Development and Utility of the Front Line Manager s Quick Reference Guide

Transcription

1 Federal Aviatio Admiistratio DOT/FAA/AM-13/10 Office of Aerospace Medicie Washigto, DC Developmet ad Utility of the Frot Lie Maager s Quick Referece Guide Daredia McCauley L. Sarah Peterso Civil Aerospace Medical Istitute Federal Aviatio Admiistratio Oklahoma City, OK S. Jaie Kig Xyat Techology, Ic. Oklahoma City, OK May 2013 Fial Report

2 NOTICE This documet is dissemiated uder the sposorship of the U.S. Departmet of Trasportatio i the iterest of iformatio exchage. The Uited States Govermet assumes o liability for the cotets thereof. This publicatio ad all Office of Aerospace Medicie techical reports are available i full-text from the Civil Aerospace Medical Istitute s publicatios Web site:

3 Techical Report Documetatio Page 1. Report No. 2. Govermet Accessio No. 3. Recipiet's Catalog No. DOT/FAA/AM-13/10 4. Title ad Subtitle 5. Report Date Developmet ad Utility of the Frot Lie Maager s Quick Referece Guide May Performig Orgaizatio Code AAM Author(s) 8. Performig Orgaizatio Report No. McCauley DL, 1 Peterso LS, 1 Kig SJ 2 9. Performig Orgaizatio Name ad Address 10. Work Uit No. (TRAIS) 1 FAA Civil Aeromedical Istitute P.O. Box Oklahoma City, OK Xyat Techology, Ic. Oklahoma City, OK Cotract or Grat No. 12. Sposorig Agecy ame ad Address 13. Type of Report ad Period Covered Office of Aerospace Medicie Federal Aviatio Admiistratio 14. Sposorig Agecy Code DOT/FAA/ 800 Idepedece Ave., S.W. Washigto, DC Supplemetal Notes Work was accomplished uder approved task AM Abstract Air traffic cotrol Frot Lie Maager s (FLMs) ifluece the prevetio of operatioal icidets (OIs) ad ruway icursios (RIs) through practices that eable safe cotroller performace ad mitigate problems related to specific cotributig factors, icludig cotroller coordiatio, cotroller-pilot commuicatio, ad cotroller memory. Proximity ad frequet iteractios with cotrollers i the operatioal eviromet place FLMs i a uique positio to impact air traffic safety through maagig cotroller performace ad by mitigatig OIs ad RIs as commo cotributig factors. With this uderstadig of the FLM role, the Huma Factors Divisio (ANG-C1), Booz Alle Hamilto (cotractor), ad Air Traffic Orgaizatio (ATO) developed the Frot Lie Maagers Quick Referece Guide (QRG).The Civil Aerospace Medical Istitute (CAMI) subsequetly helped the ATO with the reorgaizatio of materials. The guide was iteded to provide a maagemet best practices format specific to air traffic cotrol to assist FLMs perform their job. To determie the utility of the QRG, CAMI ad ATO developed ad admiistered a survey of FLMs. The survey also provided the ATO with a basis for additios or deletios to the QRG as it related to the specific factors that cotribute to OIs ad RIs. This report documets the developmet, implemetatio, ad subsequet survey of the QRG for frot lie maagers. The survey was desiged for use by all field FLMs. Suggestios ad recommedatios from survey respodets are provided, which ca keep the guide both curret ad relevat. 17. Key Words 18. Distributio Statemet Air Traffic Cotrol Frot Lie Maagers, Operatioal Icidet, Ruway Icidet, Frot Lie Maager s Quick Referece Guide, Frot Lie Maager s Best Practices Documet is available to the public through the Iteret: Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price Uclassified Uclassified 18 Form DOT F (8-72) Reproductio of completed page authorized i

4

5 ACKNOWLEDGMENTS Research reported i this paper was coducted uder the Federal Aviatio Admiistratio s (FAA s) Air Traffic Program Directive / Level of Effort Agreemet betwee the Huma Factors Research ad Egieerig Divisio (ANG-C1), FAA Headquarters, ad the Aerospace Huma Factors Research Divisio (AAM-500) of the Civil Aerospace Medical Istitute. Special recogitio is give to Edmudo Sierra (ANG-C1) ad Cidy J. Greee (TCG- MSP) for their assistace with this project. iii

6

7 CONTENTS Developmet ad Utility of the Frot Lie Maager s Quick Referece Guide Itroductio...1 Air Traffic Cotrol Frot Lie Maagers...1 Developmet of the Frot Lie Maager Quick Referece Guide...1 Method...4 Survey...4 Survey Participats...4 Survey Admiistratio...4 Results...5 Coclusio...5 Refereces...6 Appedix A: Frot Lie Maager Quick Referece Guide Survey, 2011 Overall Report...A1 v

8

9 ACRONYMS ANG Huma Factors Research ad Egieerig Divisio ARTCC Air Route Traffic Cotrol Ceter AT Air Traffic ATC Air Traffic Cotrol ATO Air Traffic Orgaizatio ATO-E Air Traffic Orgaizatio E Route ad Oceaic Services ATO-S Air Traffic Orgaizatio Safety Services ATO-T Air Traffic Orgaizatio Termial Support Services ATSAP Air Traffic Safety Actio Program BAH Booz Alle Hamilto CAMI Civil Aerospace Medical Istitute CIC Cotroller i Charge CPC Certified Professioal Cotroller CRM Crew Resource Maagemet FAA Federal Aviatio Admiistratio FEA Frot Ed Aalysis FLM(s) Frot Lie Maager(s) OHP Optimizig Huma Performace OI Operatioal Icidet QRG Quick Referece Guide RI Ruway Icursio SME Subject Matter Expert SUPCOM Air Traffic Supervisor s Committee TRACON Termial Radar Approach Cotrol vii

10

11 Developmet ad Utility of the Frot Lie Maager s Quick Referece Guide INTRODUCTION Air traffic cotrol (ATC) Frot Lie Maagers (FLMs) are istrumetal i meetig Federal Aviatio Admiistratio (FAA) Flight Pla objectives for mitigatig Operatioal Icidets (OIs) ad Ruway Icursios (RIs) by providig leadership to esure that air traffic cotrollers performace meets FAA goals for improvig the safety of the Natioal Airspace System (NAS). FLMs also ifluece the prevetio of errors by eablig exemplary cotroller performace (Greee, 2010). Iformatio to assist air traffic cotrol FLMs i the performace of their techical duties may ot be readily available, ad formal traiig may be delayed due to a umber of factors such as access to orgaizatioal traiig or operatioal metorship. Providig FLMs with iformatio that helps them supervise cotrollers may impact the overall safety of air traffic operatios. The Quick Referece Guide (QRG) was desiged as a resource to help ATC FLMs supervise their employees more effectively by promotig safety, ehacig commuicatio, ad ecouragig respect i a way that ehaces cotroller performace. The QRG is available to the FLMs olie at: media/frot_lie_maager_quick_referece_guide.pdf It is also available i hard copy i FAA field offices. Air Traffic Cotrol Frot Lie Maagers ATC FLMs provide first-lie supervisio to teams of developmetal cotrollers ad certified professioal cotrollers (CPCs). FLMs are resposible for plaig ad assigig work priorities, evaluatig ad recommedig performace ratigs, ad listeig to ad resolvig complaits of cotrollers uder their supervisio. FLMs are also resposible for idetifyig ad arragig for the traiig eeds of their staff as well as recommedig goals ad objectives ad the meas to track these accomplishmets. They also participate i the labor-maagemet process ad foster a equitable workig eviromet. FLMs must be excellet commuicators ad be able to adjust quickly to a chagig work eviromet. FLMs use a combiatio of these skills i supervisig cotrollers. These skills are also iteded to mitigate OIs ad RIs (Iteratioal Civil Aviatio Orgaizatio, 2005). The Air Traffic Orgaizatio (ATO) supported a survey to evaluate the QRG s efficacy. This report describes the developmet of the Air Traffic Cotrol Frot Lie Maagers Quick Referece Guide ad summarizes the fidigs of the survey desiged by the Civil Aerospace Medical Istitute (CAMI) to examie the operatioal fuctioality of the QRG. Developmet of the Frot Lie Maager Quick Referece Guide Booz Alle Hamilto (BAH) coducted studies sposored by the FAA s Huma Factors Divisio ad Egieerig Divisio (ANG-C1) ad the ATO Offices of Safety (ATO-S), E Route ad Oceaic (ATO-E), ad Termial Support (ATO-T) services to idetify best practices of FLMs for promotig safety ad, therefore, icrease the likelihood of reducig the occurrece of OIs ad RIs throughout the Natioal Airspace System. Backgroud research for the QRG was developed i three phases, usig differet methodology for each group of FLMs: e route, tower, ad termial radar approach cotrol (TRACON). The first FLMs to be assessed were e route, followed by tower, ad TRACON. E route QRG developmet. The e route QRG backgroud research was coducted ad reported i 2006 (Duley, Capriglioe, Kalita, Guether, & Nagy, 2006a). This research focused o geeral supervisory best practices used by exemplary FLMs i respose to a variety of idetified problems ecoutered i the areas of selectio ad assigmet, skills ad kowledge, motivatio ad icetives, ad operatioal eviromet. BAH used the Optimizig Huma Performace (OHP) Frot Ed Aalysis (FEA) methodology as a huma performace model to idetify performace with the goal of improvemet (Saba Software, 2001). The compoets of optimal performace were idetified as skills ad kowledge, motivatio ad icetive, ad operatioal eviromet. The FEA approach also allowed for likage to other systems such as rewards, learig maagemet, ad assigmet of FLMs. BAH provided certified expertise with this methodology. Withi the FEA framework, BAH reviewed literature related to OIs withi the e route eviromet ad collaborated with ATO-E to idetify exemplary FLMs ad cotrollers to serve as subject matter experts (SMEs) to provide expertise. Next, BAH coducted workig group 1

12 Table 1. QRG Problem Format Example Problem Statemet: Cosistet performace maagemet practices are ot always utilized FLM Challeges: 1. Maagig performace ad coduct of cotrollers who are ot uder the watch supervisor s admiistrative resposibility 2. Kowig the performace history or backgroud of aother supervisor s cotrollers to effectively maage their performace 3. Icosistetly documetig both positive ad egative feedback 4. Kowig what costitutes correct performace 5. Relayig performace cocers ad praises to the supervisor of record Mitigatio Topic: Fosterig a Positive Work Culture Mitigatio Topic: Active Listeig Mitigatio Topic: Accoutability Mitigatio Topic: Sharig Iformatio Mitigatio Topic: Commuicatio meetigs to establish project expectatios ad develop lists of FLM target job duties ad resposibilities. Maagers desigated as SMEs were the iterviewed to validate project expectatios with levels of FLM target duties ad resposibilities. Prior to site visits, BAH reviewed relevat FAA documets, OI reports, ad other related reports. Site visits were coducted at six Air Route Traffic Cotrol Ceters (ARTCCs) for structured FEA iterviews with FLMs ad reviews of site documetatio relevat to safety ad best practices. BAH focused o aalyzig factors idetified as beig both barriers to ad facilitators of accomplished performace. Cosistet with the OHP process, the data collected were orgaized aroud the idetified compoets of optimal performace: selectio ad assigmet, skills ad kowledge, motivatio ad icetive, ad operatioal eviromet. Sice both barriers to ad facilitators of best practices were reported, patters i the data permitted further categorizatio ito problem statemets that focused o factors facilitatig best practices ad factors hiderig best practices. The SMEs agreed that these workplace problem areas were commo to the ATC work eviromet. With problem areas reported, viable workplace itervetios or solutios were also idetified ad documeted for each problem statemet. Idetified ATC FLM problem areas with viable solutios became the format for the developmet of the E Route QRG (Duley, Capriglioe, Kalita, Guether, & Nagy, 2006b). Table 1 illustrates this problem statemet format. The QRG was developed to provide precise maagemet focus to the ATC workig eviromet with the itet of prevetig ad mitigatig both OIs ad RIs. Tower QRG developmet. The 2008 Tower FLM study used the Threat ad Error Maagemet methodology to idetify best practices to prevet errors ad their cotributig factors (Capriglioe, Williams, Daiels, & Phillips, 2008). The model described the relatioship betwee huma performace ad error i a coceptual framework. Prior to site visits, BAH reviewed OI ad RI data ad literature related to OIs ad the maager s role. Through site visits to 16 air traffic cotrol towers ad through iterviews with 30 FLMs, factors ifluecig supervisor effectiveess i the tower eviromet were defied. Relevat coordiatio, commuicatio, ad memory best practices were documeted. Specifically, coordiatio focused o cotroller-tocotroller iteractios ad trasfer of iformatio. Commuicatio focused o cotroller-to-pilot exchage of iformatio. Memory focused o cotroller recall of procedures ad actios, as well as situatioal awareess. 2

13 Table 2. QRG Cotet Example Error Idetificatio ad Mitigatio: Coordiatio Error FLM Coordiatio Error: 1. Problem Detectio 2. Best Practices Mitigatio Team Buildig Mitigatio Coflict Resolutio Mitigatio Observatio Mitigatio Active listeig I additio to idetificatio of cotroller best practices i these areas, this study idetified a three-step process used for applyig these practices: 1) problem detectio, 2) causal determiatio, ad 3) problem mitigatio or elimiatio respose. Table 2 provides a QRG cotet example for coordiatio errors. The tower study idetified FLM error maagemet best practices i the tower operatioal eviromet; however, much of the iformatio was relevat to all FLMs. While the iformatio provided permitted a expaded best practices format, it also provided the idetificatio of specific problem mitigatio topics to be icluded i the QRG, alog with idetificatio ad mitigatio techiques. To icorporate this iformatio i the Tower versio of the QRG, BAH expaded the ATC supervisory best practices idetified by the e route study ad added some best practices for tower FLMs idetified i the tower study. Specifically, the Problem Statemet sectio was expaded to iclude tower-relevat situatios ad mitigatig factors, as well as expad the set of best practices applicable to all FLMs. TRACON QRG developmet. The third study, which focused o TRACON FLMs, was completed by BAH i 2010 (Hoeicke, Rudolph, & Hill, 2010). The BAH team coducted a Gap Aalysis o iformatio collected from the e route ad tower studies to idetify factors commo to TRACON. After idetifyig commo factors of situatioal awareess, traiig, compressio/complexity, memory, positio relief, commuicatio, performace maagemet, staffig, equipmet, ad motivatio ad icetives, these commo factors were reviewed for the TRACON eviromet as related to error prevetio. I additio, the proportio of developmetal cotrollers supervised ad weather were added as factors to be cosidered i error prevetio ad mitigatio. The TRACON study also idetified leadership skills used by effective FLMs, which promoted operatioal maagemet ad admiistrative effectiveess. This resulted i the additio of a Leadership sectio, which followed the Problem Statemet sectio i the QRG. QRG cosolidatio. BAH research cosolidated the cotet iformatio provided for all three services (e route, tower, ad TRACON). The TRACON iformatio was icorporated ito the existig QRG draft from the e route ad tower studies. The draft documet was made available for review. The FAA s Air Traffic Supervisor s Committee (SUPCOM) provided SME review of the documet. The origial SME research coordiators were retaied for cotiuity. These reviewers worked with ATO-S, ANG-C1, ad CAMI to review the BAH draft QRG documet. The documet was revised to remove some locatio-specific maagemet situatios, makig the overall documet applicable to maagerial coditios across all facility types. Laguage was carefully reviewed ad adjusted to reflect curret AT professioal use. I additio, sectios were icluded for Crew Resource Maagemet, Safety Culture, Air Traffic Safety Actio Program, ad Additioal Refereces ad Resources as supportig documetatio. To summarize, more tha 60 E Route, Tower, ad TRACON FLMs ad their supervisors were iterviewed from 30 facilities for the etire project. Research ad field iterviews focused o the circumstaces that cotribute to errors ad the FLM best practices used to mitigate or prevet them. This made the QRG particularly relevat to the FLM populatio as a maagemet tool. Accordig to the Executive Summary of the 2006 Supervisory Best Practices for Operatioal Icidet Prevetio, The guide serves as a job-aid that addresses the gap betwee existig Operatioal Supervisor traiig resources that deal with admiistrative fuctios but overlook the people skills that are critical for avoidig operatioal icidets (Duley et al., 2006a, p. vi). Although the QRG was developed as a stadaloe referece tool, it was also icorporated ito existig traiig ad could be used as a basis for the developmet of ew traiig specific to air traffic FLMs. This provided for idetificatio ad applicability of maagemet skills specific to those eeded by air traffic FLMs, with a focus o the reductio 3

14 Table 3. QRG Orgaizatio AT FLM Quick Referece Guide FLM Tabbed Topics: 1. Problem Statemets 2. Leadership 3. Performace Maagemet 4. Commuicatios 5. Mitigatig Coordiatio Error 6. Mitigatig Commuicatio Error 7. Mitigatig Memory Error 8. Mitigatig Complexity Error FLM Tabbed Supportig Iformatio Topics: 9. Safety Culture 10. Air Traffic Safety Actio Program (ATSAP) 11. Crew Resource Maagemet (CRM) 12. Additioal Refereces ad Resources ad mitigatio of errors. Table 3 provides a illustratio of the QRG cotet topics ad supportig iformatio. The FLM QRG was delivered to maagers at air traffic cotrol facilities i hard copy ad to FLMs electroically i December I order to assess the impact, ATO supported a survey to evaluate the efficacy of the QRG. This survey was desiged to evaluate the utility of the FLM-QRG ad determie if additios would be valuable to ATC FLMs ad could be cosidered for updates to the QRG at a later date. The survey was opeed electroically i Jue of 2011 ad remaied ope through July of METHOD Survey To assess the utility of the QRG i the field, a survey cosistig of 22 questios was distributed to all ATC FLMs. The survey was desiged to assess FLM perceptios of the QRG, icludig cotet orgaizatio, usefuless, ad appropriateess. The survey also solicited feedback regardig potetial cotet errors, problems with QRG utility, additioal cotet areas that should be addressed, ad geeral suggestios for improvemet. I additio to their perceptio of the QRG, the survey collected demographic iformatio about the participats, icludig curret supervisory status (e.g., curret FLM), air traffic optio (e route or termial), termial facility type (TRACON, tower, tower/tracon), facility level, ad certified professioal cotroller (CPC), cotroller i charge (CIC), ad FLM teure. The survey ad its admiistratio methodology were submitted to ad approved by the CAMI Idepedet Review Board (IRB). Survey participat cofidetiality was assured. Survey Participats A list of FAA air traffic supervisory employees was provided by the FAA s Iformatio Techology, Iformatio Delivery ad Collaborative Services office. The list (N=3,151) icluded all supervisory employees withi the air traffic cotrol specialist job series who were assiged to a facility resposible for the separatio of air traffic. Supervisory persoel at the FAA s Washigto Headquarters, Techical Ceter, ad Aeroautical Ceter were removed from the list, reducig the pool to 2,956 persos. The list was further refied to exclude area maagers, operatios maagers, higher-level seior maagers, ad osupervisory maagers, resultig i a sample of 2,130 FLMs. Of those, 11 survey ivitatios were udeliverable ad 1 FLM was removed upo the supervisor s request, leavig a fial adjusted sample of 2,118 FLMs. Ivitatios to complete the survey were distributed to FLMs iteral FAA electroic mail addresses. Completed surveys were volutarily submitted olie by 849 participats, resultig i a 40.1% overall respose rate. Amog respoders, 58 idicated that they were ot FLMs ad were excluded from further reportig. The remaiig 791 respodets were icluded i the fial reportig ad aalysis. Survey Admiistratio The survey was desiged ad distributed usig Sap Survey Versio software. The software eabled dissemiatio via electroic mail ivitatios with embedded hyperliks. A electroic mail message aoucig the survey ad ecouragig participatio was distributed to participats FAA addresses o Jue 1, The aoucemet icluded a embedded hyperlik to view ad dowload the FLM QRG. 4

15 Ivitatios to complete the survey were set via electroic mail o Jue 15, To cotrol access to the survey ad protect respodet aoymity, a uique hyperlik was embedded i each ivitatio ad all subsequet electroic mail remiders. The hyperliks eabled the survey software to track survey completios, geerate a uique idetificatio umber i a database to track submitted surveys, ad automatically geerate electroic mail remiders to participats who had ot submitted a completed survey. Participats without completed surveys were ed up to four remiders at 14-day itervals. The remiders ecouraged participatio ad icluded a embedded hyperlik to access the survey. The survey closed o August 1, Olie survey resposes were dowloaded. RESULTS Most participats (564; 71.4%) idicated that they had reviewed the QRG. Of those who had reviewed the QRG, the majority (72.5%) reviewed a electroic copy. Overall FLM perceptios of the QRG were positive. Most FLMs respoded that the iformatio cotaied i the QRG was somewhat or very appropriate to their job (82.8%), ad 79.2% felt that it was well orgaized. Over half of all FLMs idicated that they would probably or defiitely use the QRG as a referece tool i the future (60.8%). Whe split by FLM teure (6 or fewer years, 7 to 10 years, more tha 10 years), lower teure (60.6%) ad higher teure (70.7%) FLMS were more iclied to use the QRG as a referece tool for their jobs i the future. Overall, 85.6% of FLMs reported that the QRG was somewhat to very useful. Item level results of the survey are preseted i Appedix A of this report. FLMs were asked if they had suggestios for improvig the QRG. Sixty-seve (12%) idicated yes ad subsequetly provided at least oe suggestio for improvemet. Their verbatim resposes were cotet coded usig 21 uique topics codes orgaized ito three major areas related to the QRG Cotet, Implemetatio, ad Techology with a fourth cotet area for commets ot directly related to the QRG. Each topic code was assiged oly oce to each commet item (items 10-13) for each respodet, with the umber of codes assiged varyig from oe to five codes. A total of 148 codes were assiged to the 67 survey respodets who provided commets ad suggestios. The majority of commets were coded ito the QRG Cotet area (106 codes). Withi the QRG Cotet area, most of the commets idicated that the QRG was too log or cotaied too much iformatio, while some provided suggestios for additioal cotet areas such as Huma Resources Cotet ad Labor Maagemet Relatios Sectios. Of the 12% of FLMs who made suggestios for improvemets, oe oted ay cotet errors i the existig QRG. The QRG Implemetatio codig cotet area (25 codes) cotaied commets focused primarily o providig the QRG i a codesed pocket versio or quick guide, providig hard copies or electroic copies of the guide, ad commets about iadequate time or staffig to use the QRG ad the eed to promote the QRG ad trai FLMs to use it. The Techology cotet area commets recommeded simplifyig the electroic versio usig highlighted sectios, icludig electroic liks or bookmarks, ad ehacig or facilitatig dowload ad prit optios. 5 CONCLUSION Survey data idicate that FLMs perceive that the QRG is well orgaized ad cotais appropriate iformatio to be a useful referece tool. Both less ad more experieced FLMs fid the QRG to be helpful, as idicated by reported iteded use, with the more experieced FLMs idicatig higher iteded use. There were also recommedatios for improvig the utility of the QRG by makig it dowload more quickly ad providig ehaced avigatio tools. Recommedatios for imbedded liks to other relevat FAA documets were also suggested. The QRG was first evisioed as a best practices guide to aid both ew ad experieced FLMs i dealig with their job resposibilities. Overall, it has served as a traiig support whe other traiig opportuities may ot be readily available. The QRG is available both i hard copy ad olie ad ca be kept i each office where FLMs are resposible for aidig cotrollers i their duties. Orgaizatioal support has bee documeted for the QRG i FLM traiig ad through SUPCOM Workshops, ad it has bee frequetly accessed through the website, with over 1,000 dowloads i Through icorporatio ito traiig programs ad website access, it has become a itegral part of the FLM toolkit to advise cotrollers o elemets of air traffic cotrol. Idividual areas of the QRG may be cosidered for updates ad there are several additioal areas recommeded for iclusio that would ehace utility. Fatigue risk maagemet is oe of these areas recommeded for iclusio, as well as sectios o huma resources ad labor maagemet relatios. The cotiued maiteace ad utility of this documet will deped o a commitmet by maagemet ad the orgaizatioal sposorship of the QRG.

16 REFERENCES Capriglioe, C., Williams, K., Daiels, M., & Phillips, J. (2008). Tower supervisor best practices for operatioal error prevetio. McLea, VA: Booz Alle Hamilto. Duley, J., Capriglioe, C., Kalita, S., Guether, L., & Nagy, J. (2006a). Supervisory best practices for operatioal error prevetio. McLea, VA: Booz Alle Hamilto. Duley, J., Capriglioe, C., Kalita, S., Guether, L., & Nagy, J. (2006b). Air traffic operatios supervisor quick referece guide. McLea, VA: Booz Alle Hamilto. Greee, C.J. (2010). Frot lie maager quick referece guide for improved air traffic safety: a tool for mitigatig errors by ehacig maagemet ad cotroller performace. Presetatio to The Natioal Air Traffic SUPCOM Coferece. Washigto, DC. Hoeicke, B., Rudolph, F.M., & Hill, B. (2010). TRACON frot lie maager best practices for operatioal error prevetio. McLea, VA: Booz Alle Hamilto. Iteratioal Civil Aviatio Orgaizatio. (2005). Threat ad error maagemet i air traffic cotrol. Motreal, Caada: ICAONET. Saba Software Ic. (2001). Optimizig huma performace. Redwood Shores, CA. 6

17 APPENDIX A Frot Lie Maager Quick Referece Guide Survey, 2011 Overall Report Frot Lie Maager Quick Referece Guide Survey 2011 Overall Report Federal Aviatio Admiistratio Civil Aerospace Medical Istitute Aerospace Huma Factors Research Divisio Oklahoma City, Oklahoma Ad Xyat Techology, Icorporated Oklahoma City, Oklahoma A1

18 Explaatio of Report Cotet The Federal Aviatio Admiistratio s (FAA) Civil Aerospace Medical Istitute distributed a survey to Air Traffic Frot Lie Maagers (FLMs) durig Jue ad July The survey was desiged to gather iformatio about job requiremets, ad examie maagers' perceptios of the Air Traffic FLM Quick Referece Guide (QRG). The results of this survey will be used to assist the FAA ad the Techical Operatios Supervisor's Committee (SUPCOM) i the developmet of FLM traiig ad referece materials. This report provides the results of the survey. Example of Report Format 5. How well orgaized is the QRG? Respose Distributio (%) m sd Very disorgaized Somewhat disorgaized Neither Somewhat orgaized Very orgaized Descriptive Statistics Number of Respodets (). The umber of people that provided a usable (i.e., valid) respose for a item. Mea (m). The mea is the arithmetic average, or the sum of all scores for a item divided by the umber of people who aswered that item. Meas are reported for items aswered o iterval scales (e.g., Usefuless). Each respose optio i the scale is assiged a umber from 1 (low) to 5 (high). For example, o the Usefuless scale, the first respose optio (Not useful) would be assiged a score of 1, ad the last respose optio (Very useful) would be assiged a score of 5. Stadard Deviatio (sd). The stadard deviatio is a measure of dispersio, or spread of scores aroud the mea. Smaller stadard deviatio values idicate higher levels of agreemet amog respodets. Miimum (Mi). The lowest, or miimum, score provided for a item. Maximum (Max). The highest, or maximum, score provided for a item. Media (Med). The exact middle data poit i a set of rak-ordered scores. It is less affected by extreme scores i compariso to the mea, ad thus, is relied upo whe extreme scores are preset i a data set (e.g., year became a CPC cotroller, item 17). Respose Distributios (%). Distributios ca show where perceptios are egative or positive by lookig at the percetage of the respodets choosig low (1 ad 2) or high (4 ad 5) respose optios. Items are writte so that a respose of 4 or 5 is positive. A2

19 2011 Frot Lie Maager Quick Referece Guide Survey This survey was desiged to gather iformatio about job requiremets, ad to examie Frot Lie Maagers' (FLM) perceptios of the Air Traffic FLM Quick Referece Guide (QRG). Therefore, oly those respodets who idicated that they were curretly servig i a supervisory capacity at a air traffic facility were icluded i the fial reportig for items 2 through 22. Some items were used to brach or route the respodet to differet sectios of the survey. These items were required ad are deoted followig the item text. Frot Lie Maager Respose Distributio (%) 1. Are you curretly a active Frot Lie Maager? (Required) Yes No (ot icluded i items 2-22) The remaider of this report (items 2 through 22) icludes oly those respodets who idicated that they were a active Frot Lie Maager ('Yes' o item 1). 2. I prior years, do you recall participatig i a study of Frot Lie Maagers to idetify best practices for promotig safety ad prevetig safety icidets? Yes No Quick Referece Guide (QRG) 3. Have you reviewed the Frot Lie Maager Quick Referece Guide (QRG)? (Required) Yes No (skip to item 14) Items 4 through 13 iclude oly those respodets who idicated that they had reviewed the FLM QRG ('Yes' o item 3). 4. Did you review the QRG as: Hard copy Electroic copy Both A3

20 2011 Frot Lie Maager Quick Referece Guide Survey Quick Referece Guide (QRG) Respose Distributio (%) 5. How well orgaized is the QRG? m sd Very disorgaized Somewhat disorgaized Neither Somewhat orgaized Very orgaized 6. How useful is the QRG? m sd Not useful Somewhat useful Neither Moderately useful Very useful 7. How appropriate was the iformatio cotaied i the ORG to your job? m sd Very iappropriate Neither Very appropriate 8. Do you thik you will use the QRG as a referece tool for your job i the future? m sd Defiitely ot Probably ot Usure Probably yes Defiitely yes 9. Do you have suggestios for improvig the QRG? (Required) Yes No (skip to item 14) A4

21 Quick Referece Guide (QRG) 2011 Frot Lie Maager Quick Referece Guide Survey Items 10 through 13 iclude oly those respodets who both idicated that they had reviewed the FLM QRG ('Yes' o item 3), ad had suggestios for improvig the QRG ('Yes' o item 9). 10. Idetify additioal cotet areas that eed to be addressed i the QRG: Idetify ay potetial cotet errors that you foud i the QRG: Idetify ay problems with the orgaizatio or utility of the QRG: Do you have ay other suggestios or commets cocerig the QRG? 46 Demographics Respose Distributio (%) 14. At which type of facility are you curretly a active Frot Lie Maager? (Required) E Route (skip to item 17) 56.1 Termial Items 15 ad 16 iclude oly those respodets who idicated that they were a FLM at a Termial facility ('Termial' o item 14). 15. Facility type: Cotrol Tower with TRACON Tower Cab TRACON 16. Facility level: Levels 5-7 Levels 8-10 Levels I what year did you become a CPC cotroller? mi med max A5

22 2011 Frot Lie Maager Quick Referece Guide Survey Demographics 18. Were you a CIC? (Required) 782 Respose Distributio (%) Yes No (skip to item 20) Item 19 icludes oly those respodets who idicated they were a CIC ('Yes' o item 18). 19. I what year did you become a CIC? mi med max I what year did you become a Frot Lie Maager? mi med max Have you received a performace award? Yes No 22. Has your facility received a performace award? Yes No A6

23