CRM in USAF Flight and Family Medicine Clinics Michael D. Jacobson, DO, MPH Colonel, USAF, MC, SFS USAF School of Aerospace Medicine Wright-Patterson AFB, OH RAM 2013 Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 1
Disclosure Information 84th Annual Scientific Meeting Michael D. Jacobson, DO, MPH I have no financial relationships to disclose. I will not discuss off-label use and/or investigational use in my presentation. Definitions Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) Patient Centered Medical Home (PCMH) Aviation Crew Resource Management (acrm) Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 2
Background Crew Resource Management (CRM): effective use of all available resources (human, equipment, information) to ensure mission completion and reduce mishaps CRM: significant + impact in other industries Much less studied in medicine CRM in AFMS initiatives (TeamSTEPPS, PCMH) Limited analyses of major DoD HS CRM initiatives Few outcome studies (really reduce errors?) No papers found re: use SAQ in DoD HS Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 3
Research Questions/Hypotheses Q1: Do CRM-related initiatives (TeamSTEPPS, PCMH, acrm) result in more positive patient safety-related attitudes in AF physicians? H1a: SAQ (No TeamSTEPPS training) H1b: SAQ (No PCMH training) H1c: SAQ (No acrm training) < SAQ (TeamSTEPPS training) < SAQ (PCMH training) < SAQ (acrm training) Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 4
Research Questions/Hypotheses Q2: Which variables explain the variance in individual domain scores? H2a: SAQ TeamSTEPPS varies w/{group/rank/gender/ MAJCOM} H2b: SAQ PCMH varies with {group/rank/gender/ MAJCOM} H2c: SAQ acrm varies with {group/rank/gender/ MAJCOM} Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 5
Methods Design: Cross-sectional quantitative analysis Inclusion: AF base MTF flight/family physicians Study apparatus: AFeSAQ online Safety Attitude Questionnaire (SAQ)-Short (36) 6 Domains: Teamwork Climate, Safety Climate, Job Satisfaction, Stress Recognition, Perceptions Mgt (Unit/Hosp), Work Conditions Four questions directly inquiring perspective as to Impact of TeamSTEPPS, PCMH, acrm training Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 6
Methods Statistical Analysis Multivariate analysis of covariance (MANCOVA) IV: Training in TeamSTEPPS, PCMH, or acrm DV: SAQ domain scores Covariates: group (FS vs. FP), rank, gender, MAJCOM Interaction effects examined between training types To further analyze significant covariate effects Spearman s correlation One-way ANCOVA Extra questions re: training effect ANOVA Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 7
Results: Participants FS FP Other Total Survey Pool 322 300 0 622 Responses 143 78 38 269 Retained 123 66 0 189 Gender M-142 (75%) F-47 (25%) O-3 O-4 O-5 O-6 Rank 84 (44.4%) 51 (27.0%) 40 (21.2%) 14 (7.4%) MAJCOM: ACC-37,AMC-34 AFGSC-9,PACAF-4,AFCENT-0 Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 8
Mean Scores SAQ Domains Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 9
Were CRM-like initiatives assoc/with > assoc/with >SAQ domain scores? MANCOVA Multivariate Tests (Wilks lambda) No significant fixed effect on SAQ domains seen with any of the types of training i.e. fail to reject null hypothesis: TeamSTEPPS, PCMH, acrm were NOT associated with improved SAQ scores (i.e. with more positive attitudes) EFFECT Value F Hyp df Err df Sig Power TeamSTEPPS 0.955 1.163 7 172.326.045 PCMH 0.976.597 7 172.758.024 acrm 0.963.946 7 172.472.037 Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 10
Did any covariates have an effect on SAQ domain scores? RANK had a significant effect (.017) on most SAQ domains Teamwork Climate Job Satisfaction Perceptions of Management (Unit AND Hosp/Clinic) Work Conditions MAJCOM: significant effect (.013) on Job Satisfaction (but clinically significant?) Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 11
Correlation of RANK with SAQ Domain Scores (Spearman s) Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 12
Covariate Effects for MAJCOM ANOVA: MAJCOM has effect on SAQ Job Satisfaction Sum of Sq df Mean Sq F Sig. Between Groups 14822.609 9 1646.957 2.058 0.036 Within Groups 143263.7 179 800.356 Total 158086.31 188 However post-hoc failed to show signif. diff b/w MAJCOM When ANCOVA was run with RANK as a covariate, effect was seen: i.e., MAJCOM s effect only in presence of RANK Only: AETC (80.6) > AMC (53.5) Weak partial eta 2 (0.108) Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 13
Perceived Impact of Initiatives (Additional questions on AFeSAQ) TeamSTEPPS/PCMH/aCRM has improved the effective use of available human, equipment & information resources improved communication skills, teamwork, task allocation, and decision-making contributed to a working environment in which fewer mistakes are made I enjoy my working environment more as a result of [TeamSTEPPS/PCMH/aCRM] Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 14
Attitudes re: Impact of Training Initiatives (pooled ANOVA) P<0.005 Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 15
Conclusions and Recommendations TeamSTEPPS/PCMH/aCRM training ineffective at increasing patient safety-related positive attitudes (i.e., SAQ scores) SAQ: validated instrument for measuring safetyrelated attitudes in health care Use tools, e.g., SAQ/AFeSAQ, to objectively assess impact of initiatives and training e.g., <50% family physicians trained in PCMH TeamSTEPPS: some say it s just check the box Can easily be utilized to assess climate at unit level Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 16
Bibliography Whitmore, Christine ; Borsky, Amanda ; Christensen, Eric. Evaluation of Staff Satisfaction at the National Naval Medical Center Medical Home. Center for Naval Analyses, Alexandria, VA. 2011. Savage AI, Lauby T, Burkard JF. Examining selected patient outcomes and staff satisfaction in a primary care clinic at a military treatment facility after implementation of the patient-centered medical home. Mil Med 2013;178(2):128-34. Christensen EW et al. Impact of a patient-centered medical home on access, quality, and cost. Mil Med. 2013;178(2):135-41. Dorrance KA et al. Protecting the culture of a patient-centered medical home. Mil Med. 2013;178(2):153-8. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Services Research 2006;6:44 Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 17
Questions? Distribution A: Approved for public release; distribution is unlimited. Case Number: 88ABW-2013-2285, 13 May 2013 18