The use of the central venous catheter (CVC) has become an

Transcription

1 Empirical Investigations Short-term and Long-term Impact of the Central Line Workshop on Resident Clinical Performance During Simulated Central Line Placement Torrey A. Laack, MD; Yue Dong, MD; Deepi G. Goyal, MD; Annie T. Sadosty, MD; Harpreet S. Suri, MBBS; William F. Dunn, MD Introduction: The Central Line Workshop (CLW) was introduced at our institution to better train residents in safe placement of the central venous catheter (CVC). This study sought to determine if immediate performance improvements from the CLW are sustained 3 months after the training for residents with various levels of experience. Methods: Twenty-six emergency medicine residents completed the CLW, which includes online modules and experiential sessions in anatomy, ultrasound, sterile technique, and procedural task training. Demonstration of the synthesis of these skills including placement of both internal jugular and subclavian CVCs was assessed using a task trainer. Each resident was also tested approximately 3 months before and 3 months after the CLW. Residents were assessed using a validated CVC proficiency scale. Results: Residents CVC proficiency scores (percentage of items performed correctly during the assessment station) improved after CLW (0.6 vs. 0.93, P G 0.05). At 3 months after CLW testing, there was apparent skill decay from the CLW but overall improvement compared with baseline testing (0.6 vs. 0.8, P G 0.05). There was no significant difference in procedure time after CLW training. The postgraduate year 1 group showed the greatest improvement of CVC skill after CLW training. Conclusions: Resident CVC placement performance improved immediately after the CLW. Although performance 3 months after the CLW revealed evidence of skill decay, it was improved when compared with initial baseline assessment. Novice learners had the greatest benefit from the CLW. (Sim Healthcare 9:228Y233, 2014) Key Words: Central venous catheter, Simulation, Patient safety, Resident education, Procedure, Skill retention, Skill decay, Emergency medicine. The use of the central venous catheter (CVC) has become an indispensable resuscitation tool in the emergency department and intensive care unit. However, CVC placement has risks including both mechanical and infectious complications of insertion. 1Y5 Previous studies have shown that more than 15% of patients who receive a CVC experience one or more complications. 6Y8 Many system improvements have been postulated in an attempt to decrease these risks. 9 The Institute for Healthcare Improvement has proposed the Central Line Bundle, 10 which includes ultrasound guided placement and full sterile barrier precautions during insertion. Although adherence to the Central Line Bundle should decrease the number of complications, providing training and experience for all physicians involved in CVC placement can be difficult. Traditionally, such experience has come entirely from patient From the Department of Emergency Medicine (T.A.L., D.G.G., A.T.S.), Mayo Clinic Multidisciplinary Simulation Center (Y.D.), and Division of Pulmonary and Critical Care Medicine (H.S.S., W.F.D.), Department of Internal Medicine, Mayo Clinic, Rochester, MN. Reprints: Torrey A. Laack, MD, Department of Emergency Medicine Mayo Clinic 200 First St SW, Rochester, MN ( laack.torrey@mayo.edu). The authors declare no conflict of interest. This work was supported by the Mayo Foundation for Medical Education and Research, Y&S Nazarian Family Foundation. Copyright * 2014 Society for Simulation in Healthcare DOI: /SIH encounters, potentially putting patients at risk, especially with junior trainees. Procedure complication rates are inversely related to the experience of the operator. Although the exact number of times a procedure must be performed to develop sustained competency is unclear, operators who have placed more than 50 CVCs have fewer complications than those who have placed fewer than 50 CVCs. 7,8 Given that most house staff (including fellows) have placed far fewer than 50 CVCs and to decrease risk to patients as trainees acquire this experience, there is a need to safely improve learning methods to shorten the learning curve. Current focus on quality improvement, innovation, and patient safety has increased the use of simulation-based instructional methods, especially in settings associated with significant patient risk. 11 A simulation-based, multispecialty Central Line Workshop (CLW) was developed at the Multidisciplinary Simulation Center to offer trainees a safe, experiential learning opportunity for CVC placement. This 4-hour curriculum on ultrasound-guided CVC placement has been offered to residents and fellows involved with CVC insertion since 2005 and has been required of all emergency medicine (EM) residents since Residents demonstrate competency to perform the procedure safely and effectively in a simulated environment before being allowed to place CVCs on actual patients. Recent studies have shown improved performance after simulation-based CVC training. 12Y19 Skill decay has been 228 Impact of the Central Line Workshop Simulation in Healthcare

2 shown to occur over time after training sessions, and skill retention is improved with deliberate practice. 19Y25 We predicted that the CLW would improve performance of CVC placement for EM residents. Although some skill decay at 3 months after the CLW was anticipated, our primary aim was to determine if there is sustained improvement compared with the baseline assessment before the CLW. Secondarily, we sought to determine variability in the performance after the CLW based on the level of experience with CVC placement. METHODS Settings The Mayo Clinic Institution Review Board approved this prospective observational cohort study, which was conducted at a teaching hospital with a multidisciplinary simulation center. The EM residency is a 3-year program with 8 to 9 residents each year. The Accreditation Council for Graduate Medical Education requires EM graduates to have placed a minimum of 20 CVCs, whereas the Mayo Clinic EM residency requires 30 CVCs before graduation. Because of perceived benefits to patients, the CLW is a requirement for each EM resident; therefore, no control group was available. To measure the impact of CLW training, between November 2008 and May 2009, each EM resident s central line procedure skill was assessed 3 months before CLW (T1), immediately on completion of CLW (T2), and 3 months after CLW (T3) (Fig. 1). Three-month assessment intervals were chosen to allow assessment of skill decay while completing the study before advancement of residents to the next academic year. The baseline skill assessment (T1) was part of an annual simulation-based competency assessment course with residents unaware that CVC placement would be included. Approximately 3 months after T1 (average, 100 days; range, 82Y119 days), each resident was scheduled for the CLW (T2). Before the CLW, each resident completed online prerequisites including a combination of literature review and a case-based self-study module. The CLW has previously been described in detail. 26 Residents participated in interactive learning stations (ultrasound station, gloving and gowning station, and a procedural skill laboratory, both cadaveric and with a CVC task trainer) with hands-on teaching and deliberate practice. Experiential practice was followed by reflective debriefing with an EM attending faculty or respiratory therapist specifically trained to instruct the CLW. Finally, each resident performed a simulated internal jugular vein (IJ) CVC placement uninterrupted followed by instructions to obtain isolated subclavian vein (SC) venipuncture (without repeating other items assessed on the checklist). The session concluded with individualized instructor debriefing of the assessment station and an additional opportunity to demonstrate missed items from the checklist. Approximately 3 months after the CLW course (T3; average, 109 days; range, 91Y127 days), residents were retested at the simulation center. Similar to T1, residents did not know they would be undergoing assessment of CVC placement before arrival, as it was included during a regularly scheduled simulation-based educational session. For all T1, T2, and T3 assessments, residents were given identical instructions at the time of the testing to place a CVC in accordance with institutional policy on a simulated task trainer (Blue Phantom, Redmond, WA). Identical assessment stations involving both IJ and SC catheterization were used. Video of each resident s performance was captured and rated by investigators (Y.D. and H.S.S.) who had no previous exposure to the residents and were blinded to their level of training. We previously confirmed high interrater agreement (88.9%) using a validated assessment instrument, the Central Venous Catheterization Proficiency Scale for scoring (Table 1). 26,27 The score was calculated as the FIGURE 1. Study workflow. Vol. 9, Number 4, August 2014 * 2014 Society for Simulation in Healthcare 229

3 TABLE 1. Checklist for Central Line Procedure Items Yes/No 1. Preprocedure ID verification Yes Ì No Ì 2. Informed consent communication Yes Ì No Ì 3. Trendelenburg position Yes Ì No Ì 4. Operator maximal barrier precautions Yes Ì No Ì 5. Hand hygiene Yes Ì No Ì 6. Chlorhexidine skin antisepsis Yes Ì No Ì 7. Sterile gloving and gowning Yes Ì No Ì 8. Patient maximal barrier precautions Yes Ì No Ì 9. Ultrasound sterile technique Yes Ì No Ì 10. Identify IJ by ultrasound Yes Ì No Ì 11. Procedural pause Yes Ì No Ì 12. Successful independent IJ venipuncture Yes Ì No Ì 13. Transduction to verify venous access Yes Ì No Ì 14. Secure the catheter correctly Yes Ì No Ì 15. Successful independent SC venipuncture Yes Ì No Ì percentage of 15 dichotomous items performed correctly. In a previous study of 105 participants, the checklist was able to demonstrate high score reliability and discriminate levels of trainee experience in CVC proficiency. 26 We measured procedure time from the initial greeting of the patient until successful insertion of the IJ CVC. We did not include the SC venipuncture in the procedure time. During the study period, each resident continued to self-report and record every central line placed in a required procedure log. The residents had no additional simulated central line training or performance between testing sessions (T1, T2, and T3). Statistical Analysis The composite performance and number of procedures were summarized as a median (range). The Kruskal-Wallis test was used for comparison among postgraduate year (PGY) groups followed by a pairwise Wilcoxon rank sum tests between two groups. Two-sided 5% type 1 error was used to determine statistical significance. All statistical analyses were performed using statistical software package JMP 8.0 (SAS Institute Inc, Cary, NC). RESULTS A total of 26 residents were enrolled in the study (Table 2). Because of schedule conflicts, one resident missed the T2 assessment station, whereas 3 did not participate in session T3; however, the assessment stations they attended were included in the overall data set. Figure 2 and Table 3 show the composite TABLE 2. Resident Characteristics n (%) or Median (Range) Sex Female 14 (39) Male 22 (61) CVC placed in last 2 yr (T1) 16 (0Y48) CVC placed in last 3 mo (T2) 3 (0Y10) CVC placed in last 3 mo (T3) 4 (0Y16) Training levels PGY1 8 (30.8) PGY2 9 (34.6) PGY3 9 (34.6) T1 indicates baseline; T2, CLW; T3, 3 months after CLW. FIGURE 2. Composite score of residents performance. scores for each session with Table 3 revealing time to insertion of the IJ CVC. After the CLW, there was overall improvement of the performance score. The median performance composite scores were improved (0.6 vs. 0.93, P G 0.001) after the CLW with no change in median procedure time (20.4 vs minutes). However, performance decay was noted after CLW (T3) compared with CLW. The 3 items demonstrating the largest drop from T2 to T3 include (percent score at T2 vs. T3): hand hygiene (92% vs. 35%); procedural pause (96% vs. 57%), and preprocedure ID verification (80% vs. 57%). Although thepost-clwscoreswerelowerthanthoseattheclw,there was an overall improvement in the performance score comparedwiththebaseline(0.8vs.0.6,p G 0.05), and the procedure times were shorter but did not reach statistical significance (19.0 vs minutes, P =0.2). When stratifying the residents by training year, PGY3 residents self-report placement of more CVC lines than PGY1 residents during the study period (5.5 vs. 1.5, P G 0.001) (Table 4). The baseline performance of PGY2 and PGY3 residents was significantly higher than that of PGY1 residents. There were no significant differences between groups for the performance improvement at T2 and decay at T3 for the PGY1 and PGY2 residents. Figure 3 shows composite scores by PGY of training. The PGY1 group showed greater improvement compared with the PGY2 and PGY3 groups in the median performance score after the CLW training with the effect lasting more than 3 months (Table 5). DISCUSSION The format of the CLW, constructed around the principles of Bloom s taxonomy, 28 proved highly effective in achieving predefined learning objectives. Performance was TABLE 3. Residents Performance Variable T1 (n = 26) T2 (n = 25) T3 (n = 23) Composite score 0.6 (0.4Y0.8) 0.93 (0.73Y1)* 0.8 (0.53Y1)* Time (min) 20.4 (11.7Y32.1) 20.4 (15.7Y26.3) 19.0 (12.9Y28.8) *P G 0.05, compared with T1. P G 0.05, compared with T2. Data are presented as median (range) T1 indicates baseline; T2, CLW; T3, 3 months after CLW. 230 Impact of the Central Line Workshop Simulation in Healthcare

4 TABLE 4. Number of Self-Reported CVC Lines Placed by Resident Class PGY1 PGY2 PGY3 Total CVC (n) Baseline (T1) 3.5 (0Y7) 12 (8Y31) 39 (23Y48) CLW (T2) 4.5 (1Y8) 20 (10Y41) 39 (29Y55) Post (T3) 9.5 (5Y12) 22 (14Y41) 52 (33Y59) Delta (CLW-Base) 1.5 (0Y3) 3 (1Y10) 5.5 (1Y7)* Delta (Post-CLW) 4 (1Y7) 2 (0Y12) 4 (0Y16) *P G compared with PGY1. Data are presented as median (range). assessed using a previously validated assessment instrument checklist, which is able to discriminate levels of trainee experience and has been shown to have high scoring reliability. 26 In an effort to standardize best practices across specialties within the institution, the items on the checklist (Table 1) were chosen prospectively by a multidisciplinary team to include key elements of proper technique while seeking to reduce both mechanical and infectious complications of CVC placement. CVC line placement by EM residents was substantially improved after the CLW, similar to results reported by others. 12Y19 Composite scores decreased in the 3 months after the CLW, but scores after CLW remained higher than baseline. The appearance of skill decay during the 3 months after CLW has been demonstrated previously 19 and may suggest a need for more frequent deliberate practice after the CLW. The skill decay was equal among the novice and senior residents, despite the senior residents performing more CVC placements between sessions. The items demonstrating the largest decay 3 months after the CLW include hand hygiene, procedural pause, and preprocedure ID verification. Given the high risk of complications for CVC placement in the emergency department, these items are important to ensure patient safety and potentially decrease complications such as infection and placement in the wrong site or patient. Future research should identify curricula that maintain sustained gains on these items. Proper reinforcement of skills through deliberate practice is critical to maintain competency at all levels. Whether the baseline to post-clw improvement is from the CLW or from additional clinical experience is unclear. It is possible that repeated exposure to the CLW might better maintain competency, but this must be balanced by the resources required for such a simulated experience. With consideration of the previous experience of the residents, our study also revealed that the biggest performance gain was observed for those with less CVC experience. The novice learners (PGY1) have more performance improvement through simulationbased CVC training. Despite less experience and lower baseline scores, the PGY1 residents performed nearly identically to the PGY2 and PGY3 residents at the time of the CLW and sustain a similar level of performance 3 months after the CLW (Fig. 3). The CLW may have the greatest impact on patient safety improvement for the most novice trainees. CLW seems to accelerate the procedural performance of novice learners, diminishing the performance gaps between the PGY1 and more senior residents. Our study has several limitations. First, it is limited to a single center with a small sample size, and the results might be different with larger populations and different programs. Second, there was no control group without CLW training to compare with the simulation group. Third, although the checklist was designed to ensure standardization of best practices across specialties, one could argue that not all components of the checklist used for scoring are of equivalent importance. Finally, we did not assess clinical FIGURE 3. Composite scores by PGY. Vol. 9, Number 4, August 2014 * 2014 Society for Simulation in Healthcare 231

5 TABLE 5. Comparison of Performance Score by Training Year Performance Score PGY1 PGY2 PGY3 P Baseline (T1) 0.47 (0.4 to 0.73) 0.67 (0.47 to 0.8) 0.67 (0.53 to 0.8) 0.04 CLW (T2) 0.9 (0.73 to 1) 0.93 (0.7 to 1) 0.87 (0.8 to 1) 0.36 Post (T3) 0.8 (0.6 to 1) 0.83 (0.67 to 0.93) 0.8 (0.53 to 0.87) 0.7 Delta (CLW-Base) 0.44 (0 to 0.53) 0.33 (to 0.1 to 0.47) 0.27 (0.13 to 0.4) * 0.07 Delta (Post-CLW) j0.13 (j0.27 to 0.07) j0.1 (to 0.33 to 0.2) j0.07 (j0.4 to 0.07) 0.99 Delta (Post-Base) 0.33 (j0.13 to 0.47) 0.17 (j0.07 to 0.47) 0.17 (0 to 0.2)* 0.10 *P = 0.03, compared with PGY1. Data are presented as median (range). performance because there might be performance gaps between simulation-based assessment and clinical practice. 29 Our study is unique in that residents were unaware that CVC placement would be assessed before arrival to the baseline and post-clw assessment sessions (T1 and T3), thus eliminating the opportunity to specifically prepare before the testing session and making this design generalizable for other similar studies. This allowed assessment truly reflective of each resident s skill level and not a manifestation of a testing scenario. We also captured the baseline characteristics and improvement variations between learners, enabling us to explore different course designs in the future to be tailored to a learner s level of experience. Further studies are needed to investigate the ideal frequency and method of reinforcement for CVC placement to most efficiently improve clinical care. 30,31 CONCLUSIONS With the use of experiential learning through simulation, the CLW improved EM resident CVC placement performance. Skill decay did occur during the study period, but there was improvement in CVC placement at 3 months after the CLW compared with baseline performance. The improvement is greatest for learners who have placed fewer CVCs in actual patients, with performance of novices after CLW approaching that of more experienced learners without increased rates of decay. Providing a simulation-based CLW experience before CVC placement on clinical rotations allows novice learners to learn without risk and may enhance patient safety. Future study should focus on approaches to maintain competency over time and optimize the skill transfer from the simulation setting to clinical practice. REFERENCES 1. Polderman KH, Girbes AJ. Central venous catheter use. Part 1: mechanical complications. Intensive Care Med 2002;28:1Y McGee DC, Gould MK. Preventing complications of central venous catheterization. N Engl J Med 2003;348:1123Y Deshpande KS, Hatem C, Ulrich HL, et al. The incidence of infectious complications of central venous catheters at the subclavian, internal jugular, and femoral sites in an intensive care unit population. Critical Care Medicine 2005;33:13Y LeMaster CH, Agrawal AT, Hou P, Schuur JD. Systematic review of emergency department central venous and arterial catheter infection. Int J Emerg Med 2010;3:409Y Marik PE, Flemmer M, Harrison W. The risk of catheter-related bloodstream infection with femoral venous catheters as compared to subclavian and internal jugular venous catheters: a systematic review of the literature and meta-analysis. Crit Care Med 2012;40:2479Y Merrer J, De Jonghe B, Golliot F, et al. Complications of femoral and subclavian venous catheterization in critically ill patients: a randomized controlled trial. JAMA 2001;286:700Y Sznajder JI, Zveibil FR, Bitterman H. Central vein catheterization. Failure and complication rates by three percutaneous approaches. Arch Intern Med 1986;146:259Y Taylor RW, Palagiri AV. Central venous catheterization. Crit Care Med 2007;35:1390Y Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006;355:2725Y Institute for Healthcare Improvement. Implement the Central Line Bundle Available at: ImplementtheCentralLineBundle.aspx. Accessed December 2, Schmidt E, Goldhaber-Fiebert SN, Ho LA, McDonald KM. Simulation exercises as a patient safety strategy: a systematic review. Ann Intern Med 2013;158:426Y Britt RC, Novosel TJ, Britt LD, Sullivan M. The impact of central line simulation before the ICU experience. Am J Surg 2009;197:533Y Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med 2009;4:397Y Barsuk JH, McGaghie WC, Cohen ER, O Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med 2009;37:2697Y Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med 2009;169:1420Y Evans LV, Dodge KL, Shah TD, et al. Simulation training in central venous catheter insertion: improved performance in clinical practice. Acad Med 2010;85:1462Y Khouli H, Jahnes K, Shapiro J, et al. Performance of medical residents in sterile techniques during central vein catheterization: randomized trial of efficacy of simulation-based training. Chest 2011;139:80Y Ma IW, Brindle ME, Ronksley PE, Lorenzetti DL, Sauve RS, Ghali WA. Use of simulation-based education to improve outcomes of central venous catheterization: a systematic review and meta-analysis. Acad Med 2011;86:1137Y Smith CC, Huang GC, Newman LR, et al. Simulation training and its effect on long-term resident performance in central venous catheterization. Simul Healthc 2010;5:146Y Lammers RL, Davenport M, Korley F, et al. Teaching and assessing procedural skills using simulation: metrics and methodology. Acad Emerg Med 2008;15:1079Y Wayne DB, Siddall VJ, Butter J, et al. A longitudinal study of internal medicine residents retention of advanced cardiac life support skills. Acad Med 2006;81:S9YS Lammers RL. Learning and retention rates after training in posterior epistaxis management. Acad Emerg Med 2008;15:1181Y Impact of the Central Line Workshop Simulation in Healthcare

6 23. Mitchell EL, Lee DY, Sevdalis N, et al. Evaluation of distributed practice schedules on retention of a newly acquired surgical skill: a randomized trial. Am J Surg 2011;201:31Y Yang CW, Yen ZS, McGowan JE, et al. A systematic review of retention of adult advanced life support knowledge and skills in healthcare providers. Resuscitation 2012;83:1055Y Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79:S70YS Dong Y, Suri HS, Cook DA, et al. Simulation-based objective assessment discerns clinical proficiency in central line placement: a construct validation. Chest 2010;137:1050Y Dong Y, Suri HS, Kashani KB, Dunn WF. Development of checklist based performance assessment instrument of central line placement. Chest 2009;136:13S. 28. Bloom BS. Taxonomy of Educational Objectives. New York, NY: David McKay Co Inc; Sturm LP, Windsor JA, Cosman PH, Cregan P, Hewett PJ, Maddern GJ. A systematic review of skills transfer after surgical simulation training. Ann Surg 2008;248:166Y Cook DA. One drop at a time: research to advance the science of simulation. Simul Healthc 2010;5:1Y Weinger MB. The pharmacology of simulation: a conceptual framework to inform progress in simulation research. Simul Healthc 2010;5:8Y15. Vol. 9, Number 4, August 2014 * 2014 Society for Simulation in Healthcare 233