DOCUMENTING proficiency in procedural EDUCATIONAL ADVANCES. Procedural Competency in Emergency Medicine: The Current Range of Resident Experience

Transcription

1 728 COMPETENCY Hayden, Panacek PROCEDURAL COMPETENCY IN EM EDUCATIONAL ADVANCES Procedural Competency in Emergency Medicine: The Current Range of Resident Experience STEPHEN R. HAYDEN, MD, EDWARD A. PANACEK, MD Abstract. Objectives: To evaluate the recorded range of procedures tracked by emergency medicine (EM) programs, and to determine whether differences in procedural experience occur in various types of residency or hospital settings. Methods: The program directors of 112 approved EM programs were asked to send actual procedure logs. The requested information included the average total number of a given procedure per graduating resident, for all procedures that were tracked. Data were categorized by program format, hospital type, and ED volume. To assess the global procedural experience among programs, a set of 22 index procedures were identified; all procedures the EM residency review committee (RRC-EM) required to be tracked were included in this set. The means per graduating resident for each index procedure were added together to generate a mean index procedure sum (MIPS) per graduating resident for each residency program. These MIPSs for a residency were then compared by program format, hospital type, and ED volume. A similar analysis was performed for all resuscitations, and a mean index resuscitation sum (MIRS) per graduating resident was generated. Results: An overall response rate of 82% was achieved; a number of programs had not graduated a residency class and were not included. Sixtyfive of 85 eligible programs (76%) provided procedural data. The average number of a given procedure per graduating resident (95% CI in parentheses) for selected procedures is as follows: oral intubation 65 (46 to 85), intubation unspecified 75 (62 to 87), nasal intubation 6 (4 to 9), cricothyroidotomy 2 (1 to 2), subclavian catheter 23 (16 to 30), chest tubes 17 (14 to 20), intraosseous line 2 (1 to 3), thoracotomy 3 (2 to 5), and vaginal deliveries 17 (13 to 21). The only statistically significant differences in subgroup comparisons were in diagnostic peritoneal lavage, trauma resuscitations, and pediatric medical resuscitations when compared by postgraduate year format, and intubation unspecified and cricothyroidotomy when compared by hospital type. There was no statistically significant difference when MIPSs were compared by format, hospital type, or ED volume. Conclusions: To the authors knowledge, this is the first study of the range of EM resident procedure experience across the spectrum of EM residency types and settings. Overall, there are few statistically significant differences in procedure experience among different program formats. Similar experiences are recorded in a variety of different hospital types or ED volumes. However, some programs report very limited EM resident experience with selected critical procedures. There is a large variation in the types and numbers of procedures recorded by EM programs. Key words: emergency medicine; procedure; resident physician. ACA- DEMIC EMERGENCY MEDICINE 1999; 6: DOCUMENTING proficiency in procedural skills is becoming increasingly important for all physician training programs, including emergency medicine (EM). The core content in EM 1 contains 81 distinct procedures. Applicants to the American Board of Emergency Medicine (ABEM) may be tested on each of these, and it is expected From the Department of Emergency Medicine, University of California San Diego Center, San Diego, CA, and the Division of Emergency Medicine, University of California Davis Center, Davis, CA (SRH, EAP). Received January 1, 1999; revision received January 23, 1999; accepted February 16, Presented at the SAEM annual meeting, Denver, CO, May Address for correspondence and reprints: Stephen R. Hayden, MD, Department of Emergency Medicine, UCSD Center, 200 West Arbor Drive, San Diego, CA Fax: ; shayden@ucsd.edu that EM training programs should provide residents experience or education in all of them. The Residency Review Committee for Emergency Medicine (RRC-EM) identifies 16 emergency procedures and four types of resuscitation that EM residencies (EMRs) must track in the form of a procedure log. 2 Although the RRC-EM requires procedure tracking, there are currently no published competency criteria, or minimum number of procedures required for board certification in EM. It is also not clear what range of procedure experience occurs in EM training programs. Emergency medicine is not alone in the lack of specific criteria for establishing procedural competency. The American Boards of Anesthesia, s, and Family Medicine do not currently require formal documentation of procedural expe-

2 ACADEMIC EMERGENCY MEDICINE July 1999, Volume 6, Number rience. The American Board of Surgery requires submission from applicants for specialty certification of 500 operative cases, 150 of which must have been done in the year prior. There is no set number of individual cases required but the board publishes a list of procedures that qualify. In 1991, the American Board of Internal Medicine approved a policy specifying seven procedures that candidates to the board must master. These include abdominal paracentesis, arterial puncture, joint aspiration, central venous access, lumbar puncture, thoracentesis, and nasogastric tube placement. 3 New York State requires hospitals to credential all houseofficers in a specified set of procedures. 4 Anecdotally, hospital credentials committees are increasingly asking emergency physicians (EPs) for documentation of procedural experience, including numbers of specific procedures, as part of the application for hospital privileges. Our objective was to evaluate the recorded range of procedures that are being tracked by EM programs. Before any statements are made regarding recommended EM resident procedural experience, a description of what currently exists is necessary. Additionally, we sought to determine whether significant differences in procedural experience occur in various types of residency or hospital settings. METHODS Study Design. This was a descriptive study of procedures performed by EM residents from approved EM residency programs. Comparisons were made of total procedures done as reported in procedural logs, as well as procedures by program format, hospital type, and ED volume. This study was considered exempt from institutional review board approval. This project was discussed with the SAEM Residency Committee and approved by the Council of Emergency Medicine Residency Directors (CORD). Study Population and Protocol. Letters were initially sent to the program directors of the 112 EM programs approved as of January Program directors were asked to provide actual procedure logs. If this detailed information was not available, we then asked for a copy of the procedure summary page from the Program Information Forms (PIFs) of the program s most recent RRC review. The average total number of a given procedure per graduating resident was the main outcome measure, and this was tabulated for all procedures that were tracked by a given program. We also asked whether the information represented actual recorded numbers or a best-guess estimate. To increase the response rate, three rounds of letters requesting information were sent to program directors on CORD letterhead, and multiple requests were sent via the CORD list server over the next several months. All information was handled in a confidential manner. As each package was received, it was opened by a support staff member and given a study number. Data were entered into the database according to this number. Subgroup comparisons were made by program format (PG-123, PG-1234, PG-234), hospital type (university, community, county/public), and ED volume of the primary training site (<50,000 annually or 50,000). Measures. To assess the global procedural experience between programs, a set of index procedures were identified. These included arthrocentesis, transvenous cardiac pacing, external cardiac pacing, central venous catheterization (subclavian, internal jugular, and femoral routes), cricothyroidotomy (needle, and surgical), diagnostic peritoneal lavage (DPL), dislocation reduction, closed fracture reduction, intraosseous line, oral and nasal endotracheal intubation, laceration repair, lumbar puncture, pericardiocentesis, thoracostomy tube and needle thoracostomy, open thoracotomy, venous cutdown, and vaginal deliveries. This set of procedures was chosen for several reasons. It includes all the procedures the RRC-EM specifies in the program information forms and several additional critical procedures performed by EM practitioners. In addition, among respondents these procedures were the ones most consistently tracked by program directors and represent a broad range of critical procedural experience. The means per graduating resident for each index procedure were added together to generate a mean index procedure sum (MIPS) per graduating resident for each residency program. These MIPSs for a residency were then compared by postgraduate format, hospital type, and ED volume. The MIPS represents an estimate of the global procedural experience a graduating resident records at a given institution. It is important to note, however, that this approach has not been validated and therefore provides only an educated guess of global experience. A similar analysis was performed for resuscitations (medical and trauma resuscitations for adults and pediatrics), and mean index resuscitation sums (MIRSs) per graduating resident were compared by postgraduate format, hospital type, and ED volume. Data Analysis. Descriptive statistics were used for most data, and subgroups were compared by analysis of variance for groups demonstrating a normal distribution and the Kruskal-Wallis test for nonparametric comparisons.

3 730 COMPETENCY Hayden, Panacek PROCEDURAL COMPETENCY IN EM TABLE 1. Average Number of Procedures per Graduating Resident Arthrocentesis Transvenous External Subclavian Internal Jugular Femoral Cutdown Mean (95% CI) 11 (8, 14) 2 (1, 3) 6 (4, 8) 23 (16, 30) 14 (9, 18) 18 (10, 27) 2 (1, 2) 1 (0, 2) 3 (2, 5) Range IQR* Chest Tube Diagnostic Peritoneal Lavage Dislocation Fracture Oral Nasal Unspecified Intraosseous Line Mean (95% CI) 17 (14, 20) 8 (6, 10) 17 (14, 21) 25 (18, 33) 65 (46, 85) 6 (4, 9) 75 (62, 87) 2 (1, 3) Range IQR Laceration Repair Lumbar Puncture Needle Pericardiocentesis Thoracotomy Vaginal Delivery Mean (95% CI) 147 (116, 177) 39 (33, 46) 5 (4, 6) 3 (2, 5) 17 (13, 21) Range IQR Mean (95% CI) 107 (70, 144) 19 (13, 26) 74 (49, 100) 13 (9, 18) Range IQR * IQR = interquartile range. RESULTS Of the 112 approved EM programs, 92 responded, for an overall response rate of 82%. Twenty-seven stated that they either had not graduated a residency class or did not have complete procedure information and therefore did not have data for the primary outcome measure. These programs were not included in the data analysis. Of the 85 programs eligible for inclusion, 65 (76%) sent us procedural information. Basic characteristics of the nonresponding programs were evaluated by UCSD support staff according to the information in the electronic residency catalog, and no significant difference was found compared with respondents with respect to program format, location, and hospital type. Seventy-two percent of the respondents were PG-123 format, 15% PG-234, and 9% PG Based on the primary training site, 58% could be classified as university hospital-based, 24% were community hospital programs, and 18% were county/public hospitals. The respondents were fairly well distributed geographically, with 23% from the central region, 23% eastern region, 21% midwest region, 11% southern region, and 22% the west. Seventy-seven percent sent actual procedure logs; 23% sent program information forms. Seventy-four percent stated that information represented actual recorded numbers; 26% noted that information represented a best-guess estimate. We split the respondents into the group that reported actual numbers and those that reported a best-guess estimate and found no statistically significant difference in the numbers of index procedures, thus, we thought we could combine the information in these groups for further comparisons. Reported ED volumes ranged from 34,000 to 190,000 per year, with a mean volume of 58,000. All the respondents included in this study reported the average total number of a given procedure per graduating resident. This information is presented by selected procedures in Table 1. Only 24 respondents were able to delineate procedures by individual postgraduate year, making comparisons among year groups within programs difficult. Respondents were categorized by program postgraduate year format. The average number of selected procedures per graduating resident by program type is shown in Table 2. The only statistically significant differences found were the following (95% confidence intervals in parentheses): DPL PG-123, 6 (4 to 8); PG-1234, 13 (5 to 21); PG-234, 11 (4 to 19), p = trauma resuscitations PG-123, 58 (39 to 77); PG-1234, 209 (57 to 540); PG-234, 92 (17 to 166), p = medical resuscitations PG-123, 16 (10 to 23); PG- 1234, 52 (17 to 97); PG-234, 16 (4 to 27), p = Responding residencies were next grouped into three different hospital types based on the primary training site: university, community, and county/ public hospitals. The results of this com-

4 ACADEMIC EMERGENCY MEDICINE July 1999, Volume 6, Number TABLE 2. Average Number of Procedures per Graduating Resident by Program (PG) Type Mean (95% CI) Arthrocentesis Transvenous External Subclavian Internal Jugular Femoral Cutdown PG (6, 12) 2 (1, 2) 5 (3, 7) 21 (9, 31) 13 (7, 19) 14 (8, 20) 2 (1, 2) 1 (0, 2) 2 (1, 4) PG (3, 32) 2 (1, 3) 16 ( 9, 40) 27 (2, 52) 11 (4, 23) 31 (3, 85) 1 (1, 2) 1 (0, 3) 6 ( 6, 18) PG (3, 35) 4 (1, 7) 4 (3, 6) 27 (15, 39) 17 (4, 30) 24 (8, 69) 3 (2, 4) 1 (0, 3) 5 (1, 9) Chest Tube Diagnostic Peritoneal Lavage Dislocation Fracture Oral Nasal Unspecified Intraosseous Line PG (13, 20) 6 (4, 8)* 17 (13, 20) 25 (16, 35) 72 (44, 100) 5 (3, 7) 75 (60, 91) 1 (1, 2) PG (10, 31) 13 (5, 21)* 19 (4, 33) 45 (29, 61) 68 (48, 87) 12 (3, 19) 94 (58, 130) 4 ( 21, 29) PG (12, 25) 11 (4, 19)* 19 (5, 33) 19 (7, 30) 45 (18, 71) 9 (4, 13) 56 (36, 76) 3 (1, 5) Laceration Repair Lumbar Puncture Thoracotomy Vaginal Delivery PG (109, 185) 37 (30, 45) 5 (3, 6) 2 (1, 3) 16 (12, 20) PG (73, 286) 48 (21, 74) 8 (1, 15) 13 ( 7, 32) 17 (4, 28) PG (53, 196) 43 (24, 63) 5 (2, 9) 3 (1, 5) 22 (3, 40) PG (56, 139) 16 (10, 23)* 58 (39, 77)* 13 (8, 17) PG (81, 500) 52 (17, 97)* 209 (57, 540)* 20 (1, 84) PG (38, 108) 16 (4, 27)* 92 (17, 166)* 11 (2, 25) * p < parison are shown in Table 3. The only statistically significant differences found were the following: university, 2 (1 to 2); community, 2 (1 to 3); county, 4 (1 to 6), p = Intubation unspecified university, 88 (72 to 105); community, 58 (33 to 83); county, 57 (27 to 87), p = The last comparison made of individual procedures was based on ED volume. The median ED volume of respondents was 52,000, and therefore TABLE 3. Average Number of Procedures per Graduating Resident by Hospital Type Mean (95% CI) Arthrocentesis Transvenous External Subclavian Internal Jugular Femoral Needle Pericardiocentesis Cutdown University 12 (8, 17) 7 (4, 10) 7 (4, 10) 25 (15, 35) 12 (7, 18) 22 (10, 34) 2 (1, 2)* 1 (0, 2) 3 (1, 5) Community 8 (3, 13) 5 (2, 8) 5 (2, 8) 19 (7, 39) 20 (4, 32) 15 (13, 19) 2 (1, 3)* 1 (0, 1) 3 ( 2, 7) County 9 (5, 13) 4 (2, 8) 5 (2, 8) 20 (3, 37) 16 (4, 27) 6 ( 3, 16) 4 (1, 6)* 2 (0, 3) 6 (2, 11) Chest Tube Diagnostic Peritoneal Lavage Dislocation Fracture Oral Nasal Unspecified Intraosseous Line University 19 (14, 23) 9 (6, 11) 19 (13, 25) 29 (17, 40) 64 (36, 92) 6 (3, 9) 88 (72, 105)* 2 (1, 3) Community 16 (10, 21) 5 (3, 6) 14 (10, 19) 24 (10, 39) 78 (33, 120) 3 (1, 9) 58 (33, 83)* 1 (0, 4) County 14 (9, 19) 9 (2, 15) 15 (9, 20) 20 (5, 35) 65 (10, 120) 9 (5, 13) 57 (27, 87)* 3 (2, 4) Laceration Repair Lumbar Puncture Needle Pericardiocentesis Thoracotomy Vaginal Delivery University 165 (122, 208) 41 (32, 50) 5 (4, 7) 4 (1, 7) 17 (13, 21) Community 116 (60, 171) 31 (22, 41) 5 (3, 7) 3 (1, 4) 19 (5, 32) County 124 (52, 196) 42 (19, 64) 5 (2, 8) 3 (1, 5) 19 (6, 32) University 127 (71, 182) 23 (14, 33) 87 (51, 124) 16 (10, 22) Community 73 (20, 126) 9 (1, 18) 27 (1, 58) 5 (1, 16) County 69 (44, 93) 13 (7, 19) 58 (26, 90) 10 (4, 17) * p < 0.05.

5 732 COMPETENCY Hayden, Panacek PROCEDURAL COMPETENCY IN EM TABLE 4. Average Number of Procedures per Graduating Resident by ED Volume Mean (95% CI) Arthrocentesis Transvenous External Subclavian Internal Jugular Femoral Cutdown Small (<50,000) 12 (5, 16) 2 (1, 3) 5 (1, 9) 21 (11, 27) 10 (3, 16) 6 (1, 11) 1 (1, 2) 1 (0, 1) 2 (0, 3) Large ( 50,000) 10 (8, 14) 3 (2, 4) 5 (3, 6) 25 (15, 37) 17 (11, 23) 4 (1, 8) 2 (1, 3) 1 (0, 1) 2 (0, 2) Chest Tube Diagnostic Peritoneal Lavage Dislocation Fracture Oral Nasal Unspecified Intraosseous Line Small (<50,000) 19 (13, 25) 8 (4, 10) 18 (9, 22) 17 (5, 29) 53 (27, 75) 7 (2, 10) 80 (52, 103) 2 (1, 3) Large ( 50,000) 16 (13, 18) 8 (5, 11) 17 (14, 23) 20 (13, 27) 78 (45, 107) 6 (4, 9) 71 (59, 87) 2 (1, 3) Laceration Repair Lumbar Puncture Needle Pericardiocentesis Thoracotomy Vaginal Delivery Small (<50,000) 101 (67, 135) 36 (24, 39) 4 (2, 6) 4 (1, 10) 17 (10, 22) Large ( 50,000) 142 (99, 184) 42 (35, 54) 5 (3, 6) 3 (2, 4) 18 (12, 24) Small (<50,000) 123 (46, 167) 21 (10, 30) 92 (29, 134) 15 (6, 23) Large ( 50,000) 90 (56, 158) 18 (9, 29) 56 (41, 96) 12 (7, 18) There were no statistically significant differences between groups. we arbitrarily defined large programs as having 50,000 or more visits per year, and small programs with less than 50,000. The results of this comparison are shown in Table 4. There was no statistically significant difference found in the average number of a given procedure per graduating resident by ED volume. In addition to making comparisons for individual procedures, we wanted to assess whether there were any significant differences globally in the reported number of procedures between different residency settings and formats. The MIPS and MIRS were calculated as specified in the Methods section and are shown in Tables 5 and 6, respectively. The total MIPS per graduating resident was 403 (95% CI = 342 to 465). For PG-123 programs it was 383 (95% CI = 310 to 455), PG-1234 formats, 544 (95% CI = 447 to 642); and PG-234 residencies, 418 (95% CI = 233 to 603) per graduating resident, p = The MIPS for community hospitals was 318 (95% CI = 206 to 431); county hospitals, 369 (95% CI 232 to 505); and university hospitals, 453 (CI = 364 to 542) per graduating resident, p = The MIPS for small-volume programs was 364 (95% CI = 272 to 456); and for large volume programs, 428 (95% CI = 345 to 511), p = The MIRS for PG-123 programs was 181; PG programs, 466; and PG-234 residencies, 188 per graduating resident, p = The MIRS for community hospitals was 106; county hospitals, 147; and university hospitals, 248, p = Finally, the MIRS for small-volume programs was 223; and large-volume programs, 198, p = It was apparent the MIRS data were being skewed by two programs that reported numbers of resuscitations more than double those reported by other programs. A repeat analysis was performed after excluding the number of medical resuscitations from these outlier programs. In this analysis, the MIRS total was 141 (95% CI = 92 to 191); MIRS for PG-123, 125 (95% CI = 75 to 175); PG-1234, 311 (95% CI = 105 to 728); and PG-234, 120 (95% CI = 31 to 209), p = A wide range of recorded resuscitations among proportionally fewer PG-1234 programs explains why this still did not achieve statistical significance. The MIRS for community hospital programs was 49 (95% CI = 3 to 96); county hospitals, 86 (95% CI = 31 to 141); and university hospitals, 190 (95% CI = 110 to 269), p = The MIRS for small programs was 161 (95% CI = 66 to 256); and large programs, 129 (95% CI = 71 to 186), p = DISCUSSION To our knowledge, this study represents the most complete information to date on the global procedural experience of EM residents. A few studies in the EM literature have previously reported tracking resident procedures in a single program. Langdorf and Strange published their experience in developing a computerized tracking system for EM residents. 5 Dire and Kietzman reported their experience tracking procedures with a computerized system for nine EM residents over a 36-month residency program. 6 Our results are similar to the experience reported by these authors. Several prior studies have attempted to evaluate certain aspects of EM resident procedural training, and approaches to procedure tracking. Homan et al. in

6 ACADEMIC EMERGENCY MEDICINE July 1999, Volume 6, Number TABLE 5. Mean Index Procedure Sums (MIPSs) MIPS Mean (95% CI) Range Interquartile Range Total 403 (342, 465) 95 1, PG (310, 455) PG (447, 642) PG (233, 603) 179 1, Community 318 (206, 431) County 369 (232, 505) University 453 (364, 542) 95 1, Small 364 (272, 456) Large 428 (345, 511) 133 1, There were no statistically significant differences between groups. TABLE 6. Mean Index Resuscitation Sums (MIRSs) MIRS Mean (95% CI) Range Interquartile Range Total 208 (144, 273) PG (117, 245) PG ( 182, 1,115) PG (79, 299) Community 106 (19, 195) County 147 (95, 199) University 248 (154, 343) Small 223 (101, 356) Large 198 (121, 274) There were no statistically significant differences between groups reported their experience using an advanced surgical procedure laboratory to teach medical students and EM residents the performance of tube thoracostomy. 7 They found that a short didactic session, followed immediately by an animal laboratory that emphasized skill repetition, led to significantly improved procedural speed and retention of thoracostomy tube skills over time. Chapman et al. determined the differences between computer simulations, paper exercises, and real psychometric assessments in a pig model for the performance of open thoracotomy. 8 They found that observation of actual psychometric procedure performance, as in the pig model, was a superior method of assessment of procedural competency compared with paper or computer simulations. Gallagher et al. recently reported the results of a cross-sectional survey of EM residents, in which they found that EPs in EDs that have active EM residents perform on average 50% of a set of index procedures. 9 Numerous other authors have reported experience in teaching EM procedural skills. 10,11 Some authors have used animal laboratory settings and procedures on the newly dead in the ED 15,16 as well as computer-based technologies in teaching EM procedural skills. 17,18 The focus in the EM literature thus far has been on methods of acquiring initial procedural skills and approaches to documenting procedural skills in individual programs. To our knowledge, this study represents the first effort to report the combined experience across EM residents. A number of interesting findings have resulted from this information. First, there is a wide range of reported procedural experience among EM residents. Likewise, there is great variation in the methods of recording procedures. Some programs record only the procedures identified in the program information forms (PIFs) of the RRC application. Other programs are tied into their hospital computer billing system and track nearly every billable procedure, even some that are not identified in the EM core content. Despite these differences, however, it would seem that similar procedural experience is recorded in a variety of different settings. The only statistically significant differences found among subgroups were the following: DPL, pediatric medical resuscitation, and adult trauma resuscitation when compared by program format; and cricothyroidotomy and unspecified intubation compared by hospital type. These differences are not likely to represent clinically significant differences in procedural experience. It may be surprising to discover that greater differences do not exist between program types, hospital types, or ED volumes. This contradicts popularly held beliefs that adequate procedural experience can be gained only at largevolume, inner-city-based programs. The data from this study do not bear this out. In fact, no global difference in MIPS or MIRS was found when comparisons were made by format or setting. Another finding of this study is that many EM residents have limited reported experience in certain critical procedures such as cricothyroidotomy, thoracotomy, and intraosseous lines. Some programs even reported limited experience in intubation and central venous access. Although with some common procedures such as laceration repair, arterial lines, or lumbar puncture, residents may stop rigorously tracking them as more experience is gained, procedures such as cricothyroidotomy, intubation, and thoracotomy are generally much more carefully recorded by residents. The numbers represented in this study are likely to be accurate for such procedures. This is a finding that confirms anecdotal experience and anecdotal reports. While this study does not assess the quality of training in critical procedures that are performed infrequently, such limited clinical experience raises the issue of needing alternative meth-

7 734 COMPETENCY Hayden, Panacek PROCEDURAL COMPETENCY IN EM ods of gaining this experience, such as animal, cadaver labs, or computer simulations. However, are such experiences directly translatable to the clinical ED setting? Is it possible to achieve competency in certain procedures from simulated patient encounters? The answers to these questions remain unknown at present but deserve further investigation. Historically, training programs in all specialties have stressed quantity of procedures as a surrogate of procedural competency and established minimum numbers of given procedures that must be performed with little attention to the quality of procedural training. These numbers are typically derived in an arbitrary fashion by a consensus of experts. Our results demonstrate a widely divergent range of reported numbers of procedures performed by EM residents and yet there does not seem to be significantly different global procedure experiences between EM residency formats. While repeated psychomotor performance of procedures seems necessary to developing adequate proficiency, there is currently no literature available that establishes minimum or optimum numbers of such repetitions. In addition, any such minimum numbers could vary for each procedure and certainly vary from resident to resident. It may not be possible to empirically derive what the minimum numbers should be and, therefore, a different approach that emphasizes quality and not quantity would be more meaningful. LIMITATIONS AND FUTURE QUESTIONS There are a number of limitations to this study. First, the data were generated from the procedure information sent by the respondents, which was not standardized. There was a great deal of variability in both recording and reporting methods. This was self-reported, not objectively measured data. This was particularly important when recognizing that there is a lack of uniform definitions of procedures. Additionally, there was a wide variation in reported numbers of medical and traumatic resuscitations. A few programs reported a great many more resuscitations than the average EM residency. The most likely explanation for the skew in the resuscitation data lies in what a given program defines as resuscitation. It is impossible to determine from the information sent how each program defines a medical or trauma resuscitation. Is it just full codes, all patients who get intubated, any use of pressors, or any level 5 (comprehensive) level of care patient? This lack of uniformity may explain the wide variation in numbers for the medical and trauma resuscitations. For most EM procedures reported in this study, the definition of the procedure is clearer. For instance, there is little ambiguity regarding the definition of central line placement or a lumbar puncture. The wide variation in resuscitations could also be due to disparity in resident self-reporting or these could represent real experiential differences. Further study will need to be performed to clarify this issue. Most programs depend on resident self-reporting to complete their procedure logs. This may lead to underreporting or overreporting of many common procedures; however, in the authors experience, most residents carefully track more critical or infrequent procedures and thus we believe the data in this study are a fair approximation of the actual experience for these procedures. The data likely do not underestimate this experience. No programs distinguished whether their numbers included laboratory (animal or cadaver) or alternative experiences to actual patient encounters. Additionally, information was not available on whether the recorded procedural experience was obtained in an ED setting, in an out-of-hospital setting, or on other off-service rotations. Last, the data for PG-1234 programs may be skewed somewhat higher than other program formats because they record four years worth of procedural experience. This study also reports only the average number of procedures per graduating resident, not the variability within individual residency programs. While the average for the residency may be adequate, individual residents may receive significant variation in the range of procedures they perform and program directors may need to establish some means of monitoring this and providing for the requisite experience when it is necessary. One solution to this problem may be the development of standardized performance checklists for procedures that are tracked by the residency program. Another limitation in interpretation of the results is that the MIPS, and MIRS as a measure of global procedural experience, has not been previously validated. Essentially these two values represent the sum of a set of index procedures and resuscitations per graduating resident. These sums were derived from information that was easily obtained in the data we collected. More complicated analyses of global experience could have been performed; however, we thought that there was enough lack of uniformity in procedure reporting to make such comparisons unwarranted. The MIPS and MIRS were generated for the convenience of reporting global procedure comparisons and are not intended for use by training programs. Though this survey has a number of limitations, the information should still be useful to EM residency programs in the course of evaluating their own resident procedure experience. This will allow some measure for comparisons. Previously,

8 ACADEMIC EMERGENCY MEDICINE July 1999, Volume 6, Number there were no published standards that could be used for this purpose. It would be difficult to overcome many of the limitations encountered in this study without standardizing the process by which procedures are defined, recorded, and evaluated, at all EM residency training programs. Such standardization could allow for more meaningful comparisons between programs in the future. Despite the limitations of this study, and no matter how the data were analyzed, there were very few clinically significant differences found in the number of reported procedures between different EM residency settings and formats. This suggests that similar procedural experiences can be obtained in a variety of EM programs. This supports the notion that it may be best to allow individual residency programs to determine how to provide procedural training to their residents, perhaps within certain guidelines. Each program may develop an acceptable approach unique to its system, and evidence provided by this study would indicate that globally there are not major differences in procedural training among different EM residency settings and formats. Moreover, it is possible that a minimum quantity of procedures for every resident may be less important than the quality of each procedural experience in terms of both associated instruction and process of evaluation. CONCLUSIONS To our knowledge, this is the first study of the range of EM resident procedure experience across the spectrum of EM residency types and settings. Overall, there are few statistically significant differences in procedure experience among different program formats. Similar experiences are recorded at a variety of different hospital types or ED volumes. However, some programs report very limited EM resident experience with selected critical procedures. There is also a large variation currently in the types and numbers of procedures recorded, and lack of uniformity in the definition of certain procedures, between programs. The authors thank the Council of Emergency Medicine Residency Directors (CORD) and the SAEM Residency Committee for their thoughtful input into the planning of this project. This project was approved and supported by the CORD Board of Directors. The authors also gratefully acknowledge Paul Schragg (supported by CRC grant NIH M01 RR00827) for his assistance with the statistical analysis. References 1. Task Force on the Core Content for Emergency Medicine Revision. Core content for emergency medicine. Acad Emerg Med. 1997; 4: Residency Review Committee for Emergency Medicine. Program Information Forms. Chicago, IL: American Board of Emergency Medicine, American Board of Internal Medicine. Guide to Evaluation of Residents in Internal Medicine, Philadelphia, PA: American Board of Internal Medicine, New York, Official Compilation of Codes, Rules, and Regulations of the State of New York, Part 405, Subchapter A of Chapter V ( Facilities Minimum Standards), Effective January 1, Langdorf MI, Strange G. Computerized tracking of emergency medicine resident clinical experience. Ann Emerg Med. 1990; 19: Dire DJ, Kietzman LI. A prospective survey of procedures performed by emergency medicine residents during a 36-month residency. J Emerg Med. 1995; 13: Homan CS, Viccellio P, Thode HC Jr, Fisher W. Evaluation of an emergency-procedure teaching laboratory for the development of proficiency in tube thoracostomy. Acad Emerg Med. 1994; 1: Chapman DM, Rhee KJ, Marx JA, et al. Open thoracotomy procedural competency: validity study of teaching and assessment modalities. Ann Emerg Med. 1996; 28: Gallagher EJ, Coffey J, Lombardi G, Saef S. Emergency procedures important to the training of emergency medicine residents: who performs them in the emergency department? Acad Emerg Med. 1995; 2: Nelson MS. Models for teaching emergency medicine skills. Ann Emerg Med. 1990; 19: Thomas H. Teaching procedural skills: beyond the see one do one. Acad Emerg Med. 1994; 1: Olshaker JS, Brown CK, Arthur DC, Tek D. Animal procedure laboratory surveys: use of the animal laboratory to improve physician confidence and ability. J Emerg Med. 1989; 7: Sternbach GL, Rosen P. Use of laboratory animals in the teaching of emergency procedures. J Am Coll Emerg Physicians. 1977; 6: Chapman DM, Marx JA, Honigman B, Rosen P, Cavanaugh SH. Emergency thoracotomy: comparison of medical student, resident, and faculty performances on written, computer, and animal-model assessments. Acad Emerg Med. 1994; 1: McNamara RM, Monti S, Kelly JJ. Requesting consent for an invasive procedure in the newly dead [abstract]. Acad Emerg Med. 1994; 1(Mar/Apr):A52 A Olsen JC, Spilger SP, Windisch T, Fisher DL, Chaney D. Teaching cricothyrotomies on the recently deceased in the emergency department [abstract]. Acad Emerg Med. 1994; 1(Mar/Apr):A Chapman DM. Use of computer-based technologies in teaching emergency procedural skills. Acad Emerg Med. 1994; 1: Chapman DM, Marx JA, Honigman B, Rosen P. Final report: computer-based teaching and evaluation of critical emergency medicine procedure skills. Dallas, TX: Emergency Medicine Foundation, 1990.