The 1999 Institute of Medicine report increased national

Transcription

1 The Value of Monitoring Frozen Section Permanent Section Correlation Data Over Time Stephen S. Raab, MD; Joseph A. Tworek, MD; Rhona Souers, BS; Richard J. Zarbo, MD, DMD Context. The effectiveness of the long-term monitoring of errors detected by frozen section permanent section correlation is unknown. Objective. To determine factors important in laboratory improvement in frozen section permanent section discordant and deferral rates by participation in a multiinstitutional continuous quality improvement program. Design. Participants in the College of American Pathologists Q-Tracks program self-reported the number of anatomic pathology frozen permanent section discordant and deferred cases in their laboratories by prospectively performing secondary review of intraoperative consultations. Laboratories participated in the program for 1 to 5 years and reported their data every quarter. We calculated mean and median discordant and deferred case frequencies and used mixed linear modeling to determine if length of participation in the program was associated with improved performance. Participants. One hundred seventy-four laboratories self-reported data. Main Outcome Measures. Mean frozen permanent section discordant and deferred diagnostic frequencies and changes in these frequencies over time were measured. Results. The mean and median frozen permanent section discordant frequencies were 1.36% and 0.70%, respectively. The mean and median deferred diagnostic frequencies were 2.35% and 1.20%, respectively. Longer participation in the Q-Tracks program was significantly associated (P.04) with lower discordant frequencies; 4- or 5-year participation showed a decrease in discordant frequency of 0.99%, whereas 1-year participation showed a decrease in discordant frequency of 0.84%. Longer participation in the Q-Tracks monitor was associated with lower microscopic sampling frequencies for discordant diagnoses (P.04). Increased length of participation in the Q-Tracks program was significantly associated (P.04) with lower deferred diagnostic frequencies. Conclusions. Long-term monitoring of frozen permanent section correlation is associated with sustained improvement in performance. (Arch Pathol Lab Med. 2006;130: ) The 1999 Institute of Medicine report increased national awareness of medical errors and patient safety. 1 Medical specialties were encouraged to develop methods of error reporting and to share error data for benchmarking and developing quality improvement initiatives. The College of American Pathologists (CAP) established the Q- Probes program in part to benchmark particular types of anatomic pathology error, such as those detected through forms of secondary review. 2 In the past 15 years, the CAP performed 3 large studies (encompassing 232, 297, and 416 institutions, respectively) measuring frozen section permanent section discrepancies that established fixed benchmarks for those organizations participating and sharing data. 3 5 Dankwa and Davies 6 hypothesized that the frozen permanent section discrepancy frequency was not reducible to less than 2%. Assuming the absence of reporting bias, this hypothesis was disproved by the 3 frozen permanent Accepted for publication October 27, From the Department of Pathology, University of Pittsburgh, Pittsburgh, Pa (Dr Raab); the Department of Pathology, St Joseph Mercy Hospital, Ann Arbor, Mich (Dr Tworek); the College of American Pathologists, Northfield, Ill (Ms Souers); and the Department of Pathology, Henry Ford Hospital, Detroit, Mich (Dr Zarbo). The authors have no relevant financial interest in the products or companies described in this article. Reprints: Stephen S. Raab, MD, Department of Pathology, University of Pittsburgh, Centre Ave, Pittsburgh, PA ( raabss@upmc.edu). section Q-Probes studies that reported a mean concordant diagnostic frequency between 98.6% and 98.2%. 3 5 However, the 90th percentile for individual institutional discordant frequency was 3.13%, indicating room for improvement for some institutions. 3 5 Reducing error is a multifaceted task, and many barriers may prevent the success of error reduction programs. 7,8 In addition, confounding variables may make it difficult to determine whether error reduction initiatives have been effective; for example, establishing an error reporting mechanism may lead to increased error frequencies because when one looks for error, more errors tend to be identified. 8 In anatomic pathology, institutions have limited opportunities to share and track error data over time, a necessary component of a continuous quality improvement program designed to decrease errors. In 1998, the CAP established the Q-Tracks program as a continuous quality improvement tool, and one of the Q-Tracks studies measured frozen permanent section discordant and deferred diagnostic frequencies. 9 In this study, we report the frozen permanent section Q-Tracks data that illustrate how individual institutions performed over time and how continuously monitoring and benchmarking data affect error rates. We compared institutional performance with prior performance and determined if the length of time participating affected error frequencies. MATERIALS AND METHODS An overview and specific data from the Q-Tracks program were described previously. 9 The frozen permanent section dis- Arch Pathol Lab Med Vol 130, March 2006 Frozen Section Correlation Raab et al 337

2 cordance Q-Tracks study was based on 3 Q-Probes studies that were reported previously. 3 5 In the Q-Probes program, institutions were compared with each other. In the Q-Tracks program, institutions also may compare themselves with other institutions. However, in this study, we measured how institutions performed over time by comparing an institution to itself. In the frozen permanent section discordance Q-Tracks program, institutions participated on a year-to-year basis, and in each Q-Tracks study, individual institutions participated a variable number of years. The frozen permanent section discordance Q-Tracks study was performed from January 1, 1999, to December 31, Study Population A total of 174 institutions participated in the frozen permanent section discordance Q-Tracks program. Twenty-six institutions participated 4 or 5 years, 17 participated 3 years, 36 participated 2 years, and 91 participated 1 year. Of the participating institutions, 32.2% were teaching hospitals, 60.3% were nonteaching hospitals, and the teaching status of 7.5% was unknown. Thirtyseven of the participating institutions had a bed size between 1 and 250, 84 had a bed size between 251 and 500, and 37 had a bed size greater than 500. The participating institutions reported data quarterly to the CAP, and data were collected on standardized report forms. Program participants who did not submit data for at least 2 quarters during the study period were excluded from analysis. Some study laboratories submitted data for as few as 2 quarters ( 1 year), whereas other laboratories submitted data for up to 20 quarters (5 years). To ensure a uniform data collection process, definitions of the data elements were provided to all participants. The definitions of terms used in this Q-Tracks study are listed below. Definitions Surgical Pathology Case. A single accession number (eg, S03-250) per patient surgical episode for which tissue was submitted for histopathologic examination and reporting. A surgical pathology case may contain any number of specimens (eg, S03-250A, S03-250B). Frozen Section Case. A surgical pathology case that is given a unique case accession number (composed of 1 or more specimens), in which at least 1 frozen section was performed. More than 1 specimen may be evaluated by frozen section in a single case. One or more blocks of tissue that are frozen may be evaluated for an individual specimen. Therefore, a single case may have multiple frozen section diagnoses for correlation. Frozen Section Diagnosis. Diagnosis rendered by frozen section evaluation of a given specimen. A specimen may have more than 1 frozen section diagnosis, which may be based on 1 or more frozen section blocks. In addition, a frozen section block may entail 1 or more frozen section diagnoses (eg, 1 frozen section may be used to evaluate both the type of neoplasm and the status of the adjacent margin). Frozen Section Diagnostic Event. Frozen section study resulting in an intraoperative diagnosis that is concordant or discordant with permanent sections or deferred at the time of the intraoperative consultation. Concordance. An adequate frozen section study with an intraoperative diagnosis that has complete agreement with permanent section. The concordant diagnostic frequency was calculated as the total number of concordant diagnoses divided by the total number of concordant and discordant diagnoses. Deferred Diagnosis. Diagnoses that are indeterminable at the time of frozen section examination. This type of diagnosis was not considered discordant. A deferred diagnosis may involve 1 or more specimens of a given case. The frozen section evaluation of the specimen may result in 1 or more deferred frozen section diagnoses. The deferred diagnostic frequency was calculated as the total number of deferred diagnoses divided by the total number of concordant, discordant, and deferred diagnoses. Discordance. An adequate frozen section study with an intraoperative diagnosis that has diagnostic disagreement with the permanent section. Discordant frozen section diagnoses may involve 1 or more specimens of a given case. Frozen section evaluation of a specimen may result in 1 or more discordant frozen section diagnoses; a discordant diagnosis may involve 1 or more frozen section blocks. The discordant diagnostic frequency was calculated as the total number of discordant diagnoses divided by the total number of concordant and discordant diagnoses. Exclusions (Not Included as Discordant Diagnoses). The following discordant diagnoses were excluded from analysis because we considered these observations beyond the scope of a typical frozen section evaluation: (1) discordance in type of carcinoma, except for small cell carcinoma versus other types of carcinoma (eg, adenocarcinoma, squamous cell carcinoma, large cell carcinoma); (2) discordance in the degree of differentiation or grade of carcinoma (usually this applies to squamous or adenocarcinoma and not to large cell carcinoma or small cell carcinoma); (3) discordance in the grade of dysplasia or dysplasia versus carcinoma in situ (intraepithelial neoplasm); (4) discordance in breast biopsy or excision specimens undertaken because of a mammographic abnormality (especially calcification) in which a gross lesion is not identified; (5) discordance in thyroid gland lobectomy specimens in which a well-circumscribed nodule exhibiting follicular morphology was found; (6) discordance in breast or other excision specimens for which the frozen section was undertaken only to determine the adequacy of tissue for tumor concentrations of estrogen and/or progesterone binding protein; and (7) discordance in frozen sections performed for evaluation of tumor margins that are oriented on the cryostat so as to utilize and exhaust the true margin (en face). Permanent sections of the original frozen section block, therefore, would not represent true margin. Data Collection Process The participating laboratories reviewed each case evaluated by frozen section each month. 9 They compared each frozen section diagnosis with the permanent section diagnosis. For each case, the laboratory recorded the date, specimen identification number, and diagnosis type (ie, concordant, discordant, or deferred). The laboratories tracked the number of concordant diagnoses between discordant diagnoses by recording a sequential tally of the comparisons performed and whether the diagnosis was concordant, discordant, or deferred. These data were used to trend performance over time. The participating laboratories were provided worksheets for tracking the number of cases between discordant diagnoses, although laboratories could track these data in other ways, depending on their laboratory information system (eg, using computer reports or existing logs of cases). For each discordant diagnosis, the laboratory listed an anatomic site/organ system (chosen from a specified list) and the reason for the frozen section. The laboratories listed the probable reason for the discordance (ie, misinterpretation, microscopic sampling, or gross sampling). A microscopic sampling error occurred when the diagnostic tissue was seen on a permanent section of the frozen section block, but not seen on the frozen section itself (ie, the block was not sectioned deep enough to obtain the diagnostic material at the time of the intraoperative consultation). A gross sampling error occurred when the tissue itself selected for the frozen section did not have the diagnostic material (ie, deeper sections of the block did not show diagnostic tissue, but other blocks, representing material not selected for frozen section, contained diagnostic material). The laboratories optionally could list the pathologist and a consultant pathologist (using deidentification codes). For each deferred diagnosis, the laboratory listed the anatomic site/organ system, reason for the frozen section, pathologist (optional), and consultant pathologist (optional). At the end of each month, the laboratory listed the total number of frozen sections performed during the month. Laboratory demographic and study-specific questionnaires were completed for each year of participation. These questionnaires obtained information regarding institutional characteristics and intra- 338 Arch Pathol Lab Med Vol 130, March 2006 Frozen Section Correlation Raab et al

3 operative frozen section policies and practices. Each institution could also submit 5 characteristics from a master list to construct their own custom peer group. Peer groups could be defined by such information as institutional type, training programs, administrative services, and bed size. This information allowed participating institutions to benchmark against other institutions using information they thought was important for comparison. Table 1. Variables Analyzed in the Mixed Linear Model Number of participating quarters Quarter of participation Institution is inspected by the College of American Pathologists (yes or no) Institution is inspected by the Joint Commission on Accreditation of Healthcare Organizations (yes or no) Institution is inspected by the Commission on Office Laboratory Accreditation (yes or no) Institution is inspected by the Food and Drug Administration (yes or no) Institution is inspected by the state (yes or no) Teaching hospital (yes or no) Nongovernmental institution (yes or no) Governmental, nonfederal institution (yes or no) Governmental, federal institution (yes or no) Institution trains pathology residents (yes or no) Number of licensed beds Table 2. Distribution of Yearly Concordant Diagnostic Frequencies Arch Pathol Lab Med Vol 130, March 2006 Frozen Section Correlation Raab et al 339 Year Table 3. Year No. of Institutions All Institutional Percentiles 10th 25th 50th 75th 90th Distribution of Yearly Discordant Diagnostic Frequencies No. of Institutions All Institutions Percentiles 10th 25th 50th 75th 90th Analysis We calculated the diagnostic discordance frequency as the number of discordant frozen section diagnoses divided by the total number of concordant and discordant diagnoses, multiplied by 100. The diagnostic concordance frequency was the total number of concordant frozen section diagnoses divided by the total number of concordant and discordant diagnoses, multiplied by 100. The deferred diagnosis frequency was the number of deferred frozen section diagnoses divided by the total number of frozen section diagnoses (including concordant, discordant, and deferred diagnoses). For the quarterly data summary, change in institutional performance was assessed with a rare event analysis report. Because the average number of discordant diagnoses was low (ie, 1 per month in some institutions), we decided that an ordinary control chart of the discordant frequency was not sufficiently sensitive to accurately track performance. Therefore, instead of calculating this frequency at regular intervals, we monitored the number of concordant diagnoses between consecutive discordant diagnoses. We summarized institutional performance by providing the institution s position relative to the peer group percentile distributions. In general, the peer group used to calculate percentiles consisted of all reporting institutions. In some instances, individual participants were compared with a subset of other participants when an uncontrollable variable was known to exert a strong influence on the outcome of interest. To correct for the impact of program disenrollment, performance was tabulated according to the number of quarters of continuous enrollment. Trends were assessed by linear regression; if the median participant had a performance-over-time regression slope greater than 0 at the P.05 confidence level, overall improvement over time was assumed to be associated with the duration of participation. Longitudinal trends were edited for partial-year participants who exited before the year end to ensure that any decline or trend was due to improvement in performance. We compared the performance of each institution with prior performance as the number of years of participation increased. The hypothesis that longer participation resulted in a decrease in the frozen section correlation frequency and a change in breakdown percentages was formally tested using the PROC MIXED procedure in SAS (SAS, Cary, NC). 10 This procedure performs mixed-model analysis. Because the frozen permanent section discordance Q-Tracks program was offered as a yearly subscription, some laboratories joined or dropped out of the program during the 5-year study period. We excluded participants with random data submission patterns. We then determined the quarter of participation, institutional practice characteristics, and demographic information for each datum value (eg, discordant diagnostic frequency). The longitudinal data examined for this study include multiple equally spaced data points for each institution. These data are referred to as repeated measures. Repeated measures from the same institution are likely more correlated than measurements from any 2 random institutions. An institution s repeated measures are likely more correlated for quarters closer together than for quarters farther apart. The variances of repeated measures can also change over time. To account for this complicated covariance structure, we fit a mixed linear model. The mixed linear model is a generalization of the standard linear model, with the generalization being that the data can exhibit correlation and nonconstant variability. The data were introduced into the model to determine if continued program participation was associated with improved performance and if there were institutional practice characteristics or demographic variables with improved performance. The independent variables analyzed in the mixed model are shown in Table 1. RESULTS Aggregate Data For the 5 years of the study, institutions monitored frozen section diagnoses and found a total of concordant diagnoses and a concordant diagnostic frequency of 98.9%. The total number of discordant diagnoses was 2649, and the discordant diagnostic frequency was 1.1%. The total number of deferred diagnoses was 4611, and the deferred diagnostic frequency was 1.8%. The yearly institutional frequency distributions of the concordant, discordant, and deferred diagnoses are shown in Tables 2, 3, and 4, respectively. The characteristics of discordant diagnoses are shown in Table 5. The anatomic sites for deferred diagnoses are shown in Table 6. Longer participation in the Q-Tracks monitor was associated with lower discordant frequencies (P.04) (Figure 1). The institutions that participated for 4 or 5 years showed a greater reduction in the discordant diagnostic

4 Table 4. Year Distribution of Yearly Deferred Diagnostic Frequencies No. of Institutions All Institutional Percentiles 10th 25th 50th 75th 90th frequency than institutions that participated fewer years. Institutions that participated for only 1 year still showed a decrease in discordant frequency over time. Lower numbers of licensed hospital beds was significantly associated (P.04) with lower discordant frequencies; the median discordant frequencies for institutions with 1 to 250 beds, 251 to 500 beds, and more than 500 beds were 0%, 0.53%, and 1.20%, respectively. Longer participation in the Q-Tracks monitor was associated with lower deferred diagnostic frequencies (P.04) (Figure 2). Institutions that participated for 3 or more years showed almost a 2% reduction in their deferred diagnostic frequency. Institution type was significantly associated (P.001) with deferred diagnostic frequencies; governmental institutions showed a lower median deferred diagnostic frequency (0.52%) compared to nongovernmental institutions (1.85%). Longer participation in the frozen permanent section discordance Q-Tracks monitor was associated with lower microscopic sampling frequencies for discordant diagnoses (P.04) (Figure 3). Institutions that participated for at least 3 years showed a decrease in sampling error frequency, whereas institutions that participated 2 years or fewer showed an increase in sampling error frequency. COMMENT Data from the frozen permanent section discordance Q- Tracks monitor show that participation led to overall improvement and a reduction in error frequencies. In this Q- Tracks monitor, we compared the performance of each institution with prior performance. The mean decrease in discordant frequency for institutions that participated for 4 or 5 years was 0.99%. In practical terms, this means that an institution that started with the median discordant frequency of 1.33% could have reduced its discordant fre- Table 5. Reason for diagnostic discordance Microscopic sampling Gross sampling Misinterpretation Technical problems Lack of important clinical data Specimen labeling error Other reason Anatomic site Skin Lymph nodes for metastatic neoplasm (all sites) Female genital system Thyroid and parathyroid gland Breast Oral cavity, nasopharynx, pharynx, sinus, ear, and larynx Lungs, pleura, and mediastinum Gastrointestinal tract Soft tissue, bone, and joint Peritoneum and omentum Male genital system Central nervous system Urinary system Liver, gall bladder, and biliary tree Lymph nodes for disease other than metastatic neoplasm Pancreas Adrenal gland and paraganglionic system Cardiovascular system Other sites Diagnostic mission Establish/confirm a primary malignancy Determine adequacy of resection margins Establish/confirm a metastatic malignancy Expedite the availability of the diagnosis for the surgeon Confirm sufficient tissue is present Expedite the diagnosis for family members Microscopic identification of specimen Expedite the diagnosis for patient management No reason or unknown reason Confirm nature of tissue to direct tissue sampling Liability concern Acedemic or teaching protocol Requested in error Other Aggregate Characteristics of Discordant Diagnoses No. of Diagnoses Percentage of Diagnoses Arch Pathol Lab Med Vol 130, March 2006 Frozen Section Correlation Raab et al

5 Table 6. Anatomic Site of Deferred Diagnoses Anatomic Site No. of Diagnoses Percentage of Diagnoses Thyroid and parathyroid gland Soft tissue, bone, and joint Lymph nodes for disease other than metastatic neoplasm Central nervous system Female genital system Lungs, pleura, and mediastinum Breast Skin Lymph nodes for metastatic neoplasm (all sites) Oral cavity, nasopharynx, pharynx, sinus, ear, and larynx Gastrointestinal tract Urinary system Liver, gallbladder, and biliary tree Peritoneum and omentum Male genital system Pancreas Adrenal gland and paraganglionic system Cardiovascular system Other sites quency to 0.34% after 5 years of participation. This absolute reduction in errors (and not a percentage drop in errors) is tremendous. Institutions that participated for 4 or 5 years made 1 less error per 100 frozen sections; the fact that the error frequency is very low to begin with indicates that this change reflected marked improvement. The Institute of Medicine report argued for the establishment of national error reporting systems as an initial step to improve patient safety. 1 The frozen permanent section discordance Q-Tracks monitor created 1 of the first pathology patient safety databases, and only a few other error databases have been created specifically for pathology-detected errors. 9 The effectiveness of national patient safety databases in error reduction is only now being measured. 11,12 The frozen permanent section discordance Q- Tracks program was designed to improve practice through several avenues. 9 First, laboratories were able to benchmark their practices by comparison with other laboratories. Second, year-end Q-Tracks reports shared laboratory and institutional characteristics of Best Performers that other laboratories presumably could adopt to improve practice. Other studies showed that sharing error data through centralized databases resulted in program improvement, as exemplified by The Minnesota Society of Thoracic Surgeons database. 13,14 In 1993, the Minnesota Society of Thoracic Surgeons database was organized in response to a third-party payer demand for data about practice protocols and patient outcomes. The Minnesota Society of Thoracic Surgeons developed a physician-driven, standardized computerized process for the collection of individual provider and institutional performance measures and created a risk-adjusted statewide database. Over the years, a progressive decrease in operative mortality and length of stay has been observed, and the database is being increasingly used for national analyses, education, and policy decision making. In anatomic pathology, Raab et al 11,12 reported that a national anatomic pathology error database could be used to drive improvements in laboratory and clinical care. In this Q-Tracks study, the specific mechanisms by which long-term participants decreased the frequency of discrepant and deferred diagnoses are unknown. Initial improvement may have been an example of the Hawthorne effect, in which a positive impact resulted from increased attention placed on the task by the workers. 15,16 In this case, pathologists recorded their activities for each frozen section, thus increasing their attention on their work, leading to better performance. Lied and Kazandjian 16 reported that the Hawthorne effect formed the basis of some forms of quality improvement. 16 By using external observations, workers exhibited increased internal commitment that resulted in continuously improved performance. This improvement was based on individual responsibility maintained by periodic reinforcement of behaviors that lead to better performance. We do not know if certain factors such as changes in case type or specimen mix led to decreases in error frequencies for individual institutions. However, these factors probably were not the simultaneous drivers of improvement for all institutions. The findings from this study reinforce the data presented by Zarbo et al, 9 based on only 2 years of data. The frozen permanent section discordance Q-Tracks program did not monitor how laboratories used Best Performers characteristics, or even if these characteristics were correlated with fewer errors. Inherent in these characteristics were features, such as redundancy and subspecialization, which have been adopted by nonmedical fields in order to reduce error. 1,17 19 For example, one Best Performers laboratory policy was presentation of frozen section discrepancies to staff pathologists, and error revelation and discussion is a method reported to improve patient safety. 3,4 An error disclosure policy is part of a safety culture that encourages individuals to learn from previous mistakes Laboratories also could have adopted other Best Performers policies, such as double-viewing frozen sections, and Novis 23 showed that double-viewing cases prior to signout decreased errors, as evidenced by decreasing the number of amended reports. In this Q-Tracks program, it is possible that the reduction in errors was a result of biased reporting or the attrition of the more poorly performing laboratories. Controlling for these variables in a voluntary enrollment program is extremely difficult, and most voluntary patient safety programs similarly are limited in scope. These limitations do not devalue the improvement that some laboratories achieved, but illustrate the difficulties in establishing and utilizing national error benchmarking systems. Arch Pathol Lab Med Vol 130, March 2006 Frozen Section Correlation Raab et al 341

6 Figure 1. Mean reduction in discordant diagnostic frequency versus years of participation in the Q-Tracks program. Figure 2. Mean reduction in deferred diagnostic frequency versus years of participation in the Q-Tracks program. Figure 3. Mean change in microscopic sampling breakdown for discordant diagnoses versus year of participation in the Q-Tracks program. References 1. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; Zarbo RJ. Monitoring anatomic pathology practice through quality assurance measures. Clin Lab Med. 1999;19: Zarbo RJ, Hoffman GG, Howanitz PJ. Interinstitutional comparison of frozen section consultations: a College of American Pathologists Q-Probes study of 76,547 consultations in 297 North American Institutions. Arch Pathol Lab Med. 1991;115: Gephardt GN, Zarbo RJ. Interinstitutional comparison of frozen section consultations: a College of American Pathologists Q-Probes study of 90,538 cases in 461 institutions. Arch Pathol Lab Med. 1996;120: Novis DA, Gephardt GN, Zarbo RJ. Interinstitutional comparison of frozen section consultation in small hospitals: a College of American Pathologists Q- Probes study of 18,532 frozen section consultation diagnoses in 233 small hospitals. Arch Pathol Lab Med. 1996;120: Dankwa EK, Davies JD. Frozen section diagnosis: an audit. J Clin Pathol. 1985;38: Grzybicki DM. Barriers to the implementation of patient safety initiatives. Clin Lab Med. 2004;24: Eccles M, Grimshaw J, Campbell M, Ramsay C. Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual Saf Health Care. 2003;12: Zarbo RJ, Jones BA, Friedberg RC, et al. Q-Tracks: a College of American Pathologists program of continuous laboratory monitoring and longitudinal performance tracking. Arch Pathol Lab Med. 2002;126: Littell RC, Milliken GA, Stroup WW, Wolfinger RD. SAS Systems for Mixed Models. Cary, NC: SAS Institute Inc; Raab SS, Grzybicki DM, Zarbo RJ, Meier F, Geyer SJ, Jensen C. Anatomic pathology databases and patient safety. Arch Pathol Lab Med. 2005;129: Raab SS, Grzybicki DM, Janosky JE, et al. Clinical impact and frequency of anatomic pathology errors in cancer diagnosis. Cancer. 2005;104: Arom KV, Petersen RJ, Orszulak TA, et al. Establishing and using a local/ regional cardiac surgery database. Ann Thorac Surg. 1997;64: Grover FL. The Society of Thoracic Surgeons National Database: current status and future directions. Ann Thorac Surg. 1999;68: O Sullivan I, Orbell S, Rakow T, Parker R. Prospective research in health service settings: health psychology science and the Hawthorne effect. J Health Psychol. 2004;9: Lied TR, Kazandjian VA. A Hawthorne strategy: implications for performance measurement and improvement. Clin Perform Qual Health Care. 1998;6: Leape LL, Berwick DM. Five years after To Err is Human: what have we learned? JAMA. 2005;293: Bates DW, Cohen M, Leape LL, Overhage JM, Shabot MM, Sheridan T. Reducing the frequency of errors in medicine using information technology. JAm Med Inform Assoc. 2001;8: Leape LL, Kabcenell AI, Gandhi TK, Carver P, Nolan TW, Berwick DM. Reducing adverse drug events: lessons from a breakthrough series collaborative. Jt Comm J Qual Improv. 2000;26: Weissman JS, Annas CL, Epstein AM, et al. Error reporting and disclosure systems: views from hospital leaders. JAMA. 2005;293: Sorokin R, Riggio JM, Hwang C. Attitudes about patient safety: a survey of physicians-in-training. Am J Med Qual. 2005;20: Banja JD. Disclosing medical error: how much to tell? J Healthc Risk Manag. 2003;23: Novis D. Routine review of surgical pathology cases as a method by which to reduce diagnostic errors in a community hospital. Pathol Case Rev. 2005;10: Arch Pathol Lab Med Vol 130, March 2006 Frozen Section Correlation Raab et al