Performance-Based Assessment of Radiology Practitioners: Promoting Improvement in Accordance with the 2007 Joint Commission Standards Lane F. Donnelly, MD a,b New guidelines for medical credentialing and privileging were published by the Joint Commission in 2007. These guidelines require practitioner-specific data collection in the areas of patient care, medical and clinical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and system-based practice. The authors describe their current process for privileging on the basis of practitioner performance with the aim that the description may serve as a template for other departments of radiology working on similar processes. The parameters chosen in each category were picked on the basis that they would contribute to quality improvement while creating minimal additional data collection beyond what was already occurring in the department. Key Words: Credentialing, privileging, competence, process improvement, quality improvement J Am Coll Radiol 2007;4:699-703. Copyright 2007 American College of Radiology a Department of Radiology, Cincinnati Children s Hospital Medical Center, Cincinnati, Ohio. b University of Cincinnati, College of Medicine, Cincinnati, Ohio. Corresponding author and reprints: Lane F. Donnelly, MD, Cincinnati Children s Hospital Medical Center, Department of Radiology, 3333 Burnet Avenue, MLC 5031, Cincinnati, OH 45229-3039; e-mail: lane.donnelly@ cchmc.org. 2007 American College of Radiology 0091-2182/07/$32.00 DOI 10.1016/j.jacr.2007.05.007 INTRODUCTION In medical care facilities, the medical staff governing bodies have the ultimate authority and responsibility for the delivery and oversight of care rendered by licensed independent practitioners who are privileged through the medical staff process [1]. The Joint Commission [1] defines an independent practitioner as any individual permitted by law and by the organization to provide care, treatment, and services without direction or supervision. Credentialing and privileging in medical institutions are the formal recognition of the professional and technical competence of these individual practitioners [2,3]. It is the responsibility of the medical staff governance to provide the ongoing evaluation of the competency of practitioners who are privileged [1]. In 2004, the Joint Commission [2] published new guidelines for the granting, renewal, and revision of privileges for health care providers [2]. The major change with the 2004 guidelines was that practitionerspecific data must be accumulated and used in the process of reappointment. Practitioner-specific data were to be gathered in the areas of clinical judgment and technical skills, professionalism, communication, and continued education and clinical self-improvement [2]. New guidelines for medical credentialing and privileging were published by the Joint Commission [1] in 2007. There are several new concepts embedded into the new standards. First and foremost, the areas in which practitioner-specific data need to be collected have been aligned with the Accreditation Council for Graduate Medical Education and the American Board of Medical Subspecialties joint initiative to create 6 areas of general competencies [1]. These areas for data collection on a practitioner-specific level are as follows: patient care, medical and clinical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and system-based practice. Another change in concept is that there is to be ongoing professional evaluation of independent practitioners. Traditionally, the evaluation of medical staff members has been a cyclical process in which privileges are initially 699
700 Journal of the American College of Radiology/ Vol. 4 No. 10 October 2007 Table 1. Parameters used in category of professional performance Fellow evaluations of faculty performance in category role model for professionalism (acceptable score of 3.0 or 4.0) Fellow evaluations of faculty performance in category role model for interacting with patients and families (acceptable score of 3.0 or 4.0) Zero violations of site marking and hold point procedure for invasive procedures Unapproved abbreviations: mean of 0.1% for use of CC discovered via radiology report monitoring system Completion of Joint Commission patient safety test annually Note: CC cubic centimeter. granted and then reevaluated every 2 years. The new Joint Commission [1] standards state that evaluation of practitioners performance should be continuous and ongoing. As has been previously noted, the qualities that contribute to a radiologist s providing excellent care are difficult to define and even more difficult to measure [2]. Coming up with a list of ideal performance measures that are evidence based, agreed-on standards, reproducible, attributable to individual radiologists, and occurring in adequate numbers so that statistical evaluation can be meaningful is extremely difficult [2,4,5]. In fact, the new Joint Commission [1] standards state that determining the competency of practitioners to provide high quality, safe patient care is one of the most important and difficult decisions an organization must make. Those parameters that are easy to measure often do not reflect the true nature of what it means to be a competent radiologist. However, the standards of the Joint Commission indicate that we must try our best to create a list of parameters in which we collect provider-specific information. Efforts should be made to meet these external requirements in a way that fosters quality improvement in patient care. Working with the medical staff office at our institution, we have created a privileging plan based on provider performance to promote improvement and meet the 2007 Joint Commission standards. Data are collected in 6 areas, on the basis of our translation from the Joint Commission standards: professional performance, interpersonal and communication skills, medical and clinical knowledge, clinical judgment, clinical and technical skills, and systems-based practice. We describe this process with the aim that the description may serve as a template for other departments of radiology working on similar processes. The parameters chosen in each category were picked on the basis that they would contribute to quality improvement while creating minimal additional data collection beyond what was already occurring in the department. The parameters are by no means ideal and will be a continuous work in progress. Sometimes it was somewhat arbitrary as to which category a specific parameter was placed, because it could fit in multiple categories. PROFESSIONAL PERFORMANCE The new Joint Commission [1] guidelines state that with regard to professionalism, individual practitioners are expected to demonstrate behaviors that reflect a commitment to continuous professional development, ethical practice, and understanding and sensitivity to diversity and a responsible attitude to their patients, their profession, and society. The list of parameters we are currently using for professional performance are shown in Table 1. In our department, there are between 7 and 10 radiology fellows per year. Biannually, the fellows evaluate faculty members performance with regard to multiple parameters covering many areas. Parameters are graded on a scale of 1 to 4, with 4 being the best. Because ours is a children s hospital, the pediatric radiology fellows are with us for an entire year, while the radiology residents rotate on and off the service. For these reasons, we chose to use fellow evaluations of faculty members over resident evaluations of faculty members. This is not true peer evaluation, because a trainee is evaluating a faculty member. However, the evaluator does not have to be a peer. Two of the parameters used, Role model for professionalism and Role model for interacting with patients and families, are used as parameters for faculty professional performance. INTERPERSONAL AND COMMUNICATION SKILLS The Joint Commission [1] standards state that practitioners must demonstrate interpersonal and communication skills that enable them to establish and maintain
Donnelly/Performance-Based Assessment 701 Table 2. Parameters used in category of interpersonal and communication skills Physician report sign-off in 24 hours (acceptable compliance rate 95%) Number of parent, patient, referring physician, and coworker complaints to radiologist in chief concerning behavior, communication, and professionalism (acceptable 1 per year) Zero violations of policy for documentation and communication of changes between preliminary and final reports Fellow evaluations of faculty performance in category effective communication skills (acceptable score of 3.0 or 4.0) Completion of one-time CARES customer service standards test Note: CARES courteous, attentive, respectful, enthusiastic team members, and safe. professional relationships with patients, families, and other members of health care teams. The parameters used in our privileging plan in the category of interpersonal and communication skills are listed in Table 2. One of the most common ways that radiologists communicate with referring physicians is via written reports. Parameters measured in the category of communication can include physician report sign-off times. We also have instituted a complaint tracking system [2]. Complaints from parents, physicians, or hospital coworkers regarding radiology faculty members behavior, professionalism, or communication are investigated. If a complaint is legitimate, the event is logged into the tracking system. We use measurements related to such complaints as one of the parameters to follow for interpersonal and communication skills. The department policy on the documentation and communication of changes between preliminary and final reports is such that preliminary reports remain part of the final radiology reports, and any substantial changes are listed at the bottom of the preliminary reports in a defined fashion. It is expected that faculty members will always follow this policy when adding significant changes to preliminary reports. Also, again used is fellows evaluations of faculty members performance for effective communication skills. In addition, completion of a onetime CARES customer service standards test is required for all faculty along with all other employees. CARES is an acronym used at our institution demonstrating the characteristics that employees should embody: courteous, attentive, respectful, enthusiastic team members, and safe. MEDICAL AND CLINICAL KNOWLEDGE The Joint Commission [1] guidelines state that practitioners are expected to demonstrate knowledge of established and evolving biomedical, clinical and social sciences, and the application of their knowledge to patient care and the education of others. The parameters used in the category of medical and clinical knowledge are listed in Table 3. At most institutions, the major way radiology departments measure performance in the area of medical and clinical knowledge is through a process of peer review. There are multiple potential systems for peer review, including randomly assigned double interpretations, comparison between dictations and surgical or autopsy reports, and the use of commercially or professional available products [2,6]. We have a well-defined system for peer review for the evaluation of the frequency of faculty members errors [2,6]. Errors categorized as technical, perception, interpretation, or communication in the peer review database are used as parameters related to performance in medical and clinical knowledge. Other parameters used include continued medical education requirements for Ohio license, the placement of instructional cases into the digital teaching file for training and educational purposes, and the presentation or publication of applications of pediatric radiology. Table 3. Parameters used in category of medical and clinical knowledge Meets CME requirements to maintain Ohio license Assign and de-identify imaging studies for instructive cases to a digital teaching file for use in trainee education At least 1 presentation or publication of applications of pediatric radiology perception, interpretation, or communication (within 2 SD from group mean) Note: CME continuing medical education.
702 Journal of the American College of Radiology/ Vol. 4 No. 10 October 2007 Table 4. Parameters used in category of clinical judgment perception, interpretation, or communication (within 2 SD from group mean) Fellow evaluations of faculty performance in category demonstrates balance between supervision & delegation of responsibility defines lines of responsibility well (Acceptable score of 3.0 or 4.0) Fellow evaluations of faculty performance in category role model for interacting with referring physicians (Acceptable score of 3.0 or 4.0) CLINICAL JUDGMENT The Joint Commission [1] standards for credentialing in the category of patient care state that practitioners are expected to provide patient care that is compassionate, appropriate, and effective for the promotion of health, prevention of illness, treatment of disease, and care at the end of life. We have divided this section of our provider performance based privileging into the categories of clinical judgment and clinical and technical skills. Parameters used in clinical judgment are listed in Table 4. Parameters for clinical judgment include data again obtained from the peer review database. Fellows evaluations of faculty members performance in the categories Demonstrates balance between supervision and delegation of responsibility Defines lines of responsibility well and Role model for interacting with referring physicians are used for the category of clinical judgment. For the future, we are considering a survey of referring physicians and perhaps radiology departmental peers for evaluating the clinical judgment of faculty members, because these peers may be more qualified to render such opinions. CLINICAL AND TECHNICAL SKILLS Under the Joint Commission category of patient care, we also have a section for clinical and technical skills. Parameters used for evaluating clinical and technical skills are listed in Table 5. Parameters used include enrollment and compliance with maintenance certification programs, either through the American Board of Radiology or the American Board of Nuclear Medicine or, for foreign medical graduates, that faculty members are enrolled in and compliant with programs leading to American Board of Radiology certification and maintenance of certification. From the fellow evaluations, faculty members performance is evaluated for Role model for taking care of patients/clinical care. Pediatric advanced life support certification or enrollment in and completion of radiology advanced life support certification through our own institution are required, as well as certification in cardiopulmonary resuscitation. In addition, data are used from the peer review system database in clinical and technical skills as well. We have separate specific parameters to follow for the interventional radiologists that are not part of the process for general radiologists, described here. SYSTEM-BASED PRACTICE The Joint Commission s [1] standards for system-based practice state that practitioners are expected to demonstrate both an understanding of the context and systems in which health care is provided, and the ability to apply this knowledge to improve and optimize health care. This category is the most likely to be new to most departments and is in line with the efforts of the American Table 5. Parameters used in category of clinical and technical skills perception, interpretation, or communication (notification to division chief when greater than 2 SD from group mean) Enrolled and compliant with maintenance of certification (MOC) program with either American Board of Radiology (ABR) or American Board of Nuclear Medicine or for foreign medical graduates: enrolled and compliant with program leading to ABR certification and MOC program Fellow evaluations of faculty performance in category role model for taking care of patients/clinical care (Acceptable score of 3.0 or 4.0) Pediatric advanced life support certification or radiology advanced life support certification Cardiopulmonary resuscitation certification (by end of fiscal year 2008 and then each year forward) Note: CPR cardiopulmonary resuscitation; PALS pediatric advanced life support.
Donnelly/Performance-Based Assessment 703 Table 6. Parameters used in category of systems-based practice Completed error prevention training or any required updates Completed assignment for standardization of indications, protocols, and reporting for a specific clinical scenario and imaging modality as assigned by department (fiscal year 2007: focus on CT protocols) Follows CCHMC guidelines for safe handoffs when involved in handoffs of patients undergoing interventional procedures or undergoing sedation Participation in independent verification of intravenous contrast orders At least one of the following ΠFunctions as a safety coach ΠParticipates in quality improvement activity at institutional or national level ΠParticipation in MOC required quality improvement project Note: CCHMC Cincinnati Children s Hospital Medical Center; CT computed tomographic; MOC maintenance of certification. Board of Radiology s maintenance of certification process and other national initiatives. Parameters used for the category of system-based practice at our institution are listed in Table 6. At our institution, error prevention training to teach behavioral expectations that minimize the likelihood of human error has been rolled out as part of an institution-wide patient safety campaign. One of the parameters in system-based practice is that our faculty members have completed this training. Second, each faculty member has been assigned a role in the standardization of indications, protocols, and reporting for a specific clinical scenario and imaging modality. The roles of some of the faculty members are participation and group discussion related to improvements in these areas. Participation in these process improvement efforts is one of the parameters measured. Many of these tasks are assigned by the division within the department to the faculty members who rotate through that specific division commonly. Updates of progress will be given at annual faculty evaluations. Faculty members are expected to follow institutional guidelines for the safe handoff of patients. Participation in the independent verification of intravenous contrast orders as mandated by the Joint Commission is also a monitored parameter. In addition, each of the faculty members must participate in at least one of the following: function as a safety coach, participate in quality improvement activity at the institutional or national level, or participate in a quality improvement project required for maintenance of certification. In this sense, the maintenance of certification process for the American Board of Radiology parallels the Joint Commission s process, and involvement in the same quality improvement project by an individual or group can meet the criteria for both processes. Safety coaches are part of our new institutional safety program and actively promote behavioral expectations and techniques to reduce errors actively in the workplace. SUMMARY Departments of radiology should look on the new Joint Commission [1] guidelines for credentialing and privileging as a means to foster quality improvement within their own departments. With changes in technology and hospital information systems that make measurable parameters having meaning easier to collect, the parameters used in such a provider performance system will constantly be evolving. We describe the system we currently have in place, which attempts to capitalize on many of the positive events that are already occurring in our department and minimize the amount of need for additional data collection. We hope that the description of this program may help others who are working on their own programs for provider performance-based privileging. REFERENCES 1. Joint Commission. Comprehensive accreditation manual for hospitals: The official handbook. Oakbrook Terrace, Ill: Joint Commission; 2007. 2. Donnelly LF, Strife JL. Performance-based assessment of radiology faculty: a practical plan to promote improvement and meet JCAHO standards. AJR Am J Roentgenol 2005;184:1398-401. 3. O Connor ME. Medical staff appointment and delineation of pediatric privileges in hospitals. Pediatrics 2002;110:414-8. 4. Ondategui-Parra S, Bhagwat JG, Gill IE, et al. Essential practice performance measurement. J Am Coll Radiol 2004;1:559-66. 5. Landon BE, Normand SLT, Blumenthal D, Daley J. Physician clinical performance assessment: prospects and barriers. JAMA 2003;290:1183-9. 6. Halsted M. Radiology peer review as an opportunity to reduce errors and improve patient care. J Am Coll Radiol 2004;1:984-7.