Image Gently: A Web-Based Practice Quality Improvement Program in CT Safety for Children

Transcription

1 Pediatric Imaging Technical Innovation Goske et al. CT Safety for Children Pediatric Imaging Technical Innovation FOCUS ON: Marilyn J. Goske 1 Rebecca R. Phillips 2 Keith Mandel 3 Daniel McLinden 4 Judy M. Racadio 1 Seth Hall 1 Goske MJ, Phillips RR, Mandel K, McLinden D, Racadio JM, Hall S Keywords: CT, maintenance of certification, patient safety, pediatrics, practice quality improvement, online learning DOI: /AJR Received October 7, 2009; accepted without revision October 23, Supported by a grant from the Society for Pediatric Radiology Research and Education Foundation N. Thorne Griscom Education Award. This project was approved by the American Board of Radiology for credit as a performance quality improvement project. 1 Department of Radiology, Children s Hospital Medical Center, ML 5031, 3333 Burnet Ave., Cincinnati, OH Address correspondence to M. J. Goske (marilyn.goske@cchmc.org). 2 Department of Education and Training, Children s Hospital Medical Center, Cincinnati, OH. 3 Division of Health Policy and Clinical Effectiveness, Children s Hospital Medical Center, Cincinnati, OH. 4 Assistant Vice President, Children s Hospital Medical Center, Cincinnati, OH. AJR 2010; 194: X/10/ American Roentgen Ray Society Image Gently: A Web-Based Practice Quality Improvement Program in CT Safety for Children OBJECTIVE. Radiologists want to improve quality and safety to benefit their pediatric patients and to comply with new requirements of the American Board of Radiology for maintenance of certification. The purpose for this article is to describe the development, construction, and content of a free, Web-based practice quality improvement (PQI) module in CT safety for children. CONCLUSION. We describe an online tutorial accessible on the Image Gently Website that enables radiologists nationwide to perform PQI in CT safety for pediatric patients. T he quality of health care in the United States has been the subject of recent, serious scrutiny by government, medical benefit companies, health care providers, and the public. This scrutiny escalated after a report, To Err Is Human: Building a Safer Health Care System, was published by the Institute of Medicine in The report documented that somewhere between 44,000 and 98,000 patients die in the United States each year as a result of preventable medical errors [1]. The report concluded that the majority of medical errors are caused by faulty systems, processes, and conditions that either lead people to make mistakes or fail to prevent them, as opposed to resulting from individual recklessness [1]. The authors suggested that mistakes can best be prevented by improving the health care system to make it safer; they challenged the medical profession to chart a course of self-improvement. To produce predictable quality improvement results from a complex system such as health care, practice quality improvement (PQI) is critical [2]. PQI approaches problem solving by looking at a system as a series of steps and evaluating for any suboptimal areas of flow, thereby attempting to anticipate problems and improve the process. Radiology is in the center of national efforts to improve health care quality and safety, partly because of the high cost of modern imaging and the many opportunities for medical errors [3]. The American Board of Radiology (ABR), in part to improve patient safety, instituted sweeping changes in the process of accreditation for radiologists in Instead of lifetime certificates, radiologists now maintain their certification over 10-year cycles and are required to perform and document PQI projects. This provides a significant opportunity for the radiology community to take the lead in developing PQI initiatives [4]. In addition to the ABR efforts, the Accreditation Council for Graduate Medical Education (ACGME) lists practice-based learning and improvement as one of the core competencies for graduate medical education [5]. The ACGME requires that residents learn to continuously improve patient care on the basis of constant self-evaluation and lifelong learning, expecting them to be able to systematically analyze practice using quality improvement methods and implement changes with the goal of practice improvement. We describe an example of a PQI initiative, using an online PQI module regarding safety of children undergoing CT. Materials and Methods: Design Process Development of Metrics Pediatric CT safety was chosen as the area of practice for which the PQI module would be designed. This area for improvement was chosen because of the increase in the number of CT examinations in the United States and concern regarding radiation risk to children and the developing fetus [6, 7]. Potential renal toxicity and life threatening events from contrast agent allergy were also included in the module [8, 9]. Communication between AJR:194, May

2 Goske et al. referring physicians and radiologists and between radiologists and families was selected for inclusion in the module due in part to renewed emphasis on medical literacy that has resulted from the Institute of Medicine s report, Health Literacy: A Prescription to End Confusion [10]. After these initial focus areas were identified, one of the authors analyzed the practice of pediatric CT, from the initial order to the communication of results. Each step was examined for any area that had the potential to contribute to errors or diminished patient safety and for which modifications or improvements would be possible. Through this process, 10 metrics were identified that had published evidence documenting their critical role in CT practice or were the subject of American College of Radiology (ACR) standards. The final list of metrics to be evaluated for each patient, with the goal of achieving 100% for each metric, is as follows: 1. Justification for examination provided by referring physician 2. Parent or patient education provided by hospital or practice 3. Communication of critical patient information from parent to radiologist before scanning 4. Documentation of last menstrual period or pregnancy test status before performance of CT 5. Documentation of history of renal disease or renal function 6. Documentation of allergy history before administration of IV contrast agent 7. Use of breast shields or technique modulation in girls and women 8. Use of single phase scans for most examinations 9. Limitation of scan to portion of body necessary to answer the medical question 10. Consistency of hospital or practice with stated pediatric protocols A numeric value (from 0 to 3) was assigned to the response for each metric so the results could be measured and compared, either from practice to practice or over multiple cycles of measurement. An additional learning exercise also was included that allows the radiologist to review the CT dose display on a PACS and evaluate the relationship to patient dose. Development of Education Team and Peer Review An education team was formed, with the following core members: an adult learning specialist who provided expertise in curriculum design and adult learning methods (learning objectives and evaluation tied to desired outcomes, robust interactivity, skill practice in work context, and job aids to support application of learning after the learning intervention), a quality improvement advisor skilled at working with improvement tools, a testing and evaluation specialist, and an information technology (IT) expert [11]. This group met periodically to review the content and made modifications throughout the 15-month development process. An instructional designer, graphic designer, and medical illustrator were also consulted early in the process. As the module took shape and was ultimately placed within a preexisting Website using Web pages with a specific template, these experts did not remain part of the core development team. Once the content was in a final draft stage, it was sent as a Microsoft Word document to five pediatric radiologists and two medical physicists who agreed to act as outside peer reviewers. Significant edits were incorporated, including replacement of metric 10 with a new metric and the addition of a learning exercise. At this point, a formal application was made to the ABR for qualification as a maintenance of certification (MOC) project. Approval was granted, pending final online review. Once the Website was posted to a development site, a focus group of 10 pediatric radiologists at the authors institution reevaluated the module before posting on the Internet. After subsequent modifications were made, the ABR reviewed the final product and approved the PQI project as qualifying for part IV of the MOC process. Development of Technical Infrastructure The content was written and saved by the primary author as Word documents or PDF files. Fig. 1 Home page of Website. This practice quality improvement project was created on preexisting Website and used cascading Web pages and preexisting design template. Although amount of space on each Web page was limited, cost was minimal because no new Website was developed for project. The Word content was converted to HTML and placed in the professional medical society s Web content manager. The content manager used cascading style sheets for a consistent look and feel to the Website. This format was chosen for three reasons: First, the authors wanted the content to reside on the professional medical society s hosting site (Society for Pediatric Radiology). Second, they wanted to avoid the financial burden of building a new site. Third, the content manager provided rapid deployment of content to the Website. The built-in survey tool was housed on a secure commercial Website (Webmasters.com, NetTuner.com) and linked to the primary Website. The IT expert used PHP (Hypertext Preprocessor Language) and MySQL (both free database software) to develop the survey tools for each metric, demographic data, and user feedback to authors. The module may be accessed directly at no charge on the Image Gently Website ( org/), through the Society for Pediatric Radiology Website ( or through the ABR Website ( Results: Module Description The online module begins with a brief overview of the science of PQI, followed by a justification for PQI as a part of MOC part IV (Fig. 1). The radiologists are guided through the collection of representative data from a small sample of their practices. The radiologists are asked to randomly select 25 pediatric abdomi AJR:194, May 2010

3 CT Safety for Children nopelvic CT scans with specific Current Procedural Terminology codes that were performed at their institutions and that they personally interpreted: five in children less than 3 years, 10 in children 4 11 years old, and 10 in children more than 12 years old. A metric worksheet is provided online for them to print out and use to record the data for each metric (Fig. 2). For each patient, there is a place to record the score from each metric. For example, for metric 2 (parent or patient education provided by your institution or practice), a score of 0 is given if no information was provided about the CT examination to parents, a score of 1 if the parents had an opportunity to view departmental information about CT but it was not documented, or 2 if parents received departmental information about CT and the documentation of receipt of CT informational materials by parents is archived and easily accessible by the interpreting radiologist. Once the radiologists have completed the data collection on the worksheet, they begin working through the individual metrics in the module (Fig. 3). For each metric, there is a survey tool that uses aggregate data collected from the patient samples to allow the radiologists to develop insights into how their practices are performing (Fig. 4). Again, using metric 2 as an example, radiologists would enter the number of patients for which the metric 2 score was 0, the number for which it was 1, and the number for which it was 2. The module then converts these patient numbers to percentages. The goal for each metric is 100% compliance at the optimal rating on the scale. If for metric 2, for example, none of the radiologist s patients received any information about the CT scan before it was performed; there is evidence for significant room for improvement for this metric within the radiologist s practice. Alternatively, if all 25 parents were documented to have received educational material that informed them about the nature and risks of CT, the radiologist has reached 100% compliance for this metric (the optimal value), and there is little room for improvement. As various radiologists use the PQI module, the data from each user are compiled and a cumulative score for all the radiologists or practices that have been through the module is created. The radiologist s reported scores and percentages are then displayed on the screen along with the scores and percentages for all the other radiologists or practices. This provides direct feedback to users so that they can compare their practice to others who have Fig. 2 Metric worksheet. This worksheet organizes radiologist s data before entry into survey page for each metric. It also serves as partial documentation of performance of this performance quality improvement project. This page may be saved as electronic or hard copy. It is not saved in central database or documented for maintenance of certification purposes by program. taken the survey. Although this survey tool is not scientific research, it does allow the radiologists to get a sense of where their practices stand relative to others who have entered data. Because the information provides no identifiable patient information and only aggregate data are recorded, there are no patient privacy concerns involved with this project. For each metric, there is a section of learning tools provided. This section supplies links or PDF files of evidence-based references for each metric; it also provides links to ACR guidelines to improve the user s understanding of safety issues related to the performance of CT in children. Permission was obtained from societies, publishers, and Fig. 3 Evidencedbased metrics. There is scientific justification for each of 10 metrics selected. Links to scientific articles are provided. Practice performance tools such as CT parent pamphlet are provided via links to downloadable PDF files. authors and is listed on the home page under a tab under Permission From Publishers. Practice interventions are provided for each metric. These interventions can be implemented to improve a particular area of CT safety in the radiologist s practice. For example, there is a job aid, a one-page Parent/Patient Questionnaire that fulfills many of the requirements for metrics one through six; when completed, this questionnaire can be scanned into the radiology PACS or electronic medical record (Fig. 5). The questionnaire documents patient name, age, and medical history, including history of chronic illness; prior surgery; prior CT; allergies to contrast material; kidney or liver disease; blood disorders; recent CT or MRI con- AJR:194, May

4 Goske et al. trast administration; and use of total parenteral nutrition, lipids, or metformin. For girls who are 12 years and older, menstrual history and chance of pregnancy are also documented. The questionnaire asks the parent or patient to explain in his or her own words why the CT is being performed, to acknowledge receipt of information about CT and what to expect, and to sign the form. The radiologists may modify this form to suit their practices. Once scanned into the PACS or saved as a hard copy, the radiologist is able to access this information before making a decision as to whether to proceed with CT. A second part of this form allows the radiologist to structure the individual patient scans and provides an area to modify the CT protocol, for example designating a lower resolution scan in a child with a ventriculoperitoneal shunt to assess for shunt malfunction [12]. When the radiologists have completed each of the 10 metrics and the learning exercise, they are asked to examine their results and then complete a Practice Improvement Plan worksheet (Fig. 6). This worksheet has space for each radiologist to construct a plan to improve his or her practice in any or all of the metric areas. Although suggestions are made within the module for practice improvements for each metric, the suggestions can be customized by each radiologist for an individual practice. Practice improvement tools provided within the module may be selected and documented by the radiologist in this section of the PQI process. Demographics and Copyright The module is copyrighted by the education team s academic institution and accompanied by a standard disclaimer as to its use as an educational tool. Users agree to provide nonidentifying demographic data that may be used for scientific research. By entering data in the survey, users agree to participate. Fig. 5 CT parent information questionnaire and protocol form. This form is meant to be downloaded and used as practice improvement tool for radiologist s practice. It includes patient medical history, parent communication, and other critical patient information for radiologist and may be scanned into electronic medical record. Fig. 4 Survey tool. For each metric, there is survey tool that allows radiologists to input their aggregate data for metric and compare how their practice is doing compared with others who have taken survey. This is not scientific research tool, but simply interactive component of module that provides radiologist with snapshot of other practices. Feedback to Authors At the end of the module, users are asked to provide feedback to the authors as to how well the information was presented and the ease of use of the module. At the conclusion of the module, there is a brief User Feedback form with six questions developed by the testing and evaluation specialist and the education group. The users are asked to rank the module as to whether it was useful, instructions were easy to follow, the user learned more about evidenced-based performance of CT practice, and they believe practice improvement will result from using the module. Finally, those who use the module are asked to comment on whether this project was a worthwhile investment in their professional development. Free text areas are provided for the users to report what they did or did not learn from this program. Discussion PQI is a tool medical professionals can use to improve the quality and safety of their practice. Any PQI program should incorporate the selection and analysis of an area of 1180 AJR:194, May 2010

5 CT Safety for Children practice to be changed or enhanced, the division of the change into a manageable series of steps, and analysis of each step [3]. Data collection is a critical component of any meaningful PQI project; the project should allow accurate data collection so that variation both within a practice or from practice to practice can be identified and analyzed and a particular practice can be compared with national benchmarks. PQI is not just one process or cycle; it is by definition continuous and incorporates a repeated cycle of improvements, which Applegate [2] has reported as the PDSA cycle. The cycle involves four steps: plan, do, study, act, and then repeat. For PQI, the plan step would be identifying a target area for improvement; the Do step incorporates baseline data collection; study is the analysis of the data and comparison with evidence-based ideal practice; and act involves practice modification to make improvements [5]. This provides a process for sustainable improvement as the cycle is repeated and metrics are reevaluated. On a practical level, we used some guidelines when designing the PQI module. First, the PQI project should be relevant to the practice. Second, the project should be able to be performed with the resources available to each practitioner. With the increased use of computerized hospital records, finding the pertinent information for the 10 metrics should not be difficult for any practicing radiologist. Third, the results should be measurable and Fig. 6 Practice improvement plan worksheet. This form is provided on Website for radiologists to fill in to organize their practice improvement plan. It also documents this portion of project. This form is not saved in online module and must be kept by radiologist either as electronic or hard copy. should be able to be measured repeatedly. By assigning a number score to each, the metrics could easily be measured and compared. Finally, the PQI initiative should effect quality improvement for the patients. Any area that is identified and improved through this module should increase patient safety. The module was also designed to work as part of a PQI cycle. After completing the program and making practice modifications, the radiologists would reevaluate the metrics in follow-up measurements to evaluate any improvement. During the subsequent cycle, they may adjust the process slightly, such as using the same or similar metrics to evaluate patient safety during head or chest CT. This module does not analyze all the steps that relate to quality and safety during the performance of body CT in a child. For example, report turn-around time was not included as a metric. In future cycles of improvement, the radiologist might choose to customize this PQI project and add that quality indicator as a metric to be measured repeatedly over time. Another pitfall of this project is that it does not document performance of the module for the radiologist. The radiologists are responsible for documenting their performance of this PQI module; either by saving the printed metric worksheet and practice performance plan or creating a computer file of the materials. While this program is able to save responses, it is beyond the scope of this module to support a central database. This PQI module is not able to store individual data and link it to users over multiple cycles of improvement. The radiologists are encouraged to access the ABR s Website and enlist in the MOC process and document performance of this PQI project. Conclusion PQI is a tool medical professionals can use to improve the systems involved in patient care. By completing PQI, radiologists eliminate potential sources of error and thereby improve patient safety. This PQI module allows the practitioner to capture how an individual practice performs body CT in children and to compare that practice to safe practice as documented in the literature and in ACR guidelines. By identifying areas of concern, changes can be implemented to improve practice. Following the process through multiple cycles allows continuous, sustainable quality improvement. Acknowledgments We thank the following individuals for their review of the manuscript: Alan Brody, Priscilla Butler, Donald P. Frush, Sue Kaste, Thomas Slovis, Keith Strauss, and Janet Strife. Focus group participants included Christopher Anton, Alan Brody, Maria Calvo, Kathleen Emery, Robert Fleck, Kathy Helton, David Larson, Tal Laor, Michael Nasser, Daniel Podberesky, and Janet Strife. Administrative support was provided by Coreen Bell. References 1. Corrigan JM, Kohn LT, Donaldson MS, eds. To err is human: building a safer health system. Washington, DC: National Academies Press, Applegate KE. Continuous quality improvement for radiologists: critical thinking skills symposium. Acad Radiol 2004; 11: Dunnick NR, Applegate KE, Arenson RL. Quality: a radiology imperative report of the 2006 intersociety conference. J Am Coll Radiol 2007; 4: Strife JL, Gary LE, Becker J, et al. The American Board of Radiology perspective on maintenance of certification: part IV practice quality improvement in diagnostic radiology. AJR 2007; 188: Accreditation Council for Graduate Medical Education Website. Educational program requirements: ACGME competences practice-based learning and improvement. acwebsite/navpages/commonpr_documents/ AJR:194, May

6 Goske et al. IVA5c_EducationalProgram_ACGMECompetencies_PBLI_Explanation.pdf. Published January 15, Accessed August 29, Amis ES, Butler PF, Applegate KE, et al. American College of Radiology white paper on radiation dose in medicine. J Am Coll Radiol 2007; 4: Berlin L. Radiation exposure and the pregnant patient. AJR 1996; 167: Turner NM. A practical approach to paediatric FOR YOUR INFORMATION emergencies in the radiology department. Pediatr Radiol 2009; 39: Heinrich MC, Haberle L, Muller V, et al. Nephrotoxicity of iso-osmolar iodixanol compared with nonionic low osmolar contrast media: meta-analysis of randomized controlled trials. Radiology 2009; 250: Institute of Medicine. Health literacy: a prescription to end confusion. Washington, DC: National Academies Press, Reid JR, Goske MJ, Hewson MG, Obuchowski N. Creating an international comprehensive webbased curriculum in pediatric radiology. AJR 2004; 182: Singh S, Kalra MK, Moore MA, et al. Dose reduction and compliance with pediatric CT protocols adapted to patient size, clinical indication, and number of prior studies. Radiology 2009; 252: For more information on this subject and for practical steps to create a patient radiation safety program, see "For One Radiologist, CT Dose Safety is a Personal Matter," by Steven B. Birnbaum, in ARRS InPractice, Winter 2009, vol. 3, issue 2, page AJR:194, May 2010