Automated Critical Test Result Notification System: Architecture, Design, and Assessment of Provider Satisfaction

Transcription

1 Health Care Policy and Quality Original Research Lacson et al. Automated Critical Test Result Notification Health Care Policy and Quality Original Research FOCUS ON: Ronilda Lacson 1 Stacy D. O Connor 1 Katherine P. Andriole 1 Luciano M. Prevedello 2 Ramin Khorasani 1 Lacson R, O Connor SD, Andriole KP, Prevedello LM, Khorasani R Keywords: alert notification, automated system, critical test result, provider communication, satisfaction survey DOI: /AJR Received April 25, 2014; accepted after revision June 13, Supported by grant R18HS from the Agency for Healthcare Research and Quality (AHRQ) Partnerships in Implementing Patient Safety II initiative; National Library of Medicine (NLM) grant T15LM007092; and two grants from the Controlled Risk Insurance Company Risk Management Foundation (CRICO-RMF). 1 Center for Evidence-Based Imaging and Department of Radiology, Brigham and Women s Hospital, Harvard Medical School, 75 Francis St, Boston, MA Address correspondence to R. Khorasani (rkhorasani@partners.org). 2 Present address: The Ohio State University Wexner Medical Center, Columbus, OH. WEB This is a web exclusive article. AJR 2014; 203:W491 W X/14/2035 W491 American Roentgen Ray Society Automated Critical Test Result Notification System: Architecture, Design, and Assessment of Provider Satisfaction OBJECTIVE. Communicating critical results of diagnostic imaging procedures is a national patient safety goal. The purposes of this study were to describe the system architecture and design of Alert Notification of Critical Results (ANCR), an automated system designed to facilitate communication of critical imaging results between care providers; to report providers satisfaction with ANCR; and to compare radiologists and ordering providers attitudes toward ANCR. MATERIALS AND METHODS. The design decisions made for each step in the alert communication process, which includes user authentication, alert creation, alert communication, alert acknowledgment and management, alert reminder and escalation, and alert documentation, are described. To assess attitudes toward ANCR, internally developed and validated surveys were administered to all radiologists (n = 320) and ordering providers (n = 4323) who sent or received alerts 3 years after ANCR implementation. RESULTS. The survey response rates were 50.4% for radiologists and 36.1% for ordering providers. Ordering providers were generally dissatisfied with the training received for use of ANCR and with access to technical support. Radiologists were more satisfied with documenting critical result communication (61.1% vs 43.2%; p = ) and tracking critical results (51.6% vs 35.1%; p = ) than were ordering providers. Both groups agreed use of ANCR reduces medical errors and improves the quality of patient care. CONCLUSION. Use of ANCR enables automated communication of critical test results. The survey results confirm overall provider satisfaction with ANCR but highlight the need for improved training strategies for large numbers of geographically dispersed ordering providers. Future enhancements beyond acknowledging receipt of critical results are needed to help ensure timely and appropriate follow-up of critical results to improve quality and patient safety. T he Joint Commission has made improved communication of critical results of diagnostic procedures among caregivers a National Patient Safety Goal [1]. To meet this goal, our institution established an enterprise-wide communication of critical test results policy [2] with guidelines and procedures for notifying referring providers when critical imaging results are identified. Initial policy implementation relied on mostly analog processes, including labor-intensive alert creation for a substantial volume of critical imaging results. Therefore, we developed an application, Alert Notification of Critical Results (ANCR), to facilitate critical imaging test result communication. Because the ANCR database contains details of closedloop communication for critical results, it serves as a component of the electronic health record (EHR) and a source of data for relevant regulatory and safety audits. Therefore, as long as use of ANCR is documented in the relevant radiology report, the dependence on documenting details of the closedloop communication in the radiology report with its associated delays can be eliminated. We describe the ANCR system components and the design decisions made to enhance workflow efficiency. We also report on provider satisfaction with ANCR and compare radiologists and ordering providers attitudes toward ANCR. Materials and Methods The institutional review board granted approval for administering an electronic survey to providers at the institution to assess ANCR implementation for this HIPAA-compliant study. ANCR was designed and implemented at a 753-bed urban adult AJR:203, November 2014 W491

2 Lacson et al. tertiary referral academic medical center in January The institution has a fully integrated EHR, and all physicians and care providers have access to hospital and clinic notes and laboratory, radiology, pathology, and microbiology result retrieval. All diagnostic imaging orders are placed via a computerized physician order entry system. System Purpose and Components ANCR was designed to provide urgent and semiurgent alert notification to referring providers when the radiologist deems imaging examination results critical or unexpected at the time of interpretation [3]. The ANCR functional and technical requirements are publicly available [3]. ANCR is a web-based application installed on an HP Pro- Liant DL380 G5 server (Hewlett-Packard) with two 3.00-GHz physical central processing units (eight cores total) and 8 GB random access memory running Microsoft Windows Server 2003 R2 Standard 64 edition as the operating system with Microsoft Internet Information Services enabled to manage the website. Storage requirements for 6000 users are 20 MB for the application, 1.5 GB for the database, and 35 GB for audit logs. The ANCR SQL Server 2005 database (Microsoft) stores user, patient, examination, critical result, notification, and acknowledgment information in related tables. A directory of current and previous users with their contact information, titles, departments, and affiliations is authenticated through the Active Directory and updated through the Paging Directory web services of SQL Server 2005 database each time a radiologist logs on to ANCR to generate an alert or select a provider as an alert recipient. ANCR takes advantage of common application interfaces to interact with multiple clinical applications (Fig. 1), including the PACS, the hospital paging system, and the EHR. Integration points with clinical applications are described as they are invoked in each step of the critical result alert communication process. The ANCR alert communication process includes user authentication, alert creation, alert communication, alert acknowledgment and management, alert reminder and escalation, and alert documentation. The following descriptions of each step highlight features and design decisions intended to improve efficiency and safety. Fig. 1 Chart shows Alert Notification of Critical Results (ANCR) integration with clinical applications. SMTP = simple mail transfer protocol, EPL = enterprise patient list, IIS = Microsoft Internet Information Services, SQL = Microsoft Structured Query Language, EHR = electronic health record. User Authentication When users access ANCR from outside of the institutional intranet, secure access is accomplished through the https protocol and requires authentication through Active Directory web services. Within the intranet, the user accesses ANCR while viewing images from an examination on an integrated PACS workstation (the radiologist clinical workstation). The integration allows Active Directory web services authentication obtained during login to the workstation to be passed to multiple clinical applications through single sign-on access. Single sign-on capability provides the radiologist access to the paging system (and physician directory services), the EHR, and ANCR without having to perform multiple login authentications. Alert Creation After identifying a critical result while reviewing an imaging study, the radiologist activates ANCR and chooses an appropriate alert level from three progressively more urgent levels of yellow, orange, and red (e.g., red alerts are immediately life threatening, requiring communication within 60 minutes). ANCR maintains user-context sensitivity, and integration with the PACS adds unidirectional automatic sharing of patient and examination context from the PACS, removing the need for manual data entry, improving efficiency, and reducing potential for human error. To further automate the process, the event of a radiologist opening an examination in the PACS starts the Windows Middle Management feature, which launches ANCR in a separate browser window. This keeps patient and examination information in ANCR and the PACS in sync and allows immediate alert creation when a critical finding is identified. To identify the provider who can quickly act on a critical alert, referred to as the responding clinician, our institution maintains an enterprise patient list system for inpatients that contains the current care provider and is updated with every coverage change. The enterprise patient list is integrated into ANCR through web services so that the responding clinician can be selected from a pull-down list with a single click. The pulldown list contains other members of the patient care team, including the attending clinician and the admitting clinician, and specifies these roles. For outpatients, the ordering provider is identified from the electronic radiology requisition, generated via computerized order entry. A type-ahead feature in ANCR displays names of providers in the ANCR user directory with increasing specificity so that the radiologist can confidently select the ordering provider from a pull-down menu. Alert Communication ANCR requires alert receipt to be acknowledged and logged. Once created, alerts go into pending status until acknowledged by the receiver or to immediate closure if the radiologist documents face-to-face or telephone (interruptive) alert communication. More urgent red and orange alerts require interruptive communication in accordance with institutional policy. ANCR sends an alert notification via web services to the paging system (for red or orange alerts) or the simple mail transfer protocol (SMTP) server (for yellow alerts via ). Pages contain patient s name, radiologist s name and telephone number, and alert level. User accounts are verified through an institutional database automating W492 AJR:203, November 2014

3 Automated Critical Test Result Notification direct paging of alert receivers through the paging system. notification allows asynchronous noninterruptive communication of alerts. In addition, ordering of additional imaging examinations based on the critical result can be performed from the ANCR notification page (Fig. 2). Each alert requires that a sender and receiver both be licensed independent practitioners (attending physicians, physician assistants, nurse practitioners). Our policy dictates that alerts generated by radiology trainees require the selection of an attending radiologist from an authenticated list of attending physicians at the time of creation. This policy can be enforced by, and is configurable within, ANCR. Alert Acknowledgment and Management For high urgency (red or orange) critical alerts, which require interruptive communication through the paging system, the radiologist closes the communication loop by acknowledging the alert in ANCR. For less urgent yellow alerts, the responding clinician either opens the alert from the ANCR website or uses the link in an ANCR to securely access and review alert details and to acknowledge receipt. For alerts sent to an incorrect responding clinician, ANCR has a link for the clinician to send the alert back to the radiologist with a comment that also keeps the alert in pending status. ANCR tracks alerts that are generated as pending until they are acknowledged and as overdue if the communication loop is not closed within the window dictated by policy. All unacknowledged alerts are visible in the ANCR work list (Fig. 3) presented at ANCR login to referring clinicians and to radiologists. Communication of critical test results can be monitored across the entire radiology practice by use of ANCR performance metrics and tracking logs. Alert Reminder and Escalation An important ANCR component is an alert reminder system, which sends escalation pages for overdue red and orange level alerts. ANCR also sends reminders when alerts have not been acknowledged within a predetermined time frame. All overdue alerts are combined into a single reminder to limit alert fatigue for providers who have multiple overdue alerts [4, 5]. s continue daily until alerts are acknowledged and the communication loop is closed. Alert Documentation When an alert is created, all relevant information critical result, alert level, patient name, radiologist, responding clinician, date and time of alert is recorded in the ANCR database. Our Fig. 2 Notification generated with yellow level alert. Centricity = Centricity PACS (GE Healthcare), Percipio = Percipio computerized physician ordering system (Medicalis), BWH = Brigham and Women s Hospital, QA = quality assurance. Fig. 3 Work list for unacknowledged alerts. RAD = radiology. institution has established ANCR as part of the EHR for maintaining details regarding closedloop communication of critical results. Therefore, radiologists can finalize reports containing critical results immediately after sending an ANCR alert rather than waiting for a response from the responding clinician. Radiologists can insert a macro in the report generation system to automatically insert critical results were communicated via the automated ANCR system into radiology reports, even while awaiting alert acknowledgment. Before ANCR implementation, many radiologists used their unsigned reports as a work list to ensure closed-loop documentation of critical results, delaying availability of the finalized report in the EHR. Furthermore, integration of ANCR alerts into the EHR result manager enables acknowledgment of ANCR alerts in the EHR to automatically pass the acknowledgment to ANCR. Integration With Analytics System ANCR is closely integrated with our radiology quality analytics system [6]. The ANCR audit and documentation capabilities enable capture of granular data for assessing adherence to the critical results communication policy at the institution, practice, or provider level and by radiology AJR:203, November 2014 W493

4 Lacson et al. TABLE 1: Satisfaction with Alert Notification of Critical Results (ANCR) Support and Utility Radiologist (n = 126) Ordering Provider (n = 1187) Category section. It also enables tracking the number of alerts in any specific time frame by alert level and the timeliness of acknowledgment (e.g., on time, late) of critical result alerts. Assessing Provider Satisfaction Surveys were administered to all radiologists (n = 320) and ordering providers (n = 4323) who sent or received ANCR alerts 3 years after ANCR implementation [3]. One survey for radiologists and another for ordering providers were adopted from an internally developed and validated questionnaire [7] and divided into four domains of general user information (three questions), ANCR usage patterns (five questions), user satisfaction with ANCR (five questions) and ANCR support (two questions), and provider attitudes toward ANCR (three questions). Responses to general questions were categorical. Usage and opinion questions were answered on a 7-point Likert scale. survey invitations included a link to the survey form. Surveys took less than 5 minutes to complete. A second mailing was sent to nonresponders after 2 weeks, and three more mailings were made, so that nonresponders received five total invitations. Research Electronic Data Capture (REDCap), a secure, web-based software application, was used to administer and manage the surveys and to compile and analyze the results. We dichotomized Likert scale responses so that responses greater than 4 indicated agreement or satisfaction. Categorical variables were compared by chi-square test. To reduce the chance of committing a type I error from multiple comparisons, we adjusted our alpha level of significance to using Bonferroni adjustment. We used chi-square Satisfied/Agree Dissatisfied/Disagree Satisfied/Agree Dissatisfied/Disagree ANCR support Training adequacy 89 (70.6) 16 (12.7) 254 (21.4) 655 (55.2) < a Access to help 69 (54.8) 25 (19.8) 231 (19.5) 511 (43.0) < a ANCR functionality Ease of use 62 (49.2) 41 (32.5) 519 (43.7) 279 (23.5) 0.24 Satisfaction for communication 68 (54.0) 33 (26.2) 559 (47.1) 289 (24.3) 0.14 Satisfaction for documentation 77 (61.1) 31 (24.6) 513 (43.2) 314 (26.5) a Satisfaction for tracking 65 (51.6) 38 (30.2) 417 (35.1) 306 (25.8) a Overall satisfaction 58 (46.0) 35 (27.8) 518 (43.6) 284 (23.9) 0.61 Attitude toward ANCR Reduces medical error 58 (46.0) 32 (25.4) 564 (47.5) 241 (20.3) 0.75 Improves patient care quality 63 (50.0) 39 (31.0) 540 (45.5) 272 (22.9) 0.33 Note Values in parentheses are percentages. Statistically significant. two-tailed analysis and R programming language (University of Auckland, New Zealand). Results The survey response rate was 50.4% for radiologists and 36.1% for ordering providers. Satisfaction With ANCR Support Ordering providers were generally dissatisfied with the training they received for using ANCR (Table 1) and with access to available help and support. In contrast, radiologists were satisfied with ANCR support more than half of the time. Satisfaction with training and support differed significantly between the two groups of providers (p < ). Satisfaction With ANCR Functionality Overall, 46.0% of radiologists and 43.6% of ordering providers were satisfied with ANCR. Radiologists were more likely than ordering providers to be satisfied with documenting critical result communication (61.1% vs 43.2%; p = ) and tracking critical results (51.6% vs 35.1%; p = ). Providers Attitudes Toward ANCR Compared with ordering providers, radiologists tended to agree more that ANCR reduces provider workload (42.1% vs 33.7%, p = 0.06). However, the difference did not reach the adjusted level of significance (p < 0.005). Both groups agreed that use of ANCR reduces medical errors (46.0% and 47.5%) and improves quality of patient care (50.0% and 45.5%). Discussion Key ANCR design features are highlighted in Table 2. Factors that enhanced workflow efficiency include reduction in the amount of required manual data input and provider lookup that needed to be performed by care providers and communication of concisely organized information in a less interruptive manner. Although ANCR can function as a standalone application, integration with relevant paging and systems, the PACS, and the EHR allows efficient access to ANCR for both radiologists and ordering providers. For radiologists, enterprise patient list integration facilitates one-click identification of responding clinicians for inpatients. Receivers can be automatically paged directly through the paging system or be contacted by generated in ANCR for less urgent alerts. PACS integration allows automatic recording of patient and examination information into critical result alerts, reducing manual data entry. In addition, documentation of ANCR communication is facilitated with a report generation macro, which enables radiologists to finalize radiology reports without delay. For ordering providers, the important contribution of ANCR is allowing asynchronous yet verifiable communication for less urgent yellow alerts, substantially reducing interruptive pages. Combining reminders for all overdue alerts into a single daily reduces the risk associated with alert fatigue. Radiologists were more likely than ordering providers to be satisfied with ANCR functionality, training, and support. Our re- p W494 AJR:203, November 2014

5 Automated Critical Test Result Notification TABLE 2: Key Features of Alert Notification of Critical Results (ANCR) Alert Communication Step Features User authentication Alert creation Alert communication Alert acknowledgment and management Alert reminder and escalation Alert documentation Single sign-on access to multiple clinical applications after authentication at an integrated PACS workstation Three progressively more urgent levels: yellow, orange, and red ANCR integration with PACS enables automatic extraction of patient and examination information, reducing time and potential for manual data entry error Enterprise patient list integration (inpatient) through web services allows identification of the responsible clinician at the time of alert generation; that provider can be selected for contact with a single click User authentication at ANCR launch enables automatic receiver paging through the hospital paging system web services notification allows alerts to be sent asynchronously without interrupting provider workflow with nonurgent pages or telephone calls Each critical alert requires a sender who is an independent practitioner; thus alerts generated by radiology trainees require documentation of a supervising physician ANCR enables closing the communication loop All unacknowledged alerts are visible on an ANCR work list ANCR sends escalation pages when urgent alerts are not acknowledged within the designated time frame according to policy All overdue alerts are combined into a single notification to limit alert fatigue for providers who have multiple overdue alerts When an ANCR alert is acknowledged within the electronic health record, it is also simultaneously acknowledged in ANCR Radiologists can insert the macro critical results were communicated via the automated ANCR system in the radiology report at the time of report generation and finalize the report without delay ANCR is part of the electronic health record, maintaining work lists for unacknowledged alerts and documentation of closed-loop communication for critical results sults underscore the relative simplicity of providing support to 320 radiologists in relatively limited physical distribution compared with the geographically dispersed users receiving alerts across the health system. The latter received limited in-person training owing to their numbers and geographic distribution. In addition, it was simpler to identify and train the more limited number of new physicians joining the radiology practice in the 3 years since initial implementation of ANCR than to ensure timely training for every new provider who joined our health care delivery system. Part of the providers dissatisfaction with ANCR support may also be due to negativity bias (selective recall of decreased support during initial implementation) 3 years before the survey was distributed [8]. Strategies for improved live support and streamlined ANCR training for every new provider joining the health care enterprise will likely improve user experience and satisfaction. [9, 10]. Nearly one half of the providers agreed that use of ANCR reduces medical errors and improves the quality of patient care, though less than one half of providers thought that ANCR improved provider workload for ensuring closed-loop communication of critical results. Our findings also highlight opportunities for further enhancements to ANCR. These include functionality to document communication of critical results to patients and to ensure timely follow-up of critical results to further improve quality and safety [11, 12]. Currently, the granularity of information collected in the ANCR database enables complete and comprehensive documentation of critical results communication. The centralization and standardization of these data facilitate quality initiatives to enhance and measure provider performance, efficiency, and productivity [13]. Most important, ANCR enables creation and documentation of critical result alerts, tracking of unacknowledged alerts, and facilitation of closed-loop communication between care providers. Conclusion ANCR enables automation of critical imaging test result communication between care providers, addressing a national patient safety goal. Although providers agreed that use of ANCR reduces medical errors and improves quality, radiologists were more satisfied with ANCR training and support than were ordering providers. Our results highlight the need for improved strategies for efficient training of large numbers of geographically dispersed providers. We also identified the need to expand the capabilities of critical results management information technology tools to include documented communication of critical results to patients and functionality to help ensure timely follow-up of critical results. Acknowledgments We thank Loraine Wu for providing help with Figure 1 and Laura E. Peterson for reviewing the manuscript. References 1. The Joint Commission website national patient safety goals. information/npsgs.aspx. Accessed January 1, Anthony SG, Prevedello LM, Damiano MM, et al. Impact of a 4-year quality improvement initiative to improve communication of critical imaging test results. Radiology 2011; 259: Brigham and Womens Hospital website. CEBI (Center for Evidence-Based Imaging). How AJR:203, November 2014 W495

6 Lacson et al. ANCR works. normal test results and ambulatory decision support qualitative study. BMC Med Inform Decis Mak search/labs/cebi/cctr/ancr.aspx. Modified systems. Int J Med Inform 2003;71(2 3: ; 13:67 February 27, Accessed July 8, Spaniol J, Voss A, Grady CL. Aging and emo- 11. Boohaker EA, Ward RE, Uman JE, McCarthy 4. Kreimer S. Quality and safety: alarming Joint tional memory: cognitive mechanisms underlying BD. Patient notification and follow-up of abnor- Commission, FDA set to tackle alert fatigue. the positivity effect. Psychol Aging 2008; mal test results: a physician survey. Arch Intern Hosp Health Netw 2011; 85: Phillips J. Clinical alarms: complexity and common sense. Crit Care Nurs Clin North Am 2006; 18: Khorasani R. Objective quality metrics and personal dashboards for quality improvement. J Am Coll Radiol 2009; 6: Murff HJ, Gandhi TK, Karson AK, et al. Primary care physician attitudes concerning follow-up of ab- 23: Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013; 20:e9 e Simon SR, Keohane CA, Amato M, et al. Lessons learned from implementation of computerized provider order entry in 5 community hospitals: a Med 1996; 156: McCarthy BD, Yood MU, Boohaker EA, Ward RE, Rebner M, Johnson CC. Inadequate followup of abnormal mammograms. Am J Prev Med 1996; 12: Mukundan G, Seidenwurm D. Performance measures, efficiency, productivity. Neuroimaging Clin N Am 2012; 22: W496 AJR:203, November 2014