Notifications Received by Primary Care Practitioners in Electronic Health Records: A Taxonomy and Time Analysis

Transcription

1 CLINICAL RESEARCH STUDY Notifications Received by Primary Care Practitioners in Electronic Health Records: A Taxonomy and Time Analysis Daniel R. Murphy, MD, MBA, a,b,c Brian Reis, BE, a Dean F. Sittig, PhD, d Hardeep Singh, MD, MPH a,b a Houston VA HSR&D Center of Excellence and Houston VA Patient Safety Center of Inquiry, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Tex; b Section of Health Services Research, Department of Medicine and c Department of Family and Community Medicine, Baylor College of Medicine, Houston, Tex; d University of Texas School of Biomedical Informatics and the UT- Memorial Hermann Center for Healthcare Quality & Safety, Houston, Tex. ABSTRACT BACKGROUND: Asynchronous electronic health record (EHR)-based alerts used to notify practitioners via an inbox-like format rather than through synchronous computer pop-up messages are understudied. Our objective was to create an asynchronous alert taxonomy and measure the impact of different alert types on practitioner workload. METHODS: We quantified and categorized asynchronous alerts according to the information they conveyed and conducted a time-motion analysis to assess practitioner workload. We reviewed alert information transmitted to all 47 primary care practitioners (PCPs) at a large, tertiary care Veterans Affairs facility over 4 evenly spaced 28-day periods. An interdisciplinary team used content analysis to categorize alerts according to their conveyed information. We then created an alert taxonomy and used it to calculate the mean number of alerts of each type PCPs received each day. We conducted a time-motion study of 26 PCPs while they processed their alerts. We used these data to estimate the uninterrupted time practitioners spend processing alerts each day. RESULTS: We extracted 295,792 asynchronously generated alerts and created a taxonomy of 33 alert types categorized under 6 major categories: Test Results, Referrals, Note-Based Communication, Order Status, Patient Status Changes, and Incomplete Task Reminders. PCPs received a mean of 56.4 alerts/day containing new information. Based on 749 observed alert processing episodes, practitioners spent an estimated average of 49 minutes/day processing their alerts. CONCLUSIONS: PCPs receive a large number of EHR-based asynchronous alerts daily and spend significant time processing them. The utility of transmitting large quantities and varieties of alerts to PCPs warrants further investigation. Published by Elsevier Inc. The American Journal of Medicine (2012) 125, 209.e1-209.e7 KEYWORDS: Asynchronous alerts; Classification; Electronic health records; Health information technology; Medical informatics; Primary care; Taxonomy; Time and motion studies Funding: This work was funded by the Veterans Affairs National Center for Patient Safety, Baylor College of Medicine Department of Family & Community Medicine Post Doctoral Fellowship program, a SHARP contract from the Office of the National Coordinator for Health Information Technology (ONC # ), and in part by the Houston VA HSR&D Center of Excellence (HFP90-020). These sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. Conflict of Interest: None. Authorship: All authors had full access to all of the data in the study, participated in the writing of the manuscript, and take responsibility for the integrity of the data and the accuracy of the data analysis. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs. Requests for reprints should be addressed to Daniel R. Murphy, MD, MBA, Michael E. DeBakey Veterans Affairs Medical Center (MEDVAMC), HSR&D Center of Excellence (152), 2002 Holcombe Boulevard, Houston, TX address: drmurphy@bcm.edu /$ -see front matter Published by Elsevier Inc. doi: /j.amjmed

2 209.e2 The American Journal of Medicine, Vol 125, No 2, February 2012 Most electronic health records (EHRs) use alerts to facilitate information delivery to practitioners at the point of care. Two main alert classes exist. Synchronous alerts pop up immediately in response to certain practitioner actions in the EHR (eg, drug-drug interaction alerts during medication order entry 1 ). Asynchronous alerts communicate potentially important and actionable clinical information in an inbox-like format (much like ), irrespective of active practitioner-computer interaction, that is, the sender and receiver need not be engaged simul- CLINICAL SIGNIFICANCE taneously. 2 Recent evidence suggests that synchronous alerts cause information overload and are often ignored; 3-5 thus, these alerts have received significant recent attention. Conversely, little is known about asynchronous alerts that might rapidly accumulate in message inboxes. Asynchronous alerts often convey valuable and actionable information between traditional face-to-face patient-practitioner encounters. 6 Unlike synchronous alerts, asynchronous alerts require practitioners to actively access their EHR s inbox application to read alerts. If important alerts are missed, practitioners might not become aware of critical information until a future patient visit, leading to potential delays in diagnosis or treatment. 7 For instance, when we studied asynchronous alerts related to abnormal imaging and laboratory test results, we found that practitioners did not always acknowledge or follow-up these alerts in a timely manner. 8,9 In our subsequent qualitative work, primary care practitioners (PCPs) reported receiving a large number and variety of asynchronous alerts each day and believed that some alerts did not provide critical information and were therefore unnecessary. 10,11 Although no empiric evidence exists, information overload may make practitioners vulnerable to overlooking important data such as abnormal test results Our objective was to categorize asynchronous alerts according to the information they conveyed and measure their impact on practitioner workload in terms of time spent processing them. Documenting the information load, in terms of quantity and type, conveyed by asynchronous alerts, is necessary for improving safety and efficiency of care through EHR-based notification systems. METHODS Setting We evaluated alerts received by PCPs practicing at 10 Veterans Affairs (VA) primary care clinics affiliated with a Electronic health record-based alerts are increasingly used to notify practitioners about actionable patient information, much like asynchronous messages. Primary care practitioners receive large numbers and types of asynchronous alerts daily, and spend an estimated 50 minutes of daily non-face-to-face time processing them. Current and future electronic health record-based notification systems must balance providing sufficient information to make informed decisions without overburdening practitioners to the point of information overload. large tertiary care hospital. The study was approved by the local Institutional Review Board. The VA uses an integrated EHR, the Computerized Patient Record System, at all facilities, nationwide. At log-in and when switching between patient records, the Computerized Patient Record System displays the View Alert inbox that contains asynchronous alerts for a practitioner s patients (Figure 1). This inbox holds alerts until processed by the practitioner; most alerts are automatically removed if unprocessed after a prespecified time-period. While this functionality is uniform across the VA, most decisions on which alerts practitioners receive are made at the facility level, usually by EHR committees that have both information technology and clinical representatives. Similar inbox functionality is available in many other commercially available EHRs. Data Collection All asynchronous alerts are recorded in a centralized tracking file, which holds information about the origin, content, and recipients of each alert transmitted during the prior 30 days. We wrote software programs to extract information from this file on all asynchronous alerts delivered electronically to any physician, nurse practitioner, or physician assistant designated in the EHR as a full-time PCP at the study sites. Because this file is organized as a 30-day running list, we removed the first and last days of every sample to eliminate inclusion of days with partial data. To reduce bias from seasonal variation, alert logs were extracted every 3 months in 4 periods of 28 consecutive days. Three fields were extracted for each alert: alert message; transmission date; and recipient-practitioner identification number. Alert Categorization Using a qualitative content analysis method 15,16 and prior work on a clinical decision support taxonomy 17 as a methodological framework, individuals with expertise in primary care, computer programming, and informatics systematically analyzed alert logs to create a taxonomy. Each alert was evaluated based on its message field, which included 2 sub-fields: generic templated-text common to all alerts of the same type (eg, Abnormal Lab ), and patient-specific information (eg, WBC 14.1 ). We used structured query language programs to automatically parse the text of all alerts in our log files and group alerts with the same generic templated text as a single alert type. We then manually analyzed a randomly selected sample of alert message fields from each type and abstracted inferences on 2 themes:

3 Murphy et al Electronic Health Record-based Notifications in Primary Care 209.e3 Figure 1 The View Alert portion (lower half, boxed) of the Patient Selection window with sample alerts. information conveyed to the practitioner (such as laboratory result or patient status update); and the event that triggered the alert (such as when a referral note was created or a patient was discharged). Four types had message fields that were unclear, and thus we performed chart reviews to identify context. Based on team discussions and consensus, all alert types were iteratively grouped into 6 unique categories to develop a taxonomy. Analysis of Information Load After categorization, we tallied the number of alerts of each type and calculated mean alerts/day by dividing the total number of alerts by the number of work days in the study period. Using the total number of unique practitioner identification numbers, we then calculated the mean number of alerts per practitioner per work day. Data storage and analysis were performed using Microsoft Access and Excel (Microsoft Corporation, Redmond, Wash). Alert Impact on Practitioner Time We conducted a time-motion analysis 18,19 by observing PCPs and recording the time they spent processing their alerts. PCPs were invited to participate via announcements at monthly departmental meetings and through s. Immediately before each observation session, we printed a screenshot of the PCP s alert inbox to facilitate data collection without practitioner interruption. Using this screenshot as a guide, a study investigator recorded alert processing time (time between clicking on the alert and exiting the patient chart) for each alert using a stopwatch. Interruptions unrelated to alert processing (eg, visits from other clinicians) were recorded but manually removed from the data. The practitioner was observed nonintrusively and encouraged to work at his or her normal pace. Incompletely processed alerts, identified by the provider during the session, and alerts related to inpatient care, were excluded from analysis. RESULTS We extracted information from 295,792 unique asynchronous alerts transmitted to PCPs over 4 28-day time periods between July 29, 2009 and May 25, 2010 that included a total of 78 working days. The mean number of alerts transmitted each work day was 3607 (range ), and 426

4 209.e4 The American Journal of Medicine, Vol 125, No 2, February 2012 Figure 2 Alert taxonomy. ( ) for weekends and holidays. We identified a mean of 47 recipient PCPs in the 4 study periods. Alert Categorization We identified 33 unique alert message types through content analysis. These alert types were grouped into the following 6 categories according to the alert s purpose (Figure 2): 1) Test Result Alerts informed practitioners when test results became available, including Laboratory, Imaging, and Pathology. 2) Referral Alerts transmitted information to practitioners about the progression of their patients through the referral process and included all messages connected to a specific referral request. 3) Note-Based Communication Alerts facilitated communication between practitioners via progress notes, such as informing practitioners when an addendum was added to their own electronic note or when another clinician or trainee requested their signature. 4) Order Status Alerts informed practitioners when an order s status had changed, its expiration was imminent, or when clarifications regarding orders were requested by other clinicians. 5) Patient Status Change Alerts communicated major changes in patient status, including admission, discharge, and patient death. 6) Incomplete Task Reminders served as reminders of yet-unfinished tasks, indicating that a note or order had been initiated in the EHR but not yet completed or canceled. These were self-generated by PCPs and did not convey new information and thus were removed from subsequent analysis. Additionally, each subsequent alert for the same note or order replaced its prior instance, preventing accurate quantification. Alert Quantification Alert quantities in each category were tallied and divided accordingly to yield the number of alerts per practitioner per work day (Table, Column C). Practitioners received a mean of 56.4 alerts each work day. A significant proportion (44.9%) communicated information unrelated to test results. Most of these non-test-result alerts were triggered by referral status changes (19.8%) and note-based communication within the EHR (17.2%). Abnormal Laboratory Result was the most common type of alert transmitted (14.9/practitioner/work day; 26.5% of all alerts), followed by Additional Signature Request alerts (9.3; 16.5%) and Normal Laboratory Result alerts (8.6; 15.2%). Workload Involved in Alert Processing (Time-Motion Analysis) Of 47 full-time PCPs, 26 responded to requests to participate in the time-motion study, which allowed an observation of 749 individual alert-processing episodes. A total of 14 hours and 25 minutes of provider processing time was observed (excluding setup and interruption times, and transition times between finishing one alert and beginning the next). The mean time taken to process an alert was 85 seconds, with wide variations depending on alert type (95% confidence interval, seconds), (Table, Column D). We were unable to observe the processing of 8 rarely encountered alert types. For these, we substituted the overall mean time for all alert types to estimate their contribution to the total processing time. In addition to being one of the most frequent types, Additional Signature Request alerts required the highest average processing time of 113 seconds (95% confidence interval, seconds). We estimated the contribution of each alert type to practitioners processing time by multiplying the mean alert processing time by the mean number received/day for each alert type (Table, Column E). After aggregation, we estimated that, at the minimum, practitioners spent a mean uninterrupted time of 49 minutes and 6 seconds to process all of their alerts each day. DISCUSSION We analyzed EHR-based asynchronous alerts communicated to PCPs in order to determine their information load (content, quantity, and workload impact) on daily schedules. We identified 33 unique alert types and developed a taxonomy to advance the understanding of what alert types

5 Murphy et al Electronic Health Record-based Notifications in Primary Care 209.e5 Table Analysis of Alerts Received by Primary Care Practitioners A B C D E Total Alerts Alert Type n (%) Average Alerts per Practitioner per Day n Average Time to Process One Alert (m:s) Average Time to Process Each Day (m:s) Test result alerts Laboratory results Normal laboratory result 31,464 (15.2) 8.6 0:37 5:16 Abnormal laboratory result 54,733 (26.5) :42 10:26 Critically abnormal laboratory result 11,110 (5.4) 3.0 0:54 2:42 STAT laboratory result 932 (0.5) 0.3 0:26 0:07 Imaging results Noncritical imaging result 8217 (4.0) 2.2 1:30 3:22 Abnormal imaging result 6438 (3.1) 1.8 1:20 2:21 STAT imaging result 375 (0.2) 0.1 1:51 0:11 Imaging result amended 140 ( 0.1) :06 0:03 Pathology results Pathology result 610 (0.3) 0.2 0:33 0:05 Total test result alerts 114,019 (55.1) 31.1 Referral alerts Referral status changes Referral request received 1362 (0.7) 0.4 Appointment scheduled 13,871 (6.7) 3.8 0:15 0:57 Referral cancelled 4991 (2.4) 1.4 0:56 1:17 Referral reactivated 9 ( 0.1) :25* 0:01* Referral forwarded 1588 (0.8) 0.4 0:16 0:07 Consult completed 18,593 (9.0) 5.1 0:31 2:36 STAT consult completed 30 ( 0.1) :25* 0:01* Consult results modified 463 (0.2) 0.1 0:34 0:04 Referral communication Comment added to referral request 11,904 (5.8) 3.2 0:13 0:43 Total referral alerts 52,811 (25.5) 14.4 Note-based communication Clinical notes Co-signature request 1547 (0.7) 0.4 0:55 0:23 Additional signature request 34,023 (16.5) 9.3 1:53 17:30 Note addendum added 83 ( 0.1) :06 0:03 Total clinical communication alerts 35,653 (17.2) 9.7 Order-related alerts Order status changes Laboratory order canceled 459 (0.2) 0.1 0:27 0:03 Imaging order canceled 1608 (0.8) 0.4 0:22 0:10 Imaging order held 525 (0.3) 0.1 0:08 0:01 Imaging order modified 68 ( 0.1) :25* 0:02* Imaging order started 714 (0.3) 0.2 1:25* 0:16* Order near expiration Medication order expiring 704 (0.3) 0.2 1:25* 0:16* Order clarifications Order clarification request 2 ( 0.1) :37 0:01 Total order status alerts 4080 (2.0) 1.1 Patient status change alerts Patient admitted 141 (0.1) :25* 0:03* Patient discharged 44 ( 0.1) :25* 0:01* Patient expired 2 ( 0.1) :25* 0:01* Total patient status 187 (0.1) 0.05 Total alerts Totals 206,750 (100%) :06 *We were unable to time the processing of certain alert types due to their low frequency. The overall mean processing time was used to estimate their contribution to total alert processing time each day. These alerts were received by only a select few practitioners who have taken on additional responsibilities. Because they are not typical of alerts received by primary care practitioners, they were excluded in our totals. practitioners receive through their EHRs. PCPs received a mean of 56.4 alerts with new clinical information each day, and spent an estimated 50 minutes processing them; certain alert types led to higher contributions to workload. This information lays the foundation for future research to improve the safety and efficiency of care through EHR-based notification systems. Our study has several implications. PCPs appear to receive a large quantity of electronically delivered information each day and spend a significant proportion of their

6 209.e6 The American Journal of Medicine, Vol 125, No 2, February 2012 work day processing it. We previously found that about 7% of abnormal laboratory and imaging test result alerts were missed in EHR systems. 8,9 Information overload from too many alerts might be a potential reason, given that practitioners must sort through a large number of alerts daily This phenomenon has been described as alert fatigue and can lead busy practitioners to overlook critical information 3 and further lead to adverse patient outcomes. 4 This is, to our knowledge, the first study that quantifies EHR-based asynchronous alerts and their impact on PCPs work day. These implications are significant for current and future EHRbased notification systems, which will need to balance providing sufficient information to practitioners to make informed decisions without overburdening them to the point of information overload. Unexpectedly, we found that only half of the alerts transmitted were related to test results. Our proposed taxonomy establishes the groundwork to decipher what alert types are absolutely necessary for safe and efficient clinical care. Certain types, such as Additional Signature Request alerts that relay note-based messages between practitioners (eg, informing them of patient phone calls or need for medication refills), were particularly prevalent, comprising over one sixth of all alerts and requiring up to a quarter of the processing time. This is not surprising because processing them requires accessing multiple sections of the chart and calling patients or other clinicians. As opposed to abnormal test results, the clinical relevance of these alerts and their impact on safe patient care is largely unknown. Because they potentially transmit clinically actionable information, further evaluation is needed before concluding that such alerts are ideal targets for overload reduction. While practitioners have always managed an influx of patient information in paper-based systems, electronic notifications exert a remarkable change on practitioners clinical workflow. Prior research has shown that practitioners may not manage this influx of electronic information as efficiently. 10,20-23 For instance, practitioners might spend a substantial amount of time sifting through electronic records to locate information. 10 Additionally, because it is far easier to create additional electronic alerts than paper-based ones, institutions might be more willing to keep adding new alerts to their EHRs to improve communication. Therefore, the potential for overload is greater because practitioners can more easily receive potentially unnecessary information they traditionally would not have. Despite this impact on workflow, few studies have reported on the direct effect of electronic alerts on provider non-face-to-face work time. One study suggests that data acquisition activities have increased after EHR implementation; 24 however, the role alerts played in this difference was not specifically evaluated. Based on our findings, we identify several next steps to improve the effectiveness of EHR-based notification systems. First, alert information should be quickly and easily understood by practitioners. We found certain alert types with ambiguous text, such as New Result Associated. Practitioners often override warning messages from synchronous alerts when they are presented with an abundance of incorrect, redundant, confusing, 25 or clinically insignificant information. 26 Asynchronous alert messages also must be succinct, intuitive, and lack confusing abbreviations. 27 Second, the interface used to display alerts should be optimized to increase processing efficiency and situational awareness of the practitioner, and practitioners must be educated on the presence of an alert system s efficiencyenhancing features. For instance, we found during categorization that all alert texts displayed only a single piece of information, which almost always lacked the detail required to make a clinical decision without opening the patient s record to review it. Providing certain additional patientspecific information in the alert text, such as a prior laboratory value in the case of an abnormal laboratory result, would permit better situational awareness and improve efficiency. 28 Third, during our time-motion study, interruptions were common and were reportedly more frequent between patient visits. Because interruptions are known to adversely affect care, 29 practitioners might benefit from setting aside designated non-face-to-face time to process these alerts. 33 Fourth, further research is needed to discern clinically relevant alerts from those that have little impact on patient outcomes. To reduce information load on practitioners, alerts irrelevant to clinical care or having little effect on patient outcomes can be removed from alert systems or activated by practitioners only on an opt-in basis. Identifying alerts that were ignored by PCPs also may help illuminate the clinical relevance of individual alert types. Finally, multiple factors beyond the EHR system itself influence how information is delivered to practitioners, including organizational policies (decision to transmit certain alert types) and provider behavior and culture (excessive use of additional signature alerts for FYI [for your information] reasons). Thus, efforts to improve alerting systems must consider a broad range of sociotechnical factors that might affect alert quantities. 34 Several limitations merit mention. First, our results might not generalize outside the VA and to other EHRs. Nevertheless, asynchronous alerts are being increasingly used in other EHRs, and thus, many of our findings will be applicable. Furthermore, our methods and findings will likely be useful to practitioners who are now adopting EHRs, and perhaps motivate institutions to carefully consider which alerts to transmit during initial stages of EHR configuration and implementation. Second, we were unable to exclude inpatient-related alerts, potentially inflating our findings, as some PCPs also provide occasional inpatient care. Third, practitioners can customize alert settings and disable certain nonmandatory alerts 35 and hence, information load might vary in response to practitioner preferences. However, our methods use an aggregation of close to 300,000 alerts sent to 47 different practitioners, which par-

7 Murphy et al Electronic Health Record-based Notifications in Primary Care 209.e7 tially mitigates this effect. Fourth, we may have introduced a Hawthorne effect 36 during our observations and altered practitioner behavior. Fifth, we were unable to recruit all PCPs for the time motion study, and thus, participants may not be a fully representative sample. Additionally, practitioners may have specifically chosen observation session times when they expected fewer than normal interruptions. However, this would lead to an underestimation of alert processing time. Finally, we did not evaluate the association between alert load and medical errors, although this remains a focus of our future research. CONCLUSION Practitioners receive a substantial information load from EHR-based asynchronous alert notification systems and spend a significant proportion of each day processing alerts. The utility of transmitting large quantities and varieties of alerts to PCPs and their potential to cause information overload warrants further investigation. Future research is needed to reduce the likelihood of causing patient harm from missing critical information among these large numbers of alerts. References 1. Kuperman GJ, Bobb A, Payne TH, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc. 2007;14(1): Coiera E, Tombs V. Communication behaviours in a hospital setting: an observational study. BMJ. 1998;316(7132): Magnus D, Rodgers S, Avery AJ. GPs views on computerized drug interaction alerts: questionnaire survey. J Clin Pharm Ther. 2002; 27(5): Weingart SN, Toth M, Sands DZ, et al. Physicians decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003;163(21): Taylor LK, Tamblyn R. Reasons for physician non-adherence to electronic drug alerts. Stud Health Technol Inform. 2004;11(Pt 2): Poon EG, Wang SJ, Gandhi TK, Bates DW, Kuperman GJ. Design and implementation of a comprehensive outpatient Results Manager. J Biomed Inform. 2003;36(1-2): Wahls TL, Cram PM. The frequency of missed test results and associated treatment delays in a highly computerized health system. BMC Fam Pract. 2007;8: Singh H, Thomas EJ, Mani S, et al. Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential? Arch Intern Med. 2009; 169(17): Singh H, Thomas EJ, Sittig DF, et al. Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain? Am J Med. 2010;123(3): Hysong SJ, Sawhney MK, Wilson L, et al. Provider management strategies of abnormal test result alerts: a cognitive task analysis. JAm Med Inform Assoc. 2010;17(1): Hysong SJ, Sawhney MK, Wilson LA, et al. Understanding the management of electronic test result notifications in the outpatient setting. BMC Med Inform Decis Mak. 2011;11(1): Wahls T. Diagnostic errors and abnormal diagnostic tests lost to follow-up: a source of needless waste and delay to treatment. J Ambul Care Manage. 2007;30(4): Fava GA, Guidi J. Information overload, the patient and the clinician. Psychother Psychosom. 2007;76(1): Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc. 2004;11(2): Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1): Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2): Wright A, Sittig DF, Ash JS, et al. Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems. J Am Med Inform Assoc. 2011;18(3): Bratt JH, Foreit J, Chen PL, et al. A comparison of four approaches for measuring clinician time use. Health Policy Plan. 1999;14(4): Finkler SA, Knickman JR, Hendrickson G, Lipkin M, Thompson WG. A comparison of work-sampling and time-and-motion techniques for studies in health services research. Health Serv Res. 1993;28(5): Poon EG, Gandhi TK, Sequist TD, et al. I wish I had seen this test result earlier! : Dissatisfaction with test result management systems in primary care. Arch Intern Med. 2004;164(20): Schumacher RM. Commentary: electronic health records and human performance. J Oncol Pract. 2010;6(3): Linder JA, Schnipper JL, Tsurikova R, et al. Barriers to electronic health record use during patient visits. AMIA Annu Symp Proc. 2006: Simon SR, McCarthy ML, Kaushal R, et al. Electronic health records: which practices have them, and how are clinicians using them? J Eval Clin Pract. 2008;14(1): Pizziferri L, Kittler AF, Volk LA, et al. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. J Biomed Inform. 2005;38(3): Weingart SN, Simchowitz B, Shiman L, et al., others. Clinicians assessments of electronic medication safety alerts in ambulatory care. Arch Intern Med. 2009;169(17): Shah NR, Seger AC, Seger DL, et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc. 2006;13(1): Feldstein AC, Smith DH, Robertson NR, et al. Decision support system design and implementation for outpatient prescribing: the Safety in Prescribing Study. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation, Vol 3: Implementation Issues. Rockville, MD: Agency for Healthcare Quality and Research; Singh H, Petersen LA, Thomas EJ. Understanding diagnostic errors in medicine: a lesson from aviation. Qual Saf Health Care. 2006;15(3): Biron AD, Loiselle CG, Lavoie-Tremblay M. Work interruptions and their contribution to medication administration errors: an evidence review. Worldviews Evid Based Nurs. 2009;6(2): Doerr E, Galpin K, Jones-Taylor C, et al. Between-visit workload in primary care. J Gen Intern Med. 2010;25(12): Farber J, Siu A, Bloom P. How much time do physicians spend providing care outside of office visits? Ann Intern Med. 2007;147(10): Ammenwerth E, Spötl H-P. The time needed for clinical documentation versus direct patient care. Methods Inf Med. 2009;48(1): Margolius D, Bodenheimer T. Transforming primary care: from past practice to the practice of the future. Health Aff (Millwood). 2010; 29(5): Sittig DF, Singh H. Eight rights of safe electronic health record use. JAMA. 2009;302(10): Singh H, Wilson L, Reis B, et al. Ten strategies to improve management of Abnormal Test Result Alerts in the Electronic Health Record. J Patient Saf. 2010;6(2): Mayo E. The Human Problems of an Industrial Civilization. New York: MacMillan; 1993.