Evidence to Support Application of a Usability Assessment Tool for Nursing Documentation

Janette Coble, MS, BSN, RN Staff Nurse, Neuroscience St. Louis Children s Hospital jcc6154@bjc.org Evidence to Support Application of a Usability Assessment Tool For Nursing Documentation Janette Coble, BSN, MS, RN St. Louis Children s Hospital Background Implementation Testing Results Outcomes Usability issues in Nursing Documentation Systems can impact patient safety. At our hospital we currently monitor and address usability issues reported through a Safety Event Monitoring System, but there may be non-reported usability problems that affect patient safety. In addition, usability issues can significantly decrease RN satisfaction, efficiency, and effectiveness causing RN s to spend more time and effort charting their care with increased frustration. PICO Question For pediatric inpatients, would application of a usability assessment tool to a nursing documentation system result in improved patient safety and improved nursing satisfaction, efficiency, and effectiveness? A team of staff nurses, nursing managers, and nurses from clinical information systems worked together to perform an initial round of usability testing for the current nursing documentation system. We used the Think-Aloud protocol for usability testing and Morae recording software to support the tests and analysis. This provided a current level of usability and identified specific usability issues. Changes were made to the system to address selected priority issues and a second round of usability testing occurred to determine if the changes improved the system s usability. We tested with 9 Staff RN s representing non-icu floors in both of Rounds 1 and 2. The RN s performed typical tasks, such as an initial shift assessment, a discharge, an admission, and a midday assessment. System Usability Scale (SUS) score: 72.5 68-100: Above average score < 68 Below average score Quality of Use Metric 4 Questions after each task Overall score: 73 Quality of Use Scores Persistent Usability Issues: Extra mouse / keyboard clicks Some are informative and should stay Some are simply extra work for RN Some promote errors of omission Extra scrolling Most can be redesigned to eliminate scrolling Incorrect charting Documentation of actual time of event vs. time documenting event Duplicate charting Conclusions Correcting usability issues after production is less effective than during the design stage & only minimal corrections possible Correcting usability issues after production is more costly and system limitations impact ability to improve the HER Evidence Summary The Healthcare industry calls for more usable Electronic Health Records (EHR). The National Institute of Standards and Technology has created three separate documents (NISTIR 7804, NISTIR 7741, NISTIR 7865) from three separate efforts stating the need and approach for improving usability of EHR s. The HIMMS Usability Task Force said in their report, Defining and Testing EMR Usability: Principles and Proposed Methods of EMR Usability Evaluation and Rating that usability has a strong often direct relationship with error rates and user fatigue. For Round 2 we added a task to document patient/family education due to a significant change in this feature. Q of Use Meaning of Score 90 100 (A) Users consistently accomplish tasks, sometimes exceed expectations 80 90 (B) Most users easily accomplish tasks 70 80 (C) Users accomplish tasks, but rough edges have an impact 60 70 (D) Many users accomplish tasks, but many reject system < 60 (F) Most will not accomplish tasks and most will reject system Usability testing early and often in the development process is critical Selected References Keenan, G., Yakel, E., Dunn, L. K., Tschannen, D., Ford, Y. B., (2013). Challenges to nurses efforts of retrieving, documenting, and communication patient care information. Journal of American Medical Informatics Association, 20, 245-51. Kjeldskov, J., Skov, M. B., Stage, J. (2010). A longitudinal study of usability in health care: does time heal?. International Journal of Medical Informatics, 79, e135-43. National Institute of Standards and Technology (2010). NIST guide to the processes approach for improving the usability of electronic health records. (NISTIR 7741). Gaithersburg, MD. National Institute of Standards and Technology (2012). Technical Evaluation, Testing, and Validation of the Usability of Electronic Health Records. (NISTIR 7804). Gaithersburg, MD. National Institute of Standards and Technology (2012). A human factors guide to enhance EHR usability of critical user interactions when supporting pediatric patient care. (NISTIR 7865). Gaithersburg, MD. Rogers, M. L., Sockolow, P. S., Bowles, K. H., Hand, K. E., George J. (2013). Use of a human factors approach to uncover informatics needs of nurses in documentation of care. International Journal of Medical Informatics, 82, 1068-1074. Despite this focus on creating more usable EHR s, there have been minimal studies applying usability assessments to nursing documentation systems. 2016 1

Healthcare industry calls for more usable Electronic Health Records: National Institute of Standards and Technology NISTIR 7804 NISTIR 7741 NISTIR 7865 HIMSS Usability Task Force: Usability has strong often direct relationship with error rates and user fatigue Minimal published usability studies Usability issues in nursing documentation systems (KIDDOS): Compromise Patient safety Decrease RN satisfaction, efficiency, and effectiveness 2016 2

For inpatients at St. Louis Children s Hospital, would application of a usability assessment tool to a nursing documentation system result in improved patient safety, and improved nursing satisfaction, efficiency, and effectiveness? Form a Team Usability Test Make Changes to KIDDOS New KIDDOS 2016 3

Evaluate the usability of a system Effectiveness Efficiency Satisfaction Patient safety Our approach: Think-Aloud Protocol Production system Morae Recording Software Specific Users: 18 Staff RN s general, non-icu units. Specific Context of Use (Tasks): Initial Shift Assessment Discharge Direct Admission Mid-shift Assessment (Round 1 only) EU Admission Education Log (Round 2 only) 2016 4

Overall: 84 73 Extra mouse/keyboard clicks Extra Scrolling Incorrect charting Variations/Confusion in charting 2016 5

Improved KIDDOS usability Correcting usability issues is challenging: More costly, less effective when system is in production Inherent system limitations Transitioning to EPIC Establish usability testing process early and often! NICU research project 2016 6