Back to the Future: Moving Towards Real-Time, Actionable Outcome Measures Roni H. Amiel Scott M. Klein, MD, MHSA John Settembrini Jill Wegener, RN, MSN 95 Bradhurst Avenue Valhalla, NY 10595 www.blythedale.org Objectives Understand fundamental concepts of outcome design and measurement Learn how to identify key performance measures Learn techniques to create real-time, actionable outcome dashboards Format Understand fundamental concepts of outcome design and measurement 15 minute presentation Learn how to engage staff in the outcome process 15 minute presentation Learn techniques to create real-time, actionable outcome dashboards 30 minute presentation Facilitated Discussion 30 minutes 1
Understand Fundamental Concepts of Outcome Design and Measurement Theory of outcome measurement Examples of current published outcome measures Good and bad aspects of current measures Need for additional outcomes unique to individual organizations Data collection and IT barriers Theory of Outcome Measurement Evaluations of quality of care, Donabedian, 1966 Structure Process Outcomes Theory of Outcome Measurement Crossing the Quality Chasm Berwick, 2002 Six dimensions of quality Safe Effective Efficient Timely Patient centered Equitable 2
Theory of Outcome Measurement Milestones Along the Quality Measurement Journey Milestone Activities performed at this milestone 1 Develop a measurement philosophy 2 Identify the concepts to be measured (types and categories of measures) 3 Select specific measures 4 Develop operational definitions for each measure 5 Develop a data collection plan and gather the data (giving special consideration to stratification and sampling) 6 Analyze the data using statistical process control measures (especially run and control charts) 7 Use the analytic results (data) to take action (implement cycles of change, test theories, and make improvements) Lloyd in The Healthcare Quality Book, 2008 Theory of Outcome Measurement Concept versus specific measure Concepts often are ambiguous For example, Patient Satisfaction What aspect of Patient Satisfaction to measure? For example, Food Service What specific measures to track? For example, Percentage of trays delivered within 15 minutes of scheduled meal Percentage of meals with correct allergy alerts Theory of Outcome Measurement Operational definition of specific measures Gives communicable meaning to a concept or idea Clear and unambiguous Specifies the measurement method, procedures and equipment Provides decision-making criteria when necessary Enables consistency in data collection Lloyd in The Healthcare Quality Book, 2008 3
Theory of Outcome Measurement Good Performance Measures Relevant Reliable Valid Cost-effective Under the control of the provider Precisely defined and specified Interpretable Risk adjusted or stratified Loeb et al, Statistical Tools for Quality Improvement, The Healthcare Quality Book, 2008 Theory of Outcome Measurement Dashboard Ongoing performance of critical processes Lead to organizational success but not the success itself Dashboard in Car Fuel, Speed, Temperature How you get to destination, but not the destination itself Scorecard Record and report past performance not real-time performance Outcome measures rather than process measures Investment disclaimers Past performance does not guarantee future results Examples of Current Published Outcome Measures Volume/Demographic Total Discharges Length of Stay Case Mix Index Discharge to Community Hours of Therapy a Day 4
Examples of Current Published Outcome Measures National Benchmarks (CMS, NQF, etc) Percent of Residents or Patients with Pressure Ulcers that are New or Worsened (Short Stay) National Healthcare Safety Network Catheter Associate Urinary Tract Infection (CAUTI) Outcome Measure Change in Self-Care Score for Medical Rehabilitation Patients Change in Mobility Score for Medical Rehabilitation Patients Discharge Self-Care Score for Medical Rehabilitation Patients Discharge Mobility Score for Medical Rehabilitation Patients Influenza Vaccination among Healthcare Personnel Percent of Residents or Patients who were Assessed and Appropriately Given the Seasonal Influenza Vaccination All Cause Unplanned 30 day post-irf Discharge Readmission Measure Facility-Wide Inpatient Hospital-Onset Methicillin-Resistant Staphylococcus aureus (MRSA) Bacteremia Outcome Measure Facility-Wide Inpatient Hospital-Onset Clostridium difficile Infection (CDI) Outcome Measure The Good and Bad Current Outcome Measures Good Benchmarks Comparable between like institutions Research Target Serious Complications on a National Level Bad Does not measure process Implies broken process, but doesn t guide next steps Often these outcomes are not the areas the institution really wants to focus upon Concerns about universal applicability (i.e. case mix, etc) The Need for Better Outcomes What are my problem areas? What are the outcomes that need help? What are the processes that lead to those outcomes? How can I measure them? How can I impact the outcomes before they show up on a scorecard? 5
The Need for Better Outcomes Example Pressure Ulcers as an outcome We don t have many pressure ulcers Good! (is it case mix or excellent processes?) We have to monitor outcomes anyway Not a lot of interest or buy-in Especially for inservices since we don t have a problem with ulcers We do have issues with correct documentation of wound staging and interventions This is my problem area and important to me I m more likely to be engaged in the process Unfortunately, there aren t any NQF measures for this problem, so it has to be homegrown Stay tuned Data Collection and IT Barriers Why are you collecting these data? Will the data add value to your quality improvement efforts? How will these data be used to make a difference? What process will be monitored? What specific measure will be collected? What are the operational definitions of the measures? Where is the data? How will you collect the data? Who will collect the data? What costs will be incurred by collecting these data? How will the data be coded, edited and verified? Analyzing Data Data refers to the raw facts and figures which are collected as part of the normal functioning of the hospital. Information, on the other hand, is defined as data, which have been processed and analyzed in a formal, intelligent way, so that the results are directly useful to those involved in the operation and management of the hospital. Austin, 1983 6
Stakeholder Buy-In Staff Engagement STAFF ENGAGEMENT = Satisfied Staff Energized + Staff + Productive Staff Staff Engagement Individual Leaders ** Organization Hess, V. (2013). Lead and Succeed in a Do-More with-less World: 6 Shortcuts to Employee Engagement 7
Strategies For Enhancing Staff Engagement 1:1 Meetings Strategies For Enhancing Staff Engagement Communication Strategies For Enhancing Staff Engagement Traffic Light Check In 8
Best Practices for Outcome Measurement Shift the culture: Punitive Opportunity for Improvement Reactive Proactive It s our policy It s best practice Evidenced Based Practice drives the outcomes Front line staff are the KEY Creating Real-Time, Actionable Outcome Dashboards Blythedale's Trek Joint Commission preparation consultant identified: "Reassessment and staging of wounds not clearly documented" 9
Blythedale s Trek Action Plan: Revise Policy and Procedure Define wounds Distinguish between pressure ulcers and all other wounds Educate MD s and RN s Monitor process and outcomes Blythedale s Trek Follow-up tasks Accountable: Jill, Scott IT wound rounding tool and identify wounds in the EMR Build template for documentation by physicians Define process for consult by wound nurses Blythedale s Trek What are my problem areas? What are the outcomes that need help? What are the processes that lead to those outcomes? How can I measure them? How can I impact the outcomes before they show up on a scorecard? 10
Blythedale s Trek Structure = Wound Care Nurses Exist Process = Policy on Wound Management, Consults and Frequency Outcomes = % of patients with wounds seen by Wound Team within policy timeframe Blythedale s Trek Dashboard shows Outcome Can t intervene on things that already happened How to use real-time data to facilitate process and change outcomes before they even show up on the dashboard? Ask Wayne Gretzky Blythedale s Trek A great hockey player skates to where the puck is going to be. Wayne Gretzky 11
Roles and Responsibilities: IT and Clinicians Need for Partnership Clinicians and IT need to understand how to frame questions in a way they each will understand Clinicians come up with variables to study IT identifies where the data resides and how to access, manipulate and present the data Clinicians and IT work together to find best ways to present the data in order for clinicians to know what to do next Clinical Infrastructure EMR System Nurse Call System Medication Dispensing System Environmental Monitoring System Clinical Portable Devices Administrative systems: Finance, HR, Procurement Clinical Data Repositories Data Warehouse Visualization, Dashboards, Reports, Ad hoc query, Push\pull alerts Integration, Collaboration, Access Control Administrative Systems Knowledge Clinical Systems Information Data Data Warehouse 12
Actionable Outcomes Demonstration Take-Home Points Some Outcomes just have to be measured/reported Other Outcomes are more important operationally Clinicians are the best people to identify the steps in the processes that need studying IT are the best people to identify automated methods that produce timely information and improves outcomes Learning to understand each other is a critical step Iterative process Unless outcomes are actionable (and action is taken when needed), they are just numbers on a wall Your 100 Day Plan A strategy around laying the foundation towards using Actionable Data Inventory: Build an inventory of your data, beginning with your most frequently accessed and time-relevant data Identify: Look for any manual, time-consuming data curation processes for potential reengineering and automation using technology Detect: Locate the data silos within your organization (e.g., HR, finance, engineering), along with corresponding data needs that are currently unmet across the hospital. Combine: Simplify access to data. Create a strategy for standardizing data access via a data platform. 13
Your 100 Day Plan A strategy around laying the foundation towards using Actionable Data Prioritize: Select data and develop a road map for implementing to scale External sources: While building your platform, start looking outside your hospital for external data sources that can be incorporated to complement existing data and help lead to more complete insights. Pilot: Target a single hospital function. Once proven, each subsequent process of data will build upon this initial framework. Objectives Understand fundamental concepts of outcome design and measurement Learn how to identify key performance measures Learn techniques to create real-time, actionable outcome dashboards How to Contact Us Roni Amiel, CIO ramiel@blythedale.org Scott M. Klein, CMO sklein@blythedale.org John Settembrini, Clinical Manager jsettembrini@blythedale.org Jill Wegener, CNO jwegener@blythedale.org Blythedale Children s Hospital (914) 592-7555 14