Overview of the Hospital Safety Score September 24, 2013 Missy Danforth, Senior Director of Hospital Ratings, The Leapfrog Group
Presentation Overview Who is getting a Hospital Safety Score? Changes to the Scoring Methodology Since May 2013 New Measures Details of the data review process Important Dates Questions 2
What is the Hospital Safety Score? The Hospital Safety Score is an A, B, C, D, or F letter grade reflecting how safe hospitals are for patients. The Hospital Safety Score launched in June 2012. This October will be the fourth release. More information at www.hospitalsafetyscore.org 3
What Hospitals Will Receive a Hospital Safety Score? General, acute-care hospitals for which there is adequate public data Excluded Hospitals include: Specialty Hospitals (i.e. surgical centers, cancer hospitals, women s hospitals, etc.) Critical Access Hospitals Free Standing Pediatric Facilities Non-IPPS participating hospitals (hospitals from the state of MD) Hospitals Missing Too Much Data: More than 9 process measures More than 4 outcome measures 4
SCORING OVERVIEW 5
Measure Selection Criteria Measures are publicly-reported from national data sources, reflecting individual hospital results Leapfrog Hospital Survey Centers for Medicare and Medicaid Services data sets Measures are endorsed or in use by a national measurement entity Measures are linked to patient safety ( freedom from harm ) Directly quantifying patient safety events Assessing processes that lead to better outcomes Identified by experts as important to patient safety 6
Measures included in the Hospital Safety Score Measure Name Primary Data Source Secondary Data Source Process and Structural Measures (15) Computerized Physician Order Entry (CPOE) 2013 Leapfrog Hospital Survey* 2012 HIT Supplement I* ICU Physician Staffing (IPS) 2013 Leapfrog Hospital Survey* 2011 AHA Annual Survey i Safe Practice 1: Leadership Structures and Systems 2013 Leapfrog Hospital Survey* Safe Practice 2: Culture Measurement, Feedback and Intervention 2013 Leapfrog Hospital Survey* Safe Practice 3: Teamwork Training and Skill Building 2013 Leapfrog Hospital Survey* Safe Practice 4: Identification and Mitigation of Risks and Hazards 2013 Leapfrog Hospital Survey* Safe Practice 9: Nursing Workforce 2013 Leapfrog Hospital Survey* Safe Practice 17: Medication Reconciliation 2013 Leapfrog Hospital Survey* Safe Practice 19: Hand Hygiene 2013 Leapfrog Hospital Survey* Safe Practice 23: Care of the Ventilated Patient 2013 Leapfrog Hospital Survey* SCIP INF 1: Antibiotic within 1 Hour CMS Hospital Compare* SCIP INF 2: Antibiotic Selection CMS Hospital Compare* SCIP INF 3: Antibiotic Discontinued After 24 Hours CMS Hospital Compare* SCIP INF 9: Catheter Removal CMS Hospital Compare* SCIP VTE 2: VTE Prophylaxis CMS Hospital Compare* Outcome Measures (13) Foreign Object Retained CMS HACs* Air Embolism CMS HACs* Pressure Ulcer Stages 3 and 4 CMS HACs* Falls and Trauma CMS HACs* CLABSI 2013 Leapfrog Hospital Survey* CMS Hospital Compare* CAUTI 2013 Leapfrog Hospital Survey CMS Hospital Compare SSI: Colon CMS Hospital Compare PSI 4: Death Among Surgical Inpatients CMS AHRQ PSI PSI 6: Iatrogenic Pneumothorax CMS AHRQ PSI PSI 11: Postoperative Respiratory Failure CMS AHRQ PSI* PSI 12: Postoperative PE/DVT CMS AHRQ PSI PSI 14: Postoperative Wound Dehiscence CMS AHRQ PSI PSI 15: Accidental Puncture or Laceration CMS AHRQ PSI 7 [i] AHA Annual Survey 2011 Health Forum, LLC *Updated Since May 2013
Weighting Process Two (2) measure domains, each weighted 50%: 1. Process/structural measures 2. Outcome measures Three (3) criteria for weighting individual measures Strength of Evidence (rating of 1 or 2) Opportunity (rating of 1, 2, 3), based on coefficient of variation Impact (rating of 1, 2, or 3), based on no. of patients possibly affected by the event and severity of harm to individual patients Weight score: [Evidence + (Opportunity x Impact)] 8
Z-Score Methodology Standardizes data from individual measures with different scales Counts how many standard deviations a hospital s score on the measure is away from the mean Mean always equals 0; worse than mean = negative z-score ; better than mean = positive z-score Translate raw score on measure to z-score: Process/Structural Measures = [(Hospital Score Mean)/Standard Deviation] Outcome Measures = [(Mean Hospital Score)/Standard Deviation] 9
Overall Score Summation of z-score for each measure weight for each measure 3.0 + CPOE z-score CPOE weight + IPS z-score x IPS weight + CLABSI z-score CLABSI weight.... etc. If measure has missing data, then weight for that measure is re-apportioned to other measures within the same domain 3.0 was added to each hospital s final score to avoid possible confusion with interpreting negative patient safety scores 10
CHANGES TO THE SCORING METHODOLOGY SINCE MAY 2013 11
Two New Measures CAUTI Primary Data Source: Leapfrog Hospital Survey Secondary Data Source: CMS HAI Data Element: SIR SSI: Colon Primary Data Source: CMS HAI Data Element: SIR 12
DETAILS OF THE DATA REVIEW PROCESS 13
Secure Website http://www.leapfroggroup.org/data-validation/validationlogin Details on what public reports were used to obtain source data. Links to source data with instructions. Hospitals must confirm their Medicare Provider Number before moving on to the next page. 14
Source Data Instructions for hospital to review their source data and a link to the help desk For each measure, hospitals are provided with the measure name, type of measure, data source (primary or secondary), reporting period, measure score (i.e. points, rate, SIR, etc) Hospitals are asked to confirm that each Measure Score matches their score (i.e. rate, SIR, etc) from the Data Source (i.e. Leapfrog Hospital Survey Results, CMS Hospital Compare, etc) 15
Where to Locate Source Data Hospitals That Submitted a Leapfrog Hospital Survey by August 31, 2013 CPOE: Visit www.leapfroggroup.org/cp, find the column labeled prevent medication errors. Leapfrog score used to determine points towards safety score (i.e. 4 bars equals 100 points, 3 bars equals 50 points, etc). ICU Physician Staffing: Visit www.leapfroggroup.org/cp, find the column labeled Appropriate ICU Staffing. Leapfrog score used to determine points towards safety score. CLABSI and CAUTI Standardized Infection Ratio (SIR): Visit www.leapfroggroup.org/cp, find the column labeled prevent ICU infections. The SIR was used to calculate in the safety score calculation, and is located in the first sentence in the first paragraph above the table. 16
Where to find your hospital's CLABSI SIR on Leapfrog s results website 17
Where to Locate Source Data Hospitals That Did Not Submit a Leapfrog Hospital Survey by August 31, 2013 Safe Practice Points Log into your 2013 Leapfrog Hospital Survey 18
Where to find your hospital s CLABSI, CAUTI, SSI SIR on CMS results website 19
20
Where to Locate Source Data CMS Data for All Hospitals SCIP Measures: Hospital rates are published by CMS at https://data.medicare.gov/hospital-compare/hospital-process-of- Care-Measures-Surgical-Care-Im/iz5u-4mk2 HACs: Use the downloadable database on the Data Preview Website PSI 11: Use the downloadable database on the Data Preview Website Other PSIs: Hospital rates are published by CMS at https://data.medicare.gov/hospital-compare/agency-for- Healthcare-Research-And-Quality-Measure/vs3q-rxc5 You can download your hospital s results into a CSV or XLS formatted file. 21
Where to Locate Source Data AHA Data for Hospitals That Did Not Complete a Leapfrog Hospital Survey by August 31, 2013 ICU Physician Staffing - 2011 AHA Annual Survey CPOE 2012 HIT Supplement Please contact AHA Health Forum Please note: corrects will only be accepted to through October 7th 22
What if the Measure Score doesn t match the public report? Hospitals are asked to contact the help desk immediately once they have confirmed the measure and reporting period. Hospitals must provide a copy of the public report that shows a different score If we find a recording error, we will update the score and re-issue a numerical safety score 23
Preview Numerical Safety Score 24
Important Dates September 2013 - Letter sent to CEOs of hospitals receiving a Hospital Safety Score. Letter included: Information about the Hospital Safety Score A username/password to a secure website where hospitals can review the source data that Leapfrog used to calculate their numerical safety score Links to the Hospital Safety Score help desk and helpful documents September 19 October 7- Courtesy Data Preview Period Mid-October Hospitals will be able to preview letter grades 48 hours prior to the pubic release (www.hospitalsafetyscore.org). Late October Letter Grades will be published at www.hospitalsafetyscore.org. For more information about important dates, visit: http://www.hospitalsafetyscore.org/for-hospitals/updates-andtimelines-for-hospitals 25
More Information Hospital Safety Score Help Desk - scorehelp@leapfroggroup.org Hospital Safety Score Website www.hospitalsafetyscore.org Data Review Website - http://www.leapfroggroup.org/datavalidation/validation-login 26