Star Rating Method for Single and Composite Measures CheckPoint uses three-star ratings to enable consumers to more quickly and easily interpret information about hospital quality measures. Composite ratings combine individual measures into a single rating to summarize the overall quality of care for a specific clinical condition or quality topic. The method used for CheckPoint composite ratings combines the methods used by HealthPartners 1 and The Joint Commission 2. For composite measures the method uses the following four-step process. Single measure star ratings only use the first step. Determine whether each measure within the composite is statistically different from the Wisconsin average for that measure; Calculate the Quality Score for each measure, based on the statistical difference; Calculate the Composite Score for the group of measures assigned to the composite; and Assign a star rating to composite measures. 1) Determine statistical difference for each measure A confidence interval is calculated for each measure and then compared to the state average or a Target Range for that measure. a) Calculation of Confidence Intervals i) Confidence intervals will be calculated for measures with a denominator >25. ii) When a confidence interval is supplied with the data source, that confidence interval is used to determine statistical significance. This includes Centers for Medicare and Medicaid Services (CMS) mortality, readmission and infection measures. It also includes mortality and patient safety indicators calculated with Agency for Healthcare Research and Quality (AHRQ) software. iii) When a confidence interval does not already exist one is calculated using the Wilson Score Interval method. This method is more reliable than the Normal Approximation Interval method, particularly when sample sizes are small and/or the performance level is near 100 or zero. Wilson Score Interval Method Calculation b) Compare to State Average or Target Range If the confidence interval was supplied with the data or can be calculated, the individual hospital s confidence interval is compared to the state average for that measure. A target range is used when a confidence interval cannot be calculated. c) Classify the Difference Between Hospital Performance and the State Performance Compare confidence interval for hospital, on each measure, to the state average or target range and classify as No Different, Better Than or Worse Than the overall state performance. For single measure star ratings these classifications are converted to two, three and one stars respectively and calculations are complete. i) HCAHPS analysis is based on Always survey responses and utilizes the following logic and targets for comparing each survey domain, overall satisfaction and willingness to recommend 1 : (a) If the hospital score is >75 the hospital is Better Than other hospitals in the state.
(b) If the hospital score is >70 or <75 the hospital is No Different from other hospitals in the state. (c) If the hospital score is <70 the hospital is Worse Than other hospitals in the state. ii) For measures where lower performance is better, such as mortality, infections and readmissions, the following logic is used 2 : (a) If the state average for the measure overlaps a hospital s lower and upper confidence intervals the hospital is No Different from other hospitals in the state. (b) If the state average is entirely above a hospital s upper confidence interval the hospital is Better Than other hospitals in the state. If a hospital has a rate of zero, they will be scored as Better Than regardless of the sample size and confidence interval. (c) If the state average is entirely below a hospital s lower confidence interval the hospital is Worse Than other hospitals in the state. (d) Hospitals having no infections but too small a denominator for NHSH to compute a SIR or confidence interval will have zero infections reported instead of + and a rating of two stars, as there is no statistical basis to determine that they were different from the state average. iii) For measures where higher performance is better, such as process measures, the following logic is used 2 : (a) If the state average for the measure overlaps a hospital s lower and upper confidence intervals the hospital is No Different from other hospitals in the state. (b) If the state average is entirely below a hospital s lower confidence interval the hospital is Better Than other hospitals in the state. If a hospital has a rate of 100 they will be scored as Better Than regardless of their sample size or confidence interval. (c) If the state average is entirely above a hospital s upper confidence interval the hospital is Worse Than other hospitals in the state.
2) Calculate a Quality Score for Each Measure A Quality Score is assigned to each measure based on their statistical difference from the other hospitals in the state. a) Calculation of Raw Quality Score i) Measures that were Better Than other hospitals receive a score of 1.0 ii) Measures that were No Different from other hospitals receive a score of 0.5 iii) Measures that were Worse Than other hospitals receive a score of 0.0 Example: If Target Range = 85.2-96.5% Hospital Numerator Denominator Hospital Rate Statistical Difference Raw Quality Score A 24 25 95% No Different 0.5 B 42 61 69% Worse Than 0 C 99 99 100% Better Than 1.0 D 50 53 94.3% No Different 0.5 E 9 12 75% NA Sample is <25 NA F 20 20 100% Better Than 1.0 b) Weighting of Raw Score i) The weighted score for HCAHPS domains/questions equal the Raw Quality Score x 1.0. ii) The weighted score for process measures equal the Raw Quality Score x 1.0. Process measures are measures that indicate whether the appropriate care was provided. iii) The weighted score for outcome measures equal the Raw Quality Score x 1.5. Outcome measures are measures that reflect the patient s response to care, including mortality, readmissions and hospital acquired conditions. Example for Stroke: Raw Weighted X = Measure Quality Score Weight Quality Score STK-2 - Antithrombotics on discharge 1.0 1.0 1.0 STK-3 - Anticoag for atrial fib/flutter 0.5 1.0 0.5 STK-5 - Early antithrombotics 0.5 1.0 0.5
STK-6 Discharged on statins 0.5 1.0 0.5 STK-8 Patient education 0 1.0 0 STK-10 Assessed for rehab 1.0 1.0 1.0 Stroke mortality 0.5 1.5 0.75 Weighted Score 4.25 3) Calculate a Composite Score for a Group of Measures a) A hospital must participate in (authorize reporting of) all of the component measures within a composite measure to receive a score. If a hospital has chosen to not report data for any of the component measures they will be reported as DNR Did Not Report. b) Each composite will have three or more component measures, which are listed in Appendix A. c) The Patient Safety (PSI-90) composite measure is computed using AHRQ software employing a different method from the one described here. For further information visit http://www.qualityindicators.ahrq.gov/downloads/modules/psi/v50/techspecs/psi_90_patient_safety_for _Selected_Indicators.pdf. d) A hospital must have a denominator >25 for the measure to be included in the Composite Score. At least half of the measures must have valid denominators to calculate a Composite Score. If the hospital participates in all of the measures but does not meet the minimum sample size for at least half of the measures their composite will be reported as + Insufficient Sample Sizes to Calculate a Composite. e) The Composite Score is the sum of the weighted scores divided by the total possible weighted points. Example for Stroke: Raw Weighted X = Measure Quality Score Weight Quality Score STK-2 - Antithrombotics on discharge 1.0 1.0 1.0 STK-3 - Anticoag for atrial fib/flutter 0.5 1.0 0.5 STK-5 - Early antithrombotics 0.5 1.0 0.5 STK-6 Discharged on statins 0.5 1.0 0.5 STK-8 Patient education 0 1.0 0 STK-10 Assessed for rehab 1.0 1.0 1.0 Stroke mortality 0.5 1.5 0.75 Totals 7.5 4.25 Composite Score 0.57 (4.25/7.5) 4) Assign a Star Rating A three-star rating is assigned using the Composite Score calculated in step 3. a) Composite Scores <0.33 receive a one-star rating b) Composite Scores >0.33 and <0.67 receive a two-star rating c) Composite Scores >0.67 receive a three-star rating Exhibit 3: Composite Ratings 0 0.33 0.67 1.0
Example and Composite Key Weighted Quality Score Composite Rating Comments Hospital A 0.59 2 star Hospital B 0.85 3 star Hospital C 0.25 1 star Hospital D DNR Hospital did not participate in all measures Hospital E + Hospital participates in all measures but did not have a sample size of >25 in at least half of the measures Hospital G NA Hospital does not provide the services rated by this composite References: 1. HealthPartners ; Cost and Quality Assessment Documentation - https://www.healthpartners.com/ucm/groups/public/@hp/@public/documents/documents/cntrb_033165.pdf 2. The Joint Commission 2011 Health Care Professional Quality Report User Guide - http://www.qualitycheck.org/assets/hospital_prof_user_guide_june%202011.pdf Appendix A Composite Ratings and Component Measures Composite Measure Birth Hip and Knee Infections Mortality Component Measures Cesarean Section (PC-02) Early Elective Delivery (PC-01) Exclusive Breastfeeding (PC-05) Newborn Screening Turnaround Time Hip and Knee Complications Hip Fracture Mortality Hip Replacement Mortality Hip and Knee Replacement 30-day Readmission Abdominal Hysterectomy Surgical Site Infection (SSI) Catheter Associated Urinary Tract Infection (CAUTI) Central Line Associated Blood Stream Infection (CLABSI) Clostridium difficile Infection Colon Surgery Surgical Site Infection (SSI) Methicillin Resistant Staph Aureus Bacteremia (MRSA) Abdominal Aortic Aneurism (AAA) Repair Mortality Acute Stroke Mortality Carotid Endarterectomy (CEA) Mortality Coronary Angioplasty (PTCA) Mortality Coronary Bypass (CABG) Mortality Craniotomy Mortality Gastrointestinal Hemorrhage Mortality Heart Attack 30-day Mortality Heart Failure 30-day Mortality
Composite Measure Patient Safety Patient Satisfaction Readmissions Component Measures Hip Fracture Mortality Hip Replacement Mortality Pneumonia 30-day Mortality The AHRQ PSI-90 Patient Safety Composite is computed from: PSI-3 Pressure Ulcers PSI-6 Pneumothorax/Collapsed Lung PSI-7 Central Venous Catheter-Related Blood Stream Infection Rate PSI-8 Post-Op Hip Fracture PSI-12 Post-Op Blood Clot PSI-13 Post-Op Sepsis Rate PSI-14 Post-Op Abdominal Wound Complication PSI-15 Accidental Puncture or Cut Doctor Communication Hospital was Quiet at Night Nurse Communication Overall Satisfaction (Rated Hospital High) Pain Control Patient Room was Clean Patient Would Recommend Hospital Patients Received Help They Needed Patients Understood Their Care When They Left Staff Explained Medications Staff Provided Discharge Instructions All Cause Readmission COPD 30-Day Readmission Heart Attack 30-Day Readmission Heart Failure 30-Day Readmission Hip and Knee 30-Day Readmission Pneumonia 30-Day Readmission Stroke 30-Day Readmission