Dashboards, Data and Information Overload: Using ACS NSQIP as a Reality Check

Transcription

1 Dashboards, Data and Information Overload: Using ACS NSQIP as a Reality Check Mary Hawn MD MPH Professor and Chief, Division of Gastrointestinal Surgery Associate Quality Officer: Perioperative Services The University of Alabama at Birmingham

2 Why Dashboards? Public reporting of mortality, PSIs, HACs, readmissions. is the latest paradigm of quality measurement. 100% 90% 80% 70% 60% 50% 40% CMS has published the 30% plan for value based 20% purchasing (VBP) which 10% is upon us. 0% VBP by Year Process (SCIP) Experience( HCAHPS) Outcome( mort/psi) Efficiency($ /pt)

3 Dashboards for Surgical Quality Surgical Care Improvement Project University HealthSystem Consortium State Surgical Site Infection reporting Patient Safety Indicators Hospital Acquired Conditions Never Events Readmissions HCAPS CGCAPS And so on and so on..

4 Hospital Compare Data SCIP Core Measures UAB Natl Avg Top 10% ABX within 1hr ABX Selection ABX D/C time Card pts controll ed 6am Post Op Glucos e A ppropriate Hair Removal Patients w/ recommended VTE prophy l axis Patient who receiv ed appropri ate VTE prophylaxis

5

6 UAB Hospital Jan - Mar 2010 (Q1) Po st - Su r g ical Mo r t alit y ( Ex cl u d in g Tr au m a/ Bu r n ) n it io n - Po st - Su r g ical M o r t a lit y ( Ex clu d in g Tr au m a / Bu r n ) m ortality O/ E ratio for inpatient s age 18 and older undergoing any surgical procedure excluding t raum a, m aj o 265, , , , , , , , , , , , , , , , , , , , , , nt quart er. Bad data, nonviable neonates, organ harvest cases, and records with a null expected m ort alit y ar im e. Rel at iv e Per f o r m an ce Ob ser v ed Tar g et UHC Med ian Ran k nt Quarter / 109 t Year / 109 Cu r r en t Qu ar t er Last Qu ar t er Recen t Year ( denom.) 3,269 3,300 13,538 ved Deat hs ted Deaths ved/ Expected Rat io ved Mort ality(% ) ted Mortality ( % ) t (A) Po st - Su r g ical Mor t alit y ( Ex clu d i n g Tr au m a/ Bu r n ) 2008 Q Q Q Q42009 Q Q Q32009 Q Q1 Expected Mortality ( % ) Observed Mortality( % )

7 University of Alabama at Birmingham Hospital General/Vascular Report No PAGE 3 Site No /06-06/07 01/07-12/07 07/07-06/08 01/08-12/08 07/08-06/09 01/09-12/09 07/09-06/10 01/10-12/10 07/10-06/11 01/11-12/11 GV Mortality GV Morbidity H L 0.76 L 0.73 L GV Cardiac H 2.18 H GV Pneumonia 0.34 L 0.30 L 0.20 L H 2.09 H 1.54 H L 0.65 GV Unplanned intubation GV Ventilator > 48 hours GV DVT/PE L L GV Renal Failure H GV UTI GV SSI L GV ROR 0.91

8 What is the Truth? Process versus Outcome? SCIP versus SSI/VTE/MI Different data definitions NHSN versus CDC SSI Clinical versus Administrative data Risk adjustment HAC/PSI versus outcome

9 Surgical Care Improvement Project (SCIP) Implemented in 2006 by Centers for Medicare and Medicaid Services 3 Focus Areas: Surgical site infections Adverse cardiac events Venous thromboembolism (VTE) Reduce Surgical Complications by 25% in 2010 Bratzler DW, Hunt DR. Clin Infect Dis. Aug ;43(3):

10 Surgical Care Improvement Program 6 SCIP Infection Measures (at the time of assessment) Timely Antibiotic Administration Appropriate Coverage Timely Discontinuation Glucose control for cardiac patients Normothermia for colon cases Appropriate Hair removal Two VTE measures VTE Prophylaxis ordered VTE Prophylaxis received Cardiac Measure Beta blocker given within 24 hours prior to surgery

11 SCIP Measures: By definition, they re measureable Denominator specified Which surgical cases Exclusion Criteria Numerator is number of cases receiving measure Publically reported on Hospital Compare Are used for Value Based Purchasing 11

12 Association between Adherence and SSI SSI Unadjusted Adjusted SCIP Measure Met % OR 95% C.I. OR* 95% C.I. Timely Yes 5.0 No Appropriate Yes 5.3 No Discontinue Yes 5.6 No Hair Removal Yes 6.3 No Normothermia Yes 15.8 No Composite Yes 4.7 No *Adjusted for age, work RVU, operative time, specialty, diabetes, COPD, steroid use, ASA class, wound class, smoking status, dyspnea, alcohol, history of radiation therapy and gender

13 100% Hospital Infection Rate by Adherence 112 VA Hospitals R² = 0.03 Hospital Adherence (%) 80% 60% 40% 20% 0% 0% 2% 4% 6% 8% 10% 12% 14% 16% Infection Rate (%) * Bubble size represents

14 Adherence and Adjusted SSI Over Time 100% SCIP Adherence 80% 60% 40% 20% 0% Time Period (6 Month Intervals) Composite Timely Discontinue Appropriate Hair Removal Normothermia

15 100% Adherence and Adjusted SSI Over Time 25% SCIP Adherence 80% 60% 40% 20% 20% 15% 10% 5% Adjusted SSI Rate 0% 0% Time Period (6 Month Intervals) Composite Timely Discontinue Appropriate Hair Removal Normothermia Adjusted SSI Rate Hawn et al Ann Surg :

16 Association between Adherence and VTE: Patient Level Total Unadjusted Adjusted N 30,531 % OR 95% CI OR 95% CI c-index 89. VTE Adherence 27, *Adjusted for age, gender, BMI, smoking status, weight loss, emergent case status, operative time Composite SCIP-VTE adherence was not associated with VTE outcomes even after adjusting for patient factors Altomet al Am J Surg :591-7

17 Association between Adherence and VTE: Hospital Level Altomet al Am J Surg :591-7

18 Association between Adherence and VTE: Secular Trends Altomet al Am J Surg :591-7

19 Richman et al JAMA Surgery in press Whole Cohort

20 Conclusion SCIP Performance Has Little Correlation with Outcomes

21 What about Patient Safety Indicators and Hospital Acquired Conditions? UAB Hospital 12/2012 through 8/ ,899 pts admitted to or discharged from DOS 552 total adverse outcomes in 448 patients 235 deaths (2.0%) 253 PSI (at least one) 26 HAC (at least one)

22 PSI by Preventability and Reason Table 3a: PSI Outcomes by Admission Type and Preventability Overall Not Preventable Possibly Preventable Preventable N (%) N (%) N (%) N (%) p-value Overall (30.7) 171 (59.0) 30 (10.3) Admission Type Emergent 107 (36.9) 39 (43.8) 58 (33.9) 10 (33.3) 0.47 Urgent 26 (9.0) 9 (10.1) 15 (8.8) 2 (6.7) Elective 157 (54.1) 41 (46.1) 98 (57.3) 18 (60.0) % of total PSI n = Respiratory 27 DVT/PTE 27 Hemorrhage 23 Puncture 27 Escalation Protocol Preop Opt Judgment Supervision Technical

23 Do Patient Safety Indicators Correlate with Clinical Outcomes? 10 AHRQ PSI Post-op Physiologic and Metabolic Derangement 11 Post-op Respiratory Failure 12 Post-op PulmonaryEmbolism (PE)/ Deep Vein Thrombosis (DVT) VASQIP Surgical AE Actual Renal Failure -Failure to Wean - Reintubation -PE -DVT 13 Post-op Sepsis Systemic Sepsis 14 Post-op Wound Dehiscence Dehiscence

24 Data Sources VA Patient Treatment File (PTF) VASQIP Database Data Element Source Data Element Source Demographic information (age, sex); Diagnostic information (ICD-9 CM); Summary information on episode of care (dates, setting of care, DRG, discharge status) PTF-Main Sex, principal disease and comorbidities Surgical Clinical Nurse Reviewer (SCNR) collected pre-operative data One primary and up to four secondary bedsectiondiagnoses, DRG and length of stay information for each bedsection stay PTF- Bedsection Emergency operations, principal anesthesia technique, intra operative blood loss SCNR collected intra operative data ICD-9-CM procedure codes, date, time, and site of all procedures during the inpatient stay PTF- Procedure Systemic sepsis, pneumonia, unplanned intubation, pulmonary embolism SCNR collected post operative data ICD-9-CM surgery codes and surgical specialty for each hospitalization PTF-Surgery 24 selected adverse events including morbidity/mortality SCNR collected 30 days post surgery data

25 Example: Respiratory Failure Example of PSI#11 Events eligible for PSI #11, Respiratory Failure N=109,916 VASQIP AEs N = 2,151 surgeries VASQIP only n=683 Overlap n=1,468 PSI AEs N = 2,047 surgeries PSI only n=579 Comparingthe Chart Chart Reviewed Reviewed PSI VASQIPonly VASQIP gold Casesstandard PSI-only to the Differences in in AE A 13 (65%) Sensitivity definition -PSI codes 68.3% Specificity PSI for reintubationhave false negative 6 (30%) 1 (7%) 99.4% different timing criteria Unableto Positive than VASQIP determine without predictive value input Unableto from determine VASQIP SCNR without input from VASQIP SCNR 71.7% 1 (5%) 14 (93%)

26 Results PSI Validity Comparison Group Results for PSI version 4.1a, FY03-07 Data PSI VASQIP AE #10 PMD ARF #11 RF #12 PE/DVT #13 Sepsis SS #14 WD Dehiscence Cases eligible for PSI(%)* 131,711 (49%) FW R/UI 109,916 (41%) PE 253,090 DVT (95%) 47,523 (18%) 74,251 (28%) VASQIP- Detected AEs (%)** 442 (0.3%) 2,151 (2.0%) 1,588 (0.6%) 570 (1.2%) 1,254 (1.7%) *Percent of the PTF-VASQIP matched sample of 268,771 hospitalizations. **Percent of the hospitalizations eligible for the corresponding PSI. PSI- Detected AEs (%)** 322 (0.2%) 2,047 (1.9%) 3,323 (1.3%) 397 (0.8%) 544 (0.7%) PSI Sensitivity (95% CI) 48% (44-53%) 68% (66-70%) 65% (63-67%) 31% (27-35%) 31% (29-34%) PSI PPV (95% CI) 66% (61-72%) 72% (70-74%) 31% (30-33%) 44% (39-49%) 72% (68-76%)

27 Opportunities for Improvement: PSI Hemorrhage (9): Need more data: balance between the enemy of good is perfect, anticoagulation to prevent DVT and never leaving the OR Respiratory failure (11): Escalation: (previous slide conclusions) Documentation: Elective extubation within 48 hours of operation NOT a PSI DVT/PTE (12): Protocol: need to work together on high risk services to balance bleeding vs anti-coagulation Protocol: need to ensure that each patient is on appropriate prophylaxis Accidental Puncture/Perforation (15): Education: rules for documentation of inherent conditions and therefore NOT a PSI

28 Conclusions PSI and Outcomes Measures Have Little Overlap

29 UAB O and E Mortality by Quarter Expected Observed Q3CY12 Q4CY12 Q1CY13 Q2CY13 Q3CY13 Q4CY13

30 UHC Mortality Ratios UHC data based on discharge service only Many surgical services transferred patients to other services within the hospital where they died. palliative care MICU Unclear the contribution of the surgical services to the patients morbidity or mortality Reviewing mortality based on whether a patient was admitted to or discharged from a surgical service captures all aspects of surgical care.

31 Understanding the O/E ratio Observed Deaths 10 patients Expected Death O (actual number of people who died) E (Each ptassigned a risk of dying b/w 0 and 1 based on coding) Sum all 10 ptsrisk to create total expected risk of dying (denominator) 3 E If E > 3 (i.e. 3.5) that means lesspeople died than expected (Good News) If E < 3 (i.e. 2.5) that means morepeople died than expected (Bad News)

32 DOS OE Mortality Ratio by Quarter Education M&M change POA Statements More CDS Quality Restructure POA App OE ratio p<0.05 Q3CY12 Q4CY12 Q1CY13 Q2CY13 Q3CY13 Q4CY13

33 UHC IQI90 %ilerank by Quarter 0 Q3CY12 Q4CY12 Q1CY13 Q2CY13 Q3CY IQI

34 Accurate documentation with service specific present on admission (POA) statements (increased E ) We have compiled the top 10 DRGs for each service for the FY From this we have extracted the important drivers of the UHC model for determining the individual patients mortality risk by DRG. Each category has language that the coders need to appropriately code.

35 Conclusions Surgeon led, systematic quality improvement efforts improve patient care and documentation. These efforts positively effect our external measures of quality. Significant leadership support, political will and perseverance are critical. this is not going away Teamwork with the physicians, administration, data resources and the coding team are essential.

36 Understanding the differences among the various measures will help direct QI efforts ACS NSQIP provides the gold standard to double check your other measures against.

37 Lack of Cohesive Snapshot of your Hospital? Disparate surgical populations Different definitions of outcomes Different assessment/screening Coding versus clinical data Time point of measure Index hospitalization