CONSORTIUM LOIRE-ATLANTIQUE AQUITAINE RHÔNE-ALPES POUR LA PRODUCTION D INDICATEURS EN SANTE TE Conditions of Use & Reporting Methods of Patient Safety Indicators in OECD Countries State of knowledge Dr M.A. Le Pogam, MD, MPH Consortium CLARTE Pôle IMER, Hospices Civils de Lyon HCQI PSI SUBGROUP MEETING 9 MAY 2012 1
Context (1) French Ministry of Health (DGOS) and National Authority for Health (HAS) program of Development and Validation of French Health Care (Hospital) Quality Indicators Endorsement by the Steering Committee for National Implementation of Hospital Care Quality Indicators Integration into the National Quality Improvement Framework Mandatory public reporting Comparative benchmarking Hospital accreditation P4P ( future project) HCQI PSI SUBGROUP MEETING 9 MAY 2012 2
Context (2) CLARTE (Consortium Loire-Atlantique-Aquitaine-Rhône-Alpes pour la production d indicateurs de qualité en sante) Human Resource Management (HRM) Turn over and absenteeism of non-medical staff Medical and non-medical staff satisfaction Patient Safety (PS) Patient Safety Culture Patient Safety indicators French PSIs : Set of AHRQ PSIs Adapted from ICD-9-CM to ICD-10 by IMeCCHI (International Methodology Consortium for Coded Health Information) Adapted to French coding methodology and procedure classification (CCAM) Refined and validated HCQI PSI SUBGROUP MEETING 9 MAY 2012 3
Context (3) Report on Conditions of Use & Reporting Methods of Patient Safety Indicators in OECD Countries» between 2001 and 2010 Systematic review of PSI literature published between 2001 and 2010 Semi-structured interviews and grey literature review Guidelines on Conditions of selection, adaptation and refinement of French PSIs Methods of estimation, validation and reporting of PSI rates at hospital- and area-level Institutional uses of PSIs for hospital care security assessment and enhancement HCQI PSI SUBGROUP MEETING 9 MAY 2012 4
CONSORTIUM LOIRE-ATLANTIQUE AQUITAINE RHÔNE-ALPES POUR LA PRODUCTION D INDICATEURS EN SANTE TE Conditions of Use & Reporting Methods of Patient Safety Indicators in OECD Countries Qualitative approach Le PogamM.A, Januel J.M, Colin C. Consortium CLARTE HCQI PSI SUBGROUP MEETING 9 MAY 2012 5
Method (1) Semi-structured telephone interviews with 10 international experts in the field of PSIs Countries Organizations Experts Canada Canadian Institute for Health Information / IMECCHI Dr Chantal Couris Canada University of Calgary / IMECCHI Pr William Ghali Germany University of Niederrhein / IMECCHI / OECD HCQI Pr Saskia Drösler USA University of California (UC Davis) / IMECCHI / OECD HCQI Pr Patrick Romano Belgium Federal Public Service (FPS) - Public Health / OECD HCQI Dr Margareta Haelterman United Kingdom Imperial College London - School of public Health / Dr Foster Unit Dr Paul Aylin United Kingdom Imperial College London - School of public Health / Dr Foster Unit Pr Alex Bottle Switzerland University of Lausanne - Institute of Social and Preventive Medicine / IMECCHI Pr Bernard Burnand OECD OECD HCQI Pr Niek Klazinga OECD OECD HCQI Pr Vladimir Stevanovic EU European Commission - DG Health and Consumers - Healthcare Systems Unit Dr Agnieszka Daval-Cichon HCQI PSI SUBGROUP MEETING 9 MAY 2012 6
Method (2) Questionnaire Historical development of PSIs Routinely used PSIs PSIs selection, adaptation, validation and reporting processes Stakeholder uses of PSIs for patient safety improvement in hospital Grey literature Selected government and patient safety organization websites, journal articles and reports, technical and working papers HCQI PSI SUBGROUP MEETING 9 MAY 2012 7
Results (Updated) Canada (1) PSIs routinely used in Canadian Hospital Reporting Project (CHRP) In-Hospital Hip Fracture in Elderly (65+) Patients (rate per 1000 discharges) modified PSI#8 Obstetric Trauma - Vaginal Delivery with Instrument (rate per 100 instrumentassisted vaginal deliveries ) PSI#18 Obstetric Trauma - Vaginal Delivery without Instrument (rate per 100 unassisted vaginal deliveries ) PSI#19 Nursing-Sensitive Adverse Events for Medical Patients (per 1,000 medical discharges) Urinary tract infections, Pressure Ulcers (PSI#3), In-hospital fractures, Pneumonia Nursing-Sensitive Adverse Events for Surgical Patients (per 1,000 surgical discharges) Urinary tract infections, Pressure Ulcers (PSI#3), In-hospital fractures, Pneumonia No Pediatric Safety Indicator (AHRQ PDI) HCQI PSI SUBGROUP MEETING 9 MAY 2012 8
Results (Updated) Canada (2) Results reporting Hospital and peer group reports Risk-adjusted rates (logistic or Poisson regression model) and 95%CI limits Hospital level Health administration region level Provincial/territorial level Risk factors Age, gender and selected pre-admit comorbid diagnoses Bullet and trend graphs HCQI PSI SUBGROUP MEETING 9 MAY 2012 9
Results (Updated) Canada (3) Validation work Assessment of health administrative data quality Development of Internal Consistency Tools based on logic checks Influence of coder characteristics Effect of physician alternate payment plans Assessment of chart documentation quality Impact on administrative data quality Criterion validation studies : medical record reviews as «gold standard» PSI#5, #7, #12, #13, #15 Effect assessment of Canadian coding standards on PPVs (diagnosis type) Construct validation studies LOS, Re-admission rates Validation of a risk-adjusted model for national comparisons Review of the differences across country-specific ICD-10 clinical modifications Effect on the international comparability of morbidity data International cross-validation of ICD-10 coding algorithms PSIs, Charlson index, Elixhauser index HCQI PSI SUBGROUP MEETING 9 MAY 2012 10
Results (Updated) Germany (1) PSIs not used yet for mandatory external quality assurance (AQUA-Institut quality reports and hospital benchmark reports ) 3 modified patient safety indicators based on clinical data modified PSI#3 Pressure ulcer grade I-IV in older hospitalized patients (age 75 years, LOS 5 days, PU POA or acquired after admission) Pressure ulcer grade II-IV in older hospitalized patients (age 75 years, LOS 5 days, PU POA or acquired after admission) modified PSI#12 Postoperative DVT (selected surgical procedures) Postoperative PE (selected surgical procedures) modified PSI#9 Wound hematomas / postoperative bleeds (selected surgical procedures) HCQI PSI SUBGROUP MEETING 9 MAY 2012 11
Results (Updated) Germany (2) German health care policy (Hospital Remuneration Law, KHEntgG ) plans to produce national quality assurance indicators based on hospital inpatient administrative data by 2014 PSI #2 #19 adapted by 3M* company to German coding system Hospital care security self assessment and management PSI#1 complications of anesthesia : not coded PSI#20 Obstetric trauma cesarean section : clinically irrelevant HCQI PSI SUBGROUP MEETING 9 MAY 2012 12
Results (Updated) Germany (3) Results reporting Case-based (patients) and Hospital-based aggregate results Risk-adjusted rate (95% CI), median, range Time trend Between hospitals comparison Comparison with a reference range (target range, tolerance range) Risk-adjusted rates (95%CI) Stratification Logistic or Poisson regression model Risk factors Age, gender, selected pre-admit comorbid diagnoses, health related behaviours, socioeconomic factors Sentinel indicators HCQI PSI SUBGROUP MEETING 9 MAY 2012 13
Results (Updated) Germany (4) Validation work Public reporting : multi-stage assessment procedure Modified Delphi method Statistical test Recommendations for public reporting Criterion validation studies Medical record reviews Cross-validation with extensive hospital databases Construct validation studies Validation of a risk-adjusted model for national comparisons Validation of a risk-adjusted model for international comparisons Stratification on age, gender, number of SDx, LOS Test-retest reliability of PSIs Impact of national coding standards on the variability of PSIs Country-specific ICD versions PDx definition POA flag for SDx Admission type HCQI PSI SUBGROUP MEETING 9 MAY 2012 14
Results (2011) USA (1) PSIs routinely used in USA Hospital individual measures : PSI#2 - PSI#20 Modified PSI#3 Pressure Ulcer stage 3 or 4 Modified PSI#4 Death among surgical inpatients with serious treatable complications Modified PSI#7 Central venous catheter-related bloodstream infections Modified PSI#15 Accidental puncture or laceration Hospital composite measure : PSI#90 Patient safety for selected indicators Composite of PSI#3, PSI#6, PSI#7, PSI#8, PSI#9, PSI#10, PSI#11, PSI#12, PSI#13, PSI#14, PSI#15 Area level : PSI#21 PSI#27 Analogs of PSI#5, PSI#6, PSI#7, PSI#9, PSI#14, PSI#15, PSI#16 NQF-endorsed PSIs PSI#2, PSI#4, PSI#5, PSI#6, PSI#11, PSI#12, PSI#14, PSI#15, PSI#16, PSI#17 PSI#90 PSIs adopted for CMS RHQDAPU PSI#90, PSI#4, PSI#6, PSI#14, PSI#15 HCQI PSI SUBGROUP MEETING 9 MAY 2012 15
Results (2011) USA (2) Results reporting Individual measures Provider observed rates Provider stratified observed rates (age, gender, payer, user-defined patient characteristics) Provider expected rates (risk-adjusted rates) Time trends Comparison with a reference population rate Risk-adjusted rates Stratification Logistic or Poisson regression model Risk factors Age, gender, DRG and comorbidity categories HCQI PSI SUBGROUP MEETING 9 MAY 2012 16
Results (2011) USA (3) Validation work PSIs Identification, selection and evaluation : multi-stage assessment procedure Step 1 : Concepts and Evaluation Framework Step 2 : Literature review to identify potential PSIs Step 3 : Candidate list of PSIs Step 4 : Structured review of each PSI (face validation using modified Delphi method) Step 5 : Empirical analysis Many criterion validation studies (PPV, Se, Sp) Medical record reviews Clinical data from VHA, CMS, CDC Volontary AEs reporting IHI Global Trigger Tool Construct validation studies LOS, mortality rates, charges Validation of a risk-adjusted model for national comparisons Complex sampling method allowing sensitivity estimation Risk-adjustment method incorporating data on Present on Admission (POA) HCQI PSI SUBGROUP MEETING 9 MAY 2012 17
Results (2010) UK (1) PSIs routinely used in United-Kingdom (Dr Foster Hospital Guide) Hospital individual measures PSI#2, PSI#7 - PSI#15, PSI#18, PSI#19 Modified PSI#2 Deaths in low-risk conditions Modified PSI#7 Central venous catheter-related bloodstream infections Modified PSI#15 Accidental puncture or laceration Hospital composite measure Composite of PSI#3, PSI#4, PSI#9, PSI#11, PSI#13, PSI#15 HCQI PSI SUBGROUP MEETING 9 MAY 2012 18
Results (2010) UK (2) Results reporting Individual measures Crude rates Expected rates (risk-adjusted rates) Mean number of SDx coded ( coding depth ) Real time monitoring tool (CUSUM control charts) Comparison with national average rate and 99,8%CI (banding) Funnel plots Risk-adjusted rates Logistic or Poisson regression model Risk factors Age, gender HCQI PSI SUBGROUP MEETING 9 MAY 2012 19
Results (2010) UK (3) Validation work Hospital Guide Consultation (Dr Foster Intelligence) Criterion validation studies (PPV, Se) medical record reviews Construct validation studies LOS, mortality rates, non-elective hospital re-admission rates Validation of a risk-adjusted model for national comparisons 2012 : Campaign for a POA flag (Dr Foster Intelligence) Impact of POA flag on criterion validity and on trust rates HCQI PSI SUBGROUP MEETING 9 MAY 2012 20
Results (2010) Belgium (1) PSIs routinely used by the Belgian Ministry of Health (SPF-SPSCAE) PSI#1 - PSI#20 Modified PSI#4.1 Failure to rescue acute kidney failure Modified PSI#4.2 Failure to rescue sepsis Modified PSI#4.3 Failure to rescue DVT/PE Modified PSI#4.4 Failure to rescue cardiac shock or arrest Modified PSI#4.5 Failure to rescue pneumonia Modified PSI#4.6 Failure to rescue digestive hemorrhage or acute gastric ulcer HCQI PSI SUBGROUP MEETING 9 MAY 2012 21
Results (2010) Belgium (2) Results reporting Rate-based indicators : PSI#3, PSI#4, PSI#11, PSI#12, PSI#13, PSI#15 Hospital observed rates National observed rates (benchmarks) Sentinel indicators AEs lists Validation work Criterion validation study (PPV, NPV) : PSI#3, PSI#12, PSI#13 medical record review HCQI PSI SUBGROUP MEETING 9 MAY 2012 22
Variability across countries Choice Specifications Risk-adjustment Conclusion PSIs can be routinely used for national comparisons or public reporting after: A standardized selection procedure Administrative data quality assessment Chart documentation quality assessment Criterion and construct validation studies Implementing diagnosis and procedure classifications modifications Validation of a risk-adjusted model Coding standards should be modified PDx POA HCQI PSI SUBGROUP MEETING 9 MAY 2012 23
Thank you for your attention www.clarte-sante.fr