PATH: Preview of indicators A-L. Guisset World Health Organization regional office for Europe agu@euro.who.int
Preview of indicators Rationale, generic definition Results and lessons learnt from PATH-pilot and PATH-II International organizations, partners Key issues for data collection Assess relevance/interest, burden of data collection (+ when alternatives: admin. database vs. ad-hoc data collection?) Recommend priorities
Preview of indicators Balance Dimensions Source data : Prospective retrospective; Databases ad-hoc data collection (audits, surveys) Structure process- outcomes International local relevance
Priorities If you had to select only 1(or 3) indicators in each dimension? If you had to exclude 1 (or 3) indicators in each dimension? In what additional dimensions (sub-dimensions or domains of care) would you suggest developing indicators?
Structure of the descriptive sheets Descriptive sheet (2 pages) Definition WHAT it means (what is really being measured, how to position it in relation to the comprehensive picture of organizational performance) Rationale: WHY measure this indicator (importance: prevalence, burden, potential impact; validity) Interpretation International reference points Dissemination of results: Who is to look at the data and what questions to ask? (checklist) Action what s next? to go further : Reference to networks or audit/improvement tools
Structure of the descriptive sheets Data collection procedure: Algorithm, step by step, audit tools or questionnaires if relevant, instructions for translation or adaptation to local context if relevant (what data, where, how to get organized for data collection in the hospitals, minimum number of cases, inclusion and exclusion, computation of indicator, test for data quality or cleaning data sets, etc.) Signature of expert or partner organization ( seal ) and contact details and developer or user of the original indicator (e.g. AHRQ for c-section)
PATH-II: discussion of results Sample: number of participating hospitals on selected indicators Indicator # countries Potential # hospitals # participating hospitals Mortality 8 155 12 LOS 7 154 116 C-section 8 155 84 Prophylactic antibiotic 6 11 78 Needle injuries 8 154 67 CTM3 (discharge preparation) 5 55 47
PATH 9 Indicators Clinical Effectiveness Safety - Utilisation - Indicators derived from Hospital Patient Administrative Databases ESQH-Office for Quality Indicators DK-National Indicator Project
Indicators C-section Rate In-Hospital Case fatality rate Myocardial Infarction In Hospital Case fatality rate Stroke Postoperative Pulmonary Embolism or Deep Vein Thrombosis ESQH-Office for Quality Indicators DK-National Indicator Project
Source of Indicators technical information and rationale OECD (Heath at a Glance, Technical Manuals www.oecd.org) AHRQ Quality / Patient Safety Indicators www.qualityindicators.ahrq.gov/ PATH Data Specification Manual (Coming Soon) ESQH-Office for Quality Indicators DK-National Indicator Project
Source of Data National Patient Registries (Scandinavia) National/Regional Billing Databases DRG registries Hospital Administrative Databases Various levels of data quality e.g.: - Unique patient identifier - Verification of sources ESQH-Office for Quality Indicators DK-National Indicator Project
Minimum Information content requirements Coded primary and secondary diagnoses ICD9, ICD1 Coded Interventions (operative procedures) ICD9, Other Systems Age/Sex of Patient Date of Admission/Discharge In-Hospital Death ESQH-Office for Quality Indicators DK-National Indicator Project
Cesarean-section Rate Rationale: Utilization of Healthcare (Significant between-country and within-country variation, quality/cost considerations) Effectiveness? Definition/Inclusion Criteria: Number of C- sections/1 deliveries (Specified exclusion criteria)
PATH-II: discussion of results C-section delivery rate Definition TITLE : % of caesarean sections of total deliveries NUMERATOR: number of Caesarean sections (C-sections) DENOMINATOR: All deliveries EXCLUSION CRITERIA: exclude patient with abnormal presentation, preterm, foetal death, multiple gestation, breech procedure, delivery within 37 weeks or less of pregnancy (AHRQ definition, focus on low risk deliveries, for increase homogeneity of patient population) TAILORED number of primary C-sections over number of primary deliveries vaginal deliveries over all deliveries with a previous caesarean section http://www.qualityindicators.ahrq.gov/downloads/iqi/iqi_guide_v31. pdf
PATH-II: discussion of results C-section delivery rate International comparison on average c-section rate within country 1 9 8 7 6 5 4 3 2 1 country 1 country 2 country 3 country 4 country 5 AHRQ_US
PATH-II: discussion of results C-section delivery rate International comparison on average c-section rate within country (min, 1 st quartile, mediane, 3 rd quartile, max) 1. 8. 67.6 6. 55.4 4. 2.. 32.5 3.3 33.1 28. 22.8 21.4 19.7 21.8 16.6 1.6 15. 12.3 18.4 12.3 17.4 1.9 9.7 8.2 country 1 country 2 country 3 country 4 country 5
PATH 9: discussion issues C-section delivery rate Inclusion criteria? (All deliveries or defined subgroups) Interpretation Applications? (trends, peer comparisons, possibility of defining targets for good practice?)
PATH-II: discussion of results Compliance with antibioprophylaxis guidelines Definition % of patients who received prophylactic antibiotic according to local guidelines NUMERATOR : Patients that received the antibiotic DENOMINATOR : Patients that should have received antibiotics TRACER PROCEDURES : planned surgery for colorectal cancer, coronary artery bypass graft (CABG), hip replacement, and hysterectomy TAILORED : 1) patients whose prophylactic antibiotics was initiated within 1 hour of incision, 2) patients whose prophylactic antibiotics were discontnued within 24 h after surgery end time
C2 Compliance with antibioprophylaxis PATH-II: discussion of results guidelines Compliance with antibioprophylaxis guidelines Bottle 3/4th full or 1/4th empty? 1% 8% 6% 4% tot other under over OK 2% %
PATH-II: discussion of results Compliance with antibioprophylaxis guidelines Inter-hospital variations Inter- and within-country distribution of % of patients receiving antibioprophylaxis in compliance with local guidelines for hysterectomy (minimum, 1st quartile, 3rd quartile, maximum) 1 8 1 1 1 1 1 91,8 88,3 86,4 1 79,1 6 73,8 59,3 4 38,3 2 24,5 country 1 country 2 country 3 country 4
PATH-II: discussion of results C2 Compliance with antibioprophylaxis guidelines Compliance with antibioprophylaxis guidelines Data collection Who assessed compliance? Compliance was assessed against what guidelines? Local? National? Content (molecule, doses, timing)? How were records identified? Best practices Who is responsible for developing guidelines? Reviewing them? For communication? For monitoring cmpliance? For setting up structure to ensure proper timing? Interpretation What is your rate of compliance? Did results come as a surprise or were they expected? How do you relate those results to post-surgical infection rates? What goals do you set up? Impact To whom were the results presented? How was awareness raised? Was it assessed again? Is it part of routine (now)? Next steps?
PATH 9: discussion issues C2 Compliance with antibioprophylaxis guidelines Compliance with antibioprophylaxis guidelines Tracers Number of records to be audited per tracer? Audits performed locally or centrally? ToR for auditor? Test reliability? Compare against local guidelines, national guidelines, international guidelines? Provide a tool to assess local and national guidelines? Include elements that need to be included in the guideline (timing before/after, dose, type, exclusion criteria, etc.) provide standard algorithm as illustration How to facilitate comparisons of national guidelines before implementation of indicators? How much time is needed between Whom to involve for local development, measurement and interpretation of this indicator?
PATH-II: discussion of results C4 Readmission within 3 days Definition Numerator: Total number of unplanned admissions within a fixed follow up period (3 days) from the same hospital and with a readmission diagnosis relevant to the initial care. Denominator: Total number of patients admitted for selected tracer conditions Exclusion criteria: Patient who died during the index hospitalization or who were discharged to another acute care hospital
PATH-II: discussion of results C4 Readmission within 3 days Global readmission rates (in %) per country and tracer 12 1 8 6 4 country 1 country 2 country 3 country 4 2 AMI Stroke Community acquired pneumo Hip fracture
PATH-II: discussion of results 1 5 Stroke country 1 country 2 country 3 country 4 Global C4 Readmission within 3 days 45-64 65-79 8-89 9 and more 15 Community acquired pneumonia 1 5 country 1 country 2 country 3 country 4 Global 45-64 65-79 8-89 9 and more
PATH-II: discussion of results C4 Readmission within 3 days AMI 25 25 Stroke 2 14.88 15 13.61 1.26 1 7.75 8.4 5 4.17 2.59 4.36 3.85 2.86.93.49 country 1 country 2 country 3 country 4 Community acquired pneumonia 25 23.26 2 16.11 15.38 15 12. 1 7.22 5.64 5.17 5 4.17 2.1 1.9 1.91 1.46 country 1 country 2 country 3 country 4 Hip fracture 25 2 17.54 16.94 16.55 2 16.67 17.24 15 1 5 12.39 1.2 7.35 7.17 4.78 4. 1.21.33 country 1 country 2 country 3 country 4 15 1 5 11.76 9.71 7.72 6.6 4.62 4.86.98 1.96 country 1 country 2 country 3 country 4
PATH 9: discussion issues C2 Compliance with antibioprophylaxis guidelines Readmissions within 3 days Tracers (see OECD) Unique identifier? Alternative 2 includes algorithm to review records Unplanned? Avoidable? Agregation of tracers into summary indicator?
PATH-II: discussion of results C8 Median length of stay Definition This indicator assesses the median number of days of hospitalization (admission and discharge date count for one day) for cases admitted with acute myocardial infarction (ICD-9: 431, 433, 434, 436 and ICD-1: I63, I64, I65, I66). Data collected over a 12 months time period from the 1 st January to 31 st December 26 (unless this data was not available then the most recent data covering a 12 months period) Patients transferred to/from other hospitals were excluded. The reported data is NOT adjusted for age and sex.
PATH-II: discussion of results C8 Median length of stay Tracer 1: Acute Myocadial Infraction 3 Tracer 3: Community acquired pneumonia 3 2 2 1 1 country 1 country 2 country 3 country 4 country 5 Global country 1 country 2 country 3 country 4 country 5 Global
PATH-II: discussion of results 3 C8 Median length of stay LOS and age? ---- No risk adjustement Tracer 1: Acute Myocadial Infraction 3 Tracer 3: Community acquired pneumonia 2 country 1 country 2 country 3 country 4 country 5 Global 2 country 1 country 2 country 3 country 4 country 5 Global 1 1 45-64 65-79 8-89 9 and more 45-64 65-79 8-89 9 and more
PATH-II: discussion of results C8 Median length of stay Tracer 1: Acute Myocadial Infraction Tracer 3: Community acquired pneumonia 3 3 21.8 2 1 18. 15.4 14.3 13.5 12.8 11.9 11.17 1.6 9.9 9.15 9.9 9.6 8.4 7.44 8.5 7.3 7.1 7. 5.5 country 1 country 2 country 3 country 4 country 5 2 1 15.6 15.7 15.1 11.41 11.2 11.3 9.7 8.9 8.4 7.64 7.6 9.1 8 5 4.86 3.6 country 1 country 2 country 3 country 4 country 5
PATH 9: discussion issues C2 Compliance with antibioprophylaxis guidelines Median length of stay Tracers (see OECD) Alternative 2: risk adjustement? DRGs? Agregation of tracers into summary indicator?
Myocardial Indfarction/Stroke within hospital 3 days case fatality rates Rationale: Effectiveness safety (Outcome measure associated with evidence-based practice) Definition: Denominator Number of deaths (Age +15) in the same hospital that occurred within 3 d of admission Numerator No admission (Age +15) to hospitals with a primary diagnosis of Stroke/Myocardial Infarction
Myocardial Indfarction/Stroke within hospital 3 days case fatality rates Definition ctd: Specified ICD- codes
Myocardial Indfarction/Stroke within hospital 3 days case fatality rates Discussion issues: Risk Adjustment? (Need for additional data collection?) Interpretation? (Possibility of Benchmarking) Internal improvement activity (Utility as a trigger for audit-improvement of key processes?)
Postoperative Pulmonary embolism or Deep Vein Thrombosis: NEW INDICATOR Rationale: Patient Safety effectiveness (occurence of DVT/Pulmonary embolism is one of the major potentially lethal and preventable - complications to surgery) Definition: Denominator: Number of discharges with a secondary diagnosis of PE/DVT Numerator: All surgical discharges with a code for op. Procedure (Specified exclusions)
Postoperative Pulmonary embolism or Deep Vein Thrombosis Issues for discussion: Coding practice standardisation between hospitals? (Underreporting in administrative databases) Interpretation: Need for risk adjustment? (Patient factors operative procedure factors) Coupling to internal QI-activity?
PATH 9: discussion issues C2 Compliance with antibioprophylaxis guidelines Prevalence study pressure sores Assess both risk and presence / stade of ulcer Sample: hospital-wide or specific departement? Or specific conditions? Focus on low-risk or high-risk? Training needs? Reliability? Previous experience with such exercice?
PATH-II: discussion of results C16 score on CTM3 DEFINITION: The term care transitions refers to the movement patients make between health care practitioners and settings as their condition and care needs change during the course of a chronic or acute illness. The score (on a -1 scale) is built on responses (on 1-4 scale) to three items in questionnaire : 1. Preferences : The hospital staff took my preferences and those of my family or caregiver into account in deciding what my health care needs would be [when I left the hospital]. 2. Health management : [When I left the hospital], I had a good understanding of the things I was responsible for in managing my health. 3. Medications : [When I left the hospital], I clearly understood the purpose for taking each of my medications A higher score represents better transition from hospitals to home or other care settings
PATH-II: discussion of results C16 Score on CTM3 Sample size: INCLUSION CRITERIA : All patients discharged over the survey period. MINIMUM 6 patients per tracer condition/procedure to be included in the sample. Most hospitals have provided valid data for between 5 patients (P25) and 144 patients (P75) for 6 tracers or more. Risk adjustement: According to descriptive sheet: Age and sex BUT CTM developers indicate that the CTM is a patient centered measure that assesses the extent to which hospital staff accomplished essential care processes (...) to be extended universally, irrespective of disease burden or socio-demographic status. As a result, the CTM (...) does not employ risk-adjustment in calculating a summary score. (...). Each of the [empirical] analysis has confirmed that these variables [gender, age] does not bias CTM-3 response patterns. PATH results: no association between age and sex and CTM3 score
C16 Score on CTM3 1% 8% 6% 4% 2% % AMI Stroke Hipfracture CAP Asthma Diabetes Other* strongly agree agree disagree strongly disagree 1% 1% 8% 8% 6% 6% 4% 4% 2% 2% % 15-44 45-64 65 and more strongly agree agree disagree strongly disagree % Country 1 Country 2 Country 3 Total strongly agree agree disagree strongly disagree
PATH-II: discussion of results C16 Score on CTM3 Variations in the international sample 1, 8, 9,3 9,3 92,2 85, 85, 87,7 75,8 77,3 77,1 6, 61,9 62,9 63,8 4, 2,, Preference Health management Medication
PATH-II: discussion of results C15 Breastfeeding rate Definition: The percent of women with exclusive breastfeeding at discharge. WHO defines exclusive breastfeeding when the infant receives only breast milk from his/her mother or a wet nurse, or expressed breast milk, and no other liquids or solids with the exception of drops or syrup consisting of vitamins, mineral supplements or medicine. Numerator: Total number of mother included in the denominator breastfeeding at discharge. Denominator: Total number of delivery fulfilling criteria for inclusion. Exclusion criteria: Neither mother nor infant has a medical condition for which breastfeeding is contraindicated.
C15 Breastfeeding rate 1, 9, 8, 7, 6, WHO/UNICEF Baby Friendly Hospital Initiative: minimum threshold for label 5, 4, 3, 2, 1, Global rate, country 1 country 2 country 3 country 4 1 8 99 1 1 1 94,3 91,7 8,7 84,3 6 57,9 7,7 57,9 4 4,6 Individual indicators 2 country 1 country 2 country 3 country 4
PATH-II: discussion of results C13 Smoking prevalence NUMERATOR : number of staff smoking DENOMINATOR: total number of staff INCLUSION CRITERIA : All staff on the hospital payroll SOURCE OF DATA: The European Network of Smoke-free hospitals developed a survey measure including 13 standard questions to be able to compare differences between hospitals in various European countries. The first questions of the survey will be sufficient to gather information on staff smoking prevalence; the additional questions in the survey are optional for hospitals to fill in. Alternatively, if the information on staff smoking prevalence is already available from other sources (such as periodic staff health survey), these can be used.
C13 PATH-II: Smoking discussion prevalenceof results Participating PATH hospitals General population 1. 8. All female male 1 8 All Female Male 6. 6 4. 4 2. 2. Poland Belgium country 2 Small sample size France Poland Belgium Estonia Hungary Slovenia
PATH-II: discussion of results C13 Smoking prevalence Evidence to target health promotion activities 1% 9% 8% 7% 1 8 all health prof. all non prof. Health. 6% 5% 4% 3% 2% < 3 3-4 4-5 > 51 6 4 2 1% % other countries country 1 country 2 country 3 country 4 other countries country 1 country 2 country 3 country 4
PATH-II: discussion of results C14 Needle injuries This indicator assesses the number of needle injuries among FTE (Full time equivalent) staff. DEFINITION: Needlestick injuries are wounds caused by needles or other sharp objects that accidentally puncture the skin and may result in exposure to blood or other body fluids. Data is obtained through a survey asking about incidences of needle injuries in the last year. NUMERATOR: number of needlestick injuries over the last calendar year DENOMINATOR: Number of Full Time Equivalent (FTE) staff over the same time period
C14 Needle injuries 25 2 15 1 5 1,% 9,% 8,% 7,% 6,% 5,% 4,% 3,% 2,% 1,%,% 18,5 18,2 8,1 2,8 6,8 1,6 13,6 Nurses Doctors Technicians Housekeeping staff (N=98) (N=63) (N=61) (N=73) Poland country 1 country 2 country 3 country 4 3,1 Max = 45.5 5,4 Reference points: Wide variations in leterature but systematically much higher than PATH results: 1.4 and 5. sharp injuries per respectively 1 FTE medical or nursing staff in Australia teaching hospital 55.1% and 22.% needle injuries experienced by respectively for medical and nursing staff in a German university hospital 33.2 and 18. % incidence rate for all staff in 9 teaching and 32 non teaching US hospitals (3) nurses doctors technicians housekeeping total staff
C14 Needle injuries Higher risks in smaller hospitals? Random variations? Rate of injuries (in %) 25 Rate of injuries (in %) 25 2 15 18,5 16,5 17,8 15 16,3 2 15 18,2 14,6 1 5 12,7 4,6 1-1 (N=16) 9,5 11-15 (N=16) 6,2 3,8 3,4 3,2,8 1,6 151-25 (N=14) 2,8 251-35 (N=9) 8,8 8,1 5,5 5,1 1,8 2,2,3,3 351-5 (N=15) Hospital size (in FTE nurses) 51-75 (N=13) 8,9 1,4 6,9 2,7 75-1 (N=6) 2,7 7,9 5,2 11- more (N=8) 1 5 1,8 1 7,1 7 5,3 5 5 1,9 4 1,6 1,6 2,7 1,4 2,4,7 1-5 51-1 11-175 176-25 251-5 5 & more (N=14) (N=12) (N=12) (N=13) (N=6) (N=7) Hospital size (in FTE doctors) 5,6 9,3 Rate of injuries (in %) 25 2 15 1 5 1,4,5 3-6 (N=21) 6,4 2,4 4,6 5,8 5,3 3,9 4 5,1 1,5 1,5 2,3 1,8 2,1,3 61-9 91-14 14 & more (N=8) (N=13) (N=6) Hospital size (in FTE total staff)