Updated validation of AHRQ Prevention Quality Indicators in the USA Patrick S. Romano, MD MPH UC Davis Center for Healthcare Policy and Research Organization for Economic Cooperation and Development October 22, 2009
PQIs: Potentially Avoidable Hospitalizations Admissions for diagnoses that may have been prevented or ameliorated with access to high-quality outpatient care Two independently developed measure sets described in the 1990s literature John Billings Joel Weissman Strong independent negative correlations between self-rated access and avoidable hospitalization Correlations between avoidable hospitalization and: household income at zip code level (neg) uninsured or Medicaid enrolled (pos) maternal education (neg) Primary care physician to population ratio (neg) Weaker associations for Medicare populations
Current uses of the PQIs in the USA: Four examples National Healthcare Quality Report, Commonwealth Fund, California: Public health agencies tracking and comparing health system performance across counties, states Wisconsin Medicaid program: Comparing performance of managed care plans for low-income persons Dallas-Fort Worth Hospital Council: Exploring how community health affects hospitals General Motors: Estimating potential return on investment with improving primary care access in communities with GM employees.
Measures recommended for state Medicaid programs by the Foundation for Accountability Ambulatory care sensitive conditions ( potentially avoidable hospitalizations ) Angina Adult asthma/pediatric asthma Chronic obstructive pulmonary disease Congestive heart failure Diabetes (short-term and long-term complications, uncontrolled) Lower extremity amputation with diabetes Hypertension Other potentially avoidable conditions Perforated appendix Low birth weight
Percent Percent Percent Percent Evaluating Medicaid managed care programs in Wisconsin % ICare Enrollees with CHF Hospitalized for CHF % ICare Enrollees with Asthma Hospitalized for Asthma: 1998 & 2000 25 20 15 20.8 15.1 12 10 8 10 5 6 4 2 3.9 2.8 0 1998 2000 0 1998 2000 % ICare Enrollees with COPD Hospitalized for COPD % ICare Enrollees with Diabetes Hospitalized for Diabetes: 1998 & 2000 12 10 8 6 4 2 0 5.8 4.7 1998 2000 3.5 3 2.5 2 1.5 1 0.5 0 2.9 1998 2000 2.5
Reducing ED visits for PQIs in Medicaid managed care Comparison of total E.R. visits and ambulatory care sensitive condition (ACSC) E.R. visits for I-Care and matched FFS recipients, 1999 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% I-Care (n=1722) Comparable FFS (n=1722) No ER visit ER, not for ACSC ER for ACSC
Community Health Assessments using PQIs to understand role of hospitals
Risk Adjusted Rates per 100,000 Population 1 to 117.0 117.1 to 283.0 283.3 to 399.2 399.3 to 565.2 > 565.2 DI Hospitals AHRQ Prevention Quality Indicators Congestive Heart Failure Admission Rate - 2003 Named counties without shading have a Risk Adjusted rate of zero. Congestive Heart Failure Admission Rate Congestive heart failure (CHF) can be controlled in an outpatient setting for the most part; however, the disease is a chronic progressive disorder for which some hospitalizations are appropriate. Texas Hospital Inpatient Discharge Public Use Date File, FY2002. Texas Health Care Information Council, Austin, Texas. December, 2003. 2003 (PQI 08) Rates per 100,000 Cases County Numerator (Outcome) Denominator (Population) Observed Risk Adjusted Confidence Interval (95%) Stat. Sig. State of Texas 66,822 15,882,253 420.7 504.5 BOSQUE 21 13,486 155.7 0.0 ( 0.0, 0.0 ) CAMP 80 8,567 933.8 860.3 ( 664.7, 1055.9 ) - COLLIN 911 429,184 212.3 410.7 ( 391.6, 429.8 ) + COMANCHE 117 10,233 1,143.3 954.4 ( 766.0, 1142.8 ) - COOKE 60 28,036 214.0 142.7 ( 98.5, 186.9 ) + DALLAS 6,412 1,631,345 393.0 527.8 ( 516.7, 538.9 ) - DELTA 32 4,172 767.1 622.7 ( 384.0, 861.4 ) o DENTON 876 369,935 236.8 456.8 ( 435.1, 478.5 ) + EASTLAND 26 14,031 185.3 0.0 ( 0.0, 0.0 ) ELLIS 367 88,785 413.4 518.2 ( 471.0, 565.4 ) o ERATH 97 24,993 388.1 376.7 ( 300.8, 452.6 ) + FANNIN 171 25,024 683.3 610.2 ( 513.7, 706.7 ) - FRANKLIN 48 7,616 630.3 505.7 ( 346.4, 665.0 ) o GRAYSON 568 86,204 658.9 612.4 ( 560.3, 664.5 ) - HAMILTON 19 6,241 304.4 23.7 ( 0.0, 61.9 ) + HENDERSON 386 58,796 656.5 560.5 ( 500.2, 620.8 ) o HILL 220 25,615 858.9 776.5 ( 669.0, 884.0 ) - HOOD 149 34,883 427.1 330.1 ( 269.9, 390.3 ) + HOPKINS 37 24,315 152.2 84.1 ( 47.7, 120.5 ) + HUNT 362 59,959 603.7 630.8 ( 567.4, 694.2 ) - JACK 12 6,916 173.5 135.9 ( 49.1, 222.7 ) + JOHNSON 564 100,860 559.2 664.9 ( 614.7, 715.1 ) - KAUFMAN 390 59,401 656.6 750.8 ( 681.4, 820.2 ) - LAMAR 260 36,763 707.2 619.4 ( 539.2, 699.6 ) - MONTAGUE 21 14,900 140.9 0.0 ( 0.0, 0.0 ) MORRIS 64 10,014 639.1 507.3 ( 368.2, 646.4 ) o NAVARRO 216 34,415 627.6 596.6 ( 515.2, 678.0 ) - PALO PINTO 22 20,368 108.0 2.9 ( 0.0, 10.3 ) + PARKER 210 72,541 289.5 361.6 ( 317.9, 405.3 ) + RAINS 38 8,472 448.5 396.8 ( 262.9, 530.7 ) o ROCKWALL 106 39,433 268.8 394.3 ( 332.4, 456.2 ) + SOMERVELL 9 5,416 166.2 139.0 ( 39.8, 238.2 ) + STEPHENS 6 7,147 84.0 0.0 ( 0.0, 0.0 ) TARRANT 3,688 1,118,382 329.8 456.1 ( 443.6, 468.6 ) + TITUS 40 19,796 202.1 198.5 ( 136.5, 260.5 ) + VAN ZANDT 209 38,329 545.3 458.6 ( 391.0, 526.2 ) o WISE 58 39,967 145.1 209.0 ( 164.2, 253.8 ) + WOOD 235 30,845 761.9 599.1 ( 513.0, 685.2 ) - YOUNG 15 13,604 110.3 0.0 ( 0.0, 0.0 ) + = County s RA rate significantly lower than State RA rate - = County s RA rate significantly higher o = No statistical difference 2004, Dallas-Fort Worth Hospital Council - Data Initiative For Hospital Internal Use Only April, 2005
General Motors: Impact of PQI admissions on employer-financed health care QI Name Potential cost savings if number of admissions were reduced by specified percentage County name (all counties in MI listed), average cost of admission for QI specified, total number of cases, and total cost
General Motors mapped PQI data Name of Indicator and Data Year in Map Title Data quintiles. Green is the lowest 20% or the lowest rates. Red is the highest 20% or the highest rates. Symbol indicating number of GM covered beneficiaries, number below is average in the group. Counties with high indicator rates and higher number of beneficiaries
Using GM-mapped data Integrate action plans with other Community Initiatives projects Pay for Performance for providers in specific counties to reduce admission rates Coordination with other Community Stakeholders to achieve desired improvement Funding to implement projects at a community level Focus on indicators with highest potential cost savings
Area Potential uses of PQIs QI Comparative Reporting Payor X X Provider X X X X Pay for Performance Current application Extended applications 1 We initially assessed the internal quality improvement application for large provider groups. Following our initial rating period, panelists expressed interest in applying select indicators to the long term care setting and these applications were added to our panel questionnaire.
Revalidation methods Clinical Panel review using new hybrid Delphi/Nominal Group technique Two groups: Core and Specialist Core assesses all; Specialist only those applicable to their specialty Three indicator groups: Acute, Chronic, Diabetes Two panels: Delphi Nominal Group
Delphi vs. Nominal Delphi group Advantages: Larger, hence better reliability, more points of view, less chance for one panelist to pull the group Disadvantage: Less communication and cross-pollination across panelists, less ability to discuss and refine details of indicators/evaluation Nominal group Advantages: Can discuss details, facilitate sharing of ideas Disadvantages: Limited in size and therefore in representation, one strong panelist can flavor group and therefore poorer reliability
Composition of panels Characteristic Delphi Group (n = 42) Gender Male 62.8 73.9 Female 37.2 26.1 Urban/Rural 1 Urban 32.6 30.4 Suburban 14.0 13.0 Rural 7.0 8.7 Multiple/All areas served 16.3 30.4 Academic Affiliation 1 Academic practice 27.9 47.8 Non-academic practice 34.9 30.4 Any academic affiliation 69.8 87.0 Underserved population in practice 1 46.5 69.6 Funding 1 Public 27.9 34.8 Private and/or Non-profit 20.9 39.1 Multiple sources 7.0 0 Nominal Group (n = 23)
Specialties represented Specialty Delphi Panel Nominal Panel Internal Medicine 5 3 Family Medicine 4 1 Geriatric Medicine 2 2 Public Health Physician 4 0 Emergency Medicine 3 2 General Nurse Practitioner 2 1 Endocrinology 4 2 Vascular Surgery 2 1 Diabetes Outpatient 1 1 Management Nephrology 0 1 Cardiology 4 3 Pulmonology 3 2 Asthma Specialist 1 0 Pulmonary Rehabilitation 1 0 Infectious Disease 2 2 General Surgery 3 2 Urology 1 0
Panel Process: Exchange of Information Delphi Dephi rating Results: initial rating Delphi panel re-rates Delphi comments Nominal comment 1 st round results to panelists prior to call Diabetes call Acute call Call summaries to panels Final ratings Nominal Nominal rating Results: Initial rating Chronic call Nominal panel re-rates
Quality Improvement Applications Indicator Provider COPD and Asthma (40 yrs +) + Asthma ( < 39 yrs) Hypertension + Angina CHF Perforated Appendix + Diabetes Short Term Complications Diabetes Long-Term Complications Lower Extremity Amputation Bacterial Pneumonia UTI Dehydration + + + Major Concern Regarding Use, Some Concern General Support Full Support + Either Delphi or Nominal Panel reported higher level of support for measure than shown
Comparative Reporting Applications Indicator Area Provider COPD + Asthma ( < 39 yrs) + + Hypertension + Angina CHF + Perforated Appendix + + Diabetes Short Term + Diabetes Long-Term + LE Amputation Bacterial Pneumonia UTI Dehydration Major Concern Regarding Use, Some Concern General Support Full,Support + Either Delphi or Nominal Panel reported higher level of support for measure than shown
Potential interventions to reduce hospitalizations Acute Chronic Area Access to primary care/urgent care Payor Coverage of medications Coverage of auxiliary health services (e.g. at home nursing) Access to primary care/urgent care Provider Quality nursing triage Patient education Accurate/rapid diagnosis and treatment Appointment availability Outpatient treatment of complications Access to care Lifestyle modifications Coverage of medications Coverage of comprehensive care programs Coverage of auxiliary health services (e.g. at home nursing) Disease management programs Lifestyle modification incentives Education, disease management Lifestyle medication interventions Comprehensive care programs, care coordination, auxiliary health services
Potential improvements Defining the numerator One admission per patient per year? Using related principal diagnoses with target secondary diagnoses (i.e., pneumonia with COPD) Consider excluding first hospitalization before chronic condition diagnosed Defining the denominator Identifying patients with chronic diseases (mulitple diagnoses from outpatient claims, population survey rates, pharmaceutical data Requiring minimum tenure with payor or provider or minimum duration of residence in an area
Risk adjustment Demographics Age and gender highly rated as important Race/ethnicity depending on indicator (i.e., diabetes) Disease severity Historical vs. current data Comorbidity Lifestyle associated risk and compliance Smoking, obesity Pharmacy records Socioeconomic status May mask true disparities in access to care Panel felt benefits of inclusion outweighed problems
Policy implications Ensuring true quality improvement Shifting risk (adverse selection) Manipulating coding Cost/burden of data collection to support better risk adjustment Different perspectives of different stakeholders Does avoiding hospitalization really reflect the best quality and value?
Acknowledgments Project team: Sheryl Davies, MA (Stanford) Kathryn McDonald, MM (Stanford) Eric Schmidt, BA (Stanford) Ellen Schultz, MS (Stanford) Olga Saynina, MS (Stanford) Jeffrey Geppert JD (Battelle) Patrick Romano, MS, MD (UC Davis) This project was funded by a contract from the Agency for Healthcare Research and Quality (#290-04-0020)