The Long-Term Effect of Premier Pay for Performance on Patient Outcomes

T h e n e w e ngl a nd j o u r na l o f m e dic i n e Special article The Long-Term Effect of Premier Pay for Performance on Patient Outcomes Ashish K. Jha, M.D., M.P.H., Karen E. Joynt, M.D., M.P.H., E. John Orav, Ph.D., and Arnold M. Epstein, M.D. A bs tr ac t From the Department of Health Policy and Management, Harvard School of Public Health (A.K.J., K.E.J., A.M.E.); the Division of General Medicine (A.K.J., E.J.O., A.M.E.); and the Division of Cardiovascular Medicine (K.E.J.), Brigham and Women s Hospital; and the Veterans Affairs Boston Healthcare System (A.K.J.) all in Boston. Address reprint requests to Dr. Jha at the Department of Health Policy and Management, Harvard School of Public Health, 677 Huntington Ave., Boston, MA 02115, or at ajha@ hsph.harvard.edu. This article (10.1056/NEJMsa1112351) was published on March 28, 2012, and updated on March 29, 2012, at NEJM.org. N Engl J Med 2012;366:1606-15. Copyright 2012 Massachusetts Medical Society. Background Pay for performance has become a central strategy in the drive to improve health care. We assessed the long-term effect of the Medicare Premier Hospital Quality Incentive Demonstration (HQID) on patient outcomes. Methods We used Medicare data to compare outcomes between the 252 hospitals participating in the Premier HQID and 3363 control hospitals participating in public reporting alone. We examined 30-day mortality among more than 6 million patients who had acute myocardial infarction, congestive heart failure, or pneumonia or who underwent coronary-artery bypass grafting (CABG) between 2003 and 2009. Results At baseline, the composite 30-day mortality was similar for Premier and non-premier hospitals (12.33% and 12.40%, respectively; difference, 0.07 percentage points; 95% confidence interval [CI], 0.40 to 0.26). The rates of decline in mortality per quarter at the two types of hospitals were also similar (0.04% and 0.04%, respectively; difference, 0.01 percentage points; 95% CI, 0.02 to 0.01), and mortality remained similar after 6 years under the pay-for-performance system (11.82% for Premier hospitals and 11.74% for non-premier hospitals; difference, 0.08 percentage points; 95% CI, 0.30 to 0.46). We found that the effects of pay for performance on mortality did not differ significantly among conditions for which outcomes were explicitly linked to incentives (acute myocardial infarction and CABG) and among conditions not linked to incentives (congestive heart failure and pneumonia) (P = 0.36 for interaction). Among hospitals that were poor performers at baseline, mortality was similar in the two groups of hospitals at the start of the study (15.12% and 14.73%; difference, 0.39 percentage points; 95% CI, 0.36 to 1.15), with similar rates of improvement per quarter (0.10% and 0.07%; difference, 0.03 percentage points; 95% CI, 0.08 to 0.02) and similar mortality rates at the end of the study (13.37% and 13.21%; difference, 0.15 percentage points; 95% CI, 0.70 to 1.01). Conclusions We found no evidence that the largest hospital-based pay-for-performance program led to a decrease in 30-day mortality. Expectations of improved outcomes for programs modeled after Premier HQID should therefore remain modest. 1606 n engl j med 366;17 nejm.org april 26, 2012

Effect of Pay for Performance on Patient Outcomes T ying financial incentives to performance, often referred to as pay for performance, has gained broad acceptance as an approach to improving the quality of health care. 1-4 The Centers for Medicare and Medicaid Services (CMS) recently completed a 6-year demonstration of pay for performance for hospitals through the Premier Hospital Quality Incentive Demonstration (HQID), and the Affordable Care Act calls for CMS to expand this program to nearly all U.S. hospitals in 2012. The policy of tying financial incentives to the quality of performance has strong face validity that is, paying for better care should promote improvements in quality and, ideally, lead to better patient outcomes. Whether pay for performance will lead to better patient outcomes, however, is unclear. Although there is evidence from the Premier HQID that pay for performance is associated with modest improvements in the processes of care, 5-8 much less is known about its effect on patient outcomes. Two studies examined outcomes in the Premier HQID, and neither showed any effect on mortality rates within the first 3 years 9,10 ; however, to our knowledge, there have been no studies of the effect on outcomes over the longer term. Long-term data are particularly important because it may take years for providers to reconfigure their underlying approach to care, and improvements in outcomes from pay for performance may therefore become evident only after years of work. In this study we addressed three questions: first, did the Premier HQID lead to lower 30-day mortality among patients who received care in the hospitals that received financial incentives for 6 years? Second, if there were any benefits with respect to mortality, were they particularly pronounced for the outcomes of conditions that were explicitly linked to incentives, as compared with the outcomes of conditions that were merely linked to process measures? Third, given prior evidence that certain types of hospitals may have a greater incentive or capability to improve quality (e.g., hospitals that were poor performers at baseline or those with higher financial margins), 8,11 were the benefits of the Premier HQID with respect to patient outcomes particularly pronounced in these subgroups of hospitals? Me thods Study Design In 2003, a total of 421 hospitals in the Premier Healthcare Informatics program were invited by CMS to participate in the HQID, of which 252 (60%) joined and were available for analysis. 12 These hospitals agreed to provide data on 33 measures, including indicators for three medical conditions (acute myocardial infarction, congestive heart failure, and pneumonia) and two surgical procedures (coronary-artery bypass grafting [CABG] and total knee or total hip replacement). All five conditions were assessed with the use of process-quality indicators, and additional indicators to assess risk-adjusted mortality were used for acute myocardial infarction and CABG. Hospitals that performed in the top two deciles for any of these conditions were eligible for 1 to 2% bonuses in Medicare payments for that condition, whereas underperforming hospitals were liable for a 1 to 2% financial penalty starting in the fourth year of the program. The Premier HQID made modest changes later in the program to offer additional incentives for hospitals that made substantial improvements in care. We evaluated changes in riskadjusted mortality overall and for each condition individually, excluding hip and knee replacement because of the low mortality associated with these procedures (approximately 0.7%). 9 Because our intent was to compare the effect of the addition of pay for performance to public reporting (i.e., the Premier program) with public reporting alone (i.e., the concurrently running Medicare Hospital Compare program, which included more than 98% of eligible hospitals), we selected as our control group all non-premier hospitals that reported to Hospital Compare as of the end of 2004. We used multivariable techniques to adjust for differences between Premier and non- Premier hospitals in terms of patient population and hospital characteristics (obtained from the American Hospital Association [AHA] annual survey). The hospital characteristics used in our models were chosen on the basis of prior work showing that Premier hospitals were larger than non- Premier hospitals, were more often nonprofit or teaching hospitals, and were more likely to be located in urban areas in the southern United States. 5,13 The study was approved by the Office of n engl j med 366;17 nejm.org april 26, 2012 1607

T h e n e w e ngl a nd j o u r na l o f m e dic i n e Human Research Administration at the Harvard School of Public Health. Clinical Outcomes We used national Medicare Part A data from 2002 through 2009 to examine 30-day, risk-adjusted mortality rates for more than 6 million patients discharged with one of the principal diagnoses or surgical procedures of interest. We created five sets of patient-level logistic-regression models, accounting for clustering of patients within hospitals in each model. The first model was an overall model, which combined patients across all four conditions (acute myocardial infarction, congestive heart failure, pneumonia, and CABG); the remaining models were specific for each of the four conditions. Each patient s risk of death was adjusted for 29 coexisting medical conditions with the use of the Elixhauser risk-adjustment scheme, a validated 14-16 and widely used 17-22 tool developed for use in analyzing administrative data. (For details, see the Supplementary Appendix, available with the full text of this article at NEJM.org.) We used these models to plot quarterly adjusted mortality rates for Premier versus non-premier hospitals overall and for each condition. Covariates of Interest We also calculated three other covariates of interest according to findings reported by Werner et al. 8 indicating that the greatest improvements in process measures under Premier HQID occurred in hospitals that were eligible for large bonuses (on the basis of the proportion of their patients with Medicare insurance), that had good financial health (as indicated by positive total margins), and that Table 1. Hospital and Patient Characteristics.* Characteristic Premier Non-Premier P Value Hospitals No. of hospitals 252 3363 Hospital size (%) Small 13.49 39.77 <0.001 Medium 61.11 50.13 Large 25.40 11.09 Teaching status (%) Teaching 13.49 7.11 <0.001 Nonteaching 86.51 92.89 Location (%) Urban 94.84 81.56 <0.001 Rural 5.16 18.44 Ownership (%) Private for-profit 1.19 17.78 <0.001 Private nonprofit 90.08 61.64 Public 8.73 20.58 Region (%) Northeast 13.10 14.99 0.001 Midwest 22.22 27.65 South 51.19 38.75 West 13.49 18.61 Mean financial margin (%) 3.89 3.08 0.44 Mean Herfindahl Hirschman index 0.122 0.125 0.63 Mean proportion of patients receiving Medicare (%) 44.49 47.21 <0.001 Mean mortality (%) 12.88 13.41 0.22 1608 n engl j med 366;17 nejm.org april 26, 2012

Effect of Pay for Performance on Patient Outcomes Table 1. (Continued.) Characteristic Premier Non-Premier P Value Patients, 2009 sample No. of patients 137,287 1,069,034 Mean age (yr) 79.92±8.34 79.63±8.29 <0.001 Female sex (%) 51.49 52.52 <0.001 Race or ethnic group (%) Non-Hispanic white 85.13 85.99 <0.001 Non-Hispanic black 10.34 9.41 Hispanic 1.79 1.75 Other 2.75 2.86 Coexisting conditions (%) Diabetes 27.48 27.86 0.004 Hypertension 54.86 55.21 0.014 Chronic kidney disease 22.09 21.76 0.005 Chronic pulmonary disease 30.76 32.11 <0.001 * Plus minus values are means ±SD. This index, a commonly accepted measure of market concentration, is calculated by squaring the market share (expressed as a fraction of total market) of each firm competing in the market and then summing the resulting numbers. Markets in which the index is between 0.10 and 0.18 points are considered to be moderately concentrated, and those in which the index is in excess of 0.18 points are considered to be concentrated. Race or ethnic group was self-reported. were located in the least competitive markets (as measured by the Herfindahl Hirschman index). We used the AHA survey to obtain the proportion of Medicare patients for each hospital and the Herfindahl Hirschman index for each market; the total margin for each hospital was calculated from the Medicare Hospital Cost Reports. 23 Statistical Analysis We compared key characteristics of Premier hospitals with those of non-premier hospitals, as well as the characteristics of the patients who received care at each of these types of hospitals. Next, we compared mortality rates between Premier and non-premier hospitals in the initial baseline year (the fourth quarter of 2003 through the third quarter of 2004), using linear regression models with quarterly hospital risk-adjusted mortality as the outcome and accounting for correlation across quarters, with an indicator for Premier versus non- Premier hospitals as the predictor. This same analysis was carried out for the last year of available mortality data (the first through fourth quarters of 2009). We then compared changes in risk-adjusted mortality rates between Premier and non-premier hospitals from the fourth quarter of 2003 (the beginning of the study period) through the fourth quarter of 2009 (the end of the study period) with the use of longitudinal linear regression analyses, with the hospital as the unit of analysis. Our primary predictors were an indicator variable for pay-for-performance status (Premier vs. non-premier), a continuous variable for time (by quarter), and an interaction term between pay-forperformance status and time. Each of these models was weighted for the number of patients seen at each hospital and was adjusted for the following hospital characteristics: size (small, medium, or large, as categorized by the AHA), teaching status (member of the Council of Teaching Hospitals or not), location (urban vs. rural), ownership (public, private nonprofit, or private for-profit), region (the four U.S. census regions), financial margin, the Herfindahl Hirschman index, and the proportion of patients receiving Medicare. In addition, because there were secular trends in employment relationships between hospitals and physicians, we tested whether changes in hospitalstaffing models, as reported in the AHA survey, confounded the relationship between pay-for-performance status and changes in mortality. Since this variable had no qualitative effect, these data are not included here. We then created a model to test whether any n engl j med 366;17 nejm.org april 26, 2012 1609

T h e n e w e ngl a nd j o u r na l o f m e dic i n e incremental benefits with respect to mortality among the Premier hospitals, as compared with the non-premier hospitals, were greater for the two conditions with outcomes that were explicitly linked to incentives (acute myocardial infarction and CABG) than for the conditions with outcomes that were not explicitly linked to incentives (pneumonia and congestive heart failure), by adding an indicator for the incentives and creating a three-way interaction term among time, pay-forperformance status, and explicit incentivization for the outcome of the particular condition. We also examined interactions with hospital characteristics by means of individual models that included terms for the size of the potential bonus, the hospital s financial status, and the local levels of competition. Because of substantial policy interest in how incentives might affect poorly performing hospitals, and because prior studies have found that most quality-improvement efforts disproportionately benefit providers who are poor performers at baseline, 11 we chose, a priori, to perform subgroup analyses of the effect of pay for performance on hospitals performing in the highest quartile of mortality rates before the initiation of the Premier HQID (i.e., the first quarter of 2002 through the third quarter of 2003). We calculated baseline performance for all study conditions combined, as well as for each study condition separately. We repeated the analyses described above, first examining overall mortality for poorly performing Premier hospitals versus non-premier hospitals and then examining each condition separately. A two-sided P value of less than 0.05 was considered to indicate statistical significance. R esult s hospital and patient characteristics Hospitals participating in the Premier HQID program were larger, more often teaching hospitals, and more often nonprofit institutions than non- Premier hospitals (Table 1). Premier hospitals were also more likely to be located in the southern United States than were non-premier hospitals. As compared with Medicare patients admitted to the non-premier hospitals, those admitted to Premier hospitals were slightly older (79.9 years vs. 79.6 years, P<0.001), less likely to be women (51.5% vs. 52.5%, P<0.001), more likely to be black (10.3% vs. 9.4%, P<0.001), and more likely to have chronic kidney disease but less likely to have diabetes, hypertension, or chronic pulmonary disease (Table 1). Clinical Outcomes We found that mortality was similar at Premier and non-premier hospitals at baseline (12.33% and 12.40%, respectively; difference, 0.07%; 95% confidence interval [CI], 0.40 to 0.26) (Table 2). Mortality rates declined at similar rates among both Premier and non-premier hospitals during the study period ( 0.04% per quarter among Premier hospitals and 0.04% per quarter among non- Premier hospitals; difference, 0.01 percentage points per quarter; 95% CI, 0.02 to 0.01; P = 0.55 for the difference) (Table 2 and Fig. 1). At the end of the 6 years under the incentives program, overall mortality across these four conditions did not differ significantly between the Premier hospitals and the non-premier hospitals (11.82% and 11.74%, respectively; difference, 0.08 percentage points; 95% CI, 0.30 to 0.46). We found similar patterns in 30-day mortality when we examined each of the three medical conditions individually (acute myocardial infarction, congestive heart failure, and pneumonia), with no significant difference between Premier and non-premier hospitals in terms of mortality at baseline, rates of decline in mortality, or mortality at the end of the study period (Table 2, and Fig. 1A, 1B, and 1C in the Supplementary Appendix). For CABG, there were no significant differences at baseline or in rates of change, but mortality at the end of the study period was higher at Premier hospitals than at non-premier hospitals (4.12% vs. 3.34%; difference, 0.78 percentage points; 95% CI, 0.20 to 1.36) (Table 2, and Fig. 1D in the Supplementary Appendix). For conditions with outcomes that were specifically linked to incentives (acute myocardial infarction and CABG) versus conditions with outcomes that were not linked to incentives (congestive heart failure and pneumonia), we found no difference in trends in mortality rates between Premier and non-premier hospitals (P = 0.36 for interaction). We also found no evidence that the effect of the Premier HQID on mortality rates differed on the basis of a hospital s potential financial incentive (P = 0.47 for interaction), its financial health (P = 0.79 for interaction), or the competitiveness of the market in which it was located (P = 0.35 for interaction). 1610 n engl j med 366;17 nejm.org april 26, 2012

Effect of Pay for Performance on Patient Outcomes Table 2. Mortality at 30 Days for Study Conditions at All Participating Hospitals, According to Premier Status, 2003 2009.* Condition and Pay-for-Performance Status Mortality during Baseline Period Change in Mortality per Quarter Mortality during Terminal Period All conditions Premier % 12.33 0.04 11.82 Non-Premier % 12.40 0.04 11.74 Difference (95% CI) percentage points 0.07 ( 0.40 to 0.26) 0.01 ( 0.02 to 0.01) 0.08 ( 0.30 to 0.46) Acute myocardial infarction Premier % 17.32 0.11 15.67 Non-Premier % 17.42 0.09 15.85 Difference (95% CI) percentage points 0.10 ( 0.72 to 0.52) 0.02 ( 0.05 to 0.01) 0.18 ( 0.97 to 0.61) Congestive heart failure Premier % 10.68 0.01 11.13 Non-Premier % 10.61 0.01 10.92 Difference (95% CI) percentage points 0.07 ( 0.31 to 0.46) 0.00 ( 0.02 to 0.02) 0.22 ( 0.28 to 0.71) Pneumonia Premier % 12.87 0.07 11.71 Non-Premier % 13.13 0.06 11.85 Difference (95% CI) percentage points 0.26 ( 0.70 to 0.19) 0.01 ( 0.03 to 0.02) 0.14 ( 0.67 to 0.38) Coronary-artery bypass grafting Premier % 3.91 0.03 4.12 Non-Premier % 3.62 0.02 3.34 Difference (95% CI) percentage points 0.29 ( 0.12 to 0.69) 0.01 ( 0.03 to 0.02) 0.78 (0.20 to 1.36) * Rates have been adjusted for patient and hospital characteristics. Differences in mortality were calculated as the mortality at the Premier hospitals minus the mortality at the non-premier hospitals. None of the differences in mortality between the two hospital groups were significant except for coronary-artery bypass grafting in the terminal period (P = 0.01). CI denotes confidence interval. The 1-year baseline period extended from the fourth quarter of 2003 through the third quarter of 2004. The 1-year terminal period extended from the first quarter of 2009 through the fourth quarter of 2009. (The individual interaction terms and their confidence intervals are presented in Table 2 in the Supplementary Appendix.) In the subgroup of hospitals that were poor performers at baseline, overall baseline mortality rates were similar at Premier hospitals and non-premier hospitals (15.12% and 14.73%, respectively; difference, 0.39 percentage points; 95% CI, 0.36 to 1.15) (Table 3), with no significant difference in the rate of improvement over time ( 0.10% vs. 0.07% per quarter, respectively; difference, 0.03 percentage points; 95% CI, 0.08 to 0.02; P = 0.22) (Table 3 and Fig. 2). Similarly, there was no significant difference in overall mortality at the end of the study period (13.37% at Premier hospitals and 13.21% at non-premier hospitals; difference, 0.15 percentage points; 95% CI, 0.70% to 1.01). The findings were similar in the condition-specific analyses, with one exception: mortality among patients with pneumonia was reduced more rapidly at Premier hospitals than at non-premier hospitals ( 0.16% vs. 0.10% per quarter; difference, 0.06 percentage points; 95% CI, 0.11 to 0; P = 0.04) (Table 3). Discussion We found little evidence that participation in the Premier HQID program was associated with declines in mortality above and beyond those reported for hospitals that participated in public reporting alone, even when we examined care over a period of 6 years after the program s inception. Furthermore, we found no differences in trends in mortality between conditions for which outcomes were explicitly linked to incentives and n engl j med 366;17 nejm.org april 26, 2012 1611

T h e n e w e ngl a nd j o u r na l o f m e dic i n e 16 14 13.73 13.31 Premier Non-Premier 12 13.66 13.18 11.19 30-Day Mortality (%) 10 8 6 4 2 0 Preincentives Period Inception of Premier HQID Incentives Period 10.94 2002 2003 2004 2005 2006 2007 2008 2009 Figure 1. Mortality at 30 Days among All Hospitals, According to Pay-for-Performance Status, 2002 2009. The rates have been adjusted for patient and hospital characteristics and include all study conditions. HQID denotes Hospital Quality Incentives Demonstration. conditions for which outcomes were not linked to incentives. Even among hospitals that were poor performers at baseline, there was only a weak and inconsistent association between participation in the Premier HQID program and reductions in mortality. Taken together, these findings are sobering for policymakers who hope to use incentives such as those in Premier HQID to improve patient outcomes (e.g., 30-day mortality). In the Affordable Care Act, the U.S. Congress mandated the CMS to adopt a pay-for-performance program for hospitals nationwide. 24 CMS responded by creating the value-based purchasing (VBP) program, which provides financial incentives for both high achievement and improvement in performance 25 an approach closely modeled after the Premier HQID during the latter part of the study period. Currently, VBP focuses on incentives for process measures (along with metrics of the patient s experience), but in 2013, it will be broadened to include 30-day mortality. 26 Although the Premier HQID may be associated with marginally greater adherence to process measures, 8 we found little evidence that this program reduced mortality rates beyond the results achieved with public reporting alone, perhaps reflecting the modest relationship between the process indicators included in HQID and 30-day mortality. 21,27 Pay for performance may be intended to improve outcomes for all participants, but an important goal of most quality-improvement efforts is to motivate the poor performers (i.e., those who have the most to gain). Even though we found that poorly performing hospitals improved under the Premier HQID program, the improvements were similar to those seen under public reporting alone, with the exception of pneumonia. We are not sure why the patterns for pneumonia were different, although it is possible that the finding is factitious, reflecting the multiple outcomes tested. Prior studies of the Premier HQID showed that the early gains in process quality had mostly dissipated after 5 years under the program. 5 Werner et al. found that the modest benefits in adherence to process measures were most perceptible for hospitals that were eligible for larger bonuses, that were well financed, or that operated in less competitive markets. 8 We failed to find that these factors were associated with a significantly greater effect on outcomes. Although the data regarding the effect on processes of care are important, patients care most 1612 n engl j med 366;17 nejm.org april 26, 2012

Effect of Pay for Performance on Patient Outcomes Table 3. Mortality at 30 Days for Study Conditions at Hospitals with Poor Performance at Baseline, According to Premier Status, 2003 2009.* Condition and Pay-for-Performance Status All conditions Mortality during Baseline Period Change in Mortality per Quarter Mortality during Terminal Period Premier % 15.12 0.10 13.37 Non-Premier % 14.73 0.07 13.21 Difference (95% CI) percentage points 0.39 ( 0.36 to 1.15) 0.03 ( 0.08 to 0.02) 0.15 ( 0.70 to 1.01) Acute myocardial infarction Premier % 19.75 0.19 16.66 Non-Premier % 20.50 0.14 17.74 Difference (95% CI) percentage points 0.75 ( 2.73 to 1.24) 0.05 ( 0.16 to 0.07) 1.08 ( 3.08 to 0.92) Congestive heart failure Premier % 12.53 0.05 12.04 Non-Premier % 12.25 0.04 12.11 Difference (95% CI) percentage points 0.27 ( 0.72 to 1.27) 0.02 ( 0.08 to 0.04) 0.07 ( 1.49 to 1.36) Pneumonia Premier % 15.11 0.16 12.32 Non-Premier % 15.45 0.10 13.09 Difference (95% CI) percentage points 0.34 ( 1.25 to 0.56) 0.06 ( 0.11 to 0.00) 0.77 ( 1.82 to 0.28) Coronary-artery bypass graft surgery Premier % 4.82 0.05 4.88 Non-Premier % 4.70 0.03 3.97 Difference (95% CI) percentage points 0.11 ( 0.72 to 0.95) 0.01 ( 0.07 to 0.05) 0.91 ( 0.91 to 2.73) * Rates have been adjusted for patient and hospital characteristics. Differences in mortality were calculated as the mortality at the Premier hospitals minus the mortality at the non-premier hospitals. None of the differences in mortality between the two hospital groups were significant except for the change in mortality per quarter for pneumonia (P = 0.04). The 1-year baseline period extended from the fourth quarter of 2003 through the third quarter of 2004. The 1-year terminal period extended from the first quarter of 2009 through the fourth quarter of 2009. about outcomes. The prior evidence suggests little benefit of the Premier HQID program in the short run: Ryan found no evidence that the program affected 30-day mortality rates through mid-2006, 9 and this finding was confirmed by Glickman and colleagues for Premier hospitals participating in a disease registry for acute myocardial infarction. 10 Proponents of pay for performance might argue that benefits with respect to mortality would take longer to become evident as hospitals reconfigure their underlying approach to delivering care. We find little evidence in the Premier HQID program to support this notion, even after 6 years of financial incentives under the program. Our study has limitations. First, the Premier HQID incentives were primarily focused on processes of care, with mortality rates for acute myocardial infarction and CABG accounting for only a small fraction of the measures evaluated. Incentives for improving outcomes may have a greater effect if hospitals are not distracted by competing targets that are easier to achieve. Hospitals participating in the Premier HQID program were self-selected and potentially different from control hospitals in ways we could not account for in our models. Premier hospitals may have been more committed to quality improvement, although their mortality rates were similar to those of non-premier hospitals at baseline, which offers some reassurance. We used administrative data to adjust for underlying differences in the risk of death among patients, but these reports have inherent limitations (e.g., lack of details regarding the subtype of a patient s acute myocardial infarction). However, any differences between n engl j med 366;17 nejm.org april 26, 2012 1613

T h e n e w e ngl a nd j o u r na l o f m e dic i n e 30-Day Mortality (%) 20 18 16 14 12 10 8 6 4 2 0 17.79 17.29 Preincentives Period 16.04 15.68 Inception of Premier HQID Premier 14.19 14.13 Non-Premier Incentives Period 13.48 12.17 2002 2003 2004 2005 2006 2007 2008 2009 Figure 2. Mortality at 30 Days for Hospitals with Poor Performance at Baseline, According to Pay-for-Performance Status, 2002 2009. The rates have been adjusted for patient and hospital characteristics and include all study conditions. Premier and non-premier hospitals should have been mitigated by our difference-in-differences models. We could have failed to detect a small effect on mortality owing to the sample size; however, the HQID is the largest demonstration of hospital pay for performance to date, and it is unclear whether benefits smaller than those detectable in our study would be clinically meaningful. Conversely, our finding of significantly higher mortality during the terminal period at Premier hospitals for patients undergoing CABG is probably due to particularly poor outcomes during the first two quarters of 2009 and not reflective of broader trends in care over the entire study period. Finally, the Premier HQID is only one model of pay for performance, and the results may not be generalizable to all pay-for-performance programs. Alternative models that incorporate larger incentives and are focused on outcomes may be more effective. In summary, we found little evidence that participation in the Premier HQID program led to lower 30-day mortality rates, suggesting that we still have not identified the right mix of incentives and targets to ensure that pay for performance will drive improvements in patient outcomes. Even though Congress has required that the CMS adopt pay for performance for hospitals, expectations with regard to programs modeled after Premier HQID should remain modest. Supported by a grant from the Robert Wood Johnson Foundation. Disclosure forms provided by the authors are available with the full text of this article at NEJM.org. References 1. Berwick DM, DeParle NA, Eddy DM, et al. Paying for performance: Medicare should lead. Health Aff (Millwood) 2003; 22:8-10. 2. Greenberg JO, Dudley JC, Ferris TG. Engaging specialists in performanceincentive programs. N Engl J Med 2010; 362:1558-60. 3. Lee PV, Berenson RA, Tooker J. Payment reform the need to harmonize approaches in Medicare and the private sector. N Engl J Med 2010;362:3-5. 4. Rosenthal MB, Landon BE, Normand SL, Frank RG, Epstein AM. Pay for performance in commercial HMOs. N Engl J Med 2006;355:1895-902. 5. Lindenauer PK, Remus D, Roman S, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007;356:486-96. 6. Petersen LA, Woodard LD, Urech T, Daw C, Sookanan S. Does pay-for-performance improve the quality of health care? Ann Intern Med 2006;145:265-72. 7. Rosenthal MB, Frank RG. What is the empirical basis for paying for quality in health care? Med Care Res Rev 2006;63: 135-57. 8. Werner RM, Kolstad JT, Stuart EA, Polsky D. The effect of pay-for-performance in hospitals: lessons for quality improvement. Health Aff (Millwood) 2011;30:690-8. 9. Ryan AM. Effects of the Premier Hospital Quality Incentive Demonstration on Medicare patient mortality and cost. Health Serv Res 2009;44:821-42. 1614 n engl j med 366;17 nejm.org april 26, 2012

Effect of Pay for Performance on Patient Outcomes 10. Glickman SW, Ou FS, DeLong ER, et al. Pay for performance, quality of care, and outcomes in acute myocardial infarction. JAMA 2007;297:2373-80. 11. Rosenthal MB, Frank RG, Li Z, Epstein AM. Early experience with pay-forperformance: from concept to practice. JAMA 2005;294:1788-93. 12. Centers for Medicare and Medicaid Services. CMS/Premier Hospital Quality Incentive Demonstration (HQID) (http:// www.premierinc.com/quality-safety/ tools-services/p4p/hqi/index.jsp). 13. Jha AK, Orav EJ, Epstein AM. The effect of financial incentives on hospitals that serve poor patients. Ann Intern Med 2010;153:299-306. 14. Healthcare Cost and Utilization Project. HCUP Comorbidity Software, version 3.7. 2010 (http://www.hcup-us.ahrq.gov/ toolssoftware/comorbidity/comorbidity.jsp). 15. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care 1998; 36:8-27. 16. Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/ Deyo methods of comorbidity measurement in administrative data. Med Care 2004;42: 355-60. 17. Li B, Evans D, Faris P, Dean S, Quan H. Risk adjustment performance of Charlson and Elixhauser comorbidities in ICD-9 and ICD-10 administrative databases. BMC Health Serv Res 2008;8:12. 18. Pine M, Jordan HS, Elixhauser A, et al. Enhancement of claims data to improve risk adjustment of hospital mortality. JAMA 2007;297:71-6. 19. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. JAMA 2007; 298:984-92. 20. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA 2007;298:975-83. 21. Jha AK, Orav EJ, Li Z, Epstein AM. The inverse relationship between mortality rates and performance in the Hospital Quality Alliance measures. Health Aff (Millwood) 2007;26:1104-10. 22. Weller WE, Rosati C, Hannan EL. Relationship between surgeon and hospital volume and readmission after bariatric operation. J Am Coll Surg 2007;204:383-91. 23. Ly DP, Jha AK, Epstein AM. The association between hospital margins, quality of care, and closure or other change in operating status. J Gen Intern Med 2011; 26:1291-6. 24. United States Senate. HR 3590: The Patient Protection and Affordable Care Act, P.L. 111-148, 124 Stat. 353, Sec. 3001. 25. Centers for Medicare and Medicaid Services. Roadmap for implementing value driven healthcare in the traditional Medicare fee-for-service program. Washington, DC: Department of Health and Human Services, 2011. 26. Federal Register. Medicare programs: hospital inpatient value-based purchasing program: proposed rule. 42 CFR parts 422 and 480. 2010 (https://www.federalregister.gov/articles/2011/05/06/2011-10568/ medicare-program-hospital-inpatientvalue-based-purchasing-program). 27. Werner RM, Bradlow ET. Relationship between Medicare s hospital compare performance measures and mortality rates. JAMA 2006;296:2694-702. [Erratum, JAMA 2007;297:700.] Copyright 2012 Massachusetts Medical Society. n engl j med 366;17 nejm.org april 26, 2012 1615