Physician Response to Pay-for-performance Evidence from a Natural Experiment

Transcription

1 Physician Response to Pay-for-performance Evidence from a Natural Experiment Jinhu Li 1,2 Jeremiah Hurley 1,2,3 Phillip DeCicca 1,2 Gioia Buckley 2 Draft April 20, Department of Economics, McMaster University 2 Centre for Health Economics and Policy Analysis, McMaster University 3 Department of Clinical Epidemiology and Biostatistics, McMaster University Abstract: Over the past decade, the province of Ontario, Canada has introduced several "Primary Care Reform" models in an attempt to attract primary care physicians away from the traditional fee-for-service payment method and to a blended payment system. As part of this blended payment method, the provincial government has instituted a series of incentive payments to physicians to encourage the delivery of specific services to targeted patient populations. This study assesses the impact of eleven selected primary care incentive payments on the provision of targeted services and examines whether physicians responses differ by physician age, practice size and baseline compliance level. We exploit this natural experiment which creates cross-sectional and time-series variations in exposure to each incentive and compare the behavior of physicians exposed to the incentives against those not exposed throughout the study period. We use an administrative data source which covers the full population of the province of Ontario and nearly all the services provided by practicing primary care physicians in Ontario. With an individual-level data set, we employ a differencein-differences approach that controls for both selection on observables and selection on unobservables that may cause spurious relations between the provision of P4P incentives and the changes in physician behavior. We also implemented a set of robustness checks to control for the confounding effect of the other attributes of the Primary Care Reform models. The physician responses to the incentives are mixed. In general, out results suggest that physicians responded to the bonuses for preventive care services but not the special payments. Overall, the results provide a cautionary message regarding the effectiveness of pay-for-performance schemes for increasing quality of care. 1

2 1. Introduction Explicit financial incentives, especially pay-for-performance (P4P) incentives, have been extensively employed in the last few decades by health plans and governments in an attempt to improve the quality of health care services. A typical P4P program offers financial rewards to health care providers for meeting pre-established targets for the provision of specific health care services or for high marks on a set of quality measures. The provision of these explicit financial incentives aims to motivate health care providers at the margin within a specific compensation system. A variety of P4P programs have been established in several countries. In the United States, as of 2005 at least 100 nationwide P4P initiatives had been sponsored by a variety of health plans, employer coalitions and the Centers for Medicare and Medicaid Services (CMS) (Baker and Carter 2005). Originally the majority of the P4P programs were targeted at primary care physicians affiliated with Health Maintenance Organizations (HMO). Since 2004 there has been significant expansion of the programs to specialists and hospitals with more sophisticated measures for performance assessment (Rosenthal and Dudley 2007, Baker 2004, Baker and Carter 2005). In the United Kingdom, the British National Health Services (BNHS) introduced a pay-for-performance contract for family practitioners in 2004 which linked increases in physician income to performance with respect to 146 quality indicators covering clinical care for 10 chronic diseases, organization of care and patient experience (Doran et al. 2006). More generally, incentive programs have being used in Canada, Australia, Haiti and some other nations (Frolich et al. 2007). Using P4P programs to stimulate the health care providers behaviour is controversial. Advocates believe that P4P programs can be very powerful in improving the process and outcomes of health care. Decades of reforms of payment mechanisms have had little impact on reducing the deficiencies in health care delivery and this has led to the gradual employment of explicit P4P incentives to link financial gains and losses to quality indicators at the margin (Maynard 2008). The belief is that, as soon as the payments are at least partly contingent on indicators of provider effort to deliver high-quality care, P4P programs will induce providers to improve health care quality (Rosenthal and Frank 2006). Just as Mark McClellan, former Administrator of the Centers for Medicare and Medicaid Services (CMS), put it, You get what you pay for. And we ought to be paying for better quality (quoted in The New York Times, 2/22/06). However, critics argue that P4P programs are not as effective as commonly claimed by advocates and often create unintended consequences. The effectiveness of P4P programs is contingent on the magnitude of the payments but current P4P incentive programs are often with small-scale arrangements thus the payments are 2

3 insufficient to generate a response (Rosenthal and Frank 2006). Some argue that without careful design of the incentive structure, P4P programs will not effectively improve the quality of care because the incentives reward physicians for what they already do or should already do (Christianson et al. 2008; Lewis 2009). Some argue that even if we could observe a change in physicians behavior, it doesn t mean that physicians are truly responding to the incentives in a desired way since physicians may game the system when the P4P scheme mostly relies on physician reporting (Richards 2009). Moreover, some P4P programs create side effects such as provider focus on the clinical outcomes subject to incentives to the neglect of other aspects of care and patient selection (e.g. by turning away patients who could bring down their averages) (Rosenthal and Frank 2006; Hutchison 2008; Shen 2003). The rationale for employing P4P incentives to induce desired physician behaviour comes primarily from implications derived from principle-agent theory and incentive contract theory. The classic principal-agent and incentive contract theories have investigated how payfor-performance can be used to elicit desired behaviours from individuals when the effect of the incentive is constrained by the noisiness of the performance measures, the extent to which the performance is easily monitored, the ability of agents to handle risk, and the extent to which the desired behavior consists of multiple tasks (Prendergast 1999; Baker 1992; Hart and Holmstrom 1987; Milgrom and Roberts 1992; Stiglitz 1974). The existing health economics literature models the health care provision behavior of physicians in response to different aspects of financial incentives. Physicians respond to changes in fees or other aspects of payment schemes because they maximize their profits or seek a target income (McGuire and Pauly 1991). Models are also developed to discuss alternative payment schemes or different aspects of contractual arrangement that can be used to stimulate or regulate physician behavior in the presence of uncertainty of health services, information asymmetry between patients and physicians, and noncontractible effort of physicians. The optimal payment structure in most contexts is a mechanism that blends prospective and retrospective incentives (Robinson 2001), in particular is a mixture of capitation and fee-for-service payments for physician services (Newhouse 1996). Other aspects of contractual incentives can also be embedded within alternative payment modalities to induce the desired physician behavior in service provision (Ma and McGuire 1997; Blomqvist 1991). These theoretical pieces indicate that performance incentives can provide us some scope of improving the quality of care, despite the special characteristics of the institutional arrangements in health care market. However, the extent to which physicians respond to explicit P4P incentives and the effectiveness of the use of these P4P incentives are still unclear. First of all, although carefully designed performance incentive scheme is beneficial in terms of better resource allocation, the payment incentives are often criticized 3

4 for their limited ability to resolve the complex and conflicting sets of problems in health care (Robinson 2001). Besides the pecuniary factors, non-pecuniary factors including professional and social status, altruistic or ethical concerns, also influence the preferences of physicians (Scott 2001; Eisenberg 1985; Eisenberg 1986). As a result, physicians may also be subject to additional ethical constraints, or they face different types of trade-offs other than only earnings-related incentives when they make their choices of practice. Moreover, most of P4P incentives in health care are embedded within a complex environment in which physician behavior are affected simultaneously by local market and social environment, health plan characteristics and provider organization characteristics (Conrad and Christianson 2004; Frolich et al. 2007). The complicity of organizational structures imposes further constraints or hidden costs to physicians thus may further inhibit them from responding. Existing empirical studies on the effects of P4P incentives are rare in spite of its popular use in current health care system. Only a handful of empirical studies including both RCTs and observational studies provided direct evidence of how targeted performance incentives influence physician delivery of targeted services. The findings from these studies are mixed. Eight out of the thirteen studies identified by our systematic search found partial effect of P4P incentives in sense that, physicians respond to some of the incentives but not the others, while for the subset of incentives which did improve the performance, the magnitude of the improvement is modest. Three out the thirteen studies found significant positive effect while the other two studies found no effect of rewarding physicians with performance incentives at all. The evidence from these studies is tentative because they often suffer from poor study design. For example, the two studies (Doran et al. 2006; Campbell et al. 2007) drawn from the nationwide P4P program introduced by Britain s National Health Service in 2004 are based on only cross-sectional analysis or only before-after analysis without control group. Moreover, the small scale of the experiments conducted in RCTs and some real world P4P programs makes it difficult to identify reliable results from these studies. Therefore, the results from this empirical literature are often difficult to extrapolate to other settings thus far more inconclusive to provide policy implications to guide further implementation of P4P incentives. We will discuss these studies, and others, in more details in the following section. This study exploits a natural experiment in the province of Ontario, Canada to empirically identify the impact of pay-for-performance (P4P) incentives on the provision of targeted primary care services. The P4P scheme rewards GPs when they achieve targeted levels of service provision. Primary care reform in Ontario provides a natural experiment that allows us to employ a difference-in-differences approach that controls for potential problems that may cause spurious relations between the provision of P4P incentives and the changes in physician behaviour. Specifically, we assess the incentive effects by comparing the group of 4

5 physicians that were affected by the incentives and the group of physicians that were not affected by the incentives in both pre- and post- intervention periods. We exploit an administrative data source which covers the full population of the province of Ontario and nearly all general practitioners. The administrative databases include detailed information on services provided that constitutes over 98% of all physician activity. The population-based nature allows us to assess the P4P incentive effects for a large physician population, while the rich content of the data allows us to address a variety of potential biases that are caused by selection on observables. Also the longitudinal nature of the data allows us to partially control for potential bias that are caused by selection on unobservables by using a set of panel data models that allow for correlation between the P4P incentive treatment and unobserved individual-specific or trend-specific components. The universal public insurance system in Canada provides a good setting for studying the P4P incentives effects. Unlike the U.S. health care system, almost all of the primary care services in Canada are funded publicly and compensated by a single payer. These characteristics make it easier to disentangle the P4P incentive effects from other confounding effects. For example, studies based on U.S. data often cannot easily eliminate the effects of other simultaneous interventions because most physicians contract with multiple health plans. As Robinson noted (Robinson 2001), comprehension and compliance to any payment mechanisms will be undermined when physicians face different incentives from multiple insurers. Therefore, the estimates of the P4P incentives from the US studies are expected to be biased towards zero. Moreover, given that all the existing studies on P4P incentives are based on programs implemented in other countries, this study is a good complement to the literature of empirical evaluations on P4P programs and a valuable addition for international comparisons on performance compensation reforms. Since we also attempt to identify the context in which the P4P incentives may exert larger influence on physician behaviour, our study examines the heterogeneity of the P4P incentive effects across different physician types and different practice characteristics. We hypothesize that the impact of P4P incentives is heterogeneous because both the benefit of responding to P4P incentives and the cost of responding at the margin is likely to be different across different procedures and different practice characteristics. Specifically, we compare the incentives effects across physician age, across practices that differ in patient population size, and across practices with different baseline levels of compliance. This shall help shed some light on understanding the factors that could enable or mitigate the effect of P4P incentives in shaping physician behaviour. 5

6 This paper proceeds as follows. Section 2 provides a review of the empirical evidence on physician responses to financial incentives and particularly responses to explicit pay-forperformance incentives. Section 3 describes the analytic framework of our study including the institutional background, data source, identification strategies and empirical methods adopted. Section 4 specifies our study sample and describes the statistical properties of the study sample. Section 5 presents the regression results and Section 6 discusses our analysis and concludes the paper. 2. Empirical evidence on physician response to P4P A large body of empirical studies has examined the effect of financial incentives on physician behaviors. There is considerable evidence that physicians respond to the incentives embedded in different payment schemes (Hurley et al. 1990; Yip 1998; Nguyen and Derrick 1997; Hickson et al. 1987; Krasnik et al. 1990; Scott and Shiell 1997). There is less evidence on physician behaviour in response to explicit financial incentives in the form of targeted performance payment, and the evidence is far more mixed regarding the use of these incentives to guide targeted behaviours. Several recent survey papers (Rosenthal and Frank 2006; Christianson et al. 2008; Petersen et al. 2006; Town et al. 2005; Armour et al. 2001) have conducted systematic reviews of studies regarding the effects of pay-for-performance schemes and targeted financial incentives more generally. They have demonstrated that the current evidence in term of the effect of paying-for-quality is mixed and inconclusive. In addition to some empirical studies reviewed in these surveys, our systematic search of the literature identified 13 empirical studies that provide direct evidence of how targeted performance incentives influence the delivery of targeted services by physicians (See Appendix 1 for the identified empirical studies). Among the 13 studies, 7 of them are based on Random Control Trials (RCTs) and the remaining 6 are based on observational studies. These RCTs conducted experiments on small groups of physicians to examine the effects of alternative forms of performance incentives, such as bonus, bonus based on capitation payment, bonus with performance feedback, on the provision of targeted services by physicians. In most RCTs, the incentives are mostly targeted on preventive care services including, influenza immunizations, mammograms, Pap smear, colorectal screening and pediatric immunization. 6

7 The results from the RCTs are mixed. Three studies (Grady et al. 1997; Hillman et al. 1998; Hillman et al. 1999) didn t detect any significant effect of the P4P bonus rewards or bonus rewards combined with performance feedback on the physician compliance of a set of cancer screening practices, pediatric immunization and mammography referrals. Two studies (Fairbrother et al. 1999; Fairbrother et al. 2001) found when bonus or bonus with performance feedback incentives were provided to physicians, the coverage levels of childhood immunization rates increased significantly, but the increase is primarily due to better documentation not due to better immunizing practices. The study on using bonus smoking cessation clinics (Roski et al. 2003) found a significant improvement of documentation of smoking status and providing advice to quit, but no effect in the actual quitting rate. Only one RCT showed a significantly positive effect of using bonus payment that increased the influenza immunization rates by 7 percent. RCTs are often deemed as the gold standard to identify the causal effects, but the results from these RCTs cannot easily be generalized or extrapolated. All of these RCTs are based on small scale of the experiments which consist of fewer than a hundred physicians or practices. One study (Fairbrother et al. 2001) involved only 24 physicians who were subjected to P4P incentives. It is nearly impossible to identify statistically meaningful results based on such small sample sizes. Moreover, the design of the RCTs often makes it impossible to disentangle the pure P4P financial incentives effects from other quality management tools. Among the seven RCTs, two studies (Hillman et al. 1998; Hillman et al. 1999) bundled the bonus payment with performance feedback regarding compliance level, while one study (Grady et al. 1997) bundled financial reward with the provision of education in the form of chart reminder stickers. The observational studies are mostly based on small to large scale of pilot pay-forquality programs or quality-improvement initiatives adopted by health plans in US and UK. Unlike the RCTs which mainly focus on a small set of targeted preventive care services, some of these programs covered broader scope of quality indicators in addition to the preventive care services. A study from U.K. (Doran et al. 2006) evaluated the effect of the nationwide P4P program introduced by Britain s National Health Service in 2004 for family practitioners in the first year of this program. The program linked increases in income to performance with respect to 146 quality indicators covering clinical care for 10 chronic diseases, organization of care, and patient experience. The result from this study showed English family practices attained high levels of achievement of meeting the quality indicators, as the median reported achievement was 83.4 percent. But this study is based on a cross-sectional analysis so it only 7

8 established an association between high level of reported achievement and the P4P contracting, not the real effect of the P4P incentives. As Campbell et al. noted (Campbell et al. 2007), since a wide range of initiatives including limited use of incentive programs had been introduced since 1990, the high levels of quality attained after the 2004 contract might just reflect improvements that were already under way. Another U.K. study from Campbell et al. (2007) drew from the same intervention but used a better study design to examine the effect of 2004 P4P contracting on the quality of care. The quality indicators of three chronic conditions: asthma, coronary heart disease, and type 2 diabetes for some representative groups of general practitioners were measure two times before the P4P contracting (1998 and 2003) and one time after the contracting (2005). A before-after analysis which compared the quality score predicted by trend and the observed quality score in 2005 showed that the introduction of pay for performance in 2004 was associated with a modest acceleration in improvement for two of these three conditions: diabetes and asthma. However, since the P4P contracting is offered to all general practitioners in the U.K., this study couldn t provide a plausible control group in the before-after analysis so it cannot identify the causal effect of P4P incentives. Two U.S. studies examined the P4P incentive effects based on much smaller scale of P4P programs. One study (Amundson et al. 2003) found positive effect of using bonus with performance feedback in improving physician compliance with the tobacco treatment guidelines. But this study did not use any control group thus cannot provide reliable result of the P4P effects. The other study (Beaulieu and Horrigan 2005) examined the effect of performance bonus with group discussion process on the improvement of nine process and outcome measures for diabetic care. The results showed significant improvement on seven out of the nine measures. However, the treatment group in only consists of 21 physicians so it is difficult to draw reliable inference from this study. Moreover, both studies are looking at performance bonus bundled with other management tools so it is impossible to separate out the P4P incentive effects. The best evidence to date on the effects of P4P programs are from two observational studies in the U.S. drawn from the P4P initiatives introduced by a large network Health Managed Organization (HMO): PacifiCare Health Plan. The first study (Rosenthal et al. 2005) examined the effect of Quality Incentive Programs (QIP) provided by PacifiCare Health Plan to medical groups in California in 2002 on physician delivery of cervical cancer screening, mammography and haemoglobin A1c test. It used a difference-in-difference design by comparing provider groups in California which were affected by these incentives with provider groups in the Pacific Northwest which were unaffected by the incentives but also contracted with PacifiCare Health Plan. It found that outcomes improved significantly for 8

9 cervical cancer screening with an increase of 3.6 percentage points, but did not improve for mammography and the haemoglobin A1c test. The second study (Mullen et al. 2009) built on the first paper and examined the effect of QIP incentives along with another larger P4P program by the Integrated Healthcare Association (IHA). It also concluded that the P4P incentive effects are mixed. In line with the previous study, the analysis found evidence of a positive effect only for cervical cancer screening, but not for mammography, haemoglobin A1c test and asthma medication. In fact, the study found evidence of possible negative effects of these P4P programs on asthma medication and antibiotic usage because the substitution of resources away from these two rewarded services to other more lucrative rewarded measures. 3. Background, Data Source and Study Design As noted above, this study draws on a natural experiment in Ontario Canada with respect to general practitioners (GP) funding to address the following questions: 1) does P4P stimulate the delivery of targeted health care services by general practitioners? 2) Are P4P incentives effects heterogeneous across physician and practice characteristics? 3.1 Institutional background: The natural experiment Over the last 2 decades, the province of Ontario in Canada has launched a series of primary care renewal (PCR) models to improve the quality of health care. The main purposes of implementing the PCR models include 1) improve quality by providing pay-forperformance incentives to stimulate the delivery of targeted health care services; 2) transforming the traditional fee-for-service payment scheme to a blended payment method; 3) integrating primary care physicians, nurses and other professionals into more collaborative, multidisciplinary teams (Wilson 2006). The set of P4P incentives that we focus on includes bonuses for five preventive care services: Pap smears, mammograms, flu shot for seniors, toddler immunizations, and colorectal cancer screening; and six special payments: payments for obstetrical deliveries, hospital services, palliative care, office procedures, prenatal care, and home visits. The five bonuses are essentially service enhancement payments for preventive care which include two components, the contact payment and the cumulative preventive care bonus payment. The contact payment rewards PCR practices for contacting patients to schedule an appointment to receive a targeted preventive service. The cumulative preventive care bonus payment rewards PCR practices for achieving high rates of coverage for the preventive services in the target 9

10 populations appropriate to each service. The six special payments reward physicians for providing more services in the six areas of particular interest to the MOHLTC. Physicians received the incentive payments based on eligibility conditions that can be described by three characteristics: the target population, the time period and the threshold coverage levels. For the five bonus payments on the preventive care services, the PCR practice receives a contact payment of $6.86 for each eligible patient in the target population for a designated service and for which it provides the Ministry the required documentation. They also receive cumulative bonus payment on March 31 each year based on the proportion of its physicians eligible and rostered patients in the target population who had received the targeted service over a specified period of time. Physicians receive a certain amount of money if the proportion reaches to a pre-specified coverage threshold while the payment grows as the proportion exceeds higher thresholds. For example, if 60% of a physician s rostered female patients in the age of 35 to 69 received a pap smear for cervical cancer screening during the last 30 months as of March 31, this physician is rewarded 220 dollars. If 65% of this eligible patient population received a pap smear, this physician receives 440 dollars. The physician is compensated with 660 dollars, 1,320 dollars and 2,200 dollars for a coverage rate of 70%, 75% and 80%, respectively. For the six special payments, a physician received a fixed payment if the targeted service was delivered to a minimum absolute level of service provision, where that minimum is defined in terms of number of services, dollar value of services, number of patients, or a combination of these factors during the preceding fiscal year. For instance, if 5 or more obstetrical services were delivered to 5 or more patients in a fiscal year, a physician receives a fixed payment of 3,200 dollars (with an increase to 5,000 dollars since October 2007). The details of the other bonuses and special payments are listed in Table 1. Physicians who remained in fee-for-service practices were never eligible to receive these P4P incentives. Physicians who converted to one of the Primary Care Renewal models were exposed to these P4P incentives when the P4P incentives were in effect for their specific PCR model. The eligibility periods for the P4P incentives differ by the PCR model types and by the targeted service types. During our study period ( ), the four relevant PCR models are the Family Health Network model (FHN), the Family Health Group model (FHG), the Comprehensive Care Model (CCM) and the Family Health Organization (FHO). Table 2 presents the eligibility timing for the eleven targeted services by PCR models types. As the P4P incentives are available only to physicians in the PCR models, the policy intervention in Ontario serves as a natural experiment that we can exploit to identify the casual effect of P4P incentives. Since we can observe the practice activities of almost every GP in Ontario over 10 years and because this period spans the process of P4P incentives implementation, we can 10

11 assess the impact of P4P incentives within a difference-in-differences framework by comparing the outcome measures of the GPs not exposed to the P4P incentives with the GPs exposed to the P4P incentives. The Ontario Ministry of Health and Long-Term Care (MOHLTC) introduced these PCR models at different points of time. The earliest model introduced among the four PCR models in our study is the FHN which existed as early as 2002 and requires group practices comprised of at least 3 GPs. Funding for FHNs is a blended system of capitation for core services provided to rostered patients and fee-for-service for both non-rostered patients and for non-core services excluded from the basket of capitated services. FHGs were introduced in 2003, also required a group of 3 or more GPs and the basic payment scheme is enhanced fee-for-service. CCM model was introduced in 2005 and operates the most similar to traditional FFS practice. CCMs are tailored to family physicians who prefer to remain in a solo practice and to be compensated primarily on a fee-for-service basis. FHOs were introduced in Like FHNs, and FHGs this model requires groups practice comprised of at least 3 GPs and employs a blended capitation payment scheme. At the beginning of our study period in April 1998, all but a few hundred primary care physicians in Ontario were in the traditional fee-for-service practice, while at the end of our study period, more than half of these GPs converted to one or more of the primary care reform models. Table 3 lists the main characteristics of these four models. Unlike traditional fee-forservice practices, all of these four PCR models are entitled to receive bonuses, special payments, and several additional small lump sum payments and premiums 1. The four PCR models are also different from the traditional fee-for-service practices in the following aspects: general payment scheme, practice composition, after-hour services and patient enrolment requirement (see Appendix 2). Firstly, traditional FFS GPs are paid by FFS, while the PCR models are paid mostly by a blended scheme. Most of the PCR model GPs work in group practice or interdisciplinary teams while most FFS GPs work in solo practice. Also PCR model GPs have to provide extended services, nurse-staffed telephone health advisory services and on-call services. Lastly, patient enrolment is required in these PCR models except for FHGs but not for FFS GPs. Such differences between the PCR models and the FFS practice raise the issue of how similar the GPs who were not exposed to the incentives are compared to those who were exposed, and whether the behaviour of the FFS GPs represents a good counterfactual of the 1 It should be noted that the cumulative bonus payments are paid to the physician s practice for GPs in a FHN, while GPs in FHGs, CCMs and FHOs receive the bonus payments directly. The special payments are paid to the individual physician directly in all models. 11

12 treated GPs. The treated GPs may be prone to respond to the P4P incentives anyway due to some reasons which have made them choose to participate in the PCR models at the first place. Furthermore, different practice settings between FFS and PCR models may generate different responses in delivering the targeted services from the two groups even in absence of P4P incentives. As described in the study design part below, a careful control for the institutional differences between the FFS and PCR models is crucial to reduce the selection bias and confounding effect in the identification. Nonetheless, the differences in various aspects among different PCR models themselves provide us an opportunity to examine more refined hypotheses. 3.2 Data Sources Our study mainly draws on four administrative databases of the Ontario Ministry of Health and Long-Term Care (MOHLTC), linked by patient encrypted health number and provider encrypted number. A claims database that contains service-specific records allowed us to identify all the publicly funded physician services obtained for every person in Ontario. Because 98.5% of the primary care in Ontario is publicly financed, we observe essentially all primary care services. This data plus a roster database allowed us to match every person to a practice. A registered person s database which provides basic information on each OHIP beneficiary allowed us to identify the age and sex of each patient and if this beneficiary should be counted towards the targeted population for each incentive payment. A provider database of all licensed physicians in Ontario allowed us to identify the type of practice a GP is in at each point of time during our study period. Taken together, we constructed for each physician in the province, a measure of their practice population each year and a record of all services received by those patients during the period of 1999 to 2008 fiscal years. (See Appendix 3 lists all the data sources that we used and the correspondent information that we extracted from each source). 3.3 Identification strategy for the incentive effect As described above, the policy intervention in Ontario serves as a natural experiment that we can exploit to identify the casual effect of P4P incentives, because it provides P4P incentives only to physicians in the PCR models but not the FFS physicians. The treatment of interest is a set of P4P incentives targeted on eleven specific health care services or sets of services. Specifically, this policy intervention conditions the eligibility of the P4P incentives on the PCR model-participation status. A simple difference-in-differences approach can 12

13 provide us an estimate of the P4P incentive effects by directly comparing the mean change across the PCR model GPs and the FFS GPs. However, the non-random assignment of GPs to treatment that is generated by the voluntary participation process invalidates this simple difference-in-differences approach (Meyer 1995). In other words, we expect that the treated GPs are systematically different from the non-treated ones and these differences may contribute to the observed difference in the response of GPs to P4P incentives. Therefore, the identification of causal effect hinges on how well the selected comparison group represents the counterfactual of the treatment group, and the extent to which we could mitigate the selection bias. We expect that the treated GPs who were exposed to the P4P incentives are somewhat different from those not exposed at the first place because GPs self-selected into the PCR models for good reasons. As GPs who converted from FFS to PCR models need to comply with specific rules in their practice, it is reasonable to assume that GPs whose complying costs are relatively smaller tend to participate in the PCR models. Accordingly, we expect that the PCR GPs are different from the remaining-in-ffs GPs in the following ways. First, we expect that the PCR GPs are younger thus more flexible in practice style and more than the FFS GPs. Second, it is likely that PCR GPs are more interested in or better at team production than FFS GPs because most PCR models require work in group practice instead of solo practice. Third, we expect that PCR GPs are more flexible in working time or they have a lower preference on leisure over after-hour work since they are required to provide extended after-hour services, nurse-staffed telephone health advisory services and on-call services. These differences in physician characteristics might pose some difficulties for the identification of the incentive effects. We employ several identification strategies to mitigate the selection bias that may be generated by both observable and unobservable physician characteristics. First of all, we control for important aspects of physician characteristics and practice characteristics that might be correlated with the self-selection process and are also important in determining the provision of the targeted services. The data allows us to control for physician characteristics including physician demographics, work experience, and work load measures; and practice characteristics including practice size, geographical location of the practice and patient population characteristics of the practice. Secondly, to address selection bias generated by unobservable characteristics, we exploit the longitudinal nature of our data and employ a difference-in-differences approach with individual fixed effects. The eligibility timing of the P4P incentives in the PCR models facilitates the reduction of selection bias as soon as we carefully select our comparison and treatment groups. The policy intervention provided the P4P incentives to different PCR 13

14 models in different time periods, but it created essentially three types of physicians groups: non-incentive group, incentive group 1 and incentive group 2 (see Figure 1). The nonincentive group consists of the GPs who remain in FFS over our study period. Since they were never eligible for the incentives, they are used as the legitimate control group in the difference-in-differences design. The Incentive group 1 consists of the GPs who joined a PCR model and simultaneously became eligible for the P4P incentives. This group of physician can be used as part of the treatment group but this is problematic because the identification of the P4P effect suffers from the selection bias. Given the participation in PCR models is a voluntary process, the P4P incentive effect is perfectly confounded by the selection into the PCR model for this group of physicians. The Incentive group 2 consists of the GPs who became eligible for the P4P incentives only after they had participated for some time in a PCR model. This group of GPs pertains to the majority of physicians who were entitled with the incentives in our study. Using this group of physicians as the treatment group can mitigate the problem of selection because these physicians chose to participate in PCR unaware of the P4P incentives yet. Since the incentive effect is not perfectly confounded by the selection into the treatment, the selection bias can be reduced to the extent that the unobserved components that determine both the self-selection behaviour and the outcomes are physician-specific and time-invariant, and can be differenced out by a difference-in-differences approach with individual fixed effects. The identification of the above approach is based on the assumption of a parallel trend between treatment and control groups, but it is very unlikely that the treatment group and the control group GPs are subject to the same time trend over 10 years. In order to control for the uncommon time trends across treatment group and control group, we use the difference-in-differences adjusting for differential trends approach as suggested by Bell, Blundell and Reenen (1999). This model relaxes the assumption of parallel trends between the control and treatment group GP when these differential trends have different impact on the outcome between P4P system and non-p4p system. Other than the fairly restrictive parallel trend assumption (PTA), the limitation of the above approach comes from the lack of control for unobserved temporal individual-specific component that affected the selection into the treatment group and control group (Blundell and Costa Dias 2000). This could be a problem if some GPs self-selected into PCR models because of temporary shocks that are directly related to the targeted health care services. However, this should not be a big concern in our study for the following reasons. Firstly, participating into a PCR model is unlikely to depend on short-term changes that affect the utilization rates of the targeted services, such as a sudden demand-side change or an onset of other simultaneous policies that are targeted to these specific services. The monetary values 14

15 of these P4P incentives are a relatively very small proportion of the total income of GPs. So it is unlikely that any temporary changes related to the targeted services caused the conversion behavior. This assumption is reinforced by the fact that only a very small proportion of GPs who converted from FFS practice to PCR models switched back to FFS practice during our study period of ten years. Secondly, any unobserved temporary shocks that are correlated with PCR participation should not play an important role in determining the utilization of the specific services that are targeted by P4P incentives, because most of the our treatment group GPs already converted for some time before they were eligible for the P4P incentives. Hence, the incentives are unlikely to be the reason that has motivated the conversion behavior in any event. One might also argue that it is possible that some of the observed differences in response to P4P incentives between treatment group and controlled group are actually caused by other unobserved attributes pertaining to the PCR practice instead of the specified P4P incentives. For example, an important institutional difference between some PCR and FFS practice is that these PCR models are paid by a mixture of FFS and capitation instead of traditional FFS piece rate. One might expect that FFS physicians respond less to P4P bonus related to preventive care services because the opportunity cost may be greater for FFS physicians than for physicians paid by capitation or salary in the sense that doing more immunizations may preclude the provision of other services that generate higher fees per unit time. We argue that this confounding effect should not be a big concern because we could mitigate potential confounding effects by conducting subgroup analysis with different treatment groups. This approach is possible for our study since we can exploit the variation on several dimensions across different PCR models to examine the effects of the specific confounders. To eliminate the possibility that the difference in general payment scheme is causing the difference in response, we restrict our treatment GPs as those PCR GPs who were also compensated by fee-for-service and compare their behaviour with the remaining-in-ffs GPs. If we still observe the difference in response, we can conclude that it is likely not the payment scheme causing the observed P4P incentive effects. 3.4 Subgroup analyses Another focal point of our study is to examine the heterogeneity of the P4P incentive effects across different physician types and different practice characteristics. Physician age, practice size and the baseline compliance level are three potential factors that could reinforce or mitigate the power of the P4P incentives. Younger physicians are expected to respond more to the P4P incentives because it is relatively easier for younger physicians to set up their 15

16 practice in a way that is more compatible to the P4P incentive programs. Presumably the practice size could also alter the P4P incentive effects and we have competing arguments for the direction of the gradient. On one hand, bigger practices are likely to be administratively set up in a more systematic way to handle these incentives so physicians having bigger practice sizes tend to be more responsive. On the other hand, the target levels are easier to reach by physicians with smaller practices because the denominator is smaller, and since the financial reward associated with the bonus represents a higher proportion of income for a physician with smaller practices, so physicians having smaller practice sizes tend to respond more. Also, the specific structure of an incentive schedule matters in generating the desired responses. The recent literature of P4P incentives indicates that the target-based P4P incentive structure always discourage very low performers and very high performers from improving (Mullen et al. 2009). So the power of the incentive is largely related to the specific design of incentive schedule and the difference between the initial level and the targeted level of provision. 3.5 Empirical framework and methods Sample For each incentive, we conduct a difference-in-differences analysis. The unit of analysis is a physician. Our study period is a 10-year period beginning April and ending March In order to identify GPs who work in a typical community-based practice providing family medicine and who do not specialize, we used to following criteria to select our study sample: include physicians who are general practitioners throughout the study period; excluded part-time FPs who billed less than $30,000 each year; to limit our study sample to GPs in an established practice, we only included physicians who had at least two consecutive years of practice before study period; include GPs for whom the office-based consultations accounting for the majority of their activities; exclude locums as they are not eligible for bonuses. The sample size changes for each criterion are listed in Table 4. After we implement these selection criteria, we obtain a core sample of 2,204 GPs. In most cases, physicians switched from FFS to PCR practice and later became eligible for the P4P incentives when the P4P incentives were provided to that PCR model. But a very small proportion of physicians switched back and forth between the non-incentive practice and the incentive-practice 2. For the simplicity of the analysis, we dropped these 2 This might be switching back and forth between FFS practice and a PCR model, or switching back and forth between a PCR model which was eligible for the incentives and another PCR model which was not eligible for the incentives yet. 16

17 physicians whose treatment status turned on and off for more than one time during our study period. This led to a further reduction of the study sample. Since the eligibility scope and timing for the eleven P4P incentives are different for the four PCR models, the composition and the sample size of the treatment and control groups vary among these P4P incentives. The definitions of control and treatment groups as well as their sample sizes are presented in Table 5 divided into three subsets of targeted services. The implementation dates are different in general for the five bonuses and the six special payments. As a result, most of the P4P GPs in our study became eligible for the five bonuses within one year while most P4P GPs became eligible for the six special payments gradually in three consecutive years. Accordingly, in the regression analysis we had one group of GPs constituting the treatment group for the five bonuses but three groups of GPs constituting three treatment groups for the six special payments based on the timing of their eligibility. Moreover, as the five bonuses were provided to all four PCR models considered in our study while the six special payments were provided to only some of the PCR models (e.g. FHNs and FHOs), there are much fewer GPs constituting the treatment groups in the analysis for the six special payments than for the five cumulative bonuses. As a result, we could conduct a more complete set of regression analyses with the five bonuses but not with the six special payments because the small sample size of the treatment group for these special payments does not allow us to run more complicated models and to test more refined hypotheses by subgroup analysis Dependent Variables The outcome variables are physician-specific utilization rates or compliance level for each type of service under study. These rates were calculated using a 3-step process. Firstly, we assigned all patients in the Ontario Health Insurance Program (OHIP) physician claims database to a GP and thus defined a practice population for each GP on March 31 of each year of our study period. Physicians in traditional FFS practice do not roster patients so we defined the practice population for these physicians using the methodology developed in Hutchison et al. (Hutchison, Hurley et al. 1997). Specifically, a physician s practice population is defined as: all individuals for whom the physician billed OHIP for at least one visit during the previous fiscal year; and all additional patients for whom the physician billed OHIP for at least one visit in each of the two preceding fiscal years 3. Physicians participating in PCR models have both rostered (the majority) and non-rostered patients. For this case we define 3 Patients who met these criteria for more than one physician were assigned to the physician who billed for the largest number of visits; if the number of visits was equal, assignment was based on the physician with most recent visit. 17

18 the practice population as the set of rostered patients (as indicated by the Ministry Client Agency Program Enrollment database) plus non-rostered patients as assigned by the Hutchison et al. algorithm. As a result, all OHIP beneficiaries were assigned to a physician for each year of the study period. Secondly, we counted the services the eligible patient population received from each GP for each year. Specifically, each person in a physician s eligible practice population who received the targeted service during the relevant period is counted toward the physicians achieved coverage level. Lastly, we calculated the utilization rate or compliance level for each type of service for each GP in each year. The dependent variable for the analysis of each of the five preventive care bonuses is defined, as of March 31 each year, as the proportion of a FP s practice population that received the service in question during the relevant period prior to that March 31st 4. The dependent variable for the analysis of each of the special payments is defined dichotomously, taking on the value of 1 if the physician s service provision met the criteria for the special payment of interest, and 0 if it did not. The detailed outcome variables definitions are listed in Appendix Independent variables The data allow us to identify whether, and if so, when, a FP joined a PCR practice. From this information we constructed a treated and control dummy, a pre- and post- dummy and an interaction term equal to 1 when a GP is eligible for the incentives. We also drew on the administrative databases to extract and construct a set of covariates comprised of physician-specific variables, practice-specific variables and patient population variables. With our longitudinal data, we estimate a reduced-form specification on utilization rate, which is determined by both demand-side variables and supply-side variables. On the supply side, we control for a set of PCR models using a series of dummies that indicate GPs specific practice types. We also control for physician-specific characteristics including physician age and sex, work experience, and a set of work-load variables including days of work, number of patient visits and number of patient visits per working day. A set of practice- 4 This does not match exactly the criterion for whether a FP actually received a bonus: that criterion is defined in reference to rostered patients only, while our definition of an FPs practice population includes both rostered and non-rostered patients. We define the dependent variable as we do to ensure that we have a consistently defined measure both across all the years in the study period and across traditional FFS and PCR physicians. Because our analysis does not depend on accurately identifying who actually received a bonus, the differences between our definition and the bonus criteria as applied by the Ministry does not pose a problem. The best way to handle the Q tracking codes in the analysis is not clear. We are therefore conducting sensitivity analysis to put bounds on possible error from the fact that only PCR physicians could claim such codes for patients who received services in another setting, such as a flu-shot clinic. 18