Supplementary Appendix

Transcription

1 Supplementary Appendix This appendix has been provided by the authors to give readers additional information about their work. Supplement to: Timbie JW, Setodji CM, Kress A, et al. Implementation of medical homes in federally qualified health centers. N Engl J Med 2017;377: DOI: /NEJMsa

2 Medical Home Implementation in Federally Qualified Health Centers Methods Appendix Justin W. Timbie, Claude M. Setodji, Amii Kress, Tara Lavelle, Mark W. Friedberg, Peter Mendel, Emily K. Chen, Beverly A. Weidmer, Christine Buttorff, Rosalie Malsberger, Mallika Kommareddi, Afshin Rastegar, Aaron Kofner, Liisa Hiatt, Ammarah Mahmud, Katherine Giuriceo, Katherine L. Kahn RAND Health

3 Table of Contents List of Exhibits... iv 1. Conceptual Model FQHC APCP Demonstration Interventions... 3 Care Management Payments... 3 Technical Assistance (TA) for Implementing Practice Change... 3 Feedback Reports on Progress Toward PCMH Recognition... 4 Feedback Reports on Utilization, Quality, and Expenditures Selection of Demonstration Sites and Comparison Sites... 5 Eligibility for the Demonstration... 5 Selection of Demonstration Sites... 5 Selection of Comparison Sites... 7 Overview of Potential Comparison FQHCs... 7 Selection of Comparison FQHCs Beneficiary Attribution Attribution Methodology Accounting for Changes in Beneficiary Eligibility During the Demonstration Persistence of Attribution to Baseline Sites Medicare Claims Analysis Claims-Based Outcome Measures Utilization Measures Process Measures Medicare Expenditures Difference-in-Differences Model Specification Cumulative Difference-in-Differences Model Specification Dependent Variable Modeling Assumptions Covariate Adjustment Propensity Score Weighting Generating Estimates of Demonstration Impacts Using Recycled Predictions Year-by-Year Difference-in-Differences Analysis Cumulative Difference-in-Differences Analysis Assessing Parallel Trends Assumption Medicare Beneficiary Survey Data Collection and Analysis Medicare Beneficiary Survey Design Difference-in-Differences Model Specification Propensity Score Weighting Sampling Weights Nonresponse Weights ii

4 7. Qualitative Data Collection and Analysis Qualitative Methods Overview Nested Sampling Design Recruitment FQHC Site Leader Interviews Thematic Analysis PCA Leader Interviews Thematic Analysis Site Visits Beneficiary Cohort Characteristics and Balance Assessments Comparisons of Beneficiaries Attributed to Demonstration and Comparison Sites Cumulative and Year-By-Year Demonstration Effects Demonstration Effects Stratified by Beneficiary Time of First Attribution References iii

5 List of Exhibits Exhibit 3.1 Eligibility Requirements for FQHCs Seeking to Participate in the FQHC APCP Demonstration... 5 Exhibit 3.2 Number of Demonstration Sites Classified According to Five Characteristics... 6 Exhibit 3.3: Exclusions Applied in Identification of FQHC Comparison Sites... 9 Exhibit 3.4: Unweighted and Weighted Comparisons of Demonstration and Comparison Site Characteristics Exhibit 3.5. Analysis of Site-Level Propensity Score Balance Between Demonstration and Comparison FQHCs Exhibit 4.1. Reasons for Attrition from the Demonstration Exhibit 4.2. Loyalty to Baseline Site Exhibit 5.1: Outcomes Used in Medicare Claims Analyses Exhibit 5.2: Dependent Variables and Family, Link Function, and Eligibility Adjustment Used in Regression Model Specifications Exhibit 5.3 Covariates Used in Regression Models for Claims Analyses Exhibit 5.4: Beneficiary Cohorts Used to Produce Each Difference-in-Difference Estimate for Year-by-Year Analyses Exhibit 5.5: Parallel Trends Tests of Differences in Trends between Demonstration and Comparison site in the Two Years or One Year Before the Demonstration Exhibit 5.6: Trends in Difference-in-Differences Estimates of the Demonstration Effect on Medicare Expenditures during the Pre-Demonstration Period (Demonstration Sites vs. Comparison Sites) Exhibit 5.7: Trends in Difference-in-Differences Estimates of the Demonstration Effect on Inpatient and ED Utilization Measures during the Pre-Demonstration Period (Demonstration Sites vs. Comparison Sites) Exhibit 5.8: Trends in Difference-in-Differences Estimates of the Demonstration Effect on Non- ED Ambulatory Utilization Measures during the Pre-Demonstration Period (Demonstration Sites vs. Comparison Sites) Exhibit 6.1: Survey Response Rates Exhibit 6.2: Predictors of Responding to Both Waves of the Beneficiary Survey Exhibit 6.3 Covariates Used in Regression Models for Beneficiary Survey Analyses Exhibit 7.1. Qualitative Nested Sampling Design Exhibit 7.2. Qualitative Sample Selection Criteria and Sizes Exhibit 7.3. Timing of Qualitative Date Collection Exhibit 8.1: Characteristics of Demonstration and Comparison Beneficiaries and Balance Assessment iv

6 Exhibit 9.1: Cumulative and Year-by-Year Demonstration Effects on Measures of Utilization, Process, and Medicare Expenditures Exhibit 10.1: Year-by-Year Demonstration Effect on Claims-Based Measures of Health Care Utilization, by Year of First Attribution Exhibit 10.2: Year-by-Year Demonstration Effect on Claims-Based Process Measures, by Year of First Attribution Exhibit 10.3: Year-by-Year Demonstration Effect on Claims-Based Measures of Medicare Expenditures, by Year of First Attribution v

7 1. Conceptual Model To anchor the evaluation, we used Donabedian s quality-of-care-model, 1-3 which specifies that the structure of health care delivery increases the realization of optimal care processes, which improves patient outcomes. In this model, the structure refers to the attributes of the setting in which health services are provided, including material resources (facilities, equipment, funding) and human resources (practice organization, quality review, reimbursement methods). The process describes the diagnostic or therapeutic services provided for patients. The outcomes are reflected by the health status of patients and populations resulting from the effects of medical care. For the FQHC Advanced Primary Care Practice (APCP) Demonstration, the technical and financial assistance provided to demonstration sites was designed to spur the development of advanced primary care structures within FQHCs. We hypothesized that, consistent with the Donabedian model, the demonstration would produce a cascade effect in which the structural changes would facilitate processes, with resulting achievements of improved patient experiences and outcomes. The Centers for Medicare & Medicaid Services (CMS) expected that at least 90 percent of FQHCs receiving assistance would achieve National Committee for Quality Assurance (NCQA) Level-3 patient-centered medical home (PCMH) recognition by the end of the demonstration. 1 RAND s evaluation sought to determine the extent to which participation in the demonstration was associated with changes in structures as measured by the achievement of NCQA Level-3 PCMH recognition the highest level of PCMH recognition awarded by NCQA. Additionally, the evaluation assessed how participation in the demonstration and changes in structures are associated with changes in processes and outcomes of care for Medicare beneficiaries who seek care from FQHCs. To assess differences in patient experiences and outcomes between beneficiaries of demonstration sites and those of comparison sites, we assessed measures across multiple domains. These domains included continuity of care, timeliness, access, adherence to evidencebased care, beneficiary-rated quality of care, effective participation in decision-making, selfmanagement of health problems, experiences with the health care system, utilization of services, expenditures, coordination of care, health care disparities, and costs. We hypothesized that demonstration sites higher rate of achieving PCMH recognition would translate to larger improvements in primary care utilization and beneficiary experiences, and larger reductions in 1 NCQA confers three levels of PCMH recognition (Level 1, 2, or 3) based on the number of points scored across elements after review of a site s application and associated documentation. Sites with points are awarded Level 1 recognition, sites with points are awarded Level 2 recognition, and sites with 85 or more points are awarded Level 3 recognition. To receive recognition sites much receive at least 50% of available points on certain must-pass elements. 1

8 specialty care utilization, acute care utilization, and Medicare expenditures over time relative to comparison sites. 2

9 2. FQHC APCP Demonstration Interventions The FQHC APCP Demonstration included four interventions to support FQHC sites transformation into PCMHs: (1) quarterly care management fee payments, (2) technical assistance, (3) feedback on readiness for PCMH recognition, and (4) data and performance feedback reports on quality and utilization measures. These intervention components, designed by CMS, were delivered by a network of organizations with complementary purposes. Care Management Payments CMS provided demonstration sites with a quarterly care management fee payment of $18 for each Medicare beneficiary attributed to the site to support patient-centered medical care. To allocate payments to sites, CMS conducted its own attribution process, distinct from the methodology used by RAND for the evaluation. Beneficiaries eligible for attribution for the purposes of determining care management fee payments were continuously enrolled in Part A and Part B, continuously enrolled in fee-for-service Medicare, did not have end-stage renal disease, were not enrolled in hospice, and were alive by the end of the preceding year. FQHC visits made by eligible beneficiaries to each participating site in the year preceding each attribution quarter were identified using revenue code 520 or 521 and tallied. Beneficiaries were attributed to the demonstration site that provided a plurality of FQHC visits. In the event of ties, the beneficiary was attributed to the site providing the most recent visit. Total payments across participating sites averaged $3.8 million per quarter; payment per site averaged $7,844 per quarter for the duration of the demonstration. Technical Assistance (TA) for Implementing Practice Change Technical assistance provided by the FQHC APCP Demonstration was designed by CMS, and delivered by a network of organizations with complementary roles. Technical assistance was built on an existing support network of the National Association of Community Health Centers (NACHC), and over time incorporated regional TA level supports involving Primary Care Associations (PCAs) and Health Center Controlled Networks (HCCNs). FQHCs received training and assistance to support them in their transformation across all six NCQA standards through an extensive learning system involving the Health Resources and Services Administration (HRSA), the American Institutes for Research (AIR), and primary care associations (PCAs). AIR, the regional PCAs, and Qualis Health (Qualis) assisted demonstration sites with the preparation and completion of the biannual Readiness Assessment Surveys (RASs), which are discussed below. AIR provided office hours, conducted webinars, and distributed newsletters providing information that highlighted expectations, deadlines, successes, 3

10 and challenges. The PCAs provided one-on-one assistance and monitored the progress of demonstration sites in their region. Qualis provided an intense review and feedback of RAS submissions, and toward the end of the demonstration Qualis provided TA directly to demonstration sites. In addition to TA relating to practice change, NCQA offered TA to demonstration sites to assist them in preparing documentation required for NCQA PCMH recognition. Feedback Reports on Progress Toward PCMH Recognition Demonstration sites were required to submit information on progress toward PCMH recognition on a biannual basis. The instrument used for each submission, referred to as the NCQA Readiness Assessment Survey (RAS) within the FQHC APCP Demonstration alone, was identical to NCQA s PCMH recognition application used to formally apply for recognition, however, FQHCs were not required to provide documentation associated with their assessments. Following submission of the biannual RAS, each demonstration site received a report that provided each site s likely NCQA PCMH recognition-level (assuming documentation provided would support each site s assessment) and overall score trends. Feedback Reports on Utilization, Quality, and Expenditures Demonstration sites received quarterly cost and utilization data reports, which included sitelevel claims-based utilization measures (e.g., rates of inpatient admissions and Emergency Department visits), Medicare expenditure summary data (e.g., average total Medicare expenditures per beneficiary), and quality of care measures (e.g., glycated hemoglobin [HbA1c] testing, retinal eye exams, low-density lipoprotein [LDL] screening and nephropathy testing rates among beneficiaries with diabetes). In addition, a quarterly beneficiary-level report provided identifiable beneficiary data summarizing key claims-based outcomes (e.g., readmissions) for all beneficiaries attributed to the site. 4

11 3. Selection of Demonstration Sites and Comparison Sites Eligibility for the Demonstration CMS identified FQHCs that served at least 200 unique Medicare beneficiaries in the previous 12 months; were not specialty FQHCs; and were not FQHCs that served exclusively migrant or homeless populations, through Medicare administrative claims data, and invited them to apply to the FQHC APCP Demonstration. In order to participate, FQHCs had to meet the eligibility requirements listed in Exhibit 3.1 below. Exhibit 3.1 Eligibility Requirements for FQHCs Seeking to Participate in the FQHC APCP Demonstration 1. The FQHC is managed by a physician or nurse practitioner, meaning the overall clinical direction is provided by a physician or a nurse practitioner. 2. The FQHC provides primary care services (as opposed to providing only specialty services, such as dental or vision care) to a general population, and not exclusively to migrant workers or the homeless. 3. The FQHC is an individual physical location (multiple locations will not be considered under a single application). 4. The FQHC provides medical services to at least 200 Medicare beneficiaries with Part A and Part B coverage, and who are not enrolled in Medicare Advantage in the most recent 12-month period, including those with both Medicare and Medicaid coverage (dual eligibles). 5. The FQHC accepts the Joint Principles of a Patient-Centered Medical Home (Patient-Centered Primary Care Collaborative, undated). 6. The FQHC has a valid provider transaction number (PTAN) from CMS. 7. The FQHC is not under a corrective action plan for serious health and safety or financial issues with HRSA. 8. The FQHC is listed in the Provider Enrollment, Chain, and Ownership System (PECOS) file and able to receive electronic fund transfers (EFTs) at the time of application. FQHCs that have not recently submitted an 855A form are not listed in PECOS and therefore not eligible to participate in the demonstration. 9. The FQHC is submitting claims for payment to National Government Services (NGS) or to Noridian. Source: Medicare Federally Qualified Health Center Advanced Primary Care Practice Demonstration, Demonstration Partners, 2012b. As of November 20, 2012: Selection of Demonstration Sites A total of 1014 FQHC delivery sites applied to participate in the FQHC APCP Demonstration. Among these, CMS selected 500 demonstration sites to produce a diverse sample with respect to region, urbanicity, EHR adoption, PCMH readiness (measured according to the 2008 NCQA PCMH recognition standards that sites submitted as part of the application in the summer of 2011), and receipt of PCMH-related payments from multiple payers prior to the start of the demonstration. CMS developed a protocol to select 500 FQHCs while keeping the scope of the demonstration at or under 195,000 Medicare beneficiaries a sample size that was determined 5

12 by the available budget for care management fee payments to demonstration sites. First, CMS stratified FQHCs in the eligible applicant pool according to the five characteristics on which diversity was desired. Perfect balance would have produced a sample in which approximately 8 FQHCs were classified into each cell. CMS accepted into the demonstration all FQHCs in cells with counts of 8 or fewer a total of 115 FQHCs. The remaining eligible FQHCs were sorted in ascending order by the number of Medicare beneficiaries. The smallest remaining 385 FQHCs were then selected, regardless of strata, in order to maximize the number of FQHC sites that could participate within the demonstration s budget. Counts of the number of demonstration participants classified according to the five characteristics are displayed in Exhibit 3.2. Exhibit 3.2 Number of Demonstration Sites Classified According to Five Characteristics Rural FQHCs EMR - No EMR - Yes No Multi-Payer Support Multi-Payer Support No Multi-Payer Support Multi-Payer Support Level 0 Level 1-3 Level 0 Level 1-3 Level 0 Level 1-3 Level 0 Level 1-3 Midwest Northeast South West Urban FQHCs EMR - No EMR - Yes No Multi-Payer Support Multi-Payer Support No Multi-Payer Support Multi-Payer Support Level 0 Level 1-3 Level 0 Level 1-3 Level 0 Level 1-3 Level 0 Level 1-3 Midwest Northeast South West Source: Centers for Medicare & Medicaid Services, 2011 Notes: Level 0 and Level 1-3 correspond to NCQA PCMH Recognition levels scored by CMS using NCQA s 2008 recognition standards. The demonstration required sites to achieve Level 3 recognition according to the 2011 standards, which are more rigorous. Due to the early attrition of three demonstration sites, CMS added three additional demonstration sites within the first 9 months of the demonstration. The combined sample of 503 sites was included in all analyses that assessed the effect of the demonstration on beneficiary outcomes. Ultimately, 69 of the 503 demonstration sites (13.7%) ceased participating before the end of the demonstration. During the demonstration, 23 sites (4.6%) voluntarily withdrew from the demonstration while the remaining 46 (9.1%) were terminated by CMS for not meeting demonstration requirements (e.g., failure to complete the Readiness Assessment Survey; no longer operating as an independent site; Medicare beneficiary volume failing to meet the threshold; or failure to progress toward NCQA Level 3 PCMH recognition). 6

13 Selection of Comparison Sites Overview of Potential Comparison FQHCs We identified three types of FQHCs that might serve as comparison sites: 1) FQHCs that applied but were not selected for the demonstration, 2) nearly-eligible FQHCs that provided services to fewer than 200 Medicare beneficiaries and thus failed to meet eligibility criterion 4 in Exhibit 3.1, and 3) non-applicant FQHCs. We describe each of these three groups below. Non-Selected FQHCs A total of 318 FQHCs were eligible and applied for the demonstration, but were not among the 500 sites initially selected as participants. Three of the 318 non-selected sites were later permitted to join the demonstration to replace three sites that dropped out after the first few months of the demonstration. The non-selected sites were disproportionately high-volume sites because the criteria used by CMS to select participants from among eligible applicants favored lower-volume sites. Because the non-selected sites met all eligibility criteria for the demonstration and agreed to all obligations required of participating sites, they are likely to be comparable to participating FQHCs on non-observable characteristics, such as their motivation to achieve PCMH recognition. Nearly-Eligible FQHCs We also identified at group of nearly eligible FQHCs that were ineligible for the demonstration because they failed to provide medical services to 200 or more Medicare beneficiaries in the pre-demonstration period. The rationale for considering this group was that the volume-based eligibility criterion might have induced an artificial distinction between eligible and ineligible sites particularly in the narrow range around the 200-beneficiary cutoff. For example, an FQHC that served 190 Medicare beneficiaries in 2010 is likely to be highly comparable in terms of both observed and unobserved characteristics to a participating FQHC that treated 210 patients. While FQHCs that serve fewer than 150 Medicare beneficiaries may be viewed as less comparable to participating FQHCs, many of these sites may have total volumes (including Medicare beneficiaries and patients with other types of insurance) that are comparable to that of participating FQHCs. Thus, we prioritized sites with between 100 and 200 Medicare beneficiaries, but did not consider FQHCs that provided services to fewer than 100 Medicare beneficiaries. Although, we might assume these sites are comparable to demonstration FQHCs, we were unable to verify that the nearly eligible FQHCs would have met all other eligibility criteria. Moreover, we do not know if these sites would have been motivated to apply had they been eligible, unlike all participating FQHCs, each of whom were motivated to apply. 7

14 Non-Applicant FQHCs This group comprises 544 FQHCs that met all eligibility criteria and were invited to apply but declined to submit an application. Information on the reasons why these FQHCs did not apply was not collected. For this reason, we were unable to readily identify a subset of eligible, non-applicant FQHCs that declined to apply for reasons that would be unlikely to bias the demonstration s primary outcomes. Thus, non-applicant FQHCs may differ systematically from applicants for reasons that cannot be easily measured and accounted for in our analyses. Nevertheless, the large number of sites that are in this category and the comparability in size between these sites and demonstration sites suggested that these sites were too valuable to exclude from consideration as possible comparison sites. Selection of Comparison FQHCs After identifying potential comparison sites, we used Medicare claims from calendar year 2010 to identify the subset of comparison sites that were most comparable to demonstration sites. We used the Provider Transaction Access Number (PTAN) as the primary identifier of FQHCs on claims. We sought to replicate the method used by CMS to determine eligibility for the demonstration by identifying all FQHCs that submitted claims for all-inclusive visits (revenue center codes 521, 522, and 525) for Medicare beneficiaries who were continuously enrolled in Part A and Part B during 2010, and who were not enrolled in Medicare Advantage, did not have end-stage renal disease, and had no hospice utilization during the year. As illustrated in Exhibit 3.3, this comprised a total of 3426 FQHCs. To narrow the sample of potential comparison FQHCs, we first excluded the 503 FQHCs (14.7 percent) that were participating in the demonstration at baseline or that replaced dropout sites in the first few months of the demonstration. Because exposure to the intervention among sites that are affiliated with the same parent organization might contaminate comparison sites, we excluded 823 additional sites (24.0 percent) whose parent organization had at least one site that was participating in the demonstration. Organizational relationships were identified using Uniform Data System (UDS) identifiers. From the remaining eligible sites we excluded those that were not categorized as non-selected, nearly-eligible, or non-applicant FQHCs. We then excluded 40 additional FQHCs (1.2 percent) that were participating in the MAPCP Demonstration, which was an exclusion criterion for demonstration sites. To better match the profile of participating FQHCs we then identified and excluded 27 sites (0.8 percent) that served homeless or migrant worker populations exclusively, 33 sites (1.0 percent) that operated in special settings such as schools and correctional facilities, and 3 sites (0.1 percent) that were not brick-and-mortar facilities (such as mobile vans). To implement these exclusions we used characteristics from HRSA s Electronic Handbooks Grant Management System. For 85 sites (2.5 percent) we were not able to crosswalk PTANs to a single, unique record in HRSA s databases. These sites were excluded from consideration as comparison sites. 8

15 We excluded 2 additional sites (0.1 percent) that were located outside of the 50 states (U.S. Virgin Islands in both cases) because these sites may be less comparable to those operating in the 50 states. Finally, we excluded 17 sites (0.5 percent) that had no attributed beneficiaries at the beginning of the demonstration using a plurality rule. The resulting sample included a total of 827 FQHC comparison sites, of which 148 were non-selected, 314 were nearly-eligible, and 365 were non-applicants. Exhibit 3.3: Exclusions Applied in Identification of FQHC Comparison Sites Exclusion Number of sites (%) All FQHCs that billed Medicare for a qualifying service in (100%) RAND exclusions Demonstration site 503 (14.7) Organizational relationship with demonstration site 823 (24.0) Low volume site or otherwise ineligible site 1066 (31.1) Participating in Multi-payer Advanced Primary Care Practice Demonstration 40 (1.2) Serves exclusively homeless or migrant population 27 (0.8) Operates in a special setting a 33 (1.0) Not a brick-and-mortar site 3 (0.1) Could not crosswalk PTAN/HRSA Site ID 85 (2.5) Located in a U.S. territory 2 (0.1) No attributed beneficiaries in the year preceding the demonstration 17 (0.5) Total number of eligible FQHC comparison sites 827 a Special settings include: hospital, school, domestic violence, correctional facility, or nursing home. We then used propensity score methods to examine the extent to which the 827 comparison sites were comparable to demonstration sites in terms of observable characteristics using data derived from claims, HRSA administrative sources, U.S. Census Bureau, and other sources. Propensity score models used logistic regression to estimate the propensity to participate in the demonstration as a function of site-level characteristics. Propensity score methods are discussed more fully elsewhere in this Appendix. Exhibit 3.4 shows both unweighted and propensity-score weighted comparisons of characteristics between demonstration and comparison sites. All characteristics included in Exhibit 3.4 were included in the propensity score model except where noted. Applying weights eliminated nearly all instances of imbalance between the two groups. 9

16 Exhibit 3.4: Unweighted and Weighted Comparisons of Demonstration and Comparison Site Characteristics Characteristic Demonstration FQHCs Unweighted Comparison Comparison FQHCs P-value Demonstration FQHCs Propensity Score- Weighted Comparison Comparison FQHCs P-value Aggregated beneficiary-level characteristics Mean % <65 years (0.76) (0.62) (0.76) (0.65) Mean % years (0.43) (0.35) (0.43) (0.35) Mean % years (0.33) (0.28) (0.33) (0.28) Mean % 85pl years 4.05 (0.16) 4.03 (0.13) (0.16) 3.70 (0.13) Mean % female (0.33) (0.29) (0.33) (0.30) Mean % White (1.32) (1.00) (1.32) (0.99) Mean % Black (1.22) (0.97) (1.22) (0.90) Mean % Asian 2.95 (0.50) 2.10 (0.27) (0.50) 3.46 (0.41) Mean % Hispanic 7.17 (0.52) 7.29 (0.42) (0.52) 6.82 (0.38) Mean % Other/unknown race 2.81 (0.26) 3.28 (0.29) (0.26) 2.80 (0.21) Mean % Disabled (0.75) (0.60) (0.75) (0.63) Mean % Medicaid (0.81) (0.63) (0.81) (0.65) Mean % Institutionalized 2.13 (0.08) 2.09 (0.07) (0.08) 2.20 (0.07) HCC score of beneficiaries (mean) 1.09 (0.01) 1.07 (0.01) (0.01) 1.09 (0.01) Mean % autoimmune condition 4.36 (0.10) 4.27 (0.09) (0.10) 4.27 (0.09) Mean % cancer 7.64 (0.13) 7.58 (0.11) (0.13) 7.96 (0.16) Mean % CHF 8.92 (0.15) 8.84 (0.13) (0.15) 8.84 (0.13) Mean % chronic lung disease (0.30) (0.23) (0.30) (0.23) Mean % CVD (0.25) (0.24) (0.25) (0.25) Mean % diabetes (0.43) (0.32) (0.43) (0.35) Mean % hematologic condition 0.65 (0.03) 0.61 (0.02) (0.03) 0.70 (0.03) Mean % HIV 1.24 (0.24) 1.27 (0.18) (0.24) 1.32 (0.24) Mean % liver disease 0.46 (0.02) 0.44 (0.02) (0.02) 0.60 (0.04) Mean % mental health condition (0.47) (0.42) (0.47) (0.48) Mean % neurological condition (0.16) (0.16) (0.16) (0.18) Mean % stroke 4.05 (0.09) 3.83 (0.08) (0.09) 4.03 (0.08) Mean % substance abuse condition 4.20 (0.17) 3.74 (0.13) (0.17) 4.80 (0.21) Site-level characteristics Not rural location (%)

17 Characteristic Demonstration FQHCs Unweighted Comparison Comparison FQHCs P-value Demonstration FQHCs Propensity Score- Weighted Comparison Comparison FQHCs P-value Rural location (%).. Midwest region (%) Northeast region (%) South region (%) West region (%) % census tract Black (1.18) (0.90) (1.18) (0.85) % census tract Asian 3.14 (0.37) 3.32 (0.31) (0.37) 3.33 (0.31) % census tract Am Indian 1.59 (0.30) 1.71 (0.24) (0.30) 1.55 (0.25) % census tract Hispanic (1.13) (0.89) (1.13) (0.80) % census tract foreign born (0.59) (0.48) (0.59) (0.46) % census tract with poverty (0.51) (0.40) (0.51) (0.40) FQHC age < 5 yrs (%) FQHC age 5-10 yrs (%) FQHC age yrs (%) FQHC age yrs (%) FQHC age yrs (%) FQHC age 40+ yrs (%) Lookalike site (%) Safety Net Medical Home Participant (%) FQHC site (%) FQHC site (%) FQHC site (%) FQHC site (%) FQHC site (%) Primary Care Providers, Mean (SE) 5.53 (0.22) 5.43 (0.19) (0.22) 5.16 (0.17) Specialist Providers, Mean (SE) 0.78 (0.08) 0.80 (0.08) (0.08) 0.91 (0.07) Midlevel Providers, Mean (SE) 2.38 (0.13) 2.51 (0.10) (0.13) 2.32 (0.09) Behavioral Health Providers, Mean (SE) 0.31 (0.04) 0.32 (0.03) (0.04) 0.31 (0.03) Dental Providers, Mean (SE) 0.04 (0.01) 0.04 (0.01) (0.01) 0.03 (0.01) Vision Providers, Mean (SE) 0.10 (0.02) 0.09 (0.02) (0.02) 0.12 (0.02) Podiatry Providers, Mean (SE) 0.17 (0.02) 0.14 (0.02) (0.02) 0.18 (0.02) Other Providers, Mean (SE) 0.37 (0.04) 0.33 (0.03) (0.04) 0.42 (0.03) HRSA supplemental funding for Beacon sites (%)

18 Unweighted Comparison Propensity Score- Weighted Comparison Demonstration Comparison P-value Demonstration Comparison P-value Characteristic FQHCs FQHCs FQHCs FQHCs HRSA supplemental funding for PCMH in FY (%) HRSA supplemental funding for PCMH in FY (%) ACA Building Capacity grant funding (%) ACA Immediate Facility Improvement grant funding (%) ACA New Access Point grant funding (%) Patients per site Mean (SE) (90.17) (80.42) (90.17) (77.57) Medicare patients per site Mean (SE) (9.06) (7.59) (9.06) (8.24) Grant Revenue per site Mean (SE) 761, , ( ) ( ) ( ) ( ) Patient Revenue per site Mean (SE) 1,165, ,164, ( ) ( ) ( ( ARRA grant funding (%) NCQA PCMH Recognition Level: none (%) Level 1-2 (%) Level 3 (%) NOTES: 2008 NCQA PCMH Recognition Levels were not included in the model because of the low prevalence of two of the three categories. Shaded cells indicate P <

19 We then examined comparability of individual comparison sites to the sample of demonstration sites by comparing the distribution of propensity scores between the two groups. We divided the range of propensity scores into bins of equal width and compared the number of intervention and comparison sites (by type) within each bin. Exhibit 3.5. Analysis of Site-Level Propensity Score Balance Between Demonstration and Comparison FQHCs Propensity Score Total (N=1330) Demonstration FQHCs (n=503) volume FQHCs (n=182) Comparison FQHCs volume FQHCs (n=132) Non-Applicant FQHCs (n=365) Non-Selected FQHC (n=148) As seen in Exhibit 3.5, all comparison sites had demonstration site matches within each of the defined propensity score strata. The analysis also revealed few FQHC comparison sites in the upper tail of the propensity score distribution with only two FQHC comparison sites associated with propensity scores >= However, we elected not to exclude any demonstration sites from the analysis. Based on the results displayed in Exhibit 3.5, we included all 827 comparison FQHCs in all subsequent analyses. 13

20 4. Beneficiary Attribution Attribution Methodology For the purposes of RAND s evaluation, beneficiaries were attributed on a quarterly basis to the site that provided a plurality of the beneficiary s primary care visits over the most recent 12- month period. Eligible Medicare beneficiaries were required to be continuously enrolled in Part A and B, not enrolled in Medicare Advantage, and not have end-stage renal disease, during the most recent 12-month period. Beneficiaries were then attributed to sites on the basis of a plurality attribution rule, under which beneficiaries were attributed to the FQHC or other site (e.g., rural health clinic or primary care practice) associated with a plurality (i.e., highest count) of qualifying services during the lookback period. In the case of a tie, the beneficiary was attributed to the site with the most recent qualifying service. Qualifying services were defined to include common primary care services provided by FQHCs, rural health clinics, and services provided in other settings by physicians in defined specialties. Qualifying services were identified using Medicare claims, including revenue codes for FQHC all-inclusive visits and a combination of HCPCS codes and provider specialty codes for non-fqhc services. We defined a list of qualifying services by applying the following criteria: (1) the service is a primary care service that is likely to be delivered by providers working in FQHCs and other sites that deliver similar services; (2) the service is included in attribution rules for MAPCP states and the Medicare Shared Savings Program; (3) HCPCS code for the service is valid for separate Medicare reimbursement for the period (or replaced by an existing code for an included qualifying service), in order to ensure to the extent possible that attribution will not change over the course of the evaluation period. Applying these criteria, Medicare qualifying services were defined as: FQHC visits: Outpatient Institutional claim line items with date of service from 1/1/ /31/2010 with type of bill (TOB) code = 73x (for dates of service prior to April 1, 2010) or 77x (for dates of service on or after April 1, 2010), AND revenue code in {0521 (FQHC clinic visit), 0522 (home visit by FQHC provider)}. Rural health clinic visits: Outpatient Institutional claim line item with date of service from 1/1/ /31/2010 with type of benefit (TOB) code = 71x AND revenue code in {0521, 0522}. Other primary care services: Carrier claim line items with date of service from 1/1/ /31/2010 AND with HCPCS code in { , , , , 99315, 99316, 99318, , , , } AND provided by a primary care physician (as defined by National Plan and Provider Enumeration System Provider Taxonomy Code: internal medicine, general practice, family medicine, ob/gyn, nurse practitioner, physician assistant, geriatrics). 14

21 Beneficiaries who were attributed to sites in the demonstration or comparison groups were included in the study population; beneficiaries attributed to other sites were excluded. The attribution process was conducted initially using a lookback period of one year prior to initiation of the demonstration (11/1/ /31/2011). Attribution was then repeated on a quarterly basis for the duration of the demonstration. Consistent with the intention-to-treat principle, beneficiary outcomes were associated with the site to which each beneficiary was first attributed regardless of whether the beneficiary s attributed site changed over time. In recognition that a large number of new Medicare beneficiaries met attribution criteria over time, we included beneficiaries who were first attributed to a demonstration or comparison site during the baseline year, year 1, or year 2. We excluded all beneficiaries who were first attributed to a demonstration or comparison site during the demonstration s final year (year 3) because these beneficiaries did not have a full year of observed outcomes. Accounting for Changes in Beneficiary Eligibility During the Demonstration To account for changes in beneficiaries Part A and B enrollment status, enrollment in Medicare Advantage, and changes in entitlement due to the development of ESRD, we calculated a yearly eligibility fraction for each beneficiary. The eligibility fraction was defined as the number of months of eligibility within the preceding year divided by 12. For example, a beneficiary who is eligible for 6 months of the year would have an eligibility fraction of 0.5 for the year. The eligibility fraction is used to inflate expenditure and utilization data if a beneficiary does not have a full year of eligibility as described in Section 5. Once beneficiaries were attributed to a demonstration or comparison site, their outcomes were assessed until the end of the third year of the demonstration. Beneficiaries contributed outcomes in each year of the demonstration except when they met one or more ineligibility criteria for all 12 months of the year. To examine differences in beneficiary rates of attrition due to ineligibility, we compared attrition during the final year of the demonstration. Exhibit 4.1 shows that by the third year of the demonstration, 21.0 percent of demonstration beneficiaries and 21.1 percent of comparison beneficiaries no longer met eligibility criteria for attribution. Reasons for ineligibility were similar between demonstration and comparison sites. Exhibit 4.1. Reasons for Attrition from the Demonstration Demonstration FQHCs Comparison FQHCs Overall Number of beneficiaries who attrited, n (%) a 64,420 (21.0) 109,151 (21.1) 173,571 (21.1) Reasons for attrition, n (%) b Loss of Part A or Part B eligibility 5,537 (8.6) 9,604 (8.8) 15,141 (8.7) Enrollment in Medicare Advantage 32,927 (51.1) 55,398 (50.8) 88,325 (50.9) Medicare entitlement through ESRD 4,504 (7.0) 8,307 (7.6) 12,811 (7.4) 15

22 Death 26,035 (40.4) 44,749 (41.0) 70,784 (40.8) NOTES: This analysis uses the cohort of beneficiaries who were ever attributed to a demonstration or comparison site over the course of the demonstration, which includes four cohorts: beneficiaries first attributed during the year before the demonstration began, year 1, year 2, and year 3. a Percentages are based on the total number of beneficiaries ever-attributed to demonstration or comparison sites (306,666 beneficiaries attributed to demonstration sites and 517,672 beneficiaries attributed to comparison sites). b Percentages are based on the total number of beneficiaries attriting as of the last quarter of the demonstration. Persistence of Attribution to Baseline Sites We used an intention-to-treat design in which beneficiaries were assigned to the demonstration or comparison group based on the site to which they were first attributed regardless of their subsequent primary care utilization patterns. To understand the extent to which beneficiaries changed their usual source of care over the three-year demonstration period, we examined the rate to which beneficiaries were re-attribution to their baseline site. This analysis was limited to beneficiaries who were first attributed to a demonstration or comparison site in the year preceding the demonstration. Exhibit 4.2 indicates that only 11.8 percent of demonstration beneficiaries and 11.4 percent of comparison beneficiaries were attributed to an alternative site based on their primary care utilization during the last year of the demonstration. Exhibit 4.2. Loyalty to Baseline Site Attribution status Demonstration FQHCs N (%) Comparison FQHCs N (%) Attributed to same site 64,159 (42.1) 113,224 (41.0) Attributed to different site 17,936 (11.8) 31,507 (11.4) Ineligible for attribution 17,974 (11.8) 32,946 (11.9) No qualifying services 52,231 (34.3) 98,169 (35.6) Total 152,300 (100.0) 275,846 (100.0) 16

23 5. Medicare Claims Analysis Claims-Based Outcome Measures We used Medicare Part A and B claims files to measure utilization, quality of care, and Medicare expenditures. The outcomes used to assess the demonstration s impact in Medicare claims analyses are presented in Exhibit 5.1. Exhibit 5.1: Outcomes Used in Medicare Claims Analyses Outcome Utilization FQHC visits Non-FQHC primary care visits Specialist visits ED visits Inpatient admissions Inpatient ACSC admissions Inpatient readmissions Process HbA1c test (diabetes patients) LDL test (diabetes patients) Eye exam (diabetes patients) Nephropathy test (diabetes patients) All diabetes tests Lipid test (IVD patients) Expenditures Total Medicare expenditures Inpatient expenditures Definition Number of FQHC visits per beneficiary per year Number of visits to primary care providers in rural health clinics or non-fqhc primary care practices per beneficiary per year Number of visits to a specialist physician per beneficiary per year Number of emergency room visits per beneficiary per year (includes both visits followed by admissions and outpatient-only ED visits as well as observation stays) Number of hospitalizations per beneficiary per year Number of ACSC hospitalizations per beneficiary per year Percentage of discharges followed by an unplanned readmission within 30 days Number of diabetic patients who had a Hemoglobin A1c blood test in the past year Number of diabetic patients who had an LDL cholesterol blood test in the past year Number of diabetic patients who had an eye exam in the past year Number of diabetic patients who had a albumin urine test in the past year Number of diabetic patients who had all four above tests in the past year Number of IVD patients who had a lipid panel test in the past year Sum of total Medicare expenditures in the year Sum of inpatient file expenditures in the year Part B expenditures Sum of carrier file expenditures in the year NOTE: HbA1c=hemoglobin A1c; LDL= low-density lipoprotein; IVD=Ischemic Vascular Disease. Utilization Measures Utilization measures included FQHC visits; non-fqhc primary care visits (which include visits to physicians, nurse practitioners, or physician assistants who have primary care specialties 17

24 and who practice in either rural health clinics or office settings); specialist visits (which include visits to physicians, nurse practitioners, or physician assistants who have specialties other than primary care and who practice in an FQHC, rural health clinic, or office setting); emergency department [ED] visits (including both visits that do and do not lead to a hospitalization as well as observation stays); and both all-cause admissions and admissions for ambulatory care sensitive conditions (ACSC) at acute care short-stay hospitals. ACSC admissions are those with a primary diagnosis of: one or more diabetes short-term complications (ketoacidosis, hyperosmolarity, coma); one or more diabetes long-term complications (renal, eye, neurological, or circulatory); chronic obstructive pulmonary disease or asthma in older adults; hypertension; congestive heart failure; angina without procedure; uncontrolled diabetes; asthma in younger adults; and lower-extremity amputation among patients with diabetes. We also measured 30-day unplanned hospital-wide readmissions. All measures were adjusted for beneficiary, site, and area-level characteristics as described later in this section Process Measures The evaluation used five Healthcare Effectiveness Data and Information Set (HEDIS) process quality measures to assess site adherence to evidence-based guidelines. For patients with diabetes patients, we assessed utilization of HbA1c testing, low-density lipoprotein cholesterol testing, retinal eye exams, and nephropathy testing within the past 12 months, as well as the use of all four tests. For patients with ischemic vascular disease, we assessed utilization of a blood lipid panel within the past 12 months. All measures were adjusted for beneficiary, site, and arealevel characteristics. Medicare Expenditures Measures of Medicare expenditures included: inpatient expenditures (including all claims found in the inpatient file), Part B expenditures (including all claims in the Physician/Supplier Part B file, or carrier file); and total Medicare expenditures (comprising the sum of inpatient, outpatient, skilled nursing facility, home health, hospice, durable medical equipment, and Part B expenditures). Additional measures of expenditures are included within RAND s Final Evaluation Report. All measures were adjusted for beneficiary, site, and area-level characteristics. 18

25 Difference-in-Differences Model Specification A differences-in-differences approach was used to model the impact of the demonstration on both claims-based and survey-based outcomes. Generalized estimating equations were used to implement this approach. Covariates used in the model include beneficiary, site, and area characteristics. Repeated yearly observations for each beneficiary allowed an assessment of the impact of the demonstration over time. This model is defined as: (1) Y = α 0 + α 1 I + α 2 DemoYear1 + α 3 DemoYear2 + α 4 DemoYear3 + α 5 (I DemoYear1) + α 6 (I DemoYear2) + α 7 (I DemoYear3) + γx + ε where Y is the outcome. α 0 is the intercept, an estimate of the mean adjusted level of Y in the comparison group in the baseline year. I is an indicator for the intervention, defined here as attribution to a demonstration site (=0,1). Its parameter estimate α 1 is an estimate of the difference in levels of cost/utilization associated with the demonstration group relative to the comparison group in the baseline period. DemoYear1, DemoYear2, and DemoYear3 represent binary indicator variables of the year of observation for each outcome during the demonstration period. For example, DemoYear1=1 for the first demonstration year (i.e., November 1, 2011 through October 31, 2012) and 0 for all other years. Parameters α 2 through α 4 are estimates of the difference in beneficiary cost/utilization between the year of the indicator variable and the baseline year in the comparison group. I*DemoYear1, I*DemoYear2, and I*DemoYear3 represent interaction terms that permit the impact of the demonstration to differ for demonstration sites in each of the three demonstration years, compared with the baseline year. Parameters α 5 through α 7 are the estimates of interest in this model. These parameters convey the impact of the demonstration on a yearly basis in the demonstration period in relation to the baseline period. For example, α 5 is an estimate of how the difference between the demonstration and comparison groups in the first demonstration year differs from the difference between the demonstration and comparison groups in the baseline year. The parameter α 6 is an estimate of how the difference between the demonstration and comparison groups in demonstration Year 2 differs from the difference between the demonstration and comparison groups in the baseline year. X is a vector of covariates. Its parameter estimates γ represent the difference in beneficiary cost/utilization associated with a one-unit change in X. A list of all covariates used in the model are included later in this section. ϵ is a random error term that is assumed to follow an auto-regressive process where the error in one year is correlated with the error in the next year. The coefficient of auto-correlation is estimated through the model. This model outlined in Equation 1 allows the effect of the demonstration to vary in a nonlinear fashion between years during the demonstration period. We adjusted standard errors for clustering of beneficiary responses over time using the Huber-White sandwich estimator. With this method, standard error estimates would be more 19