Unit of Analysis, Determination of Peer Groups, and Scope of Services Issue Paper Prepared for the Provider Peer Grouping Advisory Group By Minnesota Department of Health Staff June 26, 2009 UNIT OF ANALYSIS In developing the provider peer grouping methodology, there are many questions that need to be considered to ensure that the peer grouping system is useful for its intended audiences, and that the comparisons among providers are accurate and meaningful. This issue paper addresses issues related to determining the unit of analysis and scope of analysis that are most appropriate in light of these goals, and highlights several key questions that the Advisory Group will need to consider. Two central questions related to the peer grouping system are at what level the analysis should be conducted and at what level results should be publicly reported? Should comparisons be made at the clinic/hospital level or the medical group/hospital system level? The most desirable unit of analysis may vary according to what is being measured. For example, management of chronic illness requires a team, and so it may be most appropriate to peer group at the medical group level for this purpose. In addition, different stakeholders may desire different units of analyses depending on how the information will be used. Consumers tend to be interested in more granular publicly reported comparisons of individual physicians or clinics, whereas medical groups may be most interested in comparisons at the medical group level for competitive advantage. Physicians may be interested in data at the individual physician level to inform their practice, but tend to prefer data to be reported publicly at a higher level of aggregation. Others who prefer such comparisons to be made at either a clinic/hospital or medical group/hospital system level point out that accountability for care practices and the ability to change practice patterns is greater at either of these levels than for an individual physician. On a practical level, two primary considerations influence the unit of analysis: an adequate number of observations and limitations of available data. NUMBER OF OBSERVATIONS Sufficient numbers of patients must be part of an analysis in order to generate an acceptable level of statistical validity and reliability. The issue of small cell sizes has complicated other efforts to compare providers who practice in small rural communities or in highly specialized fields of medicine. The aggregation of data through a multipayer encounter claims database will facilitate a community-wide analysis of provider performance and will help in mitigating the small cell size issue. While opinions differ as to the minimum number of observations that is needed (and may vary depending on how
the information is intended to be used), CMS currently uses 25 patients as the minimum number required to produce reliable analysis. In analyzing either total cost and quality or specific conditions, it may be desirable to modify the methodology as it applies to rural and/or independent providers. If a provider has fewer than 25 patients available, at least three options are available to increase patient numbers: 1) lengthen the time period included in the analysis; 2) roll up reporting into a broader composite score based on multiple measures; or 3) combine scores of separate similar hospitals or clinics and report them at a higher level of aggregation. In making a recommendation on this issue, the Advisory Group will likely need to consider the tradeoffs between ensuring that adequate numbers of observations are available, and the usefulness of the information (e.g., combining scores of separate providers may increase statistical reliability, but could negatively affect the usefulness of peer grouping as a tool for accountability and for quality and efficiency improvement). LIMITATIONS OF AVAILABLE DATA The claims data that will be available to inform the resource use and cost components of the peer grouping analysis will readily identify a specific physician as well as the medical group of which that physician is a part. Reliable data at the clinic site level, however, is more difficult to obtain both because of how providers submit claims data and because physicians may practice at multiple clinic locations. 1 Since May 2008, providers have been required to use a National Provider Identification (NPI) number in submitting claims for payment. Providers have the flexibility to report NPIs at essentially any level; for example, a large medical group may choose to identify each clinic in its system with a separate NPI, or it may choose to identify all clinics under one NPI. Under the latter circumstance, it is difficult to drill down from the medical group to a specific clinic level identifier. 2 A related issue to the challenge of highly aggregated NPIs is that an unknown proportion of physicians practice at multiple clinics. If one wants to peer group at the clinic site level, it would be necessary to make credible linkages between individual physicians, and the particular clinic at which that physician practiced in serving a particular patient. If peer grouping is done at the clinic site level, these linkages would be needed to ensure that care is attributed appropriately to a particular clinic and not just to a particular physician. Due to these data limitations, several health plans compare providers at the medical group level. Those who produce this analysis generally agree that clinic-level reporting would 1 In the future, it may become more feasible to collect more data directly from providers, which would enable other approaches to reporting. 2 The Minnesota Department of Health s contractor for collecting encounter and pricing data has proposed some potential strategies for mitigating this issue in the Minnesota Health Care Claims Reporting System. Since data submission for the MHCCRS will begin on July 1, 2009, it will be some time before the Minnesota Department of Health more fully understands how NPIs are reported and how successful these alternative strategies may be for drilling down under highly aggregated NPIs.
be more desirable if data was available to do so for example, MN Community Measurement has been able to report quality information for a limited number of measures at the clinic site level, and the results of this site-level measurement have shown substantial variation in performance across clinic sites within medical groups. Hospital comparisons are generally done at the individual hospital level rather than the hospital system level, because hospitals may differ significantly within the same system and because data to compare hospitals separately is readily available. SHOULD THERE BE DISTINCT METHODOLOGIES FOR PEER GROUPING WITHIN A PROVIDER TYPE? An important issue closely tied to considerations of unit of analysis is whether distinct categories of providers -- or peer groups --should be determined for the two tasks of peer grouping: creating a methodology for calculating total risk-adjusted cost and quality of a provider s patient population, and for specific conditions. Specifically, the question that needs to be addressed is whether it is appropriate to use a single methodology for peer grouping within a provider type (physicians, hospitals) or whether the use of multiple methodologies for distinct categories of physicians and hospitals is warranted. One argument in favor of creating distinct categories within a provider type is that it may be desirable to compare providers with a similar scope of service (e.g. comparing solo or small practitioners only with each other vs. comparing them with large medical groups) or who serve only a distinct population (e.g. children s hospitals). For purposes of resource use and cost analysis, the methodology for providers with different scopes of service is likely to be the same. However, the types of quality measures that are a) available, and b) likely to meet a standard for minimum number of observations will likely vary between small and large providers. Regardless of whether the Advisory Group chooses to recommend separate methodologies for distinct groups within a provider type, some method of dealing with the fact that the quality measures chosen for peer grouping will not be available for all providers is needed. The key question for the Advisory Group at this point is whether to develop a single recommended methodology for each provider type (including recommendations for how to deal with missing data or small numbers), or whether to develop multiple methodologies. A primary argument against developing multiple methodologies is that this process would likely result in a less useful tool from the consumer s perspective. From a consumer perspective, it makes the most sense to structure the peer grouping system in a way that mirrors the way consumers think about care and how they access providers. If a consumer wishes to determine the best place to receive care, they should be able to compare performance for all providers of a given type. With multiple peer grouping methodologies, a consumer would only be able to compare within groups (e.g., small providers vs large providers) and the information would be less useful than a comparison of all providers using a common methodology. For specific conditions, a key question is how to compare providers who treat the specific condition, and specifically, whether it is appropriate to directly compare primary care
physicians and specialists or whether comparisons should be made only within (but not across) those two groups. From a consumer perspective, it may be most desirable to compare all physicians both primary care and specialists who treat patients with a specific condition included in the peer grouping analysis. Comparing specialty care physicians and primary care physicians to each other may be controversial, however, due to concerns among specialists that their patient population is sicker than the population treated by primary care providers in ways that are not accounted for by risk adjustment. The Centers for Medicare and Medicaid Services (CMS), in its physician resource use reporting project, has chosen to compare specialists only to other specialists treating the same condition. A related discussion is how results of the peer grouping analysis will be risk adjusted to account for differences among providers related to severity of illness, patient demographics, and payer mix. This latter discussion will be taken up at a later point in the Advisory Group s discussions. SCOPE OF SERVICES In an analysis of total cost and quality for primary care and for hospitals, determinations need to be made about the scope of services to be included. For purposes of total cost, it will be desirable to cast as wide a net around services to be included as possible. With some limited exceptions for outlier circumstances, 3 all care that is typically covered in a health insurance benefit set (and therefore, all care that is included in the claims database that will be used for this analysis) will be included for a provider s patient population. 4 Non-covered services such as long-term care, alternative therapies, or over the counter medications will not be included. This approach to defining the scope of services for a total cost of care analysis is similar to existing methodologies that are being used in Minnesota. Most existing initiatives to measure resource use, cost, and/or quality for specific conditions rely on the use of episode groupers, which are software programs that use diagnosis codes from health care claims information to assign claims to episodes of care for a particular condition over a defined period of time (for acute conditions, there are specific start and end dates that can be defined, and for chronic conditions the definition of an episode is frequently all care provided during a 12 month period). An episode perspective provides a longitudinal view of care that is useful especially for the analysis of chronic or disabling conditions. 5 Analysis for specific conditions will be based on care coordinated by physicians, including hospital services. An additional consideration for the Advisory Group is whether it also is desirable to analyze and report separately on care provided by hospitals for specific conditions. 3 Identifying these outliers will be the topic of a future Advisory Group discussion. 4 There will likely be challenges associated with completeness of data (e.g., prescription drug claims), and a methodology for handling this will need to be developed. 5 Episode groupers will be discussed in more detail when the Advisory Group considers issues related to measuring resource use and cost for specific conditions.
Key Questions: What should the unit of analysis be for physicians and hospitals? Does it need to be the same for every audience? What should be considered the minimum number of observations required for public reporting of peer grouping results? Should separate methodologies for peer grouping within a provider type be created? If so, how should the peer groups within a provider type be determined? In an analysis of specific conditions, should primary care physicians and specialists be compared to each other, or should comparisons be made only within these two groups?