MEASURING QUALITY OF CARE AND PERFORMANCE FROM A POPULATION HEALTH CARE PERSPECTIVE

Transcription

1 Annu. Rev. Public Health : doi: /annurev.publhealth Copyright c 2003 by Annual Reviews. All rights reserved First published online as a Review in Advance on December 2, 2002 MEASURING QUALITY OF CARE AND PERFORMANCE FROM A POPULATION HEALTH CARE PERSPECTIVE Stephen F. Derose and Diana B. Petitti Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, California 91101; Stephen.F.Derose@kp.org, Diana.B.Petitti@kp.org Key Words outcome and process assessment (health care), quality/quality assurance, health care/public health practice, health care economics and organizations, patient care management Abstract Population health care is health information and clinical services provided to individuals of a defined population. From a population health care perspective, quality of care involves the health status of the entire population, and thus issues of access, cost of care, and efficiency matter. In this paper, we describe the definitions of quality health care and the framework for measuring quality, with emphasis on the performance of organizations involved in the delivery and assurance of population health care. We describe quality measurement sets and systems, criteria for the choice of measures, data sources, and how quality measurements are used to improve health care and outcomes from a population health care perspective. INTRODUCTION The population health care perspective acknowledges a multidisciplinary approach to medical care that emphasizes prevention while recognizing the need for efficiency and the consequences of resource allocation decisions (48, 19). The goals of population health care are achievable only with information on indiviuals, their health status, and the quality of their medical care. An organization that practices population health care attempts to maximize the health of individuals in a defined population given available resources and thus shares the perspectives of both personal health care and public health. The ultimate goal of measuring quality is to improve health outcomes by stimulating improvements in health care. An increasing recognition of deficiencies in the quality of health care has spawned interest in making more measurements of quality as a spur to actions to improve health outcomes (7, 32, 40). The increasing focus on measuring and improving quality has occurred at the same time that there has been a demand to demonstrate the value of health care services and to monitor /03/ $

2 364 DEROSE PETITTI the performance of insurance companies, managed care organizations, and health care providers in delivering services. In this paper, we describe the definitions of quality health care and the framework for measuring quality, with emphasis on the performance of organizations involved in population health care. We describe quality measurement sets and systems, criteria for the choice of measures (or indicators) within sets and systems, and data sources. We also describe how quality measurements are used to improve health care and outcomes. Throughout, the paper emphasizes the importance of a population perspective in defining quality and in the choice of quality measures. POPULATION HEALTH CARE Population health care is health information and clinical services provided to individuals of a defined population. In population health care, surveillance occurs and individuals are targeted for interventions based on known or predicted risk. The indicators of success are population statistics in the form of quality of care and organizational performance measures. Population health care can be practiced to some extent by any entity that delivers or assures the delivery of health services to a defined group of individuals. These organizations include insurers, whose population is defined according to enrollment; provider systems, whose population is defined by membership via insurers, corporations, or individual s choice; or by government providers of care, such as the Veteran s Administration, or payers of care, such as the Centers for Medicare and Medicaid Services (CMS), whose populations are defined by eligibility for services. Organizations that pay for or assure the delivery of care, such as CMS, have the regulatory or financial capability to monitor and affect the care of their defined populations. However, by nature of their organizational structure, integrated health care delivery systems that function as a unit across health care disciplines (e.g., primary and specialty providers, nurses, health educators) and settings (e.g., outpatient, inpatient, at home) are currently most involved in the practice of population health care. Population health care seeks to maximize the health and well-being of persons in the defined population given a set of fixed resources and operational constraints. Organizations voluntarily take on responsibility for population health it is rarely imposed externally or mandated contractually. The orientation to populations is indicative of an organization s culture, values, and philosophy of care. Other forces that motivate population health care include marketplace incentives to provide more efficient care, the health professions mission to provide more effective care, the demands by payers that providers demonstrate the quality of care, and information technology (17). Linkages to Public Health Several linkages exist between public health and personal health care, such as the provision of immunizations, cancer screening, and treatment of tuberculosis

3 POPULATION PERSPECTIVE ON QUALITY 365 (2, 43). Population health care broadens these linkages by expanding personal health care services to a population level, and thus population health care shares several features in common with public health. Most important, both seek to maintain or enhance health in a defined group. The methods of surveillance developed in public health are used to monitor the clinical care and health status of persons in population health care. Each relies on programs that deliver specific services to achieve the goal of health maintenance and promotion and emphasizes outreach and prevention. For example, a subpopulation with a high-prevalence condition such as hypertension can be targeted for prevention services that promote healthy habits by means such as flyers or posters. Both operate within systems of constrained resources and must therefore allocate resources to achieve an optimal effect. However, unlike public health services, population health care targets individuals with some common characteristic that selects them, such as enrollment in a health plan, and not geographic boundaries. Unlike public health, population health care generally does not attempt to change the physical, social, or economic environment unless done in cooperation with public health agencies. Linkages to Personal Health Care: Population Health Care Management There is a set of population health care interventions that share much in common with personal medical care and utilization management. Termed population health care management, these interventions focus on individual risk assessment, individually targeted interventions, and modification of practice patterns. Population health care management programs are tools to extend the reach of a provider organization beyond the clinician s office by using data and systems to influence both providers and patients to improve the quality and efficiency of clinical care. For example, information can be automatiaclly generated for physicians at the time of a patient visit and can include reminders about overdue screening tests and the risk of a cardiovascular disease event for persons with diabetes. Organizations that practice population health care management have access to data and, under favorable circumstances, the means to change clinical practice that are often beyond the reach of public health organizations, less integrated health systems, or small provider groups. Measurement and Perspective Table 1 describes some of the distinguishing characteristics of quality and performance measurement from the perspective of personal health care, population health care, and public health. While the primary responsibility in personal health care is the individual patient, population health care and public health must consider the maximization of health across individuals. Thus, the latter use epidemiological methods such as surveillance to explicitly consider costs and target interventions at groups of people. The performance of health care organizations is also assessed to inform consumers. Population health care management, as a distinct set of

4 366 DEROSE PETITTI TABLE 1 Distinguishing characteristics of quality and performance measurement from the perspective of personal health care, population health care, and public health Personal health care Population health care Public health Responsibility Individuals Enrollees Residents of an area Purpose of Clinical care Surveillance of Surveillance of measurement population health population health Organizational Support programs and performance assessment services for the general Support clinical systems population and for interventions for high-risk high-risk subpopulations subpopulations Care management: focus on high-risk individuals How decisions Application of clinical Prioritization of population Prioritization of are made evidence base health problems and population health consequences of resource problems and allocation consequences of resource allocation Care management: Application of public application of clinical health evidence base evidence base approaches to health care delivery within the domain of population health care, uses personal health information, population statistiscs (e.g., the distribution of glycosylated hemoglobin scores among diabetics), risk assessment (i.e., by applying prediction formulas), and evidence-based clinical guidelines to manage the health care of individuals. QUALITY OF CARE AND PERFORMANCE: CONCEPTS AND DEFINITIONS Experts struggled for decades to formulate a single concise, meaningful, and generally applicable definition of the quality of health care (3). There are now several commonly cited definitions (9, 12, 28), which differ in their emphasis on quality of life, the delivery of services, and processes of care as components of quality. A 1990 definition from the Institute of Medicine, which was arrived at after collecting and considering over 100 definitions of quality from the literature, appears often in discussions of the quality of medical care. The Institute of Medicine defined quality as the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge (28). This definition is important in the context of this paper because of its explicit recognition of the population perspective in quality.

5 POPULATION PERSPECTIVE ON QUALITY 367 Donabedian first suggested that there is more than one legitimate formulation of quality depending on the system of care and the nature and extent of responsibilities (11). Blumenthal clarified the relationship between different perspectives and the definition of quality of care (3). He describes four main perspectives on quality the health care professional perspective, the patient perspective, the perspective of health care plans and organizations, and the purchaser perspective which lead to different definitions of quality. Health care providers tend to view quality in terms of the attributes of care and the results of care, leading to definitions of quality that emphasize technical excellence and the characteristics of patient/professional interaction (11, 35). Patients tend to view quality in terms of their own preferences and values, leading to definitions of quality that encompass satisfaction with care, as well as outcomes such as morbidity, mortality, and functional status. Health care plans and organizations that deliver services tend to place greater emphasis on the general health of the enrolled or covered population and on the function of the organization (26), leading to a definition of quality that takes into account the ability of the plan to meet the needs of enrollees. This perspective encompasses decisions to limit some care to assure essential services for all and acknowledges the reality of fixed resources. Purchasers, such as health care organizations, tend to be concerned about population-based measures of quality and organizational performance (3). The purchaser perspective leads to a definition of quality that is similar to that of health care organizations. However, purchasers are very concerned about the value of care, and this concern incorporates the price of care and the efficiency of the delivery of care. A Population Perspective on Quality and Performance The distinction between quality and performance measurement is not clear in the published literature. Performance measurement is the term usually applied when an organization is being assessed (as opposed to a hospital or an individual) and when access, cost, and efficiency are being assessed as a component of quality. Organizational performance measures from a population health perspective are more encompassing than indicators typically encountered in personal health care quality assessment, where the focus is on the individual patient-provider interaction. Performance measures of cost and efficiency include, for example, the number of people a diabetes education program serves and the cost per service. These are also considered to be measures of utilization and can be unrelated to the quality of care of individuals. However, a large amount of resources for an education program that reaches few persons with diabetes might dictate a new approach to outreach or referral or a redirection of resources to prevention. From a population health care perspective, performance measures include indicators of access to care (e.g., waiting time for an appointment, outreach educational letters) because the entire population, and not just those who receive care, is of concern. Other types of performance indicators especially relevant from a population health care perspective include measures of the effectiveness of prevention and patient-centered measures of risk behavior (19). Examples include the proportion of persons in a population

6 368 DEROSE PETITTI with obesity, the proportion of persons in a population who smoke, and the proportion of a health plan s established members who have newly diagnosed diabetes. MEASURING QUALITY AND PERFORMANCE Quality of care must be objectively measured to be useful in crafting strategies for improving health. The alternative is subjective judgments and thus unclear evidence to guide decisions. Organizations must objectively measure performance to engage in successful performance improvement since systems and providers can be held accountable only for things that are measured. Thus, discussions of quality of care and performance quickly transition to discussions of ways to measure quality and performance. Structure, Process, Outcome Quality of care can be measured based on structure, process, or outcome (12 14). Structural measures are the characteristics of the resources in the health system (e.g., the number of beds in a hospital, the number of registered nurses per patient in the intensive care unit). Processes embody what is done to and for a person (e.g., immunization, prescription for a medication, recommendation to cease smoking). Outcomes are the end results of care or the effect of the care process on the health and well-being of patients and populations (e.g, death, satisfaction with care, health status). Implicit in the use of structure and process to measure quality is a link to outcomes. Structure, process, and outcome measures all can provide valid information about the quality of health care (4, 5, 12, 28). Organizational structure may be a root cause of several downstream quality problems and critical to the success of provider organizations. The main advantage of structural measures of quality is their relative ease of measurement. The disadvantage is the often indirect link between the structural variable and desired outcome. For example, staffing of intensive care units with physicians trained as intensivists is being used as a performance indicator by the Pacific Business Group on Health, a consortium of insurance purchasers operating in California. The explicit purpose of this staffing pattern is to improve the outcome of patients cared for in an intensive care unit. There is some empiric evidence of a correlation between staffing and better outcome. But the link between staffing and better outcome clearly is indirect it must be the things that are done by the intensivist or a system or institution that uses intensivists and not strictly the presence of the intensivist that improves outcome. Process measures are appealing to providers because they are directly related to what providers do. They are actionable, often pointing directly to areas where care needs to be improved. The appeal of a measure to providers is critical because changes in what providers do and say often determines the success or the failure of efforts to change outcomes. However, there are several arguments against process measures as measures of quality of care. Processes are not necessarily important predictors of outcome, and directing resources at processes that do not affect

7 POPULATION PERSPECTIVE ON QUALITY 369 outcomes may increase health care costs without producing health improvements (16). It may be difficult to achieve consensus on the correct process for many clinical problems, especially when the evidence-base is poor. Outcome measures also appeal intuitively to providers and can also be used to target quality improvement efforts. But some outcomes (e.g., cancer mortality) are rare and thus comparisons of quality based on rare outcomes are often of low statistical power (4). Many outcomes are not under the control of the health care providers, and conclusions about the quality of care based on them may be invalid. Moreover, it may be difficult to assign responsibility for outcomes to a single provider or a system of care when there are no clear points of entry or exit to the health care system (28). For example, a case of hepatitis B in an adult member of a health care organization might occur because of failure of the pediatrician in another organization to vaccinate appropriately during infancy. PERFORMANCE MEASUREMENT SETS AND SYSTEMS A performance measurement set or system is an interrelated set of process and/or outcome measures that facilitate internal and external comparisons of an organization s performance over time (27). Specific performance measurements within a set measure the quality of care or some other desired attribute of care delivery (i.e., access, efficiency). Performance measurements sets are developed by regulatory bodies, such as state and federal governments; agencies that accredit health care organizations, such as the Joint Commission on the Accreditation of Healthcare Organizations (JCAHO); voluntary organizations representing disease constituencies, such as the American Diabetes Association; professional societies, such as the American College of Cardiology; and organizations that themselves deliver health care services. The motivation for developing measurement sets is generally the same for each type of organization involved in developing them to define quality and drive its improvement and to create accountability for demonstrating and improving quality. Quality measures are used for accreditation of institutions, to provide accountability to payers of care, to educate consumers, for internal quality improvement, and for research into care delivery. There are several measurement systems for evaluation and comparison of hospital care (27). A set of quality indicators developed specifically for use with hospital discharge abstract data have been developed in the Healthcare Cost and Utilization Project (HCUP). JCAHO has recently begun to require reporting of some measures of quality and outcome and to take performance on these measures into account in making decisions about accreditation. To the extent that persons hospitalized for a given condition constitute a population, the JCAHO measures are indicators of population health care performance. The Health Plan Employer Data and Information Set (HEDIS ), first established in 1991, is a performance measurement system (8) that has been used by the National Committee on Quality Assurance (NCQA) to compare the quality of

8 370 DEROSE PETITTI care of Health Maintenance Organizations (HMOs). HEDIS measures have been extended to other kinds of insurance schemes. Medicare and Medicaid require reporting of HEDIS measures separately for beneficiaries in HMOs. Table 2 shows topic items under two types of HEDIS measures, effectiveness of care and access/availability of care, which are selected from the 2001 HEDIS measurement set (33). To follow these HEDIS topics, a potential measure of childhood immunization status is the proportion of children who have had all immunizations recommended by the American Academy of Pediatrics by 24 months of age. Another potential measure, of breast cancer screening, is the proportion of women age 50 years and older who have had a mammogram in the past two years. These measures qualify as population health care measures. HEDIS performance is considered in decisions of NCQA to accredit health plans. HEDIS scores form the basis for report cards, which are widely publicized for the use of consumers and organizations in choosing a health plan. Performance measurements to be used in public health settings have also been developed (10, 23, 36), and a national initiative is under way to assess local and state public health system quality (39). Many states and localities have developed or conducted performance assessment (29), although internal, quantitative quality assessment in local health departments is not yet routine. TABLE 2 Selected types of HEDIS 2001 measures (33) Effectiveness of care Childhood immunization status Adolescent immunization status Breast cancer screening Cervical cancer screening Chlamydia screening in women Controlling high blood pressure Beta blocker treatment after a heart attack Cholesterol management after acute cardiovascular events Comprehensive diabetes care Use of appropriate medications for people with asthma Follow-up after hospitalization for mental illness Antidepressant medication management Advising smokers to quit Flu shots for older adults Pneumonia vaccination status for older adults Medicare Health Outcomes Survey Access/availability of care Adults access to preventive/ambulatory health services Children s access to primary care practitioners Prenatal and postpartum care Annual dental visit Availability of language interpretation services

9 POPULATION PERSPECTIVE ON QUALITY 371 CHOOSING QUALITY AND PERFORMANCE MEASURES Different perspectives and definitions of quality, as described above, call for different approaches to quality measurement (11). It is helpful to clarify the purpose of measurement in order to guide the selection and development of measures. We focus our discussion here on the purposes of organizations involved in population health care. Measures are chosen or developed for internal performance improvement by provider organizations. Measures for accreditation or external reporting are developed by agencies that assure care delivery, such as NCQA. Many measures can be used for both purposes. Measures of population health care inevitably involve consideration of data resources. Measures developed by care providers may emphasize operational constraints and the organization s capacity to effect change, and may include performance indicators that are specific to the care delivery system and the problems facing the population or the organization. Population health care measures developed by agencies that assure care delivery must be applicable across a wide variety of settings. The quality of clinical care for populations is generally assessed by either process or outcome measures, since structural quality measures are the most distant from outcomes and have their effect through changes in processes. Organizational systems performance also can be assessed by carefully considered measures of utilization (such as hospitalization rates among diabetics), access to care (such as waiting time for a mental health appointment), and time-efficiency (such as the turn-around time for medical test reports). The meaning of these measures must be carefully considered because they can be dissociated from quality (e.g., high quality may exist with either high or low utilization) or have a complex relationship to quality. For example, there might be an optimal range for number of physicians and the size of their patient panels that preserve organizational viability in a competitive marketplace (having fewer physicians and larger panels) while not reducing needed access to care (having more physicians and smaller panels) and not encouraging medical errors due to time constraints (having smaller panels). Although perfect measures and measurement systems do not exist, more or less useful measures do. Several criteria are helpful to guide the choice of performance measures. These criteria (e.g., meaningfulness or relevance, feasibility, actionability, reliability, validity) have been expressed in a variety of ways in published sources, but all revolve around similar key concepts (15, 30, 31, 36). Key issues for measuring quality and performance from a population health care perspective are discussed below. Topic Areas for Measurement Organizations providing or assuring the quality of population health care will chose topic areas for measurement where a potential exists to significantly improve heathrelated quality of life and decrease morbidity and mortality among members of the defined population. An assessment of the cost effectivness of care associated with

10 372 DEROSE PETITTI a potential topic area will help determine its relative importance for measurement. One starting place for identifying areas of importance for quality assessment and improvement is the health care goals defined by Healthy People 2010 (34) and many complementaty plans by state and local health agencies. These goals are chosen to improve the quality and length of life, as well as decrease health disparities, in the U.S. population. Specific population subgroups may have needs that differ from the entire U.S. population. National data sources, such as the Morbidity and Mortality Weekly Report and publications from the National Center for Health Statistics (both under the Centers for Disease Control and Prevention), provide geographically based population health information. Information on members of a health plan is often available from electronic administrative and clinical care databases and can include causes of mortality, morbidity, and utilization. Using sources such as these, the significance of the conditions and risk factors (such as smoking) on morbidity, mortality, and health resource use can be estimated for a defined population (30). Evidence Base Measures are most successful in driving improvements in care when the actions they represent are evidence-based (or have a clear, logical link to outcomes). For example, a performance measure for diabetes care that is based on data from well-conducted randomized trials (e.g., use of ACE-inhibitors in diabetic patients with microalbuminuria) is a better candidate for successful quality improvement interventions than measures based on currently weaker evidence, such as screening everyone with hypertension for chronic kidney disease. In recent years, the evidence base with which to make personal health care decisions has received increasing emphasis. During the latter half of the twentieth century, randomized controlled trials became the standard of medical knowledge, and systematic reviews that synthesize the results of multiple scientific investigations have grown in influence (7). The evidence base for population health care is essentially the evidence base for personal health care, rather than a combination of personal and public health evidence. As the evidence base for public health becomes more established (45), and if cooperation between public health agencies and health plans grows, the public health evidence base likely will gain influence on the practice of population health care. Population health care management applies the generalizable results of scientific studies to estimate the risks an individual faces and the benefits of interventions to modify those risks the traditional domain of the health care professional. A wealth of evidence exists to support important population health care interventions. For example, the Heart Outcomes Prevention Study (HOPE) provides strong evidence for the beneficial effect of angiotension converting enzyme (ACE) inhibitors in persons with diabetes over the age of 55 years and who have cardiovascular disease or another established cardiovascular risk factor (20). On the basis of this

11 POPULATION PERSPECTIVE ON QUALITY 373 evidence, patients who satisfy these criteria but are not on the recommended medication can be identified, and information or prompts to action for physicians or patients can be delivered. For example, a specific intervention used at Kaiser Permanente Southern California is a computer-generated datasheet for scheduled appointments that lists medications, relevant lab data, other useful information, and specific recommendations for care that are evidence-based, tailored to the individual patient (e.g., ACE inhibitor by HOPE study criteria, as described above), and easy for providers to act upon at the point of service to the patient. For clinical quality improvement, it is helpful to clarify the objectives and process of care. Evidence-based guidelines that explicitly state actions for specific objectives can help define performance measures and provide a model for improving the process of care. Rigorously developed guidelines, such as those of the U.S. Preventive Services Task Force (47), are an excellent source of information. However, the evidence base, and guidelines produced from evidence, are necessary but not sufficient for identifying measures that have the potential to improve quality. Actionability Actionability the ability of resources applied by the health care provider or system to significantly affect the measure is also an important consideration in choosing measures that can be used to drive performance improvement. For example, computerized reminders to physicians at the time of an appointment have the potential to produce improvements in routinely scheduled services, such as cancer screening tests (41) or monitoring tests for a person with diabetes (e.g., LDL, HgbA1c). Providers and managers are much more supportive of proposed measures when change for the better is possible. Measures may also be more actionable when supported by other stakeholders, such as payers of care (business, the government), or patients, whose preferences provide justification for additional resource diversion into the area of concern. An example of an actionable measure, developed by the Picker Institute (38) for assessing hospital care, is Were you given enough privacy when discussing your condition or treatment? The responses are Yes, always, Yes, sometimes, and No. This type of question can lead directly to actions that improve care. Compare this to a common type of satisfaction measure, How satisfied are you with your pain management? with a five-point response from Very dissatisfied to Very satisfied, and it is obvious that the latter measure is not readily actionable because reasons for the level of satisfaction are not specified by the question. High levels of performance sometimes can be associated with diminishing returns for increasing costs, or a level can be reached at which the cost associated with achieving better performance in one area may negatively affect quality in other areas. For example, resources may be better used to screen previously untested members for cervical cancer than to increase the return rate for repeat mammography screening. These considerations affect resource distribution and the actionability of measures.

12 374 DEROSE PETITTI STANDARDIZATION AND AUDIT OF QUALITY AND PERFORMANCE MEASURES When comparing the performance of organizations or providers, it is essential that any differences in the measurements are not due to differences in measurement methodology and thus permit valid conclusions about variations among the units being compared. The validity of comparisons is achieved by standardization of the methods for making the measures. This is done by provision of detailed rules for data collection and reporting and development of protocols for collecting data for the measures. These descriptions and protocols are called specifications. Table 3 gives example specifications for three measures from a diabetes measurement set used to compare performance across Kaiser Permanente. Kaiser Permanente is America s largest not-for-profit health maintenance organization, serving 8.1 million members, including approximately 380,000 members with diabetes, at 29 medical centers in nine states. Valid conclusions about time trends in performance requires that the methodology for the measure be stable over time. Changes in the definitions of numerator or denominator for a clinical performance measure will change the measure. In addition, changes in the accuracy or completeness of the data that generate the measure can affect trendability. For example, improvements in the recording of immunizations in health plan medical records of immunizations received outside the health plan would results in an improvement in a measure that is not accurately indicative of improvement in the quality of care, only improvements in the quality of the data. Quality and performance measurements in some performance sets are subject to routine audit by external agencies. Audit is especially likely when the measures will be publicly reported and might affect decisions of individuals or organizations to purchase or pay for health care from the reporting organizations. For example, HEDIS clinical measures are audited by a large accounting firm and only measures that pass the audit can be publicly reported. DATA SOURCES AND DATA QUALITY Computer-Stored Administrative and Clinical Data It is attractive to use already collected computer-stored data for quality and performance measures because there are no costs associated with collecting these data, the data are easily accessible, the number of people or events included in many databases is large, and the data are often population-based. Computer-stored data available to health systems can be divided broadly into two categories administrative data and clinical data. Administrative data are collected for business purposes and include information collected at the time of enrollment or premiums/dues payment (e.g., name, address, date of birth, social security number) and information used for billing and claims processing (e.g., dates

13 POPULATION PERSPECTIVE ON QUALITY 375 TABLE 3 Example specifications of year 2000 diabetes performance measures from Kaiser Permanente Measure Specification The percentage of members with Denominator: The number of members identified as diabetes who were tested with a having diabetes a hemoglobin A1c test during Numerator: The number of members in the denominator the reporting period who had at least one of the following tests for glycemic control with a test date during the reporting period: Hemoglobin A1c or total glycated hemoglobin Calculation: Laboratory data are used to identify those members with diabetes who receive at least one test for hemoglobin A1c during the reporting period (one year) The percentage of members with Denominator: The number of members identified as having diabetes whose LDL-C is (a) diabetes who were 18 years of age by 01/01 of the <130 mg/dl for patients without reporting period coronary artery disease (CAD), Numerator: The number of members in the denominator and (b) <100 mg/dl for patients with a most recent LDL-C value that is (a) <130 mg/dl with CAD for patients without CAD, and (b) <100 mg/dl for patients with CAD Calculation: A CAD case-identification database is used to identify members with CAD. Laboratory data are used to identify members LDL-C values during the reporting period (one year) The age-adjusted rate of hospital Denominator: The number of members identified discharges for medical/surgical as having diabetes care per 1000 members with Numerator: The number of discharges from an inpatient diabetes care facility, with a principle diagnosis that excludes nonmedical or nonsurgical care (ICD-9 codes are identified), with a discharge date during the reporting period Calculation: Hospital discharge data are used to identify discharges for members with diabetes during the reporting period (one year). Members are classified into six age classes that are used to age-adjust to the total diabetic membership a Members with diabetes are identified by any of the following criteria, which are abbreviated for clarity: (i) one or more dispensations of medications used to treat diabetes; (ii) an inpatient admission with a principle or secondary discharge diagnosis of diabetes mellitus; (iii) two outpatient visits in a 24-month period with diabetes mellitus as a principle or secondary diagnosis; (iv) an elevated glycosylated hemoglobin test result. Members with a diagnosis of gestational diabetes are excluded. Pharmacy, laboratory, hospital discharge, and outpatient visit data are used to identify members with diabetes. A case-identification database is created.

14 376 DEROSE PETITTI TABLE 4 of hospitalization, DRG or ICD-9 codes). Clinical data are collected to support clinical practice and patient care and include information collected in electronic medical records. There is overlap, and some data collected mainly for administrative purposes have clinical uses. For example, data from prescriptions may be computer stored to facilitate administrative functions such as refills and drug costaccounting, but the information may also be used clinically to manage patients. For many types of computer-stored data, there is no clear distinction between administrative and clinical uses. For example, the use of hospital discharge data used to monitor hospital discharges for procedures is both administrative and clinical. Computer-stored data collected strictly for administrative reasons generally do not contain clinical detail (see Table 4). The accuracy of both computer-stored administrative and clinical data is uncertain until evaluated because of the circumstances under which the data are collected. Data in both computer-stored administrative and clinical databases may not be complete. Data fields (e.g., birth date, race/ethnicity) may be missing in a membership database. Information on services received outside the system may not be captured. For example, influenza shots given at a community clinic may not be recorded in a health plan s immunization database. Prescriptions filled in the local drugstore will not be recorded in the prescription database. Other well-recognized limitations associated with coded data from hospital discharges and ambulatory visits include imprecise coding of diagnoses and procedures, lack of specificity of codes, undercoding of comorbidities and complications, and different levels of coding completeness across institutions. All of the sources of inaccuracy and incompleteness affect the usefulness of computer-stored data for quality and performance measurement and the validity of conclusions based on the measure. Data from outside a health system may be of interest for measuring performance, especially for research purposes. Linking these data to those available within a health plan poses the problem of matching individual records since unique person Strengths and limitations of administrative versus clinical data Characteristics of data Administrative data Clinical data Clinical detail Low High Accuracy Uncertain Uncertain Lower if judgment Higher if a system of or memory is required standardization exists (e.g., for coding laboratory test calibration) Completeness Varies Varies Higher if data is required, Higher if data entry is automatic entry is automatic, there or access to health plan are significant financial services is good implications, or access to health plan services is good

15 POPULATION PERSPECTIVE ON QUALITY 377 identifiers are not shared. Moreover, it may be difficult for health plans to assess the accuracy of data external to their systems, and data like those on mortality can be delayed up to a year. Medical Records Medical records, both paper and electronic, are an important source of information for performance measurement. Information from the medical records may be the sole basis for a performance measure or it can be used to supplement computerstored data. For example, the accuracy of a measure of beta-blocker use after acute myocardial infarction as a measure of quality could be improved by reviewing the paper charts or the electronic medical records of people who did not receive a betablocker to determine whether this is because of a contraindication to beta-blocker use. Medical records, both paper and electronic, are not good sources of data on mortality outcomes, functional status, or patient satisfaction unless special attention has been placed on collecting such information in the course of clinical practice. Medical records may not themselves be complete. For example, the contraindication to beta-blocker use may not be recorded in the medical record. Finally, there is almost never a protocol for the recording of information in a medical record. Providers may differ in their threshold for recording whether the patient had a comorbid condition or may use different criteria to make a diagnosis. Patient Report Statements from individuals about care or about their perceptions are another source of information on processes and outcomes. They are the only source of data on satisfaction with care and most measures of functional status. Performance measurement based on reports from individuals can be costly when based on surveys of large populations. All surveys are subject to bias due to nonresponse. Nonresponse tends to be high in surveys done for measurement of the quality of care. For example, in Kaiser Permanente, the Picker Institute survey of hospital performance (18, 38) had a response rate of 40%, and an internal survey of satisfaction with pregnancy and newborn care had a response rate of 33% (25). Little work has been done to determine whether respondents to surveys of the quality of care are representative. Finally, problems exist with the accuracy of self-reported data. Patients do not always accurately report their receipt of services. For preventive services, there is a tendency to telescope, reporting receipt of services more recently than they were actually performed (44). Some individuals may not know which services they have received. A person may, for example, know that they had a shot but they do not whether the shot was for influenza or pneumococcal pneumonia. Consumer satisfaction ratings may not always be reliable indicators of quality because consumers cannot always deem whether services are appropriate or technically good (1), and the ratings may not reflect the outcome for the entire population at risk

16 378 DEROSE PETITTI (e.g., a patient with a contagious disease may dislike a treatment program even though it is beneficial to the patient and others). MEASUREMENT TO IMPROVE POPULATION HEALTH Population health care is dependent on performance measures to objectively assess the effectiveness of its programs. In clinical medicine, one-on-one patient-provider contact and individual test results are used to judge the status and progress of patients. In population health care, individuals are risk-assessed and interventions prompted, but the measures of success are population statistics. Success is judged by comparisons with past performance, benchmarks, or the performance of similar organizations. Population health care measures generally do not delve into the mechanism of action of complex care interventions, such as the diverse effects of a self-care educational program for a chronic disease. Detailed analysis of such interventions is the realm of program evaluation. Rather, the receipt of services, the immediate results they produce, and ultimate outcomes are monitored over time. Performance measures thus point toward opportunities for improving care. Since it is impractical to measure all aspects of care, even by rotating measures periodically into and out of use, performance measures cannot reveal every reason for a given level of performance. For that, subject matter knowledge and creativity are required to extract information that is useful for performance improvement. For example, if satisfaction with sexually transmitted disease and birth control counseling services is low among teens, then use of these services may decline and disease or unintended pregnancies may result. Efforts should be made to understand the reasons for low satisfaction, such as fear of an overly visible waiting area, so that corrective measures can be taken and result in improved performance. Invalid conclusions can sometimes be drawn from measures that are not fully interpretable without understanding data limitations, potential confounding factors (i.e., covariates), and what constitutes good performance. For example, an influenza outbreak can significantly change hospitalization rates among diabetics, but without that knowledge, an increase in rates might be attributed to a variety of reasons, and may not include a discussion of rates of influenza immunization. Thus, the users of performance results need to be educated about the limits and interpretation of results. Organizational Conditions for Performance Improvement For performance measures to have an effect on performance improvement, clear objectives must exist, and there must be an infrastructure of individuals with subject matter knowledge and an ability to change the system of care. Objectives are often chosen that require a strong effort but are potentially achievable, with an idealized goal in mind. For example, a goal for breast cancer screening might be to maximize mammography rates among eligible women, recognizing that there will always be

17 POPULATION PERSPECTIVE ON QUALITY 379 members who chose not to be screened or continually defer making a conscious choice even after appropriate outreach is made. The objective for a given period of time might be to raise the absolute screening rate from 75% to 85% after considering the current rate, what screening rate is potentially achievable given the resources at hand, and the opportunity costs involved. The question of what organizational structure will achieve the best results is not well understood and may depend on the organization s mission, culture, leadership, resources, and accountabilities. Integrated health systems with hospitals, clinics, primary care, specialty care, comprehensive data systems, and a focus on prevention are clearly in a position of strength to act on and alter care relative to other organizations that are more fractionated, have less access to comprehensive data, and focus less on the long term. In addition, accountability to shareholders or the needs of one large payer may limit an organization s response to problems regarding current and future performance by focusing improvement only on publicly reportable performance measures or a subpopulation of their membership. Population health care is focused on risk reduction and therefore is more likely to be practiced by organizations with a long-term, broad view of health care. There is no question that health professionals who understand medical evidence and ways to create changes in the care delivery system are needed to accomplish continuous, positive change within any provider system. Such experts should work within an organizational structure that supports both creativity and the reaching of consensus, the authority to overcome stasis and take action, and the responsibility to monitor and be accountable for results. One example is a model employed at Kaiser Permanente Southern California, in which lead physicians, nurses, pharmacists, educators, and others form teams in each of several areas served by a Kaiser Foundation Health Plan medical center. These teams meet periodically to compare performance across sites, discuss programs in place, identify problems and opportunities for improvement, brainstorm solutions, plan a response, and monitor the results in a continuous cycle. Because the members are leaders within their clinical areas (e.g., diabetes care), and the organizational culture to support this approach exists, they are able to create positive changes in this large and complex health care system. For example, with a host of interventions employed for diabetes care, improvements in several measures of screening for dyslipidemia and microalbuminuria, and therapy for dyslipidemia and hyperglycemia, were seen over several years, and are believed to be largely due to performance monitoring, feedback, and multifaceted population health interventions, rather than simply due to changes in practice patterns over time (22). A longstanding and successful population-based approach to chronic illness care and prevention has been described at Group Health Cooperative of Puget Sound (19, 46, 48). Performance feedback, at a group or individual provider level, is one method of creating change (24, 37, 49). Sometimes, incentives accompany performance (21). Using the method of feedback, the details of change are often left to the provider groups whose performance is assessed. Although engaging providers by