Tilburg University. Measuring health system performance Heijink, Richards. Document version: Publisher's PDF, also known as Version of record

Transcription

1 Tilburg University Measuring health system performance Heijink, Richards Document version: Publisher's PDF, also known as Version of record Publication date: 2014 Link to publication Citation for published version (APA): Heijink, R. (2014). Measuring health system performance Enschede: Gildeprint General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. - Users may download and print one copy of any publication from the public portal for the purpose of private study or research - You may not further distribute the material or use it for any profit-making activity or commercial gain - You may freely distribute the URL identifying the publication in the public portal Take down policy If you believe that this document breaches copyright, please contact us providing details, and we will remove access to the work immediately and investigate your claim. Download date: 28. apr. 2018

2 Measuring Health System Performance Richard Heijink Heijink.indd :15:42

3 The research described in this thesis was carried out at the Centre for Prevention and Health Services Research, National Institute for Public Health and the Environment (RIVM), Bilthoven, the Netherlands, and at the Scientific center for care and welfare (Tranzo), Tilburg University, Tilburg, the Netherlands. The studies described in this thesis could not have been performed without the financial support of the National Institute for Public Health and the Environment (RIVM) and the Dutch Ministry of Health, Welfare and Sport (VWS). Cover design: Diana de Man Lay-out and printing: Gildeprint Drukkerijen, Enschede, the Netherlands ISBN/EAN: Copyright R. Heijink, 2013 All rights reserved. No parts of this publication may be reproduced in any form without permission of the author. Heijink.indd :15:42

4 Measuring Health System Performance Proefschrift ter verkrijging van de graad van doctor aan Tilburg University op gezag van de rector magnificus, prof. dr. Ph. Eijlander, in het openbaar te verdedigen ten overstaan van een door het college voor promoties aangewezen commissie in de Aula van de Universiteit op vrijdag 17 januari 2014 om uur door Richard Heijink geboren op 14 juli 1982 te Diepenveen Heijink.indd :15:42

5 Promotiecommissie Promotor: Prof. Dr. G.P. Westert Copromotor: Dr. A.H.E. Koolman Overige leden: Prof. Dr. J.A.M. Maarse Prof. Dr. F.T. Schut Prof. Dr. D.M.J. Delnoij Prof. Dr. D.H. de Bakker Dr. P.P.T. Jeurissen Heijink.indd :15:42

6 Table of Contents Chapter 1 General Introduction 7 Chapter 2 Decomposing cross-country differences in Quality Adjusted Life Expectancy: the impact of value sets 23 Chapter 3 International comparison of experience-based health state values 51 Chapter 4 Cost of illness: an international comparison Australia, Canada, France, Germany and the Netherlands 77 Chapter 5 Spending more money, saving more lives? The relationship between avoidable mortality and healthcare spending in 14 countries 97 Chapter 6 International comparison of chronic care coverage 123 Chapter 7 Measuring and explaining mortality in Dutch hospitals; The Hospital Standardized Mortality Rate between 2003 and Chapter 8 Effects of regulated competition on key outcomes of care: Cataract surgeries in the Netherlands 163 Chapter 9 Benchmarking and reducing length of stay in Dutch hospitals 183 Chapter 10 General Discussion 199 Summary 222 Samenvatting 228 Dankwoord 234 Curriculum Vitae 237 List of publications 238 Heijink.indd :15:42

7 Heijink.indd :15:42

8 Chapter 1 General Introduction Heijink.indd :15:42

9 Background Dutch health care world-class [1]; Time to learn from the Dutch champions how to build value-for-money healthcare [2]; Dutch health care pretty good [3]; Too much variation in quality of care in the Netherlands [4]; Managed Competition for Medicare? Sobering Lessons from the Netherlands [5] This is just a small sample of recent quotes on the performance of the Dutch health system. Although these conclusions create quite different pictures, they have one thing in common. They reflect the ongoing search for health system performance information by researchers, policy makers and the general public. In recent decades, the demand for public accountability and transparency in health systems has increased internationally [6,7]. Patients and citizens need information on the performance of health care providers in order to choose where to be treated and where to get the best care available; health insurers require performance information for negotiations with health care providers; and policy makers need to track the performance of the health system to evaluate and prepare policies and reforms. In recent years, various health system reforms have been implemented internationally that require close monitoring, such as marketbased reforms, the introduction of pay-for-performance mechanisms and integrated care. Besides, policy makers may want to assess whether public resources are well-spent and whether the continuously rising health expenditures provide sufficient value [8,9]. In 2008, the World Health Organization (WHO) Member States in the European Region even signed an agreement, the Tallinn Charter, committing themselves to promote transparency and be accountable for health system performance to achieve measurable results [10]. Health system performance information was considered one of the main building blocks of stronger and more valuable health systems; Health systems need to demonstrate good performance. This thesis includes a set of studies developed as background research for the Dutch Health Care Performance Report [11]. From 2006 onwards, the Dutch Ministry of Health has commissioned the National Institute for Public Health and the Environment (RIVM) to produce this report on a regular basis, in order to monitor the performance of Dutch health care. Similar studies have been published in other countries. There are examples from Australia (Australia s Health), the US (National Healthcare Quality Report), Canada (Health Indicators), and Sweden (Quality and Efficiency in Swedish Health Care) [11-15]. In addition, several international agencies performed cross-country comparisons of health system performance, such as Health at a Glance of the Organisation for Economic Co-operation and Development (OECD) and the health system reports of the Commonwealth Fund [16,17]. These studies all aim to translate a great amount 8 Chapter 1 Heijink.indd :15:42

10 of information into conclusions about the quality and efficiency of the health system. Do health systems meet their objectives and at what expense? 1 Glimpse of the literature Early attempts of performance assessment in health systems, dating back to the beginning of the 20 th century, were aimed at tracking individual patients after a particular hospital treatment [18,19]. The few pioneering investigators at that time focused on treatment outcomes in terms of patients health. Nowadays, improving health outcomes is still considered the main goal of health services and health systems. Consequently, a comparison of the health status of populations, in relation to the amount of resources invested in health systems, may reveal how well health systems perform. As argued by WHO, it is achievement relative to resources that is the critical measure of a health system s performance [20]. Figure 1 depicts this relationship for 191 countries in 2009, using per capita health expenditure (total resources invested in personal medical care plus prevention and public health services) and life expectancy at birth. The figure demonstrates a positive association between total health expenditure and life expectancy at birth. It suggests that greater investment in health systems provides better population health. This may be the result of greater coverage (in terms of patients, services, or reimbursement) or the use of more expensive and more effective treatments. The figure also indicates that the marginal returns to health spending decrease as the level of health spending increases. Furthermore, countries with similar levels of health spending reach different levels of health, suggesting that some health systems perform better than others. However, before drawing strong conclusions, it must be considered that things may be more complex. Several factors confound the association between health spending and population health, such as socioeconomic conditions. A number of studies published in the 1960 s and 1970 s clearly pointed to this issue, in critical reviews on the role of medicine [21,22]. In these studies, it was argued that the mortality decline between the mid-19 th century and the mid-20 th century largely occurred before the introduction of major medical treatments. Therefore, improvements in population health were attributed to improved economic and social conditions and better nutrition, but not to better or more health services. Not surprisingly, these conclusions generated widespread discussion on the benefits of health systems and various researchers in the fields of medicine, demography, epidemiology, and health economics have aimed to unravel the issue since [23-25]. In this area of research, different types of empirical studies can be distinguished with regard to their perspective and type of data used. Various studies analyzed the association between General Introduction 9 Heijink.indd :15:42

11 90 80 Life expectancy at birth Per capita health expenditure (US$ PPP) Figure 1: Relationship between per capita health expenditure (in US$ PPP) and life expectancy at birth for 191 countries in 2009* Source: WHO Global Health Observatory, Accessed February 2013, * PPP = Purchasing Power Parities health spending and life expectancy using aggregated cross-country (panel) data and controlling for confounding variables such as national income, environmental factors, or lifestyles (for an overview see [26]). Most of these studies found a positive association between health spending and population health. Others used a disease-perspective, investigating disease-specific mortality trends in combination with information on the effectiveness and the timing of the introduction of medical treatments [9,27-29]. The general conclusion from these studies seems to be that, especially in recent decades and for specific conditions as infectious diseases and cardiovascular disease, medical care did play a significant role in reducing mortality rates. Other studies applied a regional approach. For example, it was shown that in Canada higher spending regions achieved lower mortality rates, after controlling for socioeconomic and lifestyle factors [30]. Fisher et al. showed that higher spending regions in the US did not achieve better mortality, functional status or satisfaction with care, after controlling for various patient characteristics [31,32]. More recent 10 Chapter 1 Heijink.indd :15:42

12 studies from the UK combined the regional-level and disease-level approach, showing that for most of the disease categories studied, health care spending had a demonstrably positive effect on health outcomes, after controlling for differences in need between regions [33,34]. 1 The World Health Report 2000 published by WHO is generally considered one of the landmark studies on health system performance [20,35]. In this study, WHO examined the average relationship between health expenditures and health, but also attributed systematic variation between countries to the countries health systems. In other words, given the amount of resources invested, countries were held accountable for achieving worse population health compared to other countries. The WHO researchers did control for differences in the level of education between countries, because it may affect health outcomes beyond the control of health systems. At the same time, they did not adjust for lifestyle factors that may affect population health, because these were considered within the control of health systems. Overall, France showed the best-performing health system, reaching the highest level of population health (healthy life expectancy) given the available resources (total health spending). Instead of life expectancy or healthy life expectancy, researchers have used more specific health measures to assess health system performance. One of the main concepts used is avoidable mortality, which focuses on a group of diseases where clinical evidence has shown that health services affect mortality [36]. The concept of avoidable mortality was introduced in the 1970 s as indicator of the quality of health systems [37]. It was shown that avoidable mortality rates declined significantly faster than all other mortality rates in recent decades, pointing to a non-negligible contribution of medicine to population health [36]. In addition, various studies showed that the level of avoidable mortality differed significantly between and within countries [36], indicating that certain countries (or regions) performed better than others. Alternative performance measures that do not directly reflect health outcomes have been proposed too, such as the concept of health system coverage [38]. Health system coverage concentrates on whether health systems are able to deliver services to people in need of care, which is considered an important way through which health systems contribute to health outcomes. WHO has published countrylevel coverage estimates for different preventive interventions, such as (DTP3) immunization coverage among 1-year olds (see In addition to these macro-level and disease-level approaches, many performance studies have been conducted at the organizational level, concentrating on the performance of particular providers of health services in terms of quality or efficiency (see e.g. [18,39]). The main idea of these studies is to attribute variation in health outcomes or other performance measures to individual institutions. As such, they may provide information about specific actors within the health General Introduction 11 Heijink.indd :15:42

13 system with lacking performance. Organizational performance studies predominantly focused on hospital care [18]. These hospital performance studies have commonly used mortality rates (e.g. in-hospital mortality or 30-day hospital mortality) as performance measure. Other output measures that have been used are e.g. the number of patients treated (assuming that treating more patients equals producing more health), in-hospital length of stay (efficiency indicator), and readmission rates or disease-specific complication rates (both quality measures) [6,40]. Conceptual and methodological issues Given the increased interest in and use of health system performance studies, it becomes all the more important to identify, clarify, and address conceptual and methodological issues at hand. As shown by the responses to WHO s World Health Report 2000, performance studies can be heavily discussed [41-45]. Recently, Smith argued: Despite widespread acceptance that the pursuit of health-system productivity (ratio of some valued output(s) to resources consumed) should be a central goal, its measurement remains elusive [46]. In this section, we first describe a general framework that can be used as starting point for health system performance studies. Subsequently, we highlight specific methodological and conceptual issues that arose from the literature. Health system performance framework A conceptual framework provides better understanding of the relationship between the input(s) and output(s) of the health system, and helps to reflect the goals, the setup, and the nature of the functioning of the system in question [47]. Various health system performance frameworks have been developed (see [48] for an overview), though, most probably, a perfect health system performance framework does not exist [47]. Therefore, a more generic conceptual framework is presented here in figure 2, based on Jacobs et al. [6]. The middle column of figure 2 shows the basic input output relationship: inputs such as labor (e.g. doctors and nurses) and capital are transformed into output such as better health, through activities or interventions. This process can be assessed at different levels; the individual doctor, a health care institution, a chain of providers and services, or the entire health system. As defined by WHO, the health system comprises all actors, institutions and resources that undertake health actions, where the primary intent of a health action is to improve health. Consequently, the health system is a broader entity than the health care system, which includes all personal medical care and public health activities [48]. Health system performance reports commonly apply a system-level perspective complemented with analyses of different sectors, diseases, or providers. Jacobs et al. identified some generic concerns regarding the unit of analysis in the context of performance analysis [6]. First, the unit 12 Chapter 1 Heijink.indd :15:43

14 External output: social benefits (productivity gains) Output: health improvement, responsiveness (average and distribution) Joint output: research & training 1 Endowments year t-x Activities in unit X Endowments year t+x Exogenous factors: e.g. socioeconomic conditions, health behavior, demographic Input: capital, labor System constraints: e.g. policy and physical constraints structure Figure 2: Generic health system performance framework* * Jacobs et al. ([6], p.38), adjusted by the author of analysis should capture the entire production process of interest. Second, the unit of analysis should be a decision making unit, i.e. it should convert resources into products and outputs or be able to influence this process through regulation. Third, the units compared should be comparable, in other words, produce a similar set of services or products. As mentioned in the previous section, the health production process can be influenced by exogenous factors beyond the control of health systems. Figure 2 shows this can involve population characteristics in terms of socioeconomic conditions (e.g. income, unemployment), health behavior (e.g. lifestyle habits) or demographics (e.g. age structure). Such factors can influence the use of resources and health outcomes, or other outputs. As far as such factors are considered beyond the control of health systems, they should be controlled for. The latter is commonly referred to as risk adjustment [49]. Figure 2 gives a rather generic list of possible risk-adjusters. The exact operationalization will depend on the outputs and inputs measured and the unit of analysis, as different units may have different functions and objectives. Furthermore, the role of e.g. population characteristics may differ between output measures. For example, the General Introduction 13 Heijink.indd :15:43

15 impact of age on mortality rates most likely differs from the impact of age on hospital waiting times [49]. As figure 2 shows, there are additional factors affecting the health production process. This includes system constraints, such as policy constraints (e.g. budget constraints), physical constraints (population density or a country s geographical characteristics) and societal preferences. Furthermore, certain dynamics are involved as previous investments in health systems may affect current output, and current input-choices may affect future results. Finally, the health system may produce additional outputs considered valuable to society including direct outputs such as education or research and innovation and indirect or external outputs such as productivity gains. Defining and measuring input and output The next question is how to define the input(s) and output(s) of the health system, not only in terms of quantities but also in terms of value [50]? There is broad consensus that health is the primary output of health services and health systems. However, performance studies often discuss the meaning and operationalization of health to a limited extent only. Mortality is frequently used as health measure, because it is the most widely and systematically registered health outcome. Nonetheless, it is generally accepted that health services not only aim to prolong life but also aim to improve health status during life. There are different approaches to measuring non-fatal health outcomes [51-53]. Widely used measures of population health, such as Disability Adjusted Life Years (DALY) or Health Adjusted Life Expectancy (HALE), have incorporated information on the prevalence of diseases to cover non-fatal health outcomes [35]. In most clinical studies and economic evaluations, disease-specific and/or generic health instruments such as the EQ-5D or the SF-36 are often used [52]. These measures cover different health dimensions, such as physical and mental health. Recently, a group of researchers proposed to redefine the concept of health as the ability to adapt and to self-manage, including physical, mental and social elements [53]. Because of the multidimensional nature of health, health values are needed to combine different health dimensions and to determine whether overall health improves or not. For example, if physical health improves, but mental health deteriorates to a similar extent, do we consider this a health improvement on aggregate? In other words, do we value mental health and physical health equally or differently? The valuation of health is an important element of all summary measures of health (such as HALE or Quality Adjusted Life Years (QALY)). There is ongoing discussion about the approaches to elicit such values (see [52] for a complete overview), for example regarding the types of questions and instruments used. Brazier et al. concluded that there is no compelling basis for choosing a particular instrument at this stage. In addition, values have been elicited from different groups; patients, the general public and experts. Whose values count? Some have argued that the values of the general public count, since public resources should be spent 14 Chapter 1 Heijink.indd :15:43

16 in line with societal values [54]. Others have argued that the general public is unable to imagine what certain health states are like, which biases their valuation of hypothetical health states. In response to these issues, the approach of experience based values was proposed which uses the valuation of health states people currently experience (instead of values that are based on stated preferences over hypothetical states) [55]. In general, it is also unclear to which extent the valuation of health differs across populations, an important issue for cross-country population health research [52]. 1 As mentioned before, several alternative output measures have been developed to evaluate health system performance, such as avoidable mortality [36]. Most previous studies analyzed avoidable mortality trends, but not the relationship between avoidable mortality and health system inputs (health spending). The studies that did perform such input-output analysis did not take into account methodological issues such as the role of confounders and dynamic effects as shown in figure 2. The output measure health system coverage has been used in a more descriptive way, showing differences in performance between countries or regions. Two studies aimed to further explain variation between regions, relating coverage to population and health system characteristics [56,57]. The most challenging issue in this area is to broaden the scope of these studies, as they largely focused on preventive interventions so far [58]. This requires a conceptual discussion on the measurement of need. The commonly studied preventive interventions are targeted at groups that are rather easy to identify (based on e.g. demographic characteristics), but this may not be the case for many other health services. As figure 2 demonstrates, the health system also produces benefits in terms of non-health outcomes. The concept of responsiveness was introduced to cover non-health aspects that are valued by patients and the general public [7,59-61]. It reflects the ability of health systems to meet the needs of the population in the health care process, aside from health improvements. This could include aspects of care such as communication, confidentiality, and dignity. Measuring responsiveness relies on survey questions and one of the main issues is the comparability of these survey questions across populations, given that norms and experiences will influence response behavior. Although possible solutions were proposed in the literature they have not been applied extensively [61]. The above issues do not just hold for system-level performance studies, but also for performance studies at the organizational level. For example, mortality has often been used as health outcome measure for hospital services. However, even though it may be a relevant output for certain (life-saving) hospital treatments, other types of health measures or non-health measures may be needed in addition. Several provider-level studies used alternative output measures, such as General Introduction 15 Heijink.indd :15:43

17 the number of patients treated sometimes complemented with quality indicators as the number of readmissions [39]. An issue particularly relevant to organizational-level performance studies, is to take into consideration the interrelationships between different types of providers in the health system. For example, health outcomes of hospital patients or costs of hospital care may be influenced by the availability and performance of health services before and after a hospital stay [40]. Finally, health spending is often used as main input measure. Broad definitions include all expenditures on personal medical care (e.g. hospitals, general practitioners, medicines) and public health services. Several studies disaggregated input into labor (e.g. the number of doctors) and/or capital (e.g. the number of hospital beds). Here again, the choice between input measures depends on the goal and scope of the analysis [62], and on which input factors are considered within control of the health system. For example, some have chosen not to measure input in terms of labor or capital, because it was argued that the choice of (combinations of) inputs and even their respective prices are within control of the health system [35]. Furthermore it is important to keep in mind that inputs should be related to outputs as precisely as possible. A final issue is the comparability of input or expenditure data across units, as classifications and allocation methods may vary between countries and providers [63]. Aims and outline The aim of this thesis is to add to and improve the empirical evidence on the performance of health systems, addressing conceptual and methodological issues that arose from the literature. We focus on different dimensions of performance (inputs, outputs, exogenous factors, constraints) and aim to include different perspectives (system-level, organizational-level and disease-level). Each of these perspectives may provide different but complementary pieces of information on the performance of health systems. In particular, we focus on: exploring and explaining differences in health outcomes between countries and health care providers, in terms of (avoidable) mortality, self-reported health, (healthy) life expectancy, and in-hospital mortality the valuation of health; studying the value of experienced health-states across populations and analyzing the impact of health values on health outcome measurement exploring output measures that may complement population health measures, i.e. avoidable mortality and health system coverage 16 Chapter 1 Heijink.indd :15:43

18 comparing health system inputs between countries and providers, in terms of health expenditures and prices of hospital treatments measuring performance at the organizational level, in particular the hospital-level, in terms of health outcomes (in-hospital mortality), quality indicators, responsiveness, prices, and efficiency the relationship between input and output (efficiency) across health systems and health care providers 1 In chapter 2, we study international differences in population health combining fatal and nonfatal health outcomes into a single measure: Quality Adjusted Life Expectancy (QALE). We use a generic health instrument (EQ-5D) that is widely used in clinical trials and economic evaluations, yet to a lesser extent in studies at the population-level. Differences in population health are decomposed to analyze the impact of mortality, health status and health state values. Chapter 3 deals with the valuation of health states across countries. We examine international differences in the valuation of experienced health states, a relatively new approach that has been applied in the national context only [64]. The study investigates whether health limitations are valued differently across populations. In chapter 4, the main input measure of health systems is studied: health expenditures. This chapter includes a comparison of the level and distribution of health spending across six countries. In particular, the distribution of health spending across disease groups is analyzed. The study looks at conceptual issues, the comparability of expenditure data, and policy implications of such cross-country comparisons of health spending. In chapter 5 and chapter 6, the output measures health system coverage and avoidable mortality are studied. The objective of chapter 5 is to explore the relationship between avoidable mortality and health care spending across countries using health production functions and taking into account macro-level confounders and dynamic effects. Furthermore, the health production functions are used to assess cross-country differences in performance. Using the health system coverage concept, we evaluate the extent to which health systems are able to reach those in need of care in chapter 6. We explore health system coverage in the area of chronic care, focusing on international differences and the role of population characteristics. We use a probabilistic approach to measure health care need, based on disease-specific symptomatic screening questions. The remaining methodological and conceptual issues of measuring chronic care coverage are discussed and recommendations for future research are given. General Introduction 17 Heijink.indd :15:43

19 Thereafter, this thesis moves from system-level to organizational-level performance analysis. We focus on hospital care, because hospitals consume the largest part of health system resources and commonly the best data are available for this sector. First, health outcomes are studied. Chapter 7 focuses on one of the main health outcomes of hospital care, in-hospital mortality, aiming to explain variation in the Hospital Standardized Mortality Rate (HSMR) between Dutch hospitals. The main goal of this study is to find out whether hospital mortality is associated with hospital characteristics and environmental factors, on top of the patient-level variables included in the HSMR. Close attention is given to the interpretation of HSMR variation between hospitals. In chapter 8, we compare the performance of hospitals focusing on elective hospital care, in particular cataract surgery. We investigate key outcomes of care, i.e. price, volume and quality (complication rates, process indicators and patient experiences) and the relationship between these variables. Finally, we examine the role of system characteristics in terms of market structure and relate the findings to recent policy-changes in this area of Dutch hospital care. Finally, in chapter 9, another widely used performance (efficiency) indicator is studied, i.e. length of stay in hospitals. We investigate the extent to which hospitals, in particular hospital departments, differ in terms of length of stay, after controlling for patient characteristics. In addition, the study estimates the potential reduction in bed-days at the macro-level, if hospitals are able to reach a specified norm. The final chapter 10 summarizes and interprets the findings of the previous chapters, provides recommendations for future research, policy implications, and a general conclusion. 18 Chapter 1 Heijink.indd :15:43

20 References 1. Visser De E. Nederlandse zorg hoort bij wereldtop [Dutch health care world-class]. Volkskrant /06/ Powerhouse HC. Time to learn from the Dutch champions how to build value-for-money healthcare! General press release of the Euro Health Consumer Index Brussels: Burgers J, Faber MJ, Voerman G, Grol R. Zorg in Nederland scoort best goed [Dutch health care performance pretty good]. Medisch Contact. 2011;2: RVZ. Sturen op gezondheidsdoelen. Den Haag: Raad voor de Volksgezondheid en Zorg, Okma KG, Marmor TR, Oberlander J. Managed competition for Medicare? Sobering lessons from The Netherlands. The New England journal of medicine. 2011;365(4): Jacobs R, Smith PC, Street A. Measuring Efficiency in Healthcare. Analytic Techniques and Health Policy. Cambridge: Cambridge University Press Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health system improvement: experiences, challenges and prospects. Cambridge: Cambridge University Press; Bodenheimer T. High and rising health care costs. Part 1: seeking an explanation. Annals of internal medicine. 2005;142(10): Cutler DM, Rosen AB, Vijan S. The value of medical spending in the United States, The New England journal of medicine. 2006;355(9): WHO Europe. The Tallinn Charter: Health Systems for Health and Wealth. Tallinn: World Health Organization Europe, RIVM. Dutch Health Care Performance Report Bilthoven: National Institute for Public Health and the Environment, AIHW. Australia s Health. Canberra: Australian Institute of Health and Welfare, AHRQ. National Healthcare Quality Report Rockville: Agency for Healthcare Research and Quality, Health Canada. Healthy Canadians 2010: A federal report on comparable health indicators. Ottawa: Health Canada, SALAR, Socialstyrelsen. Quality and Efficiency in Swedish Health Care: Regional Comparisons Stockholm: Swedish Association of Local Authorities and Regions SALAR and Swedish National Board of Health and Welfare Socialstyrelsen, OECD. Health at a Glance: Europe Paris: Organisation for Economic Co-operation and Development, Commonwealth Fund. [cited /07/2013]; Available from: org/topics/international-health-policy.aspx. 18. Loeb JM. The current state of performance measurement in health care. International journal for quality in health care: journal of the International Society for Quality in Health Care / ISQua. 2004;16 Suppl 1:i McIntyre D, Rogers L, Heier EJ. Overview, History, and Objectives of Performance Measurement. Health Care Financing Review. 2001;22(3): WHO. The World Health Report 2000; Health Systems Improving Performance. Geneva: World Health Organization, McKeown T. The role of medicine: dream, mirage, or nemesis? London: The Nuffield Provincial Hospitals Trust; Cochrane AL, St Leger AS, Moore F. Health service input and mortality output in developed countries. Journal of epidemiology and community health. 1978;32(3): General Introduction 19 Heijink.indd :15:43

21 23. Colgrove J. The McKeown thesis: a historical controversy and its enduring influence. American journal of public health. 2002;92(5): Bynum B. The McKeown thesis. Lancet. 2008;371(9613): Nolte E, Bain C, McKee M. Population Health. In: Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance Measurement for Health System Improvement: Experiences, Challenges, Prospects. Cambridge: Cambridge University Press Baal van P, Obulqasim P, Brouwer W, Nusselder W, Mackenbach J. The influence of health care expenditures on life expectancy. Panel paper 35. Tilburg: Netspar Tilburg University, Bunker JP, Frazier HS, Mosteller F. Improving health: measuring effects of medical care. The Milbank quarterly. 1994;72(2): Mackenbach JP. The contribution of medical care to mortality decline: McKeown revisited. Journal of clinical epidemiology. 1996;49(11): Cutler DM, McClellan M. Is technological change in medicine worth it? Health Aff (Millwood). 2001;20(5): Cremieux PY, Ouellette P, Pilon C. Health care spending as determinants of health outcomes. Health economics. 1999;8(7): Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. The implications of regional variations in Medicare spending. Part 2: health outcomes and satisfaction with care. Annals of internal medicine. 2003;138(4): Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. The implications of regional variations in Medicare spending. Part 1: the content, quality, and accessibility of care. Annals of internal medicine. 2003;138(4): Martin S, Rice N, Smith PC. Does health care spending improve health outcomes? Evidence from English programme budgeting data. Journal of health economics. 2008;27(4): Martin S, Rice N, Smith PC. Comparing costs and outcomes across programmes of health care. Health economics. 2012;21(3): Murray CJ, Frenk J. A framework for assessing the performance of health systems. Bulletin of the World Health Organization. 2000;78(6): Nolte E, McKee M. Does health care save lives? Avoidable mortality revisited. London: The Nuffield Trust, Rutstein DD, Berenberg W, Chalmers TC, Child CG, 3rd, Fishman AP, Perrin EB. Measuring the quality of medical care. A clinical method. The New England journal of medicine. 1976;294(11): Shengelia B, Murray CJL, Adams OB. Beyond Access and Utilization: Defining and Measuring Health System Coverage. In: Murray CJL, Evans DB, editors. Health Systems Performance Assessment; Debates, Methods and Empiricism. Geneva: World Health Organization; Hollingsworth B. The measurement of efficiency and productivity of health care delivery. Health economics. 2008;17(10): Häkkinen U, Joumard I. Cross-country analysis of efficiency in OECD health care sectors: options for research. Paris: Organisation for Economic Co-operation and Development, Blendon RJ, Kim M, Benson JM. The public versus the World Health Organization on health system performance. Health Aff (Millwood). 2001;20(3): McKee M. Measuring the efficiency of health systems. The world health report sets the agenda, but there s still a long way to go. BMJ. 2001;323(7308): Williams A. Science or marketing at WHO? A commentary on World Health Health economics. 2001;10(2): Almeida C, Braveman P, Gold MR, Szwarcwald CL, Ribeiro JM, Miglionico A, et al. Methodological concerns and recommendations on policy consequences of the World Health Report Lancet. 2001;357(9269): Chapter 1 Heijink.indd :15:43

22 45. Nord E. Measures of goal attainment and performance in the World Health Report 2000: a brief, critical consumer guide. Health Policy. 2002;59(3): Smith PC. Measuring and improving health-system productivity. Lancet. 2010;376(9748): Arah OA. Performance Reexamined; concepts, content and practice of measuring health system performance. Amsterdam: University of Amsterdam; Arah OA, Westert GP, Hurst J, Klazinga NS. A conceptual framework for the OECD Health Care Quality Indicators Project. International journal for quality in health care: journal of the International Society for Quality in Health Care / ISQua. 2006;18 Suppl 1: Iezzoni LI. Risk-adjustment for performance measurement. In: Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance Measurement for Health System Improvement; Experiences, Challenges and Prospects. Cambridge: Cambridge University Press; Street A, Häkkinen U. Health system productivity and efficiency. In: Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance Measurement for Health System Improvement; Experiences, Challenges and Prospects. Cambridge: Cambridge University Press; Williams A. Comments on the response by Murray and Lopez. Health economics. 2000;9(1): Brazier J, Ratcliffe J, Salomon JA, Tsuchiya A. Measuring and Valuing Health Benefits for Economic Evaluation. Oxford: Oxford University Press; Huber M, Knottnerus JA, Green L, van der Horst H, Jadad AR, Kromhout D, et al. How should we define health? BMJ. 2011;343:d Brazier J, Akehurst R, Brennan A, Dolan P, Claxton K, McCabe C, et al. Should patients have a greater role in valuing health states? Applied health economics and health policy. 2005;4(4): Dolan P, Kahneman D. Interpretations of utility and their implications for the valuation of health. The Economic Journal. 2008;118: Lozano R, Soliz P, Gakidou E, Abbott-Klafter J, Feehan DM, Vidal C, et al. Benchmarking of performance of Mexican states with effective coverage. Lancet. 2006;368(9548): Liu Y, Rao K, Wu J, Gakidou E. China s health system performance. Lancet. 2008;372(9653): Murray CJ, Frenk J. Health metrics and evaluation: strengthening the science. Lancet. 2008;371(9619): Franken M, Koolman X. Health system goals: A discrete choice experiment to obtain societal valuations. Health Policy Valentine NB, Silva de A, Kawabata K, Darby C, Murray CJL, Evans DB. Health System Responsiveness: Concepts, Domains and Operationalization. In: Murray CJL, Evans DB, editors. Health Systems Performance Assessment: Debates, Methods and Empiricism. Geneva: World Health Organization; Valentine N, Prasad A, Rice N, Robone S, Chatterji S. Health systems responsiveness: a measure of the acceptability of health-care processes and systems from the user s perspective. In: Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance Measurement for Health System Improvement; Experiences, Challenges and Prospects. Cambridge: Cambridge University Press; McGlynn EA. Identifying, Categorizing, and Evaluating, Health Care Efficiency Measures. Final Report. Rockville: Agency for Healthcare Research and Quality, 2008 Contract No.: AHRQ Publication No Mosseveld van CJPM. International Comparison of Health Care Expenditure; Existing frameworks, Innovations and Data Use. Rotterdam: Erasmus University Rotterdam; Leidl R, Reitmeir P. A value set for the EQ-5D based on experienced health states: development and testing for the German population. PharmacoEconomics. 2011;29(6): General Introduction 21 Heijink.indd :15:43

23 Heijink.indd :15:43

24 Chapter 2 Decomposing cross-country differences in Quality Adjusted Life Expectancy: the impact of value sets Richard Heijink, Pieter van Baal, Mark Oppe, Xander Koolman, Gert Westert. Decomposing cross-country differences in quality adjusted life expectancy: the impact of value sets. Population Health Metrics 2011, 9: 17. Heijink.indd :15:43

25 Abstract The validity, reliability and cross-country comparability of summary measures of population health (SMPH) have been persistently debated. In this debate, the measurement and valuation of nonfatal health outcomes have been defined as key issues. Our goal was to quantify and decompose international differences in health expectancy based on health-related quality of life (HRQoL). We focused on the impact of value set choice on cross-country variation. We calculated Quality Adjusted Life Expectancy (QALE) at age 20 for 15 countries in which EQ-5D population surveys had been conducted. We applied the Sullivan approach to combine the EQ-5D based HRQoL data with life tables from the Human Mortality Database. Mean HRQoL by country-genderage was estimated using a parametric model. We used nonparametric bootstrap techniques to compute confidence intervals. QALE was then compared across the six country-specific time trade-off value sets that were available. Finally, three counterfactual estimates were generated in order to assess the contribution of mortality, health states and health-state values to crosscountry differences in QALE. QALE at age 20 ranged from 33 years in Armenia to almost 61 years in Japan, using the UK value set. The value sets of the other five countries generated different estimates, up to seven years higher. The relative impact of choosing a different value set differed across country-gender strata between 2% and 20%. In 50% of the country-gender strata the ranking changed by two or more positions across value sets. The decomposition demonstrated a varying impact of health states, health-state values, and mortality on QALE differences across countries. The choice of the value set in SMPH may seriously affect cross-country comparisons of health expectancy, even across populations of similar levels of wealth and education. In our opinion, it is essential to get more insight into the drivers of differences in health-state values across populations. This will enhance the usefulness of health-expectancy measures. 24 Chapter 2 Heijink.indd :15:43

26 Background Summary measures of population health (SMPH) have been calculated to represent the health of a particular population in a single number, combining information on fatal and nonfatal health outcomes [1,2]. SMPH have been applied to various purposes, e.g., to monitor changes in population health over time, to compare population health across countries, to investigate health inequalities (the distribution of health within a population), and to quantify the benefits of health interventions in cost effectiveness analyses [3-5]. In this study, we focus on using SMPH to compare the level of health across populations. 2 Although different types of SMPH have been developed [6-10], they usually comprise three elements: information on mortality, nonfatal health outcomes, and health-state values. Healthstate values reflect the impact of nonfatal health outcomes on a cardinal scale, commonly comprising a value of 1 for full health and a value of 0 for a state equivalent to death. In SMPH, the number of years lived in a particular population (taken from life tables) is combined with information on the (proportional) prevalence of health states or diseases and the value of these nonfatal health outcomes. In this way, the number of life years lived in a population is transformed into the number of healthy life years lived. 1 The value sets provide the link between the information on nonfatal health outcomes and the information on mortality. There has been much debate on SMPH, in particular regarding the validity, reliability, and crosscountry comparability of different methods. A complete discussion on the pros and cons of different methods is beyond the scope of this paper and can be found elsewhere [6,11,12]. In short, crucial and persistent issues have been the measurement and valuation of nonfatal health outcomes and the incorporation of other values such as discounting or equity. In cases where SMPH are used to compare population health across countries, it is essential to use the same concepts and measurement methods for mortality, nonfatal health outcomes, and value sets across countries. Furthermore, it is crucial to understand in what way the method chosen may affect cross-country variation in the summary measure. In this study, we performed a cross-country comparison of Quality Adjusted Life Expectancy (QALE). We included information on health-related quality of life (HRQoL) to represent nonfatal health outcomes. EQ-5D (HRQoL) population surveys were used, and we included the 15 countries in which an EQ-5D population survey had been conducted. The EQ-5D is a standardized and validated questionnaire for measuring HRQoL. It comprises five dimensions such as mobility and self-care. The information on HRQoL, in combination with one of the available value sets, can be used to calculate QALE. As far as we know, a HRQoL-based approach Decomposing cross-country differences in Quality Adjusted Life Expectancy: the impact of value sets 25 Heijink.indd :15:43

27 has rarely been used in SMPH [1], particularly in international comparisons. The approach may prove interesting, since the value sets are calculated on the basis of choice-based methods, which have a theoretical foundation in economic theory [13]. Furthermore, data requirements of an EQ-5D type of instrument may be limited compared to other approaches such as using disease prevalence, particularly in international comparisons [14,15]. There are several other validated HRQoL instruments besides the EQ-5D, such as the SF-36 and the Health Utility Index mark 2 and mark 3 (HUI-2 and HUI-3) [16-18]. Muennig et al. used EQ-5D data to estimate Health Adjusted Life Years (HALY) in the American population [19]. They found differences across income groups, yet they did not provide insight into the uncertainty in their estimates. In Canada, the HUI was used to calculate a national SMPH [20,21]. Feeny et al. used the HUI-3 and a single Canadian value set to compare health expectancy between Canada and the US [21]. Significant health differences between the two countries were found. Health-state profiles have also been included in SMPH in combination with information on diseases and disability [7]. Our first aim was to provide more empirical evidence on international differences in HRQoLbased health expectancy. Additionally, we aimed to explore the impact of the value set choice. In the context of international comparisons, a choice has to be made between country-specific values and cross-country (global) values. The issue of value set choice has not been extensively discussed in the literature, however. It can be argued that if SMPH serve (international) health system performance assessments, country-specific value sets are preferred. Health systems should deliver outcomes in accordance with the preferences of the population they serve and whose means are put in use. Country-specific value sets may not always be available, however. Some have used foreign value sets, e.g., from neighboring countries. For example, Feeny et al. compared health-utility-based health expectancy between the US and Canada using the Canadian value set for both countries [21]. The authors remarked this as a limitation because the true preferences of the US population may not exactly resemble the Canadian values. Some have used a single global value set in international comparisons. For example, Mathers et al. calculated Health Adjusted Life Expectancy (HALE) by combining data on disease incidence (from the WHO Global Burden of Disease [GBD] study) with, for a subset of countries, survey data on health states [7]. Global value sets were applied to both the diseases (values were called severity weights in this context) and the health states. International comparisons of disabilityadjusted life years (DALYs) and of disability-adjusted life expectancy (DALE) also used a single value set across countries [22-24]. It has been argued that the valuation of health domains shows reasonable consistency across countries, justifying the use of a global value set from an empirical perspective [25]. Nevertheless the need for more empirical evidence was acknowledged. Others did find differences in disease/disability-related values across countries and raised doubts about the universality of health values [26]. Another consideration that could support the use of global 26 Chapter 2 Heijink.indd :15:43