Case Study Work in groups At most 7-8 page, double-spaced, typed critical appraisal of a published CEA article Start with a 1-2 page summary of the article, answer the following ten questions, and then include a maximum 1 page overall assessment. Check-List for Assessing Economic Evaluations (Drummond, Chap. 3) 1) Was a well-defined question posed in answerable form? 2) Was a comprehensive description of the competing alternatives given (i.e., can you tell who did what to whom, where, and how often?)? 3) Was the effectiveness of the program or services established? 4) Were all the important and relevant costs and 5) Were costs and consequences measured accurately in appropriate physical units (e.g., hours of nursing time, number of physician visits, gained life-years, etc )? 1 2 Check-List for Assessing Economic Evaluations (continued) 7) Were costs and consequences adjusted for differential timing? 8) Was an incremental analysis of costs and consequences of alternatives performed? 9) Was allowance made for uncertainty in the estimates of costs and consequences? 10) Did the presentation and discussion of study results include all issues of concern to users? Sample Critical Appraisal of Boyle M, et al. Economic evaluation of neonatal intensive care of very-low-birthweight infants. NEJM 1983; 308:1330-7 O. Alagoz 3 Background Clinical evidence that NICUs increase survival of Very- Low-Birth-Weight Infants (VLBWI) and costs for their care No economic evaluation of Level III NICUs Methods Societal perspective is considered Long-terms consequences (benefits and costs) are projected CBA, CE A, and CUA are used The VLBWI belonging to Hamilton-Wentworth County are included in the study (Group 1: 373 infants that were born before the hospitals introduced neonatal intensive care and Group 2: 265 infants that were born afterwards Two groups: 500g-999g and 1000g-1499g 1
Methods Program Description 3 levels for complexity and costs of perinatal program have been defined 68% of VLBWI in the region received care in the regional neonatal intensive-care unit in the test period (1973-77) Health Outcomes They evaluated health status using 6 levels for physical function, 5 levels of role function, 4 levels of social and emotional function and 8 levels of health problems, totally 960 health states. The relative value of health states is determined through utility measurements (ranging from 0.39 to 1) Neonatal-Intensive-Care Costs All costs are in 1978 Canadian dollars The actual cost for each infant was determined by multiplying the quantity of the services used by the infant, by the unit price of the service, and summing across all services. Unit prices include also overhead costs. The simultaneous-equation method was used to allocate all costs among support departments serving each other. Other costs include physician charges, transport charges etc. Physician charges from the billings. Hospital-specific per diem rates served to approximate the costs of hospital care outside the neonatal units. Follow-up and Projected Costs and Earnings Analysis Earnings are the lifetime earnings of the survivors. Follow-up costs include the resources used by children after discharge from the hospital (obtained from the parents by interview) and are converted to 1978 dollars. Forecasts of lifetime outcomes are obtained by an interval estimate rather than a point estimate for each patient. Detailed analyses are shown in the Appendix (CEA, CUA and CBA) A discount rate of 5% per year is applied to costs, earnings and effects because, neonatal intensive care requires early expenditure of large sums of money to achieve later gains. Sensitivity analyses were also performed for 4 variables. The discount rate (0 to 10%) Life expectancy (extremes of the forecast) Loss to follow-up Utility values (over their ranges of uncertainty) Results Health Outcomes Rate of survival increases with the neonatal intensive care Life years per live birth also increased. However, for infants weighting 500 to 999 g QALY earnings are considerably less than life years. Costs The costs were greatly increased with the provision of neonatal intensive care- for all time periods Economic Evaluation They present the CEA, CUA and CBA results for neonatal intensive care with and without discounting for future costs separately. When 5% discount rate is applied, the costs per QALY gained drop drastically. They also present the net economic benefit of neonatal intensive care according to 250-g subgroups at a 5% discount rate. The analyses by 250-g subgroups reveal dramatic differences in the net economic benefits of the neonatal intensive care between the groups. 2
Economic Evaluation For newborns weighing 1000 to 1499 g the cost (in 1978 Canadian dollars) was $59,500 per additional survivor $2,900 per life-year gained $3,200 per QALYs gained intensive care resulted in a net economic gain when figures were undiscounted but a net economic loss when future costs, effects, and earnings were discounted at 5%. Economic Evaluation For infants weighing 500 to 999 g, the corresponding costs were $102,500 per additional survivor, $9,300 per life-year gained $22,400 per QALYs gained Intensive care resulted in a net economic loss for this group of babies By every measure of economic evaluation, the impact of neonatal intensive care was more favorable among infants weighing 1000 to 1499 g than among those weighing 500 to 999 g. Does not change with sensitivity analysis 1) Was a well-defined question posed in answerable form? Yes, the authors note that previous clinical outcomes studies have shown that neonatal intensive care units have increased the survival and costs of VLBWI but no CEA, CUA analysis has been reported. 2) Was a comprehensive description of the competing alternatives given (i.e., can you tell who did what to whom, where, and how often?)? Yes, they present the details of the mortality and morbidity of all VLBWI born to the residents of a southern Ontario county before (1964 to 1969) and after (1973 to 1977) the introduction of neonatal intensive care. Although never stated as a question, they perform CEA, CBA, CUA using the societal perspective. If the research question were posed as a question in an answerable form it would be, From the viewpoint of society, are neonatal intensive care units (NICUs) preferable for treating VLBWI weighing between 500g and 1499g? 15 The infants born in each of the study s time periods were subdivided the two cohorts by weight, 500g-999g and 1000g-1499g. The assessment of survivors health and costs were compared from birth to hospital discharge, to age fifteen (projected) and to the time of death (projected). 16 3) Was the effectiveness of the program or services established? 14.6% increase of Yes, efficacy survival to discharge established by for 1000g-1499g previous clinical trials Same techniques used at Level III NICU 11.8% increase for 500g-999g 17 4) Were all the important and relevant costs and Yes, lifetime costs/consequences were estimated for all: neonatal care follow-up health care other special services such as special education needs handicapped or institutionalization The relevant effects considered include mortality at the time of discharge 960 distinct possible health states are used to estimate the utility scores Utility scores for each of the 960 health states were obtained from a random sample of parents from the region (adjusting for parent sex 18 and child educational placement) 3
4) Were all the important and relevant costs and Forecasts for lifetime effects and costs were calculated based on answers from two pediatricians who were asked to provide a probability distribution of life expectancy health-state-classification productivity health care consumption 5) Were costs and consequences measured accurately in appropriate physical units (e.g., hours of nursing time, number of physician visits, gained life-years, etc )? Yes, Costs Hospital services used summed and multiplied by price MD, other healthcare, transportation, etc Projected for duration of lifetime Yes, Consequences 6 levels for physical function 5 levels of role function 4 levels of social emotional function and 8 levels of health problems=960 total health states. 19 Projected earnings subtracted utility measurements ranging from 0.39 to 1 Health outcomes were obtained from hospital records Questionnaires from the parents of the infants (mailed) The outcomes, at certain points in time, were obtained from interviews from a random sample of survivors The relative values that society attached to the various health states of the survivors were determined from a random sample of local parents of Ontario school children who were asked to rate each of the 960 states from 0-1 (0 dead and 1 healthy) based on desirability and undesirability The range that resulted was -.39 to 1 because parents rated some heath states worse than death To value the costs: first determined a unit of measure for allocating the costs (such as patient days, etc.) then used 1978 financial and service data to determine a fully allocated unit price for each service Follow-up and projected costs were determined by interviews of the parents of the infants to find out the types of services that are used by the survivors and projected probability distributions from the pediatricians. 7) Were costs and consequences adjusted for differential timing? Yes, 1978 Canadian dollars were used discount rate of annual 5% was applied to all future costs of care, earnings by the surviving infants and the effects Applied different discount rates (0-10%) to the effects and the costs in the sensitivity analysis 8) Was an incremental analysis of costs and consequences of alternatives performed? Yes, the results of the cost effectiveness evaluation: 1000g to 1499g $59,500/addl survivor discharge 500g to 999g $102,500/addl survivor at discharge 4
9) Was allowance made for uncertainty in the estimates of costs and consequences? Yes and No Yes, Sensitivity analyses performed for 4 variables: - The discount rate (0 to 10%) - Life expectancy (extremes of the forecast) - Loss to follow-up - Utility values (over their ranges of uncertainty) 9) Was allowance made for uncertainty in the estimates of costs and consequences? A missing point in their analysis of future costs and consequences is the lack of statistical analyses of the future costs, effects and benefits. They forecasted the costs and effects in the future independently; however, this procedure is not described in detail. For example, they asked the pediatricians to specify a probability distribution of possible outcomes and used those responses for make projections. However, they do not provide information about these probability distributions and they do not present evidence that they have done statistical analysis of these data. Also, it is very likely that physicians made errors in these estimations. 10) Did the presentation and discussion of study results include all issues of concern to users? Yes, they interpreted the values of the ratio indexes one by one and derived conclusions that address issues for other users. For example, they calculated the ratios for neonatal intensive care for infants weighing 1000g to 1499g and found that the CEA ratio is $6,100 per life-year gained until age 15, the CUA ratio is $7,700 per QALY until age 15, and the net economic loss was $2600 per live birth. There are factors that are not included in the measures (emotional impact on parents of infant deaths etc) 10) Did the presentation and discussion of study results include all issues of concern to users? Although they do not compare their results to similar studies, they justify this lack of comparison by stating that their methodologies are different They discuss the generalizability of the results of this study to other environments and conclude that their findings can be applied to other countries with slight modifications (i.e. countries with fewer resources may wish to give priority to infants in the group weighing 1000g to 1499 g) They also discuss the applicability of their findings to similar urban environments within Canada. Conclusion Yes, in summary, I think the study of Boyle and colleagues satisfies most of the expected attributes of a well designed economic analysis 5