Emergency Medical Services Outcomes Project (EMSOP) II: Developing the Foundation and Conceptual Models for Out-of-Hospital Outcomes Research

Transcription

1 EMS/METHODOLOGY Emergency Medical Services Outcomes Project (EMSOP) II: Developing the Foundation and Conceptual Models for Out-of-Hospital Outcomes Research From the Arizona Emergency Medicine Research Center, Division of Emergency Medicine, The University of Arizona College of Medicine, Tucson, AZ * ; the Department of Emergency Medicine, University of Michigan Medical Center, Ann Arbor, MI ; the Department of Emergency Medicine, The Brody School of Medicine at East Caroling University, Greenville, NC ; the Division of Emergency Medicine and the Loeb Health Research Institute, University of Ottawa, Ottawa, Ontario, Canada II ; the Institute for Trauma and Emergency Care, New York Medical College, Valhalla, NY ; EMSSTAR Group, Annapolis, MD # ; School of Hygiene and Public Health, Johns Hopkins University, Baltimore, MD ** ; Health- Span Transportation Services and Allina Health System, St. Paul, MN ; and Pediatric Emergency Services, Massachusetts General Hospital and Emergency Medical Services for Children, Massachusetts Department of Public Health, Boston, MA. Received for publication June 6, Revision received February 8, Accepted for publication February 20, Presented in part at the Society for Academic Emergency Medicine annual meeting, San Francisco, CA, May 22, Supported by the US Department of Transportation, National Highway Traffic Safety Administration (Contract No. DTNH22-96-H-05245). Daniel W. Spaite, MD * Ronald Maio, DO, MS Herbert G. Garrison, MD, MPH Jeffrey S. Desmond, MD Mary Ann Gregor, MHSA Ian G. Stiell, MD, MSc II C. Gene Cayten, MD, MPH John L. Chew, Jr., MS # Ellen J. MacKenzie, PhD ** David R. Miller, MBA Patricia J. O Malley, MD Address for reprints: Daniel W. Spaite, MD, Division of Emergency Medicine, Arizona Health Sciences Center, P.O. Box , Tucson, AZ Copyright 2001 by the American College of Emergency Physicians /2001/$ /1/ doi: /mem Development of methodologically acceptable outcomes models for emergency medical services (EMS) is long overdue. In this article, the Emergency Medical Services Outcomes Project proposes a conceptual framework that will provide a foundation for future EMS outcomes research. The Episode of Care Model and the Out-of-Hospital Unit of Service Model are presented. The Episode of Care Model is useful in conditions in which interventions and outcomes, especially survival and major physiologic dysfunction, are linked in a time-dependent manner. Conditions such as severe trauma, anaphylaxis, airway obstruction, respiratory arrest, and nontraumatic cardiac arrest are amenable to this methodology. The Out-of-Hospital Unit of Service Model is essentially a subunit of the Episode of Care Model. It is valuable for evaluating conditions that have minimal-to-moderate therapeutic time dependency. This model should be used when studying outcomes limited to the out-of-hospital. An example of this is pain management for injuries sustained in motor vehicle crashes. These models can be applied to a wide spectrum of conditions and interventions. With the scrutiny of health expenditures ever increasing, the identification of clinical interventions that objectively improve patient outcome takes on growing importance. Therefore, the development, dissemination, and use of meaningful methodologies for EMS outcomes research is key to the future of EMS system development and maintenance. [Spaite DW, Maio R, Garrison HG, Desmond JS, Gregor MA, Stiell IG, Cayten CG, Chew JL Jr, MacKenzie EJ, Miller DR, O Malley PJ. Emergency Medical Services Outcomes Project (EMSOP) II: developing the foundation and conceptual models for out-of-hospital outcomes research. Ann Emerg Med. June 2001;37: ] JUNE :6 ANNALS OF EMERGENCY MEDICINE 657

2 INTRODUCTION This is the second article in a series reporting on the work of the Emergency Medical Services Outcomes Project (EMSOP). EMSOP is a 5-year project funded by the National Highway Traffic Safety Administration with the purpose of developing a foundation and a framework for out-of-hospital outcomes research. 1 The main objectives of the project are to identify conditions that should be emphasized in future emergency medical services (EMS) outcomes research (priority conditions), risk-adjustment measures for the priority conditions, and outcome measures for the priority conditions. 2 BACKGROUND The scope and delivery of out-of-hospital has come under increased scrutiny in recent years. Although it is acknowledged that timely transport may be necessary for some patients, many have questioned the value of the range of out-of-hospital services currently provided Although some EMS professionals are suggesting a radical streamlining of EMS systems, 5 others support a more expanded role. 10,12 In the broader health community, there is a persistent concern about the lack of proof of effectiveness related to most aspects of out-of-hospital Most experts on both sides of the argument agree that methodologically sound outcomes research that identifies the role of out-of-hospital is long overdue. 1-3,16-19 The 1994 National Highway Traffic Safety Administration Workshop on Methodologies for Measuring Morbidity Outcomes in EMS 1 concluded that EMS outcomes research was essential. However, it was noted that the methods applicable to out-of-hospital outcomes, especially those with nonmortality measures, had never been developed. There is evidence that out-of-hospital has a profound effect on survival for some conditions (eg, nontraumatic cardiac arrest). In addition, it is possible that outof-hospital affects morbidity and mortality in subsets of patients with other conditions, such as major trauma. However, methodological inadequacy has left many important questions unanswered. For instance, in major trauma, the implementation of fully integrated trauma systems have improved patient survival However, it has been very difficult to identify how much of this effect is related directly to the rendered in the out-ofhospital setting The next objective for EMSOP is the identification of risk-adjustment and outcome measures for each of the EMS priority conditions identified in EMSOP I. However, the reporting of these measures without the prior development of a methodological and conceptual framework for their use would be premature and potentially counterproductive. Therefore, EMSOP II is presented to provide the methodological foundation for future out-of-hospital outcomes research. It will also guide the use of risk-adjustment and outcome measures as tools for identifying the effect of specific out-of-hospital interventions on the priority conditions. GENERAL MODELS FOR OUTCOMES RESEARCH Kane and colleagues have proposed a model for analyzing health that assumes outcomes to be a result of several factors that can be classified as risk-adjustment measures and treatment characteristics. Risk-adjustment measures include items like severity, comorbidity, demographic characteristics, and psychosocial characteristics, whereas treatment characteristics include interventions and setting. In Kane s model, the goal is to isolate the relationship between the outcome variable of interest and treatment by controlling for the effects of other relevant variables. This method is referred to as risk adjustment. In The Algebra of Effectiveness, a more complex model for analyzing outcomes, Iezzoni and colleagues point out that given the diversity of outcomes in the model, a uniform definition of what comprises risk is neither possible nor desirable. In other words, the risk adjusters that are selected will depend on the outcome measures of interest. Both models include risk-adjustment and outcome measures and will be discussed in detail in future EMSOP publications Of particular concern for out-of-hospital outcomes research will be the feasibility of obtaining accurate riskadjustment and outcome measures in this setting. This is challenging because of the operational complexity of the environment. The usefulness of these measures is also affected by the purpose for which they are collected; that is, whether they are gathered as part of routine patient or required as a part of specific research endeavors. In the former, investigators will have to depend on structural resources that are routinely available in the EMS system. In the latter, funding for evaluation methodologies that are not routinely available will be required. Even if all of the major hurdles discussed here can be overcome, the information available from current out-ofhospital databases will be insufficient to answer many important outcomes questions. This is due, in part, to the fact that it is impossible to answer many questions about in the out-of-hospital environment without detailed data from provided after the patient is delivered to 658 ANNALS OF EMERGENCY MEDICINE 37:6 JUNE 2001

3 the hospital. This will require significant attentiveness to the linkage of data from emergency department, inpatient, outpatient, and, in some cases, autopsy records. 37,38 METHODOLOGICAL FOUNDATIONS FOR EMS OUTCOMES RESEARCH For the majority of out-of-hospital patients, the therapies provided before arrival at the hospital are the first interventions in a continuum of that extends into the ED, inpatient units, and subsequently to the outpatient setting. Because therapeutic interventions delivered along the entire continuum of have the potential to affect outcomes that are measured at or near the end of the clinical course, it is an enormous task to identify the effect of out-of-hospital. Thus, the ability to identify the individual effect of a specific therapy is compromised. Because of this, the concept of the episode of is particularly meaningful for out-of-hospital outcomes research (Figure 1). From this perspective, each site of can be identified as a separate unit of service along a continuum termed the episode of In this model, numerous units of service typically occur during an episode of. In the case of EMS, the episode of is initiated by an event that leads to a response by the EMS system. In patients who require the entire continuum of, outof-hospital is followed by ED, emergency subspecialty, inpatient, and follow-up. At the end of follow-up, patient has been completed, and long-term outcomes are determined. Obviously, not all patients require or receive all of the potential units of service. By using the Episode of Care Model, it would be optimal to identify the effect of rendered during each unit of service, on short-, intermediate-, and long-term outcomes. Figure 2 shows how risk-adjustment and outcome measures are used to yield a model that allows this to occur. Early in each unit of service, risk-adjustment measurements are selected, therapeutic interventions are performed, and outcome measurements are collected. Use of this methodology in a stepwise fashion allows the effect of therapeutic interventions rendered in each unit of service to be measured and identified. The model is presented here in its simplest form. Obviously, variations occur. For instance, emergency subspecialty may not be necessary for a given patient, even though the other units of service occur. On the other hand, surgical interventions may occur repeatedly throughout the continuum from the ED through follow-up. In addition, multiple therapeutic interventions often occur within a given unit of service. Thus, it is possible that multiple cycles of risk-adjustment measurement, followed by therapeutic intervention, followed by outcome measurement, may be repeated within a single unit of service. Although it may be possible to complete multiple cycles of risk adjustment and measurement in the hospital, this may have limited applicability in the out-of-hospital environment. This model is enlightening in several respects. First, the logistics of identifying the incremental effect of an intervention within a unit of service is inherently easier moving from left to right in the model. This is because the rapid rate of change in risk-adjustment and outcome measurements decreases as the long-term outcomes are approached. In addition, the operational realities of measuring risk adjustment and outcomes are far less complicated and far more controlled as a patient moves along the episode of. For instance, the predictability of the presentation of a given patient is highly controlled (indeed Figure 1. The Episode of Care. *Such as surgery and interventional radiology; **such as specialty follow-up, physical therapy, and occupational therapy. Figure 2. The Episode of Care Model for identifying the effect of treatment from each unit of service in the episode of. RA, Risk-adjustment measures; T, therapeutic outcome; OUT, outcomes measure(s). Out-ofhospital Precipitating event ED Emergency subspecialty * Units of service Inpatient Follow-up ** Care complete; long-term outcomes determined Out-ofhospital Precipitating event ED Emergency subspecialty Inpatient Follow-up Long-term outcomes JUNE :6 ANNALS OF EMERGENCY MEDICINE 659

4 scheduled) in follow-up, whereas it is entirely unpredictable in out-of-hospital and ED. In addition, many of the tools that can be used to measure risk adjustment and outcomes are easily applied in the inpatient or follow-up settings, whereas they may be difficult or impossible to use in the out-of-hospital setting. For instance, a 30-minute survey tool for risk adjustment or outcomes might easily be completed in a clinic setting, although it would be operationally impossible to apply in a patient with a brief out-of-hospital unit of service. Another operational issue that directly relates to the length of the units of service is length of patient contact. Out-of-hospital is generally rendered within minutes and is nearly always complete within 1 hour. However, the units of service at the opposite end of the continuum range in length from days to years. Thus, any risk-adjustment or outcome measurement occurring in the out-ofhospital setting must be applied within an extremely narrow time window. Finally, the environment in which out-of-hospital is rendered is often uncontrolled and sometimes dangerous. Another obvious implication of this model is that it is easier to identify the incremental effect on outcome from a given unit of service when the number of units of service in the episode are few. The ultimate expression of this occurs in patients who receive only out-of-hospital. All identified improvements in outcome are automatically attributable to the out-of-hospital therapeutic interventions (above any spontaneous improvement that would be predicted apart from any therapy at all). This model also explains why out-of-hospital cardiac arrest research has advanced so far beyond any other aspect of EMS research. It predicts that the incremental effect of out-of-hospital would be easy to identify in a condition with the following attributes: (1) identification of patients with the specific condition is easy and unambiguous; (2) the condition is identified by readily obtainable risk-adjustment measures; (3) the pertinent final outcome is relatively easy to obtain and is dichotomous; and (4) the final outcome is solely determined (or nearly so) by the therapeutic interventions rendered in the outof-hospital unit of service. Given this description, it is clear that out-of-hospital cardiac arrest was a providential gift to EMS outcomes research. This discussion leads to an important reality in EMS outcomes research that is most easily understood by evaluating the frequency graph in Figure 3. This figure shows several approximations of the association between the frequency of EMS responses and the severity of physiologic derangement in the population of out-of-hospital patients. 4 Clinical experience and the frequency evaluations published in EMSOP I reveal a strong association between the severity of physiologic derangement and the time dependency of outcomes. In other words, the most pertinent outcome of patients with severe physiologic derangement (survival) may be strongly affected by even a brief delay in therapeutic interventions. On the other hand, survival and physiologic derangement among patients with minor physiologic abnormalities is far less dependent on how quickly is rendered. If Figure 3 represents a reasonable rendering of therapeutic time dependency and the frequency of EMS responses, then it is not difficult to explain why the effect of EMS has only been verified in a small handful of conditions. Conceptually combining Figures 2 and 3 explains why it is relatively easy to show the effect of out-of-hospital interventions on cardiac arrest, whereas it is very difficult to identify their effect on uncomplicated seizures, abdominal pain, or ankle sprains. Major trauma resides somewhere to the right of center in Figure 3. Interestingly, it represents a condition for which there is some evidence of delineating the effect of out-of-hospital on outcomes. Thus, evidence for the effect of EMS on major trauma is better than what exists for the majority of conditions that lie on the minor end of the scale. However, it is not as convincing as the evidence for an effect of out-of-hospital on cardiac arrest. The Episode of Care Model reveals how important it will be for outcomes researchers to identify and use riskadjustment and outcomes measures for each unit of service. This is particularly true in major trauma because we still do not know whether the out-of-hospital unit of ser- Figure 3. Frequency, severity, and therapeutic time dependency in EMS. Frequency of EMS responses Minor, minimally time-dependent conditions Severity and therapeutic time dependency Severe, extremely time-dependent conditions 660 ANNALS OF EMERGENCY MEDICINE 37:6 JUNE 2001

5 vice is substantially responsible for any improvements in outcome Furthermore, even if the out-of-hospital treatments do have a significant effect, it may remain unclear which interventions within the package lead to improvements in outcome. For instance, it is possible that out-of-hospital intubation of critically injured patients has led to some of the improvements in outcome identified in modern trauma systems. On the other hand, it is also possible that most (or all) of these improvements could be explained solely by the systematic early delivery of trauma patients to definitive facilities. Thus, identification of the effect of the out-of-hospital package on patients experiencing major trauma has been inadequate. However, the Episode of Care Model may help identify this effect. In addition, this model may provide the framework to identify whether there is any effect from individual interventions within the out-of-hospital unit of service. ISOLATING THE EFFECT OF OUT-OF-HOSPITAL INTERVENTIONS IN OUTCOMES RESEARCH ment measures and outcome measures that can be obtained during the out-of-hospital. This prevents the complication of mistakenly attributing patient improvement to out-of-hospital when the effect may actually be the result of subsequent interventions. The identification of these risk-adjustment and outcome measures for use in the out-of-hospital setting will be a major focus of future articles in the EMSOP series. Throughout the EMSOP process, the investigators have been sensitive to the issue of isolating EMS outcomes research from the rest of the continuum of. This has long been considered a weakness of EMS research. In current research, many distal outcomes have been inappropriately attributed to out-of-hospital interventions, and most can easily be explained by subsequent interventions. The Episode of Care Model is intended to emphasize a balance between these extremes. Identifying the effect of interventions within the out-of-hospital unit of service is essential for many conditions, but linkage between units of service is also extremely important when logically supportable and methodologically sound. By expanding and focusing on the early part of the episode of, a model can be developed to help isolate the effect of rendered before arrival at the hospital. Figure 4 shows what we have termed the Out-of-Hospital Unit of Service Model and includes a detailed rendering of the out-of-hospital. This is taken from a previously validated time- model. 45 After arrival of EMS personnel at the patient s side, the assessment begins. Figure 4. Out-of-Hospital Unit of Service Model. Precipitating event It is clear from the discussion of frequency, severity, and time dependency that the vast majority of EMS patients fall into the minimally time-dependent category when survival and physiologic derangement are considered as the outcomes. 4 Thus, a model that might delineate the effect of out-of-hospital interventions, if they exist, in patients with only minor or moderate physiologic abnormalities is necessary. Although mortality is rarely a relevant outcome for these patients, several other outcomes from the 6 Ds (ie, death, disability [physiologic derangement], disease, discomfort, dissatisfaction, and destitution [cost]) may be affected by out-of-hospital interventions. Some would say that resources should not be expended on patients without life-threatening problems in the out-of-hospital setting. However, it is notable that a similar severity-frequency curve exists for all health. Thus, if it is considered necessary to have a system in place for saving patients from out-of-hospital cardiac arrest and severe trauma, it is possible that the incremental cost required to affect other outcomes may be relatively small However, such a discussion will always remain philosophical unless proper research models are applied to this population. For conditions that have relatively minimal therapeutic time dependency, the ability to make conclusions about effect becomes difficult, if not impossible, unless short-term outcomes are identified within, or in immediate proximity to, the out-of-hospital. Thus, it becomes necessary to identify meaningful risk-adjust- Out-ofhospital ED Assessment Emergency subspecialty Out-of-hospital * Scene treatment Patient removal Inpatient Follow-up Transport Riskadjustment Intervention(s) Intervention(s) Outcome measurements measurements * Modified from Ann Emerg Med. 1993;22:639. JUNE :6 ANNALS OF EMERGENCY MEDICINE 661

6 During this time, appropriate risk-adjustment measurements are taken. This is followed by interventions that occur during on-scene treatment, transport s, or both. Outcome measurements would typically be collected during the transport after the pertinent interventions have occurred. Figure 4 shows the model in its simplest form. Obviously there is the potential to have multiple interventions evaluated in series. This requires that risk-adjustment measurements be made before and that outcome measurements be collected after each of the interventions being investigated. The Out-of-Hospital Unit of Service Model can be used in a variety of ways. It can be used when the only outcome of interest occurs in the out-of-hospital setting. For example, this would be an effective approach for studying the treatment and nontransport of diabetic patients experiencing a hypoglycemic reaction. In this case, even though follow-up information would be needed from the patient or a proxy, the for this patient would have been completed in the out-of-hospital unit of service. Other outcomes would also be appropriate for this model. For instance, patient satisfaction could, and probably should, be evaluated in the out-of-hospital, rather than lumping all units in the episode together. Relief of pain is another example of an outcome that is relatively easy to study within the out-of-hospital setting without necessarily continuing the evaluation on into later units of service. In summary, development of methodically acceptable outcomes models for EMS is long overdue. We present 2 models for future EMS outcomes research on the basis of methodological work done by health outcomes researchers in other environments. The Episode of Care Model is useful in conditions in which interventions and outcomes are linked in a time-dependent manner. Thus, conditions such as nontraumatic cardiac arrest, airway obstruction, anaphylaxis, respiratory arrest, and perhaps severe trauma are amenable to this methodology when survival, physiologic derangement, and probable long-term disability are considered as outcomes. Nontraumatic cardiac arrest is the prototypical condition for use of this model. For conditions that have minimal-to-moderate therapeutic time dependency, the Out-of-Hospital Unit of Service Model is presented. This model is a subunit of the Episode of Care Model. Both of these models can be applied to a wide spectrum of conditions, interventions, and outcomes. With the scrutiny of health expenditures ever increasing, the identification of clinical interventions that objectively improve patient outcomes takes on growing importance. Therefore, the development, dissemination, and use of meaningful methods for EMS outcomes research is a key to the future of EMS system development and maintenance. REFERENCES 1. National Highway Traffic Safety Administration. EMS Outcomes Evaluation: Key Issues and Future Directions. Proceedings from the NHTSA Workshop on Methodologies for Measuring Morbidity Outcomes in EMS; Washington, DC, April Washington, DC: US Department of Transportation; Maio RF, Garrison HG, Spaite DW, et al. Emergency Medical Services Outcomes Project I (EMSOP I): prioritizing conditions for outcomes research. Ann Emerg Med. 1999;33: Callaham M. Quantifying the scanty science of prehospital emergency. Ann Emerg Med. 1997;30: Koenig KL. Quo vadis: scoop and run, stay and treat, or treat and street? Acad Emerg Med. 1995;2: Koenig KL. Unscheduled access to health : reengineering the 911 system. Acad Emerg Med. 1996;3: Reines HD, Bartlett RL, Chudy NE, et al. Is advanced life support appropriate for victims of motor vehicle accidents: the South Carolina highway trauma project. J Trauma. 1988;28: Smith JP, Bodai BI, Hill AS, et al. Prehospital stabilization of critically injured patients: a failed concept. J Trauma. 1985;25: Spaite DW, Criss EA, Valenzuela TD, et al. Emergency medical services systems research: problems of the past, challenges of the future. Ann Emerg Med. 1995;26: Seidel JS, Henderson D, Tittle S, et al. Priorities for research in emergency medical services for children: results of a consensus conference. Ann Emerg Med. 1998;33: Spaite DW, Criss EA, Valenzuela TD, et al. Developing a foundation for the evaluation of expanded-scope EMS: a window of opportunity that cannot be ignored. Ann Emerg Med. 1997;30: Stiell IG, Wells GA, Spaite DW, et al. The Ontario Prehospital ALS (OPALS) Study II: rationale and methodology for trauma and respiratory distress patients. Ann Emerg Med. 1999;34: O Connor RE, Cone DC, De Lorenzo RA, et al. EMS systems: foundations of the future. Acad Emerg Med. 1999;6: Raskin IE, Maklan CW. Medical treatment effectiveness research: a view from inside the Agency for Health Care Policy and Research. Eval Health Prof. 1991;14: Relman AS. Assessment and accountability: the third revolution in medical. N Engl J Med. 1988;319: Roper WL, Winkenwerder W, Hackbarth GM, et al. Effectiveness in health : an initiative to evaluate and improve medical practice. N Engl J Med. 1988;319: Delbridge TR, Bailey B, Chew JL, et al. EMS agenda for the future: where we are... where we want to be. Ann Emerg Med. 1998;31: National Highway Traffic Safety Administration. Emergency Medical Services: Agenda for the Future. Washington, DC: US Department of Transportation; Publication No. DOT HS Brice JH, Garrison HG, Evans AT. Study design and outcomes in out-of-hospital emergency medicine research: a ten-year analysis. Prehosp Emerg Care. 2000;4: Spaite DW. Outcome analysis in EMS systems [editorial]. Ann Emerg Med. 1993;22: Bass RR, Gainer PS, Carlini AR. Update on trauma system development in the United States. J Trauma. 1999;47(suppl):S15-S Mann NC, Mullins RJ, MacKenzie EJ, et al. Systematic review of published evidence regarding trauma system effectiveness. J Trauma. 1999;47(suppl):S25-S ANNALS OF EMERGENCY MEDICINE 37:6 JUNE 2001

7 22. MacKenzie EJ. Review of evidence regarding trauma system effectiveness resulting from panel studies. J Trauma. 1999;47(suppl):S34-S Jurkovich GJ, Mock C. Systematic review of trauma system effectiveness based on registry comparisons. J Trauma. 1999;47(suppl):S46-S Mann NC. Assessing the effectiveness and optimal structure of trauma systems: a consensus among experts. J Trauma. 1999;47(suppl):S69-S Mann NC, Mullins RJ. Research recommendations and proposed action items to facilitate trauma system implementation and evaluation. J Trauma. 1999;47(suppl):S75-S Spaite DW, Criss EA, Valenzuela TD, et al. Prehospital advanced life support for major trauma: critical need for clinical trials. Ann Emerg Med. 1998;32: Kane RL. Approaching the outcomes question. In: Kane RL, ed. Understanding Health Care Outcomes Research. Gaithersburg, MD: Aspen Publishers; 1997:1-17, Kane RL. Risk adjustment. In: Kane RL, ed. Understanding Health Care Outcomes Research. Gaithersburg, MD: Aspen Publishers; 1997: Smith M. Severity. In: Kane RL, ed. Understanding Health Care Outcomes Research. Gaithersburg, MD: Aspen Publishers; 1997: Nitz NM. Comorbidity. In: Kane RL, ed. Understanding Health Care Outcomes Research. Gaithersburg, MD: Aspen Publishers; 1997: Derose S. Demographics and social factors. In: Kane RL, ed. Understanding Health Care Outcomes Research. Gaithersburg, MD: Aspen Publishers; 1997: Iezzoni LI. Risk and outcomes. In: Iezzoni L, ed. Risk Adjustment for Measuring Health Outcomes. 2nd ed. Chicago, IL: Health Administration Press; 1997: Iezzoni LI. Dimensions of risk. In: Iezzoni L, ed. Risk Adjustment for Measuring Health Outcomes. 2nd ed. Chicago, IL: Health Administration Press; 1997: Daley J, Schwartz M. Developing risk adjustment methods. In: Iezzoni L, ed. Risk Adjustment for Measuring Health Outcomes. 2nd ed. Chicago, IL: Health Administration Press; 1997: Daley J. Validity of risk adjustment methods. In: Iezzoni L, ed. Risk Adjustment for Measuring Health Outcomes. 2nd ed. Chicago, IL: Health Administration Press; 1997: Hughes JS, Ash AS. Reliability of risk adjustment methods. In: Iezzoni L, ed. Risk Adjustment for Measuring Health Outcomes. 2nd ed. Chicago, IL: Health Administration Press; 1997: Patterson L. Weiss H, Scharno P. Combining multiple databases for outcomes assessment. Am J Med Qual. 1996;11:S73-S Waien SA. Linking large administrative databases: a method for conducting emergency medical services cohort studies using existing data. Acad Emerg Med. 1997;4: Hornbrook ML, Hurtado AV, Johnson RE. Health episodes: definition, measurement, and use. Med Care. 1985;42: Solon JA, Feeney JJ, Jones SH, et al. Delineating episodes of medical. Am J Public Health. Nations Health. 1967;57: Young KM, Fisher CR. Medi episodes of illness: a study of hospital, skilled nursing facility and home health agency. Health Care Financing Rev. 1980;2: Grady ML, ed. Medical Effectiveness Research Data Methods. AHCPR pub Rockville, MD: US Department of Health and Human Services; Steinwachs DM. Episode of Care Framework: Utility for Medical Effectiveness Research. Rockville, MD: Agency for Health Care Policy and Research, Publication No Valenzuela TD, Criss EA, Spaite DW, et al. Cost-effectiveness analysis of paramedic emergency medical services in the treatment of prehospital cardiopulmonary arrest. Ann Emerg Med. 1990;19: Spaite DW, Valenzuela TD, Meislin HW, et al. Prospective validation of a new model for evaluation EMS systems by in-field observation of specific time s in prehospital. Ann Emerg Med. 1993;22: JUNE :6 ANNALS OF EMERGENCY MEDICINE 663