JOINT POLICY AND PLANNING COMMITTEE. Funding Hospital Based Ambulatory Care in Ontario

JOINT POLICY AND PLANNING COMMITTEE Funding Hospital Based Ambulatory Care in Ontario A Review of CIHI s National Ambulatory Care Reporting System Final Report Prepared by the Ambulatory Care Funding Working Group of the JPPC Reference Document #RD7-3, August 1998 For additional copies of this report, please visit our web site at http://www.jppc.org/ A Review of CIHI s National Ambulatory Care Reporting System

In March 1998 the Ambulatory Care Funding Working Group (ACFWG) received a three-month mandate from the JPPC Hospital Funding Committee to evaluate CIHI s National Ambulatory Care Reporting System (NACRS). The directives were to assess the feasibility of using NACRS as a means of funding hospital based ambulatory care in Ontario. The ACFWG endeavored to meet its objectives through input from various stakeholders through focus groups and telephone surveys. The Working Group s evaluation of NACRS also included a preliminary cost-benefit analysis. The Working Group identified eleven criteria to be used to evaluate NACRS. These were presented to six focus groups that encompassed Ambulatory Clinics, Emergency, Rehabilitation, Mental Health, Health Records and Administration. Each of the focus groups were asked to: 1) Provide feedback on criteria developed by the Working Group and rank them in order of importance; 2) To identify measurable indicators for each criteria, and; 3) To specify implementation issues that need to be considered. Clinical relevance, administrative simplicity, resource homogeneity, complexity of a visit and cost-benefit were identified as the most significant issues that should be considered when evaluating NACRS. Findings from the focus group sessions were further corroborated by data gathered from telephone survey responses provided by CIHI ambulatory care pilot site participants and OCCP ambulatory care costing sites. Based on these various sources of information, NACRS was evaluated on its ability to meet specific requirements, defined by criterion-specific indicators. In addition, implementation issues were identified and grouped according to three categories: timing, costs, and current pressures. A preliminary cost-benefit analysis identified that the major cost categories associated with NACRS implementation were associated with registration and coding activities. Based on this evaluation the ACFWG concluded that it was feasible to use NACRS as a means of funding hospital ambulatory care (excluding qualifying day surgery), in Ontario. The ACFWG also proposed the following implementation considerations: 1) Ambulatory care reporting should be mandated; 2) An implementation team should be established; 3) Hospitals need to be provided with detailed implementation timelines; 4) Emergency Services data should be reported in 1999 with clinics to follow in 2000; and 5) The data should be used for funding. These recommendations were approved by the JPPC HFC and forwarded to the MoH for their review. A Review of CIHI s National Ambulatory Care Reporting System

Table of Contents BACKGROUND...1 Ontario Recognizes the need for Reliable and Valid Ambulatory Care Data...1 CIHI s National Ambulatory Care Reporting System (NACRS)...4 Minimum Data Set...6 Grouping Methodology...7 The JPPC Ambulatory Care Funding Work Group...8 METHODOLOGY...9 A Framework for Evaluating NACRS...9 Focus Group Sessions... 10 CIHI Pilot Survey... 12 Evaluation of NACRS... 12 KEY FINDINGS... 13 Focus Group Sessions... 13 NACRS Pilot Site Survey... 14 Evaluation of NACRS... 14 IMPLEMENTATION ISSUES... 15 Timing... 15 Costs... 15 Current Pressures... 15 COST-BENEFIT ANALYSIS... 16 Identifying Costs... 16 Benefits of Collecting Ambulatory Care Data... 18 CONCLUSIONS... 19 SUMMARY OF RECOMMENDATIONS... 20 LISTING OF APPENDICES 1. NACRS CORE MANDATORY DATA ELEMENTS...1 2. AMBULATORY CARE WORKING GROUP: TERMS OF REFERENCE AND MEMBERSHIP...5 3. DEVELOPMENT OF THE RAI-MH: AN UPDATE...8 4. PROPOSED TIME LINE FOR NACRS IMPLEMENTATION...9 A Review of CIHI s National Ambulatory Care Reporting System

A Review of CIHI s National Ambulatory Care Reporting System Page 1 BACKGROUND Ontario Recognizes the need for Reliable and Valid Ambulatory Care Data In 1988, The Honorable Elinor Caplan, Minister of Health, promised to evaluate and modify the hospital funding system in collaboration with the hospital industry. Through an initiative called Transitional Funding, a case-mix adjustment approach was introduced in Ontario which made adjustments to a hospital global budget based on the level of patient care activity and associated costs. Initially, case-mix-funding systems excluded ambulatory care. This was undesirable because, a considerable amount of surgery is performed on an outpatient basis, and a strictly inpatient focused case mix funding system does not provide any incentives to perform more outpatient surgery. Another reason why the omission of ambulatory care data from case-mix funding systems was undesirable was that creative accounting practices by hospitals was used to shift costs out of inpatient care into other categories. Thus lowering their inpatient cost per weighted case. As this was the basis for making funding adjustments these creative hospitals therefore received more funding than they should have. Secondly, ambulatory care accounts for approximately 40% of total hospital costs, yet no incentive is built into the case mix funding systems to supply ambulatory care more efficiently (i.e., at a low cost per case). Figure 1 below, illustrates the considerable variation in hospital ambulatory care activity by specific geographic region in Ontario. Due to a lack of reliable data regarding case-mix, health care planners are unable to conduct comparative evaluations of ambulatory activity in relation to costs, outcomes or effectiveness. Figure 1: Variation in Hospital Ambulatory Activity Acute Hospital 1995/96 Outpatient Clinic & ER Visits Per 1,000 Resident Population 1400 1200 1000 ER OP 800 600 400 200 0 Hamilton Halton Peel York Toronto Durham

A Review of CIHI s National Ambulatory Care Reporting System Page 2 In the Canadian health sector, hospitals and healthcare providers have responded to the challenge of providing quality care in the most cost-effective manner by shifting services from inpatient to ambulatory care services was often the suggested solution to address this challenge. This shift has been evident in the Canadian hospital sector, where during a recent seven-year period (1986-1993), the ratio of outpatient visits to inpatient days for acute care facilities rose 39% (from 62% in 1986 to 86% in 1993) 1. Consequently, the rapid growth in ambulatory care services has underscored the need for data on this client population to support planning, evaluation, management, research and funding activities. Figure 2, shows that based on 1995/96 Ontario Hospital costs, ambulatory care represents approximately 19% (excluding day surgery costs) of the province s hospital expenditures that translate to approximately $1.5 billion. It is not known however, whether this is a cost-effective or a cost-efficient use of resources. Figure 2: Breakdown of Hospital-Based Care in Ontario 1995/96 Ontario Hospital Costs Outpatient 19 % or $1.5 Billion Acute & SDS 37% Teaching 5% Rehab. 2% Chronic 7% Fixed 30% In 1992, the Ontario Ministry of Health (MoH) stated in several reports that hospitals need to maximize their ambulatory care services as part of the hospital reform agenda (Health Services Framework, January 1992). The MoH also commented that funding formulas need to provide specific incentives for hospitals to substitute ambulatory services for acute inpatient services. 1 CIHI (1998) National Ambulatory Reporting System: Project Report, page 1

A Review of CIHI s National Ambulatory Care Reporting System Page 3 The importance of this matter was reflected in a resolution passed by the Board of Governors of the Ontario Hospital Association (OHA) in the fall of 1992. In this resolution, they urged the MoH to: take immediate action to rectify the disincentives in the Equity Funding Formula respecting ambulatory care and day surgery to support hospitals to use such delivery methods to provide care and allocate resources on the most appropriate and cost effective basis. Hospitals should not be financially penalized for shifting services from inpatient to day surgery. Following this resolution, the Ontario Joint Policy and Planning Committee (JPPC) adopted Day Procedure Groups (DPG s), incorporated Qualifying Day Surgery into the current funding methodology and developed the Day Surgery Incentive Model (DSIM). The DSIM addresses inconsistencies between stated objectives and funding methodologies by providing an explicit financial incentive for hospitals to complete more cases on a day surgery basis. Hospitals with a higher proportion of day surgery cases relative to the Ontario average are rewarded. Those cases with a lower proportion of day surgery cases relative to the Ontario experience are penalized. For more detailed information regarding the model see JPPC reference document #3-1: The Day Surgery Incentive Model. Ambulatory Care consists of Medical and Surgical Clinic visits, Day/Night visits, Emergency visits and Day Surgery. While the Day Surgery Incentive model represented a positive first step in the development of an ambulatory care funding methodology, quantifying the remaining components of ambulatory activity remain as a pre-requisite for hospitals before ambulatory care can be accurately funded. Throughout the late 1990s financial pressures have resulted in unprecedented restructuring efforts by Ontario hospitals. What has resulted, is a focus on the need to measure all types of hospital activity through the use of case-mix classification systems. Since its inception in 1996, Ontario s Health Services Restructuring Commission (HSRC) has relied on proxy measures for ambulatory care when restructuring hospital service delivery in the province, including the conversion of former inpatient services to ambulatory care. Early in its mandate, the HSRC ignored the impact of ambulatory growth estimates on facility requirements. However, in recent reports 2, the HSRC has projected ambulatory growth using a ratio of visits to beds. The HSRC stated that the absence of recommendations for investment in hospital-based ambulatory care from their directives is attributed to the absence of good cost and activity data in ambulatory care services. 2 Metro GTA Health Services Restructuring Report

A Review of CIHI s National Ambulatory Care Reporting System Page 4 This conclusion was also reached by a joint Emergency Services working group of the Ontario Ministry of Health and the Ontario Hospital Association who reported in April 1998 that: Throughout the course of this study, a major obstacle was the lack of consistent data regarding emergency services. A consistent theme throughout the course of the interviews with stakeholders was a lack of system-wide data that is comparable, accurate and complete... (consequently).. A provincial emergency minimum data set should be established that would enable continued monitoring and research into system-wide solutions and would provide information to assist in the operational management of emergency services at hospitals 3 Thus, the need for reliable and valid ambulatory care data has come to the forefront in recent years. CIHI s National Ambulatory Care Reporting System (NACRS) In anticipation of this need for reliable and valid ambulatory care data, the Canadian Institute for Health Information (CIHI) initiated the expansion of its national ambulatory care database to include all hospital-based and community-based acute care services, resulting in the National Ambulatory Care Reporting System (NACRS). The classification of ambulatory care activity has largely occurred through the use of DPG s. However, Ambulatory care includes a wide range of services, such as diagnostic, consultative, therapeutic and client teaching activities. While the focus of ambulatory care is often the hospital outpatient department, the majority of ambulatory care services are provided outside the hospital setting, such as from private practitioners offices, community health centers, and in the home. Although national statistics for these services are currently not available, general consensus is that utilization rates of these ambulatory services have increased in recent years. Consequently, the need for accurate and reliable ambulatory care data has correspondingly increased. Currently, CIHI is the national repository for ambulatory day procedure data (i.e., DPG's), with more that 450 hospitals Canada-wide reporting over 1.5 million ambulatory surgery visits annually. The objective of their efforts in developing a National Ambulatory Care Reporting System (NACRS) was to develop a national database of ambulatory care data for a broader range of services. Specific project objectives included: (1) development of national minimum data set, (2) development of a client classification system (grouping methodology), and (3) development of a national reporting mechanism. 3 OHA (1998) OHA Region 3 Emergency Services Working Group Final Report

A Review of CIHI s National Ambulatory Care Reporting System Page 5 For the purposes of CIHI s ambulatory care project, their Steering Committee defined ambulatory care as: Ambulatory care encompasses all health services that are provided to clients who are not residing in health care institutions at the time that care is given. Ambulatory care includes: emergency services; day/night care; specialty clinics; non-specialty clinics; community clinics; day surgery; private practice; and home care While the NACRS project includes all these areas, a multi-phase approach was adopted. Phase 1, includes the areas of hospital-based ambulatory care services and some services delivered in community health centers. Phase 2 will include home care and additional community services 4. The NACRS developmental process included the following activities: Identification of information requirements; Development of minimum data set; Pilot testing of data set; Development and/or modification of a grouping methodology; and Development of output reports. While these activities were implemented in a general progression, there was substantial interaction among the activities. For example, prior to the development of the minimum data set, it was essential to know potential grouping variables in order to ensure their inclusion in the data set. Similarly, output reports were produced prior to the development of resource intensity weights, or a grouping methodology; however, their utility is optimized when the other activities have been performed. It should be noted that Phase 1 activities did not include development of resource intensity weights, the current lack of Canadian cost data being the primary deterrent. Resource intensity weights will be developed at such time when data is sufficient for complete analysis (see Working Group recommendations). 4 For detailed information regarding CIHI s development of the NACRS, see: CIHI s National Ambulatory Care Reporting System Project Report, March 1998.

A Review of CIHI s National Ambulatory Care Reporting System Page 6 Minimum Data Set A data set is a collection of all possible elements within a subject area, in this case, ambulatory care. A minimum data set is the smallest collection of data elements which meets the common needs of the stakeholders. A minimum data set can generally be divided into a number of components, specifically a mandatory and optional section. The mandatory portion of a minimum data set includes those data elements that are collected and reported on a comprehensive, regular basis. Optional data elements are not collected routinely or comprehensively but may be collected to support specific needs or interests. The minimum data set for the National Ambulatory Care Reporting System (APPENDIX 1) is the collection of data elements which describe the ambulatory care service event. That is, the minimum data set describes the service provided by a service provider to a service recipient at a delivery site during a period of time. Included in the mandatory component on the NACRS data set, is a subsection of data elements which are required by the grouping methodology to place clients into clinically and resourcehomogenous groups i.e. grouper data elements. The additional mandatory elements may be needed to develop ambulatory care indicators, which will be included in output reporting. An iterative process was used in the development of the minimum data set. As an overall principle, the NACRS Steering Committee agreed that the data set must meet the key information needs of a wide range of stakeholders while minimizing the burden of data collection. They subsequently developed a set of decision criteria by which to determine the data elements to be included in the minimum data set. Specifically, each data element must: Be useful to multiple stakeholders; Be readily available; Able to be collected in a cost-effective manner; Be flexible and versatile; Be clearly defined and unequivocal; Facilitate comparisons; and Apply or be pertinent to the majority of service events.

A Review of CIHI s National Ambulatory Care Reporting System Page 7 A draft data set was developed based on these criteria, which was revised several times, prior to pilot testing. The final data set, excluding day surgery, includes 40 data elements 5 (see table 1). Table 1: National Ambulatory Care Data Set MANDATORY DATA ELEMENTSDATA TYPE # of Data Elements Demographic6 Administrative2 Clinical9 Emergency Services4 Day Surgery3 OPTIONAL DATA ELEMENTSOptional Data Elements12 Provincially defined3 Weight Calculation4 TOTAL43 Grouping Methodology A grouping methodology is a client classification system. That is, it defines discrete clusters of client types. This is in contrast to disease and procedure classification systems, such as ICD-9, ICD-9-CM, and CCP which defines disease conditions. To define the client groups or clusters, consideration is given to factors such as client characteristics, e.g. gender and age; purpose of visit, e.g. annual check-up; the client s specific health problem, usually a diagnosis; and procedures performed. The grouping methodology organizes these factors into coherent clusters, based on similar clinical characteristics, similar patterns of resource use, or some combination of criteria. Depending on the philosophy of the grouping methodology, clients are assigned to different groups, based on certain characteristics and/or patterns of resources use. For example, if clinical relevance is the fundamental principle of the grouping methodology, clusters will form primarily on the bases of client diagnosis and interventions performed. However, if resource use is the primary driver, client types will be grouped on the basis of similar patterns of resource use, without regard to diagnosis or other clinical factors. Therefore the grouping philosophy and underlying principles will determine the various client clusters and the information produced therefrom. 5 It is important to note here that NACRS day surgery elements are not mandatory or will not be required for ambulatory care reporting as this information is already being collected in Ontario via Day Procedure Groups

A Review of CIHI s National Ambulatory Care Reporting System Page 8 The JPPC Ambulatory Care Funding Work Group CIHI has published the Minimum data set, defined data elements, developed submission guidelines and completed a pilot test with a sample of Canadian Hospitals and Community Health Centres. The purpose of this pilot test was to assist CIHI in the development of the National Ambulatory Care Reporting System. The Grouping methodology has also been completed and is currently being evaluated by the United States Medical Department Centre and School. The results of this evaluation are expected by August 1998. Throughout the developmental process, the Ontario Joint Policy and Planning Committee through its Funding Integration Sub-Committee were reviewing the progress of CIHI in its developmental efforts with the intention to conduct their own evaluation of the National Ambulatory Care Reporting System once completed. In March, 1998 the JPPC formed an Ambulatory Care Funding Working Group with the mandate to evaluate the feasibility of using CIHI s National Ambulatory Care Reporting System as a basis for hospital ambulatory care funding in Ontario (APPENDIX 2). The objective of the working group was to assess the appropriateness and applicability of CIHI s NACRS and to make recommendations regarding its use to the JPPC Hospital Funding Committee. The remainder of this document will describe the efforts of the JPPC Working Group in fulfilling its mandate. It should be noted, that while pilot tests and reviews by the United States Medical Department occurred during the development and validation of NACRS, the JPPC Ambulatory Care Funding Working Group did not have an abundance of data and information available to conduct their evaluation of the reporting system. Consequently, focus group sessions and telephone surveys to CIHI pilot sites were held to generate the primary data needed to appropriately conduct an evaluation. Because of the lack of comprehensive data for some components of the evaluation, the implementation committee, that is recommended at the end of this document, will need to conduct additional evaluation as more comprehensive data become available.

A Review of CIHI s National Ambulatory Care Reporting System Page 9 METHODOLOGY A Framework for Evaluating NACRS The mandate of the JPPC Ambulatory Care Funding Working Group (ACFWG) was to assess the feasibility of using CIHI s National Ambulatory Care Reporting System (NACRS) for funding hospital ambulatory care in Ontario. In particular, the Working Group was asked to evaluate the appropriateness and applicability of the system. The flow-chart, shown in Figure 3 below, depicts the methodological framework developed by the ACFWG for the purpose of evaluating NACRS. It is a useful illustrative reference when reading the following section. Figure 3: Methodological Framework Development of Evaluation Criteria Literature Review Testing of Criteria Stakeholder Review: MoH, Hospitals, Clinicians, Researchers Focus Groups Cost - Benefit Recommendations and Plan for Implementation Working Group Evaluation Workshop CIHI Pilot Survey In order to evaluate a system, one must first begin with an understanding of the desirable outcomes of a classification system. The Working Group conducted an extensive literature review in order to identify evaluation criteria that were generically applicable when evaluating a classification system. These criteria were presented to focus group participants for their review and to suggest additional criteria if needed. The second consideration when evaluating a system is who will be using or benefiting from the implementation of an Ambulatory Care Funding System. That is, what are the relevant evaluation criteria to use, according to various stakeholders who will benefit from the system? Thus, evaluation criteria needed to consider the perspectives of various stakeholders that would be affected by the implementation of NACRS and the use of data that is generated by the system. Once the evaluation criteria were identified and defined, several steps were necessary to evaluate the feasibility of implementation and appropriateness of NACRS in the Ontario hospital ambulatory care setting.

A Review of CIHI s National Ambulatory Care Reporting System Page 10 There was no clearly identified data source that was available to the working group for this purpose; therefore, it was necessary to develop several measures to gather data for the evaluation. This was achieved through four mechanisms: focus Groups with broad stakeholder involvement, a Questionnaire directed at CIHI Ambulatory Care Pilot sites, a workshop where ACFWG members would present the results of their evaluation of NACRS according to the criteria identified by focus groups, and a cost benefit analysis Each of these components of the evaluation conducted is described in the following section. Focus Group Sessions In order to determine whether the evaluation criteria to be used by the Working Group was appropriate for capturing the perspectives of various stakeholders, focus groups were arranged with the intention to solicit the opinions of these stakeholders. Four stakeholders in particular were considered: the Ministry of Health (funder), hospitals (utilization), clinicians (case management, outcomes measurement) and researchers. Focus groups were held to (1) identify and review the evaluation criteria the ACFWG should use, (2) identify indicators to test whether NACRS fulfilled this criteria and (3) to rank which criteria are the most important for the purpose of this evaluation. The sessions were organized into six focus groups, Emergency services (ER), Rehabilitation services (REH), Mental Health services (HM), Ambulatory Clinics (CLI), Finance/ Administration (FAD) and, Data Processing/Health Records (PRO). Consultants from the Hay Group of Health Care Consultants facilitated these sessions and a companion document of the focus group discussions is available through the JPPC 6. Analysis of the focus group discussions revealed eleven criteria that the ACFWG should use in their evaluation of NACRS. 1. Clinical Relevance Clinical Relevance is determined by whether the proposed data elements to be captured under NACRS describe in a clinically meaningful way an ambulatory care visits in a clinic or ER setting. In addition, can those data elements support the development of clinically meaningful groups? 2. Groups are Resource Homogenous Cases are grouped within the various ambulatory care service categories that are similar in terms of the course of treatment and consumption of hospital resources. In addition, the use of a common measure (e.g. resource intensity weight) should be used to measure and demonstrate the relative difference in resource consumption between various ambulatory care categories and with other hospital services (e.g. acute inpatient, day surgery, chronic care etc) 6 Ambulatory Care Focus Group Summary: ACFWG Final Report Companion Document

A Review of CIHI s National Ambulatory Care Reporting System Page 11 3. Focus on Patient/Client Characteristics The patient characteristics that materially affect the nature and cost of patient care. Patients are clustered on the variables of age, gender and diagnosis 4. Administratively Simple In order to evaluate whether a classification system is administratively simple, one must consider the following: How many data elements are collected? How long does it take to collect all data elements? Does the collection require special expertise? Is data collection part of health records abstraction or registration? What additional costs if any, result from the data collection? 1. Dynamic The reporting system and grouper must be relatively easy to modify to reflect ongoing changes in practice and technology. NACRS should provide a seamless interface with other grouping methodologies, to allow movement between analysis of inpatient and outpatients. 2. Builds on current knowledge and has multiple purposes NACRS can abstract from existing data sets that are used for purposes other than ambulatory care reporting 3. Comprehensive The reporting system can be used in a variety of settings for all outpatient types 4. Demonstrates complexity of a visits The data accurately differentiate/recognize the impact of visit complexity on clinical and other resources. 5. Easily convertible to electronic capture The power/speed and costs of the computer (hardware) and the complexity and costs of the computer programs ( software) required to collect the information and transmit it to CIHI and the Ministry of Health should be minimal. 6. Cost Benefit The value/utility of the information in assisting the hospital and other stakeholders in making patient care, utilization, planning, research and funding decisions should be greater than the effort required/costs associated with collecting the data 7. Coding Ambiguity The grouper must represent well-defined and distinct types of patient, thus minimizing over and under coding.

A Review of CIHI s National Ambulatory Care Reporting System Page 12 CIHI Pilot Survey A total of 10 facilities that participated in CIHI s pilot site study in the development of the NACRS minimum data set and the grouping methodology were surveyed to confirm the evaluation criteria and identify implementation and cost issues. The survey was conducted by telephone using a three part questionnaire that identified the characteristics of the facility, the ambulatory care areas (e.g., emergency, rehabilitation) implemented and the requirements for this implementation. Seven respondents were project managers primarily from medical records, and two were ER physicians. Five of the sites contacted were in Ontario Evaluation of NACRS The evaluation of NACRS was conducted using data compiled from the Focus Groups and Pilot Site Survey. A one-day workshop was held to discuss the evaluation conducted by each working group member. This workshop included all of the working group members and one representative from each of the focus groups to ensure issues identified by focus groups were appropriately reflected in the discussions. Prior to the one-day workshop, each member of the working group was assigned a specific evaluation criterion. Using indicators identified by focus group participants, the working group member was asked to assign a rating or score regarding whether NACRS satisfied the specific indicator. Table 2 below shows the rating scale each member was to use when evaluating the indicators. The sum total of the ratings resulted in an overall score for a criterion. Once all of the criterion were ranked and discussed, working group members would then decide whether or not to endorse that NACRS was feasible and appropriate for implementation in Ontario. Table 2: Indicator Rating Scale Rating Scale Description 1 NACRS does not meet indicator 2 3 NACRS partially satisfies indicator 4 5 NACRS fully satisfies indicator NA Information is not available to support the assessment of NACRS using this indicator

A Review of CIHI s National Ambulatory Care Reporting System Page 13 KEY FINDINGS Focus Group Sessions There was a tremendous amount of interest among stakeholders in assessing the ambulatory care system. Of the 76 invited guests, 72 in total attended the sessions. The focus group sessions provided context-rich information around the delivery of hospital ambulatory services within the province. Focus Group participants identified implications for NACRS, which clustered around 5 specific themes. Clinical relevance, administrative simplicity, resource homogeneity, complexity of a visit and cost-benefit were identified as the most significant issues for the working group to consider when evaluating NACRS. At the end of each focus group, focus group participants were given six importance votes. Focus group participants were instructed by the focus group facilitators to identify which of the eleven criteria they consider to be the most important when evaluating an ambulatory care classification system. Focus group participants were to do this by allocating their six votes to the eleven criteria using a specific voting scheme. All six votes could be allocated to a single criterion, or three votes could be allocated to the most important, two votes to the second most important and one vote to the third most important. Focus group participants could not allocate one vote to each of six criteria. Table 3. 0 shows the results of the ranking of importance of each criterion by focus group participants. Table 3: Focus Group Ranking of Evaluation Criteria Criterion ER CLI PRO FAD MH REH Total Rank Clinical Relevance 11 15 5 29 6 13 79 1 Administratively simple 9 11 21 2 15 7 65 2 Resource Homogeneity 8 14 9 4 10 16 61 3 Complexity of visits 11 13 3 16 9 2 54 4 Costs Benefit 20 10 2 8 1 2 43 5 Comprehensive 0 0 0 9 10 12 31 6 Patient Characteristics 0 4 0 0 17 1 22 7 Build on current knowledge 0 0 5 0 8 1 14 8 Electronic capture 0 0 4 2 1 4 11 9 Dynamic 1 3 0 2 1 2 9 10 Coding Ambiguity 0 2 5 0 0 0 7 11

A Review of CIHI s National Ambulatory Care Reporting System Page 14 NACRS Pilot Site Survey CIHI s pilot study conducted during the development of NACRS provided the opportunity for various types of ambulatory care providers to participate in the development of the minimum data set and grouping methodology. The pilot occurred over three months in 1996. Consequently, the level of implementation of NACRS varied and was limited to that which could be accomplished within the time frame of the pilot. The survey findings from the pilot sites by the JPPC s Ambulatory Care Funding Working Group (ACFWG) were consistent with the discussions among the focus groups. Specifically, the pilot sites agreed that the evaluation criteria the ACFWG were proposing to use in their evaluation were applicable and appropriate for evaluating NACRS. The component of the survey which inquired about implementation and costing issues, will be discussed later in this report. Evaluation of NACRS Table 4.0 illustrates the results of the ranking of the criteria by the members of the ACFWG. Each score reflects how well, according to ACFWG members, NACRS was able to satisfy the evaluation criteria. Table 4: Criterion Ranking and Scores Rank Criterion Score out of 5 1 Clinical Relevance 3.00 2 Administratively simple 4.20 3 Resource Homogeneity 3.44 4 Complexity of visits 3.00 5 Costs Benefit 3.00 6 Comprehensive 4.13 7 Patient Characteristics 3.50 8 Build on current knowledge 4.23 9 Electronic capture 3.00 10 Dynamic 4.00 11 Coding Ambiguity 3.13 The ACFWG members discussed each criterion. The average score for all criteria was 4.0 with no criterion scoring below 3.0 (NACRS partially satisfies indicator). Some of the criteria used in the evaluation could not be fully assessed by working group members due to the unavailability of data. For example, Clinical Relevance received a score of 3.0 due to the lack of data available regarding the grouper. However, as previously mentioned, the grouper is currently being reviewed by the United States Medical Department Centre. It is expected that their final report will be available in August 1998. This limitation is addressed in the recommendations at the end of this report through the creation of an implementation team responsible for conducting additional work.

A Review of CIHI s National Ambulatory Care Reporting System Page 15 IMPLEMENTATION ISSUES Throughout the evaluation, implementation issues were identified and recorded. These were consolidated into three major areas of concern: timing, costs, and current pressures on hospitals. These implementation issues were incorporated in the recommendations at the end of this report. It should be noted, focus group participants recommended that, the implementation of NACRS should be sensitive to the activities of other groups involved in the development of classification systems and data collection methodologies that may impact on Ambulatory Care Services. This specifically relates to the work of the OHA Region 3 Emergency Services Working Group and the JPPC Psychiatric Working Group, RAI-MH project 7 (APPENDIX 3). Timing Focus group participants specified the need for a specific timeline for implementation that would begin with immediate direction from the Ministry of Health to all hospitals to begin collecting ambulatory care data using NACRS. Costs Participants consistently identified costs as a major implementation concern. In particular, costs of technology requirements (hardware and software), education, and staffing were identified as potential costs of implementation. Focus group participants raised questions regarding whether hospitals would be expected to absorb these costs of implementation. Current Pressures Hospitals are faced with competing resource pressures from many external forces. The resources associated with the implementation of NACRS will compete for limited resources already earmarked for such issues as hospital restructuring and Year 2000 preparation. To address these concerns, focus group participants suggested a phased approach to implementation starting with emergency services. 7 The April 1998, OHA Region 3 Emergency Services Working Group Final Report addresses recommendations related to data collection and reporting of Emergency Services Data. The JPPC Psychiatric Working Group is currently developing a classification system for mental health encompassing inpatient and outpatient activities. More information on this project is available through the JPPC Reference Document # RD4-12, March, 1997 Working Towards Developing a Resident Assessment Instrument for Mental Health"

A Review of CIHI s National Ambulatory Care Reporting System Page 16 COST-BENEFIT ANALYSIS As part of the ACFWG mandate, its first objective was to evaluate NACRS. The second objective was to provide a cost analysis of implementing the system. To evaluate costs associated with ambulatory data collection, information was gathered through a survey of the 10 NACRS pilot sites and 12 hospitals participating in the Ontario Case Cost Project (OCCP). The OCCP is currently attempting to begin collecting ambulatory care data among its case cost hospitals. Identifying Costs Identifying the costs associated with ambulatory care data collection was extremely difficult due to the unavailability of good cost data. Moreover, the level of implementation of NACRS by pilot sites varied. For example, some pilot sites implemented the reporting system for specific ambulatory clinics, while others restricted implementation to emergency services only. The variability in this implementation process resulted in data that, for purposes of cost-benefit analysis, was incomparable, of poor quality or non-existent. In order to generate a gross approximation of costs, a telephone survey of OCCP and CIHI pilot site hospitals was conducted by the ACFWG. Through this survey, a set of generic costs were identified that would likely be associated with ambulatory care data collection. Table 5 lists these estimated costs and the suggested ways to measure these costs. Table 5: Measuring Costs COSTS Capital Costs Staffing Education Productivity Costs Software operating costs Data collection processes MEASURE bytes/visit visits/fte hours/fte charts/hour Maintenance and licensing fees pick lists vs. Health Records coding As per the recommendations of the focus groups, additional estimates of costs for implementation of NACRS in emergency services were also generated. For example, Figure 4.0. shows that the primary costs for collecting Emergency services data is associated with registration activities and health records departmental functions. According to the surveyed sites, most ER patients are currently being registered which would suggest that the major costs associated with ambulatory data collection for ER services, in these hospitals, would be coding and abstracting of the ER chart. Figure 4 identifies these components.

A Review of CIHI s National Ambulatory Care Reporting System Page 17 Figure 4: Costs of ER Data Collection Patient to ER Patient Registered ER CHART Patient Treated Admission or Discharge Document On chart CHART To Health Records COST Staffing Training Software Operating COST Staffing Training Software Operating FILED NO CODE? YES HRT* HRA* Abstracting *HRT = Health Records Technician *HRA = Health Records Assistant Capital costs relate to start-up hardware and software expenditures. These costs are directly related to the amount of processing power and storage required by hospitals, based on the number of ambulatory care visits per year. Staffing and education costs are directly related to the type of work processes that are implemented to collect the data. For example, some of the NACRS pilot sites implemented pick lists to capture ambulatory care clinic visits, thereby reducing the costs associated with health records coding and abstracting. It is recognized that this may not be appropriate for all sites and that the method of data collection will vary based on the type and volume of ambulatory services provided. Consequently, costs will also vary based on the type and level of implementation. As previously mentioned, in attempting to identify implementation costs, the ACFWG found considerable variation in the information received from the sites surveyed. The working group found that NACRS pilot sites reported costs of $.04 to $3.74 per record 8. As none of the sites surveyed could provide accurate actual costs associated with collecting ambulatory care data in Emergency Services or for ambulatory care Clinics, the cost data provided was a best estimate by each site. However, all sites surveyed acknowledged they were registering all of their ER visits. As a result, figure 5 demonstrates the data collection cost estimates based on information provided by each site. 8 CIHI (1997) NACRS Pilot Test Presentation Handout

A Review of CIHI s National Ambulatory Care Reporting System Page 18 Figure 5: Visits per FTE for ER Coding 12 Sites Surveyed 38,000 Visits per FTE 28,000 Visits per FTE 10,000 Visits per FTE 38 33 30 28 28 28 27 27 25 20 17 10 7 30 34 28 50 60 60 50 60 80 108 99 ER-Visits per FTE ER Visits ER Visits per year (x1000) The right portion of Figure 5 shows the number of ER visits per year (x1000 visits) and the left part of the graph demonstrate the number of visits per FTE (x1000 visits) required for coding. Based on the information provided by those hospitals that currently code and abstract ER data most required, on average, 1 FTE per 28,000 visits. Clearly, more work needs to be done to ensure a more accurate cost analysis. Benefits of Collecting Ambulatory Care Data The purpose of implementing a reporting system for ambulatory care is to have consistently good quality data for hospital planning, reporting and accurate funding. Based on survey responses from CIHI s pilot sites and OCCP hospitals, the benefits they identified with collecting ambulatory care data can be grouped into four broad areas: 1) equitable access to services; 2) identification of utilization efficiencies within hospitals; 3) enable accurate funding and 4) enable quality research. Figure 1 in the Background section of this report showed considerable variation in the use of ambulatory care services by geographic region in Ontario. It was acknowledged that moving beyond this recognition without an ambulatory care classification system was not possible. Clinical complexity of patients could be construed as a reasonable explanation for geographic variations in service utilization across the province; however, lacking data regarding resource intensity or clinical complexity, variations in service delivery remain unexplained. Ensuring equitable access to hospital services regardless of geographic residence is a requirement of our publicly funded health care system. Unless accurate data is available, ensuring equitable access or better understanding the reasons for regional disparities is not possible.

A Review of CIHI s National Ambulatory Care Reporting System Page 19 Recently, Ontario has undertaken one of the largest restructuring efforts in its history. Cost efficiency has been promoted as one pre-requisite to ensuring the continued maintenance of our health system. Hospitals have increasingly focussed on achieving utilization efficiencies in order to provide quality care at a lower cost. Ambulatory care services in 1995/96 accounted for $1.5 billion of hospital based expenditures. As hospitals attempt to achieve efficiencies outpatient services are seen as an appropriate substitute for inpatient services. However, very little is known about the characteristics of such patients or whether this trend is appropriate. Accurate ambulatory care data is needed in order to be able to answer these questions. Thus, accurate funding of a growing component of hospital services, as well as clinical outcomes oriented research leading to evidence-based decisions, can only be achieved through the availability of accurate data. Implementing an ambulatory care reporting system is seen as a positive step toward achieving these desirable objectives. Moreover, a National Ambulatory Care Reporting System will facilitate comparisons and understanding of changing practice patterns across provinces and hospitals in Canada. CONCLUSIONS The Working Group unanimously agreed that CIHI s National Ambulatory Care Reporting System was feasible to implement in Ontario for the purposes of hospital based ambulatory care funding (see recommendations at the end of this report for additional detail). To guide implementation of NACRS, the working group identified a time line (APPENDIX 4) based on a projected goal of funding ambulatory care by 2001. This goal would require that NACRS be mandated as soon as possible by the Ministry of Health. In addition, an implementation team with relevant expertise should be assembled to: develop a detailed implementation plan develop an impact analysis for hospitals facilitate hospital ambulatory care data submission to the MoH educate hospital users such as administrators, clinicians, health records and clerical staff develop a data quality review process for assessing the reliability and validity of the data Based on the feedback from the focus group participants, implementation of NACRS could be phased-in through collection of Emergency Services data by July 1999 and all additional ambulatory care clinics by April 2000. The working group acknowledges the challenges implementation will raise with respect to resource requirements and therefore have recommended that strategies be identified to consider the unique requirements of all hospitals in Ontario.

A Review of CIHI s National Ambulatory Care Reporting System Page 20 SUMMARY OF RECOMMENDATIONS In response to its mandate, the JPPC Ambulatory Funding Working Group unanimously agrees that the Canadian Institute for Health Information s (CIHI) National Ambulatory Care Reporting System is feasible to be used as a basis for Ambulatory Care data collection and funding hospitals in Ontario. The Working Group recommends: That the Ontario Ministry of Health mandate the collection of Ambulatory Care data using the Canadian Institute for Health Information s National Ambulatory Reporting System for hospitals in Ontario. Data collection be mandated for Emergency Services (by July 01, 1999) and for all other Ambulatory Care activity (by April 01, 2000) using the full Minimum Data Set, excluding day surgery 9, and incorporate any modifications suggested by the implementation team described in recommendation two. That Ministry of Health establishes an implementation team to implement Ambulatory Care data collection in Ontario hospitals. A representative from this team is identified to provide quarterly progress reports to the Hospital Funding Committee regarding the implementation effort. That the implementation team review CIHI s NACRS final report and recommend any modifications to the minimum data set and grouper resulting from the evaluation conducted by the United States Army Medical Department Center and School. That CIHI provide the implementation team with a production schedule that must indicate: 1) CIHI s effort to facilitate data submissions from hospitals in advance of the implementation dates recommended, 2) provide education sessions and 3) the creation of a data quality review process resulting from first quarter data collection of Ambulatory care data by Ontario Hospitals. That the implementation team devises a strategy that recognizes the resource implications of implementing Ambulatory care data collection in hospitals. The resource strategy should consider the unique requirements of various Ontario hospitals with respect to data collection (such as those in smaller or rural hospitals). Moreover, the Working Group recommends that implementation proceeds in parallel to this process and that the creation of a resource strategy not inhibit progress. 9 The working group recommends that implementation of NACRS minimum data set exclude data elements related to day surgery as this information is already captured in Ontario. (As per the time line, notices of this mandate should be sent concurrently to software vendors who will design data collection systems for hospitals)

A Review of CIHI s National Ambulatory Care Reporting System Page 21 That the JPPC through the Hospital Funding Committee identify a funding mechanism to fund ambulatory care activity using the National Ambulatory Care Reporting System by April 01, 2001. As well, the JPPC through the Hospital Funding Committee request the Canadian Institute for Health Information provide an update to the Hospital Funding Committee in January, 1999 regarding the development of resource intensity weights for Ambulatory Care data.

A Review of CIHI s National Ambulatory Care Reporting System APPENDIX 1 NACRS Core Mandatory Data Elements 10 10 The Implementation Team will further assess each data element and its applicability within the various Ambulatory Care settings. It should be noted that Data Elements 38-40 are only available from case costing hospitals.

A Review of CIHI s National Ambulatory Care Reporting System Core Mandatory Data Elements Type Data Element Grouping Variable Demographic 1 Chart Number 2 Health Care number 3 Province Issuing Health Card 4 Postal Code 5 Gender X 6 Birth Date X Administrative 7 Visit MIS functional Centre Code 8 Date of Visit X Clinical 9 Provider Type(s) 10 Primary Provider Type 11 Provider Identification Number 12 Visits Disposition X 13 Main Problem X 14 Other Problem(s) 15 External Cause of Injury/Poisoning 16 Main Intervention X 17 Other Intervention (s)