Measuring the Performance of Primary Health Care

Transcription

1 Measuring the Performance of Primary Health Care Existing capacity and future information needs October 2006 August 2006 Anne-Marie Broemeling PhD Diane E. Watson PhD MBA Charlyn Black MD ScD Robert J. Reid MD PhD

2 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Library and Archives Canada Cataloguing in Publication Measuring the performance of primary health care: existing capacity and future information needs / Anne-Marie Broemeling... [et al.]. ISBN Primary health care--british Columbia--Evaluation. I. Broemeling, Anne-Marie II. University of British Columbia. Centre for Health Services and Policy Research RA427.9.M C

3 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Contents 4 About CHSPR 5 Acknowledgements 6 Executive Summary 8 Introduction 10 Methods istrative Data Population-Based Survey Data 13 Findings Population Characteristics and PHC Contexts Primary Health Care Inputs Primary Health Care Activities Primary Health Care Outputs Volume and Type Qualities Primary Health Care Outcomes Immediate Outcomes Intermediate Outcomes Final Outcomes 27 Discussion 31 Conclusion O C T O B E R

4 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S About CHSPR The Centre for Health Services and Policy Research (CHSPR) is an independent research centre based at the University of British Columbia. CHSPR s mission is to stimulate scientific enquiry into issues of health in population groups, and ways in which health services can best be organized, funded and delivered. Our researchers carry out a diverse program of applied health services and population health research under this agenda. CHSPR aims to contribute to the improvement of population health by ensuring our research is relevant to contemporary health policy concerns and by working closely with decision makers to actively translate research findings into policy options. Our researchers are active participants in many policy-making forums and provide advice and assistance to both government and non-government organizations in British Columbia (BC), Canada and abroad. CHSPR receives core funding from the BC Ministry of Health to support research with a direct role in informing policy decision-making and evaluating health care reform, and to enable the ongoing development of the BC Linked Health Database. Our researchers are also funded by competitive external grants from provincial, national and international funding agencies. Much of CHSPR s research is made possible through the BC Linked Health Database, a valuable resource of data relating to the encounters of BC residents with various health care and other systems in the province. These data are used in an anonymized form for applied health services and population health research deemed to be in the public interest. CHSPR has developed strict policies and procedures to protect the confidentiality and security of these data holdings and fully complies with all legislative acts governing the protection and use of sensitive information. CHSPR has over 30 years of experience in handling data from the BC Ministry of Health and other professional bodies, and acts as the access point for researchers wishing to use these data for research in the public interest. For more information about CHSPR, please visit 4

5 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Acknowledgements This project has benefited from the contributions of many individuals. In particular, we acknowledge the expert advice and comments provided by Dr Jan Barnsley at the University of Toronto, and Greg Webster, Shamali Gupta and Ron Wray at the Canadian Institute for Health Information. Thanks also to staff at the BC Ministry of Health who reviewed and provided comments on this report. The insights and comments of each of our reviewers contributed greatly to this effort. We would also like to acknowledge staff at the UBC Centre for Health Services and Policy Research (CHSPR) who contributed to this report, and to other primary health care research currently underway. Particular thanks go to Rachael McKendry, Allyson MacDonald and Chris Balma for their assistance. This report complements a multi-stage program of research underway at CHSPR, the goal of which is to develop capacity for system-level performance measurement and evaluation of primary health care in British Columbia. The BC Ministry of Health provided funding to support this project under a contribution agreement between the Ministry and CHSPR. The results and conclusions are those of the authors and no official endorsement by the Ministry is intended or should be inferred. This report relies on information from other organizations. All interpretation and any errors are the sole responsibility of the authors. O C T O B E R

6 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Executive Summary Primary health care (PHC) is the foundation of the Canadian health care system. The vast majority of Canadians visit a PHC provider each year and, for most, PHC is a point of first contact with health care services. PHC is the setting where short-term, acute health issues are resolved and the majority of chronic conditions are managed. Health promotion and preventive services are commonly delivered via PHC, and patients in need of more specialized services are referred by PHC providers to secondary or specialty care. The health of our aging population, and the health of those with chronic conditions, is closely linked with the accessibility and quality of our PHC system. While Canadians have expressed strong support for Canada s health care system, many have also identified health care reform, including PHC renewal, as a priority. Canadians expect improvements in PHC, providers are ready for change, and leaders have committed to ensuring that health care aligns with the needs and expectations of citizens and that governments report routinely on health system performance. This report details how existing population-based data sources can be used to describe PHC, identifies gaps in the current data landscape that hinder PHC reporting, and recommends how these gaps might be filled. This work builds on a performance measurement model, or logic model (detailed in A Results-Based Logic Model for Primary Health Care) developed at the UBC Centre for Health Services and Policy Research. Logic models link a system s resource inputs to activities performed, services delivered and outcomes achieved in order to identify domains for monitoring, evaluating and reporting on system performance. CHSPR s logic model reflects the aims and functions of Canada s PHC system by describing the sector s contexts and chain of inputs, activities, outputs and outcomes. 6

7 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Our review of existing population-based survey and administrative data collection tools confirms that data are available to populate some components of a logic model describing PHC. Data already exist to describe population characteristics and other contextual aspects of PHC, along with some aspects of the sector s human resource, material and financial inputs. Selected policy and governance, health care management and clinical activities and decisions can be described using a mix of administrative and survey data, as can some volumes and types of PHC outputs. However, we lack much of the basic information required to assess and monitor PHC renewal. We lack consistent data on organizations delivering PHC, and consistent data by geographic level of analysis. We lack data regarding some of the sector s outputs (e.g., preventive care and health promotion), qualities of those outputs (e.g., coordination and inter-personal effectiveness), and most immediate outcomes of PHC (reduced risk and effects of continuing or chronic conditions, reduced risk, duration and effects of acute and episodic conditions, and increased knowledge about health and health care). In many instances where data are available, these data are cross-sectional, whereas longitudinal data are needed to assess the impact of PHC inputs, activities and outputs on outcomes over time. Finally, our ability to attribute changes in intermediate and final health outcomes to PHC redesign is limited comprehensive measures and multivariate statistical methods are required to analyze and attribute variations in outcomes to PHC. A complete picture is necessary if we are to evaluate PHC and understand its contribution to health outcomes. To gain a complete understanding of Canada s PHC system, we need to evaluate the linkages between the sector s contexts, inputs, activities, outputs and outcomes. This, in turn, will require a comprehensive data collection strategy and the development of information systems that recognize the linkages between PHC dimensions and support the evaluation of these linkages. A comprehensive data collection strategy and the accompanying information systems would include population-based, multi-level, longitudinal data and link individual patients to providers, clinics and/or organizations, as well as comprehensive data across PHC dimensions (contexts to inputs, activities, and outputs to outcomes). A common, systematic evaluation framework and consensus process should guide Canadian efforts to develop the collection system and the next generation of PHC information systems. Few, if any, governments or organizations in Canada have dedicated the necessary resources to routine measurement and reporting on primary health care. A sustained commitment on the part of government is needed if we are to respond to these information needs and support providers in delivering patient-centred, high-quality care. This commitment to information systems is also required to inform policy development and planning, evaluate the process and outcome of change initiatives, assess the attainment of goals established for PHC and broader health system renewal, and meet the accountability expectations of Canadians. O C T O B E R

8 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Introduction Primary health care (PHC) is the foundation of Canada s health care system. For most Canadians, it is the point of first contact with health care services as well as the setting where short-term, acute health issues are resolved and the majority of chronic conditions are managed. Health promotion and preventive services are commonly delivered via PHC, and patients in need of more specialized services are referred by PHC providers to secondary or specialty care. PHC facilitates continuity and coordination of care across providers, programs, settings, and over time. Canadians have expressed strong support for our health care system but have also identified health care reform, including PHC renewal, as a priority issue. 1,2 A large majority of Canadians (88 per cent) support a strong, publicly funded health care system incorporating the principles of universality, accessibility, comprehensiveness and fairness. At the same time, there is concern among Canadians regarding the performance of the system in 2004, almost two-thirds of Canadians said health care was the issue that should receive the greatest attention from leaders. 3 Canadians opinion of health care performance has declined during the past decade: in 1991, 61 per cent of Canadians felt that the system was excellent or very good but in 2000, only 29 per cent of Canadians rated the system this favourably. Declining confidence in system performance has been accompanied by a belief that greater efficiency can be achieved by governments, providers and health care users, and that accountability can be improved. Canadians support a variety of reform initiatives to improve system performance and PHC renewal figures prominently among proposed enhancements. Suggestions include new models of service delivery, interdisciplinary teams, nurse provision of routine care, cost-effective alternatives, and a focus on wellness, prevention and education. 4 Citizens are prepared to consider additional public funding to sustain health care services but have also noted that funding alone cannot address performance concerns. Canadians require stronger accountability, improved health care management, and regular reports about how services are used, funds are expended and performance compares over time. Governments have recognized these concerns and have introduced initiatives and funding in an effort to ensure that health care aligns with the needs and expectations of citizens. First Ministers commitments to PHC renewal include the Health Transition Fund (1997), 5 Primary Health Care Transition Fund ( ), 6 First Ministers Accord on Health Care Renewal (2003), 7 and 10-Year Plan to Strengthen Health Care (2004). 8 In 2003, First Ministers set a goal of ensuring that 50 per cent of Canadians have access to an appropriate PHC provider at all times by Renewal efforts also seek to address provider concerns and improve working conditions. In response to concerns regarding accountability, Canada s First Ministers have agreed to report routinely on health system performance. In September 2000, First Ministers committed to provide comprehensive, regular public reports on health service performance. 10 The same year, the Canadian Institute for Health Information (CIHI) released the first annual report on health care in Canada. 11 Building on the National Consensus Conference Population Health Indicators framework, the report provided information on health status, non-medical determinants of health, health system performance and community/health system characteristics, using information derived from available data sources. First Ministers subsequently directed Canada s health ministers to develop additional performance indicators and to measure progress toward achieving reforms envisioned in the Health Accord of Fourteen of the 70 indicators proposed were ascribed to PHC, including measures of difficulty obtaining care, use of services, satisfaction with care, and perceived quality of care. 12 Governments subsequently agreed in 2005 to report on new health system investments, including PHC renewal initiatives, funded through the 10-Year Plan to Strengthen Health Care. 8

9 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Despite these investments, a national performance measurement and evaluation framework for the PHC system and PHC renewal does not exist. Such a framework would form a necessary foundation for the development of consistent performance indicators and data collection. A framework is key in evaluating the process and outcomes of change initiatives, enabling performance reporting across jurisdictions and over time, and assessing the attainment of goals established both for PHC and broader health system renewal defined in First Ministers agreements. Throughout 2005 and 2006, CIHI has coordinated a national consensus process with policymakers, providers and researchers to identify PHC indicators based on a common set of evaluation questions. These evaluation questions are based on expert advice, consultation and a review of PHC literature, 13 including A Results-Based Logic Model for Primary Health Care established at CHSPR in The evaluation questions formed the basis for the development of a set of 105 PHC indicators and standardized indicator definitions. 14 Options to enhance data collection to support these indicators have also been proposed. 15 This information is intended to advance PHC performance reporting across Canada, although no government or organization has yet committed to a data collection strategy that would support routine measurement and reporting of these national comparative indicators. At the same time, CHSPR has undertaken the current project a review of existing population-based data capacity to measure PHC system performance using a results-based logic model. 16 This report examines a wide variety of population-based administrative and survey data sources and PHC performance measures currently available. Many are available across Canada. This report complements a multi-stage program of research underway at CHSPR, the goal of which is to develop capacity for system-level performance measurement and evaluation of PHC in British Columbia. Components of the program include: Development of a results-based logic model to identify contexts, inputs, activities, outputs and outcomes of PHC. The logic model is designed to focus evaluative efforts by establishing a common conceptual framework, and is presented in A Results-Based Logic Model for Primary Health Care, released in September Compilation of profiles and indicators to provide a snapshot of the PHC system in British Columbia and the health of its users. This work began to populate the PHC logic model and to address information needs by those responsible for planning and monitoring PHC renewal. The report, Planning for Renewal: Mapping PHC in British Columbia, was released by CHSPR in January Development of a PHC information system to provide timely, valid and reliable performance indicators, and to support evaluation of the dimensions and features of the PHC system and the impact of renewal efforts. Work is underway to extend existing data sources and to ensure reliable and valid data are available to address additional linkages in the PHC logic model and to fill gaps in currently available information. Building on this work, this report details how existing population-based data sources can be used to describe PHC, identifies gaps in the current data landscape that hinder PHC reporting, and recommends how these gaps might be filled. We hope it will be of interest to policymakers, managers and researchers in British Columbia and across Canada, and support efforts to bolster PHC performance reporting and accountability. O C T O B E R

10 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Methods In 2003 and 2004, CHSPR developed a logic model for the PHC sector based on the Treasury Board of Canada results-based management accountability framework, as well as policy analyses, research evidence and broadly based consultation with experts across Canada. Resultsbased logic models link a system s resource inputs to activities performed, services delivered and outcomes achieved. In doing so, they identify domains for monitoring, evaluating and reporting on system performance. CHSPR s logic model reflects the aims and functions of the PHC system in Canada by describing the sector s contexts and chain of inputs, activities, outputs and outcomes. Each is important to understanding and evaluating PHC. Logic models can also help health care policy makers, managers and providers focus quality improvement efforts. We reviewed existing population-based data sources (including surveys of citizens and providers as well as reports using population-based administrative data) to illustrate existing capacity to measure the constructs identified in the PHC logic model. Our review included existing survey and administrative data collection tools, an extensive review of public sector and research centre reports, and peer-reviewed PHC research literature. The survey and administrative data and performance measures identified in this report are not an exhaustive list nor are they intended to describe the best or preferred performance measures. Rather, this information illustrates data availability and provides a guide for future information systems development by highlighting components of the PHC logic model where data are, and are not, currently available. Figure 1: A results-based logic model for primary health care 1 0

11 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H This report focuses on population-based administrative and survey data, each with unique strengths and limitations. There is potential to extend our current capacity beyond existing administrative and survey sources in order to assess the PHC system. For instance, the adoption of widely available electronic medical records and clinical information systems would deepen the scope of performance measures available. Identifying these next steps must reflect the current availability of data and measures, and an understanding of the gaps in sources that should be addressed. istrative Data In British Columbia, population-based administrative data are available through the BC Linked Health Database (BCLHD), under the custodianship of CHSPR at the University of British Columbia (UBC). Data are available for research purposes, subject to access approval by stewards responsible for each source. Data files include Vital Statistics birth and death records, and encounter and transaction records for medical, hospital, continuing care, mental health and PharmaCare services. istrative data are available for the population overall, as well as subgroups of the population by age, sex, geographic area of residence and other units of aggregation. the British Columbia Ministry of Health has used medical, hospital and pharmaceutical administrative data to develop chronic disease registries. 24 These registries have been used to identify and deliver disease management services and to estimate treatment prevalence rates for specific chronic conditions, an important measure for planning and evaluating PHC. istrative data are readily available and relatively inexpensive to access, compared with large paper-based clinical data sources. The strength of administrative data stems from its population basis other data sources are based on samples of the population. Linked administrative data provide a source of information to assess specific dimensions of the PHC system and to populate the PHC logic model. istrative data from the BCLHD have been used to measure morbidity in the population, 18 chronic health conditions and comorbidity, 19 utilization of specific services, 20,21 utilization of services by high users, 22 and continuity of care. 23 The same administrative data are also available to provincial ministries of health and regional health authorities for planning and management. For example, istrative data can be categorized at various geographic levels to address specific local questions and enable comparative analyses across areas, and can support longitudinal analyses and linkages across patients and providers. The reliability of administrative data in investigating specific research questions has been confirmed. 25 However, the depth of variables currently collected limits the use of administrative data in assessing the performance of PHC. O C T O B E R

12 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Survey Data Surveys of the general population or provider groups are also available to assess some dimensions of PHC and the impact of renewal initiatives. While census data are population-based, most surveys provide data for a representative sample of the population or of providers. Examples include the Canadian Community Health Survey (CCHS), the National Longitudinal Survey of Children and Youth (NLSCY), the National Physician Survey (NPS) and the National Survey of the Work and Health of Nurses. Surveys, collected either in-person, by telephone or on paper, provide a range of data from self-assessed health status and self-reported prevalence In many cases, survey data are collected nationally on a one-time or periodic basis and are structured to enable analyses at the provincial or sub-provincial (health region or health service delivery area) level. In some cases, longitudinal survey data are available to allow analysis of trends. For example, the common content of the National Physician Survey enables temporal comparisons. Sampling frames limit the use of survey data for some questions, and some data are not available at the regional or sub-regional level or for specific populations (e.g., institutionalized populations). For the most part, survey data cannot be linked to individual-level administrative data because of issues relating to privacy and confidentiality. However, when aggregated, they can provide ecological measures to support evaluation of PHC. Should existing population-based surveys be deemed useful to the policy and planning community, sampling frames for these tools could be expanded to be representative of other levels of geography. rates for specific conditions, to access to services and satisfaction with care received. Supplements to surveys have also been used to address specific issues, such as the CCHS Health Services Access Survey supplement, which collects data on access to firstcontact and specialist services

13 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Findings Population Characteristics and PHC Contexts It is important to understand the demographic, health status and other population characteristics that influence the need and demand for PHC. Population size, age distribution, dispersion and projected growth are each important factors influencing need for and use of these services. Health status measures are also important, and may include summary measures such as premature mortality as well as condition-specific measures (e.g., teen pregnancy rates), morbidity (e.g., prevalence of chronic health conditions) and mortality (e.g., alcohol-related death rates). Behavioural, psychosocial, social support and health-related attitudes and preferences must also be recognized. 27 Similarly, contextual factors social cohesion, social capital and income disparity, 28,29 economic, cultural, physical, and policy and legislative environment also influence health and patterns of use and delivery of PHC services. These may include individual, community and system-wide factors. Since population characteristics and contextual factors vary across jurisdictions, the need for different types of health care services also varies across geographic or administrative boundaries. 30 It is both expected and desired that PHC inputs, activities and outputs will vary across jurisdictions commensurate to differences in requirements. Moreover, variation in immediate, intermediate and final outcomes may be attributed to variation in these contextual factors, since outcomes are heavily influenced by factors outside the PHC system. Indeed, immediate outcomes are more attributable to PHC inputs, activities and outputs and are less likely to be affected by larger contextual factors than intermediate and final outcomes. Understanding population characteristics and contexts is, therefore, key to evaluating the performance of PHC. Table 1 describes examples of population and contextual measures that are available or can be derived from existing data sources. istrative data can be used to estimate morbidity in the population and among population groups, and treatment prevalence rates for specific chronic health conditions and chronic health conditions overall. Mortality data can also be used to describe health status common measures include infant mortality, life expectancy, standardized mortality ratios and potential years of life lost. CHSPR has used mortality data to derive premature mortality rates by geographic area, a measure that has been described as the single best estimate of a population s need for services. 31,32 Survey data also provide important information on population characteristics. CCHS measures include selfrated health, disability, activity restrictions, self-reported chronic health conditions, behavioural risks including physical inactivity, obesity, tobacco use, risk drinking, illicit drug use, risky sexual practices, and medical interventions. Existing data sources that can inform our understanding of social and economic contexts include socio-economic risk measures such as children at risk and overall socioeconomic risk, occupation, unemployment rates, educational attainment and early childhood readiness to learn measures, seniors receiving the Guaranteed Income Supplement, and seniors living alone. Physical contexts such as remoteness from health services and estimated travel times are also available with current data sources. Data sources currently available describe the population, health status, selected health behaviours, and other contextual factors. However, less information is available for comparative reporting on policy and legislative contexts. O C T O B E R

14 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Table 1: Population characteristics and PHC contexts MEASURES/INDICATORS DATA SOURCES REFERENCES Demographics Population estimates and forecasts: distribution by age, gender, geography Population estimates: distribution by age, sex, geography Population geographic distribution: population density, dispersion Survey Census data Statistics Canada: census 33 BC Stats PEOPLE projections 34 Provincial medical plan registration, vital statistics and utilization data: PHC population research registry Provincial medical plan registration data: postal codes Broemeling et al McGrail et al Births: birth rates, fertility rates Vital statistics BC Vital Statistics Agency 36 Health Status Self-reported health Survey Canadian Community Health Survey 2005 cycle 3.1: GEN_Q01-02 Self-rated mental health Survey CCHS : GEN_Q02B CCHS Mental Health and Wellbeing (MHWB) 2003: SCR_Q08-02 Statistics Canada: CCHS 37 Statistics Canada: CCHS 37 Statistics Canada: CCHS MHWB 38 Two week disability: self-assessed Survey CCHS : TWD_Q1 Q6 Statistics Canada: CCHS 37 Activity restrictions Survey CCHS :RAC Q1-Q8 Statistics Canada: CCHS 37 Morbidity status Provincial medical plans and Hospital Discharge Abstracts Database; Johns Hopkins ACG software 39 Chronic condition prevalence rates: selfreported conditions Chronic condition treatment prevalence rates Chronic condition prevalence rates: provincial registry Chronic condition incidence and prevalence rates Low birth-weight births, births by gestational age Reid RJ et al Reid RJ et al Survey CCHS :CCC_Q Statistics Canada: CCHS 37 Provincial medical plan and hospital administrative data: diagnoses Provincial medical, hospital and pharmaceutical administrative data Provincial registry: records abstraction Broemeling et al Huzel et al Hux et al BC Ministry of Health 24 Watson et al Statistics Canada 43 Vital statistics BC Vital Statistics Agency 36 Mortality: infant mortality Vital statistics BC Vital Statistics Agency 36 Mortality: deaths by cause of death, by Vital statistics BC Vital Statistics Agency 36 age, sex, geography Mortality: life expectancy, potential years of life lost Vital statistics BC Vital Statistics Agency 36 Premature mortality rate Derived variable using vital statistics data and population estimates Watson et al Martens et al Cohen et al

15 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Health Behaviours Health behaviours: current smoking, regular physical activity, fruit/vegetable consumption, overweight/obese, heavy alcohol consumption Health behaviours: alcohol, illicit drug dependence Survey Survey CCHS :SMK_Q PAC_Q1-6 FDC_Q1-3 FVC_Q1-6 HWT_Q4 ALC_Q01-Q15 CCHS :ALC_Q1-6 DRG_Q1-26 CCHS MHWB, 2003 ALD_Q01-Q14 DRG_Q01-Q25 Statistics Canada: CCHS 37 Statistics Canada: CCHS 37 Statistics Canada: CCHS MHWB 38 Changes made to improve health: activity, weight, diet, smoking, alcohol, received medical treatment, vitamins Survey CCHS :CIH Q1-8 Statistics Canada: CCHS 37 Social, Economic, Cultural, Physical and Policy Contexts Children at risk index, youth at risk index Provincial data sources BC Stats 45 Socioeconomic risk index Provincial data sources BC Stats 45 Martens et al Income Survey Census 2001 long form Q51 Statistics Canada: census Income Survey CCHS :INC_Q1-4 Statistics Canada: CCHS 37 Labour force: participation, occupation Survey Census 2001 long form Q34-50 Education Survey Census 2001 long form Q26-31 Social support Survey CCHS :SSA_Q01-20 SSU_Q21-24 Cultural: visible minority groups, immigrant status Survey Census 2001 long form Q9-22 Statistics Canada: census Statistics Canada: census Statistics Canada: CCHS 37 Statistics Canada: census Dunn et al O C T O B E R

16 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Primary Health Care Inputs Inputs describe the human, material and fiscal resources that PHC systems rely on to carry out activities, deliver care and achieve results. 48 Inputs including the health human resources required to provide care, capital investments in physical facilities and equipment with which to deliver services, and financial resources to meet the costs of service delivery are essential to the sustainability of PHC. Table 2: PHC inputs MEASURES/INDICATORS DATA SOURCES REFERENCES Health Human Resources: Physicians GP/FP or specialist counts or rates by population: individuals, FTEs, physicians billing provincial medical plans, physicians actively billing provincial plans Provincial colleges of physicians and surgeons; provincial medical plans; national, provincial, territorial regulatory and licensing bodies, professional organizations, government, educational institutions: Scott s Medical Database; National Physician Database Watson et al Watson et al CIHI CIHI Physicians by type, years since graduation Scott s Medical Database CIHI and geographic area: family medicine, specialists GP/FP: specialist ratios Provincial colleges; provincial Chan medical plans; Scott s Medical Database;National Physician Database GP/FP age, sex distribution Scott s Medical Database CIHI GP/FP age, sex distribution, specialist age, sex distribution Survey Survey (NPS) 2004: Q31, 32 NPS GP/FP number or distribution by geographic area Provincial colleges of physicians and surgeons; provincial medical plans Watson et al Pong et al Pong et al GP/FP distribution by geographic area Survey NPS 2004: Q29e (new) NPS GP/FP hours worked per week Survey NPS 2004 (Q10a) NPS GP/FP Practice: primary population Survey NPS 2004 (Q4a), 2001 (Q5) NPS 2004, served: inner city, urban/suburban, small town, rural, geog isolated or remote, other Patterns of GP/FP geographic stability Survey NPS 2004 (Q18), 2001 (Q29, 31) NPS 2004, Physician migration Scott s Medical Database CIHI Students entering medical school Canadian medical education statistics, 2004 Sex of students enrolling in medical school Canadian medical education statistics, 2004 Decision to train in family medicine Canadian Resident Matching Service: Residency Match Report 2004 Association of Faculties of Medicine of Canada cited in CIHI Association of Faculties of Medicine of Canada cited in CIHI Cited in CIHI Harvey et al Internationally trained physicians Scott s Medical Database CIHI

17 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Table 2 offers examples of information available from existing administrative and survey data that enable us to assess PHC inputs. Health human resources are a particularly significant focus of these data sources. Additional data sources are being developed regarding health human resources, including CIHI s Health Human Resources Databases Development Project and the CIHI, Statistics Canada and Health Canada national survey of the work and health of nurses. Health Human Resources: Other PHC Team Members Health personnel: number and rate per 100,000 population Health Personnel Database: CIHI CIHI Health personnel: numbers Provincial professional association registration data Health professions: distribution by type of profession Survey Statistics Canada Labour Force Survey Health professions: age distribution Survey Statistics Canada Labour Force Survey RN: average age, sex, education level, geographic area, place of employment, FT, PT or casual RN, LPN, RPN: education, work history, employment, satisfaction, work hours, role overload, absences, exposure to risk, general health, stress, depression, medication use RN, LPN, RPN: registrations by age, gender, education, employment, geography Survey Survey Workforce Trends of Registered Nurses in Canada, 2004 National Survey of the Work and Health of Nurses Provincial registration data: registered nurses databases RN migration patterns Provincial registration data: registered nurses database Nursing programs offered at Canadian schools of nursing Material Resources GP/FP IT use: Internet access in primary patient care settings GP/FP use of electronic information, systems, video conferences, PDAs Medical imaging technologies (MRIs, CTs) Fiscal Resources Federal, provincial, territorial, health authority expenditures on physician and other health services (e.g., provincialterritorial government physician expenditure per capita by age and sex) Survey The National Student and Faculty Survey of Canadian Schools of Nursing CIHI Wong et al Cited in CIHI Cited in CIHI Pyper CIHI Workforce Trends of RNs, CIHI CIHI Pitblado et al Canadian Nurses Association 65 Survey NPS 2004 (Q22 (new)) NPS Survey and survey NPS 2004 (Q23), 2001 (Q43 (expansion)) Canadian MIS database, Annual Survey of Medical Imaging Equipment National Health Expenditure database Provincial medical plan expenditures (e.g., BC MSP fee-for-service expenditures, alternate payment program expenditures, out-of-province expenditures) Family spending on health care Survey Survey of Household Spending: Personal and Health Care Spending NPS 2004, CIHI CIHI CIHI Watson et al CIHI Statistics Canada cited in CIHI O C T O B E R

18 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Primary Health Care Activities PHC activities encompass the actions and decisions linking inputs to outputs and outcomes. Policy- and governance-level activity measures include policies related to insured medical services, physician payment methods, professional roles, and the licensing of physician and non-physician providers. Managementlevel decisions and activities describe work settings, practice models and shared resources. Clinical-level measures describe the activities and decisions of Table 3: PHC activities MEASURES/INDICATORS DATA SOURCES REFERENCES Policy and Governance Activities and Decisions Health care financing: percentage of total health expenditure financed by private sector Distribution of medical expenditures by physician payment category: FFS, salary, sessional, capitation, block funding, others; distribution of physician payments by category: alternative clinical, rural incentives, hospital-based, benefits Share of medical spending via alternative payment plans; proportion of physicians who receive alternative payments Remuneration method: FFS, capitation, salary, sessional, service contract, incentives, blended, other (by FP/GP, age group, gender, province) Benefit coverage and cost-sharing for medical and pharmaceutical services Primary health care educational programs availability: number of graduates from selected professional programs and survey Survey National health expenditure database Provincial medical plan billing and expenditure data Alternative payment plan data, National Physician Database NPS 2004 (Q14), 2001 (Q19 (expansion)) CIHI CIHI CIHI NPS , Provincial medical plan and pharmaceutical information Health personnel database CIHI Regulation of health care professionals Health Canada, Health Care Strategies and Policy Directorate Health Care Management Activities and Decisions GP/FP work setting: private office, community clinic, walk-in, academic, community hospital, emergency dept, nursing home, administrative, research unit, lab or diagnostic clinic, other-all, main GP/FP patient care setting: solo practice, group practice, practice network, other Primary care physicians in solo or group practice GP/FP patient care setting, shared resources: office space, equipment, expenses, patient records, staff, on call, other Survey NPS 2004 (Q1, 2a, b); 2001 (Q1, 2 (expansion)) Survey NPS 2004 (Q3a), 2001 (Q3) Janus Project Provincial medical plan data and college of physicians and surgeons registration data Health Canada, Health Care Strategies and Policy Directorate, 2003 cited in CIHI NPS 2004, NPS 2004, College of Family Physicians of Canada 56 Reid et al McKendry et al Survey NPS 2004 (Q3a), 2001 (Q3) NPS 2004,

19 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H providers regarding range and scope of clinical activities, interdisciplinary team activities, and after-hours or on-call arrangements. Table 3 describes examples of policy or governance, management and clinical activities and decisions that can be measured using existing administrative and survey data sources. Many of the identified health care management and clinical activities and decisions focus on physician indicators and data sources there is relatively little information available on interdisciplinary primary care teams. GP/FP patient care setting, shared providers: Survey NPS 2004 (Q3b), 2001 (Q4) NPS 2004, FP, specialist, NP, nurses, dieti- cian, psychologist, OT, PT, SW, pharmacists, technicians, midwives, others GP/FP professional income: by payment Survey NPS 2004 (Q14a), 2001 (Q19) NPS 2004, method Fee for service payments National Physician Database CIHI GP/FP preferred method of remuneration: Survey NPS 2004 (Q14b) NPS FFS, capitation, salary, sessional, service contract, incentives, blended, other GP/FP funding source: APP, RHA, academic, Survey NPS 2004 (Q14c (new)) NPS blended, block funding, rural/north- ern, CMPA, third-party, other GP/FP arrangements for patient care Survey NPS 2004 (Q15 (new)) NPS outside usual office hours GP/FPs accepting new patients: open, Survey NPS 2004 (Q17), 2001 (Q11) NPS partially closed, completely closed GP/FP activities:clinical, CME, vacation, Survey NPS 2004 (Q12), 2001 (Q14) NPS 2004, other GP/FP absences: illness or disability, Survey NPS 2004 (Q13 (new)) NPS personal GP/FP entering or leaving the workforce Scott s Medical DatabaseNational CIHI Physician Database GP/FP decisions to change practice during Survey NPS 2004 (Q18), 2001 (Q29 and 31) NPS 2004, past two years: relocation, scope of practice, teaching, work hours, on-call hours GP/FP plans to change practice during Survey NPS 2004 (Q18), 2001 (Q29 and 31) NPS 2004, coming two years: relocation, scope of practice, teaching, work hours, on-call hours GP/FP most significant changes made or Survey NPS 2004 (Q19 a, b (new)) NPS planned GP/FP reasons for change Survey NPS 2004 (Q19 c (new)) NPS GP/FP IT use: EHR, appointments Survey NPS 2004 (Q23), 2001 (Q43 expansion) NPS 2004, O C T O B E R

20 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Table 3: PHC activities (continued) Clinical Activities and Decisions GP/FP practice profile: patient groups making up more than 10 per cent of practice (e.g., HIV) GP/FP professional activity: range of practise (academic or research, acute) Survey NPS 2004 (Q5), 2001 (Q8) NPS 2004, Survey NPS 2004 (Q7), 2001 (Q10 expansion) NPS 2004, Procedures performed by GP/FP Survey NPS 2004 (Q8), 2001 (Q20) NPS 2004, GP/FP participation in maternity and newborn care Family physician range of practice: IP, surgical assist, surgery, anaesthesia, obstetrics GP/FP weekly work hours by type of activity GP/FP treating patients in home care, hospital, emergency, other institutions On-call activity by type: none, obstetrical, hospital IP, non-hospital telephone, non-hospital patients, ER, NH/LTC, other Survey NPS 2004 (Q9), 2001 (Q21) NPS 2004, Provincial medical plan data, National Physician Database Tepper CIHI 2005(b) 57 Survey NPS 2004 Q10a, 2001 Q13 NPS 2004, Survey NPS 2004 Q10b, 2001 Q13 NPS 2004, Survey NPS 2004 Q11a, 2001 Q15 expansion NPS 2004, On-call hours per month Survey NPS 2004 Q11b, c, d; 2001 Q16 NPS 2004, GP/FP provision of non-office care (emergency, IP, nursing home, house call, obstetrics, anaesthesia) Physician chronic disease management activities: patient registers, monitoring, regular call-back Provincial medical plan billing records Provincial medical plan, chronic disease registries, participation in CDM toolkits Chan BC Ministry of Health 75 Referrals to specialists Provincial medical plan fee for Chan and Austin service data FP activity ratio Derived from medical plan data: Watson et al FTEs per number of FPs Shared patient care with other health care providers Survey NPS 2004 Q3b NPS Teamwork between nurses and physicians; collaboration between nurses and physicians Survey National Survey of the Work and Health of Nurses CIHI 2005 (d) 63 Primary Health Care Outputs Outputs are the direct products and services resulting from the interaction between patients and PHC providers the volume, type and qualities of services provided and received. PHC Outputs: Volume and Type Table 4 describes examples of PHC outputs beginning with the use of PHC services by the entire population, and by subgroups stratified by age, sex, geographic area and morbidity level. For example, the proportion of the population using PHC services within their local health area (LHA) provides a measure of self-sufficiency. Types of PHC services include physician visits for specific conditions or specific types of visits, including preventive, episodic, chronic and palliative care. Information is available from both administrative and survey data for measures such as annual check-ups, screening tests, physician visits for mental health conditions, and use of other health services (specialist physician services, emergency department, hospital, and home care services). istrative data sources quantify the number and type of visits covered through publicly funded services, while survey data describe physician and other health provider use both from the patient perspective and the provider perspective. 2 0

21 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Table 4: PHC outputs: volume and type MEASURES/INDICATORS DATA SOURCES REFERENCES Volume and Type of Outputs FP/GP or PHC visits and visit rates per population by age, gender, morbidity, use patterns Proportion of the population using GP/FP or PHC services during the year Patient visits per week, excluding on call (provider perspective: GP/FP) Contacts with GP/FP, nurse, PT, OT, social worker-counsellor, psychologist, and other non-phc providers (population self-reports) Proportion of population reporting use of telephone health line or telehealth services Use of emergency services,emergency department visits per 1,000 population Contacts with complementary health providers (population self-report) Provincial medical plan billing data; census population data Provincial medical plan billing data; census Watson et al Watson et al Menec et al Roos et al Martens et al Watson et al Survey NPS 2004 Q6, 2001 Q9 NPS 2004, Survey CCHS : HC_Q02, Q03 Statistics Canada CCHS 37 Survey CCHS : ACC_Q40 Statistics Canada CCHS 36 Survey Provincial medical plan fee for service data Chan and Ovens Chan et al Watson et al Survey CCHS : HC_Q04, Q05 Statistics Canada CCHS 37 Use of pharmaceuticals Survey CCHS : MED_Q1 Statistics Canada CCHS 37 Use of pharmaceuticals Provincial pharmaceutical program information Proportion of patients receiving physician services in their own LHA versus other LHA Use of GP/FP services by those with specific conditions or chronic health conditions Contact with services or support for problems with emotions, mental health, use of alcohol or drugs Survey Provincial medical plan billing data; census Provincial medical plan billing data; diagnosis data from provincial medical plan and hospital data CCHS : CMH_Q1 CCHS MHWB 2003 SER_Q002-Q107 Use of GP/FP visits at the end of life Provincial vital statistics and medical plan data Home nursing and home rehab clients per population, or visits per population Home support or home-maker hours per population Provincial continuing care data systems, census data Provincial continuing care data systems, census data Morgan et al Martens et al Watson et al Broemeling et al Watson et al Tomiak et al Martens et al Statistics Canada CCHS 37 Statistics Canada CCHS MHWB 38 Menec et al Watson et al Menec et al Martens et al Watson et al. 2005(a) 17 Home care utilization: type of services used Survey CCHS : HMC_Q09, Q17 Statistics Canada CCHS 37 Residential care services or units per 1,000 population Provincial continuing care data systems Martens et al Frohlich N et al Menec et al While administrative data are available regarding a number of publicly funded health care services, an increasing proportion of medical services are funded through alternative payment programs (APP). 88,89 APP data are only available at an aggregated level, limiting their use for performance indicators and reporting. O C T O B E R

22 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S PHC Outputs: Qualities PHC output qualities focus on the unique and distinguishing features of PHC accessibility, continuity, comprehensiveness and coordination, as well as technical and interpersonal effectiveness. 90 Responsiveness indicators describe the contact or point of entry provided by PHC, and can be assessed using both survey and administrative data. Surveys may include perceptions of accessibility of, and barriers to, care, availability of services when needed, convenience of PHC services and the degree to which there is unmet need. Surveys of accessibility are available both from resident and provider perspectives. istrative data provide measures of actual or realized access use of services. Continuity of care has been defined as including relational, informational and management continuity. 91 Relational continuity can be assessed using survey data from patients to identify their regular source of care, while administrative data has been used to derive continuity of care indices such as usual care provider or sequential continuity of care. Measures of informational and management continuity are generally not available from administrative data sources and require either survey or clinical information to assess. Continuity and accessibility measures are complementary dimensions and increasing accessibility can contribute to reduced continuity. 92 PHC comprehensiveness recognizes the array of services required to meet patients health care needs, including health promotion, disease prevention, diagnosis and treatment of common conditions, management of chronic conditions, referral to specialist providers and palliative care. 93 A key attribute of PHC, comprehensiveness reflects the scope of services provided and is related to the prevalence of conditions: common conditions are usually managed in PHC, whereas less common conditions or presentations are referred to specialty care. Measures of comprehensiveness may include use of primary or secondary preventive services as well as provider activity and referrals to specialist and other providers. Comprehensiveness of services can be assessed using both self-reported survey data as well as administrative data. Existing administrative and survey data support measurement of some qualities associated with PHC but are less able to assess coordination and interpersonal effectiveness. Coordination, another key attribute of PHC, can only be partially assessed using linked administrative data to identify use of a range of services and programs. However, coordination implies that services are integrated and information from one source is used to inform delivery of other services. Using administrative data, it is not possible to assess whether services were actually coordinated (shared information, care management) or simply co-occurred. Similarly, follow-up of referrals to specialty services by a patient s GP or usual provider cannot be assessed using administrative data. Assessing coordination requires input from patients and providers using survey methods not currently used in Canada. Technical effectiveness refers to the use of treatments, procedures and interventions demonstrated to be effective for specific patients or conditions the extent to which care is consistent with evidence-based clinical practice guidelines. Evidence-based guidelines can be compared to care provided or received using administrative data sources, to the extent that the necessary fields are available in existing data. For example, receipt of recommended diagnostic testing for hemoglobin levels, lipids and eye examinations for patients with diabetes are available from administrative data. 94 However, current administrative data limit the use of case mix adjustment when assessing effectiveness. A second measure of effectiveness interpersonal effectiveness incorporates the concepts of communication, shared decision-making, interpersonal style and use of a patient-centred approach to provision of care (e.g., respect for patient preferences). A number of instruments have been designed to measure interpersonal processes of care, but no items representative of this construct have yet been included in Canadian survey instruments. Existing administrative and survey data support measurement of some qualities associated with PHC but are less able to assess coordination and interpersonal effectiveness. 2 2

23 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Table 5: PHC outputs: qualities MEASURES/INDICATORS DATA SOURCES REFERENCES Responsiveness Difficulty obtaining routine or on-going health services Difficulty obtaining health information or advice Difficulty obtaining immediate care for a minor health problem GP/FP assessment of accessibility for patients to selected services: specialists, anaesthesia, ER, palliative, hospital Barriers to accessing mental health services Continuity Number of different physicians seen by individual patients Relational or provider continuity: usual care provider, preponderance of care Primary care provider continuity versus practice continuity Proportion of the population reporting having a regular family doctor Survey CCHS ACC_Q50 Health Services Access Survey, 2001 Survey CCHS ACC_Q41 Health Services Access Survey, 2001 Survey CCHS ACC_Q61 Health Services Access Survey, 2001 Survey NPS 2004 (Q16), 2001 (Q25 expansion) Survey CCHS MHWB 2003 SER_Q Statistics Canada 94 Statistics Canada CCHS 37 Berthelot, Sanmartin 95 Statistics Canada 94 Statistics Canada CCHS 37 Statistics Canada 94 Statistics Canada CCHS 37 NPS , Statistics Canada: CCHS MHWB 37 Provincial medical plan data Barer et al Provincial medical plan data Reid et al McGrail et al Menec et al Provincial medical plan data, college of Reid et al physicians and surgeons registration data Survey CCHS HCU_Q01 Health Services Access Survey, 2001 Berthelot and Sanmartin Statistics Canada CCHS 37 Comprehensiveness Use of primary prevention (e.g., immunization) and secondary prevention (screening) tools (e.g., childhood immunization, smoking cessation, PAP smears, Provincial cancer agencies; provincial medical plan data BC Cancer Agency, 2003/04 98 Lix et al Haggerty et al mammography, colorectal screening) Use of preventive measures (e.g., blood Survey CCHS :BPCQ , Statistics Canada CCHS 37 pressure check, PAP test, mammography, breast exams, eye exams, physical checkup, flu shots) PAP_Q , MAM_Q ; BRX_Q110-BSX_Q122; EYX_Q ; PCU-Q ; FLU_Q Comprehensiveness: scope of practise Provincial medical plan data Watson et al Referral rates (e.g., referrals to specialists) Provincial medical plan data Chan and Austin Population reporting contact with a telephone Survey CCHS QACC_40A Statistics Canada CCHS 37 health line or telehealth service Technical Effectiveness Chronic disease monitoring and receipt of recommended care (e.g., HbA1C testing) Appropriateness of care: potentially inappropriate use of benzodiazepines for older adults or first-line antibiotics in children Provincial medical plan data BC Ministry of Health: Chronic Disease Management, Katz et al Provincial medical or pharmaceutical Katz et al data Immunizations for influenza among seniors Survey CCHS : FLU_Q610 Statistics Canada CCHS 37 Proportion of females aged 18 to 69 with Survey CCHS : PAP_Q020 Statistics Canada CCHS 37 at least one PAP smear in last three years Proportion of women aged 50 to 69 obtaining Survey CCHS : MAM_Q Statistics Canada CCHS 37 mammography in the last two years O C T O B E R

24 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S Primary Health Care Outcomes The PHC logic model identifies three levels of outcomes, ranging from immediate or direct outcomes for which PHC is most directly responsible, to intermediate and final outcomes. Attribution of outcomes to PHC decreases as we move from immediate to intermediate and final outcomes, and as other contextual and external factors intervene. Primary Health Care Immediate Outcomes Immediate or direct outcomes of PHC focus on patients, populations and the PHC workforce. Patient and population immediate outcomes include: increased knowledge about health and health care; reduced risk, duration and effects of acute and episodic conditions; and reduced risks and effects of continuing or chronic conditions. Workforce outcomes focus on maintaining or improving the work life of providers, an outcome often associated with the sustainability of PHC services. Increased knowledge of health and health care enables individuals to maintain or improve their own health, as well as the health and well-being of others. Health knowledge and self-management of risky behaviors could be assessed through self-reported measures regarding whether advice from a PHC provider influenced decisions to stop smoking, to exercise, or to eat a healthy diet. Survey data are available on self-reported changes to improve health, but attributing these changes to PHC remains a challenge. Patient and population outcome measures of reduced risk, duration and effects of acute and chronic health conditions are not readily available through existing administrative or survey data sources. For example, existing measures of chronic conditions focus on population-demographic measures (e.g., prevalence rates for chronic health conditions), clinical activities (e.g., provider participation in chronic disease management) and outputs (visits by those with chronic health conditions, and technical effectiveness measures such as receipt of recommended care). Immediate outcomes are limited to use of hospital services by those with specific chronic conditions, an indicator of unfavourable outcomes. Chronic disease management outcomes, such as the proportion of diabetes patients with hemoglobin or blood pressure below recommended levels, are not available and would require access to clinical data sources such as those available with an electronic health record. Measures assessing the impact of chronic disease management on longer-term health, including complications arising from specific conditions, are obscured by attribution issues and the difficulty of linking complication rates to PHC, rather than other factors. Perhaps the most useful information to assess PHC immediate outcomes relates to provider satisfaction and work life. Aspects of provider satisfaction are available through existing survey data sources such as the National Physician Survey. Primary Health Care: Intermediate Outcomes Intermediate outcomes associated with PHC include acceptability of health care, appropriateness of place and provider, health care system efficiency and health care system equity. Acceptability measures focus on patient satisfaction with services received, as well as overall population satisfaction with, and confidence in, health care services. Measures of satisfaction are available through survey data such as the Canadian Community Health Survey. These data include patient satisfaction with overall health care services, with specific services such as community-based care and tele-health services, and perceived quality of health care. Appropriateness of place and provider reflects PHC s key roles: providing the right service by the right person at the right time, and acting as a source of first-contact care and referral to specialty services. Appropriateness of place requires assessing whether the patient s condition matches the setting in which care is provided. For example, do patients using emergency departments actually require emergency care or do some use the emergency department in the absence of a regular primary health care provider? 2 4

25 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Table 6: PHC immediate outcomes MEASURES/INDICATORS DATA SOURCES REFERENCES Maintain or Improve Work Life of PHC Workforce GP/FP satisfaction: relationship with patients, hospitals, other providers GP/FP satisfaction: continuing medical education or professional development opportunities, availability of locums for CME coverage Survey NPS 2004 (Q20), 2001 (Q33 and 37) NPS 2004, Survey NPS 2004 (Q20), 2001 (Q33 and 37) NPS 2004, GP/FP satisfaction with professional life Survey NPS 2004 (Q20), 2001 (Q33 and 37) NPS 2004, GP/FP satisfaction with personal and professional commitments Work life experiences of nurses: job satisfaction, role overload, nursing work index Survey NPS 2004 (Q20), 2001 (Q33 and 37) NPS 2004, Survey National Survey of Work and Health of Nurses Concern with exposure to risk Survey National Survey of Work and Health of Nurses Overall job satisfaction by employees by health sector Increased Knowledge about Health Care Among Population Changes in risk behaviours Survey CCHS CIH_Q1-8 Smokers who received help or information to quit smoking CIHI CIHI Survey Workplace and Employee Survey Statistics Canada, cited in CIHI Survey CCHS SPC_Q13-14 Statistics Canada: CCHS 37 Statistics Canada: CCHS 37 Reduced Risk and Effects of Continuing Conditions Hospitalization rates/readmissions to hospital for specific chronic health conditions Hospital discharge abstract data CIHI Ambulatory care sensitive condition hospitalization rates Emergency department visits for complications of chronic conditions Hospital discharge abstract data Manitoba Centre for Health Policy: Concept Dictionary 105 CIHI Provincial medical plan data, provincial Booth and Fang chronic disease registry Complication rates for chronic conditions Hospital discharge abstract data and medical service plan data Booth and Fang Appropriateness of provider measures may focus on the mix of providers delivering services (including the mix of primary care and specialist providers) and the role of non-physician providers (dietitians, nurses, pharmacists, social workers, rehabilitation professionals and others) in the ongoing care and management of chronic health conditions and in health promotion and disease prevention. Data sources are not readily available to assess the role of non-physician providers in the delivery of PHC, the appropriateness of each provider, or the mix of providers on interdisciplinary teams. Health care system efficiency as distinct from the efficiency of PHC inputs, activities and outputs reflects the optimal use of resources and allocative efficiency between marginal benefits and marginal costs. O C T O B E R

26 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S One of the principal expectations Canadians hold is that care, including PHC, should be delivered according to need and, specifically, delivered according to relative need rather than ability to pay. Age and gender standardized use of family physician and other health care services, stratified by geography, socioeconomic status and other measures of need, can be used to assess equity in the use of services. Previous reports by CHSPR have plotted local health area measures for a range of PHC services against premature mortality rate to assess the degree of association between supply or use of services and need for care. The level of association between these two measures offers insight regarding equity in geographic distribution of services. Others have used Canadian Community Health Survey data to assess equitable access to medical care across OECD nations. 107 Some measures may be available to assess appropriateness of place and provider, health system efficiency, and health care system equity from existing data sources. As an example, utilization rates for emergency rooms and specialist services are available from administrative sources and can be linked to the use of PHC as well as socio-demographic measures. However, attributing these outcomes to PHC, as opposed to other factors, necessitates caution and the use of more sophisticated analyses. For this reason, specific indicators for intermediate and final outcomes have not been identified in this report. Primary Health Care Outcomes: Final Outcomes A sustainable health care system, improved or maintained individual health and function, and improved level and distribution of population health and wellness are final or ultimate outcomes for PHC. As was the case with intermediate outcomes, the complex interplay of other factors and contexts limit the extent to which final outcomes can be attributed to PHC as compared with other background, behavioural and contextual factors. 2 6

27 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Discussion istrative and survey data are available to populate some components of a logic model designed to describe the primary health care (PHC) system. Specifically, data are available to describe population characteristics and other contextual factors. Data are also available to assess some aspects of PHC s human resource, material and financial inputs at the national, provincial and sub-provincial level. Selected policy and governance, health care management and clinical activities and decisions can be described using a mix of administrative and survey data, as can some outputs of Canada s PHC system. Despite this availability of data, few (if any) governments or organizations in Canada have dedicated the necessary resources to routine measurement and reporting on PHC using these sources. Such an investment would offer important evidence and insights to individuals responsible for planning PHC renewal efforts, monitoring change in contexts and PHC performance, and reporting to Canadians on their investments in this sector. While data are available to measure some aspects of PHC performance, there are notable gaps. The following points describe the most critical information gaps, in no particular order. There are no consistent data available for organizations delivering PHC at the local level. Some information is available to describe PHC inputs, activities and outputs at the provincial and, occasionally, at regional levels. However, no such comparative information is available for individual organizations delivering PHC. Canada has made a significant investment in organizationlevel data collection in some sectors (e.g., acute care hospitals) but no such information has been collected for PHC. Thus, standardized data are not available to support evaluation and comparisons of PHC models across jurisdictions. Specifically, the extent, composition and functioning of interdisciplinary teams, provider turnover rates, availability of services, and other PHC activities and outputs are not collected in a comprehensive or consistent way. The issue of data availability by geographic level of analysis must also be addressed. Currently, some survey data focus on the national or provincial level but do not support performance measurement at the health region, local health area or organization levels. In other cases, survey data are available at the health region or health service delivery area level but not available at the local or organization level. While surveys can provide valuable information to describe PHC, the quality of survey data depends on sampling frames that are representative of the population or provider group. Clearly, information from all providers and organizations is not necessary to assess performance, identify areas for improvement or support public reporting. However, rigorous sampling methods are needed to reduce the risk of bias. We should focus on feasible approaches to data collection that generate reliable and generalizable information and balance the expense of surveys against routine administrative data collection. Existing data capacity is relatively rich in the areas of PHC inputs and activities, and some aspects of PHC outputs, but less robust regarding other PHC outputs. Measures of preventive care and health promotion where there is high consensus (e.g., smoking status during pregnancy, recorded follow-up of patients with abnormal Pap or mammogram results 108 ) are not available. Monitoring these measures would require detailed, patient-level data, potentially from population-based surveys or electronic health records. Canadian jurisdictions lack the most basic information systems that would enable us to assess and monitor immediate outcomes attributable to PHC. Identifying links between the supply of services (inputs, activities and O C T O B E R

28 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S outputs) and outcomes is a clear priority for Canadians. Quality improvement efforts must be guided by information on the impact of inputs, activities, services and qualities most likely to positively influence population outcomes. Some information is available regarding maintaining or improving the work life of the PHC workforce (an immediate outcome of the sector). Physician surveys, such as the National Physician Survey, provide cross-sectional and longitudinal data to begin to assess this immediate outcome. However, while survey data are available for some aspects of physician and nursing work life, fewer data are available for other primary care providers. Limited data are available to measure the immediate outcome of reduced risk and effects of continuing or chronic health conditions. Most chronic care measures focus on activities and outputs, such as visit rates, prescribing practices and the proportion of patients receiving care according to recommended guidelines. A few measures address immediate outcomes (hospitalization rates, emergency room visits). However, little information is available to assess the impact of PHC in managing chronic conditions, disability, functional status or activity limitations associated with chronic conditions. Additional data are required to evaluate chronic care management in association with PHC. Data are not readily available to measure increased knowledge about health and health care among the population, or reduced risk, duration and effects of acute and episodic health conditions. Indicators recommended for measuring management of acute conditions such as record that abnormal urinalysis results have been followed up are not available. While CCHS surveys do collect information on changes in health behaviours, it is not possible to link these changes to PHC practices or interventions. PHC systems evaluation needs to consider data required to expand immediate outcome measures and to address gaps in data availability. Intermediate and final outcomes are those outcomes that are influenced by, but less directly attributable to, PHC. Selected performance measures are available from existing administrative and survey data sources. However, they are not comprehensive, nor can they be fully attributed to PHC. For example, a number of measures are available to assess improved level and distribution of population health and wellness, including premature mortality, mortality by cause, morbidity and self-assessed health. Yet, we know that many factors, in addition to PHC, contribute to these population health outcomes. Health and wellness are acknowledged to be long-term measures that are not typically responsive in the short-term to service delivery and policy changes. In order to attribute changes in intermediate and final outcomes to PHC, multivariate statistical methods are required to identify, isolate and remove confounding factors. Only through these statistical methods can we analyze and attribute variations in outcomes to PHC. A number of PHC performance measures from existing sources are cross-sectional available only at a single point in time. Longitudinal survey or administrative data sources are needed to assess the impact of PHC inputs, activities and outputs on immediate outcomes, over time. Longitudinal measures are particularly important in evaluating the impact of PHC practices and interventions to manage chronic health conditions, currently an issue for at least 36 per cent of our adult population and 68 per cent of seniors (and a need that will grow as chronic condition prevalence rates increase with population aging). The issues of longitudinal data, attribution, and long-term impact must be addressed before we can evaluate the role of PHC in achieving population health improvements. While Canada does have population-based data capacity to begin to report on some aspects of PHC, additional information and information systems are clearly required in order to evaluate patterns of PHC delivered by providers and received by patients, and to evaluate renewal initiatives. 2 8

29 U B C C E N T R E F O R H E A LT H S E R V I C E S A N D P O L I C Y R E S E A R C H Work is underway in a variety of forums that may address some of these needs: E-health initiatives such as the development of electronic health records, work by CIHI to identify consensus-based PHC indicators, work by Statistics Canada, CIHI and Health Canada to develop survey data on the work life and health of nurses, and research such as the PHC program underway at CHSPR. However, no government or agency has committed to a comprehensive collection strategy to support routine measurement and reporting of national comparative PHC data, information and indicators. PHC inputs, activities and outputs and, in turn, improve the contribution of PHC to the level and distribution of health and wellness and a sustainable health care system. Gaining a fuller understanding of our PHC system requires the development of a comprehensive data collection strategy and information system. Attributes of such a system would include: population-based, multi-level data regarding individual residents, patients, providers, clinics and organizations; Additional efforts are needed if we are to extend our capacity to evaluate PHC renewal initiatives. Too often, we look only at specific pieces of the complex puzzle that is PHC. Focusing on specific parts of the system by populating selected components of the PHC logic model will only permit a partial description of the sector. If we are to gain a comprehensive understanding of our PHC system, we need to evaluate the linkages between contexts, inputs, activities, outputs and outcomes. To do so requires a comprehensive data collection strategy that recognizes the links between PHC dimensions and that supports evaluation of these links. For example, do interdisciplinary teams improve continuity of care and reduce the impact of chronic health conditions? Evaluating the linkages between PHC dimensions is necessary to identify the most important contributors to quality. For example, what patterns of use and level of continuity reduce the impact of chronic health conditions? Only then can we truly understand and evaluate PHC in a way that supports attribution of outcomes to longitudinal data that support temporal analyses, as opposed to multiple cross-sectional surveys that illuminate temporal trends but do not permit longitudinal analyses of factors that influence the health of Canadians; multi-level data that links individual patients to providers, clinics and organizations; comprehensive data collection across all aspects of the PHC system (from contexts to inputs, activities and outputs to outcomes) that recognizes the linkages between each of these aspects; and enhanced data collection of the evolving complement of encounters and interactions that patients have with PHC, as well as services provided by a complex array of providers. It is possible to build data collection systems with these attributes. In fact, other countries have implemented working data collection systems that incorporate some of these features. 109,110 However, even these systems are not built on a systematic framework or prospectively defined objectives for a comprehensive data collection strategy. Canada has a real opportunity to build a data collection system based on a common, systematic evaluation framework or consensus process to identify objectives relevant to those responsible for maintaining and improving quality and accountability to Canadians. O C T O B E R

30 M E A S U R I N G T H E P E R F O R M A N C E O F P R I M A R Y H E A LT H C A R E : E X I S T I N G C A P A C I T Y A N D F U T U R E I N F O R M A T I O N N E E D S The use of a common, systematic evaluation framework and consensus process should guide Canadian efforts to develop a comprehensive, population-based data collection system and the next generation of PHC information systems. We strongly encourage provincial, territorial and national governments to work collaboratively to develop a data collection strategy that reflects the foundational nature of PHC and builds on the qualities and strengths of population-based data. A common evaluative framework and comprehensive, pan-canadian data collection strategy will provide the necessary building blocks for PHC evaluation and performance measurement. This will require considerable effort and investment a sustained commitment is needed if we are to help providers deliver patient-centred care, organize their practices and simplify their work responsibilities, support performance monitoring, and provide incentives for high-quality care. This commitment and information is also needed to inform policy and planning, to evaluate the process and outcomes of change initiatives, to assess progress toward goals established for PHC and broader health system renewal, and to meet the accountability expectations of Canadians. 3 0