Developing a method to estimate practice denominators for a national Canadian electronic medical record database

Family Practice 2013; 30:347 354 doi:10.1093/fampra/cms083 Advance Access publication 10 January 2013 The Author 2013. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. Developing a method to estimate practice denominators for a national Canadian electronic medical record database Michelle Greiver a, *, Tyler Williamson b, Terri-Lyn Bennett c, Neil Drummond d, Colleen Savage e, Babak Aliarzadeh a, Richard V. Birtwhistle b, Shahriar Khan b and the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) a Department of Family and Community Medicine, University of Toronto, Ontario, Canada, b Department of Family Medicine, Queen s University, Kingston, Ontario, Canada, c Public Health Agency of Canada, Ottowa, Ontario, Canada, d Department of Family Medicine, University of Alberta, Edmonton, Alberta, Canada and e Department of Community Health and Epidemiology, Queen s University, Kingston, Ontario, Canada. *Correspondence to Dr. M. Greiver, Department of Family and Community Medicine, University of Toronto, 240 Duncan Mill Road, Suite 705, Toronto, Ontario M3B 3S6, Canada; E-mail: mgreiver@rogers.com Received 7 September 2012; Revised 30 November 2012; Accepted 11 December 2012. Background. Calculating disease prevalence requires both a numerator (number of persons with a disease) and a matching denominator (the population at risk being studied). Determining primary care practice denominators is challenging. Objective. To develop and test a method to calculate primary care practice denominators. Methods. We compared a corrected yearly contact group, or practice population, with the number of patients enrolled with practices. The yearly contact group was the set of patients with a visit noted in the electronic medical records during the past year. The correction factor was the proportion of patients that reported contacting their physician in the past year. Eighty-one physicians from Toronto and Kingston, Ontario, provided data. The main outcome measure was the ratio of practice population to the number of enrolled patients. Other measures included the change in ratio over 2 years, differences between locations, and differences by provider, practice and patient characteristics. Results. The ratio of practice population to enrolled patients was 1.03 in 2010 (95% confidence interval 1.00 to 1.05) and 1.03 in 2011 (95% confidence interval 1.00 to 1.05). There was no change in the ratio over time. Ratios by location, provider or practice characteristics differed by less than 10%. There was a slight under-estimation of practice population for younger male patients and over-estimation for female patients. Conclusion. This method provided a denominator that was reasonably similar to the enrolled population and was stable over time and by location, provider and practice characteristics. In regions without patient enrollment, this may provide an estimate of practice denominators. Keywords. Electronic health records, epidemiologic methods, health care evaluation mechanisms, morbidity prevalence, primary health care, sentinel surveillance. Introduction The Canadian Primary Care Sentinel Surveillance Network The use of electronic medical records (EMRs) in Canadian primary health care is growing: 49% of practices reported using an EMR system in 2010. 1 This provides an important opportunity to collect clinical data for purposes of research and epidemiological surveillance without significantly increasing physician workloads. The Canadian Primary Care Sentinel Surveillance Network (CPCSSN) is Canada s first multi-disease electronic medical record surveillance system. 2 We aim to collect valid and reliable data about common chronic diseases (chronic obstructive lung disease, hypertension, depression, osteoarthritis, diabetes, Parkinson s disease, epilepsy and dementia) for epidemiological surveillance and research. CPCSSN has recruited sentinel primary care practices; 3 information collected from sentinels is intended to be representative of a larger population 3 in order to study and monitor public health events of interest. 4 The majority of interactions with the health system occur in primary care; 5 7 CPCSSN fills an important gap using clinical information collected 347

348 Family Practice The International Journal for Research in Primary Care from primary care health records that can enrich claimbased data obtained through traditional administrative surveillance systems. 8 Similar primary care surveillance systems exist in the UK, the USA, the Netherlands, Belgium, Australia, Scotland, France, Italy and New Zealand. 9 15 These surveillance systems provide data on the incidence and prevalence of specific diseases and can monitor health trends over time. As well, they identify at-risk groups and analyse disease management and drug safety issues. Primary care providers in 10 practice-based Primary Care Research Networks located across Canada contribute EMR data to CPCSSN. The data are rendered anonymous and are sent via secure electronic file transfer protocols to CPCSSN s central data repository where they are aggregated into a single national database. Posters describing the system are present in the waiting rooms of all participating practices and patients may withdraw their data if they so choose. 2 Practice denominators Explicitly and systematically defined practice populations, or denominators, are crucial for public health surveillance and epidemiological studies using primary care sentinel surveillance data. 16 Denominators enable comparison of incidence and prevalence of disease, risk factors or other public health metrics across time and between clusters of practices, health systems and geographical (e.g. provincial) units. 4,17 At a minimum, age and gender data should be available for the denominator, as this allows stratification and age/gender-based standardization. 18 A practice population may be defined as the set of all individuals in a given community who, if they had need to, would seek health care from that practice on any given day. 16 In other words, a primary relationship or sustained partnership may be reasonably assumed to exist between a patient, a family physician and his or her practice colleagues. 19 Challenges in determining denominators in primary care contexts have been the subject of international debate. 20 22 At the centre of the problem is the fact that although numerators for conditions are usually drawn from observations about patients attending their family physician s practice, the denominator consists both of patients who do attend within a certain period of time, and those who do not but still consider that physician to be their principal primary care provider. 16 Approaches to solving the denominator problem in the Canadian context The data available on the population seeking medical care in a particular location depends partly on the local health-care system. In locations where persons are enrolled (or rostered) with a specific provider, practice populations may be more readily estimated. Although non-enrolled or inappropriately enrolled persons are potential sources of error, 23,24 delays and errors in the addition and removal of patients may even out over time. 23 In Canada, Ontario currently offers family physicians the opportunity to enroll patients. Each enrolled (or rostered) patient has signed a form designating a particular physician as their personal family physician and the practice has endorsed and submitted the registration form to the Ontario Ministry of Health and Long Term Care. 25 Parents or guardians can enroll children less than 16 years old or non-competent persons that they are responsible for. 25 There are financial incentives accruing to physicians for enrolling patients, such as capitation payments and additional fees for chronic disease management. 26 However, other Canadian provinces do not currently have broadly based primary care patient enrollment systems. A model basing the denominator on enrolled practice populations is therefore not applicable to a Canadian national primary care surveillance network. Another method for addressing the denominator issue is to measure practice attendance over a fixed period of time. 22 Variations to this approach include measuring attendance in the previous 2 years 27 or in the previous year plus in any of the 2 years prior to that year. 16 Approximately 80% of the patients treated in a 2-year time frame attend over a 1 year period. 27 Following previous work by Schlaud 18 and Mayo et al., 28 Bartholomeeusen et al. concluded that in Belgium a practice population could be reasonably estimated using the yearly contact group (patients who have attended over the past year) combined with age- and gender-specific correction factors derived from population-level estimates of primary care utilization such as surveys or medical billing information. 10 This approach has not been validated in the Canadian setting. Canadian provincial and national data on primary care utilization rates by age and gender are available from the Canadian Community Health Survey (CCHS). 29 The availability of utilization data across provinces and territories makes a corrected yearly contact group approach feasible for a pan-canadian sentinel-based network. Age- and gender-based utilization correction factors do not take into account variation in clinic attendance due to socio-economic factors or levels of chronic morbidity. However, variation in factors such as income or education may have a smaller influence on primary care utilization in Canada than in systems without populationbased health insurance, possibly due to fewer financial barriers to access. 30,31 Variation in population levels of chronic morbidity may be responsible for differences in utilization between groups of physicians. 30,32 The purpose of this study was to develop and test a pragmatic method to calculate practice denominators

Estimation of primary care practice denominators 349 applicable to a pan-canadian primary care surveillance network. This would enable the estimation of disease incidence and prevalence rates across Canada and would therefore allow comparability of findings between participating primary care research networks, health regions and provinces. As well, it would enable comparisons with other population-based surveillance systems within Canada and abroad, and with Canadian population-based disease registries. We also explored denominator comparisons by provider and patient characteristics in order to provide information for denominator adjustment for these groups. Methods Participants and data sources About 46 primary care providers in Toronto, Ontario, and 35 in Kingston, Ontario, who contribute data to CPCSSN were included in this study. We collected data on all encounters between 1 January 2010 and 31 December 2011 for patients aged 12 and older who had not opted out of CPCSSN. Each patient with at least one encounter recorded during each calendar year was counted once in the analysis, regardless of the actual number of encounters they made. We collected patient enrollment status, as recorded in the EMR, for all active patients aged 12 and older in each practice as of 31 December 2010 and 31 December 2011. We used the 2009 10 CCHS data on yearly primary care contact by age and gender as the basis for the utilization correction factor. 33 This is a large national population health survey that targets all Canadians aged 12 or older with the exception of people living on aboriginal reserves, residents of institutions and fulltime members of the Canadian Forces. The survey asks respondents whether, in the past 12 months, they have seen or talked to a family physician or general practitioner about their physical, mental or emotional health. This question was used to estimate the proportion of the population reporting at least one contact with their primary care provider in a given year, by age, gender Table 1 Patients reporting at least one contact with a family physician or general practitioner in the past 12 months, province of Ontario, Canada, Canadian Community Health Survey 2009 31 Patient gender Age group (years) Patients reporting contact in past year (%) Mean (SD) 95% Confidence interval Male 12 14 71.7 (2.7) 66.5 77.0 15 19 70.5 (2.4) 65.7 75.3 19 24 67.0 (2.9) 61.3 72.8 25 29 68.2 (3.1) 62.2 74.2 30 34 77.3 (2.3) 72.9 81.7 35 39 72.7 (2.3) 68.2 77.3 40 44 69.9 (2.8) 64.5 75.3 45 49 72.8 (2.7) 67.5 78.2 50 54 80.3 (2.1) 76.2 84.5 55 59 78.7 (3.0) 72.9 84.6 60 64 85.7 (2.3) 81.3 90.2 65 79 89.5 (1.6) 86.5 92.6 70 74 89.7 (1.8) 86.3 93.2 75 79 93.1 (1.3) 90.6 95.5 80 or more 92.0 (2.2) 87.7 96.4 Female 12 14 71.3 (2.9) 65.5 77.0 15 19 80.8 (2.2) 76.6 85.1 19 24 82.5 (2.5) 77.6 87.4 25 29 81.6 (2.3) 77.1 86.1 30 34 86.6 (1.8) 83.0 90.2 35 39 85.6 (1.6) 82.5 88.6 40 44 83.3 (2.1) 79.1 87.5 45 49 86.3 (1.9) 82.6 90.1 50 54 82.9 (2.3) 78.4 87.4 55 59 87.5 (2.3) 82.9 92.1 60 64 91.3 (1.4) 88.5 94.1 65 79 90.5 (1.4) 87.9 93.2 70 74 90.4 (1.5) 87.6 93.3 75 79 90.7 (2.0) 86.9 94.6 80 or more 88.2 (2.6) 83.1 93.3

350 Family Practice The International Journal for Research in Primary Care Table 2 Provider and practice characteristics per year and per location Table 3 Ratios of practice population and number of enrolled patients by provider and practice characteristics for each year Year 2010 2011 Provider and practice characteristics Number of providers (%) a Number of providers (%) a Provider age, Kingston b 25 44 4 (12.1) 4 (11.4) 45 64 14 (42.4) 13 (37.1) 65+ 2 (6.1) 2 (5.7) Missing 13 (39.4) 16 (45.7) Total 33 (100) 35 (100) Provider gender, Female 17 (51.5) 17 (48.6) Kingston b Male 14 (42.4) 16 (45.7) Missing 2 (6.1) 2 (5.7) Total 33 (100) 35 (100) Number of enrolled patients per provider, Kingston c <600 22 (66.7) 21 (60.0) 600 1000 9 (27.3) 12 (34.3) >1000 2 (6.1) 2 (5.7) Provider age, Toronto b 25 44 18 (40.9) 19 (41.3) 45 64 23 (52.3) 23 (50.0) 65+ 2 (4.5) 2 (4.3) Missing 1 (2.3) 2 (4.3) Total 44 (100) 46 (100) Provider gender, Female 27 (61.4) 29 (63.0) Toronto b Male 17 (38.6) 17 (37.0) Missing 0 (0.0) 0 (0.0) Total 44 (100) 46 (100) Number of enrolled patients per provider, Toronto c <600 5 (11.4) 5 (10.9) 600 1000 18 (40.9) 20 (43.5) >1000 21 (47.7) 21 (45.7) a Percentages may not add up to 100 due to rounding. b Obtained from physician surveys. c Obtained from EMR data. and province of residence. CCHS data were weighted to be representative of the Canadian population and 95% confidence intervals (CIs) were calculated using a bootstrap resampling technique to account for the complexity of the CCHS sampling design. We used sentinel demographic information collected through a yearly CPCSSN survey to characterize the practices for the two geographical areas included in this study. Outcome measures In order to estimate the practice population, we first divided the number of patients for each provider, for each age and gender stratum with at least one encounter in the EMR in each calendar year ( yearly contact group ) by the appropriate age- and gender-specific utilization correction factors. We then summed these products in order to estimate the practice population. Analysis In order to compare the two denominators, we calculated the ratio of estimated practice population (derived Year using our corrected yearly contact group approach) and number of enrolled patients for each practice (as recorded and extracted from the EMRs) for each year of interest. We grouped ratios by calendar year, by city (Toronto or Kingston), and by provider characteristics (age, gender, size of practice by patient enrollment status). The results were described using mean ratios, as well as the corresponding standard deviations and CIs using standard methods. We also calculated ratios of estimated practice population to enrollment numbers for each patient age gender group. The analyses were performed using SAS version 9.3. CPCSSN has ethics approval from the Research Ethics Boards of the host universities for all participating networks and all participating CPCSSN sentinel primary care providers have provided written informed consent to the collection and analysis of their EMR data. Results Provider and practice characteristics Ratio of practice population and number of enrolled patients Mean (SD) 95% confidence interval 2010 Provider age 25 44 1.00 (0.13) 0.95 1.06 45 64 1.03 (0.10) 1.00 1.07 65+ 1.05 (0.11) 0.88 1.22 Provider gender Female 1.03 (0.09) 0.99 1.06 Male 1.02 (0.14) 0.97 1.08 Number of enrolled patients per provider <600 1.11 (0.08) 1.08 1.15 600 1000 0.99 (0.11) 0.93 1.04 >1000 0.98 (0.09) 0.94 1.02 2011 Provider age 25 44 1.04 (0.13) 0.99 1.09 45 64 1.01 (0.07) 0.98 1.03 65+ 1.00 (0.07) 0.89 1.11 Provider gender Female 1.02 (0.09) 0.99 1.05 Male 1.02 (0.11) 0.97 1.06 Number of enrolled patients per provider <600 1.02 (0.09) 0.97 1.06 600 1000 1.02 (0.09) 0.98 1.06 >1000 1.02 (0.11) 0.97 1.07 The estimated Ontario age- and gender-specific primary care contact rates and 95% CIs 33 are presented in Table 1. Provider and practice characteristics are presented in Table 2. About 77 providers were included in 2010 and 81 providers were included in 2011. The total number of enrolled patients was 62,615 in 2010 and 63,604 in 2011. The mean number of enrolled patients per provider was 813 (SD 395) in 2010 and 785 (SD 364) in 2011. The mean ratio of practice population to enrolled patients was 1.03 in 2010 (95% CI: 1.00 1.05) and 1.03 in 2011 (95% CI: 1.00 1.05). There was a difference of

Estimation of primary care practice denominators 351 Table 4 Ratios of practice population and number of enrolled patients by patient characteristics for each year Year Patient characteristics Ratio of practice population and number of enrolled patients Mean (SD) 95% confidence interval 2010 Patient age ranges 12 14 0.92 (0.27) 0.87 0.97 15 19 0.96 (0.22) 0.92 0.99 20 24 0.98 (0.42) 0.91 1.05 25 29 1.02 (0.34) 0.97 1.08 30 34 0.98 (0.36) 0.92 1.04 35 39 1.02 (0.25) 0.98 1.06 40 44 1.04 (0.22) 1.00 1.07 45 49 1.03 (0.18) 1.00 1.06 50 54 1.07 (0.17) 1.04 1.10 55 59 1.10 (0.23) 1.07 1.14 60 64 1.04 (0.15) 1.01 1.06 65 69 1.06 (0.19) 1.03 1.10 70 74 1.08 (0.15) 1.06 1.11 75 79 1.04 (0.16) 1.02 1.07 80+ 1.08 (0.27) 1.04 1.12 Patient gender Female 1.06 (0.26) 1.04 1.07 Male 1.00 (0.25) 0.98 1.01 2011 Patient age ranges 12 14 0.91 (0.31) 0.86 0.96 15 19 0.98 (0.23) 0.95 1.02 20 24 0.97 (0.49) 0.90 1.05 25 29 1.01 (0.46) 0.94 1.09 30 34 0.99 (0.38) 0.93 1.05 35 39 1.02 (0.27) 0.97 1.06 40 44 1.04 (0.26) 1.00 1.08 45 49 1.01 (0.18) 0.99 1.04 50 54 1.04 (0.18) 1.01 1.06 55 59 1.06 (0.16) 1.04 1.09 60 64 1.03 (0.14) 1.01 1.05 65 69 1.06 (0.15) 1.04 1.09 70 74 1.08 (0.16) 1.06 1.11 75 79 1.08 (0.18) 1.05 1.11 80+ 1.09 (0.40) 1.02 1.15 Patient gender Female 1.06 (0.29) 1.05 1.08 Male 0.99 (0.28) 0.97 1.00 0.07 between Toronto and Kingston in 2010 (95% CI: 0.02 0.12), and of 0.03 in 2011 (95% CI -0.01 to 0.08). Ratios by provider and practice characteristics are presented in Table 3, and ratios by patient age and gender are presented in Table 4 and Figure 1. There were no consistent differences in the ratio by provider characteristics (age, gender, practice size). There was an under-estimation of practice population for younger male patients and an over-estimation for female patients in most age groups, which persisted over the 2 years studied. Estimated practice populations generally differed from the number of enrolled patients by less than 10%. Discussion This method generated a denominator that was similar to that derived from patient enrollment. There were no large differences in the ratio of estimated practice population (derived using our corrected yearly contact group method) to number of enrolled patients over time, by location of provider groups, by provider or by practice characteristics. The method slightly, but not unreasonably, over-estimated the denominator for female patients. We acknowledge certain limitations in this work. Sentinel practices and their populations may not be representative of the general population; CPCSSN is currently expanding to include more sentinels and sites according to an explicit sampling frame that adjusts for clustering effects. Further studies could compare sentinel characteristics and their patient characteristics to population-based estimates. There are many factors that influence why a patient may or may not consult, such as the way a person experiences or understands symptoms, the accessibility of

352 Family Practice The International Journal for Research in Primary Care Figure 1 Ratios of estimated practice population and number of enrolled patients by patient age and gender for each year primary health care in a given area, or social and cultural influences. As well, higher levels of morbidity and poor mental health are associated with increased primary care utilization. 30,32 The correction factor proposed in the current methodology is not able to take all of these variables into account. However, we were able to demonstrate reasonable levels of agreement between our method and enrolled patient status. The use of the population-level utilization correction factor makes the assumption that patients within this study consult at a similar rate to the general population. If for some reason patients within a specific practice consult at a higher or lower rate, this methodology would then over-estimate or under-estimate the practice population. For example, a practice that routinely provides reminders and recalls for preventive periodic screening may end up having a practice population that is more likely to consult at least once in a given year than the average population. In turn, the current methodology applied to that practice would over-estimate the denominator and subsequently cause a potential under-estimation of any disease prevalence or incidence rates. Again, we believe that the level of variation expressed in our data is within acceptable tolerances. There are limitations to the reference standard we chose (enrollment data). Patient enrollment may be constantly changing due to new patients being added and departures of other patients. There may be a delay in updating the patient lists. However, in practice, unless dramatic changes occur in the practice environment

Estimation of primary care practice denominators 353 itself or in the community it serves, these are unlikely to lead to unknown and epidemiologically significant biases; enrollment status has been used in other studies to define practice population. 16,34 Another assumption inherent in this methodology is that the patients at a given practice are all attending from the same geographical region, which is the level at which the correction factor is estimated and applied. In this study, this was province specific. Thus, if a practice looked after a significant number of out-of-province patients, the correction factor would be inaccurate. We had no a priori reason to assume a high number of outof-province patients in either of the study sites. Conclusion Using a yearly contact group with a correction factor derived from the CCHS is a feasible method of generating systematic practice denominators. This method provides a practice population estimate that is close to enrolled patient status for different primary care provider groups and patient populations and therefore represents a pragmatic denominator option for Canadian sentinel-based, primary care surveillance. It may be applied in jurisdictions that do not have patient enrollment to a designated primary care provider, subject to careful judgement about the applicability of the correction factor to the population being studied. Declaration Funding: Public Health Agency of Canada. The views expressed herein do not necessarily represent the views of the Public Health Agency of Canada. Ethical approval: CPCSSN has ethics approval from the Research Ethics Boards of the host universities for all participating networks and all participating CPCSSN sentinel primary care providers have provided written informed consent to the collection and analysis of their EMR data. Conflict of interest: none. References 1 National Physician Survey. Ottawa: College of Family Physicians of Canada and Canadian Medical Association, 2010. 2 Birtwhistle R, Keshavjee K, Lambert-Lanning A et al. Building a pan-canadian primary care sentinel surveillance network: initial development and moving forward. J Am Board Fam Med 2009; 22: 412 22. 3 Green LA, Miller RS, Reed FM, Iverson DC, Barley GE. How representative of typical practice are practice-based research networks? A report from the Ambulatory Sentinel Practice Network Inc (ASPN). Arch Fam Med 1993; 2: 939 49. 4 Fleming DM, Schellevis FG, Paget WJ. Health monitoring in sentinel practice networks: the contribution of primary care. Eur J Public Health 2003; 13(3 Suppl): 80 4. 5 Green LA, Fryer GE Jr, Yawn BP, Lanier D, Dovey SM. The ecology of medical care revisited. N Engl J Med 2001; 344: 2021 5. 6 Manuel DG, Maaten S, Thiruchelvam D, Jaakkimainen L, Upshur R. Primary Care in the health care system. In: Jaakkimainen L UR, Klein-Geltink JE, Leong A, Maaten S, Schultz SE, Wang L (eds). Primary Care in Ontario: ICES Atlas. Toronto: Institure for Clinical Evaluative Sciences, 2006. 7 Schultz SE TJ, Guttmann A, Jaakkimainen L. Characteristics of primary care practice. In: Jaakkimainen L UR, Klein-Geltink JE, Leong A, Maaten S, Schultz SE, Wang L (eds). Primary Care in Ontario: ICES Atlas. Toronto: Institute for Clinical Evaluative Sciences, 2006. 8 de Lusignan S, Metsemakers JF, Houwink P, Gunnarsdottir V, van der Lei J. Routinely collected general practice data: goldmines for research? A report of the European Federation for Medical Informatics Primary Care Informatics Working Group (EFMI PCIWG) from MIE2006, Maastricht, The Netherlands. Inform Prim Care 2006; 14: 203 9. 9 García Rodríguez LA, Pérez Gutthann S. Use of the UK General Practice Research Database for pharmacoepidemiology. Br J Clin Pharmacol 1998; 45: 419 25. 10 Bartholomeeusen S, Kim CY, Mertens R, Faes C, Buntinx F. The denominator in general practice, a new approach from the Intego database. Fam Pract 2005; 22: 442 7. 11 Biermans MC, Spreeuwenberg P, Verheij RA, de Bakker DH, de Vries Robbé PF, Zielhuis GA. Striking trends in the incidence of health problems in The Netherlands (2002-05). Findings from a new strategy for surveillance in general practice. Eur J Public Health 2009; 19: 290 6. 12 Hueston WJ, Mainous AG III, Ornstein S, Pan Q, Jenkins R. Antibiotics for upper respiratory tract infections. Follow-up utilization and antibiotic use. Arch Fam Med 1999; 8: 426 30. 13 Hall J, Tomlin A, Martin I, Tilyard M. A general practice minimum data set for New Zealand. N Z Med J 2002; 115: U200. 14 Stocks NP, McElroy H, Sayer GP, Duszynski K. Acute bronchitis in Australian general practice. A prescription too far?aust Fam Physician 2004; 33: 91 3. 15 Lagasse R, Desmet M, Jamoulle M et al. European situation of routine medical data collection and their utilization for health monitoring: Euro-Med-Data final report. Brussels: Universite Libre de Bruxelles, 2001. 16 Hutchison BG, Hurley J, Birch S, Lomas J, Stratford-Devai F. Defining the practice population in fee-for-service practice. Health Serv Res 1997; 32: 55 70. 17 Schach E. Numerators and denominators of rates estimated on the basis of physicians practice data. Gesundheitswesen 1993; 55(1 Suppl): 18 21. 18 Schlaud M. Comparison and Harmonisation of Denominator Data for Primary Health Care Research in Countries of the European Community. IOS Press, 1999. 19 Lasko TA, Atlas SJ, Barry MJ, Chueh HC. Automated identification of a physician s primary patients. J Am Med Inform Assoc 2006; 13: 74 9. 20 Bass M. Approaches to the denominator problem in primary care research. J Fam Pract 1976; 3: 193 5. 21 Cherkin DC, Berg AO, Phillips WR. The practice denominator. J Fam Pract 1982; 15: 611. 22 Cherkin DC, Berg AO, Phillips WR. In search of a solution to the primary care denominator problem. J Fam Pract 1982; 14: 301 9. 23 Fraser RC, Clayton DG. The accuracy of age-sex registers, practice medical records and family practitioner committee registers. J R Coll Gen Pract 1981; 31: 410 9. 24 Fraser RC. The reliability and validity of the age-sex register as a population denominator in general practice. J R Coll Gen Pract 1978; 28: 283 6. 25 Memorandum of Agreement Between the Ontario Medical Association and the Ontario Ministry of Health and Long Term Care. Toronto: Government of Ontario, 2004. 26 Glazier RH, Klein-Geltink J, Kopp A, Sibley LM. Capitation and enhanced fee-for-service models for primary care reform: a population-based evaluation. CMAJ 2009; 180: E72 81.

354 Family Practice The International Journal for Research in Primary Care 27 Jones NF, Marshall R. Evaluation of an electronic general-practitioner-based syndromic surveillance system Auckland, New Zealand, 2000-2001. MMWR Morb Mortal Wkly Rep 2004; 53 Suppl: 173 8. 28 Mayo F, Marsland D, Wood M et al. Denominator definition by the utilization correction factor method. Fam Pract 1986; 3: 184 91. 29 Canadian community health survey 2008: http://www.statcan. gc.ca/cgi-bin/imdb/p2sv.pl?function=getsurvey&sdds=322 6&lang=en&db=imdb&adm=8&dis=2. (accessed on 14 April 2010). 30 Sibley LM, Moineddin R, Agha MM, Glazier RH. Risk adjustment using administrative data-based and survey-derived methods for explaining physician utilization. Med Care 2010; 48: 175 82. 31 Glazier RH, Agha MM, Moineddin R, Sibley LM. Universal health insurance and equity in primary care and specialist office visits: a population-based study. Ann Fam Med 2009; 7: 396 405. 32 Reid R, Bogdanovic B, Roos N, Black C, MacWilliam L, Menec V. Do Some physician Groups See Sicker Patients Than Others? Implications for Primary Care Policy in Manitoba. Winnipeg: Department of Community Health Sciences, University of Manitoba, 2001. 33 Canadian Community Health Survey Annual Component. Ottawa: Statistics Canada, 2011 [updated July 192012; cited 2012 July 19]. http://www23.statcan.gc.ca:81/imdb/p2sv.pl?function= getsurvey&sdds=3226&lang=en&db=imdb&adm=8&dis=2. 34 Anderson JE, Gancher WA, Bell PW. Validation of the patient roster in a primary care practice. Health Serv Res 1985; 20: 301 14.