Exploring Quebec s Billing Data (RAMQ) as a Potential Source for Percutaneous Coronary Intervention (PCI) Data for the Calculation of the PCI Rate Indicator Prepared by the Tertiary Cardiology Evaluation Unit at the Institut national d excellence en santé et en services sociaux and the Canadian Institute for Health Information
Project Team Autors Édition Manager Coordinator Kinga David (ICIS), Yongling Xiao (INESSS) Yana Gurevich (ICIS) Peter Bogaty (INESSS, Hôpital Laval, Qc) Laurie Lambert (INESSS) Diane Guilbault Véronique Baril Published by the Direction des communications et du transfert de connaissances Legal deposit Bibliothèque et Archives nationales du Québec, 2012 ISBN 978-2-550-68657-6 (PDF) Gouvernement du Québec, 2013. This document may be reproduced in whole or in part provided that the source is cited. To cite this document: Institut national d excellence en santé et services sociaux (INESSS). Exploring Quebec s Billing Data (RAMQ) as a Potential Source for Percutaneous Coronary Intervention (PCI) Data for the Calculation of the PCI Rate Indicator. Montréal, Qc : INESSS ; 2013. Informations Institut national d excellence en santé et en services sociaux (INESSS) 2535, boulevard Laurier, 5 e étage Québec (Québec) G1V 4M3 Téléphone : 418-643-1339 Télécopieur : 418-646-8349 inesss.qc.ca inesss@inesss.qc.ca 2021, avenue Union, bureau 10.083, Montréal (Québec) H3A 2S9 Téléphone : 514-873-2563 Télécopieur : 514-873-1369
Table of contents Introduction... 1 Background... 2 Data sources and methods.... 3 Results... 5 Conclusions et recommendations.... 8 References... 9
Introduction The purpose of this report is to summarize the collaborative work of the Tertiary Cardiology Evaluation Unit of the Institut national d excellence en santé et en services sociaux (INESSS) and the Canadian Institute for Health Information (CIHI) that assessed the appropriateness of using Quebec s physician claims database (Régie de l assurance maladie du Québec, or RAMQ) for the calculation of the percutaneous coronary intervention (PCI) rate in Quebec. 1
Background In recent years, PCI rates have been increasing, and important variations in rates of PCI have been observed across jurisdictions [CIHI, 2010]. CIHI has been reporting a PCI indicator since 2006 as part of its annual Health Indicators report, using data from four different clinical administrative data holdings: the Discharge Abstract Database (DAD), National Ambulatory Care Reporting System (NACRS), Alberta Ambulatory Care Reporting System (AACRS) and MED-ÉCHO. In March 2011, a collaborative project by INESSS and CIHI sought to determine whether Quebec s Hospital Inpatient and Day Surgery Database (MED-ÉCHO) was an appropriate data source to count PCIs [INESSS, 2011]. The study concluded that it was not appropriate to use MED-ÉCHO to count the number of PCIs performed in Quebec due to a lack of uniformity in provincial admitting practices for PCI. In 2007 2008, 26% of PCI procedures documented in Quebec s physician claims database were not reported to MED-ÉCHO by the performing PCI centre, and 14% of PCIs were not reported to MED-ÉCHO by any hospital. Hospital reporting practices for PCIs varied widely across Quebec s 15 PCI centres and in those non-pci performing centres that referred patients for PCI. In the absence of a comprehensive PCI registry in Quebec, RAMQ data was considered as a gold standard for this previous analysis. RAMQ is designed for the purposes of physician billing. Given the limited patient documentation available in RAMQ data, the Quebec MSSS requested that CIHI and INESSS work together to assess the appropriateness of RAMQ billing data for the calculation of the PCI rate and evaluate the capture and comprehensiveness of data elements required for the PCI indicator. 2
Data Sources and Methods In the absence of a province-wide PCI registry, it was determined that chart review data from acute myocardial infarction (AMI) patients would be used as the gold standard to assess the appropriateness of using RAMQ data for counting the number of PCIs in Quebec. A chart review was conducted for AMI patients discharged between October 1, 2008, and March 31, 2009, as part of a field evaluation conducted by the Tertiary Cardiology Evaluation Unit (UECT) and mandated by Quebec s MSSS [Lambert et al., 2012; 2010]. The evaluation included all acute care Quebec hospitals (n = 82) that had treated at least 30 AMI patients in the preceding year. These 82 hospitals treat more than 95% of all AMIs occurring in Quebec and are distributed across 16 of Quebec s 18 administrative health care regions. All AMI patients who belonged to one of the following groups were included in this validation study:1 Those who were transferred to a PCI centre or to a catheterization lab (for patients with a direct admission to a PCI centre) within four hours of triage and underwent a PCI; Those who were transferred to a PCI centre or to a catheterization lab (for patients with a direct admission to a PCI centre) within four hours but did not undergo PCI; or Those who were not sent to a PCI centre or to a catheterization lab during the episode of care. As part of the field evaluation s original data analysis plan, RAMQ billing data was linked to the chart review data using the encrypted RAMQ ID and the index date. Data linkage was performed by RAMQ according to methods approved by the Commission d accès à l information du Québec. The index date was defined, based on the chart review data, as either the date when the PCI was conducted for those patients who did have a PCI or the date when the patient presented to the first emergency room for those patients who did not have a PCI. The RAMQ billing codes used to identify a PCI were 00632 (Angioplasty portion) and 00662 (Angioradiography and angioplasty combined). For a given patient visit, all PCIs within a specific time frame (index date ± t days) were included. First, a successful linkage rate was calculated, which was defined as the proportion of patients in the chart review data who were found in RAMQ. Second, to assess the capture of PCI procedures in RAMQ data, the following statistics were calculated, using the chart review data as the gold standard: True positive (TP): the number of patients who actually received PCI and were reported as having PCI in RAMQ False negative (FN): the number of patients who actually received PCI but were not reported as having PCI in RAMQ False positive (FP): the number of patients who actually did not receive PCI but were reported as having PCI in RAMQ True negative (TN): the number of patients who actually did not receive PCI and were not reported as having PCI in RAMQ 1 A patient may be admitted more than once for AMI during the study period; thus the same patient may appear two or more times in the data. Different episodes of care for the same patient are treated as independent episodes in our data analysis. In this document, the word patient refers to a patient visit. 3
The sensitivity, specificity and positive predictive value were also calculated, as follows: Sensitivity: the proportion of patients who received PCI (according to chart review) and were reported as having PCI in RAMQ, among all patients who received PCI according to chart review TP/(TP + FN) Specificity: the proportion of patients who did not receive PCI (according to chart review) and were not reported as having PCI in RAMQ, among all patients who did not receive PCI according to chart review TN/(FP + TN) Positive predictive value: the proportion of patients who received PCI (according to chart review) and were reported as having PCI in RAMQ, among all patients who were reported as having PCI in RAMQ TP/(TP + FP) The above analyses were conducted by linking chart review data to RAMQ billing data using different time windows to identify billings for PCI in RAMQ: Index date only Index date ± 1 day Index date ± 2 days Index date ± 3 days Index date ± 7 days Finally, a separate analysis was conducted to examine the distribution of patient age, sex and region of residence among PCI records in RAMQ in order to better understand the capture of the data elements required for the calculation of the PCI indicator. The proportion of missing values for those variables was evaluated. This was done for three fiscal years (2005 2006, 2006 2007 and 2007 2008). 4
Results A total of 4,241 AMI patients were ascertained for the chart review. Of these, 1,548 patients underwent a PCI and a total of 2,693 patients did not undergo a PCI during their episodes of care. Table 1: AMI Patients Ascertained for Chart Review Patient Group Those who were transferred to a PCI centre or to a catheterization lab (for patients with a direct admission to a PCI centre) within four hours of triage and underwent a PCI Those who were transferred to a PCI centre or to a catheterization lab (for patients with a direct admission to a PCI centre) within four hours but did not undergo PCI Those who were not sent to a PCI centre or to a catheterization lab during the episode of care N Percentage of Total (N = 4,241) 1 548 36.5 % 387 9.1 % 2 306 54.4 % Almost all patients in the chart review data were successfully linked to the RAMQ billing database. When examining the successful linkage rates within different time frames, 97% of all AMI patients in the chart review cohort were linked to RAMQ using the index date. Using the one-day time window, almost all patients were found to have a RAMQ billing record (99.6%). Increasing the length of the time window to more than one day only slightly increased the successful linkage rate, from 99.6% to 99.8%. Among patients who did not undergo a PCI, the successful linkage rate was 95.4% when using the exact index date; it increased to 99.6% when the time frame was expanded to one day. Table 2: Successful Linkage Rates Within Different Time Frames Time Window (Days) All AMI Chart Review Patients (N = 4,241) Found in RAMQ Linkage Rate (%) PCI Chart Review Patients (N = 1,548) Found in RAMQ Linkage Rate (%) Non-PCI Chart Review Patients (N = 2,693) Found in RAMQ Linkage Rate (%) 0 4,112 97.0 1,542 99.6 2,570 95.4 1 4,226 99.6 1,543 99.7 2,683 99.6 2 4,231 99.8 1,543 99.7 2,688 99.8 3 4,231 99.8 1,543 99.7 2,688 99.8 7 4,232 99.8 1,543 99.7 2,689 99.9 The sensitivity, specificity and positive predictive value (PPV) were calculated to assess the appropriateness of RAMQ data for identifying a PCI. Measures of sensitivity ranged from 95.7% to 98.6% across the different time windows; specificity ranged from 90.5% to 98.5% and PPV ranged from 94.1% to 97.5%. The number of true positive (TP) PCIs remained the same when the time window was increased from one to three days. Most PCI billings were found within one day of the date of the PCI. While using the seven-day time window captured more true positive PCIs, it also significantly increased the number of false positive (FP) PCIs. Thus increasing the 5
time window by more than one day increases sensitivity but substantially decreases specificity and PPV. Overall, the RAMQ billing database was found to be an accurate data source to identify PCI among patients hospitalized for AMI in Quebec. PCIs are most likely to be billed within one day of the PCI date. Table 3: Sensitivity, Specificity and PPV for Identifying PCI Using RAMQ Database, Compared With Chart Review Data: Different Time Windows Considered PCI Date ± Time Window (Days) TP FN FP VN Sensitivity (95% CI) 95.7 (94.6 à 96.7) 98.2 (97.5 à 98.9) 98.2 (97.5 à 98.9) 98.3 (97.6 à 98.9) 98.6 (98.0 à 99.2) PPV (95% CI) 97.5 (96.7 à 98.3) 96.8 (95.9 à 97.7) 94.9 (93.9 à 96.0) 93.5 (92.3 à 94.7) 90.5 (89.1 à 91.9) Specificity (95% CI) 98.5 (98.1 à 99.0) 98.1 (97.6 à 98.7) 97.0 (96.3 à 97.6) 96.1 (95.4 à 96.8) 94.1 (93.2 à 94.9) 0 1,475 67 38 2,532 1 1,515 28 50 2,633 2 1,515 28 81 2,607 3 1,516 27 105 2,583 7 1,521 22 160 2,529 The appropriateness of the RAMQ database for identifying PCIs in subsets of the study population was also examined. The false positive rate for the two groups of non-pci patients was calculated. It was defined as the proportion of patients who actually did not receive a PCI but who had a billing for a PCI in RAMQ (that is, FP [FP + TN]). Overall, the RAMQ database accurately documented PCI status for those who were never sent to a catheterization lab, with false positive rates ranging from 0.2% to 2.4%. False positive rates were higher among patients who were sent to a catheterization lab in four hours but did not undergo a PCI (9.1% to 26.7%). Table 4: Accuracy of RAMQ Database for Identifying PCI in Two True Non-PCI Groups Sent to a Catheterization Group Laboratory, but PCI Not Performed (N = 387) Not Sent to a Catheterization Laboratory (N = 2,306) Time Window False Positive False Positive Found in RAMQ Found in RAMQ (Days) (%) (%) 0 373 34 (9.1%) 2,197 4 (0.2%) 1 386 42 (10.9%) 2,297 8 (0.4%) 2 387 62 (16.0%) 2,301 19 (0.9%) 3 387 73 (18.9%) 2,301 32 (1.4%) 7 387 104 (26.7%) 2,302 56 (2.4%) Further analysis indicated that a substantial proportion of false positive patients underwent arterial puncture in the catheterization laboratory. Thus one reason for the higher false positive rate might be that attempted PCIs are sometimes billed as a PCI. For example, within the same day as the index date, 85% of false positive patients underwent arterial puncture in the catheterization lab. Within one day (before/after) of the index date, 74% of those false positives underwent arterial puncture in the catheterization lab. 6
The multiplicity of PCI billings in the RAMQ database was also examined. The multiplicity of PCI billings might be explained by the involvement of different physicians for the same PCI service and thus all of them billed for the service or by the multiple billing of a single physician for a PCI service during the same time window. Since the chart review data recorded only the first PCI for each episode of care, it was not possible to ascertain from chart review data whether a patient had multiple PCIs within the same episode of care. Table 5: Multiplicity of PCI Billings in RAMQ Database for Different Time Windows Time Window PCI in Chart Review Number of PCI Billings PCI in RAMQ (Days) Data 1 2 3 4 0 1,548 1,511 1,499 11 3 1 1,548 1,565 1,525 35 5 2 1,548 1,596 1,528 62 5 1 3 1,548 1,621 1,527 85 8 1 7 1,548 1,681 1,555 117 8 1 Finally, a separate analysis to examine the proportion of missing values of patient age, sex and region among all PCIs in RAMQ was performed in order to better understand the capture and comprehensiveness of data elements required for the calculation of the PCI indicator. Between 2005 2006 and 2007 2008, on average, less than 0.7% of patient region of residence values were missing and no age or sex values were missing. Thus, the results indicated that missing values are a non-issue. 7
Conclusions and recommendations Overall, the RAMQ billing database was found to be an accurate and reliable data source for identifying PCI among patients hospitalized for AMI in Quebec. Using AMI patient chart review data as the gold standard, almost all patients were found to have a PCI billing record (99.6%) within the one-day time window. PCIs were also most likely to be billed within one day of the PCI date. Further, measures of sensitivity, specificity and PPV were all favourable (sensitivity 98.2, specificity 98.1 and PPV 96.8 for one-day time window). There were also very few missing values for patient region of residence, and none missing for age and sex information. These findings suggest that it is appropriate to use the RAMQ billing database to count the number of PCIs performed in Quebec. It is therefore recommended to proceed with the calculation of the PCI rate for Quebec and its 18 health regions using RAMQ data and review indicator results in terms of face validity and comparability to previously published PCI results in Quebec and other jurisdictions (for example, Ontario). 8
References Canadian Institute for Health Information (CIHI). Health indicators 2010. Ottawa, ON: CIHI; 2010. Available at: https://secure.cihi.ca/free_products/healthindicators2010_en.pdf. Institut national d excellence en santé et services sociaux (INESSS). Le fichier MED-ÉCHO du Québec est-il la source de données appropriée pour dénombrer les interventions coronariennes percutanées (ICP)? Préparé à l intention du ministère de la Santé et des Services sociaux (MSSS) du Québec. Préparé par l Institut national d excellence en santé et en services sociaux (INESSS) et l Institut canadien d information sur la santé (ICIS). Montréal, Qc: INESSS; 2011. Available at: http://www.inesss.qc.ca/fileadmin/doc/inesss/cardio/med-echo_inesss.pdf. Lambert LJ, Xiao Y, Rinfret S, L Allier P, Maire S, Kouz S, et al. Short term mortality in ST-elevation myocardial infarction (STEMI) patients treated with primary angioplasty (PPCI): A provincewide, systematic field evaluation. Can J Cardiol 2012;28(5 Suppl):S379-S380 [abstract 728]. Lambert L, Brown K, Segal E, Brophy J, Rodes-Cabau J, Bogaty P. Association between timeliness of reperfusion therapy and clinical outcomes in ST-elevation myocardial infarction. JAMA 2010;303(21):2148-55. 9