Appendix: Data Sources and Methodology

Appendix: Data Sources and Methodology This document explains the data sources and methodology used in Patterns of Emergency Department Utilization in New York City, 2008 and in an accompanying issue brief, Time and Again: Frequent Users of Emergency Department Services in New York City. Both reports are available on the United Hospital Fund s website, www.uhfnyc.org. Data Sources This report uses four sources of data: 1) the SPARCS ED visit dataset, 2) hospital cost reports, 3) Community Health Profiles prepared by the New York City Department of Health and Mental Hygiene (NYCDOHMH), and 4) the SPARCS hospital inpatient dataset. SPARCS ED Visit Dataset This report examines all-payer emergency department (ED) utilization in 2008 for treat-andrelease visits (ED visits that did not result in hospitalization) within the Statewide Planning and Research Cooperative System (SPARCS) dataset. SPARCS is a comprehensive data reporting system mandatory for all hospitals within New York State. SPARCS collects patient level data on specific characteristics related to each hospital discharge, ambulatory surgery, and, starting in 2003, emergency department (ED) visits in New York State. The dataset includes only ED visits that did not result in hospital admission. ED visits that resulted in admission can be identified in SPARCS hospital discharge datasets, but we did not analyze these visits in our study. We also left out ED visits that took place in specialty hospitals (Manhattan Eye, Ear, & Throat Hospital; Memorial Hospital for Cancer and Allied Diseases; and New York Eye and Ear Infirmary). Our study explores only ED visits by New York City residents to New York State hospitals, not all ED visits occurring at hospitals located in NYC. We deleted the overflow records (records with sequence number greater than one). They have been used in SPARCS since 1994 if more than five UB-92 Accommodation Codes or more than 20 Ancillary Services Codes were reported for a patient stay. According to a SPARCS representative, if there are multiple records for a particular patient the only difference between the records is in the Accommodation and/or Ancillary Services codes. All other data elements for the same patient are repeated, that is, all diagnoses, procedures codes, etc., are the same for each record. Overflow records account for about 1 percent of all records and were deleted. In addition, we excluded patients with Emergency Department Indicator A for Ambulatory Surgery from Emergency Department or blank for Ambulatory Surgery only. We also decided to exclude patients transferred to a short-term general hospital for inpatient care, designated cancer center or children s hospital. United Hospital Fund, September 2012 1

Most importantly, the dataset includes patient identifiers, permitting the analysis of not only ED visits, but also individual patients inpatient hospital use over time. Hospital Cost Reports Prior to the release of SPARCS ED data, the only available data on ED visits in New York City were reported in hospitals Institutional Cost Reports (ICRs) that are filed annually with the New York State Department of Health (NYSDOH). ED visit data in ICRs include only total counts of ED visits and counts of ED visits not resulting in admission by the hospital or hospital system, by payer. Because patient origin data are not available in ICRs, ED visits can be identified only by hospital location and not by patient residence. For each category of ED visit (admit and treat-and-release), payer class information is also available in ICRs. We used ICR data in this report to assess the completeness and reliability of SPARCS ED visit data by comparing visit counts and payer mix by hospital from both sources. Community Health Profiles The Community Health Profiles prepared by the NYCDOHMH provide a comprehensive set of population demographic/ses and health status indicators for each of the 42 UHF neighborhoods. Indicators of health status in the profiles are obtained from a community survey that was most recently conducted in 2009. We also used 2008 New York City Department of Health Population Estimates for neighborhoods. SPARCS Inpatient Dataset The patient-level SPARCS dataset for 2008 describes all inpatient services provided within New York State. This dataset includes patient characteristics, diagnoses, treatments, services, and payer classes. Most importantly, UHF had access to patient identifiers that linked inpatient data with SPARCS ED, which gave us a unique opportunity to compare inpatient and ED utilization of individual patients. United Hospital Fund, September 2012 2

Methodology Upweighting Process We first evaluated the SPARCS ED visit dataset to determine if it was sufficiently complete and reliable to conduct the study. To assess the completeness of the data, we compared counts of ED visits without admission by hospital in SPARCS to those reported in hospitals ICRs. Citywide, visits were underreported in SPARCS by 13 percent. 70 percent of hospitals had variances under 20 percent. Many researchers who work with SPARCS discharge data correct for underreporting by grossing up discharges by hospital to counts reported in ICRs. This method assumes that unreported discharges have the same characteristics (demographics, diagnoses, procedures) as reported discharges. If reported data are not representative, grossing up may magnify the effects of hospitals that have atypical patient populations (e.g., safety net hospitals with large concentrations of low income, HIV, mentally ill, and substance abuse patients have more underreporting than other types of hospitals). A second consideration, especially relevant to our study, was that ED visits in some key low-income neighborhoods would be significantly underreported if we did not gross up visits. To evaluate this possibility, we examined the patient characteristics (age, sex, clinical mix) in 2007 for hospitals with underreporting in 2008 of 20 percent or more. We found that patient characteristics in both years were similar for all of these hospitals. Thus, we decided to gross up or upweight ED visits for all hospitals to ICR counts. For two hospitals (Parkway and Caritas) that had much better reporting in other years, we substituted earlier data (2005 in the case of Parkway and 2006 data in the case of Caritas) in our 2008 dataset. Since 2008 ICR numbers were not available for Our Lady of Mercy and Peninsula we used 2007 figures for the upweights. We also decided to exclude specialty hospitals (Manhattan Eye, Ear, & Throat Hospital; Memorial Hospital for Cancer and Allied Diseases; and New York Eye and Ear Infirmary) and that minimum weight would be 1 (ignoring cases where ICR < SPARCS). We evaluated the reliability of data elements to be used in our study (patient ZIP code, patient demographics, payer class, and clinical diagnoses) through comparison with alternative data sources where they were available (e.g., payer class data in ICRs) or through tests of reasonableness where alternative data sources were not available. Our findings from this evaluation are summarized below. Duplicate Records When discovered, 9.6 percent of the records (294,094) were deleted based on: unique personal identifier (UPIDE), encrypted date of birth (DOBE), sex, date of admission, time of admission and primary diagnosis code. United Hospital Fund, September 2012 3

Linking Patients The dataset includes patient identifiers. We were able to identify individual patients using the ED by linking patients using UPIDE, DOBE, and sex. Race/Ethnicity Some values were missing or coded as other. Health and Hospitals Center (HHC) data were less complete than those of nonprofit hospitals. We obtained improved data from HHC, which we used to adjust for the race/ethnicity analysis. UHF Neighborhoods The 42 UHF neighborhoods consist of adjoining ZIP code areas with similar characteristics, designated to approximate New York City Community Planning Districts, and based on the demographic, economic, and social diversity found there. The assignment of ZIP code areas to neighborhoods, the decisions about which community planning districts were most appropriate to combine, and the delineation of neighborhoods were made by UHF staff in consultation with staff of the New York City Planning Commission and the New York City Health Systems Agency. Originally developed in 1982, this neighborhood listing was updated in 2002 to reflect sociodemographic changes. UHF Neighborhood 999 A small subset of records within the SPARCS dataset contain a NYC county code but do not link to a specific UHF neighborhood because they have an incomplete or missing 5-digit ZIP code. These data were pooled into a single category, coded 999, and account for 0.3 percent of people and 0.2 percent of all ED visits in the 2008 SPARCS dataset. ED Use Definitions As no uniform definition of frequent ED users exists, for the purposes of this analysis we categorized use in multiple ways. An ED user was any person with at least one ED visit in 2008. We also examined those with 2, 3, 4, and 5 or more visits in 2008. Super-users were those with serial use, who had 5 or more visits in each of three consecutive years (2006, 2007, and 2008). United Hospital Fund, September 2012 4

Movers To assign ED users with at least two visits residing in more than one neighborhood to one neighborhood we used the following decision rules: Patients with two visits who resided in two neighborhoods: one of the two neighborhoods was randomly selected. Patients with 3 or more visits: the most frequent neighborhood was used (mode). Where there was no mode, one of the neighborhoods was chosen at random. Clinical Data We used H-CUP Clinical Classifications Software (CCS) level two (about 140 categories) to analyze data on diagnoses and procedures. Payer Mix To determine the accuracy of the payer reported in the SPARCS ED dataset, we compared each hospital s payer mix as reported in SPARCS to its ICR for years 2005 through 2008. We believe the ICR is more accurate because it is submitted at the end of the year and is reviewed by an independent auditor. The SPARCS ED dataset has two data fields that identify payer: Source of Payment and Expected Principal Reimbursement. We examined hospital payer mix by comparing both SPARCS data fields to the ICR and determined that the Expected Principal Reimbursement field is more accurate (Figure 1). The percent of total admissions reported as Medicaid in the SPARCS ED Source of Payment field varied from the ICR by more than 50 percent at 28 of 44 hospitals. The Expected Principal Payment field for Medicaid varied from ICR data by more than 50 percent at only one hospital. The Expected Source of Payment field also proved more accurate with other payer classes. United Hospital Fund, September 2012 5

Figure 1. Hospital Payer Mix Reported in SPARCS and ICR, 2008 Medicare Medicaid Self-Pay All Other Number of hospitals 25% over/under Source of Payment 28 34 13 36 Expected Principal Payment 6 9 18 14 Number of hospitals 50% over/under Source of Payment 5 28 6 34 Expected Principal Payment 2 1 4 10 Total number of hospitals = 44 We found that in SPARCS data, hospitals tend to overcount Medicaid visits and undercount self-pay visits (Figure 2). Data accuracy improved by combining these two payer classes into one category (which we labeled Safety Net ), though this action came at the expense of losing some detail of the data. Figure 2. Payer Data: SPARCS EXPECTED and ICR Medicaid Self-Pay Medicaid + Self-Pay Number of hospitals 25% over/under 25% over 6 8 2 25% under 3 10 0 Number of hospitals 50% over/under 50% over 1 0 0 50% under 0 4 0 Total number of hospitals = 44 Hour of Discharge Six percent of our sample had a negative average length of stay (LOS) in the ED. After closely examining those records we decided that the negative numbers reflected times that should have been recorded for the next day, and we corrected those cases accordingly. We also decided to delete records with zero LOS (3 percent of our sample) since they were randomly distributed in terms of age and hospitals. Twenty-four hospitals (including HHC), representing 35 percent of ED visits, had the hour of discharge coded as 99 (unknown). Three hospitals (Lutheran, LI Jewish, and North General) United Hospital Fund, September 2012 6

had extremely high average LOS (10 hours or more), and two (Cabrini and LIJ Schneider s Children) reported inexplicably low visit volumes; we decided to exclude these from the analysis as these data are likely inaccurate. Our analysis of the average length of stay in the ER was limited to 36 voluntary hospitals in the city for which data were complete. Neighborhood Quartile Analysis We divided UHF neighborhoods by quartiles based on adjusted ED rates per 100 population. There are 42 UHF neighborhoods so we assigned 11 neighborhoods in the quartiles with the highest and lowest ED rates and 10 neighborhoods to the middle quartiles. Analysis of Variance (ANOVA) Analysis Analysis of Variance (ANOVA) is a powerful statistical test used to determine whether the means between two or more groups are equal (the null hypothesis) or different. We performed one-way ANOVA analyses to determine the association of multiple population and health system factors with ED use in neighborhood groupings characterized by low, medium, and high ED use rates. Statistically significant results indicate more difference between groups of neighborhoods than within ED user groups. United Hospital Fund, September 2012 7

Study Strengths and Limitations To our knowledge, this is one of few in-depth analysis of ED use employing mandated data reported to New York State s SPARCS system. Specifically, ED data reporting was mandated starting in 2003 and, as a result, hospitals have had the opportunity to address inaccuracies or incompleteness of reporting. However, as mentioned, some of these data were not accurately reported in SPARCS and as a result, we used appropriate statistical techniques to account for these inaccuracies. Our ability to examine small area variations in health services use by UHF neighborhood provides a unique opportunity to draw attention to potential gaps in care. In addition, our ability to link UHF neighborhood ED use with income and education level provides additional valuable context for this analysis. We were able to link SPARCS ED data to inpatient data, providing unique insight into the relationship between ED use and hospitalizations. In addition, we present some novel findings that show a possible link between ED use and unstable housing, which to date has been difficult to demonstrate using large administrative datasets. ED visits and other person-level characteristics were assigned to a neighborhood based on the person s reported ZIP code of residence. However, we cannot be certain that the health services use of those within specific UHF neighborhoods actually occurred within those neighborhoods. While portions of the study examined data over multiple years, the majority was based on a single year of data, 2008. United Hospital Fund, September 2012 8