Using your Race/Ethnicity Data Quality Databooks Nov. 29, 2016 Jennifer Rienks, PhD; Linda Remy, PhD; Adrienne Shatara, MPH Family Health Outcomes Project, UCSF
Background Family Health Outcomes project (FHOP) analyzes: birth certificate data Hospital patient discharge data emergency department data Develop 12-year data summary spreadsheets on key maternal, child, and adolescent health (MCAH) outcomes for California s 61 local health jurisdictions (LHJs). Data also analyzed by race/ethnicity to enable LHJs to identify and address disparities Recently noticed alarming increase in undefined (missing, unknown, or other) race/ethnicity
Background Systematic collection of race, ethnicity, language and birthplace important for: monitoring and improving hospital practices, population- based studies of health equity, 1,2 and studies of quality care and comparative and cost effectiveness 3 Availability of race and ethnicity (R/E) central to reliably measuring population health disparities CDC data quality standard = no more than 1% of records undefined (other, unknown); national median ranging from 0.4% to 0.5% CA above 1% for many years Growing problem directly undermines the mission of public health organizations to reduce race/ethnic disparities Problem magnified in longitudinal research
Background Lack of specific guidelines for data collection has contributed to inconsistency in reporting, variable validity across R/E groups, and low completeness 1 CA Hospitals required to report R/E data to Office for Statewide Health Planning and Development (OSHPD) Data auditing at time of submission but only general benchmarks for data completeness and consistency are examined 2 OSHPD occasionally exempts a hospital from having to submit R/E data for a specified period of time In general, patient demographic data collected by hospitals is inconsistent, inaccurate, incomplete, fragmented, & collected in silos 3, 4 In 2011, Gomez et. al. 5 surveyed of 56% of hospital in CA (n=205) on hospital practices in collection of patient race, ethnicity, and language data Despite near universal collection of language and race/ethnicity, variability in hospital policies may compromise quality and consistency of data Data quality issues present significant barrier to understanding magnitude of health disparities
Methods DATA SOURCES: California Birth Certificate data in the Birth Statistical Master File (BSMF) from Center for Health Statistics and Informatics Patient Discharge Data (PDD) and Emergency Department Data (EDD) from Office of Statewide Health Planning and Development BSMF and PDD available for the period 2002-2013. EDD available for period 2005-2013. Data quality rates were calculated per 100 births or discharges of females age 15 to 44
Methods (cont.) DATA QUALITY: Calculated separately for race and ethnicity, and then for a combined race/ethnicity (R/E) variable. Birth Certificate Data: R/E is undefined when ethnicity is NOT Hispanic and Mother's RACE1 is other, unknown, or missing Hospital Patient Discharge Data and Emergency Department Data: PPD and EDD Data: R/E is undefined when ethnicity is NOT Hispanic and Race is other, unknown or missing, where other includes natives of Central and South American and multi-race
Methods How hospital R/E is coded Data is collected separately for R/E with ethnicity collected first For ethnicity, data should be collected on whether or not a person is of Hispanic or Latino culture or origin Sample patient demographic questionnaire: 1. Are you of Hispanic, Latino, or Spanish origin? (Mark ONE box.) Yes (specify (e.g. Mexican, Puerto Rican, Cuban, etc.)) No, not Hispanic, Latino, or Spanish origin 2. What is your race? (Mark one or more boxes.) White/Caucasian Asian Black/African American Native Hawaiian or Other Pacific Islander American Indian/Alaska Native Some other race: (specify) Prefer not to answer 3. IF MORE THAN ONE RACE (Question #2) IS CHECKED: Do you identify with any one race in particular? Yes No (specify)
R/E Data Quality in California: The BIG PICTURE
Results: Rates of Undefined R/E data quality in CA over time 12 10 8 6 4 Hosp. Dis. Birth Cert. Emer. Dept. CDC Stand. 2 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
CA hospitals and undefined R/E Patient Discharge Data 12-year averages for undefined R/E 332 (82%) hospitals exceed CDC s 1% standard 28 (6%) hospitals > 10% 87 (18%) hospitals < 1% Birth Certificate Data 12-year averages for undefined R/E 64 (22%) hospitals exceed CDC s 1% standard 16 (5%) hospitals > 5% 230 (77%) hospitals < 1% Emergency Dept. Data 12-year averages for undefined R/E 289 (87%) hospitals exceed CDC s 1% standard 14 (4%) hospitals > 20% 46 (14%) hospitals < 1%
CA Rates of Undefined Ethnicity, Race, and combined R/E over time in PPD
R/E Data Quality Databooks FHOP has produced 3 data quality databooks for each LHJ: Hospital Patient Discharge Data 2002-2013 (files named PRACEQ2013_XX) Hospital Emergency Department Data 2007-2013 (files named ERACEQ2013_XX) Birth Certificate Data 2002-2013 (files name BRACEQ2013_XX) R/E data quality databooks available on Fhop s website: http://fhop.ucsf.edu/raceethnic-data-quality-databooks Data quality databooks based on place of residence (NOT place of occurrence)
R/E Data Quality Databooks Tabs Local numerators (number of records missing race, ethnicity, and race/ethnicity) and denominators (total number of records) State numerators (number of records missing race, ethnicity, and race/ethnicity) and denominators (total number of records) Race Rates Race Graphs Ethnicity Rates Ethnicity Graphs Race/Ethnicity rate Race/Ethnicity graphs
Tab: Local Numerators and Denominators
Tab: Local Race Rates
Tab: Local Race Graphs From 2006-2013, significant upward trend in undefined race in BC
Tab: Local Hispanic Rates
Tab: Local Hispanic Graphs From 2008-2013, significant upward trend in undefined Hispanic ethnicity in BC
Tab: Race Ethnicity Rates
Tab: Race Ethnicity Graph From 2002-2013, significant upward trend in combined undefined Race ethnicity in BC
Implications for your LHJ Databooks If you have high rates of undefined R/E, the quality of your R/E data suffers because there are fewer cases with which to calculate rates. Fewer cases to calculate rates can result in: The need to pool the data over the years. Instead of yearly rates, you will get 2-year, 3-year, or 4-year rates Reduction the statistical power to identify significant changes in rates and trends Loss of ability to generate reliable rates by race/ethnicity, and identify changes in trends over time. This means that it will be harder to monitor the health of smaller R/E groups and identify disparities
Optional: Assessing Data quality at the hospital level If you have high rates of undefined R/E, see hospital level data quality spreadsheets available at: Hospital R/E Hospital level R/E data quality spreadsheets based on place of occurrence (NOT place of residence) 3 Hospital Level R/E data quality spreadsheets for 2002-2013 Patient Discharge data (PRACEHQ) Emergency Department data (ERACEHQ) Birth Certificate data (BRACEHQ) Often there is significant variation in R/E data quality within the SAME hospital
12-year Hospital Averages (2002-2013) for undefined R/E in San Francisco Birth Emer. Hosp. Cert. Dept. Discharge Kaiser Foundation Hospital - San Francisco 0.8 14.2 4.8 Laguna Honda Hospital and Rehabilitation Center 3.3 Langley Porter Psychiatric Institute 5.5 CA Pacific Medical Center - Pacific Campus 0.5 3.7 1.2 San Francisco General Hospital 0.1 22.2 2.4 St. Francis Memorial Hospital 6.4 11.5 CA Pacific Medical Center - St. Luke's Campus 1.1 0.1 1.0 St. Mary's Medical Center, San Francisco 4.5 17.6 UCSF Medical Center 0.5 13.7 12.4 Chinese Hospital 1.1 0.2 San Francisco County Overall (by occurrence) 1.0 10.0 5.0
Rates of missing R/E data quality at UCSF Med. Center over time
Hospitals with highest rates of undefined R/E in Birth Cert. 2002-2013 Ave. % Missing R/E 12 year 2002-2004 2011-2013 County Hospital Santa Clara Lucile Packard Children's Hospital - Stanford 57,736 23.4 2.6 49.5 9 9 Alameda Kaiser Foundation Hospital - Oakland/Richmond 21,833 20.0 2.5 38.6 9 9 San Diego Sharp Memorial Hospital 97,283 17.8 6.6 6.0 11 9 San Diego Grossmont Hospital 41,475 13.5 4.7 19.6 10 10 Los Angeles Henry Mayo Newhall Memorial Hospital 15,105 12.2 0.3 33.7 7 6 Contra Costa San Ramon Regional Med. Center 9,584 11.1 12.1 0.5 8 7 Santa Clara El Camino Hospital 61,836 9.4 14.0 3.2 7 5 Alameda Valleycare Medical Center 16,942 8.2 3.4 8.2 10 7 Los Angeles Ronald Reagan UCLA Medical Center 23,776 7.7 10.1 9.0 11 9 Fresno Fresno 14,453 7.2 0.1 6.5 6 5 Kaiser Foundation Hospital - Contra Costa Walnut Creek 46,328 6.9 19.8 2.0 3 3 Total Discharges Total # of Years Over 3% Over 5%
Assessing hospital R/E data quality for quality improvement efforts Focus on SIZE of the problem and of the hospital: Hospitals with the largest number of discharges AND with highest rates of undefined R/E: Highest 12-year averages over 5% for multiple years over 3% for multiple years AND focus on trends Is R/E data quality getting better or worse? Examine hospital R/E data quality from all sources Birth Certificate Patient Discharge Death Certificate
Hospital level data quality example Total Discharges Ave. Rate undefined R/E 2002-2013 Undefined R./E 2002-2004 Undefined R./E 2011-2013 # of years undefined rate higher than 3% # of years undefined rate higher than 5% TOTD TOTP SOPP EOPP TOTH3 TOTH5 419,891 7 8 7 12 12 010735 Alameda Hospital 2,772 22 12 24 12 12 010776 Childrens Hospital and Research Center at Oakland 7,114 14 9 18 12 12 010844 Alta Bates Summit Medical Center - Herrick Campus 11,495 14 9 17 12 12 014226 Telecare Willow Rock Center 1,294 12 14 7 5 010739 Alta Bates Summit Medical Center - Alta Bates Campus 111,088 11 14 11 12 12 013687 Mpi Chemical Dependency Recovery Hospital 1,437 13 18 9 12 11 010987 Washington Hospital - Fremont 41,033 3 0 8 4 2 010846 Alameda County Medical Center - Highland Campus 42,602 6 6 8 12 10 014050 Valleycare Medical Center 24,255 4 2 8 8 2 010967 St. Rose Hospital 21,996 4 3 6 7 1 010937 Alta Bates Summit Medical Center - Summit Campus - Hawthorne 22,911 10 12 6 12 12 010887 Kindred Hospital - San Francisco Bay Area 122 18 16 5 11 11 014207 Telecare Heritage Psychiatric Health Facility 3,067 3 5 3 2 013619 San Leandro Hospital 3,145 3 1 3 8 1 014326 Kaiser Foundation Hospital - Oakland/Richmond 38,495 6 11 2 7 5 010858 Kaiser Foundation Hospital - Hayward/Fremont 50,421 3 7 1 4 4 014233 Eden Medical Center 23,793 4 10 1 3 2 014034 Fremont Hospital 12,769 0 0 1 0 0 014113 S.T.A.R.S. - Psychiatric Health Facility 82 0 0 0 0
Rates of missing R/E data quality in Alameda County over time (by residence) 25 20 15 10 Hosp. Dis. Birth Cert. Emer. Dept. CDC Stand. 5 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Implications for Identifying Local R/E Disparities Data quality varied significantly within county, over time, and by source. The most significant problems were in the Bay Area, San Diego, and Ventura counties Birth certificate indicators with self-contained denominators were the least affected Poor R/E data quality reduces ability to: Calculate reliable jurisdiction level rates for smaller race/ethnic Identify disparities Evaluate the statistical significance of changes in rates Identify and monitor trends
Discussion Potential reasons for increasing rates of undefined R/E in birth certificate, hospital discharge, and Emergency dept. data Rise in number or people who are multiracial and don t identify with one particular race Failure of hospital personnel to effectively collect R/E data Problems at hospital level with information technology (i.e. Electronic Health Record) How hospitals use R/E data 5 Ensuring availability of interpreter services - 36% Quality improvement/disease management programs - 36% Program/benefit design - 16% Marketing - 13%
Discussion Strategies with the most hospital support to improve quality and completeness of patient information 1 : Collecting data at a patient s first visit Offering routine staff training, Incorporating questions into existing admissions forms Developing and enforcing of hospital policies regarding data collection Availability of a frequently asked questions and answers document for staff Strategies that are identified as most effective: Standardized forms Audit procedures
Recommendations to Improve Data collection and quality Why every patient should be asked about R/E 2 ALL patients should be asked about their race/ethnicity, and language Self-reporting is the most accurate source of information Self-reporting will increase consistent reporting within a health care institution Patients are more likely to select the same categories to describe themselves over time than staff who are assuming or guessing Best way to ask based on research 2 : In order to guarantee that all patients receive the highest quality of care and to ensure the best services possible, we are asking all patients about their race, ethnicity, and language. Standardize race and ethnic reporting across data sources (birth certificates, hospital discharge, and emergency department)
References 1. Andrews RM. Race and ethnicity reporting in statewide hospital data: progress and future challenges in a key resource for local and state monitoring of health disparities. J Public Health Manag Pract. 2011 Mar Apr; 17(2): 167 73. http://dx.doi.org/10.1097/phh.0b013e3181f5426c 2. Office of Statewide Hospital Planning and Development (OSHPD). MirCAL Edit Flag Description Guide, Inpatient Data 2012. Sacramento, CA: OSHPD, 2012. Available at: http://www.oshpd.ca.gov/hid/mircal/text_pdfs/manualsguides/ipeditflagdescguide.pdf 3. Hasnain- Wynia R, Baker DW. Obtaining data on patient race, ethnicity, and primary language in health care organizations: current challenges and proposed solutions. Health Serv Res. 2006 Aug; 41(4 Pt 1): 1501 18. 4. Higgins PC, Taylor EF. Measuring racial and ethnic disparities in health care: efforts to improve data collection. Washington, DC: Mathematica Policy Research, 2009. 5. Gomez SL, Le GM, West DW, et al. Hospital policy and practice regarding the collection of data on race, ethnicity, and birthplace. Am J Public Health. 2013 Oct; 93(10): 1685 8. http://dx.doi.org/10.2105/ajph.93.10.1685 6. Holland AT, Wong EC, Lauderdale DS, et al. Spectrum of cardiovascular diseases in Asian- American racial/ethnic subgroups. Ann Epidemiol. 2011 Aug; 21(8): 608 14. http://dx.doi.org/10.1016/j.annepidem.2011.04.004 7. Quality Alliance Steering Committee. Identifying Racial and Ethnic Disparities in Hospital Quality: Montgomery County Hospital Care Equity Initiative 2010. Washington, DC: Quality Alliance Steering Committee, 2010. 8. Gomez SL, Lichtensztajn DY, Parikh P, Hasnain-Wynia R, Ponce N, Zingmond D. "Hospital practices in the collection of patient race, ethnicity, and language data: a statewide survey, California, 2011." Journal of Health Care for the Poor and Underserved, 2014. 9. Hasnain Wynia, R. Race, Ethnicity, and Language Data Collection: Nuts and Bolts Northwestern University, Feinberg School of Medicine. https://www.hcupus.ahrq.gov/datainnovations/raceethnicitytoolkit/ca11.pdf
Contact Information Jennifer Rienks, PhD Family Health Outcomes Project University of California, San Francisco 500 Parnassus Ave., MUE-313 San Francisco, CA 94143-0900 Phone: 415-476-5288 Email: Jennifer.Rienks@ucsf.edu Web site: http://fhop.ucsf.edu