Lessons Learned Reuse of EHR Data for Research and Quality Improvement Bonnie L. Westra, PhD, RN, FAAN Assistant Professor, Co-Director ICNP Center for Nursing Minimum Data Set Knowledge Discovery University of Minnesota, School of Nursing 4/29/2014 1
Contributors Note: It takes a team! Lynn Choromanski, MS, RN; Mary Dierich, PhD-C, RN; Madeleine Kerr, PhD, RN; Karen Monsen, PhD, RN; Kay Savik, MS; Fang Yu, PhD, RN 1 Genevieve Melton-Meaux, MD 2 Cristina Oancea, PhD-C 3 Debra Solomon, MSN, RN, CNP 4 John H. Holmes, PhD 5 Sanjoy Dey, BSc; Gang Fang, M.Sc; Michael Steinbach, PhD; Vipin Kumar, PhD 6 Karen Dorman Marek, PhD,RN, FAAN 7 1 UMN School of Nursing, 2 UMN, Medicine, 3 UMN, Public Health, 4 Fairview Lakes HomeCaring & Hospice, 5 University of Pennsylvania, 6 UMN Computer Science, 7 Arizona State University
Grant Support National Institute of Nursing Research (Grant #P20 NR008992; Center for Health Trajectory Research) University of Minnesota, Grant-In-Aid Digital Technology Initiative The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Nursing Research or the National Institutes of Health.
Purpose 1. Describe steps in the process for discovering new knowledge from reuse of EHR data 2. Examine data selection and data quality issues 3. Explain methods data preparation and transformation for developing knowledge from EHR data 4. Compare different ways of developing predictive models 5. Explore lessons learned from reuse of data for quality improvement
Knowledge Discovery Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI Magazine, pp. 37 54. http://www.kdnuggets.com/gpspubs/aimag-kdd-overview-1996-fayyad.pdf. P. 41 5
Start with a Question in Mind 1. Test the feasibility of abstracting, integrating, and comparing the effective use of the Omaha System data across multiple software vendors and home care agencies. 2. Discover predictors for various outcomes build evidence for best practices from clinician data a. Hospitalization b. Improvement urinary and bowel incontinence c. Pressure ulcers d. Improvement in ambulation e. Improvement in oral medication management 3. Compare methods of developing predictive models
Data Selection - EHRs Home health care data 2 software vendors Convenience sample - 15 Homecare agencies Primarily Midwest also East coast All open admissions in 2004 Initial Data 18,067 OASIS records 989,772 Omaha System interventions 3,199 patients (1 74 OASIS records/ patient) 65,000 medication records
OASIS data Selected Data Clinical record items Demographics & patient history Living arrangements/ supportive assistance Health status Functional status Emergent care use/ discharge Omaha System interventions Problems Environmental, Psychosocial, Physiological, Other Health Related Problems Category of Action (HTG, T&P, CM, S) Targets focus of intervention Medication data 8
Data Quality Issues Know the Strengths and Limitations of Your Data Documentation issues Consistency of processes for documenting Copy forward or copy/paste Incomplete/ inappropriate data in the database Rules for data collection Charting by exception Rules i.e. the Joint Commission, CMS, billing Database / data model Field type Relationship of fields how do you link data Patient outliers Data with too little variance
Data Preparation / Cleaning Data Cleaning Duplicate data Plausible responses Missing data Consistency checks Type of data numeric, character, text Creating episodes for prediction What data are collected at specific points in time Multiple episodes per patient Linking all data to episodes of care
Causes of Missing Data Charting by exception Skip patterns Alternative documentation processes Wrong patient incomplete data Patient discharged before next data collection point or dies System errors
Figure 1. OASIS Integumentary Skip Pattern Skin Lesion or Open Wound No Go to Respiratory Status Yes Pressure Ulcer No Stasis Ulcer No Surgical Sound Yes Yes Yes Number of Ulcers Stage of Most Problematic Ulcer Status of Most Problematic Ulcer Number of Ulcers Status of Most Problematic Ulcer Number of Wounds Status of Most Problematic Wound
Characteristics of Data and Plausibility (M0700) Ambulation/Locomotion: Ability to SAFELY walk, once in a standing position, or use a wheelchair, once in a seated position, on a variety of surfaces. 0. Able to independently walk on even and uneven surfaces and climb stairs with or without railings (i.e., needs no human assistance or assistive device). 1. Requires use of a device (e.g., cane, walker) to walk alone or requires human supervision or assistance to negotiate stairs or steps or uneven surfaces. 2. Able to walk only with the supervision or assistance of another person at all times. 3. Chairfast, unable to ambulate but is able to wheel self independently. 4. Chairfast, unable to ambulate and is unable to wheel self. 5. Bedfast, unable to ambulate or be up in a chair. UK Unknown If response > 6, not a valid response If UK or NA, then data displays as character vs numeric
Inconsistency Checks
Distribution of Responses Skewed Data Ability to Transfer 2000 1500 1000 500 0 1 2 3 4 Little Variance Race/ Ethnicity 0 Total 5 100 80 60 40 20 0 Caucasian Non-caucasian
Heterogeneity of Population The goal is to Have an interpretable model Increase generalizability to a specific population Considerations Date range Type of population Age groups Location Length of stay Severity of illness Latent class analysis
Managing Data Characteristics
Other Data Preparation Steps OASIS data matched Start or Resumption of Care with Discharge Assessment Matched interventions by date to episodes Matched prescribed and over the counter medications by date to create a count for the number of unique medications Selected episodes appropriate for the outcome Improvement Oral Medication Management Only those who could improve or had a problem on admission
Episodes of Care 20
Transformation Converting data from one format to another Reasons Data reduction Format for to meeting assumptions for analyses Increase interpretability of results Decrease chaos
Transformation Clinical Classification Software Primary diagnoses and then reduced into 51 smaller groups within 11 major categories Charlson Index of Comorbidity Additional medical diagnoses Interventions Theoretically grouped into 23 categories Scales Created indicator variables (dummy codes) For non-normally distributed data e. g. Level of anxiety 3 levels, reference is No Anxiety
Clinical Classification Software Data reduction strategy for ICD diagnoses and procedures used experts Example: CCS 108 - CHF '4280 ' '4281 ' '42820' '42821' '42822' '42823' '42830' '42831' '42832' '42833' '42840' '42841' '42842' '42843' '4289 11 51 Groups 13,000 Dx http://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp http://www.nursing.umn.edu/icnp/otherprojects/index.htm
Charlson Index of Comorbidity http://www.erpho.org.uk/viewresource.aspx?id=15069
Karen Monsen, PhD, RN
Omaha System Interventions Intervention = Problem + Category + Target 42 Problems Environmental, Psychosocial, Physiological, Other Health Related problems Interventions Category & Target 4 Categories Monitoring, Coordinating, Providing Care, Teaching 72 Targets i.e. exercise, coping, cardiac care Intervention = Problem + Category + Target (12,096 terms)
Omaha System Intervention Groups 1 2 Monitoring Respiration and Circulation 11 Providing Respiration & Circulation Therapy Monitoring Emotional & Cognitive Status 12 Providing Pain Treatment 3 Monitoring Pain 13 Providing Medication Treatment 4 Monitoring Medications 14 Providing Injury Prevention Treatment 5 Monitoring Injury Prevention 15 Providing Wound Care Treatment 6 Monitoring Skin 16 Providing Bowel and Bladder Treatment 7 Monitoring Other 17 Providing Other Treatment 8 Coordinating Supplies & Equipment 18 Teaching Respiration & Circulation 9 Coordinating Community Resources 19 Teaching Medications 10 Coordinating Other 20 Teaching Disease Process 21 Teaching Disease Treatment 22 Teaching Emotional & Cognitive Issues 23 Teaching Other http://www.nursing.umn.edu/icnp/otherprojects/index.htm
Scales Scale Range OASIS Data Items indicated by M0xx numbering* Prognosis 0-2 M0260 Overall Prognosis M0270 Rehabilitative Prognosis Pain 0-4 M0420 Frequency of Pain M0430 Intractable Pain Pressure Ulcer 0-20 M0450 Stages 2-4 Pressure Ulcers (number of pressure ulcers multiplied by the stage of the pressure ulcer) Stasis Ulcer 0-8 M0470 Number Stasis Ulcers M0474 Unobserved Stasis ulcer M0476 Status of Most Problematic Surgical Wound 0 8 M0484 Number of Surgical Wounds M0486 Unobserved Surgical Wound M0488 Status of Most Problematic Respiratory 0 7 M0490 When Dyspneic M0500 Respiratory Treatments
Creating Normal Distribution Assumption of normal distribution for many analyses Methods to manage Log transformation (software does) Clinician s decision about important cut points Split the data into 2, 3, 4, or more groups Split by quartiles using software Even number of records for each variable Must be interpretable to clinician
Transforming Variables Logistic regression Assumes normal distribution Created dummy codes Discriminative pattern analysis Requires all variables to be binary Used expert judgement Ripper classification Uses binary, ordinal or continuous data No assumption of normality of the data Little transformation needed
Data Mining - Analysis Multiple steps iterative process Selecting variables/ features Creating results Descriptive statistics Simple relationships Chi square Predictive modeling Logistic regression Data mining Rules-classification Associations
Data Analyses Methods Traditional Statistics KDD Variable/ Feature Selection Chi-Square, bivariate analysis Chi-Square InfoGain CFS evaluation BestFirst Greedy Stepwise Genetic Clustering Latent Class K Means EM Predictive Modeling Logistic Regression Rules classifiers Discriminate pattern analysis Decision Trees Bayesian Network 32
Interpretation/ Evaluation If it doesn t make sense to a clinician, then it doesn t make sense Parsimony Validity, reliability, trustworthiness of findings depend on the type of analysis Traditional statistical analysis Statistical significance i.e. p <.01 Data mining K-fold cross validation Measures similar to sensitivity and specificity Overall accuracy of model, precision, and recall
Example KDD and Comparison of Methods Identify predictors for improvement of oral medication management for home care patients patient and support system characteristics clinician interventions number of medications
Logistic Patient Characteristics Predictor Variable OR (95% CI) Current shopping.17 (.05 -.55) No prior inpatient stay previous 14 days.29 (.16 -.54) Cognitive Functioning Some assistance and direction.33 (.15 -.74) Considerable assistance in routine situations..33 (.11 -.94) Totally dependent.06 (.01 -.43) Current: Toileting.60 (.37 -.98) Current: Prepare Light Meals.60 (.37 -.98) Oral Medication Management at Admission 11.69 (8.27 16.51)
Predictor Variables Providing injury prevention treatment Logistic Regression - Interventions OR (95% CI).52 (.32-0.82) Teaching medications 2.18 (1.47-3.23) Total number of medications.97 (.94-1.00)
Discriminate Pattern Analysis Predictive Variables No Imp Imp Cum % Yes Diff % Odds Ratio Vision: No impairment 61% 2% 2% 59% 69.4 Oral medication management: Only requires setup/ reminder Oral medication management: Needs assistance Admitted from an inpatient facility Respiratory problems: None to moderate Cognition: No impairment 27% 88% 90% 60% 18.5 59% 39% 93% 19% 2.19
Ripper Classification Rules Rule (Using Down Sampling Repeated 10x) Total No Improve Oral Medication Management (Min) => No improvement 180 174 6 If Confusion (> Min) & Transferring (> Min) & Toileting (Min/ Mod) & No Vision Prob & No Monitoring Pain => No improvement If No Surgical Wound & and Age > 85 & No Monitoring Injury & Has Other Lesion & Prognosis (Fair Good) => No improvement If Male & Prognosis (Poor) & LOS < 30 days Has Monitoring Pain => No improvement Toileting (Mod/ Severe) LOS < 30 days & Charlson Index (Mod/ Severe) => No improvement 19 17 2 14 13 1 13 11 2 9 8 1 Else => Improvement 301 45 256 Total 536 268 268 Precision/ Recall.75/.94 Improvement,.92/.69 No Improvement
Comparison of Variables Variables LR RIPPER DPA Prior Inpatient Stay X X LOS in Home Care < 30 Days X Age > 85 / Male X Prognosis/ Charlson Index X Cognition X X X Vision Problem X X Surgical Wound/ Other Lesion X Oral Medication Management X X X ADL X X IADL X
Interventions Interventions LR RIPPER DPA Injury Prevention Treatment X Teaching Medications X Monitoring Pain Monitoring Injury X X Respiration X
Nurses are Knowledge Workers 41 American Nurses Association, Scope and Standards of Nursing Informatics Practice, 2007 4/29/2014 41
Bonnie Westra, PhD, RN, FAAN Assistant Professor & Co-Director ICNP Center University of Minnesota, School of Nursing 6-135 Weaver-Densford Hall W - 612-625-4470 F - 612-626-3255 westr006@umn.edu 42