Lessons Learned Reuse of EHR Data for Research and Quality Improvement

Transcription

1 Lessons Learned Reuse of EHR Data for Research and Quality Improvement Bonnie L. Westra, PhD, RN, FAAN Assistant Professor, Co-Director ICNP Center for Nursing Minimum Data Set Knowledge Discovery University of Minnesota, School of Nursing 4/29/2014 1

2 Contributors Note: It takes a team! Lynn Choromanski, MS, RN; Mary Dierich, PhD-C, RN; Madeleine Kerr, PhD, RN; Karen Monsen, PhD, RN; Kay Savik, MS; Fang Yu, PhD, RN 1 Genevieve Melton-Meaux, MD 2 Cristina Oancea, PhD-C 3 Debra Solomon, MSN, RN, CNP 4 John H. Holmes, PhD 5 Sanjoy Dey, BSc; Gang Fang, M.Sc; Michael Steinbach, PhD; Vipin Kumar, PhD 6 Karen Dorman Marek, PhD,RN, FAAN 7 1 UMN School of Nursing, 2 UMN, Medicine, 3 UMN, Public Health, 4 Fairview Lakes HomeCaring & Hospice, 5 University of Pennsylvania, 6 UMN Computer Science, 7 Arizona State University

3 Grant Support National Institute of Nursing Research (Grant #P20 NR008992; Center for Health Trajectory Research) University of Minnesota, Grant-In-Aid Digital Technology Initiative The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Nursing Research or the National Institutes of Health.

4 Purpose 1. Describe steps in the process for discovering new knowledge from reuse of EHR data 2. Examine data selection and data quality issues 3. Explain methods data preparation and transformation for developing knowledge from EHR data 4. Compare different ways of developing predictive models 5. Explore lessons learned from reuse of data for quality improvement

5 Knowledge Discovery Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI Magazine, pp P. 41 5

6 Start with a Question in Mind 1. Test the feasibility of abstracting, integrating, and comparing the effective use of the Omaha System data across multiple software vendors and home care agencies. 2. Discover predictors for various outcomes build evidence for best practices from clinician data a. Hospitalization b. Improvement urinary and bowel incontinence c. Pressure ulcers d. Improvement in ambulation e. Improvement in oral medication management 3. Compare methods of developing predictive models

7 Data Selection - EHRs Home health care data 2 software vendors Convenience sample - 15 Homecare agencies Primarily Midwest also East coast All open admissions in 2004 Initial Data 18,067 OASIS records 989,772 Omaha System interventions 3,199 patients (1 74 OASIS records/ patient) 65,000 medication records

8 OASIS data Selected Data Clinical record items Demographics & patient history Living arrangements/ supportive assistance Health status Functional status Emergent care use/ discharge Omaha System interventions Problems Environmental, Psychosocial, Physiological, Other Health Related Problems Category of Action (HTG, T&P, CM, S) Targets focus of intervention Medication data 8

9 Data Quality Issues Know the Strengths and Limitations of Your Data Documentation issues Consistency of processes for documenting Copy forward or copy/paste Incomplete/ inappropriate data in the database Rules for data collection Charting by exception Rules i.e. the Joint Commission, CMS, billing Database / data model Field type Relationship of fields how do you link data Patient outliers Data with too little variance

10 Data Preparation / Cleaning Data Cleaning Duplicate data Plausible responses Missing data Consistency checks Type of data numeric, character, text Creating episodes for prediction What data are collected at specific points in time Multiple episodes per patient Linking all data to episodes of care

11 Causes of Missing Data Charting by exception Skip patterns Alternative documentation processes Wrong patient incomplete data Patient discharged before next data collection point or dies System errors

12 Figure 1. OASIS Integumentary Skip Pattern Skin Lesion or Open Wound No Go to Respiratory Status Yes Pressure Ulcer No Stasis Ulcer No Surgical Sound Yes Yes Yes Number of Ulcers Stage of Most Problematic Ulcer Status of Most Problematic Ulcer Number of Ulcers Status of Most Problematic Ulcer Number of Wounds Status of Most Problematic Wound

13 Characteristics of Data and Plausibility (M0700) Ambulation/Locomotion: Ability to SAFELY walk, once in a standing position, or use a wheelchair, once in a seated position, on a variety of surfaces. 0. Able to independently walk on even and uneven surfaces and climb stairs with or without railings (i.e., needs no human assistance or assistive device). 1. Requires use of a device (e.g., cane, walker) to walk alone or requires human supervision or assistance to negotiate stairs or steps or uneven surfaces. 2. Able to walk only with the supervision or assistance of another person at all times. 3. Chairfast, unable to ambulate but is able to wheel self independently. 4. Chairfast, unable to ambulate and is unable to wheel self. 5. Bedfast, unable to ambulate or be up in a chair. UK Unknown If response > 6, not a valid response If UK or NA, then data displays as character vs numeric

14 Inconsistency Checks

15 Distribution of Responses Skewed Data Ability to Transfer Little Variance Race/ Ethnicity 0 Total Caucasian Non-caucasian

16 Heterogeneity of Population The goal is to Have an interpretable model Increase generalizability to a specific population Considerations Date range Type of population Age groups Location Length of stay Severity of illness Latent class analysis

17 Managing Data Characteristics

18 Other Data Preparation Steps OASIS data matched Start or Resumption of Care with Discharge Assessment Matched interventions by date to episodes Matched prescribed and over the counter medications by date to create a count for the number of unique medications Selected episodes appropriate for the outcome Improvement Oral Medication Management Only those who could improve or had a problem on admission

19

20 Episodes of Care 20

21 Transformation Converting data from one format to another Reasons Data reduction Format for to meeting assumptions for analyses Increase interpretability of results Decrease chaos

22 Transformation Clinical Classification Software Primary diagnoses and then reduced into 51 smaller groups within 11 major categories Charlson Index of Comorbidity Additional medical diagnoses Interventions Theoretically grouped into 23 categories Scales Created indicator variables (dummy codes) For non-normally distributed data e. g. Level of anxiety 3 levels, reference is No Anxiety

23 Clinical Classification Software Data reduction strategy for ICD diagnoses and procedures used experts Example: CCS CHF '4280 ' '4281 ' '42820' '42821' '42822' '42823' '42830' '42831' '42832' '42833' '42840' '42841' '42842' '42843' ' Groups 13,000 Dx

24 Charlson Index of Comorbidity

25 Karen Monsen, PhD, RN

26 Omaha System Interventions Intervention = Problem + Category + Target 42 Problems Environmental, Psychosocial, Physiological, Other Health Related problems Interventions Category & Target 4 Categories Monitoring, Coordinating, Providing Care, Teaching 72 Targets i.e. exercise, coping, cardiac care Intervention = Problem + Category + Target (12,096 terms)

27 Omaha System Intervention Groups 1 2 Monitoring Respiration and Circulation 11 Providing Respiration & Circulation Therapy Monitoring Emotional & Cognitive Status 12 Providing Pain Treatment 3 Monitoring Pain 13 Providing Medication Treatment 4 Monitoring Medications 14 Providing Injury Prevention Treatment 5 Monitoring Injury Prevention 15 Providing Wound Care Treatment 6 Monitoring Skin 16 Providing Bowel and Bladder Treatment 7 Monitoring Other 17 Providing Other Treatment 8 Coordinating Supplies & Equipment 18 Teaching Respiration & Circulation 9 Coordinating Community Resources 19 Teaching Medications 10 Coordinating Other 20 Teaching Disease Process 21 Teaching Disease Treatment 22 Teaching Emotional & Cognitive Issues 23 Teaching Other

28 Scales Scale Range OASIS Data Items indicated by M0xx numbering* Prognosis 0-2 M0260 Overall Prognosis M0270 Rehabilitative Prognosis Pain 0-4 M0420 Frequency of Pain M0430 Intractable Pain Pressure Ulcer 0-20 M0450 Stages 2-4 Pressure Ulcers (number of pressure ulcers multiplied by the stage of the pressure ulcer) Stasis Ulcer 0-8 M0470 Number Stasis Ulcers M0474 Unobserved Stasis ulcer M0476 Status of Most Problematic Surgical Wound 0 8 M0484 Number of Surgical Wounds M0486 Unobserved Surgical Wound M0488 Status of Most Problematic Respiratory 0 7 M0490 When Dyspneic M0500 Respiratory Treatments

29 Creating Normal Distribution Assumption of normal distribution for many analyses Methods to manage Log transformation (software does) Clinician s decision about important cut points Split the data into 2, 3, 4, or more groups Split by quartiles using software Even number of records for each variable Must be interpretable to clinician

30 Transforming Variables Logistic regression Assumes normal distribution Created dummy codes Discriminative pattern analysis Requires all variables to be binary Used expert judgement Ripper classification Uses binary, ordinal or continuous data No assumption of normality of the data Little transformation needed

31 Data Mining - Analysis Multiple steps iterative process Selecting variables/ features Creating results Descriptive statistics Simple relationships Chi square Predictive modeling Logistic regression Data mining Rules-classification Associations

32 Data Analyses Methods Traditional Statistics KDD Variable/ Feature Selection Chi-Square, bivariate analysis Chi-Square InfoGain CFS evaluation BestFirst Greedy Stepwise Genetic Clustering Latent Class K Means EM Predictive Modeling Logistic Regression Rules classifiers Discriminate pattern analysis Decision Trees Bayesian Network 32

33 Interpretation/ Evaluation If it doesn t make sense to a clinician, then it doesn t make sense Parsimony Validity, reliability, trustworthiness of findings depend on the type of analysis Traditional statistical analysis Statistical significance i.e. p <.01 Data mining K-fold cross validation Measures similar to sensitivity and specificity Overall accuracy of model, precision, and recall

34 Example KDD and Comparison of Methods Identify predictors for improvement of oral medication management for home care patients patient and support system characteristics clinician interventions number of medications

35 Logistic Patient Characteristics Predictor Variable OR (95% CI) Current shopping.17 ( ) No prior inpatient stay previous 14 days.29 ( ) Cognitive Functioning Some assistance and direction.33 ( ) Considerable assistance in routine situations..33 ( ) Totally dependent.06 ( ) Current: Toileting.60 ( ) Current: Prepare Light Meals.60 ( ) Oral Medication Management at Admission ( )

36 Predictor Variables Providing injury prevention treatment Logistic Regression - Interventions OR (95% CI).52 ( ) Teaching medications 2.18 ( ) Total number of medications.97 ( )

37 Discriminate Pattern Analysis Predictive Variables No Imp Imp Cum % Yes Diff % Odds Ratio Vision: No impairment 61% 2% 2% 59% 69.4 Oral medication management: Only requires setup/ reminder Oral medication management: Needs assistance Admitted from an inpatient facility Respiratory problems: None to moderate Cognition: No impairment 27% 88% 90% 60% % 39% 93% 19% 2.19

38 Ripper Classification Rules Rule (Using Down Sampling Repeated 10x) Total No Improve Oral Medication Management (Min) => No improvement If Confusion (> Min) & Transferring (> Min) & Toileting (Min/ Mod) & No Vision Prob & No Monitoring Pain => No improvement If No Surgical Wound & and Age > 85 & No Monitoring Injury & Has Other Lesion & Prognosis (Fair Good) => No improvement If Male & Prognosis (Poor) & LOS < 30 days Has Monitoring Pain => No improvement Toileting (Mod/ Severe) LOS < 30 days & Charlson Index (Mod/ Severe) => No improvement Else => Improvement Total Precision/ Recall.75/.94 Improvement,.92/.69 No Improvement

39 Comparison of Variables Variables LR RIPPER DPA Prior Inpatient Stay X X LOS in Home Care < 30 Days X Age > 85 / Male X Prognosis/ Charlson Index X Cognition X X X Vision Problem X X Surgical Wound/ Other Lesion X Oral Medication Management X X X ADL X X IADL X

40 Interventions Interventions LR RIPPER DPA Injury Prevention Treatment X Teaching Medications X Monitoring Pain Monitoring Injury X X Respiration X

41 Nurses are Knowledge Workers 41 American Nurses Association, Scope and Standards of Nursing Informatics Practice, /29/

42 Bonnie Westra, PhD, RN, FAAN Assistant Professor & Co-Director ICNP Center University of Minnesota, School of Nursing Weaver-Densford Hall W F westr006@umn.edu 42