Data Quality Why It Matters October 19, 2015
Agenda Introduction Broad thoughts about Data and Data Quality Impact to PACE Benchmarks Presentation by LIFE St. Francis on Data Quality efforts undertaken by the organization Data Quality efforts at NPA Q and A
Data Quality Data Quality is a global concern Business decisions must be based on quality data to have significant intended impact on an organization s performance Companies in the top third of their industry in the use of datadriven decision making were, on average, 5% more productive and 6% more profitable than their competitors If data is not managed correctly then it can become a risky liability instead of a valuable asset
What is Good Data Quality and How it Relates to PACE Data Quality is a complex measure of data properties from various dimensions Are all instances captured? Is data in different tables linked? Is it entered correctly without any errors? Duplicate IDs No Dis-Enroll Reason Enrollment File Participant Disenroll Disenroll Medicaid Medicare Living Date of ID Enroll Date Date Reason Date Date Situation Birth 002 11/1/2007 11/26/2009 Deceased 11/1/2007 N/A 1/3/1940 002 11/9/2007 7/31/2009 Financial 11/9/2007 N/A 1/7/1940 001 12/9/2007 12/9/2007 Home Alone 1/5/1940 010 12/1/2007 12/1/2007 Home Alone 1/4/1935 008 2/28/2014 4/30/2014 3/14/2010 2/28/2014 Home Alone 1/8/2040 009 11/1/2007 11/11/2015 11/1/2007 Home Alone 1/6/1940 015 10/16/2007 10/1/2007 Other 10/16/2007 Nursing Home 1/12/1940 016 9/1/2008 6/23/2012 Deceased 9/4/1008 01/01/0209 Group Home 1/13/1940 No Living Situation Date of Birth in the Future Dis-Enroll Date < Enroll Date Incorrect Formatting of Date
What is Good Data Quality and How it Relates to PACE Data Quality is a complex measure of data properties from various dimensions Certain measures cannot be calculated because data is not submitted. It has to be searched Up to date information is available Overall Test At the end of the day, can data be put to the use it is intended for?
Data Quality is a Continuum Physical Therapy PMPM 2013Q2 2013Q3 2014Q2 Minimum 0.40 0.44 0.23 Maximum 22.42 30.99 16.06 Why is there a vast difference between Minimum PMPM and Maximum PMPM? What is creating the fluctuation in the Maximum range? State Mandates, Services Contracted and Participant Needs Create Variation in Data Some state regulations allow for services to be rendered by aides/assistants Not all states require aides/assistants to be supervised by a licensed professional Different levels of occupational, physical and speech therapy might be contracted Age, frailty and disease burden add another layer of variation
Data Quality is a Continuum Physical Therapy PMPM 2013Q2 2013Q3 2014Q2 Minimum 0.40 0.44 0.23 Maximum 22.42 30.99 16.06 Why is there a vast difference between Minimum PMPM and Maximum PMPM? What is creating the fluctuation in the Maximum range? Program Practices Create Variation in Data Not tracked Systems not in place to capture services provided by contracted professionals or services for which claims are received Systems not in place to capture services provided in facilities Therapies provided by PACE employees not entered into EMR consistently and accurately
The Common Data Set puts us on the pathway to Data Quality Data quality starts with the right processes in an organization Partnered with LIFE St. Francis who will present their recommendations NPA will create a Common Data Set manual where all data elements will be fully defined We will solicit member input for this manual
DATA PACE 2 ASSURING DATA INTEGRITY LIFE St. Francis, Trenton, NJ Lisa Zavorski, M.S., B.S. Karen Hillman October 19 th, 2015
Why LIFE St. Francis is Concerned with Data Integrity Ensures the accuracy and consistency of data over time Instills trust and confidence in your data when it is reliable Data inaccuracies can have negative impact on business decisions including achievement of goals and outcomes Provides a basis for resource utilization and allocation Garbage In, Garbage Out
Enrollment File Recommendations Assure file conforms to CSV conventions Verify that addresses contain no commas Assure specificity of Living Situation 1) Other check for more accurate code 2) Assisted Living use Home Alone 3) All codes, excluding NH, are community living Work with EHR vendor to assure all fields are extracted properly to CSV file. For LSF, we must have date with Medicare/Medicaid # to be included in file.
Program Level Data Recommendations Metric Tips Data Extraction Days Attending PACE Center * Enrolled Participants Only Actual Days Attended Sick/Cancel Days Not Counted Acute Hospital Admissions * Excludes observation Excludes Psych Excludes IP Hospice Excludes LTAC Excludes Sub-acute Rehab Coordinate with PCP, Facility, Finance on status (i.e. IP, Observation, ER) Acute Hospital Days * Do not count discharge date Split and put days in quarter they occurred Schedule Excel Excel * Keep Current Process until Q1 2016
Program Level Data Recommendations Metric Tips ER Visits * Include if treated and released from ER Excludes Observation Psychiatric IP Admits * Includes Inpatient Hospital Psych Unit Includes Inpatient Detox for Alcohol or Drugs Includes Standalone Facility for Behavioral/Mental Health Data Extraction Excel Excel Psychiatric IP Days* Do not count Discharge Date Excel * Keep Current Process until Q1 2016
Program Level Data Recommendations Metric Tips Data Extraction # Participants in Long Term NF Placement >90 Days * Participants in a nursing facility placement, defined as a placement that has lasted 90 + days Excel Short Term NF Days <90 Days * Days spent by participants in a nursing facility placement, i.e. 89 or less days Does not pertain to Living Situation, strictly by # of days Long Term NF Days >90 Days * Days spent by participants in a nursing facility placement, i.e. 90+ days Does not pertain to Living Situation, strictly by # of days Excel Excel * Keep Current Process until Q1 2016
Counting ST NF and LT NF Days If the PO is providing a Participant Level Admission File, the DataPACE system counts a ST or LT NF day on the following basis: Step 1 - The system calculates NF Length of Stay (LOS) which is Discharge Date - Admit Date. Please note that the actual discharge date is used even if it is after the end of the quarter. If the Discharge Date is blank, as in the case of participants in permanent placement, then the system uses quarter end date of the quarter in consideration. Step 2 - If LOS in NF > 89 days, then system tags that stay as LT, otherwise ST. Please note that this day count is only being used to tag the stay.
Counting ST NF and LT NF Days Step 3 The system stages the Start Date and End Date for the quarter from which the counts will be derived based on the following: The Start Date is the Admit Date if the Admit Date is within the quarter. If the Admit Date is prior to the start of the quarter, then the Start Date is the 1 st of the quarter (this is to ensure that only days within the quarter are counted). Please see ID 10 in the illustration below. The End Date is the Discharge Date if the Discharge Date is within the quarter. If the Discharge Date is after the end of the quarter, then the End Date is the last date of the quarter. Please see ID 12 in the illustration below. Step 4 - The system counts LT and ST days in the quarter. *Note: Participants must meet the 90 day threshold for the stay to be considered long term. When participant s living situation changes to permanent placement prior to meeting the 90 day threshold, the count of days begins on date of admission; therefore, this participant would be considered short term in the day count despite the change in Living Situation to Long Term Care.
Counting ST NF and LT NF Days Situation 1: When a participant lives at home and the status changes to permanent placement, length of stay begins on admission date and will be short term until the 90 day threshold. At the 90 day threshold, length of stay is considered long term. Situation 2: When a participant is already admitted to a NF but not yet considered permanently placed, then living situation status is changed to permanent placement, the length of stay begins on original admission date to NF and will be included in the short term day count until 90 day threshold is collectively met. The NF day count doesn t start at date of permanent placement, but rather starts with the NF date of admission.
Counting ST NF and LT NF Days The following examples illustrate how NF days are counted for 2014Q2 Step 1 Step 2 Step 3 Step 4 START DATE END DATE ID LOS (Discharge Date - Admit Date. If Discharge Date is blank use last date of quarter) If LOS in NF > 89 days then LT Admit Date. If Admit Date is prior to start of the quarter, then 1st of the quarter Discharge Date. If Discharge Date is after the end of the quarter then last date of the quarter LT Days ST Days Discharge Admit Date Date 10 3/1/2010 5/3/2014 1524 LT 4/1/2014 5/3/2014 32 11 4/9/2014 6/15/2014 67 ST 4/9/2014 6/15/2014 67 12 5/7/2012 783 LT 4/1/2014 6/30/2014 90 13 3/15/2014 7/15/2014 122 LT 4/1/2014 6/30/2014 90 14 4/15/2014 8/20/2014 127 LT 4/15/2014 6/30/2014 76 Total: 288 67 Data Quality PO s providing ST NF and LT NF days at the Program Level must provide counts using the methodology described above. PO s providing the detail level file must ensure that there are no duplicate entries in the file. For instance, EMR systems will re-insert a duplicate row if an entry for discharge date is made to a previous admit. This situation should be avoided.
Program Level Outpatient Specialists Specialist Type Count Y/N Tips Data Extraction Allergy/Immunology Yes Not in use at LSF N/A Anesthesiology Yes Not in use at LSF included with surgery N/A Bariatric Surgeon Yes Schedule Cardiology Yes Schedule Chiropractor Yes Not in use at LSF N/A Dentistry Yes Includes Onsite Schedule Dermatology Yes Schedule Endocrinology Yes Schedule Gastroenterology Yes Schedule General Surgeon Yes Schedule Gynecology Yes Schedule Hematology/Oncology Yes Schedule Infectious Disease Yes Schedule Internal Medicine Yes Not in use at LSF N/A Map to PT Massage Therapy Yes Medical Necessity Only Schedule/Paper
Program Level Outpatient Specialists Specialist Type Count Y/N Tips Data Extraction Nephrology Yes Schedule Neurosurgeon Yes Schedule Neurology Yes Schedule Ophthalmology Yes Schedule Optometry Yes Include Onsite Schedule Oral Surgery Yes Schedule Orthopedics Yes Schedule Orthopedic Surgeon Yes Schedule Otolaryngology Yes Schedule Pain Management Yes Not in use at LSF Schedule Physical Medicine/Rehab Yes Schedule Plastic Surgery Yes Schedule Podiatry Yes Includes In-house Schedule Psychiatry Yes Psychiatry Consults Future at LSF Psychology Consult Psychology Not Behavioral Therapy (future CDS) Future at LSF
Program Level Outpatient Specialists Specialist Type Count Y/N Tips Data Extraction Pulmonology Yes Schedule Radiology Yes Include Interventional & Vascular Schedule Count Radiation Oncologist Consult - not treatments Radiation Oncology Yes (future CDS) Schedule Rheumatology Yes Schedule Thoracic Surgeon Yes Schedule Transplantation Yes Schedule Vascular Surgeon Yes Schedule *Note At LSF, Outpatient Specialists are currently totaled up and submitted under Program Level as one measure. LSF uses this file for definition so clinic can do counts.
Program Level Discipline Tips Count face-to-face encounters only Do not count telephone calls (future CDS) Groups may add 1 encounter to total for each participant attending Family meetings add 1 encounter to count for each discipline only if participant and/or caregiver attends If multiple staff members see participants for multiple encounters in one day, count each visit (except aide in center) Collect location for each encounter Important to count any face-to-face encounters that are scanned
Program Level Data Metric Hemoglobin A1C test Meals Personal Care Assistance Visits Primary Care Physician Count all lab draws per quarter Count Center Meals, not Snacks Count Home Delivered Meals Count Contracted Meals Use Invoices if needed to obtain count of meals Tips Home Visits for personal care related to ADLs (examples of ADLs: Assistance with Feeding, Bathing and Hygiene, Mobility Assistance, Toileting and Incontinence Care, Transferring and Positioning) Personal Care combined with Chores Counted Count each visit per day in home, this can exceed 1 In center, count showers and other personal care but does not exceed count of 1 per day Chores, Housekeeping & IADL Assistance only visits not counted (future CDS) (examples: Laundry, Vacuuming, Kitchen and Bathroom Cleaning, Meal Preparation, Shopping, Other Home Management Activities) Includes PCP, NP and PA Scans from NH visits if not in EHR Family Meetings Include Contracted Data Extraction Report Excel Schedule, Paper Schedule
Program Level Data Metric Skilled Home Care Visits Social Worker Therapy Visits Transportation Trips Tips PCP, RN, PT, OT, SW, ST (no Registered Dietitian, future CDS) Capture any scans Home includes Assisted Living Facilities and Programs All RN Home Visits Community Agency where SW meets participant to assist, example Medicaid Application Groups Family Meetings Include Contracted SW Includes PT, OT, ST, Massage Therapy* (*Medical Necessity Only) Groups Rehab done at Facilities by Non-PACE staff* Restorative Nursing/PT supervised by licensed therapist No Rec Therapy Counted (future CDS) Count each leg of trip Count Rec Therapy trips, 1 count per participant Meds, supplies or meal deliveries are not counted Ambulance, prearranged but not emergency Taxi if arranged by LSF Participant must be on trip Data Extraction Schedule, Paper Schedule, Paper LSF currently only collecting SW Employees Schedule, Paper *Rehab at facilities not yet counted by LSF Excel
Feedback from NPA If data reflects higher or lower numbers than last quarter, investigate to identify the source of variation. Examples of this can be: 1. Changes to the center attendance model may result in higher or lower utilization. 2. Staffing patterns may result in higher or lower utilization. 3. Flu season may result in increased sick visits/utilization. Validate data accuracy with staff member who collected measure
Checking for Data Quality in NPA Check current quarter outcomes against prior quarter outcomes Internal review Check with PACE Organization if change in outcomes is due to changing PACE Organization practices or due to data errors 1.60 1.40 1.20 1.00 0.80 0.60 0.40 0.20 0.00 1.36 Peer Results Previous Quarter Primary Care Encounters PMPM 0.82 Program Results Previous Quarter 1.52 Program Results Current Quarter NPA Inquiries How was it possible to double the primary care service? Was another MD hired? Is this a result of data error in this quarter? Or, was inadequate data reported in the prior quarter? If a determination cannot be made then is current quarter outcome the correct outcome since it is closer to the peer result?
Checking for Data Quality in NPA Check the trend in outcomes for current and prior 3 quarters Inconsistencies in data counts will cause rejection if the PACE Organization is not able to provide a plausible explanation 16000 PACE Center Attendance Days PMPM COUNT 16,000 14000 14,000 12000 12,000 10000 10,000 8000 8,000 6000 6,000 4000 4,000 2000 0 PMPM 0.85 PMPM 1.06 PMPM 5.83 2014Q1 2014Q2 2014Q3 2014Q4 PMPM 5.67 2,000 0
Checking for Data Quality in NPA Check if there is consistency between different data points that are related Example of Hospital Admits and LOS from one program 8.0 Average LOS Per Hospital Admit 7.0 6.0 5.0 4.0 3.0 2.0 Does data quality have anything to do with the increasing trendline? 1.0 0.0 2012 Q1 2012 Q2 2012 Q3 2012 Q4 2013 Q1 2013 Q2 2013 Q3 2013Q4 2014Q1 2014Q2 2014Q3 2014Q4
Checking for Data Quality in NPA Check if there is consistency between different data points that are related Hospital Admits and LOS: Acquired understanding of how our systems handled the different file structures Old records with no discharge dates where participants were deceased were still adding days Acquired understanding of how different file formats were impacting this number PACE organizations submitting program level data had different understanding of how to calculate LOS Files submitted at the participant level had data patterns that varied from EMR to EMR
Checking for Data Quality in NPA Check if there is consistency between different data points that are related Hospital Admits and LOS: Processes in place result in a dropping trend line for the benchmark calculated for all programs 8,000.00 7,000.00 6,000.00 5,000.00 4,000.00 3,000.00 2,000.00 1,000.00 0.00 Acute Hospital Days per 1000 Members Per Annum
Data Quality in NPA Will be an ongoing relentless exercise To ensure that the truest benchmarks are generated, formalized processes have been created To understand data To clean data To crosswalk data To explain all aberrations and ensure that there are logical explanations To ensure that only validated data points are included To publish the reports To mail the reports We end with a very intense post mortem on how the data challenged us and what can be done to be prepared for the next quarter
Questions? Comments? Suggestions? For more information contact The NPA Data Team Alan Gay at alang@npaonline.org Berry McCarthy at berrym@npaonline.org Randi Kudner at randik@npaonline.org Ameeta Mistry at ameetam@npaonline.org Thank you!