A Historical Look at the UDSMR Program Evaluation Model

Transcription

1 A Historical Look at the UDSMR Program Evaluation Model Troy Hillman, Manager of Analytical Services Group Sarah Mullin, MS, Data Analyst Uniform Data System for Medical Rehabilitation 2015 Uniform Data System for Medical Rehabilitation, a division of UB Foundation Activities, Inc. FIM, UDS Central, UDSMR, and the UDSMR logo are trademarks of Uniform Data System for Medical Rehabilitation, a division of UB Foundation Activities, Inc.

2 Overview PEM background PEM calculations Historical outlook on PEM scores and percentiles Historical outlook on top 10% Possible improvements How to interpret your PEM score How to improve your PEM score 2

3 Background of the PEM Why do we need a performance evaluation model? UDSMR s subscribers requested one Need to constantly demonstrate the value of acute inpatient rehabilitation Need to identify and learn from high performers and/or high improvers CMS initiated and publicly reports on performance IOM has recommended to Congress that all healthcare providers be compensated on a value-based performance (P4P) basis Comments sought from IRF providers in 2009 proposed rule 3

4 Background of the PEM Stipulations of the UDSMR PEM: 1. Patterned after an accepted model HQID Premier Acute Hospital Initiative (sponsored by CMS) 2. Based on a composite measure Indicator(s) of effectiveness, efficiency, and quality (safety) 3. Easy to create and maintain No new data collection Indicators captured routinely and accepted by the field 4. Auditable (scoring patterns, etiologic diagnosis coding, etc.) 4

5 Background of the PEM The result was a composite measure of effectiveness, efficiency, and quality that utilizes IRF-PAI data already collected as part of the IRF PPS process Indicators were created at both the case level and the facility level, and the model utilizes CMGbased expectations to account for variations in impairment distribution, patient severity, or both The UDSMR PEM has been utilized in over 70% of all IRFs in the nation since

6 Inclusion/Exclusion Criteria Inclusions: Only PPS facilities Only facilities that have at least thirty cases during the year Only facilities that have at least one case in each quarter Exclusions: Expired cases 6

7 How Is the PEM Calculated? Case-level indicators: Discharge FIM total Level of functional independence at discharge FIM change Functional improvement / reduction in burden of care LOS efficiency Rate of functional improvement over time Facility-level indicators: % discharged to the community % discharged to acute care 7

8 Case-Level Indicators Case-level indicators (discharge FIM total, FIM change, and LOS efficiency) are calculated at the patient level for each patient Actual case performance is measured against expected performance (impairment and severity-adjusted [CMG-adjusted] benchmark) Credit (1 point) is given if the patient s actual score meets or exceeds the average score for cases within the patient s CMG 8

9 Case-Level Indicators The facility actual column for each indicator identifies the number of discharged patients who met or exceeded the CMG benchmark The facility target column for each indicator identifies the number of possible points that could be obtained and equals the number of patients discharged 9

10 Case-Level Indicators The facility subscore column for each indicator indicates the percentage of patients who met or exceeded the CMG benchmark The composite subscore is the sum of actual points divided by the sum of potential points (i.e., the percentage of all CMG targets achieved) 10

11 Facility-Level Indicators Facility-level indicators: % discharged to the community % discharged to acute care These indicators measure a facility s performance against impairment- and severity-adjusted (CMGadjusted) benchmarks 11

12 Facility-Level Indicators The facility actual value for each indicator is the actual percentage of patients discharged to each setting The facility targets are the CMG-adjusted expected discharge rates 12

13 Facility-Level Indicators The facility subscore for: % discharge to community is the actual percentage divided by the CMG-adjusted expected percentage % discharge to acute care is 100% aaaaaa % 100% CCC aaaaaaaa % 100 = = = 99.9% 13

14 Weighting for PEM Calculation Weighting* for final PEM score: Facility subscores are weighted as follows for the case-level and facility-level indicators: 60% case-level indicator composite Discharge FIM total, FIM change, and LOS efficiency 30% discharge to community 10% discharge to acute care Higher credit for lower D/C to acute care Lower credit (penalty) for higher D/C to acute care * Weight assignment modeled after HQI Demonstration Project 14

15 How Is the PEM Score Calculated? Facility PEM Score: Sum of the weighted indicator subscores 15

16 2014 PEM Score Distribution 16

17 How Is the Percentile Rank Calculated? Facility Percentile Rank: The facility-specific composite scores are then ranked from lowest to highest, and each facility in the UDSMR database is assigned a percentile rank from 1 to 100 relative to the other IRF subscribers in the database The percentile rank affords each facility a sense of its performance in comparison to those of the other IRFs that qualified for inclusion in the PEM that year 17

18 Basic Rules of Thumb for Interpreting Your PEM Report 1. Start with discharge to acute care % If your facility s actual value is higher than the expected value, ask yourself these questions: How has this affected your facility s discharge-to-community percentage? How does this affect case-level indicators? Acute care discharges typically have lower-than-expected outcomes, and a higher-than-expected rate is bound to affect case-level indicators 18

19 Basic Rules of Thumb for Interpreting Your PEM Report 2. Look at discharge to community % If your facility s actual value is less than the expected value, ask yourself: How does this affect case-level indicators? Because discharges to other settings typically have lower-than-expected outcomes, a lower-than-expected rate is bound to affect case-level indicators 19

20 Basic Rules of Thumb for Interpreting Your PEM Report 3. Analyze case-level indicators Are any of the facility subscores (% of target) less than 50%? This means that less than 50% of your cases are meeting or exceeding their CMGspecific expectation for that variable 20

21 PEM Targets We are frequently asked how to identify the value(s) used for each of the targets utilized in the PEM For the 2015 PEM, UDSMR published these targets for subscribers in the Annual Program Evaluation Model (PEM) Reports section of the UDS Central website 21

22 PEM Scores vs. National Percentiles for Annual PEM Reports ( ) Year 0% 25% 50% 75% 100%

23 What Does It Mean to Be in the Top 10%? If your facility is in the top 10%, its percentile rank is between 90 and 100 UDSMR considers all facilities that rank in the top 10% of the database as top program performers, recognizing their delivery of quality patient care that is effective, efficient, timely, and patientcentered 23

24 Minimum PEM Score Required for Top 10% Designation The PEM score required to achieve the top 10% has increased since 2007 and, in recent years, has begun to level off. This indicates that facilities as a whole have increased the quality of their outcomes over time. 24

25 Discharge FIM Total Distribution: Top 10% PEM Facilities ( ) Discharge FIM Subscore 25

26 FIM Change Distribution: Top 10% PEM Facilities ( ) FIM Change Subscore 26

27 LOS Efficiency Distribution: Top 10% PEM Facilities ( ) LOS Efficiency Subscore 27

28 D/C to Community Distribution: Top 10% PEM Facilities ( ) Discharge to Community Subscore 28

29 D/C to Acute Care Distribution: Top 10% PEM Facilities ( ) Discharge to Acute Subscore 29

30 Setting Realistic Goals Not every facility can be in the top 10%, but every facility can improve Improvement comes from setting goals appropriate for your facility Possibility for Improvement: Category (N=767) (N=761) (N=752) (N=752) % of facilities that improved 51.4% 48.2% 50.5% 50.5% % of facilities that improved their percentile rank by at least 10% 25.2% 22.7% 23.1% 21.1% Maximum rank improvement Maximum score improvement

31 How to Identify Opportunities for Improving Your PEM Score Step 1: Determine the accuracy of your CMGs PEM expectations are based on CMGs CMGs are based on IGCs and admission FIM ratings Therefore, the validity of your comparison to expectation is highly dependent on impairment group coding and FIM rating accuracy Utilize the UDSMR report set to identify any possible rating inaccuracies Etiologic Diagnosis by Impairment Group Code Listing Scoring Report Frequency of FIM Ratings Report 31

32 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging For case-level indicators: Generate an on-demand report for the indicator Can you identify RICs or CMGs whose average value is less than the expected national average? Can you use the case listing to identify patients who may not have met or exceeded expectations? Can you identify characteristics of groups/patients that create less-thanexpected outcomes? 32

33 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging For case-level indicators: Generate an on-demand report for the indicator 33

34 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging For case-level indicators: Generate an on-demand report for the indicator 34

35 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging For the case-level indicators discharge FIM total and FIM change: Generate the Scoring Report Can you identify FIM items whose average values are different from the expected national averages? 35

36 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging For the case-level indicators discharge FIM total and FIM change: Generate the Frequency of FIM Ratings Report Can you identify patterns for FIM items where a certain level is rated more or less frequently than the national expectation? 36

37 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging For the case-level indicators discharge FIM total and FIM change: Are you utilizing the Informatics tab for management to national expectations? Current Case FIM Scores by RCMG 37

38 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging For the case-level indicator LOS efficiency: Determine whether less-than-expected results are created by: FIM change LOS Other factors 38

39 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging For facility-level indicators: % discharge to community: What barriers prevented the patient from being discharged to the community? % discharge to acute care: What are the characteristics of patients discharged to acute care? Are some of these instances preventable? 39

40 How to Identify Opportunities for Improving Your PEM Score Step 2: Determine which indicator is lagging Typical issues identified: Outcomes for acute care discharges, short stays, and early transfers Are your patients likely to be discharged to acute care or other settings prior to the completion of rehabilitation services? FIM ratings Are you using an interdisciplinary approach and documenting all episodes? Are you using the twenty-four-hour period that yields the highest overall discharge FIM total? 40

41 How to Identify Opportunities for Improving Your PEM Score Step 3: Improve your processes Are specific policies and procedures affecting your outcomes? Pre-admission screening/admission practices Interdisciplinary assessments Safety guidelines Discharge planning 41

42 How to Identify Opportunities for Improving Your PEM Score Step 4: Schedule a PEM consultation If you are looking for an in-depth interpretation of your PEM Report, we offer an hour-long PEM consultation A clinician and a statistician from UDSMR will conduct a detailed review of your facility s reports, identifying areas for improvement The clinician and the statistician will conduct an hour-long conference call to share findings with your facility If you want more information or would like to schedule a consultation, contact Carole Stickels at

43 Thank You! Any Questions? Troy Hillman: Sarah Mullin: Analytical Services Group: