UAS-NY Data Validation-TBI 2016-17
Audit Scope IPRO conducted a validation of the UAS-NY Community Assessment and Functional Supplement tools, for a sample of the TBI waiver population in NYS. TBI eligible population consisted of approximately 360 members, for whom a Nursing Facility Level of Care (NFLOC) score of 5 or greater was not evident from UAS-NY scoring. The audit sample was 96 records, across the 9 Regional Resource Development Centers (RRDCs). A total of 44 UAS-NY elements were reviewed. The total number of reviewed elements was 4,224 (96 member records X 44 elements per record) 2
Audit Scope The review elements were in the following UAS-NY domains: Community Assessment Section B-Cognition Section C-Communication and Vision Section D-Mood and Behavior Section F-Functional Status Section G-Continence Section H-Disease Diagnosis Section J-Nutritional Status Functional Supplement Section F-Disease Diagnoses Section G-Health Conditions Section I: Skin Condition 3
Audit Scope The items validated included all of the Nursing Facility Level of Care (NFLOC) score elements, plus some additional elements requested by the DOH. Validation was accomplished through a review of: Service plans Patient Review Instruments (PRIs) Comments from UAS-NY CHA and FS assessments Home health agency documentation (if available) 4
Audit Scope A significant number of the elements reviewed contain a three (3) day window timeframe, requiring that the behavior or activity had to have occurred within the 3 day period prior to and including the assessment date. For these elements, documentation was often found to support, agree or disagree with the assessor but firm decisions were unable to be made as the documentation was dated outside of the 3 day window. 5
Audit Scope Rates for each element were calculated in the following ways: Rate of IPRO reviewer responses for which the reviewer agreed with the UAS-NY assessor s rating (excluding those items with an insufficient evidence rating due to the 3 day window requirement) Rate of IPRO reviewer responses for which the reviewer would have scored a higher rate of acuity than the assessor s rating (excluding those items with an insufficient evidence rating due to the 3 day window requirement) Rate of IPRO reviewer responses for which the reviewer would have scored a lower rate of acuity than the assessor s rating (excluding those items with an insufficient evidence rating due to the 3 day window requirement) Overall rate of insufficient evidence due to the 3 day window requirement Overall rate of responses with no evidence one way or another (IPRO reviewer unable to make a determination) Rate of IPRO reviewer responses for which the reviewer agreed with the UAS-NY assessor s rating (including those items with an insufficient evidence rating due to the 3 day window requirement) Rate of IPRO reviewer responses for which the reviewer would have scored a higher rate of acuity than the assessor s rating (including those items with an insufficient evidence rating due to the 3 day window requirement) Rate of IPRO reviewer responses for which the reviewer would have scored a lower rate of acuity than the assessor s rating (including those items with an insufficient evidence rating due to the 3 day window requirement) 6
Audit Results NFLOC scores comparison: The NFLOC scores listed for each member in the sample file submitted by the DOH matched the scores on the UAS-NY reviewed for each member Validation Results: 87.7% of reviews indicated agreement with the assessor s findings (including elements with supporting evidence outside of the 3 day window timeframe) 9.3% of reviews disagreed with the assessor, supporting a higher level of acuity (including elements with supporting evidence outside of the 3 day window timeframe) 7
Audit Results, cont d 3% of reviews disagreed with the assessor, supporting a lower level of acuity (including elements with supporting evidence outside of the 3 day window timeframe) Most of the disagreements supporting a higher level of acuity were with: Cognitive Skills for Daily Decision Making Short Term Memory Procedural Memory Meal Preparation Managing Medications Bathing Personal Hygiene Dressing Upper Body/Dressing Lower Body 8
Audit Results Significance testing was done (z test, p value<0.001) to determine if there were any significant differences between the rates of answers from the UAS-NY assessors and the IPRO reviewers. 9
Audit Results Cognitive Skills for Daily Decision Making Cognitive Skills for Daily Decision Making 8 7 71.6% 6 5 5 4 3 31.3% 1 13.5% 3.2% Independent 14.7% Modified Independence Minimally Impaired 4.2% 9.5% Moderately Impaired 1.0% 1.1% Severely Impaired 10
Audit Results-Memory Problems 10 Procedural Memory 10 Short-Term Memory 95.7% 8 80.2% 8 66.7% 6 49.0% 51.0% 6 4 4 33.3% 19.8% Yes, memory OK Memory problem 4.3% Yes, memory OK Memory problem 11
Audit Results-Mood and Behavior 10 10 96.8% Wandering 10 91.7% 83.3% Verbal Abuse 8 8 6 6 4 4 14.6% Not Present 3.2% Present, but not in last three days Not Present 5.2% Present, but not in last three days 3.1% 2.1% Exhibited in 1-2 of last three days 12
Audit Results-Mood and Behavior, cont d 10 10 95.8% Physical Abuse 96.9% 10 88.4% Resists Care 8 8 6 6 4 4 Not Present 4.2% Present, but not in last three days Not Present 7.4% 2.1% 1.0% Present, but not in last three days 4.2% Exhibited daily in last three days 13
Audit Results-IADLs -Meal Preparation (Performance) - Meal Preparation (Performance) 2.1% 1.0% 11.5% 13.5% 12.5% 9.4% 1.0% 49.0% Independent Setup Help Only Supervision Limited Assistance Extensive Assistance Maximal Assistance Total Dependence Did Not Occur 10.9% 23.9% 12.0% 16.3% 37.0% Independent Setup Help Only Supervision Limited Assistance Extensive Assistance Maximal Assistance Total Dependence Did Not Occur 14
Audit Results-ADLs - Bathing 3.1% 1.0% - Bathing 9.4% 5.2% 81.3% Independent 4.4% Setup Help Only 16.5% Supervision Limited Assistance 4.4% Extensive Assistance Maximal Assistance Total Dependence Did Not Occur 74.7% Independent Setup Help Only Supervision Limited Assistance Extensive Assistance Maximal Assistance Total Dependence Did Not Occur 15
Audit Results-ADLs, cont d 10 8 88.5% 76.9% Personal Hygiene 10 8 96.9% 84.4% Dressing Upper Body 6 6 4 4 7.3% 2.2% 16.5% 3.1% 1.0% 4.4% Independent Setup Help Only Supervision Limited Assistance 11.1% 2.1% 1.1% 1.0% 3.3% Independent Setup Help Only Supervision Limited Assistance 16
Audit Results-ADLs, cont d 10 8 96.9% 84.4% Dressing Lower Body 10 8 91.7% 82.3% Walking 6 6 4 4 3.1% 1.1% 11.1% 3.3% Independent Setup Help Only Supervision Limited Assistance Independent 5.2% 3.1% Setup Help Only 6.3% 5.2% 1.0% 2.1% 3.1% Supervision Limited Assistance Did Not Occur 17
Audit Results-Locomotion and Balance 10 93.8% 87.5% Locomotion 10 96.9% 83.3% Balance 8 8 6 6 4 4 5.2% 2.1% 1.0% 6.3% 4.2% Independent Setup Help Only Supervision Limited Assistance Not Present 5.2% 2.1% 1.0% Present, but not in last three days 11.5% Exhibited daily in last three days 18
Audit Results The majority of these elements are included in NFLOC score calculation, except for Meal Preparation, Personal Hygiene, Walking, Balance. 19
Conclusions from Audit High agreement rate: Nearly 88% of all elements validated For the majority of elements where disagreement was found, a higher level of acuity was observed by IPRO reviewers Notable levels of disagreement found with Cognitive Skills for Daily Decision Making, Short Term Memory, and Procedural Memory (all of these items are included in the NFLOC score) Other notable levels of disagreement, generally supporting a higher level of acuity, found with some elements not included in the NFLOC (e.g. Meal Preparation) 20
Conclusions from Audit Significant assistance with IADLs, not only with Meal Preparation and Managing Meds, but also with paying bills and shopping, was observed. These elements are not factored into the NFLOC score and were not part of the validation. A subset of the ADLs indicated some level of disagreement (Bathing, Dressing Upper Body, Dressing Lower Body, Personal Hygiene, Walking and Locomotion). Mood disorders were found to be prevalent in the cases reviewed, mood is not specifically addressed in the NFLOC score. Balance problems (e.g. unsteadiness while walking) were observed in the reviews, also not validated or captured in the NFLOC score. A history of substance abuse was observed in a number of records, not necessarily an active problem 21
Conclusions from Audit Recommendations RRDC staff should be consulted for case history information prior to the assessment. IPRO reviewers had access to the initial service plans, which provided a considerable amount of the member s history, including the nature of the original injury or incident prompting the TBI condition. Where the NFLOC score does not result in an eligibility determination, the assessment may be further enhanced by the addition of a subsequent clinical assessment or evaluation focusing on: Cognitive and functional deficits IADL challenges, such as with meal preparation, paying bills, shopping, managing medications Mood disorders Balance concerns Competing health needs Coordination of care across providers 22
Conclusions from Audit Recommendations (continued) Concurrently, the member record should be reviewed for a history of substance abuse, as there may be some potential for mood disorder development and other emotional disturbances. Consider additional training, with some focus on the UAS-NY elements found to have higher levels of disagreement between UAS-NY assessors and IPRO reviewers. 23
Contacts Tom LoGalbo Director, Managed Care (516) 326-7767 Extension 349 tlogalbo@ipro.org Jeanne Alicandro, MD Medical Director, Managed Care (516) 326-7767 Extension 352 jalicandro@ipro.org 24