Implementing an Innovative Approach to Outcomes Measurement ACCOM: Australian Community Care Outcomes Measure Paper presented at the Conference Quality in Aged Care. Criterion Conferences/Council On The Ageing; Sydney, 8-9 March 2016 Prof Michael Fine* Dr Beatriz Cardona* Dr Adam Stebbing Macquarie University Prof Kathy Eagar Peter Samsa Cathy Duncan AHSRI, University of Wollongong * Authors and presenters
Outcomes and Quality Inputs Service Production Process Includes 1. Funds (public and private) 2. Clients 3. Staff Organisation and conduct of work (care activities) Quality measures -Focus on quality of service delivery process Short-term or interim measures Outputs Types and amounts of services or other products provided. e.g. hours of care provided; number of meals delivered Efficiency measures Outcomes eg. Individual wellbeing; Health, Family/carer wellbeing; Residential outcome (stay at home). Effectiveness, impact measures. Longer-term tests of quality. Summary measure Production of Welfare Framework: B. Davies and M. Knap et al. (1976+)
Concept of outcomes widely used BUT + Frequently misconstrued, misapplied, + Often reinvented, renamed, + Term sometimes confused with outputs RBA: Results Based Accountability http://raguide.org/ Social Impact Assessment http://www.csi.edu.au/ Social Return on Investment (SROI) GRI Guidelines, Social Accounting Most Significant Change Third Sector Performance Dashboard SE Balanced Scorecard Results Based Accountability Project Logic and Developmental Evaluation
Outcome measurement Result, impact or consequence of a process of service provision Provide evidence of impact of service delivery Objective definition and measurement Documentation of change over time in crucial dimensions of the lives of service recipients or their carers, such as personal wellbeing, health, capacity to participate in activities and sustain meaningful personal relationships, and the viability of continued residence at home
Understanding the impact of particular interventions Match samples (eg clients) at T 0 (time zero) Compare outcomes at T 1 (time one) Standard T o T 1 Innovation T o T 1
Background to the Project ARC funded project - ARC Linkage Scheme Led by Prof Michael Fine, Macquarie University and Prof Kathy Eagar, Australian Health Services Research Institute (AHSRI) University of Wollongong in partnership with BaptistCare NSW & ACT, Community Options Australia, Aged and Community Services NSW and ACT, Kincare, Whiddon Group.
Aim and Tools To develop, test and validate a practical tool to measures outcomes of community care in Australian settings: The Australian Community Care Outcomes Measure (ACCOM)* An outcomes measurement tool for community age care that can be used during service delivery. ACCOM Data collected on an individual basis aggregated to service level and comparative system overview. *ACCOM uses the Adult Social Care Outcomes Toolkit* (PSSRU, University of Kent) as the principal basis of measurement of outcomes. ASCOT is the sole property of the University of Kent. Test and adapt to Australian circumstances + Demographic Data + Data on Needs - Functional Screen (Assessment) + Other basic outcome data Initially: Consumers and Case Managers future plans include development and testing of carers ACCOM, dementia specific, CALD, ATSI; -- HSP testing etc.
Pilot 2012-13 Research Stages The collaborative research project: to test, document and apply standard measures for the determination of client outcomes in case managed community care for older people. Stage One, the first year of the research program, a pilot study undertaken in three well established and experienced case management services in NSW. Stage Two, conducted over two years to further trial the successful approaches identified in the first stage through case managed and other suitable services for older people across the State. Stage Three: Roll out + further development NSW; National. Carer s measure, CALD, IT, further development ARC Linkage 2015-2016 2017+
Focus on easy to use tool - initial priority: one for use by staff and consumers Current Study: Testing based on use of ASCOT, demographic and function information + open questions: (sample up to 400) ACCOM administered twice. First round (Dec 2015- April 2016) test of tool viability, useability. Second round (c3 months later) test for sensitivity of tool to measure changes in outcome. Challenges: Commonly agreed goals (outcomes) to be measured Which domains? Are current domains sufficient? Are more needed? Simple to use, valid and reliable measures Validation. Is data held? If not, is it v. easily collated? How to capture it? Is a common score system possible? Trust vs Risk of Misuse Self-report systems open to bias. How to manage this?
Significance of the ACCOM tool A tool that measures changes over time in terms of capabilities and functionalities. It links to CDC emphasis on the goals of care, identifying changes associated with this goal at the start and end of an episode of care. Measures consumer s perceptions of change as well as case managers (staff). This dual process is a key innovation of the tool. -provides organisations with an instrument to identify discrepancies in case manager-client perceptions of outcomes, identify unmet consumer needs and care plan priorities. Change scores across selected key indicators provide evidence of outcomes. Measures enable service providers to assess and evaluate the impact of their interventions. It allows services to adopt systems for more rigorous and evidence-based service development.
Some early results The following slides present some of the results from the first ACCOMs, completed December 2015- March 2016 These results are illustrative only. The sample size is small and is not representative. Data presented focusses on the Quality of Life outcomes ollected through the ASCOT component of the survey.
COMPARING RESPONSES
Safety
Social contact
Food and drink
Useability Issues A total of 64 questionnaires were returned to the research team during the first 12 weeks of the field test. All the items in the instrument were completed by 62 respondents. ASCOT tool plus the 5 additional health and feedback questions. MORE THAN 20 MINUTES CLIENTS: TIME TO COMPLETE ACCOM 2 CASE MANAGERS BETWEEN 15 AND 20 MINUTES Less than 5 minutes Between 5 and 10 minutes 5 BETWEEN 10 AND 15 MINUTES BETWEEN 5 AND 10 MINUTES 6 30 Between 5 and 10 minutes Between 10 and 15 minutes Between 15 and 20 minutes Less tan 5 minutes 55 Less tan 5 minutes Between 5 and 10 minutes LESS THAN 5 MINUTES 26 More than 20 minutes 0 10 20 30 40 0 20 40 60
Completing the ACCOM Level of Difficulty Consumers Case Managers Very easy Easy Straightforward 4, 6% 1, 2% Very easy Easy 10, 17% 22, 34% 21, 33% Straightforward Not very easy 24, 41% Not easy 16, 25% 25, 42%
Completing the Survey Was help needed? Consumers Help Completing the ACCOM 11, 17% By myself 12, 19% 41, 64% With the help of family or friend With the help of a care worker
What the tool measures: Capabilities DOMAIN Control over daily life Personal cleanliness and comfort Food and drink Personal safety Social participation and involvement Occupation Accommodation cleanliness and comfort Dignity DEFINITION I can choose what to do and when to do it. I have control over my daily life and activities. I feel clean, comfortable and I look and feel like myself. I feel I have a healthy diet with variety and foods I usually eat. This reflects my culture and my preferences. I have enough to eat and drink at regular times through the day. (Does not refer to alcohol)., I feel safe and secure inside and outside my home. This means feeling free from fear of physical abuse, emotional abuse, financial abuse, falling or other physical harm I am content with my social situation. I feel I have meaningful relationships with my friends and family and I feel part of the community (should this be important to me) I feel I have access to a range of meaningful activities. This may be paid work, unpaid work, caring for others or hobby and leisure activities. I feel my environment including all the rooms are clean and comfortable I feel I am being listened to and treated with respect by my service provider. My views and preferences are valued and taken into consideration.
Functionality: The Functional Screen Current instrument: ACCOM Assessment Preview instrument ClientID Organisation BaptistCare Community Options Kincare Whiddon 5A. Part of care plan? Yes No Case Manager Name Sum of FP1 to FP5 (electronic version calculates this automatically) Date information collected Functional profile Can you do housework Without help (can clean floors etc.)? With some help (can do light housework but need help with heavy housework)? Or are you completely unable to do housework? Can you walk... Without help (except for a cane or similar)? With some help from a person or with the use of a walker, or crutches etc. Or are you completely unable to walk? 1A. Part of care plan? Yes No 6A. Part of care plan? Yes No Can you get to places out of walking distance... Without help (can drive your own car, or travel alone on buses or taxis)? With some help (need someone to help you or go with you when travelling)? Or are you completely unable to travel unless emergency arrangements are made for a specialised vehicle like an ambulance? Can you take a bath or shower... Without help? With some help (e.g. need help getting into or out of the bath/shower)? Or are you completely unable to bathe yourself? 2A. Part of care plan? Yes No Can you go out for shopping for groceries or clothes (assuming you have transportation)... Without help (taking care of all shopping needs yourself)? With some help (need someone to go with you on all shopping trips)? Or are you completely unable to do any shopping? 7A. Part of care plan? Yes No 3A. Part of care plan? Yes No Does the person have any memory problems or confusion? (Assessor judgment) Yes No Can you take your own medicine... 4A. Part of care plan? Without help (in the right doses at the right time)? With some help (able to take medication if someone prepares it for you and / or reminds you to take it) Or are you completely unable to take your own medicines? Yes No Does the person have behavioural problems (e.g. aggression, wandering or agitation) Assessor judgment Yes No Can you handle your own money... Without help (write cheques, pay bills etc.)? With some help (manage day-to-day buying but need help with managing your chequebook and paying your bills)? Or are you completely unable to handle money? Total FP score (electronic version calculates automatically)
Summary: Why measure outcomes in Community Care? An outcome is defined as a change in an individual or a group of individuals that can be attributed, at least in part, to an intervention or series of interventions (AHMAC, 1993 in Eagar, 2014) Outcome measurement, as opposed to just counting service activities (outputs), is essential for the measuring the quality of community care. Monitoring outcomes has been shown in research on LTC and in health settings to lead to improvements in service delivery over time. So far the study shows encouraging results in the areas of feasibility and ease of use for staff and consumers. Given the small sample collected so far, and the limited analysis, still too early to report research results or service quality reports.
Further Resources and Information about ACCOM Website: http://www.agedcareoutcomes.net/ The website contains reports, conference presentations and power points by project partners and the research team. AAG 2016 National Conference, Canberra 02 04 November 2016 National Convention Centre http://aagconference.asn.au/