Dr Hazel Roddam Evidence-Based Practice: approaches to measuring the impact of therapy interventions. Dr Hazel Roddam, 2014 1
Workshop outline 1. Definitions of Evidence-Based Practice (EBP) 2. Inherent challenges 3. Definitions of Practice-Based Evidence (PBE) 4. Research & service evaluations 5. Identifying opportunities 6. Personal action planning Dr Hazel Roddam, 2014 2
Dr Hazel Roddam, 2014 3
Why do we need EBP? It is not expected that all clinicians will be research-active, but it is expected that all clinicians will be active users of research Supporting Research and Development in the NHS A.Culyer, 1994 Dr Hazel Roddam, 2014 4
What is the EBP agenda? Political Professional Clinical Facing unfamiliar territory Things are not working Skeat & Roddam, 2010 Dr Hazel Roddam, 2014 5
Definition of EBP The conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients Sackett et al 1997 Dr Hazel Roddam, 2014 6
Research Clinician Client Dr Hazel Roddam, 2014 7
http://www.asha.org/members/ebp/intro/ Dr Hazel Roddam, 2014 8
Sackett s 5 steps model of EBP 1. Frame a clinically-focused question 2. Search for the best quality evidence 3. Appraise the evidence 4. Implement changes if appropriate 5. Evaluate effectiveness of practice Sackett et al 1997 Dr Hazel Roddam, 2014 9
Skills gaps and training needs Skills Experience 0-5 rating Confidence 0-5 rating Priority Actions Framing question Literature search strategy Access to publications Critical thinking Changing practice Measuring change Other Dr Hazel Roddam, 2014 10
Research Clinician Client Dr Hazel Roddam, 2014 11
What questions can research answer? Does our practice work? Treatment efficacy Treatment effectiveness How well does it work? Statistical significance Clinical significance Is it worthwhile? Clinical effectiveness Cost effectiveness Client perceptions Dr Hazel Roddam, 2014 12
Mapping the research gaps Reilly, Douglas & Oates Evidence Based Practice in Speech Pathology 2004 Urgent need for a more strategic approach for the future to acknowledge complexity, heterogeneity, low incidence cases relevant Outcome Measures for indirect interventions including education/training approaches to address the limited evidence validated for local populations Dr Hazel Roddam, 2014
Research evidence for local populations Don t assume there is a lack of research evidence before you have looked carefully Don t let colleagues use lack of research as a defence for not making any efforts to keep their practice updated Consider a working group to undertake a strategic mapping of the existing research evidence that is validated for local practice 14 Dr Hazel Roddam, 2014
Research Clinician Client Dr Hazel Roddam, 2014 15
A new definition of EBP It may be more helpful to consider EBP as a way of thinking, rather than as a body of facts McCurtin & Roddam, 2012 Dr Hazel Roddam, 2014 16
Knowledge in practice The messy reality of practice Knowledge-in-practice-in-context John Gabbay and Andree le May, 2011 Dr Hazel Roddam, 2014 17
Constructing Clinical mindlines Clinicians Are aware of formal sources of evidence Don t use those sources whilst in practice Don t ignore them Acquire new information from wide range of sources Balance many competing roles in decision-making Don t consciously apply deductive logic Rely on pattern recognition Co-construct mindlines within communities of practice Dr Hazel Roddam, 2014 18
Research Clinician Client Dr Hazel Roddam, 2014 19
Whose outcomes? Outcomes for the individual client/service user Outcomes for the patient/client group Outcomes for the service Outcomes for the commissioners 20
Range of research outcomes Efficacy The outcome of an intervention that is administered and measured in an ideal setting Effectiveness The outcome of an intervention that is administered and measured in a real world setting Efficiency Is the intervention cost-effective and time-effective Effect size Is intervention 1 better than intervention 2 Experience Patient and/or professionals 21
World Health Organisation, 2001 Impairment Activity limitation Social participation Well-being & Quality of Life 22
http://www.who.int/classifications/icf/training/icf beginnersguide.pdf 23
Measuring patient experiences Patient reported outcome measure (PROM) Patient reported experience measure (PREM) Psychometric aspects of tool development Appropriacy Reliability Validity Sensitivity/responsiveness to change Inherent challenges to self-report measures Inherent challenges to proxy-report measures 24
Generating evidence for SLT practice Plsek,PE & Greenhalgh,T (2001) The challenge of complexity in healthcare BMJ 323: 625-628 Dr Hazel Roddam, 2014 25
Methodological challenges Case presentation/populations Case management roles Interventions 1 Concordance Contexts Ethical decision-making 2 1 Medical Research Council (2000, 2008) Complex interventions for health 2 David Seedhouse (2005) Values-Based Decision-Making in the Caring Professions Chichester: Wiley Dr Hazel Roddam, 2014
Defining Practice-Based Evidence Developing an evidence base for innovative, quality and safe practice (DH, 2012; Westwood, 2013) Improving health outcomes Enhancing experiences of healthcare Creating role models/champions Dr Hazel Roddam, 2014 27
Building a research workforce Whelan, 2013 Leadership Leading research Collaboration Evidence-Based Practice Dr Hazel Roddam, 2014 28
Contributing to the research evidence base Using systematic service evaluation approaches to capture good practice exemplars Using research designs for Single case studies Small group studies Replication studies of published clinical research Data collection on larger/multi-centre research projects Development and validation of assessment tools 29 Dr Hazel Roddam, 2014
Systematic reporting of cases from clinical practice The CARE Guidelines set out quality criteria for how clinical cases can be reported and disseminated to make a valuable contribution to the collective research evidence base http://www.ncbi.nlm.nih.gov/pubmed/24 155002 30 Dr Hazel Roddam, 2014
Creating Practice-Based Evidence (PBE) This is an easy-read book directed at busy therapists in clinical practice it is not a textbook aimed at university students. It covers practical guidance on collecting and using routine clinical outcomes data to demonstrate effective practice Creating Practice-based Evidence: a Guide for SLTs Corinne Dobinson and Yvonne Wren 2013 J& R Press Ltd 31 Dr Hazel Roddam, 2014
Sharing good practice through PBE Identifying good practice Documenting outcomes Listening to the patient/client voice Developing collaborative partnerships Disseminating the evidence Dr Hazel Roddam, 2014 32
Effective partnership working Patients Academics Clinicians Dr Hazel Roddam, 2014 33
Key messages Be clear about the purpose of measuring outcomes Always use outcome measures with an understanding of their inherent limitations Ensure they meet the needs of your patient/client group and your service Identify how you will analyse and communicate the findings 34
Personal skills assessment My experience in Outcome Measures Relevance to my team & organisation My confidence My training needs Where I could access support? Strengths How am I already using these skills? How am I already sharing these skills? What support could the SLT Association offer? 35
Final thoughts The first step is to become reflective and critical reviewers of our own practice Professor Pam Enderby in Roddam & Skeat, 2010 Dr Hazel Roddam, 2014 36
Additional sources and links ASHA list of Guidelines and Systematic Reviews http://www.asha.org/members/ebp/compendium/ Cochrane Library of systematic reviews http://www.thecochranelibrary.com/view/0/index.ht ml Speech Bite free access to research publications http://speechbite.com/ 37 Dr Hazel Roddam, 2014
Useful References 1 Bury & Mead, 1998, Evidence-Based Healthcare: A Practical Guide for Therapists. Oxford: Butterworth-Heinemann Dopson, Fitzgerald, Ferlie, Gabbay & Locock, 2002, No magic targets! Changing clinical practice to become more evidencebased. Health Care Management Review 27(3): 35-47 Estabrooks, Floyd, Scott-Findlay, O Leary & Gushta, 2003, Individual determinants of research utilization: a systematic review. Journal of Advanced Nursing 43 (5): 506-520 Gabbay, J. and Le May, A. 2011 Practice-Based evidence for healthcare: Clinical Mindlines. Routledge Roddam, H. and Skeat, J. (Eds) 2010 Embedding Evidence-Based Practice in Speech and Language Therapy: International examples. Wiley-Blackwell (ISBN: 978-0-470-74329-4) Dr Hazel Roddam, 2014 38
Useful References 2 Spek, B, Wieringa-de Waard, M., Lucas, C. and van Dijk, N. (2013) Teaching evidence-based practice (EBP) to speech-language therapy students: are students competent and confident EBP users? International Journal of Language & Communication Disorders Volume 48, Issue 4, pages 444 452 Vallino-Napoli & Reilly, 2004, Evidence-based health care: A survey of speech pathology practice. Advances in Speech and Language Pathology 6 (2): 107-112 Zipoli & Kennedy, 2005, Evidence-based practice among speech-language pathologists: Attitudes, utilisation and barriers. American Journal of Speech- Language Pathology 14: 208-220 Dr Hazel Roddam, 2014 39