From the literature to evidencebased

Medicine, Nursing and Health Sciences From the literature to evidencebased care Sue Brennan, PhD Sue.brennan@monash.edu Australasian Cochrane Centre, School of Public Health & Preventive Medicine Monash University Acknowledgement: Joanne McKenzie (School of Public Health and Preventive Medicine, Monash University) and Marije Bosch (Central Clinical School, Monash University) for sharing slides and ideas. Staff of the Australasian Cochrane Centre. Please do not reproduce slides without the permission of the presenter sue.brennan@monash.edu

Translating research into evidence-based care is difficult! Consistent finding of a gap between research and practice McGlynn et al found that patients in the USA received 55% of recommended care, with similar findings globally 20-30% of patients may get care that is not needed or care that is potentially harmful In Australia, an analysis of a sample of health care encounters showed Adults received appropriate care at 57% (95% CI 1%- 43%) of encounters Varied considerably by condition Grimshaw et al 2012 Implementation Science Gap between discovery and patient care. CEO NHMRC Health Research Translation Newsletter - December 2013 Runciman et al 2012 MJA 197 (2) 22 July 2014 2

Overall goals of the presentation A super-quick introduction to implementing evidence Tools, tips, and traps to consider at each step 22 July 2014 3

From the literature to evidence-based care 1. Knowledge for translation Systematic reviews: why translate bodies of evidence? Tools that provide accessible, decisionrelevant information from research: guidelines, clinical decision support, decision aids for patients Appraising bodies of evidence: GRADE, AGREE 2. Evidence-based implementation approaches Systematic reviews of quality improvement and continuing professional education approaches Interpreting other study designs when there isn t a systematic review 3. Implementing changes to care Understanding and addressing barriers and facilitators of change Assessing the impact of implementation efforts 22 July 2014 4

Knowledge to action Graham, J Eval Clin Pract 2006, 26:13-24 22 July 2014 5

Knowledge for translation Primary research Single studies (e.g. trials, observational studies) Analysis of safety & quality data Analysis of individual, local, regional or national data can be used to: Identify variation in care (possible areas for quality improvement) Monitor the impact of introducing changes to care Graham, J Eval Clin Pract 2006, 26:13-24 22 July 2014 7

Knowledge for translation Primary research Single studies (e.g. trials, observational studies) Analysis of safety & quality data Systematic reviews Synthesis of individual studies (with or without meta-analysis) Uses all available evidence Using findings from systematic reviews: Increases certainty about the effects of an intervention Helps identify bias in the conduct of studies; reduces the chance you will be misled Provides evidence for individualising care through exploration of whether the effects of the intervention vary in patient subgroups Saves time (searching, appraisal, synthesis is done for you). Graham, J Eval Clin Pract 2006, 26:13-24 22 July 2014 9

22 July 2014 10

Acknowledgement for slide: Prof Sally Green Why translate bodies of evidence not results of individual studies? Results of individual studies need to be interpreted and translated alongside the body of evidence Single studies rarely provide sufficient evidence for practice change (especially small studies in specific populations) Early results can lessen over time studies require replication and synthesis Your Health Net: http://navigatingeffectivetreatments.org.au/. Developed by CHCP 22 July 2014 11

22 July 2014 12

Of 49 highly cited studies, 45 claimed the intervention was effective. Of these: 7 (16%) were contradicted by subsequent studies 7 (16%) found stronger effects than subsequent studies 20 (44%) were replicated by subsequent studies 11 (24%) remained largely unchallenged Of those where subsequent studies contradicted or showed weaker effects than the highly cited study: 5/6 were non-randomised studies 9/39 were randomised trials 22 July 2014 13

22 July 2014 14

22 July 2014 15

22 July 2014 16

Knowledge for translation Primary research Single studies (e.g. trials, observational studies) Analysis of safety & quality data Systematic reviews Synthesis of individual studies (with or without meta-analysis) Bodies of evidence Tools for translation Resources that provide accessible, decision-relevant information from research e.g. evidence-based guidelines, decision-support for clinicians, patient decisions aids & information Graham, J Eval Clin Pract 2006, 26:13-24 Relevance to decision-making 22 July 2014 17

Knowledge tools & products 1. Guidelines for clinical practice 2. Decision support tools for clinicians at point of care 3. Patient decision aids & information 22 July 2014 18

Tools for guidelines For rating the quality of the evidence underpinning guideline recommendations Used to summarise findings from systematic reviews Support interpretation of findings (faster, easier) For appraising the guideline development process Useful if you are considering using an existing guideline 22 July 2014 19

22 July 2014 20

See www.gradeworkinggroup.org Quality of evidence based on 5 domains: 1. Risk of bias 2. Indirectness (e.g. relevant population, intervention, comparator, outcomes) 3. Inconsistency (statistical heterogeneity: studies have non-overlapping confidence intervals, I 2, Chi 2 ) 4. Imprecision (uncertainty: wide confidence intervals, small number of participants in pooled sample) 5. Publication bias (reporting bias detected statistically, or suggested because small studies consistently report effects favoring the intervention) 22 July 2014 21

Knowledge tools & products 1. Guidelines for clinical practice 2. Decision support tools for clinicians at point of care 3. Patient decision aids & information 22 July 2014 22

Knowledge tools & products 1. Guidelines for clinical practice 2. Decision support tools for clinicians at point of care 3. Patient decision aids & information 22 July 2014 23

Evidence based implementation approaches Quality improvement and continuing professional development (practical efforts to improve care, using local knowledge, research-based evidence or both) 22 July 2014 25

Evidence based implementation approaches Terms used in the literature: translating (research, evidence, knowledge..) into practice implementing (research, evidence, knowledge..) in practice practice (behaviour) change knowledge to action Increasing recognition that methods used for QI & CPD need to be supported by evidence of their effectiveness Substantial opportunity cost in investing in ineffective strategies 22 July 2014 26

22 July 2014 27

22 July 2014 28

Designs (experimental) Randomised designs Individually randomised Cluster randomised Stepped wedge cluster randomised Other designs Non-randomised designs Uncontrolled before and after studies Controlled before and after studies Interrupted time series 22 July 2014 29

Designs (experimental) Randomised designs Individually randomised Interpret with caution: can t exclude the possibility that observed Cluster effects randomised are due to factors other than the intervention (e.g. changes over time, differences between the control and intervention groups) Stepped wedge cluster randomised Non-randomised designs Uncontrolled before and after studies Controlled before and after studies Interrupted time series 22 July 2014 30

22 July 2014 31

Look out for hallmarks of quality in reporting of QI studies 22 July 2014 32

Or use a checklist e.g. 22 July 2014 33

The process... Identify new evidence or quality problem Symptoms Assess & priotise factors that influence practice and change (barriers & enablers) Select or develop intervention Diagnosis Treatment Optimal care / practice change Cure Wensing, Implementation Science 2011, 6:103 22 July 2014 35

Theoretical Domains Framework Michie 2005 22 July 2014 36

22 July 2014 37

A range of methods are used to collect data on barriers and enablers: Individual interview methods Group interview methods Literature studies Questionnaire survey methods Observation Mixed-methods.. NICS Feb 2006; Wensing 2009 22 July 2014 38

Which intervention components? Barrier is... Consider... Lack of knowledge Perception / reality mismatch Lack of motivation Beliefs / attitudes Systems of care Education session Decision aids Audit and feedback Reminders Incentives / sanctions Peer influence Opinion leaders Process redesign 22 July 2014 39

Knowledge to action Using knowledge of barriers/ enablers + evidence-based strategies to address Graham, J Eval Clin Pract 2006, 26:13-24 22 July 2014 40

Key messages What to implement: Where possible, implement findings from a body of evidence (systematic reviews, overview of systematic reviews) OR Use guidelines or tools based on systematic reviews Use GRADE to assess the body of evidence and AGREE to assess the quality of the guideline development process How to implement: Consider (and assess) the barriers and enablers to changing practice Select evidence-based strategies to target these barriers and enablers Consider carefully the opportunity cost of using strategies that aren t yet supported by high quality evidence 22 July 2014 41

Thankyou! 22 July 2014 42

1. KNOWLEDGE FOR TRANSLATION References & resources NHMRC research translation faculty Graham, I. D., J. Logan, M. B. Harrison, S. E. Straus, J. Tetroe, W. Caswell and N. Robinson (2006). "Lost in knowledge translation: time for a map?" J Contin Educ Health Prof 26(1): 13-24. Systematic reviews translating bodies of evidence The Cochrane Library www.thecochranelibrary.com includes Cochrane Database of Systematic Reviews (CDSR) of effectiveness of healthcare interventions and diagnostic test accuracy; CENTRAL register of controlled trials, DARE. Open access in Australia. Centre for Reviews and Dissemination, University of York CRD Database of Abstracts of Reviews of Effectiveness (DARE); abstracts for >30,000 systematic reviews. Access through Cochrane Library or direct: http://www.crd.york.ac.uk/crdweb/ PROSPERO prospective register of systematic reviews: http://www.crd.york.ac.uk/prospero/ Guideline, clinical decision-support tools, patient decision aids NHMRC portal to Australian guidelines: http://www.clinicalguidelines.gov.au/ (entry indicates NHMRC approval or not) NICE UK website: guidelines, evidence-based evidence summaries for patients and practitioners: www.nice.org.uk/guidance AHRQ USA: National Guideline Clearinghouse www.guideline.gov/ EPC evidence-based reports which include summaries for patients and practitioners www.ahrq.gov/research/findings/index.html ACQSHC (some useful links to tools and resources such as patient decision aids): www.safetyandquality.gov.au Cochrane clinical answers: decision-relevant evidence summaries for clinicians (subscription based): cochraneclinicalanswers.com BMJ evidence centre: http://clinicalevidence.bmj.com/ (subscription required) Ottawa hospital Patient Decision Aid: www.decisionaid.ohri.ca Tools for assessing the quality of a body of evidence GRADE www.gradeworkinggroup.org/ Includes links to software for generating summary of findings tables. Help function in software is useful for understanding GRADE domains. AGREE www.agreetrust.org/ : tool for appraising the quality of the guideline development process and reporting 22 July 2014 43

2. EVIDENCE-BASED IMPLEMENTATION APPROACHES References & resources Sources of systematic reviews Cochrane Effective Practice and Organisation of Care Group (see COCHRANE LIBRARY) Grimshaw J, Eccles, M, Lavis J, Hill S, and Squires J (2012) Knowledge translation of resarch findings. Implementation Science 2012, 7:50 doi:10.1186/1748-5908-7-50 General articles on study designs Berenholtz SM, Needham DM, Lubomski LH, Goeschel CA, Pronovost PJ: Improving the quality of quality improvement projects. Joint Commission journal on quality and patient safety / Joint Commission Resources 2010, 36(10):468-473. [methods checklist] Bhattacharyya OK, Estey EA, Zwarenstein M: Methodologies to evaluate the effectiveness of knowledge translation interventions: a primer for researchers and health care managers. J Clin Epi 2011, 64(1):32-40. Eccles M, Grimshaw J, Campbell M, Ramsay C: Research designs for studies evaluating the effectiveness of change and improvement strategies. Quality & safety in health care 2003, 12(1):47-52. Li LC, Moja L, Romero A, Sayre EC, Grimshaw JM: Nonrandomized quality improvement intervention trials might overstate the strength of causal inference of their findings. Journal of clinical epidemiology 2009, 62(9):959-966. Quality checklists and reporting guidelines for QI studies SQUIRE: Standards for Quality Improvement Reporting Excellence http://squire-statement.org/ Ogrinc G, Mooney SE, Estrada C, Foster T, Goldmann D, Hall LW, Huizinga MM, Liu SK, Mills P, Neily J et al: The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Quality & Safety in Health Care 2008, 17 Suppl 1:i13-32. Berenholtz SM, Needham DM, Lubomski LH, Goeschel CA, Pronovost PJ: Improving the quality of quality improvement projects. Joint Commission journal on quality and patient safety / Joint Commission Resources 2010, 36(10):468-473. [methods checklist] 22 July 2014 44

3. IMPLEMENTING CHANGES TO CARE References & resources Frameworks and taxonomies to aid identification of factors that influence practice change (barriers and facilitators) Theoretical domains framework: Series of papers (including applications of use): http://www.implementationscience.com/series/tdf Original TDF paper: Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, Psychological Theory Group: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005, 14(1):26-33. Validation of TDF, with some changes to framework: Cane J, O Connor D, Michie S: Validation of the theoretical domains framework for use in behaviour change and implementation research. Implementation Science 2012, 7:x. Flottorp S, Oxman A, Krause J, Musila N et al A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science 2013, 8:35 doi:10.1186/1748-5908-8-35 Designing interventions French S, Green S, O Connor D, McKenzie J et al Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implementation Science 2012, 7:38 doi:10.1186/1748-5908-7-38 22 July 2014 45