SYSTEMATIC REVIEW METHODS Unit 1
GETTING STARTED Introduction Schedule Ground rules
EVALUATION Class Participation (20%) Contribution to class discussions Evidence of critical thinking Engagement in learning activities Systematic Review Protocol (80%) Group project Develop systematic review protocol In-class presentation
SYSTEMATIC REVIEW PROTOCOL Topic concerning the effectiveness of a health care intervention Rationale for the topic Team members/collaborators Clearly focused quantitative review question Inclusion and exclusion criteria Search strategy Assessment of methodological quality of the included studies Synthesis approach Communicating key messages
OBJECTIVES: UNIT 1 1. Differentiate between different types of reviews. 2. Identify the importance of systematic reviews in health care policy and practice decisions. 3. Identify the components of high quality systematic reviews. 4. Develop a clearly focused quantitative systematic review question about the effectiveness of an intervention.
WHAT IS A SYSTEMATIC REVIEW? A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias thus providing more reliable findings from which conclusions can be drawn and decisions made [Cochrane Handbook]
WHY DO A SYSTEMATIC REVIEW Identify what we know to inform decisions about practice, programs, policy by resolving conflicting evidence addressing questions where clinical practice is uncertain exploring variations in practice Identify what we know and/or do not know to inform the rationale for conducting research When available systematic reviews are not up to date, and/or poor quality, and/or do not sufficiently address the question asked
INFORM PRACTICE, PROGRAMS, &/OR POLICY DECISIONS
INFORM PRACTICE, PROGRAMS, &/OR POLICY DECISIONS Cochrane Database Syst Rev. 2013 Dec 5;(12):CD007458. Mobile phone messaging reminders for attendance at healthcare appointme nts.gurol- Urganci I1, de Jongh T, Vodopivec- Jamsek V, Atun R, Car J.
INFORM RATIONALE FOR CONDUCTING RESEARCH Ivers et al. (2014) Journal of General Internal Medicine: Growing Literature, Stagnant Science? Systematic Review, Meta- Regression and Cumulative Analysis of Audit and Feedback Interventions in Health Care.
SYSTEMATIC REVIEW VS. REVIEW OF THE LITERATURE Systematic review strict scientific design explicit methods where more than one person is involved in each step pre-specified methods that do not change in the course of conducting the review reproducible methods
TYPES OF REVIEWS Scoping review Traditional systematic review Network meta-analysis Overview of reviews Realist reviews Meta-syntheses
NETWORK META-ANALYSIS A B C
SYSTEMATIC REVIEW QUESTIONS Benefits and harms of interventions used in healthcare and health policy [Intervention Reviews/Aetiology Reviews] Wound care teams for preventing and treating pressure ulcers Safety of topical corticosteroids in pregnancy Probable course or future outcome(s) of people with a health problem [Prognosis] Individual recovery expectations and prognosis of outcomes in non specific low back pain Diagnostic Test Accuracy PET CT for assessing mediastinal lymph node involvement in patients with suspected resectable non small cell lung cancer Qualitative Healthcare workers perceptions and experience on using mhealth technologies to deliver primary healthcare services Cochrane Library
SYSTEMATIC REVIEW: ESSENTIAL ELEMENTS Systematic search of a number of different sources to identify relevant studies Select studies for inclusion, based on predefined criteria Evaluate studies for their strengths and limitations based on clear, predefined criteria Systematically collect data Appropriately synthesize data
REVIEW QUESTION Is the source of all other aspects of the review flow Should be identified a priori clear, unambiguous Structured PICO PECO SPPiCE
REVIEW QUESTION: PI/ECO Applicable to: Intervention Reviews Aetiology Reviews Prognosis Reviews Population(s) Intervention(s)/Indicator(s) or Exposure(s) Comparator Outcome(s)
REVIEW QUESTION: PI/ECO Population Group of participants or patients, their clinical problem, and the healthcare setting Intervention/Indicator/Exposure/Comparator Action(s) being considered e.g. treatments, processes of care, social interventions, educational intervention, risk factors Outcome Clinical changes in health state or other related changes (e.g. health resource use)
REVIEW QUESTION: TEST ACCURACY Population Test Test whose predictive value is being assessed Reference Standard gold standard test that confirms or refutes the diagnosis
REVIEW QUESTION: SPPICE Applicable to Qualitative Reviews Sample Purpose/Phenomenon of Interest Context Emphasis
REVIEW QUESTION: STUDY DESIGN Driven by the components of the question Provide the most valid answer to the question Intervention Review Appropriate way to recruit participants give participants the intervention(s) and/or comparator(s) measure the outcome(s)
REVIEW QUESTION: EXAMPLE Intervention Review Question Which of the many available microbial products improve healing in patients with chronic wounds Structured question P: patients with various forms of chronic wounds I/C: microbial products O: wound healing Study design: A study that allocates subjects with chronic wounds to alternative therapeutic interventions of interest and determines the effect of the interventions on wound healing [randomized trial]
VARIATIONS IN POPULATION How is the disease/condition defined? What are the most important characteristics that describe these people (participants)? Are there any relevant demographic factors, e.g. Age Gender Ethnicity What is the setting, e.g. Hospital Community
VARIATIONS IN INTERVENTION What are the experimental and control (comparator) interventions of interest Does the intervention have variations, e.g. dosage/intensity mode of delivery personnel who deliver it frequency of delivery, duration of delivery, timing of delivery)
VARIATIONS IN OUTCOME Essential for decision-making usually have an emphasis on patientimportant outcomes Potential as well as actual adverse effects Relevant to all potential decision makers, including economic data. Type of outcome measurements Timing of outcome measurements
REVIEW QUESTION: EXAMPLE Question: Are knowledge translation (KT) interventions directed to nurses in tertiary care are effective for improving EBP knowledge, skills, behaviours, and, as a result, patient outcomes Structured Question: P: nurses in tertiary care I: any KT intervention O: EBP knowledge, skills, behaviors O: patient outcomes
BROAD VS. NARROW: PARTICIPANTS Broad: corticosteroid injection for any tendonitis Narrow: corticosteroid injection for shoulder tendonitis Cochrane Handbook 5.6.a Advantages Comprehensive summary of the evidence. Ability to assess generalizability of findings across different implementations of the intervention Manageability for review team. Ease of reading. Disadvantages May be more appropriate to prepare an Overview of Reviews Searching, data collection, analysis and writing may require more resources. Risk of mixing apples and oranges (heterogeneity); interpretation may be difficult. Evidence may be sparse. Findings may not be generalizable to other settings or populations. Scope could be chosen by review authors to produce a desired result.
Broad: any exercise for depression BROAD VS. NARROW: INTERVENTION DEFINITION Advantages Comprehensive summary of the evidence. Ability to assess generalizability of findings across types of participants. Disadvantages Searching, data collection, analysis and writing may require more resources. Risk of mixing apples and oranges (heterogeneity); interpretation may be difficult. Narrow: supervised running for depression Manageability for review team. Ease of reading. Evidence may be sparse. Findings may not be generalizable to other formulations of the intervention. Scope could be chosen by review authors to produce a desired result. Cochrane Handbook 5.6.a
Broad: interventions for preventing bedwetting BROAD VS. NARROW: INTERVENTION/COMPARISON Advantages Comprehensive summary of the evidence. Disadvantages May be unwieldy, and more appropriate to present as an Overview of Reviews Searching, data collection, analysis and writing may require more resources. Narrow: alarms for preventing bedwetting Manageability for review team. Clarity of objectives and ease of reading. May have limited value when not included in an Overview. Cochrane Handbook 5.6.a
IN-CLASS ACTIVIT Y Frame the Systematic Review Question
IN-CLASS ACTIVIT Y 2 Minute Presentation: Systematic Review Question
IN-CLASS ACTIVIT Y Identify Assignment Groups
JUSTIFYING THE REVIEW What is the problem/issue? Why is it important to address the problem/issue? What is the magnitude of the problem/issue?
EXAMPLE Nurses are expected to use research evidence in practice to improve client and system outcomes. International standards of practice identify EBP as an important and integral component of effective nursing practice. Although awareness about EBP has increased, it remains inconsistently operationalized for use in practice.
JUSTIFYING THE REVIEW Are there relevant studies that have been done? What is the gap in the synthesized literature of these studies? Modello di ricerca delle fonti di evidenza 6 S: http://www.evidencebasednursing.it/6s.html
Search databases where systematic reviews are registered Prospero http://www.crd.york.ac.uk/prospero/ Search databases/repositories of systematic reviews, e.g. ACCESSSS Federated Search http://plus.mcmaster.ca/accessss/default.aspx?page=1 Database of Abstracts of Reviews of Effects http://www.crd.york.ac.uk/crdweb/ PubMed Clinical Queries https://www.ncbi.nlm.nih.gov/pubmed/clinical Health Evidence http://healthevidence.org/ Health Systems Evidence https://www.healthsystemsevidence.org/
Search of organizations producing systematic reviews, e.g. Cochrane Collaboration http://www.cochranelibrary.com/ Joanna Briggs Institute http://journals.lww.com/jbisrir/pages/default.aspx Search of guideline developers who conduct systematic reviews to inform their guidelines National Guideline Clearinghouse https://guideline.gov/ NICE https://www.nice.org.uk/guidance SIGN http://sign.ac.uk/ AHRQ http://www.ahrq.gov/research/findings/evidence-basedreports/index.html CTFPHC http://canadiantaskforce.ca/ EPPI Centre http://eppi.ioe.ac.uk/cms/
EXAMPLE Previous systematic reviews have focused on effectiveness of knowledge translation (KT) interventions to promote the use of research evidence among healthcare professionals No analyses were specific to nurses in these reviews. Only one systematic review by Thompson and colleagues considered the effect of KT interventions on nurses research use. With a limited number of studies of poor quality, Thompson and colleagues concluded that there was insufficient evidence to support or refute the use of any specific intervention aimed at increasing research use in nursing.
IN-CLASS ACTIVIT Y Justifying the Systematic Review Question
INTERVENTION SYSTEMATIC REVIEW: METHODS Define the research question Identify inclusion and exclusion criteria Search for studies Select studies Assess studies for methodological quality Extract relevant data from included studies Synthesize the data Interpret the results Draw conclusions
TIMELINE Cochrane Handbook
RESOURCES NEEDED TIME Funds Web applications Team Interlibrary loan
Web applications Rayyan http://rayyan.qcri.org/ Distiller http://rayyan.qcri.org/ EPPI Reviewer http://eppi.ioe.ac.uk/cms/default.a spx?alias=eppi.ioe.ac.uk/cms/er4
Team Additional reviewer(s) Content expertise Methodology expertise/support Searching Familiarity with web applications Statistical support
EXAMPLE: TEAM Who will use the findings from the review Advisory Committee Expertise in the content area Advisory Committee Team members Expertise in systematic review methods Team members, including Health Science Librarian McMaster Evidence Review Centre
IN-CLASS ACTIVIT Y Systematic Review Team