Realist Synthesis Methodology 101 Trillium Primary Health Care Research Day 2015 Judith Belle Brown, PhD Bridget L. Ryan, PhD Luan Januzi, MSc 1
Community Based Primary Health Care CIHR Team grant Principal Investigators: Moira Stewart Martin Fortin 2
Topics Covered What is Realist Synthesis? What is the process for conducting a Realist Synthesis? Example of Realist Synthesis Workshop Exercise: Creating CMOs 3
What is a Realist Synthesis? 4
Reviewing the Literature Many different types of reviews and typologies Systematic reviews Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review Meta-analysis Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results Grant & Booth 2009 5
Realist Synthesis aka Realist Review Systematic review follows highly specified, intentionally inflexible methodology, aiming for high reliability Realist synthesis follows a more heterogeneous and iterative process less amenable to prescription probably demands greater methodological expertise The process should be equally rigorous it should be possible to look behind the synthesis and see how decisions were made, evidence sought, sifted and assessed, and findings accumulated and synthesised. Pawson et al. 2004 6
RS Program Theory Conventional perspective sees interventions as tangible things (e.g. resources, equipment, personnel) In RS, interventions are theories Interventions are always based on a hypothesis that postulates If we deliver a program in this way or we manage services like so, then this will bring about some improved outcome. Pawson 2004 7
RS Definition Realist synthesis is an approach to reviewing research evidence on complex social interventions, which provides an explanatory analysis of how and why they work (or don t work) in particular contexts or settings. Pawson et al. 2004 8
RS - CMO in particular contexts or settings of how and why they work (or don t work) Context Mechanism (Success & Failure) Outcome (Positive & Negative) 9
What is the Process for Conducting a Realist Synthesis? 10
Stages in Realist Synthesis (RS) 1) Clarify the scope of the RS 2) Search for evidence 3) Quality appraisal of the evidence 4) Data extraction and coding: -Identification of Context-Mechanism-Outcome (CMO) configurations 5) Synthesize findings 6) Dissemination 11
1. Clarify the Scope of RS Identify the review questions Clarify the purpose of the RS Find and articulate program theories 12
2. Search for Evidence Search for the evidence: published papers grey literature annual reports presentations Preliminary Data Exploration: To identify a preliminary list of mechanisms 13
3. Quality Appraisal of the Evidence Two criteria for quality appraisal: Relevance (appropriate content for inclusion in the synthesis) Rigour (trustworthiness of the data) 14
4. Data extraction and coding Steps in Coding: 1. Identify the context 2. Identify the mechanism 3. Identify the positive and negative outcomes 4. Link the identified mechanism and positive or negative outcomes to the appropriate context 15
5. Synthesize findings Refine the program theories on how the intervention works. Use findings to address the purpose of the RS review. Compare and contrast findings. 16
6. Dissemination Disseminate the findings, conclusions, and recommendations: Reports Research publications Conferences 17
Example of Realist Synthesis 18
Processes of a Realist Synthesis Inspired by Pawson et al. (2004) and Sridharan (n.d.) 19
Example of CMO Configuration 1 Primary care physicians are expected to provide an increased number of preventive services, while being both evidence based and patient-centred. Current models of primary care do not allow sufficient time to address multiple health needs or to create comprehensive care plans. Patients, families, and health care professionals have reported that this program has numerous advantages. For patients and families, the extended interprofessional visit allows sufficient time and space for patients and family members to raise and discuss multiple problems with a team of professionals; indeed, a number of patients reported that the new model allowed them to tell their story and to be heard. Participating family physicians have reported that, for those patients assessed in the clinic, subsequent primary care visits were more tractable and efficient. 20
Example of CMO Configuration 1 Primary care physicians are expected to provide an increased number of preventive services, while being both evidence based and patient-centred. Current models of primary care do not allow sufficient time to address multiple health needs or to create comprehensive care plans. Patients, families, and health care professionals have reported that this program has numerous advantages. For patients and families, the extended interprofessional visit allows sufficient time and space for patients and family members to raise and discuss multiple problems with a team of professionals; indeed, a number of patients reported that the new model allowed them to tell their story and to be heard. Participating family physicians have reported that, for those patients assessed in the clinic, subsequent primary care visits were more tractable and efficient. 21
Example of CMO Configuration 2 Lack of knowledge about dementia...and limited capacity to manage challenging psychological and behavioral symptoms contribute to poor management of dementia in primary care. To address these challenges an inter-disciplinary primary care based clinic was established within which patients, referred by their family physician, are assessed and recommendations for management are formulated. This model of care represents a significant opportunity for capacity building among physicians, nurses, and other interdisciplinary care providers as evidenced by self-reported changes in knowledge and skills needed to diagnose and manage seniors with memory problems and cognitive impairment. This integrated, interprofessional model of care has ensured timely access to high quality assessment and intervention for patients with dementia. 22
Example of CMO Configuration 2 Lack of knowledge about dementia...and limited capacity to manage challenging psychological and behavioral symptoms contribute to poor management of dementia in primary care. To address these challenges a primary care clinic compromised of an interdisciplinary team was established within which patients, referred by their family physician, are assessed and recommendations for management are formulated. The clinic consists of one family physician lead, two registered nurses, one social worker, one pharmacist, and one receptionist. This model of care represents a significant opportunity for capacity building among physicians, nurses, and other interdisciplinary care providers as evidenced by self-reported changes in knowledge and skills needed to diagnose and manage seniors with memory problems and cognitive impairment. This integrated, interprofessional model of care has ensured timely access to high quality assessment and intervention for patients with dementia. 23
Questions? 24
References Yardley, S., Cottrell, E., Rees, E., & Protheroe, J. (2015). Modelling successful primary care for multimorbidity: a realist synthesis of successes and failures in concurrent learning and healthcare delivery. BMC family practice, 16(1), 23. Yardley, S., Cottrell, E., & Protheroe, J. (2013). Understanding success and failure in multimorbidity: protocol for using realist synthesis to identify how social learning and workplace practices can be optimised. Systematic reviews, 2(1), 87. Example of Realist Synthesis Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Realist synthesis: An introduction. ESRC Research Methods Programme. Pawson, R., Greenhalgh, T., Harvey, G. & Walshe, K. (2005). Realist review: A new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy, 10(S1), 21-34. Pawson, R. & Bellamy, J.L. (2006). Realist synthesis: an explanatory focus for systematic review (pp. 83-93). In Pope, J. (Ed.) Moving beyong effectiveness in evidence synthesis: Methodological issues in the synthesis of diverse sources of evidence. London: National Institute of Health and Clinical Excellence. Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal,26(2), 91-108. 25
Workshop Exercise: Creating CMOs 26
Exercise for Creating CMOs 15 minutes to create CMO configurations individually Three mechanisms are named in the next slide Extracts of 3 research papers available Read extracts individually Identify the 3 mechanisms Identify contexts and outcomes Link each of the mechanisms to their appropriate contexts and their outcomes to create CMOs (Hint: Look across papers to find CMOs) Summarize the context, mechanism, and outcomes 15 minutes to compare and finalize CMOs as a group Compare CMOs Discuss and resolve differences Ask one member to present to larger group 27