Dissemination, Uptake, and Impact: Evaluation at PCORI Rachel Witsaman, MPH, PMP Program Officer, Evaluation & Analysis 1
About Us An independent research institute authorized by Congress in 2010 and governed by a 21-member Board of Governors representing the entire healthcare community Funds comparative clinical effectiveness research (CER) that engages patients and other stakeholders throughout the research process Seeks answers to real-world questions about what works best for patients based on their circumstances and concerns
Meeting the Legislative Mandate The purpose of the Institute is to assist patients, clinicians, purchasers, and policy-makers in making informed health decisions by advancing the quality and relevance of evidence concerning the manner in which diseases, disorders, and other health conditions can effectively and appropriately be prevented, diagnosed, treated, monitored, and managed through research and evidence synthesis... and the dissemination of research findings with respect to the relative health outcomes, clinical effectiveness, and appropriateness of the medical treatments, services... --from PCORI s authorizing legislation 3
Our Mission and Strategic Goals PCORI helps people make informed healthcare decisions, and improves healthcare delivery and outcomes, by producing and promoting highintegrity, evidence-based information that comes from research guided by patients, caregivers, and the broader healthcare community. Our Strategic Goals: Increase quantity, quality, and timeliness of useful, trustworthy research information available to support health decisions Speed the implementation and use of patient-centered outcomes research evidence Influence research funded by others to be more patient-centered
Model for Evaluating the Overall Impact of PCORI
Underlying the Evaluation Framework Our Questions and How We Will Answer Them Evaluation Questions Metrics/ Indicators Methods Sources What do PCORI and PCORI stakeholders want/need to know? For each question, what are we measuring and how will we measure it? What approach will we take to answering the question? From where will we get the data to answer the question? 6
From Dissemination to Use to Impact Whether results are reported back to study participants Access to PCORI study report Presentations: Scientific/professional audiences Lay audiences Bibliometrics Number of publications Time to publication Impact Factor Citations Alternative metrics for key groups (patients, clinicians, payers, etc.): Number of manuscript downloads Number of manuscript bookmarks Media coverage Social media coverage 7
Examples of How We Track Dissemination Interim Progress Reports (6 month intervals) help us track upcoming and completed manuscripts, presentations, and other disseminations of results PIs report in Final Progress Reports: o whether they will disseminate results to study participants, engaged partners, and other investigators on the study o the methods of dissemination 8
From Dissemination to Use to Impact 9
Preparing to Track Uptake and Use PCORI will be utilizing alternative metrics to understand how study results are being incorporated into practice. Preparation for use and uptake includes implementing the use of the Altmetrics platform to track incorporation of results into curriculum, policies, and other areas. 10
Other Alternative Metric Examples: Goal 3 - Influence
Additional Resources For more information on evaluation at PCORI o visit the Evaluating Our Work page on the PCORI Website o view An Overview of the Evaluation Framework at PCORI o or contact Rachel Witsaman rwitsaman@pcori.org for any other questions about evaluation at PCORI 12