Backgrund There is a great deal f pr quality research and misleading infrmatin even in the highest quality medical jurnals. Health care rganizatins rarely recgnize this and/r rarely have systems in place and staff with skills t d needed evaluatins f scientific evidence. Health care shuld be prvided by rganizatins that bth understand the need fr evaluating science, knw hw t d s and prvide resurces fr ding this wrk. Individual circumstances apply. Yur actual findings need t take accunt f the whle r ther factrs which may serve as reasnable substitutins. Evaluatin Tl Part I. Scientific Evaluatin Capabilities Desired Outcme Prblem General Advice 1. Organizatinal Understanding Can the rganizatinal r quality imprvement leadership articulate a true understanding f the need fr a rigrus and systematic evaluatin f the quality f scientific evidence befre applying it? Nte: Leadership is vitally imprtant t help create an evidence- and value-based system. Many leaders may be able t sund like they understand, but nt actually have a true understanding. Yes: N: Red flag 2. Systematic Prcesses fr Evaluating Health Care Technlgies Des the rganizatin have a system in peratin fr rutine rigrus and systematic evaluatin f new drugs, devices and prcedures thrugh rigrus and systematic evaluatin f scientific quality? Can they demnstrate that they have created evidence-based wrk prcesses meaning d they have in place the methds, tls and staff t identify ptential wrk areas fr quality imprvement, critically appraise the medical literature and create evidence-based clinical imprvement initiatives based n valid, relevant evidence f effectiveness and value? Des the rganizatin have staff wh pssess the needed skills t critically appraise medical literature (e.g., epidemilgy skills) and are these staff utilized fr these s? Is there a frmal structure and mechanism fr regularly reviewing new drugs, devices and prcedures thrugh these prcesses? Fr example, Yes: N: Fails Reminder f ptential savings estimated at 15 t 30% f drugs and ver 30 percent f new technlgies Reminder f cnsiderable increase in care quality and reductin f patient harms www.delfini.rg 2002-2013 Delfini Grup, LLC. All Rights Reserved Wrldwide. Page 1 f 5
des the rganizatin have sme r all f the fllwing cmmittees that use evidence-based prcesses fr evaluating the strength f the evidence by evaluating interventins by lking at bth study type and methdlgy? Quality r Value Oversight Cmmittee Pharmacy & Therapeutics Cmmittee Technlgy Assessment Cmmittee Guidelines/pathways department Is the system functining successfully? 3. Understanding f Study Types Des the rganizatin use bservatinal studies r case series fr questins f the efficacy f therapy, screening r preventin? If yes, is this dne nly in absence f evidence frm valid randmized cntrlled trials (RCTs) and is this infrmatin labeled in such a way that clinicians and thers understand that the evidence is weak and maybe misleading? 4. Perfrmance f Rigrus Critical Appraisal Des the rganizatin rely n randmized cntrlled trials r systematic reviews f randmized cntrlled trials that have nt undergne a rigrus critical appraisal? 5. Critical Appraisal f Clinical Recmmendatin Cntent If the rganizatin uses clinical guidelines, pathways, perfrmance measures r ther clinical recmmendatins, d they have a rigrus and evidence-based prcess fr successfully evaluating the quality f that infrmatin? Nte: There are enrmus prblems with medical cntent and recmmendatins. Many are labeled evidence-based, when actually they are nt they may refer t a scientific study, but d nt reflect the bdy f science r may be based n a pr and misleading study. Cnsensus guidelines are highly prne t bias regardless f the develpers. Medical cntent has t be evaluated fr validity and usefulness. N: Yes: Fails Yes: N: Fails Yes: N: Fails www.delfini.rg 2002-2013 Delfini Grup, LLC. All Rights Reserved Wrldwide. Page 2 f 5
6. Critical Appraisal Cre Cmpetencies D mst f the rganizatin s individual clinicians and clinical pharmacists have an understanding f the cre principals f critically appraising research studies fr validity and usefulness? Is there sme way that clinicians and clinical pharmacists are evaluated fr this knwledge? Is there a training prgram fr them? Nte: This is an enrmus prblem in health care. Much that is published in the medical literature is pr r misleading. Mst physicians and clinical pharmacists cannt tell. This affects health care decisins daily. Applying a systems apprach, withut ensuring that their individual physicians and clinical pharmacists pssess, as a cre cmpetency, basic skills in critical appraisal and results, can be likened t trying t stp a fld withut creating a slid barrier. N: Yes: Prblem Reminder that well ver 70% f physicians fail a simple literature evaluatin quiz High risk fr patient harms, inapprpriate care and higher csts Part II. Applicatin f Valid Science Desired Outcme Prblem General Advice 7. Clinical Imprvement Implementatin Skills Can the rganizatin prvide evidence f successful implementatin f quality imprvement prjects? Nte: Many rganizatins put effrts int creating quality imprvement prjects, but lack understanding f hw t effectively implement them and create practitiner behavir change. 8. Health Care Staff Access t Quality Infrmatin Can the rganizatin demnstrate that physicians and ther clinical staff have ready access t the quality imprvement infrmatin that is t be applied fr patient care? This is especially effective when the infrmatin is available at pint-f-care. 9. Cnsumer Access t Quality Infrmatin Can the rganizatin demnstrate that it makes this infrmatin readily available t cnsumers and patients? Yes: N: Prblem Yes: N: Prblem Yes: N: Medium prblem www.delfini.rg 2002-2013 Delfini Grup, LLC. All Rights Reserved Wrldwide. Page 3 f 5
10. Perfrmance Measures & Quality Indicatrs Can the rganizatin demnstrate imprvement thrugh perfrmance measures r quality indicatrs r a fcus n quality thrugh use f such indicatrs? NOTE: Unless high quality research is dne, imprved health utcmes is nt a valid measurement. Lk fr imprvement in prcesses r services r evidence that a quality imprvement has been successfully implemented. This is a cmplicated area which may require evaluatin by experts understanding the pitfalls f such measurement. Yes: N: May be nly a minr prblem (and this is a cmplicated area which may require evaluatin by experts understanding the pitfalls f such measurement) Part III. Organizatinal Cmmitment Desired Outcme Prblem General Advice 11. Missin Statement Reflects Pririties Is a cmmitment t evidence-based quality imprvement acknwledged in the rganizatin s missin statement? 12. Leadership Supprt Is there cncrete evidence that the rganizatin s leaders are cmmitted t supprting clinical imprvements based n the best available scientific evidence? a. Statements in the missin statement, business plan, quality plan, etc.? b. Other? 13. Aligned Incentives Are incentives aligned fr quality? Example: In many rganizatins pharmacy budgets and inpatient budgets are viewed separately (sil d). If budgets are sil d, ne department may get penalized fr rising csts in their department which actually represent quality verall. Or if perfrmance measures are used, physicians might be penalized if the measure des nt take clinical judgment int accunt. Yes: N: Minr prblem Yes: N: Minr prblem Yes: N: Culd be a majr prblem Assessment Outcmes www.delfini.rg 2002-2013 Delfini Grup, LLC. All Rights Reserved Wrldwide. Page 4 f 5
Part I. Summary f Assessment f Scientific Evaluatin Capabilities: Part II. Summary f Assessment f Applicatin f Valid Science: Part III. Summary f Assessment f Organizatinal Cmmitment: Recmmendatins: Other Cmments: www.delfini.rg 2002-2013 Delfini Grup, LLC. All Rights Reserved Wrldwide. Page 5 f 5