Achieving Synergy between Designing QI Projects and Writing them up for Publication

Transcription

1 Achieving Synergy between Designing QI Projects and Writing them up for Publication Kaveh G. Shojania, MD Director, Centre for Quality Improvement and Patient Safety (C-QuIPS) University of Toronto Editor-in-Chief, BMJ Quality & Safety

2 Typical QI Report Look We did it! Introduction Hospital infections affect thousands each year Hospital staff do not wash their hands consistently We implemented a multifaceted strategy Staff education Clinical champions Empowering patients to ask staff if they have washed their hands Methods Briefly stated design methods and some mention of Plan-Do- Study-Act Results We increased hand hygiene compliance by 50% [!!!!] Discussion Our hard work paid off and patient engagement is a wonderful thing

3 1. I succeeded at X versus 2. Here s what I had to do to achieve X Only your parents or spouse care about the first one. More people will care about the second

4 Another Unhelpful Type Intro CHF affects millions Some prominent body recommends x, y, and z quality targets We implemented a multifaceted strategy educational outreach performance report cards mailed to physicians Methods Briefly stated design, analysis, and outcomes Results It didn't work Discussion We can only guess why Telephone calls to patients

5 Better Report Intro Commonly identified barriers to optimal CHF care include x, y, and z Chart reminders, performance report cards, and phone calls might address x, y, and z in the following ways Methods Measures that assess implementation fidelity Format of report cards Usability/pilot testing of automated calls system Results MDs received report cards Patients used call systems No change in targets Discussion Maybe x, y, and z aren t the dominant barriers to high quality CHF care

6 That report is an improvement, but.. Ideally, we would have recognized (through doing more background work or pilot testing) that the intervention had problems The intervention theory was wrong or incomplete Implementation required attention to additional issues There has to be a better way to plan interventions (and their evaluations), so that we increase success

7 Doing research and writing it up Various guidelines for research papers (eg, CONSORT statements for RCTs) Billed as writing guidelines but provide blueprints for the research itself It s too late for double blinding at the writing stage Similarly, in QI thinking about the eventual writing enhances the project itself

8 Key features of a good plan for developing and evaluating QI interventions Explicit theory for why intervention will work Specifies the active ingredients of the intervention Explains how those ingredients address the causes of the target safety or quality problem Outcomes that measure success but also capture the degree to which intervention is working as expected Framework for refining the intervention and addressing implementation problems

9 Don t Do Fake PDSA! Forthcoming systematic review of 73 published QI projects claiming to have used PDSA Many of these studies reported application of PDSA method that failed to accord with primary features of the method. < 20% fully document the application of a sequence of iterative cycles. A lack of adherence to the notion of smallscale change is apparent Only 15% of articles reported the use of quantitative data at monthly or more frequent data intervals to inform progression of cycles.

10 Systematic review of the application of the plan do study act method to improve quality in healthcare Taylor MJ et al 2013 Among 73 published QI projects claiming to have used PDSA. < 20% fully of studies documented iterative cycles Only 15% of articles reported the use of quantitative data at monthly (or more frequent) data intervals to inform progression of cycles.

11 Returning to Typical QI report Introduction Hospital infections affect thousands each year Hospital staff do not wash their hands consistently We implemented a multifaceted strategy Staff education Clinical champions Empowering patients to ask staff if they have washed their hands No connection between the introduction material and specific features of the intervention Results We increased hand hygiene compliance by 50% [!!!!] Discussion We showed that patient engagement is a wonderful thing

12 Improved Introduction Introduction Commonly identified barriers to hand hygiene compliance include A, B, and C Staff Education, clinical champions, and empowering patients address A, B, and C by doing X, Y, and Z This Methods introduction makes clear why this specific intervention Briefly stated design, analytic strategy and main outcomes was chosen plus some mention of PDSA Having Results this theory for the intervention up front will pay off in developing the project, not just writing the report Discussion We increased hand hygiene compliance by 50% [!!!!] We showed that patient engagement is a wonderful thing

13 Articulating theory helps vet project ideas up front 1. Why will screening for fall risk work? Because if we identify the subset of high risk patients, then we can focus on them. But, can you effectively identify such a subset? Well actually, most patients end up being `high risk 2. How will bed alarms reduce falls? Nurses love nothing better than more alarms going off

14 Multiple Theories Even Better E.g., Rapid Response Teams 1. Main theory: deteriorating patients not recognized early enough AND ward physicians often difficult to reach AND ward physicians may not escalate care in timely fashion 2. Transfers to ICU often delayed even when ward staff want them; RRTs facilitate transfers to ICU ( when staffed by ICU personnel) 3. RRTs make more patients DNR fewer unexpected deaths Specification of main and alternate theories directly suggests different outcomes worth measuring

15 Now we have a theory, what about the intervention itself? Methods Introduction what?. We implemented a multifaceted strategy Staff education Clinical champions Empowering patients to ask staff if they have washed their hands Staff education about Briefly stated design, analytic What strategy really and are main outcomes plus some mention of PDSA clinical champions and how were they selected? Results This sounds like a big deal: what exactly did you tell patients and how did you support their Discussion doing so? We increased hand hygiene compliance by 50% [!!!!] We showed that patient engagement is a wonderful thing

16 Think about your intervention like a pill We would never say we gave a small, white pill to patients with chest pain We would say We gave aspirin 325mg for patients to chew immediately etc etc Similarly, what exactly was the education about, how was it delivered, how often was it delivered? If this education matters, then these details matter. If the details do not matter, then this part of your intervention probably played no role

17 Improved description (and execution) Education was delivered as a 1 hour staff lunch, with 30 min lecture covering common misconceptions about when hand hygiene is not necessary and 30 min for staff to express concerns/questions. We visited each ward once a month for 3 months to ensure adequate exposure Clinical champions from medicine, nursing, allied health, were selected on the basis of and did x, y, and z Nurses explained empowerment policy to patients... Investigators checked in with 5 patients per ward per day to ensure they had been informed

18 Refining intervention as you implement During the educational sessions it became clear that many nurses still believed that the hand hygiene gel would dry their hands. We modified the educational material to address this misconception Staff also expressed concerns about frequency of HH gel dispensers being empty or broken. We therefore In checking in with patients, we noticed that many had not been informed of the policy. In some cases, patients had been informed but did not feel comfortable challenging staff

19 Attention to these issues aids the intervention itself not just the report Some of these issues can really be addressed with relatively minor but still noteworthy modifications (eg, the content of the education) Others might have led to a new ingredient to the intervention (eg, addressing the problems with empty/broken dispensers) Still others might lead to a major change in the intervention (eg, dealing with the problem of patients not being comfortable speaking up) or even recognition that it was not feasible (eg, if patients reported any negative interactions with staff) Whatever the case, attention to these details will improve intervention and eventual report

20 Common problems with Outcomes Unknown connection to outcomes of interest- eg, we developed a questionnaire for measuring staff perceptions of handover quality. the average score before the intervention was 2.7 and after was 4.8 (p<0.05) Process that has a known connection to outcomes but depends on other processes (e.g., smoking cessation counseling) Outcomes that don t tell whole story length of stay but not readmissions Discharges by noon cardiac arrests on ward arrests may just be happening in intensive care unit with no change in survival

21 General Strategy for Outcomes Range of outcomes (quantitative and qualitative) that capture implementation and main outcome of interest Proportion of staff who reported asking at least 1 patient ask if they had washed their hands Patient reported negative interactions with staff Monthly hand hygiene rates Nosocomial infections, including ones not expected to change from intervention

22 Impact of a Hospital wide Hand Hygiene Initiative on Healthcare Associated Infections: an interrupted time series Kirkland K 2012 Objective To improve hand hygiene rates and reduce healthcare associated infections Design: Interrupted time series over 3 years with sequential interventions and a 1 year post intervention follow up Results: Hand Hygiene compliance: 41 91% Healthcare Associated Infection: / 1000 pt days

23

24 Kirkland et al BMJ Quality & Safety 2012 S. aureus HAIs from hospital wards S. aureus HAI in Operating Rooms

25 Thinking about the writing ahead of time will help your project Articulate a theory for the intervention use theory to inform design, implementation, reporting Outcomes that capture success but also measure the intervention s impact on mediators of success Don t fake PDSA, actually do it!!! Identify your project s Achilles Heal and do something about it at outset