Orchestrated Testing Aggregate Data

Transcription

1 Orchestrated Testing Aggregate Data Michele Walsh, MD, MSE Through collaborative use of improvement science methods, reduce preterm births & improve perinatal and preterm newborn outcomes in Ohio as quickly as possible.

2

3

4

5

6

7

8

9

10

11

12

13

14 Percent of Infants with Greater than 10% Weight Loss from Birth Weight in First Seven Days De-Identified Formula Groups

15 Number of Finnegan Scores Greater than 12 in 24 Hours Prior to Starting Pharmacologic Treatment De-Identified Formula Groups

16 Highest Finnegan Score in 24 Hours Prior to Starting Treatment De- Identified Formula Groups

17 Average Length of Stay for All Opioid Exposed Infants De-Identified Formula Groups

18 Average Length of Opiate Treatment De-Identified Formula Groups

19 Average Length of Stay for Pharmacologically Treated Babies De- Identified Formula Groups

20 NAS-Orchestrated Testing Group 1 Theresa Ruby, MSN, RNC, IBCLC Southern Ohio Medical Center Through collaborative use of improvement science methods, reduce preterm births & improve perinatal and preterm newborn outcomes in Ohio as quickly as possible.

21 Group 1: 22 kcal /low lactose formula Akron Children s St Elizabeth/ Mahoning Valley/St Joseph Kettering NCH Dublin Methodist NCH Mt Carmel St Ann s NCH Riverside ProMedica Bay Park Rainbow Babies & Children s Cincinnati Children s Hospital Medical Center Soin Medical Center Southern Ohio Medical Center Southview Medical Center St Rita s Medical Center Trumbull Memorial UC Cincinnati ProMedica Bay Park Trumbull Memorial Rainbow Babies & Children s St. Rita s Medical Center Southview Medical Center Kettering NCH Riverside NCH Dublin Methodist NCH Mt. Carmel St. Ann s Akron Children s St. Elizabeth Mahoning Valley St Joseph s Soin Medical Center Cincinnati Children s UC Cincinnati Southern Ohio Medical Center

22

23

24

25

26

27

28 NAS-Orchestrated Testing Group 2 Gail Bagwell, DNP, APRN Nationwide Children s Hospital Through collaborative use of improvement science methods, reduce preterm births & improve perinatal and preterm newborn outcomes in Ohio as quickly as possible.

29 Group 2: 22 kcal /not low lactose formula Bethesda North Licking Memorial Hospital Dayton Children s Mercy Children s Hospital Good Samaritan Tri-Health Nationwide Children s Hospital Mercy Children s Hospital Dayton Children s Nationwide Children s Bethesda North Licking Memorial Good Samaritan Tri-Health Hospital

30

31

32

33

34

35

36 NAS-Orchestrated Testing Group 3 Mischel Balazs, NNP, CNP-BC Toledo Children s Hospital Through collaborative use of improvement science methods, reduce preterm births & improve perinatal and preterm newborn outcomes in Ohio as quickly as possible.

37 Group 3: 19 kcal /low lactose formula Akron Children s Hospital Medical Center Mercy Anderson Akron Children s Summa Mercy Medical Canton Akron Children s General Miami Valley Hospital Atrium Hospital Mt Carmel East Aultman Hospital Mt Carmel West Cleveland Clinic NCH Doctors Hospital Fairview Hospital NCH Grant Medical Fort Hamilton Hospital OSU Wexner Well Baby/OSU Wexner NICU Genesis Healthcare ProMedica Toledo Normal Newborn Hillcrest Medical Center ProMedica Toledo Children s Hospital Toledo Children s Toledo Normal Newborn Miami Valley Atrium Fairview Cleveland Clinic Hillcrest NCH Doctors OSU Well Baby OSU NICU NCH Grant Mt Carmel East Mt Carmel West Genesis Akron Children s ACH Summa ACH General Mercy Medical Canton Aultman Fort Hamilton Mercy Anderson

38

39

40

41

42

43

44 NAS-Orchestrated Testing Group 4 Tonia Deacon Through collaborative use of improvement science methods, reduce preterm births & improve perinatal and preterm newborn outcomes in Ohio as quickly as possible.

45 Group 4: 19 kcal /not low lactose formula Adena Regional Medical Center Mercy Health Fairfield Elyria UHCMC Mercy Health West Good Samaritan Dayton Mercy Regional Lorain Lima Memorial Hospital Springfield Medical Center Marion General Hospital The Christ Hospital MetroHealth Upper Valley Medical Center Lima Memorial Elyria Mercy Lorain MetroHealth Marion Good Samaritan Dayton Springfield Upper Valley Mercy Fairfield Medical Center Mercy West Christ Hospital Adena

46

47

48

49

50

51

52 Data Analysis of Orchestrated Testing Heather Kaplan, MD Through collaborative use of improvement science methods, reduce preterm births & improve perinatal and preterm newborn outcomes in Ohio as quickly as possible.

53 OPQC Phase II Orchestrated Testing

54 OT Analysis Team Maurizio Macaluso, MD, DPH Pierce Kuhnell, MS Heather Kaplan, MD, MSCE

55 Quality Improvement using the Model for Improvement Model for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? A P S D Changes That Result in Improvement Implementation of Change Hunches Theories Ideas 55 A P S D Very Small Scale Test Follow-up Tests Wide-Scale Tests of Change Sequential building of knowledge under a wide range of conditions

56 Orchestrated Testing OT involves planned testing across multiple sites (within or across institutions) Can use factorial design to Be more systematic about simultaneous testing of different change ideas Look at the independent and combined effects of different changes Standardization of practices and reliable implementation is necessary Can result in faster and more efficient learning

57 OPQC OT Phase II Wide scale test of change examining the role of formula in non-pharmacologic care across 54 NICU/SCN sites Two change ideas (factors): Type of formula Calorie content of formula Two levels of each factor Standard Lactose vs. Low-Lactose Standard Calorie vs. Higher Calorie Factorial Design

58 OPQC Factorial Design (2 2 ) Group Calorie Content Formula Type A 19 kcal/oz Low-Lactose B 22 kcal/oz Standard C 19 kcal/oz Standard D 22 kcal/oz Low-Lactose

59 The Model for Improvement

60 Measures: Outcomes Primary Outcome LOS (pharmacologically treated infants) Secondary Outcomes LOS (all infants) LOTx Percent pharmacologically treated Percent infants requiring dose escalation Weight Loss >10% These are similar outcomes to the outcomes examined in the RCT of standard vs. 24 kcal formula posted on clinicaltrials.gov (weight loss, LOS, LOTx, Finnegan scores)

61 Measures: Background Variables Pharmacologic treatment approach* OCHA Morphine Cincinnati Methadone Other Morphine Other Methadone Rates of breastmilk feeding Percent outborn infants + Formula compliance Time of formula initiation (early vs. time of treatment) NAS Severity (Max Finnegan Score, Scores>12) Change in typical formula treatment from Phase I to II *Centers with extreme deviation from these protocols will not be included in analysis + Infants transferring between sites in different groups will be excluded

62 Analysis Main analyses are graphical Run and control charts Explore differences on background variables Examine impact according to groups Response plots

63 Run and Control Charts Explore Background Variables Background Variable Groups: y, c, z, x

64 Run and Control Charts Examine Impact Group X Group Y Group Z Group C Factor 1 High High Low Low Factor 2 Low High Low High

65 Response Plots: Interactions No Interaction F2 High Moderate Interaction F2 High F2 Low F2 Low Low Factor 1 High Low Factor 1 High Moderate Interaction Strong Interaction F2 High F2 High F2 Low F2 Low Low Factor 1 High Low Factor 1 High

66 Replication Ultimate confirmation of findings from OT comes from replication across sites over time If positive results are seen for a change, process modification at additional sites with subsequent improvement further validates findings This is a critical step because conditions in the future (when the findings will be used) are likely different than the conditions during OT

67 How this works in real life SLUG Bug QI Project with OT 17 participating centers Examine 4 CLABSI prevention strategies Tubing change technique (sterile vs. clean) Hub care monitoring (yes vs. no) CVC access limitation (limitation vs. no limitation) CVC removal monitoring (tracking policy vs. no policy) Used factorial design ( groups) Focused on reliability 16/17 centers achieved >75% reliability on these processes Piazza AJ, et al. Pediatrics. 2016

68 4 Factors 8 Groups Variation in Baseline Rates Piazza AJ, et al. Pediatrics. 2016

69 Run and Control Charts ** ** ** ** /4 Sterile Tubing Change ** Greatest decrease with addition of HC Monitoring+ Piazza AJ, et al. Pediatrics. 2016

70 Response Plots: Impact Sterile Tubing Change Decreased CLABSI Rates by an average of 0.51 Piazza AJ, et al. Pediatrics. 2016

71 Response Plots: Interaction The addition of hub care compliance monitoring with sterile tubing change created the greatest impact showing evidence of an interaction Piazza AJ, et al. Pediatrics. 2016

72 Replication Courtesy of Lloyd Provost, data unpublished

73 How this might look for OPQC Group Calorie Content Formula Type # Hospitals Baseline LOS Phase II LOS A 22 kcal LLF ? B 22 kcal Standard ? C 19 kcal LLF ? D 19 kcal Standard ?

74 Hypothetical Control Chart Type LLF Std LLF Std Calorie 22 kcal 22 kcal 19 kcal 19 kcal

75 Hypothetical Response Plots 19 kcal 22 kcal Std LLF Std 19 kcal 22 kcal LLF Calories LOS: Formula Type LLF Std 19 kcal kcal

76 Keep in mind We are examining the effect of 2 factors in a complex system there will always be the possibility that other variables (known background variables or unknown variables) account for some of what we see BUT, the complexity is also a strength wide range of clinical conditions improves generalizability The strength of this OT approach is that is allows us to examine factors in real settings in a range of conditions (54 sites!) This is an new method of learning for OPQC and we have to figure out some things as we go (e.g., best approach to adjust for differences in baseline LOS) After June, we need to commit to implement the findings to achieve the full benefit of the OT approach

77 References for other geeks Pallotto EK, et al. Orchestrated Testing: An Innovative Approach to a Multicenter Improvement Collaborative. American Journal of Medical Quality Oct 19, [Epub ahead of print] Piazza AJ, et al. SLUG Bug: Quality Improvement with Orchestrated Testing Leads to NICU CLABSI Reduction. Pediatrics Jan;13(1):1-12. Moen RD, Nolan TW, Provost LP. Quality Improvement Through Planned Experimentation. 2 nd ed. New York: McGraw-Hill, 1999

78 Polaris C Polaris F Hallway 1 RB&C MetroHealth Dayton Children s Atrium Mercy Anderson Genesis Mt Carmel East Springfield Mercy Fairfield Elyria The Christ Hospital Licking Kettering NCH Riverside NCH Mt Carmel St Ann s UC Health St Rita s Medical Center Marion General Trumbull Hallway 2 Hallway 3 Nationwide Children s Aultman Hospital Toledo Children s Mercy Lorain Soin Medical Center Mercy West Gemini B Mt Carmel West Hillcrest Good Samaritan Dayton CCHMC Dublin Methodist Miami Valley Hospital Storyboard Rounds Steal Shamelessly; Share Seamlessly Bethesda North Fairview Hospital OSU NICU/OSU Well Baby Mercy Children s Southview Akron Children s/summa Gemini C Good Samaritan Tri-Health Mercy Canton NCH Grant Akron St E/Mahoning Valley Southern Ohio Medical Center Fort Hamilton **LUNCH is 12:30 1:15PM after Storyboard Rounds**