Protocol This trial protocol has been provided by the authors to give readers additional information about their work. Protocol for: Kerlin MP, Small DS, Cooney E, et al. A randomized trial of nighttime physician staffing in an intensive care unit. N Engl J Med 2013;368:2201-9. DOI: 10.1056/NEJMoa1302854
A Randomized Clinical Trial of Nighttime Intensivist Staffing in a Medical Intensive Care Unit Manuscript # 13-02854 AUTHORS Meeta Prasad Kerlin, MD MSCE 1 Dylan S. Small, PhD 2,3 Elizabeth Cooney, MPH 2,4,5 Barry D. Fuchs, MD MS 1 Lisa M. Bellini, MD 1 Mark E. Mikkelsen, MD MSCE 1,4,5 William D. Schweickert, MD 1 Nicole B. Gabler, PhD MHA 4,5 Michael O. Harhay, MPH MBE 4,5 John Hansen-Flaschen, MD 1 Scott D. Halpern, MD PhD 1,2,4,5,6 CORRESPONDING AUTHOR: Scott D. Halpern, MD PhD Perelman School of Medicine, University of Pennsylvania 719 Blockley Hall 423 Guardian Drive Philadelphia, PA 19104-6021 Office: 215-898-1462 Fax: 215-573-4132 Email: shalpern@exchange.upenn.edu This supplement contains the following items: 1. Original protocol, final protocol, summary of changes. 2. Original statistical analysis plan, final statistical analysis plan, summary of changes
ORIGINAL RESEARCH PROTOCOL Background Available evidence suggests that intensivist management of critically ill patients improves patient outcomes, suggesting that greater intensivist coverage might be better still. Effects of 24-hour intensivist coverage in ICUs are unknown. Two single-center studies with before-after designs suggest (1) minimal effects on patient-centered outcomes and family satisfaction, (2) improved staff satisfaction and perceived burn-out risk and patient safety, and (3) reductions in cost attributable to reduced LOS. Without knowing the effectiveness of 24-hour intensivist coverage, hospital administrators, insurers, and policy makers cannot gauge the economic impact or sustainability of this model. As 24-hour intensivist staffing rapidly is becoming the de facto standard of care at centers with the resources to provide it, there is a narrow window of opportunity to study its effects. Objective To study the comparative effectiveness and cost effectiveness of nocturnal intensivist staffing in the Medical Intensive Care Unit (MICU) of the Hospital of the University of Pennsylvania (HUP). Study Design We will conduct a randomized clinical trial of two different nocturnal staffing models: a traditional model of night coverage with in-hospital medical interns and residents (control) versus night coverage (from 7pm to 7am) by an in-hospital attending intensivist physician in addition to the traditional housestaff coverage (intervention). Under the traditional model, the attending intensivist physician who rounds on patients during the daytime hours is available to in-hospital medical residents by phone and/or beeper for case discussion and consultation, and comes into the hospital infrequently at his or her discretion based on specific patient needs. This attending physician is the same person for all consecutive nights during an assigned rotation. In the intervention, a nocturnal intensivist will be physically present in the medical ICU at HUP. The nocturnal intensivist will be a different person than the daytime attending physician and will likely be different people from one night to the next during consecutive nights of coverage. His/her responsibilities will include, but not be limited to, evaluation of newly admitted patients and any clinically deteriorating patients, supervision of bedside procedures performed by residents, triage of MICU beds, and leading nighttime rounds with the housestaff team. We will perform a block randomization scheme where we will randomly assign single weeks (i.e., seven consecutive days, Monday to Sunday) to either the traditional model or the intervention, in two week blocks that will correspond to current daytime attending assignments. We choose a block size of two weeks in order to balance the exposure with regard to daytime staffing (i.e., in each 2-week rotation staffed by an individual daytime attending, there will be seven consecutive days of traditional staffing and seven consecutive days of the intervention staffing model) and for feasibility of assigning attending physicians for night coverage. We will use electronic random-number generation to determine which week of each two-week block will be assigned to the control and intervention staffing models. We will exclude a 2-week winter holiday block from randomization because daytime fellow and attending staffing and resident staffing differs during that period compared to the rest of the year. All current MICU intensivists (board-eligible or board-certified in Critical Care Medicine and credentialed for patient-care responsibilities at the Hospital of the University of Pennsylvania) will be eligible to participate in nocturnal shifts, at a compensation of $2400 per 12-hour shift. The study period will begin September 12, 2011 and end June 30, 2012, in accordance with funding support provided by Penn Medicine.
Study Population The primary study population will include all patients admitted to the HUP MICU during the study period. We will exclude subsequent admissions for all patients with multiple admissions. Study variables and data sources The primary exposure variable will be coverage on the night of admission (intervention or control). The primary outcome variable will be ICU length-of-stay (LOS). Secondary outcomes will include hospital LOS, ICU mortality, in-hospital mortality, duration of mechanical ventilation, and readmission to the MICU among ICU survivors. Covariates to be included in analyses of patient outcomes will include APACHE III score as a measure of severity of illness and location of patient prior to ICU admission. We will also collect data on patient demographic variables, such as age, gender, and race/ethnicity. Hospital-, ICU-, and patient-level data for this study will primarily be obtained from the HUP ICU Registry, which compiles data from various online medical records, patient databases, and tracking systems. The HUP ICU Registry was created through an IRB-approved initiative titled Critical Care Outcomes (Protocol number: 810651). It includes patient level data, including certain patient demographics, clinical data such as severity of illness as measured by the APACHE score, and outcomes data such as mortality and length-of-stay. It also includes operational data, such as time of admission to the MICU and ICU census. The ICU Registry collects names and medical record numbers (MRNs) to allow for linkage from multiple automated data sources and manual data entry. To ensure privacy ICU Registry researchers only access data containing PHI on computers secure enough to have Horizon Performance Manager (HPM) access. HPM is a software program that integrates patient-level clinical data with financial data (i.e. patient billing and accounting). It is used to evaluate hospital performance based on profitability and efficiency. Due to the sensitive nature of both the patient specific and financial data, HPM is only used on the most secure computers. All patient data for this study is stored on secure servers and is not disclosed to anyone outside of the ICU Registry research team. We will obtain data from nocturnal intensivists about their nightly activities from an anonymous survey completed at the end of each night shift. We will also obtain data from medical residents about their experience during the HUP MICU rotations, including their perceptions of nocturnal intensivists, through mandatory rotation evaluations, administered through the Office of Graduate Medical Education and provided to us without linking individual residents to their responses. Subject consent and confidentiality We request a waiver of informed consent for the study population for this study, i.e. patients admitted to the MICU during the study period whose data is collected as part of the ICU registry. We also note that the attending physicians who will work during days and nights during the study period do not meet the Common Rules definition of human subjects, because there will be no study-related interaction with them and no data about them will be collected. Because they are not human research subjects, they will not be asked to consent. This study poses no more than minimal risk to patients, as the addition of a nocturnal intensivist to the standard of care (immediate availability of housestaff and attending intensivist accessible by phone) is highly unlikely to result in any harm to the patient. Indeed, many peer institutions have rolled out nocturnal intensivist coverage without examining it, assuming (but never proving) that it could only help patients. By contrast, patients admitted to the MICU on nights without nocturnal intensivist coverage
also cannot be said to be harmed by the study because HUP made an institutional decision to cover only a fraction of nights with nocturnal intensivists long before the idea to study the effects of the novel staffing model was proposed.
FINAL RESEARCH PROTOCOL Background Available evidence suggests that intensivist management of critically ill patients improves patient outcomes, suggesting that greater intensivist coverage might be better still. Effects of 24-hour intensivist coverage in ICUs are unknown. Two single-center studies with before-after designs suggest (1) minimal effects on patient-centered outcomes and family satisfaction, (2) improved staff satisfaction and perceived burn-out risk and patient safety, and (3) reductions in cost attributable to reduced LOS. Without knowing the effectiveness of 24-hour intensivist coverage, hospital administrators, insurers, and policy makers cannot gauge the economic impact or sustainability of this model. As 24-hour intensivist staffing rapidly is becoming the de facto standard of care at centers with the resources to provide it, there is a narrow window of opportunity to study its effects. Objective To study the comparative effectiveness and cost effectiveness of nocturnal intensivist staffing in the Medical Intensive Care Unit (MICU) of the Hospital of the University of Pennsylvania (HUP). Study Design We will conduct a randomized clinical trial of two different nocturnal staffing models: a traditional model of night coverage with in-hospital medical interns and residents (control) versus night coverage (from 7pm to 7am) by an in-hospital attending intensivist physician in addition to the traditional housestaff coverage (intervention). Under the traditional model, the attending intensivist physician who rounds on patients during the daytime hours is available to in-hospital medical residents by phone and/or beeper for case discussion and consultation, and comes into the hospital infrequently at his or her discretion based on specific patient needs. This attending physician is the same person for all consecutive nights during an assigned rotation. In the intervention, a nocturnal intensivist will be physically present in the medical ICU at HUP. The nocturnal intensivist will be a different person than the daytime attending physician and will likely be different people from one night to the next during consecutive nights of coverage. His/her responsibilities will include, but not be limited to, evaluation of newly admitted patients and any clinically deteriorating patients, supervision of bedside procedures performed by residents, triage of MICU beds, and leading nighttime rounds with the housestaff team. We will perform a block randomization scheme where we will randomly assign single weeks (i.e., seven consecutive days, Monday to Sunday) to either the traditional model or the intervention, in two week blocks that will correspond to current daytime attending assignments. We choose a block size of two weeks in order to balance the exposure with regard to daytime staffing (i.e., in each 2-week rotation staffed by an individual daytime attending, there will be seven consecutive days of traditional staffing and seven consecutive days of the intervention staffing model) and for feasibility of assigning attending physicians for night coverage. We will use electronic random-number generation to determine which week of each two-week block will be assigned to the control and intervention staffing models. We will exclude a 2-week winter holiday block from randomization because daytime fellow and attending staffing and resident staffing differs during that period compared to the rest of the year. All current MICU intensivists (board-eligible or board-certified in Critical Care Medicine and credentialed for patient-care responsibilities at the Hospital of the University of Pennsylvania) will be eligible to participate in nocturnal shifts, at a compensation of $2400 per 12-hour shift. The study period will begin
September 12, 2011 and end September 11, 2012, with an additional 90 days of in-hospital follow-up, in accordance with funding support provided by Penn Medicine. Study Population The primary study population will include all patients admitted to the HUP MICU during the study period. We will exclude subsequent admissions for all patients with multiple admissions. We will also exclude any patients who have a brief LOS that results in no exposure to nighttime hours; patients with missing data for APACHE III scores; and patients with an ICU LOS less than 4 hours, since they are ineligible for an APACHE III score. Study variables and data sources The primary exposure variable will be coverage on the night of admission (intervention or control). The primary outcome variable will be ICU length-of-stay (LOS). Secondary outcomes will include hospital LOS, ICU mortality, in-hospital mortality, and readmission within 48 hours to the MICU among ICU survivors. Covariates to be included in analyses of patient outcomes will include APACHE III score as a measure of severity of illness and location of patient prior to ICU admission. We will also collect data on patient demographic variables, such as age, gender, and race/ethnicity. Hospital-, ICU-, and patient-level data for this study will primarily be obtained from the HUP ICU Registry, which compiles data from various online medical records, patient databases, and tracking systems. The HUP ICU Registry was created through an IRB-approved initiative titled Critical Care Outcomes (Protocol number: 810651). It includes patient level data, including certain patient demographics, clinical data such as severity of illness as measured by the APACHE score, and outcomes data such as mortality and length-of-stay. It also includes operational data, such as time of admission to the MICU and ICU census. The ICU Registry collects names and medical record numbers (MRNs) to allow for linkage from multiple automated data sources and manual data entry. To ensure privacy ICU Registry researchers only access data containing PHI on computers secure enough to have Horizon Performance Manager (HPM) access. HPM is a software program that integrates patient-level clinical data with financial data (i.e. patient billing and accounting). It is used to evaluate hospital performance based on profitability and efficiency. Due to the sensitive nature of both the patient specific and financial data, HPM is only used on the most secure computers. All patient data for this study is stored on secure servers and is not disclosed to anyone outside of the ICU Registry research team. We will obtain data from nocturnal intensivists about their nightly activities from an anonymous, paper questionnaire completed at the end of each night shift. We will also obtain data from medical residents who complete at least one 2-week rotation during the first six months of the study period regarding their perceptions of nocturnal intensivists, through an anonymous electronic survey. Subject consent and confidentiality We request a waiver of informed consent for the study population for this study, i.e. patients admitted to the MICU during the study period whose data is collected as part of the ICU registry. We also note that the attending physicians who will work during days and nights during the study period do not meet the Common Rules definition of human subjects, because there will be no study-related interaction with them and no data about them will be collected. Because they are not human research subjects, they will not be asked to consent.
This study poses no more than minimal risk to patients, as the addition of a nocturnal intensivist to the standard of care (immediate availability of housestaff and attending intensivist accessible by phone) is highly unlikely to result in any harm to the patient. Indeed, many peer institutions have rolled out nocturnal intensivist coverage without examining it, assuming (but never proving) that it could only help patients. By contrast, patients admitted to the MICU on nights without nocturnal intensivist coverage also cannot be said to be harmed by the study because HUP made an institutional decision to cover only a fraction of nights with nocturnal intensivists long before the idea to study the effects of the novel staffing model was proposed.
RESEARCH PROTOCOL SUMMARY OF CHANGES Changes made between the original and final research protocols for SUNSET-ICU, and the rationale for them, are summarized below. Changes made to the analytic plan, including changes to the subgroup and sensitivity analyses, are detailed in a separate document. We specified additional exclusion criteria to appropriately exclude (1) patients who were ineligible for or had missing data for an APACHE III score, which we had specified a priori would be included in all analyses for purposes of risk adjustment; and (2) patients who would not be exposed to the intervention (patients whose ICU stays did not include any nighttime hours) and. These additional criteria led to the exclusion of 53 patients in total out of the 1,908 who were screened for eligibility (2.8%), and were distributed as follows: 27 patients were ineligible for APACHE calculation because their ICU stays were shorter than 4 hours, 11 patients did not have APACHE calculations due to the absence of 1 or more variables needed to calculate the score, and 15 patients had ICU stays that did not expose to nighttime hours. We clarified and further specified our secondary outcomes. We removed duration of mechanical ventilation as an outcome because we discovered that we could not reliably obtain data on the timing of intubation and extubation (duration of mechanical ventilation) through electronic or printed records. We clarified what we considered to be an ICU readmission, defining it as a readmission to the MICU within 48 hours of MICU discharge without having first left the hospital, consistent with the proposed definition of the European Society of Intensive Care Medicine. Based on the input of the residency program leaders, we changed our proposed mechanism of acquiring data regarding resident perceptions. Rather than merely using standard rotation evaluations, which we believed would not meaningfully capture specific perceptions of the intervention, we administered a separate, anonymous survey designed for this study. Because we were concerned about the possibility of seasonal effects, we had initially requested funding for a year-long trial. This was not initially granted. However, 3 months into the trial the University of Pennsylvania Health System agreed to the additional funding to extend the trial from 9 months to a full calendar year. We updated our sample size and power calculations accordingly.
ORIGINAL ANALYTIC PLAN Statistical analysis plan All analyses of patient outcomes will be at the patient level. We will use standard descriptive statistics to summarize patient demographic characteristics and study variables. Analyses of the primary outcome (ICU LOS) will be stratified by patients who do and do not survive. We will use Cox proportional hazard models to identify relationships between nocturnal intensivist coverage and time to ICU discharge. Intention-to-treat analyses will model intensivist coverage on the night of admission as present or absent; per-protocol analyses will consider intensivist coverage as a timevarying covariate throughout the ICU stay. In this latter analysis, patients who die in the ICU or who are transferred to another ICU will be censored. We will repeat analyses of the primary outcome in three pre-specified subgroups secondarily: 1. nighttime admissions from 7pm to 7am during the study period (hypothesizing that intensivist impact might be greatest during golden hours around admission) 2. admissions on Mondays and Tuesdays only (maximizing early exposed vs. nonexposed status) 3. admissions with a predicted probability of in-hospital mortality (APACHE III) in the top quartile of the sample (hypothesizing that intensivists impact would be greatest for the sickest patients) Analyses of secondary outcomes of hospital LOS and duration of mechanical ventilation will be performed using the primary exposure definition (exposure status on the first night of admission) and including all patients admitted during the study period. Analyses of secondary outcomes of ICU and hospital mortality will be performed using multivariable logistic regression models. All models in primary and secondary analyses will include covariates for APACHE III score as a measure of severity of illness and location of patient prior to ICU admission. All analyses will be performed using SAS and Stata analytic software.
FINAL ANALYTIC PLAN Statistical analysis plan All analyses of patient outcomes will be at the patient level. We will use standard descriptive statistics to summarize patient demographic characteristics, study variables, and survey data. For the primary outcome of ICU LOS, we will use Cox proportional hazard models to study relationships between nocturnal intensivist coverage (defined as exposure on the first night during a patient s admission) and time to ICU discharge. We will censor patients who die in the ICU or who are transferred to another ICU. All analyses will use an intention-to-treat approach. Analyses of secondary outcomes of hospital LOS will be performed using the primary exposure definition (exposure status on the first night of admission) for two populations: (1) all patients admitted during the study period and (2) only nighttime admissions, defined as those occurring between 5pm and 5am, as these patients are likely to be fully evaluated by nocturnal intensivists on intervention nights. Analyses of secondary outcomes of ICU and hospital mortality will be performed using multivariable logistic regression models. In sensitivity analyses to test the robustness of our assumptions, we will vary the definitions of the exposure, outcomes, and patient population in the following ways: 1. Define the exposure as a time-varying covariate representing the cumulative proportion of nights exposed to the intervention. 2. Define ICU LOS as the time to the bed request time, rather than the actual discharge time, in order to evaluate the possibility that ward bed availability influenced ICU LOS in ways that would not be amenable to change with intensivist staffing. 3. Defined all patients discharged to hospice as deaths, as opposed to considering inpatient hospice referrals as death and home hospice referrals as survivors in our primary analysis. 4. Limit the population to only those patients with 100% or 0% exposure to the intervention, to maximize the contrast between the two groups. We will also test the robustness of the methodological approach of using time-to-event models with deaths by using an alternative approach, whereby we define ICU death as equivalent to the longest LOS (i.e., the worst possible outcome) and perform log-rank analyses (ROSENBAUM, P. R. The place of death in the quality of life. Statist. Sci. 2006; 21: 313 316). We will test for interaction between the intervention and two variables: 1. APACHE III score: We will include a multiplicative interaction term between the primary exposure (coverage on the first night during the admission) and a continuous variable for APACHE III, to test whether there is a difference in the association for patients with different severity of illness. We will also evaluate the effect of the intervention stratified by quartile of APACHE III score. 2. Time of year: We will create a binary variable to identify periods of lower (July Sept) and higher (Sept June) levels of housestaff experience, to test whether there is a difference in the effectiveness of nighttime intensivists based on the experience level of the in-hospital medical residents (the so-called July effect ).
With an anticipated sample size of 1,408 patients (4 eligible patients per day for one year, excluding the winter holiday block), the primary analysis would have 86% power to detect a hazard ratio of 1.2 for the outcome of time-to-icu-discharge with a two-sided significance level of α=0.05. All models in primary and secondary analyses will include covariates for APACHE III score as a measure of severity of illness and location of patient prior to ICU admission. All analyses will be performed using SAS and Stata analytic software.
ANALYTIC PLAN SUMMARY OF CHANGES Changes made between the original and final analytic plans for SUNSET-ICU, and the rationale for them, are summarized below. Changes made to the research protocol are detailed in a separate document. With input from the statistical team (led by Dylan Small, PhD), we modified our analytic plan so that the primary analysis would be a time-to-event (ICU discharge) model, censored on death, rather than stratifying the analyses by survival status. Because a significant proportion of ICU patients die, analyses of ICU length of stay (LOS) need to account for death. Otherwise, randomized clinical trials (RCTs) of promising interventions could appear beneficial by reducing ICU LOS merely by enabling or even promoting more rapid death. Methods work led by Dr. Small concurrent with the start of the trial revealed the fallacy of an approach of stratifying LOS analyses by whether patients died or survived in the ICU, as we had originally proposed and has been done occasionally in other ICU-based RCTs. Imagine a new intervention that saved 5% more lives compared with the standard of care. These extra 5% of lives saved would have been on the margins of death (i.e., would have died without the intervention), and so would be expected to have relatively long ICU LOS based on their severity of illness. Thus, a successful intervention that reduces ICU mortality would likely increase ICU LOS among survivors. We therefore changed our approach accordingly. Technically, this approach of censoring deaths relies on the assumption that death would be independent of length of stay (i.e., there would be no informative censoring). This assumption may not always be met. However, additional work led by Dr. Small using simulated data revealed that even in the presence of informative censoring, this approach almost universally yielded accurate results. We further decided the check the assumption empirically in our RCT data by performing a rank-based test, developed by Paul Rosenbaum, PhD of the Wharton School, in which rather than censoring deaths, we coded deaths as the longest possible LOS (ROSENBAUM, P. R. The place of death in the quality of life. Statist. Sci. 2006; 21: 313 316). This approach yielded virtually identical results as the censoring approach. We expanded our sensitivity analyses to test the robustness of the assumptions underlying our primary analytic plan more thoroughly, based on input from the research team. Regarding subgroup analyses and corresponding tests for interaction, we made the following 2 changes: o o Once we received approval from the Health System to continue the trial for a full calendar year, we added the July effect analysis whereby we would compare the effect of the intervention during the last 2.5 months of the trial (post July 1, when new residents started) with the prior 9.5 months, when more experienced residents were working in the ICU. We eliminated our proposed subgroup analysis of patients admitted on Mondays and Tuesdays. The goal of this analysis had been to create a group of patients with maximal cumulative exposure to either the intervention or the control. However, on further discussion among collaborators, we decided that a better approach to achieve the same aim would be to compare all patients with 100% exposure to the intervention with those with 0% exposure to the intervention. This analysis avoids the possibility that there is something unique about patients admitted on Mondays and Tuesdays, while achieving the starkest possible contrast.