Comparative Effectiveness Research In Pediatric Surgery: Current Status and Opportunities Martin L. Blakely, MD, MS Department of Pediatric Surgery Section of Surgical Sciences Vanderbilt University School of Medicine Vanderbilt Children s Hospital
Objectives 1. Comparative Effectiveness Research Fundamentals 2. State of RCTs in Pediatric General Surgery 3. Examples of Pragmatic RCTs in Pediatric General Surgery 4. Opportunities for Pediatric Surgeons and General surgery residents and Many Others
Research v. Clinical Practice?
This scenario more accurately describes research or routine clinical practice? 1. Research 2. Clinical practice 3. Describes both about the same 33% 33% 33% 1 2 3
Milestones 2004 NIH Roadmap CTSA = Clinical and Translational Research Award Purpose: create an academic home environment to develop the discipline of clinical and translational science T1 and T2 terminology presented 2009 American Recovery and Reinvestment Act 1.1 billion for CER IOM, National Priorities for CER
An Unprecedented Investment in Comparative Effectiveness Research 2005-2009 - AHRQ received $129 million from Congress for comparative effectiveness research 2009 - The American Recovery and Reinvestment Act contained $1.1 billion for comparative effectiveness research, including $300 million to AHRQ Stakeholder Input and Involvement Horizon Scanning Evidence Synthesis Evidence Gap Identification Evidence Generation Research Training and Career Development 7
AHRQ Agency for Healthcare Research and Quality The health services research arm of the US Dept of Health and Human Services Sister agency to NIH (biomedical rsch mission) Focus: Quality improvement and patient safety Outcomes and effectiveness of care Clinical practice and technology assessment Health care organization and delivery systems
Patient Protection and Affordable Care Act Section 6301: Patient-Centered Outcomes Research Name change: Comparative Effectiveness Research = Patient- Centered Outcomes Research Patient-Centered Outcomes Research Institute Independent, nonprofit Institute with public- and private-sector funding Sets priorities and coordinates with existing agencies that support patient-centered outcomes research Prohibits findings to be construed as mandates on practice guidelines or coverage decisions and contains patient safeguards 10
www.pcori.org
$3,000,000,000 from now through 2020.
CER Question A randomized trial is planned to compare inguinal hernia repair in premature infants prior to discharge from the NICU versus repair beyond 55 weeks post-conceptual age (well after discharge). Anesthetic management will be according to the current practices at each participating site, as will other associated treatments
The inguinal hernia trial described is an example of what type of translational research? 1. T1 2. T2 3. T3 4. Not translational research 25% 25% 25% 25% 1 2 3 4
The 3T s Road Map to Transforming U.S. Health Care Basic biomedical science Clinical efficacy Clinical effectiveness T1 T2 T3 knowledge knowledge Improved health care quality and value and population health Key T1 activity to test what care works Clinical efficacy research Key T2 activities to test who benefits from promising care Outcomes research Comparative effectiveness research Health services research Key T3 activities to test how to deliver high-quality care reliably and in all settings Measurement and accountability of health care quality and cost Implementation of Interventions and health care system redesign Scaling and spread of effective interventions Research in above domains Source: JAMA, May 21, 2008: D. Dougherty and P.H. Conway, pp. 2319-2321. The 3T s Roadmap to Transform U.S. Health Care: The How of High-Quality Care.
A CRITICAL APPRAISAL OF PUBLISHED RANDOMIZED TRIALS WITHIN PEDIATRIC GENERAL SURGERY DURING THE TIME PERIOD 2000-2009 Martin L. Blakely, MD, MS 1, Lillian Kao, MD, MS 2, Rupa Seetharamaiah, MD 1, KuoJen Tsao, MD 2, Eunice Y. Huang, MD, MS 1, Kevin P. Lally, MD, MS 2. 1 University of Tennessee Health Science Center, Memphis, TN, USA, 2 University of Texas Health Science Center, Houston, TX, USA.
CONSORT Adherence Primary outcome stated Adequate sample size calculation Method of randomization sequence generation reported Method of allocation concealment reported Attrition details given Intent to treat analysis documented Any blinding (e.g. outcome assessment) Chan and Altman. Lancet 2005; 365:1159-1162
Jadad Scoring System Score range: 0 5 Elements of Jadad Score: Described as randomized trial? Randomization sequence generation appropriate? Described as double-blind? Method of blinding appropriate? Description of attrition? Jadad AR et al., Controlled Clin Trials 1996;17:1-12
Number of RCTs by Year Total N = 73 2009 2008 2007 2006 2005 2004 2003 2002 2001 2000 0 2 4 6 8 10 12 14 16
Primary Outcome Specified? Yes (43) No (30) 41% 59% Sample Size Calculation? 74% 26% Yes (19) No (54)
Results Blinding Intent to treat Attrition details Allocation concealment Yes No Randomization sequence 0% 50% 100%
Jadad Results 30 25 Trial (N) 20 15 10 30% 5 0 0 1 2 3 4 5 Score
Conclusions The number of published RCTs within pediatric general surgery (approximately 7 / year) has increased from prior decades Adherence to CONSORT guidelines is fairly low as is Jadad scoring By increasing adherence to CONSORT guidelines, the quality of randomized trials can be improved
A pragmatic RCT differs from an efficacy trial in which ways? A. Follows a strict treatment protocol, controlling many treatment aspects B. Meant to simulate typical clinical setting C. Results are likely to be less relevant for patient outside the trial compared to trial patients D. None of the above 25% 25% 25% 25% A. B. C. D.
Surgical Treatment of Perforated Appendicitis
Background / Rationale Most common GI disorder leading to hospital admission in children < 18 years of age >1/3 of children with appendicitis ruptured LBCMC perforated appendicitis rate = 35% Treatment of ruptured appendicitis is controversial among pediatric surgeons No published prospective, randomized trial evaluating early and interval appendectomy
Interventions All patients Intravenous Ceftazidime + Clindamycin q 8 hr CVL at discretion of surgeon Atb stopping criteria and patient DC criteria Early appendectomy Within 24 hours of admission Interval appendectomy Planned 6-8 weeks after initial discharge Earlier appy done if considered in best interest of pt.
Objectives / Hypotheses Hypothesis: Early appendectomy will be associated with decreased time away from normal activities without an increase in the overall morbidity rate compared with delayed (interval) appendectomy.
Primary Outcome Time away from normal activities (days) Patient-centered outcome Measured from multiple sources Composite of objective and subjective time periods In-patient Out patient with picc / i.v. antibiotics Outpatient with symptoms Outpatient in office, imaging study, ER
Sample Size Determination Clinically meaningful difference in 1 outcome = 5 days To detect 5 day difference (21 v. 16; SD 10), 64 patients / group required Alpha =.05, power = 80%, 2-tailed test Limited power to detect small differences in adverse event rates Projected 80% enrollment rate of eligible patients
FLOW DIAGRAM Assessed for Eligibility N = 247 Randomized N = 131 Excluded, n=116 -Ineligible, n=73 -Refused, n=43 Early Appy, n = 64 Received Rx = 64 Interval Appy, n = 67 Received Rx =65 Lost to Follow up = 0 Lost to Follow up = 0 Analyzed, n = 64 Analyzed, n = 67
Baseline Data Characteristic Early (n=64) Interval (n=67) P value Age - yr 10.1 (1.8 16.6) 9.5 (2.6 17.3) NS Male sex n (%) 40 (63 %) 33 (49 %) NS Race n (%) NS Black 30 (47 %) 28 (42 %) Hispanic 13 (20 %) 12 (18 %) White 20 (31 %) 26 (39 %) Other 1 (2 %) 1 (1 %) Duration of sxs d 3.4 (1 14) 3.4 (1 17) NS ED temp - C 38.0 (36.2 39.8) 38.4 (36.2 40.2).01 Admit WBC - 17.6 (5.2 34.3) 16.8 (5.6 28.7) NS IAA at admit n (%) 25 (39 %) 26 (39 %) NS SBO at admit - n (%) 6 (18 %) 9 (13 %) NS ICU admit n (%) 2 (3 %) 3 (4 %) NS
Descriptive Findings Finding Early Interval P value CVL 28/64 (44%) 58/67 (87%) <.0001 DC with CVL 6/64 (9%) 29/67 (43%) <.0001 IR procedure 11/64 (17%) 15/67 (22%) NS Total atb duration 13 (3 30) 15 (6-33) NS Interval from Dx to appy 1 51 - RA confirmed 57/64 (89%) 66/67 (99%) - Protocol violation 1/64 3/67 -
Primary Outcome Early Mean = 13.8 Median = 11 25 Range: 4-30 20 Interval Mean = 19.4 Days Days 15 10 Median = 19* 5 Range: 7-45 0 Early Interval *Wilcoxon rank sum, p=.0001
Failed Interval Patients 23/67 (34.3%) of interval appendectomy patients had appendectomy earlier than planned Reasons for failure Small bowel obstruction n = 10 (43%) Persistent symptoms n = 6 (26%) Recurrent appendicitis n = 5 (22%) Intra-abdominal abscess n = 1 Parental demand n = 1
Adverse Events Event Early (n = 63) Interval (n = 67) P value Any adverse event n (%) 18 (28.1 %) 35 (52.2 %).007 Intra abd abscess 12 (18.8 %) 25 (37.3 %).02 SBO 0 7 (10.4 %).01 Wound infection 6 (9.4 %) 6 (9.0 %) NS Re-admission 3 (4.7 %) 20 (29.9 %).0002 CVL related 1 (1.6 %) 4 (6.0 %) NS IR related 0 1 (1.5 %) NS Other 2 (3.1 %) 11 (16.4 %).02 Recurrent appendicitis in interval group: 6/67 (9 %)
Interpretation Patients treated with early appendectomy return to normal activities more quickly than those treated with interval Early appendectomy is associated with fewer adverse events Diagnostic accuracy rate of RA is ~90% Recurrent appendicitis rate is ~9% if appendix not removed early A subset of interval patients has no adverse event and has appendectomy performed as planned
Conclusions In this randomized trial, early appendectomy was associated with: decreased time away from normal activities fewer adverse events shorter length of hospital stay
The major obstacle to conducting randomized trials in surgery? 1. Lack of funding 2. Parental refusal 3. Surgeon unwillingness to randomize patients 4. Lack of dedicated academic time for surgeon investigator 25% 25% 25% 25% 1 2 3 4
Surgical Management of Severe Necrotizing Enterocolitis
Moss Trial Results Variable Lap Drain RR Mortality All patients 22/62 (35.5) 19/55 (34.5) 1.03(.63-1.69) <1000 gm 15/45 (33.3) 16/45 (35.6) 0.94(.53-1.66) >/=1000 g 7/17 (41.2) 3/10 (30) 1.37(.46-4.14) PN depend. All patients 16/40 (40) 17/36 (47.2) 0.85(.51-1.42) <1000 gm 12/30 (40) 15/29 (51.7) 0.77(.44-1.36) Moss et al., NEJM 2006;354:2225-34
UK Trial Results 6 month survival Drain 18/35 (51%) Laparotomy 21/33 (64%) Rees, Eaton, Kiely et al., Ann Surg 2008;248:44 51
Outcomes at 18-22 Months 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Mortality NDI Death or NDI Lap Drain Adjusted OR for death or NDI at 18-22 months 0.55 (0.18-1.67)
Results Summary Drainage more common initial surgical procedure NEC slightly worse prognosis than IP Tight correlation between preoperative and intraoperative diagnosis of NEC versus IP Neonatal mortality similar Trend towards increased neurodevelopmental impairment with initial drain Subgroup of infants survive with drain only Blakely et al., Pediatrics 2006;117:680-687
Necrotizing Enterocolitis Surgery Trial (NEST) NICHD Neonatal Research Network
NICHD Neonatal Research Network
Primary Hypothesis Among surgically treated ELBW infants with NEC or IP, laparotomy results in a higher rate of survival without neurodevelopmental impairment than does initial drainage (35% vs. 20%; NNT = 6-7)
18-22 month Neurodevelopmental Assessment Neurologic exam Bayley Scales of Infant Develop. III Social/Behavioral eval BITSEA Neurodevelopmental impairment (NDI) Moderate severe cerebral palsy Bayley III cognitive score <85 <20/200 bilateral vision Permanent hearing loss
NEST Enrollment 200 100
Reasons Not Consented Parent Unavailable 4% Parent Refused 39% Consent Not Requested 16% Neonatologist Refused 4% Surgeon Refused 20% Neonatologist + Surgeon Refused 13% (53%) Other 3%
Current Opportunities
Timing of IH Repair in Premature Prior to NICU discharge Reduce hernia related risks Higher anesthetic risk After discharge, beyond 55 weeks PCA No prior definitive trial Infants
CDHSG Active Centers (59)
The Congenital Diaphragmatic Study Group
Dissemination of Results and Impact of RCTs Once well designed, funded, RCT completed What if no one reads it? What if they read it, but practice is not changed? Evaluate methods to more actively disseminate trial results and study impact of trial results.
Patient and Parent-Reported Outcomes The impact of our treatment choices on families have been rarely studied Can we utilize parent reporting of important outcomes to help inform our treatment decisions? Perforated appendicitis treatment QOL comparison Parent satisfaction
RCT Design and Analysis Frequentist Analysis Bayesian Analysis Disallows prior information Disallows analysis as patients are enrolled Dichotomous outcomes p< or >.05 Uses prior information, even subjective Updates probability with each new patient outcome Results alter prior probability and result in new posterior probability
Perioperative Quality Improvement Efforts Surgical Care Improvement Project SCIP No similar effort in pediatric surgery until now Utilization of PHIS database Pediatrics NSQIP
Study Questions
Does it really matter?
Does it really matter?
Educational opportunities CONSORT guidelines Trial design Critical review Statistical analysis Bayesian versus frequentist approach
Comparative Effectiveness Research in Pediatric Surgery Many therapies aren t based on evidence Published trials do not adhere to well accepted guidelines This is all great news for current students, residents, fellows, and pediatric surgeons Many opportunities