Now We are Getting to the Hard Parts: An Analysis Files Perspective

Now We are Getting to the Hard Parts: An Analysis Files Perspective Steve Wilson Director, CDER/OTS/OB/DBIII 2010 CDISC Interchange Renaissance Hotel, Baltimore, MD November 3-4, 2010

Disclaimer Views expressed in this presentation are those of the speaker and not, necessarily, of the Food and Drug Administration

Truth-in-Advertising CDER Centric

Outline Background/Motivation Simpler Days Getting to the Hard Parts CDISC and Sponsors Reviewers Moving Ahead AFSWG FDA Track In Thinking Ahead

Other Offices and Programs Office of Policy and Planning Chief Operating Officer FDA/CDER Office of the Commissioner Management Council Chief Information Officer Strategic Planning Council Bioinformatics Board Office of Regulatory Affairs Center for Food Safety & Applied Nutrition Center for Drug Evaluation & Research Center for Biologics Evaluation & Research Center for Devices & Radiological Health Center for Veterinary Medicine National Center for Toxicological Research Bertoni, 2006

Mission of the FDA protecting the public health safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, our nation s food supply, cosmetics, and products that emit radiation. responsible for advancing the public health by helping to speed innovations

Simpler Days [1]: An Informal Survey of Statisticians Problems with Data (2002) Need to merge data sets Sensible data set names Consistent variable names Documentation of variables Derived variables Codes Drop-outs

Simpler Days [2] DIA 38th Annual Meeting, Lakeside Center at McCormick Place, Chicago, IL, USA, June 16-20, 2002 Electronic Submissions: An FDA Statistical Reviewer s Perspective Recommendations: Go Electronic! CDISC-it!

CDISC in the World of Standards 2000 ICH EU EMEA US FDA Japan MHW Health Care Providers & Pharmacies REGULATORY AUTHORITIES SDS ODM ADaM LAB Clinical Trials Pharmaceutical Industry EU USA Japan EFPIA PhRMA JPMA Standards: HL7, XML Models: NCI, OMG, RIM Dictionaries: MedDRA, LOINC

CDISC in the World of Standards 2003 International Conference on Harmonization (ICH) EFPIA EMEA JPMA MHLW KIKO U.S. FDA Protocol Std CDISC PhRMA TC: RCRIM U.S. Dept. of Health and Human Services (HHS) DICOM CDC Health Level 7 (HL7) NIH/NCI ISO NLM MedDRA ectd ADaM SDS ODM LAB Reference Information Model RIM Clinical Document Architecture LOINC SNOMED = Organization = Dictionary, Codelist = Standard = Model = Document Standard, or Architecture

CDISC in the World of Standards 2007 International Conference on Harmonization (ICH) EFPIA JPMA PhRMA World Health Organization (WHO) U.S. Dept. of Health and Human Services (HHS) EMEA MHLW U.S. FDA NIH/NCI CDC NLM RCRIM Technical Committee CDISC Health Level 7 (HL7) MedDRA CDASH ectd ADaM SDTMODM LAB Protocol Representation BRIDG Model Terminology Reference Information Model RIM LOINC CDA SNOMED = Organization = Dictionary, Codelist = Standard = Model = Document Standard, or Architecture

Getting to the Hard Parts CDISC & the Sponsors [1] What s the Meta-data team doing? Isn t this a bit Squishy? Do we have all of the Use Cases? Everyone is struggling with this issue> What s in Define 2.0? This is how we do it. Supposed to be dealing with value level Put out the draft for comment. How will it work with my internal standards.. Receptive at last two meetings, but nothing happened

Getting to the Hard Parts CDISC & the Sponsors [1] Somebody needs to work on this nomenclature Prevents us from implementing My programmers will do anything that is in there What do the reviewers really want Focus on the 80% Is it necessary to put that in there Don t you want it so that it can take everything that you can throw at it What level of complexity? It s more complicated than it needs to be Could it handle.? Got it down to a basic model Just have a big disclaimer.

Getting to the Hard Parts CDISC & the Sponsors [3] If we automate, my programmers will want to do it all We re planning a metadata repository What should this example look like? Do we need an example? Do statisticians need SDTM? Is HL7 an HHS standard? How do I transition from/modify my legacy systems? Bit of a moving target What question are we answering I ve lost track

The Hard Part -- Reviewers What is are data standards? Why should I care? Why do I need them? What is a data model? Sponsor submitted SDTM, doesn t want to submit analysis files. HL7 or CDISC? What s the difference? What is SDTM? ADaM? What is xml? What do I recommend to sponsors? Do statisticians need SDTM? CDER, CBER and CDRH? Do they have different requirements? Needs? How do I use this stuff? What are the tools: WebSDM, JStat, JMP, SAS, R? Why do these data look so weird? How do tools and data help me with review? What is review?

Some Evidence that CDER is Working on the Hard Part(s) Computational Science Center (CSC) Data Standards Plan Study Data Specifications & Analysis Transparency and the FDA Track

21 st Century Review and Analysis Files Statistical Reviewers Depend on the Sponsor s Submitted Data for this Review 17

Statistical Review: Day 1 Surprises Examples of what we were seeing: 7-study application with no disposition data Significant dropouts Took 2 months for the applicant to produce the datasets Application with no primary efficacy data Applicant stated they followed the guidance submitting SDTM Would take a statistical reviewer 6 months+ to create efficacy data Application with no treatment variable in any datasets Refuse-to-file if datasets were not updated in timely fashion Application with no data in EDR Only SDTM datasets on CD; Applicant told to submit analysis datasets by Day 74 of cycle Lost 2 months off review timeline

Office of Biostatistics Analysis Files Working Group (AFSWG) Work with CDER Computational Sciences Center Evaluate Current Guidance / Specifications Focus Groups with Statistical Reviewers Document Data Requests to Sponsors during NDA/BLA Review Work on Office/Division/Therapy-specific guidance -- template for sponsor project-level communications with medical review Establish system to respond to Requests for Information (ROIs) to assist OBI with questions from industry Improve Communications: Webpages, meetings, etc. Track, evaluate, provide feedback on statistical review of CDISC (SDTM and ADaM) submissions Assess training needs for statisticians

Study Data Specifications & Analysis

Study Data Specifications: Analysis Datasets & Programs(1) Analysis datasets are datasets created to support results presented in study reports, ISS, ISE and other analyses that enable a thorough regulatory review. Analysis datasets contain both raw and derived data. Sponsors should therefore augment SDTM with analysis data sets as described in the Analysis Datasets section.

Study Data Specifications: Analysis Datasets & Programs (2) Prior to submission, sponsors should contact the appropriate center s reviewing division to determine the division s analysis dataset needs. CDISC/ADaM standards for analysis datasets (http://www.cdisc.org/adam) may be used if acceptable to the review division Program files: The analysis datasets should be accompanied by the programs that were used to create the analysis datasets and the results presented in the application.

FDA Transparency Initiative

FDA Track: A Transparency/Metrics Opportunity Each FDA-TRACK program office collects, analyzes, and reports its performance measures and results via FDA-TRACK dashboards. These FDA-TRACK dashboards may include one or several program offices which contribute to similar public health objectives or program areas. The dashboards are published to this site quarterly following the completion of the quarterly briefing.

Office of Biostatistics Dashboard

Based on Reported Filing Meetings Scheduled During the Month Measure Apr 2010 May 2010 Jun 2010 Number of NDAs, BLAs & Efficacy Supplements w/ Electronic Datasets Percent with SDTM (one or more studies) Percent with ADaM [Version 2] (one or more studies) 9 33% 22%

Now We Can Systematically Track & Evaluate Review of CDISC Submissions! Training, Evaluation, Feedback, Communication 29

Designing and Building a Better Plane (While We Fly It) Associations between elements OMOP, Safety, Sentinel, Registrars Legacy data/comparative effectiveness Terminology/SHARE/endpoints EHRs EDC CDISC/HL7 Compliance Opportunities Life-cycle (IND, Approval, Supplements, Safety.) Tools and Review practice Regulatory Science Study designs Data standard planning Learning Lessons Continuous Improvement Common Understanding Collaboration/communication

Collaboration/Communication 2 nd Annual FDA/DIA CompSci Meeting An experiment in progress the PhUSE Wiki

PhUSE (Pharmaceutical Users Software Exchange) http://www.phuse.eu/index.aspx Independent non-profit organization run by volunteers Purpose is to create a forum for people who manage, analyze and report clinical data in the pharmaceutical industry Platform independent No product to sell Building on CDISC standards

PhUSE Wiki Sharing Program Code

Data Week in DC 2 nd Annual DIA/FDA CDER/CBER Computational Science Annual Meeting March 14-15, 2011

In Thinking Ahead This stuff is difficult/complicated Always forgetting where we came from.. We don t congratulate ourselves enough We all really need to use standards (data collection, tools, analysis, etc.) not just for submission to FDA When talking about EHRs and Standards, include regulatory research use case. Come with questions/come with solutions (or a willingness to roll-up your sleeves) Continue to explore / use CDISC standards Do this because it is the right thing to do think about what is best for the science Remember this is Science

Now We are Getting to the FUN Parts: An Analysis Files Perspective THANK YOU stephen.wilson@fda.hhs.gov

Travel Safely