PERFORMANCE IN INITIATING CLINICAL RESEARCH (PI) CLINICAL TRIAL ADJUSTMENT PROCESS HRA APPROVED TRIALS Version Control Pages 7 9 Notes/action column updated 1
Acronyms CTP DSC DSS FPR HRA MHRA REC SOP TMB VRA Clinical Trial Performance Date Site Confirmed Date Site Selected First Patient Recruitment Health Research Authority Medicines and Healthcare products Regulatory Agency Research Ethics Committee Standard Operating Procedure Trials Meeting the Benchmark Valid Research Application Purpose This Standard Operating Procedure (SOP) describes the process of data adjustment for the Performance in Initiating Clinical Research exercise which informs the NIHR performance metrics for providers of NHS services. The intention of this SOP is to outline clearly and transparently the way in which the adjustment process is applied to submitted clinical trials; to promote consistency of execution of the process and common mutual understanding of expectations and requirements. The objective of the adjustment process is to remove trials where the provider of NHS services, acting as the trial host, has not contributed to the delay of initiation of clinical trials and which therefore should not be considered when assessing provider host performance. These trials will continue to be submitted and may be used to evaluate the broader performance of the clinical research system both locally and nationally. Audience The primary audience of this SOP is the Clinical Trial Performance (CTP) Team at the NIHR Central Commissioning Facility; to use during the analysis process following quarterly submissions. This document is also intended to help and support R&D Managers, R&D Data Managers and other R&D staff, both new to the exercise and those from established submitters, to complete their quarterly returns to the NIHR CCF in a way that achieves consistency within and across providers of NHS services. Background The Government wishes to see a dramatic and sustained improvement in the performance of providers of NHS services in initiating and delivering clinical research. The aim is to increase the number of patients who have the opportunity to participate in research and to enhance the nation s attractiveness as a host for research. Performance in Initiating Clinical Research exercise (70 day benchmark) aims to assess the interval from valid research application to recruitment of the first patient to a trial. The Department of Health places a contractual obligation on providers of NHS services contracted through the NIHR; to submit data and publish outcomes against the 70 day benchmark and delivery to time and to target for commercial clinical trials. More information and all related documents to support the submission process can be accessed at http://www.nihr.ac.uk/research-and-impact/nhs-research-performance/performance-in-initiatingand-delivering-research. 2
When assessing a provider s performance, DH has accepted that it is reasonable to exclude trials delayed beyond the provider s control. Consistency in the adjustment and analysis of data is important for providers to understand their quarterly performance against other providers, for the accuracy of publicly available published data and for the assessment of financial consequences. Providers are responsible for the submission of complete and accurate information. Term Adjustment Absolute data Description Process by which trials that are within 70 days of the Date Site Selected or trials that failed the benchmark but outside of the provider's control (and have been matched) are removed from the performance metrics analysis. Full set of data submitted by providers of NHS services. Absolute performance analysis Performance of analysis based on absolute data Adjusted data Adjusted performance analysis Date of First Patient Recruitment (FPR) Date Site Confirmed Date Site Selected Matched trial Mismatched trial Need More Info Reason for delay Sub-set of data to be analysed in assessing provider performance. Performance of analysis based on adjusted data Means the date the first eligible patient consented to the study. Date of the last contract signature of all the organisations involved (i.e. sponsor, site, 3rd party) or date of final written agreement of statement of activity (as applicable) Date on the email received by the site providing the minimum defined documents to enable site to commence arrangement and/or confirmation of local capacity and capability as applicable representing that the site has been selected to take part in the study. Where the is also the site, this is the date of the HRA initial assessment letter. 1 Trial where the reason for delay and the source of delay are consistent with each other and with the durations between DSS and FPR. Trial where the selected source of delay does not correspond to the reason for delay and/or the durations between DSS and FPR. Term given to a trial where no reason and/or source of delay has been selected, or where the information given is not sufficient to match the trial. Relating to the list of possible reasons for delay to clinical trials initiation, cited by the provider of NHS services for each clinical trial not meeting the 70 day benchmark. The list of possible reasons can be found in Appendix 1 of this document. 1 http://www.nihr.ac.uk/research-and-impact/nhs-research-performance/hra-approvals-and-nihr-metrics.htm 3
Term Source of delay Submission period Submission window Trials Meeting the Benchmark (TMB) Description Indicates which party was responsible for the delay: NHS Provider//Both/. The period (usually 30 days) between the end of the quarter and the submission deadline The timeframe for which data must be submitted by the provider Trials which have recruited the first patient in 70 days or fewer from the Date Site Selected Adjustment Process There are three stages to the adjustment process: 1. Separation of trials that are within 70 days (i.e. still eligible to meet the benchmark) 2. Separation of trials that cannot be eligible to meet the benchmark because the site was not confirmed. 3. Data quality assessment 4. Adjustment for the provider s influence over delay (i.e. Matching process) 1. Separation of trials that are within 70 days The submission includes trials that, as at the end of quarter, are still within the 70 day period and have not recruited a first patient at the end of the reporting period. Since they still might meet the benchmark, they are excluded from the performance analysis. 2. Separation of trials that cannot be eligible to meet the benchmark Trials which report declined site confirmation or Site declined to participate in the Non-Confirmation Status field will consequently have a Benchmark Met status of Site not confirmed. Since these trials will not go on to recruit any patients, they are excluded from the performance analysis. 3. Data quality assessment The quality of data in each record is assessed prior to the matching process. NIHR has now established a process of routine feedback to submitters of the data quality issues identified in their submission this feedback is intended for providers to review their data in future submissions and not for in-quarter changes. Some data quality issues do not affect the matching process and the record will be analysed regardless, but others have a direct impact on the adjustment (details are provided in the table below). For any additional information about platform validation of data quality, please refer to the CTP Submission Platform Instructions. 2 2 http://www.nihr.ac.uk/research-andimpact/documents/pid%20documents/guidance%20documents/ctp%20submission%20platform%20instructi ons%20published.pdf 4
Data Issue Type REC number not recognised Incorrect date format Data outside the submission window Data type mismatch No source of delay and/or no reason for delay First Patient Recruited? status is Yes - Date Unavailable Action Feedback to trust; trials will not be excluded from the analysis solely on the basis of an unrecognised REC number If the date is invalid or in a format which will cause formulae to fail, the dates and durations will be deleted and the trial will be labelled as not having met the benchmark. If the format of the date is incorrect but the information is clear, then the record is cleaned (i.e., 1-Jan-15 to 01/01/2015). For more information, please refer to the CTP Submission Platform Instructions. 2 Usually occurring with Date of First Patient Recruitment; record will be cleaned to how it looked on the last day of the quarter (date of FPR deleted, duration from DSS to FPR deleted, First Patient Recruited? changed to No, Benchmark Met changed to No or Within 70 Days as appropriate). When numerical data appears in text fields or vice versa; these types of errors are usually picked up by the submission validation and are therefore quite rare. Where the intended information is clear (i.e., the inversion of two columns), the record will be cleaned; if there is any doubt about the intended information, the record will be excluded from analysis. Marked as Need More Info, included in the adjusted dataset and assumed that the source of the delay is the ; due to the lack of information supplied. Trials will be treated as if a first patient has not been recruited, since the Date of First Patient Recruited will be blank. 4. Adjustment for the provider s influence over delay Trials will be assessed individually. All trials not meeting the benchmark (delayed beyond 70 days) must have a reason for delay and a source of delay, plus comments if necessary. If the duration between DSS and DSC is over 40 days and/or the duration between DSC and FPR is over 30 days, trials will be marked as Need more info if an explanation is not given for the relevant delay(s). A trial will be matched if the source of delay matches the reason(s) for delay and the component durations of the delay. Where a trial is marked as a Mismatch or Need more info the trial will be included in the adjusted data set and analyses, effectively assuming that the provider contributed as a source of the delay. This will continue until the trial is adequately explained in a subsequent submission. Trials Common to the Previous Quarter s Submission All trials carried forward from the previous quarter to the current quarter will be compared to the previous quarter's matching status and handled according to the following scenarios and actions: 5
Scenario Trial data not changed, trial previously missed benchmark Action Matching as previous quarter Trial data updated Trial data not changed, but the adjustment rules have changed and trusts have been notified at least 10 days before the end of the quarter Trial previously adjusted out due to being within 70 days Matching reassessed Matching reassessed Matching as a new record 6
The following table describes the range of scenarios involving omission of data or the selection of a single delay reason, what is expected in the way of supporting information and the action in respect of match/mismatch: Scenario Sub-Category Expected Source of Delay Comments Notes/Action No reason and/or source of delay A - Relevant permissions delayed and not granted in time B - Suspended by C - Closed by D - Delays HRA approval process not completed in time Study suspended by sponsor at all sites Study suspended by sponsor at this site Study closed by sponsor at all sites Study closed by sponsor at this site delay in provision of study documentation (e.g. pharmacy or laboratory manual) delay in provision of IMP, device or equipment delay through protocol or research application amendments Please briefly explain the reason for delay to permissions (i.e., the subcategory). Please briefly explain the nature of the delay and why the chosen source of delay is appropriate. sponsor delay (i.e., the subcategory) in sponsor delay (i.e., the subcategory) in Please briefly explain the nature of the sponsor delay (i.e., the subcategory) in Will need more information, trial automatically included in analysis regardless of comments Please note if DSC to FPR duration is over 30 days, the trial will be marked Need More Info if that delay is not explained. 7
Scenario Sub-Category Expected Source of Delay Comments Notes/Action delayed site initiation sponsor delay (i.e., the subcategory) in visit E - Staff Availability Issues F No Eligible Patients Seen delayed confirmation of study open to recruitment at site (i.e., green light) Planned and agreed later start or delay intrinsic in study design (e.g. planned later start, followon study, dependency on unpredictable event e.g. flu epidemic) staff availability (annual leave, sickness absence, staff issue / shortage) training (e.g. GCP, protocol specific training, etc.) staff availability (annual leave, sickness absence, staff issue / shortage) training (e.g. protocol specific training etc.) Patients screened but no eligible patients identified sponsor delay (i.e., the subcategory) in delay (i.e., the subcategory) in the comments. Please briefly explain provider training issue (i.e., the subcategory) in the comments. Please briefly explain sponsor staffing issue (i.e., the subcategory) in the comments. Please briefly explain sponsor training issue (i.e., the subcategory) in the comments. Please indicate the reason no patients were seen (i.e., the subcategory) in NHS staff issues such as annual leave, sick leave, maternity leave, etc. are expected to be within NHS provider's control. staff issues such as annual leave, sick leave, maternity leave, etc. are expected to be within sponsor's control. Please note if DSS to DSC duration is over 40 days, the trial will be marked Need More Info if that delay is not explained. 8
Scenario Sub-Category Expected Source of Delay Comments Notes/Action G No Eligible Patients Consented H - Contracting Delays I - Rare Diseases (Please see submission guidelines 1 ) Strict patient eligibility criteria / Low recruitment target NHS Host Site's inadequate planning / feasibility arrangements to see patients Eligible patients seen chose not to participate in study (e.g. Personal reasons, frequent study visits, reluctance to participate in placebo arm, preference of treatment choice, etc.) Eligible patients seen chose not to participate in study: NHS Host Site's inadequate planning /arrangements to consent patients Contracting / costing delays (including service support costing and excess treatment costing, etc.) Rare or very rare diseases studies (as defined in the guidance) Both Please indicate the reason no patients were seen (i.e., the subcategory) in delay and why the chosen source of delay is appropriate. Please note if DSS to DSC duration is over 40 days, the trial will be marked Need More Info if that delay is not explained. Please note if DSS to DSC duration is over 40 days, the trial will be marked Need More Info if that delay is not explained. Please note if DSS to DSC duration is over 40 days, the trial will be marked Need More Info if that delay is not explained. 9
Scenario Sub-Category Expected Source of Delay Comments Notes/Action Please only use other if no other Trial will be marked Need More Info reason explains the delay. Comments if no comments are entered, or if the must always be included. Please J - Other Other - specify in comment comments do not adequately explain Both briefly explain the nature of the delay the delay and the choice of source of and why the chosen source of delay is delay. appropriate. 10
Complex delays with multiple reasons for delay and/or sources of delay It is important to capture all delays which occurred between DSS and FPR, not just the delay that tipped the trial from 70 days to 71+ days, in order to gain a more complete understanding of why trials do not meet the benchmark. As indicated in the table above, when trials have not met the benchmark, please provide reasons for delay for durations of more than 40 days between DSS and DSC and for durations of more than 30 days between DSC and FPR, if applicable. The purpose of this requirement is to align with CRN s High Level Objectives 4 and 5 and is only needed when trials have not met the benchmark. As the overall aim of the exercise is to reduce the amount of time to initiate clinical trials, it is important to capture delays which could have been avoided or mitigated in order to highlight opportunities for process improvement. If there are multiple reasons for delay, please include all of these reasons in the record. Because reasons are not currently ranked or prioritised in the submission platform, it is not possible to objectively differentiate them in the analysis. However, providers are welcome (but not required) to indicate which reason or reasons contributed most significantly to the delay. Where there are also multiple sources of delay associated with these reasons, the CTP Team will accept in good faith the source of delay which the provider associates with the most significant reason for delay as long as the rationale is sufficiently explained in the comments. 11