Guidance producer: Medicines and Healthcare products Regulatory Agency (MHRA) Guidance product: Device Bulletins Date: 20 September 2010 Final Accreditation Report Page 1 of 21
Contents Introduction... 3 Accreditation recommendation... 3 Implementation... 6 Appendix A: NHS Evidence accreditation analysis... 7 Appendix B: Bibliography... 15 Appendix C: Advisory Committee members, external advisers and NHS Evidence accreditation team... 18 Page 2 of 21
Introduction The NHS Evidence Accreditation Scheme recognises organisations that demonstrate high standards in producing health or social care guidance. Users of the accredited guidance can therefore have high confidence in the quality of the information. Organisations can publicly display a seal of approval called an Accreditation Mark for three years after their processes have been accredited. The process for accrediting producers of guidance and recommendations for practice is described in the process manual 1. Accreditation recommendation NHS Evidence has accredited the process used by the Medicines and Healthcare products Regulatory Agency (MHRA) Device Technology and Safety (DTS) Division to produce Device Bulletins (DB). Accreditation is valid for three years from September 2010 and is retrospectively applicable to guidance produced from September 2007 using the processes described in the Advisory Notice Drafting and publishing a device bulletin (published in 2002). Background to the guidance producer The MHRA was set up in April 2003 following a merger of the Medicines Control Agency and the Medical Devices Agency. The MHRA is an executive agency of the Department of Health (DH). 1 http://www.evidence.nhs.uk/accreditation/documents/nhsevidenceaccredmanual.pdf Page 3 of 21
The MHRA is the government agency responsible for ensuring that medicines and medical devices are effective and acceptably safe. The MHRA monitors the use of medicines and devices, and takes any necessary action to protect the public if there is a problem. Within the MHRA, the DTS division investigates adverse incidents involving medical devices reported by the NHS and private hospitals, care homes, manufacturers and directly from the public. Medical devices range from the most sophisticated diagnostic equipment used in hospitals, to simple bandages and other products sold over the counter to the public. Summary The Advisory Committee considered that the process for the production of Device Bulletins from the MHRA demonstrates compliance with all 22 of the applicable criteria for accreditation. The process for the development of Device Bulletins is described in the Advisory Notice Drafting and publishing a device bulletin, 2002. This documentation underpinning the guidance producer s processes is robust, comprehensive and fit for purpose. Device Bulletins provide safety information to medical device users, both professional and the public where applicable, to ensure they have the most up to date information. Device Bulletins contain guidance and information on aspects of general safe use and management of medical devices. They are written as a result of experience gained from adverse incident investigations, contacts with manufacturers and users, and other sources of information. The process used to produce Device Bulletins has a documented process for the evidence search that is fit for purpose and all evidence found is included for assessment. Suggestions to further strengthen MHRA processes for developing the Device Bulletins include: Page 4 of 21
clarifying the role of lay members in the development process for Device Bulletins, including how the Public and patient engagement strategy is used clearly documenting, within the guidance, how the strength of the evidence base was assessed David Haslam Chair, Advisory Committee September 2010 Page 5 of 21
Implementation Following accreditation, guidance from the accredited producer will be identified on NHS Evidence by the Accreditation Mark. The accredited guidance producer is also granted a royalty-free, worldwide licence to use the NHS Evidence Accreditation Mark in accordance with the Conditions and Terms of Use 2. Providing these conditions are met, a guidance producer's accreditation will last for three years from publication of approval on the NHS Evidence website. Accredited guidance producers must take reasonable steps to ensure the accredited processes are followed when generating the type of evidence for which they are accredited. Accredited guidance producers should have quality assurance mechanisms in place and must inform NHS Evidence of any significant change to a process within 30 days. Figure: The NHS Evidence Accreditation Mark 2 http://www.evidence.nhs.uk/accreditation/documents/nhsevidenceconditions.doc Page 6 of 21
Appendix A: NHS Evidence accreditation analysis The Advisory Committee considered the following analysis of the guidance producer s compliance with NHS Evidence accreditation criteria, which covers six discrete domains. The full analysis leading to the accreditation decision is shown below. Domain Criterion Evidence for meeting the criterion Accreditation decision Does the guidance producer have a policy in place and adhered to that requires them to explicitly detail: Scope and purpose 1.1 Overall objective The overall objective is Safeguarding public health and is documented on 1.2 The clinical, healthcare or social questions covered the front cover of all guidance documents reviewed. In the case of Device Bulletins b-f the questions are safety based. All examples of guidance reviewed showed the safety questions as numbered sections in the body of the guidance documents. The questions are clearly indicated and, where applicable, explained. Page 7 of 21
Domain Criterion Evidence for meeting the criterion Accreditation decision 1.3 Population and/or target audience to whom the guidance applies The Advisory Notice a section 2.2 states that inclusion of the target audience when drafting a proposal provides some evidence of need of the Device Bulletin. In the example guidance documents b-f, patient population can be inferred from the titles. 1.4 Guidance includes clear recommendations in reference to specific clinical, healthcare or social circumstances The Advisory Notice a section 1 documents how the process of defining the information and considerations of specific sections take place. Evidence of this process in practice is clear in all of the examples of guidance b-f reviewed. Does the guidance producer have a policy in place and adhered to that means it includes: Stakeholder involvement 2.1 Individuals from all relevant stakeholder groups, including patient groups, in developing guidance 2.2 Patient and service user representatives and seeks patients views and preferences in developing guidance The composition of both the drafting group and the MHRA s Committee for the Safety of Devices (CSD) show that all relevant stakeholder groups are involved in developing guidance. The CSD includes patient representatives. Public and patient engagement strategy is available on the website which further seeks the views of patients and patient representatives. Page 8 of 21
Domain Criterion Evidence for meeting the criterion Accreditation decision 2.3 Representative intended users in developing guidance. Section 2.4 of the Advisory Notice a specifies that the review group should always include a typical reader. In addition the Advisory Notice also states that any relevant professional bodies should be included in the review process. Does the guidance producer have a clear policy in place that: Rigour of development 3.1 Requires the guidance producer to use systematic methods to search for evidence and provide details of the search strategy Typical evidence searched includes trends in medical device incidents reported to MHRA (via adverse incidents tracking database) and data from registries or clinical studies. Systematic methods are used to search for evidence on which the Device Bulletin is based. 3.2 Requires the guidance producers to state the criteria and reasons for inclusion or exclusion of evidence identified by the evidence review No evidence is excluded from searches to produce a Device Bulletin. All information found by the search strategy is included. Page 9 of 21
Domain Criterion Evidence for meeting the criterion Accreditation decision 3.3 Describes the strengths and limitations of the body of evidence and acknowledges any areas of uncertainty The adverse incidents reports database is searched to provide the evidence base for the Device Bulletins. All of the information found during a specific search is used and no weighting is applied, which is fit for purpose for this type of guidance product. 3.4 Describes the method used to arrive at recommendations (for example, a voting system or formal consensus techniques like Delphi consensus) 3.5 Requires the guidance producers to consider the health benefits against the side effects and risks in formulating recommendations The process for the production of Device Bulletins is an authorship process, and therefore no voting occurs. However the guidance development process contains a clear description of the method used to develop recommendations. This is the overall objective of the MHRA and the balance between risks and benefits is shown in the Aims and objectives section of their website. The principal aim of the guidance product is to balance health benefits against risks by providing healthcare professionals with up-to-date medical device safety information so that they can make informed choices. Page 10 of 21
Domain Criterion Evidence for meeting the criterion Accreditation decision 3.6 Describes the processes of external peer review Section 2.4 and 4.3 of the Advisory Notice a outlines the process for external peer review of Device Bulletins. 3.7 Describes the process of updating guidance and maintaining and improving guidance quality Device Bulletins provide safety advice driven by new findings regarding possible safety issues for medical devices. Rather than being date driven the guidance is data driven so no regular updating schedule is available. Furthermore, the guidance itself is an update that may contain important ancillary information which is provided in addition to that produced when the medical device is licensed for production. Does the guidance producer ensure that: Clarity and presentation 4.1 Recommendations are specific, unambiguous and clearly identifiable Device Bulletins provide advice on the safe use of medical devices and additional up-to-date safety information to medical device users. The safety recommendations are specific, unambiguous and clearly shown within the Device Bulletins. Page 11 of 21
Domain Criterion Evidence for meeting the criterion Accreditation decision 4.2 Different options for the management of the condition or options for intervention are clearly presented 4.3 The date of search, the date of publication or last update and the proposed date for review are clearly stated This criterion is not applicable because Device Bulletin advise on the safe use of specific devices, not the management of conditions.. All example guidance b-f reviewed bears the date of publication and date of update where applicable. The date of literature search cannot be found. The guidance is eventrather than date-led since the updates are driven by new evidence so review dates are not applicable (see also criterion 3.1). Not applicable Applicability 4.4 The content of the guidance is suitable for the specified target audience. If patients or service users are part of this audience, the language should be appropriate. The guidance examples b-f reviewed are suitable for all audiences. Any technical terms are explained and ideas are described so the reader can understand the options (where applicable), recommendations and the evidence behind them. Does the guidance producer routinely consider: Page 12 of 21
Domain Criterion Evidence for meeting the criterion Accreditation decision 5.1 Publishing support tools to aid implementation of guidance An email alert process ensures that this guidance reaches its target audience. Since the guidance is a dissemination of safety information from the MHRA s DTS division it describes how to use a medical device with a revised safety review and is considered obligatory. 5.2 Discussion of potential organisational and financial barriers in applying its recommendations 5.3 That their guidance is current, with review criteria for monitoring and/or audit purposes within each product. This criterion is not applicable because external organisations must overcome financial and organisational barriers to implement the guidance for safe use of medical devices. This criterion is not applicable for this type of guidance product. A date of publication is stated but no evidence of a process for review criteria, audit or information about monitoring is provided (see criterion 5.2). It would not be productive to measure compliance with only those aspects of safety contained within the guidance b-f. Not applicable Not applicable Editorial Does the guidance producer: Page 13 of 21
Domain Criterion Evidence for meeting the criterion Accreditation decision independence 6.1 Ensure editorial independence from the funding body Section 5 MHRA framework document details the process of how the funding is collated and disseminated. In addition to the dispersed funding mechanism, the guidance is developed via a number of multidisciplinary groups, such as the CSD and it is peer reviewed. 6.2 Demonstrate transparency about the funding mechanisms for its guidance The MHRA is an executive agency of the DH with trading fund status. The agency is funded by fees charged for fulfilment of statutory obligations, charges for non-statutory services and the DH. Section 5 (Finance) of the MHRA framework document describes how the MHRA generates and disseminates its finances and explains its accounting and auditing arrangements. 6.3 Record and state any potential conflicts of interest of individuals involved in developing the recommendations Section 7.14 of the MHRA framework document states that all MHRA employees are required to state any potential conflicts of interest. Any declarations of interests are published on the website. Page 14 of 21
Domain Criterion Evidence for meeting the criterion Accreditation decision 6.4 Take account of any potential for bias in the conclusions or recommendations of the guidance A combination of conflict of interest declarations, guidance decisions determined by multidisciplinary groups, peer review and transparency in the way the work is funded ensures this criterion is satisfied. The webpage What principles inform the decisions? point 10 also documents the agency s decisions will as far as possible be transparent and open to public scrutiny; a reasonable person reviewing our decisions should understand the rationale. a Advisory Notice Drafting and publishing a device bulletin, 2002 (process manual). b DB2009 (02) Adverse incident reports 2008 c DB2008 (04) In vitro diagnostic medical devices used in combination d DB2008 (03) Guidance on the safe use of lasers, intense light source systems and LEDs in medical, surgical and aesthetic practises e DB2009 (01) Reporting adverse incidents and disseminating medical device alerts f DB2007 (03) Safety guidelines for magnetic resonance imaging equipment in clinical use. Page 15 of 21
Appendix B: Bibliography Appendix B lists the additional information taken into account in the analysis and considered by the Committee. Document name Description Location DB2009 (02) Adverse Example guidance incident reports 2008 supplied by guidance producer DB2008 (04) In vitro Example guidance diagnostic medical devices supplied by guidance used in combination producer DB2008 (03) Guidance on Example guidance the safe use of lasers, supplied by guidance intense light source producer systems and LED s in medical, surgical and aesthetic practises. http://www.mhra.gov.uk/publications/safety guidance/devicebulletins/con041407 http://www.mhra.gov.uk/publications/safety guidance/devicebulletins/con025728 http://www.mhra.gov.uk/publications/safety guidance/devicebulletins/con014775 DB2009 (01) Reporting Example guidance chosen Adverse incidents and arbitrarily to validate the disseminating Medical application of processes Device Alerts DB2007 (03) Safety Example guidance chosen Guidelines for Magnetic arbitrarily to validate the Resonance Imaging application of processes Equipment in Clinical Use ADVISORY NOTICES: Drafting Device Bulletins Drafting and publishing a procedure Device Bulletin http://www.mhra.gov.uk/publications/safety guidance/devicebulletins/con036318 http://www.mhra.gov.uk/publications/safety guidance/devicebulletins/con2033018 Drafting Device Bulletins procedure MHRA Framework Document outlining the http://www.mhra.gov.uk/searchhelp/search/ Searchresults/CON008369 Page 16 of 21
Document name Description Location document MHRA framework Page 17 of 21
Appendix C: Advisory Committee members, external advisers and NHS Evidence accreditation team NHS Evidence Advisory Committee Members The NHS Evidence Advisory Committee operates as a standing advisory committee of the Board of the National Institute for Health and Clinical Excellence (NICE). The Committee provides advice to the Institute on a framework for accrediting sources of evidence that should be recognised as trusted sources of information for the NHS. The Chair of the Committee is appointed by the Institute s Board and the meetings are conducted by the Chair or in his/her absence the vice chair. The current Chair is David Haslam. A full list of the Advisory Committee membership is available on the NICE website 3. The members have been appointed for a period of 3 years. This may be extended by mutual agreement to a further term of 3 years and up to a maximum term of office of 10 years. The decisions of the Committee are arrived at by a consensus of those members present. The quorum is set at 50% of committee membership. The Committee submits its recommendations to the Institute s Guidance executive which acts under delegated powers of the Institute s Board in considering and approving its recommendations. Committee members are asked to declare any interests in the guidance producer to be accredited. If it is considered that there is a conflict of interest, the member(s) is excluded from participating further in the discussions. A list of the committee members who took part in the discussions for this accreditation decision appears below. 3 http://www.evidence.nhs.uk/accreditation/documents/nhseac%20members%20names%20- %20June%2010.pdf Page 18 of 21
Title Name Surname Role Organisation Ms Judy Birch Lay member Mr Richard Brownhill Clinical Development & Nurse practitioner Calderdale and Huddersfield NHS Trust and Kirklees PCT Ms Joyce Epstein Lay member Professor David Haslam National Clinical Advisor Care Quality Commission Dr Bobbie Jacobson Dr Monica Lakhanpaul Director of London Health Observatory, Vice Chair of Association of PH Observatories Senior Lecturer / Consultant Paediatrician / Clinical Director London Health Observatory Health Education Research and Development Unit (HERADU), Department of Medical Education and Social Care, University of Leicester Professor Jon Nichol Dean School of Health and Related Research (ScHARR) University of Sheffield Professor Sandy Oliver Professor of Public Policy, Deputy Director, Social Science Research Unit Cochrane Consumers and Communication Review Group, University of London Mr Adrian Reyes- Hughes Associate Clinical Director NHS Direct Ms Sasha Shepperd Ms Ann Slee Senior Research Scientist, Department of Public Health Director of Pharmacy and Medicines Management University of Oxford University Hospitals Birmingham NHS Foundation Trust Page 19 of 21
Dr Peter Smith Vice President National Association of Primary Care Dr Mark Strong Ms Gill Swash MRC Fellow, Section of Public Health Head of knowledge and Library Services School of Health and Related Research (ScHARR) University of Sheffield NHS Western Cheshire Advisory Committee Deputies Title Name Surname Role Organisation Deputising for Ms Lynda Cox Head of Knowledge Sharing and Learning North East Strategic Health Authority Stephen Singleton Professor Bill McGuire Professor of Child Health Ms Julie Robinson Information Specialist Hull York Medical School Social Care Institute for Excellence (SCIE) Stuart Logan Amanda Edwards External Advisers for MHRA Device Technology and Safety Division Device Bulletins accreditation application Aideen Mary Tarpey, Epilepsy Specialist Nurse - Dorset, Poole Hospital NHS Foundation Trust, Poole, Dorset, UK Dr Timothy E Bates, Chief Scientific Officer, New-Use Therapeutics Limited, Nottingham, UK Page 20 of 21
NHS Evidence accreditation team for MHRA Device Technology and Safety Division Device Bulletins accreditation application Stephanie Birtles, Accreditation Technical Analyst, NHS Evidence, National Institute for Health and Clinical Excellence, Manchester, UK Dr Paul Chrisp, Associate Director Accreditation, NHS Evidence, National Institute for Health and Clinical Excellence, Manchester, UK Page 21 of 21