Guidance producer subject to accreditation: National Patient Safety Agency (NPSA) for the Rapid Response Reports Date: 6 May 2010 Draft Accreditation Report for consultation Page 1 of 19
Contents Introduction... 3 Accreditation recommendation... 3 Implementation... 5 Reapplication for accreditation... 5 Appendix A: NHS Evidence accreditation analysis... 7 Appendix B: Bibliography... 14 Appendix C: Advisory Committee members, external advisers and NHS Evidence accreditation team... 16 Page 2 of 19
Introduction The NHS Evidence Accreditation Scheme recognises organisations that demonstrate high standards in producing health or social care guidance. Evidence users can therefore have high confidence in the quality of the information. Organisations can publicly display a seal of approval called an Accreditation Mark for three years after their processes have been accredited. The process for accrediting producers of guidance and recommendations for practice is described in the process manual 1. Accreditation recommendation It is proposed that the process to produce Rapid Response Reports (RRR) guidance by the National Patient Safety Agency (NPSA) is recommended for NHS Evidence accreditation. This draft decision is subject to public consultation before a final decision is made. Background to the guidance producer The National Patient Safety Agency is an Arm s Length Body of the Department of Health (DH). The National Reporting and Learning Service (NRLS) is a division of the NPSA that aims improve patient safety by identifying and reducing risks to patients receiving NHS care. The NRLS analyses patient safety incidents received via its national reporting system. Clinicians and safety experts help analyse information in the reporting system to identify common risks and opportunities to improve patient safety. Issues are then communicated by distributing Rapid Response Reports (RRRs). The objective of Rapid Response Reports is to alert health professionals to newly identified and important risks of medicines and devices, and provide advice on how these risks can be minimised. The safety alerts are provided to healthcare professionals to guide 1 http://www.nice.org.uk/nhsevidence/aboutaccreditation/aboutaccreditation.jsp?domedia=1&mid= 27C232A0-19B9-E0B5-D4A11FA899F4C219 Page 3 of 19
decisions about the safety of drugs and devices, which are aimed at improve individual and population health. Summary The Advisory Committee considered that the process for the production of Rapid Response Reports from the National Patient Safety Agency demonstrates compliance with 23 of the 24 applicable criteria for accreditation. The process for the development of Rapid Response Reports is described in the policy documents, provided in Appendix B which were last updated in July 2008. This documentation underpinning the guidance producer s processes is robust, comprehensive and fit for purpose. The National Patient Safety Agency involves a wide range of stakeholders, and minimises bias through a combination of guidance decisions determined by multidisciplinary groups, peer review and transparency in the way the work is funded. Recommendations are clear, specific and unambiguous, and the guidance is distributed widely. Suggestions to further strengthen National Patient Safety Agency processes for developing the Rapid Response Reports include: The dates of search used in the search strategy should be clearly stated within the Rapid Response Reports or the associated documentation. A regular updating policy for the process manual should be produced and adhered to. This draft decision is now going out for public consultation, and the decision will be reviewed by the committee in the light of any feedback received before making a final recommendation. David Haslam Chair, Advisory Committee May 2010 Page 4 of 19
Implementation If accreditation is awarded, guidance from the accredited producer will be identified on NHS Evidence by the Accreditation Mark. The accredited guidance producer is also granted a royalty-free, worldwide licence to use the NHS Evidence Accreditation Mark in accordance with the Conditions and Terms of Use 2. Providing these conditions are met, a guidance producer's accreditation will last for three years from publication of approval on the NHS Evidence website. Accredited guidance producers must take reasonable steps to ensure the accredited processes are followed when generating the type of evidence for which they are accredited. Accredited guidance producers should have quality assurance mechanisms in place and must inform NHS Evidence of any significant change to a process within 30 days. Figure: The NHS Evidence Accreditation Mark 2 http://www.nice.org.uk/nhsevidence/?domedia=1&mid=5ae1d938-19b9-e0b5- D471CA81220F57DA Page 5 of 19
Reapplication for accreditation If accreditation is not granted, guidance from the non-accredited producer will still be available on the NHS Evidence site but will not be identified by the accreditation mark graphic. Guidance producers that are not accredited following the accreditation process have the opportunity to reapply from one year after the previous assessment. It is assumed that the organisation will have addressed any concerns highlighted in the original assessment before reapplying. The NHS Evidence team will provide detailed feedback and advice on areas where improvement is required to meet the criteria in a future application. Page 6 of 19
Appendix A: NHS Evidence accreditation analysis The Advisory Committee considered the following analysis of the guidance producer s compliance with NHS Evidence accreditation criteria, which covers six discrete domains. The full analysis leading to the accreditation decision is shown below. Domain Criterion Evidence for meeting the criterion Accreditation decision Does the guidance producer have a policy in place and adhered to that requires them to explicitly detail: Scope and purpose 1.1 Overall objective The overall objective is clearly defined and implemented in the Rapid Response Reports 1 3 as described in the policy documents 4 12. The description of the safety issue in the Rapid Response Reports 1 3 indicates the overall objective. 1.2 The clinical, healthcare or social questions covered The clinical questions are the safety issues for each Rapid Response Report 1 3, which meets the overall aim given by the guidance producer in response to criterion 1.1, as described in the policy documents 4 12. Page 7 of 19
Domain Criterion Evidence for meeting the criterion Accreditation decision 1.3 Population and/or target audience to whom the guidance applies The Scale of patient safety issue section of the Template for Formal Scoping Report 7 indicates that the staff and patients to whom the guidance applies should be described. In addition section 2 of the Template for literature review 8 is entitled Intended audience. In three pieces of guidance reviewed as evidence the target audience and patient populations are indicated within both the Rapid Response Reports 1 3 and the supporting information. 1.4 Guidance includes clear recommendations in reference to specific clinical, healthcare or social circumstances The recommendations are clearly linked to specific safety circumstances as described in the policy documents 4 12. The Rapid Response Reports 1 3 are single page, one topic safety guidance and so the recommendations are clearly linked to the specific circumstances described in the guidance. Does the guidance producer have a policy in place and adhered to that means it includes: Stakeholder involvement 2.1 Individuals from all relevant stakeholder groups, including patient groups, in developing guidance 2.2 Patient and service user representatives and seeks patients views and preferences in developing guidance The production of the Rapid Response Reports 1 3 includes individuals from relevant stakeholder groups as described in the policy documents 4 12. The production of the Rapid Response Reports 1 3 includes patient s views. The NPSA core stakeholder list 9 includes some generic patient groups. It would also include content-specific patient groups for instance, renal patients for haemodialysis Rapid Response Reports. Page 8 of 19
Domain Criterion Evidence for meeting the criterion Accreditation decision 2.3 Representative intended users in developing guidance. Intended users are involved in the development of the Rapid Response Reports 1 3 described in the policy documents 4-12. Does the guidance producer have a clear policy in place that: 3.1 Requires the guidance producer to use systematic methods to search for evidence and provide details of the search strategy Systematic methods were used to search for evidence on which the Rapid Response Reports 1 3 are based and the search strategy detailed as described in the policy documents 4 12, specifically the Template for literature review 8 which outlines the processes to be followed for the search strategy. Rigour of development 3.2 Requires the guidance producers to state the criteria and reasons for inclusion or exclusion of evidence identified by the evidence review The inclusion and exclusion criteria are defined in the production of the Rapid Response Reports 1 3 as described in the policy documents 1-9. The Template for literature review 8 section 5.0 states Detail search strategy used, including databases, whether grey literature or internet were searched, and any exclusions, e.g. country, language, dates. 3.3 Describes the strengths and limitations of the body of evidence and acknowledges any areas of uncertainty The strength and limitations of the evidence are defined in the production of the Rapid Response Reports 1 3 as described in the policy documents 4 12. The Template for literature review 8 evidence tables show Limitations in the comments section of the template and the guidance producer further described this process in its application. Page 9 of 19
Domain Criterion Evidence for meeting the criterion Accreditation decision 3.4 Describes the method used to arrive at recommendations (for example, a voting system or formal consensus techniques like Delphi consensus) 3.5 Requires the guidance producers to consider the health benefits against the side effects and risks in formulating recommendations The method used to arrive at recommendations in the Rapid Response Reports 1 3 are explained as described in the policy documents 4 12. The information supplied by the guidance producer can be found in Sample minutes from weekly response meeting 11. The risks versus health benefits are considered in the production of the Rapid Response Reports 1 3 as described in the policy documents 4 12. The principle aim of these guidance products is to balance health benefits against risks by providing healthcare professionals with up to date safety information to better equip health professionals to make choices and therefore make patients safe. 3.6 Describes the processes of external peer review The peer review process used in the production of the Rapid Response Reports 1 3 is clearly defined as described in the policy documents 4 12. A typical consultation will involve over fifty individuals and organisations. 3.7 Describes the process of updating guidance and maintaining and improving guidance quality The process of updating and maintaining guidance quality is observed in the Rapid Response Reports 1 3 as described in the policy documents 1-9. Rapid Response Reports 1 3 have a shelf life of six months, appoint a content lead to respond to queries, post statements of clarification on common queries and noted that the guidance refers to regular reviews. Page 10 of 19
Domain Criterion Evidence for meeting the criterion Accreditation decision Does the guidance producer ensure that: 4.1 Recommendations are specific, unambiguous and clearly identifiable The Rapid Response Reports 1 3 are single page safety notices which address one safety issue. Therefore all recommendations in the Rapid Response Reports 1 3 are specific, unambiguous and clearly identifiable as Clarity and described in the policy documents 4 12. presentation 4.2 Different options for the management of the condition or options for intervention are clearly presented Rapid Response Reports 1 3 guidance is related to the safety of medicines and devices rather than the management of a condition. Recommendations are specific to the safety issues for a medicine or a device. There is no information supplied about treating specific conditions Not applicable and therefore this criterion is considered as not applicable. Page 11 of 19
Domain Criterion Evidence for meeting the criterion Accreditation decision 4.3 The date of search, the date of publication or last update and the proposed date for review are clearly stated 4.4 The content of the guidance is suitable for the specified target audience. If patients or service users are part of this audience, the language should be appropriate. The date of publication is clearly stated on the example Rapid Response Reports 1 3 reviewed as evidence. Due to the type of guidance product the review part of the criterion is considered as not applicable. Since the subject of the guidance is updates of drug and device safety driven by new evidence, the guidance is event rather than date led. In this case the proposed date for review does not exist, as stated by the guidance producer. No dates of search are shown in the example Rapid Response Reports 1 3 and the supporting information reviewed as evidence. The Template for literature review 8 section 5 shows that dates should be used for searching, but evidence of dates could not be found in guidance examples. The content of the Rapid Response Reports 1 3 is suitable for its target audience as described in the policy documents 4 12. The Rapid Response Reports are produced for clinical and service staff as specified by the guidance producer in its response to criterion 2.1 and therefore the content is suitable. Not fully met Page 12 of 19
Domain Criterion Evidence for meeting the criterion Accreditation decision Does the guidance producer routinely consider: 5.1 Publishing support tools to aid implementation of guidance Support tools are published to aid implementation of the Rapid Response Reports 1 3 by the National Patient Safety Agency. Applicability 5.2 Discussion of potential organisational and financial barriers in applying its recommendations The organisational and financial barriers to implementing the recommendations in the Rapid Response Reports 1 3 are assumed to be overridden by the imperative of the safety issues identified. In addition the guidance producer stated that implementation considerations are raised with the Department of Health (DH). 5.3 That their guidance is current, with review criteria for monitoring and/or audit purposes within each product. The Rapid Response Reports 1 3 are current and audit of the uptake and use of this guideline is undertaken. The guidance producer provided examples of how it audits the implementation of guidance in association with other professional bodies. Does the guidance producer: Editorial independence 6.1 Ensure editorial independence from the funding body The Rapid Response Reports 1 3 are editorially independent from the funding body. The NPSA is part of the NHS. All funding for the guideline programmes is from the DH and Welsh Assembly. In addition the guidance is developed via a wide number of individual and professional groups. Page 13 of 19
Domain Criterion Evidence for meeting the criterion Accreditation decision 6.2 Demonstrate transparency about the funding mechanisms for its guidance 6.3 Record and state any potential conflicts of interest of individuals involved in developing the recommendations The guidance producer stated that Rapid Response Reports 1 3 are produced as part of NPSA core business. The guidance producer s website shows the annual accounts and breakdown of the funds. In the original submission no conflicts of interest policy could be found. As part of the guidance producer feedback the conflicts of interest policy was produced and implemented and evidence of this policy in practice was supplied. 6.4 Take account of any potential for bias in the conclusions or recommendations of the guidance The Rapid Response Reports 1 3 production minimises bias via a combination of guidance decisions determined by multidisciplinary groups, peer review and transparency in the way the work is funded, ensuring this criterion is satisfied as described in the policy documents 4 12. 1 Oxygen safety in hospitals, RRR ref 1124; 2 Mitigating surgical risk in patients undergoing hip arthroplasty for fractures of the proximal femur RRR ref 0958; 3 Reducing risk of overdose with midazolam injection in adults pathway RRR ref 1074; 4 Response unit overview ; 5 Incident Screening Summary; 6 Terms of Reference - Weekly Response Group; 7 Template for Formal Scoping Report; 8 Template for literature review; 9 List of stakeholders; 10 Template - RRR Invitation to comment letter; 11 Sample minutes from weekly response meeting; 12 Checklist for RRR development and approval Page 14 of 19
Appendix B: Bibliography Appendix B lists the additional information taken into account in the analysis and considered by the Committee. Document name Description Location A list of issued RRRs Submission document Annex A Issued RRRs Response unit overview Policy document Annex B Overview Incident Screening Summary Policy document Annex C Incident Screening Summary Terms of Reference - Weekly Response Group Policy document Annex D Terms of Reference - Weekly Response Group Template for Formal Scoping Report Policy document Annex E Template for Formal Scoping Report Template for literature review Policy document Annex F Template for literature review List of stakeholders Submission document Annex G List of stakeholders Template - RRR Invitation to comment letter Policy document Annex H Template - RRR Invitation to comment letter Sample minutes from weekly response meeting Submission document Annex I Sample minutes from weekly response meeting Checklist for RRR development and approval Policy document Annex J Checklist for RRR development and approval Oxygen safety in hospitals (Ref 1124) RRR guidance used as evidence http://www.nrls.npsa.nhs.uk/resour ces/type/alerts/?entryid45=62811 Mitigating surgical risk in patients undergoing hip RRR guidance used as evidence http://www.nrls.npsa.nhs.uk/resour ces/type/alerts/?entryid45=59867 arthroplasty for fractures of the proximal femur (Ref 0958) Reducing risk of overdose with midazolam injection in adults pathway (Ref 1074) RRR guidance used as evidence http://www.nrls.npsa.nhs.uk/resour ces/type/alerts/?entryid45=59896 Page 15 of 19
Appendix C: Advisory Committee members, external advisers and NHS Evidence accreditation team NHS Evidence Advisory Committee Members The NHS Evidence Advisory Committee operates as a standing advisory committee of the Board of the National Institute for Health and Clinical Excellence (NICE). The Committee provides advice to the Institute on a framework for accrediting sources of evidence that should be recognised as trusted sources of information for the NHS. The Chair of the Committee is appointed by the Institute s Board and the meetings are conducted by the Chair or in his/her absence the vice chair. The current Chair is David Haslam. A full list of the Advisory Committee membership is available on the NICE website 3. The members have been appointed for a period of 18 months. This may be extended by mutual agreement to a further term of 3 years and up to a maximum term of office of 10 years. The decisions of the Committee are arrived at by a consensus of those members present. The quorum is set at 50% of committee membership. The Committee submits its recommendations to the Institute s Guidance executive which acts under delegated powers of the Institute s Board in considering and approving its recommendations. Committee members are asked to declare any interests in the guidance producer to be accredited. If it is considered that there is a conflict of interest, the member(s) is excluded from participating further in the discussions. A list of the committee members who took part in the discussions for this accreditation decision appears below. 3 http://www.nice.org.uk/nhsevidence/nhseac.jsp Page 16 of 19
Title Name Surname Role Organisation Ms Judy Birch Lay member Ms Parul Desai Director of Population Health NHS Information Centre Professor David Haslam National Clinical Advisor Care Quality Commission Dr Bobbie Jacobson Dr Monica Lakhanpaul Director of London Health Observatory, Vice Chair of Association of PH Observatories Senior Lecturer / Consultant Paediatrician / Clinical Director London Health Observatory Health Education Research and Development Unit (HERADU), Department of Medical Education and Social Care, University of Leicester Mr John Lee President The Royal College of Ophthalmologists Ms Ruth Liley Professor Jon Nicholl Professor Sandy Oliver Assistant Director of Quality Improvement Professor of Health Services Research Professor of Public Policy, Deputy Director, Social Science Research Unit Marie Curie Cancer Care School of Health and Related Research (ScHARR) University of Sheffield Cochrane Consumers and Communication Review Group, University of London Dr Carl Parker PEC Chair Stockton-on-Tees Teaching PCT Mr Adrian Reyes- Hughes Associate Clinical Director NHS Direct Dr Karen Ritchie Ms Sasha Shepperd Lead Health Services Researcher Senior Research Scientist, Department of Public Health NHS Quality Improvement Scotland University of Oxford Professor Stephen Singleton Medical Director / Regional North East Strategic Health Page 17 of 19
Director of Public Health Dr Peter Smith Vice President Authority National Association of Primary Care Dr Mark Strong Ms Gill Swash MRC Fellow, Section of Public Health Head of knowledge and Library Services School of Health and Related Research (ScHARR) University of Sheffield NHS Western Cheshire Dr Sara Twaddle Director Scottish Intercollegiate Guidelines Network Advisory Committee Deputies Title Name Surname Role Organisation National Director for NHS Alliance Mrs Julie Wood Practice Based commissioning function Commissioning Deputising for Brian Fisher Ms Diane Gwynne Smith Head of Knowledge Management Social Care Institute for Excellence (SCIE) Amanda Edwards External Advisers for the National Patient Safety Agency (NPSA) Rapid Response Reports guidance accreditation application Vicki Wilkinson, Network Manager, Primary Care Research Network North West, Halton & St Helens PCT, UK Tracy NDiaye, Directorate Manager of Research, Clinical Research Facility Sheffield Children's NHS Foundation Trust, UK Page 18 of 19
NHS Evidence accreditation team for the National Patient Safety Agency (NPSA) Rapid Response Reports guidance accreditation application Stephanie Birtles, Accreditation Technical Analyst, NHS Evidence, National Institute for Health and Clinical Excellence, Manchester, UK Dr Paul Chrisp, Associate Director Accreditation, NHS Evidence, National Institute for Health and Clinical Excellence, Manchester, UK Page 19 of 19