Quality Assurance of Specialty Education and Training 2016 Pilot Activity Report

Transcription

1 December 2016 Executive Summary Quality Assurance of Specialty Education and Training 2016 Pilot Activity Report 1. Further to the publication of the Standards for Specialty Education 1 and Council agreement in April 2015, we have engaged with stakeholders to ascertain an appropriate level of quality assurance activity for specialty training. Engagement has involved pilots with five Local Education Training Board (LETBs) (prior to the recent LETB reorganisation) and two Royal Colleges, culminating in a workshop on 29 April Through this we sought to identify the appropriate and proportionate level of quality assurance activity that should be undertaken in specialty education and training. 2. This process has also informed the potential further refinement and revision of the Standards for Specialty Education and Training, to ensure they are fit for purpose. This paper serves as an update on developments and sets out the proposed next steps in developing the quality assurance process for specialty training and education in an effective and proportionate manner. Introduction and background The GDC s policy position 3. The GDC s Strategic Review of Education (2008) recommended that the Council should actively quality assure all training and awards which lead to entry to all GDC registers and listings (Dentist, Dental Care Professional (DCP) and Specialist). This was overseen by the Policy Advisory Committee (PAC) of the previous Council. It was notified of the GDC s powers and received reports from the Specialist Dental Education Board s (SDEB) initial development of models for the quality assurance of specialty education. 4. In parallel, the PAC also oversaw the Review of Regulation of the Specialties, which reported in September 2014, concluding that the specialist lists do provide some benefits for patients and we should continue to regulate the dental specialties. However, the risks to patients are relatively low and the Council decided not to make major policy changes at this stage. 5. Consideration and development of our quality assurance processes therefore apply to training programmes in all 13 current specialties. Whilst our statutory responsibilities (see section 10 below) focus on orthodontics and oral surgery we do not currently possess an evidence base, drawing upon public protection arguments to differentiate between the specialties in quality assurance activity. The GDC s powers 6. The GDC s powers in relation to specialist education and training differ from its powers for preregistration training: 7. The Dentist Act 1984 (the Act) restricts our ability to require training providers to provide information to those with Dental Authority (DA) status. Of postgraduate providers, the Royal Colleges possess DA status as do universities undertaking postgraduate or specialist dental 1 Page 1

2 training. We can request information from other postgraduate training providers such as deaneries/letbs who do not hold such status in connection with section 1(2)(a) of the Act. 8. We have powers under Section 9 of the Act to appoint visitors to inspect programmes and examinations of both undergraduate and postgraduate/specialist programmes. However, the concept of sufficiency applies only to DA and there is no formal mechanism to approve or withdraw approval from postgraduate/specialist training providers who do not possess such status. 9. The Specialist List Regulations provide us with powers to determine who is eligible to join the lists. 10. The GDC is, in relation to specialist dental qualifications in orthodontics and oral surgery, the competent authority in the United Kingdom for the purposes of the Recognition Directive and the Dental Training Directive. The Council has a statutory duty to supervise training in these two specialties. 11. We have taken legal advice and have established that our statutory duty to supervise training in orthodontics and oral surgery can support quality assurance activity across the 13 specialties. Specialty training 12. The primary route 2 by which specialists join the Specialist lists, and the route upon which the GDC has been focussing its quality assurance efforts, is successful completion of a national training programme in the individual UK specialties, where training is based upon a GDCapproved curriculum 3, overseen by one of the 15 regional postgraduate deaneries/letbs, and where the trainee also passes the relevant Royal College examination. 13. Following these successes, the trainee is recommended for entry to the GDC Specialist Lists by award of a Certificate of Completion of Specialist Training (CCST). The postgraduate deanery/letb recommend the award and the GDC awards the CCST. 14. Training in the dental specialties under the route described above is, typically, a three-year fulltime hospital-based programme. This can involve trainees receiving training in a variety of hospital settings and other clinical environments. This form of delivery, together with the provision of exit examinations by a further provider has required some changes to the GDC s current model of pre-registration QA inspection which is typically based on a single training centre under the auspices of a university or other educational body providing training. 15. In April 2015 the Council approved the Standards for Specialty Education: on.pdf 2 The other routes to entry to a specialist list where the GDC is not quality assuring training are: 1) European Economic Area (EEA) qualified specialists - relevant to the Oral Surgery and Orthodontics Specialist Lists only. Dentists who hold a qualification in Oral Surgery or Orthodontics from an EEA member state can apply for automatic recognition or via acquired rights. Specialist qualifications for EEA member states are listed in Appendix of Directive 2005/36/EC. 2) Individual assessment - dentists who do not fall into either of the above routes (CCST or EEA qualified) may be eligible to submit an application for individual assessment. This includes a very small number of individuals who join the list via the mediated entry experience gained through academic and research work. 3 The GDC approves the curriculum for each dental specialty. We consider that this step by itself, although very important in setting out detailed requirements for specialty education and training in the UK, does not constitute fully meeting our statutory responsibilities for supervision. We are currently in the process of revising the process for the submission and approval of specialty curricula, which will be discussed at Council in December Page 2

3 16. At that time, the Council also agreed stages 1 to 3 of a pilot implementation plan on the understanding that they could be delivered within the then current Quality Assurance team resources and within the 2015 business plan and budget. Developments since April 2015 Stage 1 - Publish Standards for Specialty Education 17. The consultation and subsequent stakeholder engagement on the Standards signalled the GDC s expectations in relation to specialty education and training. Publishing the Standards for Specialty Education was seen to send a clear message to the sector about the quality the GDC expects in order to protect patients and the public. 18. In addition to publishing the GDC standards, we recognised that the UK Committee of Postgraduate Dental Deans and Directors (COPDEND) already publishes a quality management tool in the form of The Gold Guide (most recently revised in 2016). We also recognised that specialty trainees are in the main already GDC registrants; and that we needed to be sensitive to the fact that specialty training (where it takes place in NHS Trusts and roles) operates in an already highly regulated environment. 19. We have been mindful that that our regulatory approach, both in its piloting and in any final form to be determined, must not introduce disproportionate or unnecessary burdens on providers. 20. We undertook to carry out a post-introduction review in Spring 2016 to establish the Standards take-up and to make recommendations about what, if any further activity is necessary. Stage 2 - Engage with providers and trainees to raise awareness of regulatory activity 21. Following publication of the standards, we have maintained an active programme of engagement with the specialist training sector in 2015 and We have used this programme to promote the standards and our role in quality assuring specialty education to providers and speciality trainees. 22. We have prepared an update to our existing Making complaints and raising concerns about an education or training provider policy to cover specialty education so that trainees have a formal mechanism to raise concerns relating to the quality of provision particularly where there are possible risks to patient safety. Stage 3 - Collect and analyse evidence from post graduate deaneries/letbs and examination providers. 23. The pilot process has been be based upon moderated self-assessment and has included: a. a data set that profiles specialty trainees and scrutinises key data including information about the trainees progression rate through programmes and exit examinations. b. a self-assessment questionnaire against the standards giving providers the opportunity to indicate how they meet the Standards. c. the opportunity to provide illustrative and supporting evidence to underpin selfassessment 24. The pilot, in addition to testing processes, has also provided a useful opportunity to form a view about the embedding of the Standards in Specialty Education and Training following publication. Stage In their deliberations in April 2015 the Council determined not to introduce any further quality assurance activity, whether that be targeted and risk-based interventions where they are required (and in response to risks identified through the stages 1-3 of the quality assurance process) or inspections of all providers based upon a regular inspection cycle. Page 3

4 26. The Council recognised that both options carried substantial resource implications. Prior to any further consideration, the pilot processes and the post-introduction review can provide evidence for the Council to take an informed view on the nature of any way forward (we may discover activity under stages 1-3 to be sufficient). The Pilot Process information collection Programme Providers 27. Following discussions with COPDEND, we were very grateful to receive agreement from three programme providers covering five LETBs to participate in a pilot process: a. London Dental Education and Training, Health Education England incorporating NE, NW and S London b. Health Education England - East Midlands (HEE EM) c. Health Education England - South West (HEE SW) 28. We circulated draft templates and guidance to the pilot sites in September 2015 and discussed them with the volunteers before their use. Based upon these discussions we refined the assessment tools and the proposed pilot process continued throughout the last quarter of 2015 and the three volunteers were asked to provide data against the final self-assessment tools in the first quarter of Examination Providers 29. We were very grateful to receive agreement from two examination providers to participate in a pilot process, the Faculties of Dental Surgery from: a. the Royal College of Surgeons of England b. the Royal College of Surgeons of Edinburgh. 30. Following discussions, a redacted version of the Standards, focussing only on those requirements relevant to examination providers was circulated and the two volunteers were asked to provide data against the final self-assessment tools in the first quarter of 2016 Joint activity 31. For the purposes of the pilot we considered a workshop-based approach to review both the standards and volunteer submissions would be appropriate and proportionate with a focus upon mutual learning for volunteer providers and GDC staff. This workshop was held on 29 April Volunteer submissions and findings from the pilot 32. All volunteers submitted information against the relevant standards and requirements set out in the Standards for Specialty Education by 4 th March We supported the pilot LETBs in this process through closer engagement, including meetings where we went through the aims and objectives of the quality assurance process, as well as giving clarity on the level of detail required regarding evidence submitted. 33. There were different approaches to the submission of evidence to accompany self-assessment information under the standards: a. Two LETB volunteers provided such evidence (one going so far as specifically to create evidence in order to meet requirements they might not otherwise have done) whilst one chose not to provide evidence. b. Of the examination providers, one chose to provide two submissions relevant to individual examinations they provide under separate governance arrangements Page 4

5 whilst the other chose to provide a holistic submission covering their quality management activities. 34. There was a further difference in approach where: Standard 1 a. Two LETB volunteers chose to provide a clear self-assessment/statement as to whether they had met or not met a requirement. b. One LETB volunteer and both examination providers did not make a clear statement of self-assessment. Key: Met Partially met; Not met N.BThe requirements under standard 1 are not applicable to examination providers. 35. Where requirements were not met or partially met under Standard 1, the reasons were: a. Acknowledgement through self-assessment on the part of the volunteer that meeting the requirement was the subject of ongoing project work; b. Focussing on the subject of equality and diversity when it is mentioned in a requirement at the expense of that requirement as a whole. 36. A number of actions were recommended to volunteers: Volunteer No of actions These ranged from requests to learn more about ongoing project work to sight of particular policies or outputs from committees that were cited in narrative but not presented as evidence. 38. Actions were also generated when a requirement was met where in the opinion of the GDC, the combination of the narrative and evidence where presented demonstrated good practice and had the potential to be a model response or the basis of an illustrative example for future programme providers working with the Standards For Specialty Education. Page 5

6 Standard 2 Key: Met Partially met; Not met Volunteers 1-3 programme providers/4-5 examination providers Requirement 11 is not applicable to examination providers 39. Where requirements were not met or partially met under Standard 2, the reasons included: a. Acknowledgement through self-assessment on the part of the volunteer that meeting the requirement was the subject of ongoing project work (multiple incidence across volunteers); b. A lack of clarity regarding the form of a LETB Quality management process : the level of its documentation and its availability to stakeholders including trusts and trainees. c. The need for further information about governance arrangements including, if possible, sight of original documentation 40. A number of actions were recommended to volunteers: Volunteer No of actions These ranged from requests to learn more about ongoing project work to sight of particular policies or documentation that were cited in narrative but not presented as evidence. 42. Actions were also generated when a requirement was met where in the opinion of the GDC, the combination of the narrative and evidence where presented demonstrated good practice and had the potential to be a model response or the basis of an illustrative example for future programme providers working with the Standards for Specialty Education. 43. There is a particular need to understand the examination providers ability to supply evidence which, whilst very useful to the quality assurance process, may possess significant data protection and business-oriented sensitivities. 44. This will be the subject of ongoing discussion with the volunteers over the remainder of Page 6

7 Standard 3 Key: Not completed Met Partially met; Not met Volunteers 1-3 programme providers/4-5 examination providers Requirement 12 and are not applicable to examination providers 45. Where requirements were not met or partially met under Standard 3, the reasons included: a. Through a GDC drafting error, one volunteer did not self-assess against three of the requirements; b. Acknowledgement through self-assessment on the part of the volunteer that meeting the requirement was the subject of ongoing project work; c. A description of process and associated documentation without details of specific numbers and outputs. An example would be the reference to ARCP appeals rules and processes, whilst recognising the value and transparency of seeing the rules in the evidence presented, the GDC would be interested in seeing evidence of actual numbers of appeals in a calendar year together with an indication of outcomes. d. The need for more information about the use of externality in assessment processes. 46. A number of actions were recommended to volunteers: Volunteer No of actions These ranged from requests to learn more about ongoing project work to sight of particular policies or documentation that was cited in narrative but not presented as evidence. 48. Actions were also generated when a requirement was met where in the opinion of the GDC, the combination of the narrative and evidence where presented demonstrated good practice and Page 7

8 had the potential to be a model response or the basis of an illustrative example for future programme providers working with the Standards for Specialty Education. Stakeholder meeting and feedback 49. On 29 April 2016 we held a very successful stakeholder engagement meeting with all programme and examination providers who contributed to the pilots plus the chair of COPDEND and a representative from HEE Quality Team. 50. This helped to clarify revisions that could be made to refine the Standards for Specialty Education and informed the discussion about a proportionate risk-based, process for future quality assurance for specialty training and education. 51. We drew together feedback from the stakeholders to ascertain if: a. The level and wording of the standards and requirements in the Standards for Specialty Education are at the appropriate level, or in need further refinement and if so, how. b. Different standards and requirements would be needed for different education providers, namely the Deaneries/LETBs and Royal Colleges c. What evidence could readily be provided by Deaneries/LETBs and Royal Colleges, which would assure us of their quality management processes, being conscious of resource implications for ourselves and our stakeholders. 52. We determined that stakeholders could learn from one another with regards to the quality management processes they presently use and the information available on a quality management level to inform and support our quality assurance needs. 53. The pilot process has given greater clarity to a proportionate level of quality assurance activity that could be undertaken by the GDC in the future. There were also a number of useful suggestions for the refinements of the standards. 54. The volunteers fed back that the level of detail set out by us in the pilots to quality assure the process, was necessary, appropriate and of the right level. 55. The volunteers made valuable suggestions to improve the Standards for Specialty Education and the proposed process as well as highlighting further areas for us to explore. 56. It was evident from the constructive feedback received that it would be appropriate and beneficial to ourselves and stakeholders, to have discrete standards for the LETBs/Deaneries and Royal Colleges, as they focus on different standards and requirements. 57. Specific feedback was given to all individual volunteers who took part in the pilots by the end of summer 2016, seeking their feedback by the end of October Next steps 58. The GDC s Quality Assurance and Education Policy team will continue to lead on the development of quality assurance processes for specialty education and training. 59. Further to feedback from the pilots and volunteers, we propose to develop a predominantly paper-based quality assurance process, which is proportionate and takes into account resources available to ourselves and the relevant stakeholders, namely the LETBs/Deaneries and Royal Colleges. 60. We will consider and consult upon, as part of this paper-based quality assurance process, a requirement for paper/electronic evidence to support a submission; we anticipate evidence will Page 8

9 be most valuable when it demonstrates both policy and process background together with an indication of actual metrics and outputs. 61. Consideration will also be given to the potential for inspection activity if required and the resources needed to support this. 62. We have developed a plan for the next steps to take forward the development of quality assurance of specialty education and training. This will include a clear rationale for the developments, the proposed process, proposed implementation, the resources needed to support the proposed activity, a clear timeline for implementation, including any necessary further consultation and delivery and identify risks and potential solutions. 63. This project will include the revision of the Standards for Specialty Education and Training, being mindful of the differing remits of the LETBs/Deaneries and Royal Colleges. 64. The planning will be completed by late 2016 and will be discussed with the Council seeking approval for the next steps in December Further to Council approval, the Education Policy and QA team will a. Share and publish the summary of the pilot findings with all relevant stakeholders by the end of 2016 b. Send out the revised Standards for Specialty Education for consultation in early 2017 for a three month period. c. Set up a further, focussed meeting with the Royal Colleges and faculties in early 2017, to identify their specific needs better and give them greater guidance and clarity about these quality assurance developments, to inform the process as well as support their future submissions. d. Liaise with stakeholders throughout 2017 about developments and the next steps regarding the delivery and implementation of the QA process which will commence by the end of the year. Page 9