HEALTH SERVICES AND DELIVERY RESEARCH

Transcription

1 HEALTH SERVICES AND DELIVERY RESEARCH VOLUME 1 ISSUE 15 DECEMBER 2013 ISSN An evauation of foundation doctor training: a mixed-methods study of the impact on workforce we-being and patient care [the Evauating the Impact of Doctors in Training (EDiT) study] S Mason, C O Keeffe, A Carter, R O Hara and C Stride DOI /hsdr01150

2

3 An evauation of foundation doctor training: a mixed-methods study of the impact on workforce we-being and patient care [the Evauating the Impact of Doctors in Training (EDiT) study] S Mason, 1 *CO Keeffe, 1 A Carter, 2 RO Hara 1 and C Stride 2 1 Schoo of Heath and Reated Research (ScHARR), University of Sheffied, Sheffied, UK 2 Institute of Work Psychoogy, Sheffied University Management Schoo, Sheffied, UK *Corresponding author Decared competing interests of authors: none Pubished December 2013 DOI: /hsdr01150 This report shoud be referenced as foows: Mason S, O Keeffe C, Carter A, O Hara R, Stride C. An evauation of foundation doctor training: a mixed-methods study of the impact on workforce we-being and patient care [the Evauating the Impact of Doctors in Training (EDiT) study] Heath Serv Deiv Res 2013;1(15).

4

5 Heath Services and Deivery Research ISSN (Print) ISSN (Onine) This journa is a member of and subscribes to the principes of the Committee on Pubication Ethics (COPE) ( Editoria contact: nihredit@southampton.ac.uk The fu HS&DR archive is freey avaiabe to view onine at Print-on-demand copies can be purchased from the report pages of the NIHR Journas Library website: Criteria for incusion in the Heath Services and Deivery Research journa Reports are pubished in Heath Services and Deivery Research (HS&DR) if (1) they have resuted from work for the HS&DR programme or programmes which preceded the HS&DR programme, and (2) they are of a sufficienty high scientific quaity as assessed by the reviewers and editors. HS&DR programme The Heath Services and Deivery Research (HS&DR) programme, part of the Nationa Institute for Heath Research (NIHR), was estabished to fund a broad range of research. It combines the strengths and contributions of two previous NIHR research programmes: the Heath Services Research (HSR) programme and the Service Deivery and Organisation (SDO) programme, which were merged in January The HS&DR programme aims to produce rigorous and reevant evidence on the quaity, access and organisation of heath services incuding costs and outcomes, as we as research on impementation. The programme wi enhance the strategic focus on research that matters to the NHS and is keen to support ambitious evauative research to improve heath services. For more information about the HS&DR programme pease visit the website: This report The research reported in this issue of the journa was funded by the HS&DR programme or one of its proceeding programmes as project number 08/1819/221. The contractua start date was in August The fina report began editoria review in August 2012 and was accepted for pubication in December The authors have been whoy responsibe for a data coection, anaysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors report and woud ike to thank the reviewers for their constructive comments on the fina report document. However, they do not accept iabiity for damages or osses arising from materia pubished in this report. This report presents independent research funded by the Nationa Institute for Heath Research (NIHR). The views and opinions expressed by authors in this pubication are those of the authors and do not necessariy refect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Heath. If there are verbatim quotations incuded in this pubication the views and opinions expressed by the interviewees are those of the interviewees and do not necessariy refect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Heath. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. Pubished by the NIHR Journas Library ( produced by Prepress Projects Ltd, Perth, Scotand (

6 Heath Services and Deivery Research Editor-in-Chief Professor Ray Fitzpatrick Professor of Pubic Heath and Primary Care, University of Oxford, UK NIHR Journas Library Editor-in-Chief Professor Tom Waey Director, NIHR Evauation, Trias and Studies and Director of the HTA Programme, UK NIHR Journas Library Editors Professor Ken Stein Chair of HTA Editoria Board and Professor of Pubic Heath, University of Exeter Medica Schoo, UK Professor Andree Le May Chair of NIHR Journas Library Editoria Group (EME, HS&DR, PGfAR, PHR journas) Dr Martin Ashton-Key Consutant in Pubic Heath Medicine/Consutant Advisor, NETSCC, UK Professor Matthias Beck Chair in Pubic Sector Management and Subject Leader (Management Group), Queen s University Management Schoo, Queen s University Befast, UK Professor Aieen Carke Professor of Heath Sciences, Warwick Medica Schoo, University of Warwick, UK Dr Tessa Criy Director, Crysta Bue Consuting Ltd, UK Dr Peter Davidson Director of NETSCC, HTA, UK Ms Tara Lamont Scientific Advisor, NETSCC, UK Professor Eaine McCo Director, Newcaste Cinica Trias Unit, Institute of Heath and Society, Newcaste University, UK Professor Wiiam McGuire Professor of Chid Heath, Hu York Medica Schoo, University of York, UK Professor Geoffrey Meads Honorary Professor, Business Schoo, Winchester University and Medica Schoo, University of Warwick, UK Professor Jane Norman Professor of Materna and Feta Heath, University of Edinburgh, UK Professor John Powe Consutant Cinica Adviser, Nationa Institute for Heath and Care Exceence (NICE), UK Professor James Raftery Professor of Heath Technoogy Assessment, Wessex Institute, Facuty of Medicine, University of Southampton, UK Dr Rob Riemsma Reviews Manager, Keijnen Systematic Reviews Ltd, UK Professor Heen Roberts Professoria Research Associate, University Coege London, UK Professor Heen Snooks Professor of Heath Services Research, Institute of Life Science, Coege of Medicine, Swansea University, UK Pease visit the website for a ist of members of the NIHR Journas Library Board: Editoria contact: nihredit@southampton.ac.uk NIHR Journas Library

7 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Abstract An evauation of foundation doctor training: a mixed-methods study of the impact on workforce we-being and patient care [the Evauating the Impact of Doctors in Training (EDiT) study] S Mason, 1 *CO Keeffe, 1 A Carter, 2 RO Hara 1 and C Stride 2 1 Schoo of Heath and Reated Research (ScHARR), University of Sheffied, Sheffied, UK 2 Institute of Work Psychoogy, Sheffied University Management Schoo, Sheffied, UK *Corresponding author Background: A major reform of junior doctor training was undertaken in , with the introduction of foundation training (FT) to address perceived probems with work structure, conditions and training opportunities for postgraduate doctors. The we-being and motivation of junior doctors within the context of this change to training (and other changes such as restrictions in working hours of junior doctors and increasing demand for heath care) and the consequent impact upon the quaity of care provided is not we understood. Objectives: This study aimed to evauate the we-being of foundation year 2 (F2) doctors in training. Phase 1 describes the aims of deivering foundation training with a focus on the roe of training in supporting the we-being of F2 doctors and assesses how FT is impemented on a regiona basis, particuary in emergency medicine (EM). Phase 2 identifies how F2 doctor we-being and motivation are infuenced over F2 and specificay in reation to EM pacements and quaity of care provided to patients. Methods: Phase 1 used semistructured interviews and focus groups with postgraduate deanery eads, training eads (TLs) and F2 doctors to expore the strategic aims and impementation of FT, focusing on the speciaty of EM. Phase 2 was a 12-month onine ongitudina study of F2 doctors measuring eves of and changes in we-being and motivation. In a range of speciaties, one of which was EM, data from measures of we-being, motivation, intention to quit, confidence and competence and job-reated characteristics (e.g. work demands, task feedback, roe carity) were coected at four time points. In addition, we examined F2 doctor we-being in reation to quaity of care by reviewing cinica records (criterion-based and hoistic reviews) during the emergency department (ED) pacement reating to head injury and chronic obstructive pumonary disease (COPD). Resuts: Phase 1 of the study found that variation exists in how successfuy FT is impemented ocay; F2 acks a ceary defined end point; there is a minima focus on the we-being of F2 doctors (ony on the few aready shown to be in difficuty ); the ED presented a chaenging but worthwhie earning environment requiring a significant amount of support from senior ED staff; and disagreement existed about the performance and confidence eves of F2 doctors. A tota of 30 EDs in nine postgraduate medica deaneries participated in phase 2 with 217 foundation doctors competing the ongitudina study. F2 doctors reported significanty increased confidence in managing common acute conditions and undertaking practica procedures over their second foundation year, with the biggest increase in confidence and competence associated with their ED pacement. F2 doctors had eves of job satisfaction and anxiety/depression that were comparabe to or better than those of other NHS workers, and adequate quaity and safety of care are being provided for head injury and COPD. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. v

8 ABSTRACT Concusions: There are ongoing chaenges in deivering high-quaity FT at the oca eve, especiay in time-pressured speciaties such as EM. There are aso chaenges in how FT detects and manages doctors who are strugging with their work. The survey was the first to document the we-being of foundation doctors over the course of their second year, and average scores compared we with those of other doctors and heath-care workers. F2 doctors are benefiting from the training provided as we found improvements in perceived confidence and competence over the year, with the ED pacement being of most vaue to F2 doctors in this respect. Athough adequate quaity of care was demonstrated, we found no significant reationships between we-being of foundation doctors and the quaity of care they provided to patients, suggesting the need for further work in this area. Funding: The Nationa Institute for Heath Research Heath Services and Deivery Research programme. vi NIHR Journas Library

9 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Contents Gossary...xi List of abbreviations....xiii Scientific summary...xv Chapter 1 Background 1 Introduction 1 Poicy context and iterature review 1 Summary 5 Chapter 2 Aims and objectives 7 Aims 7 Objectives 7 Chapter 3 Phase 1: consutation exercise and scoping study 9 Introduction 9 Aim and objectives 9 Methods 9 Resuts 10 Chapter 4 Longitudina study 25 Aims and objectives 25 Methods 25 Resuts 34 Chapter 5 A cinica case notes review of foundation year 2 doctors quaity of care 47 Introduction 47 Aims 47 Methods 48 Resuts 52 Chapter 6 Anaysing the reationship between foundation year 2 doctors job-reated characteristics, work-reated we-being and motivation, quaity of care and performance during (emergency department) pacements 59 Introduction and aims 59 Methods 59 Resuts 60 Summary 62 Chapter 7 Discussion 65 Introduction 65 Principa findings 65 Limitations 71 Future study 72 Impications for practice 72 Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. vii

10 CONTENTS Acknowedgements 75 References 77 Appendix 1 Schematic of current postgraduate training in the UK foowing the Tooke report Appendix 2 Summary of key components of the Foundation Programme 85 Appendix 3 Nationa stakehoder interview schedue 87 Appendix 4 Regiona stakehoder interview schedue 91 Appendix 5 Trainer stakehoder interview schedue 95 Appendix 6 Letter of invitation to postgraduate medica education stakehoders and emergency department consutants/training eads for participation in phase 1 interviews 97 Appendix 7 Participant information sheet for phase 1 interviews 99 Appendix 8 Consent form for phase 1 interviews 101 Appendix 9 Foundation year 2 doctors focus group schedue (phase 1) 103 Appendix 10 Letter of invitation to foundation year 2 doctors for participation in phase 1 focus groups 105 Appendix 11 Focus group participant information sheet 107 Appendix 12 Focus group consent form 109 Appendix 13 Website information/screenshot 111 Appendix 14 Letter of invitation to foundation year 2 doctors for participation in phase Appendix 15 Survey participant information sheet 115 Appendix 16 Survey consent form 119 Appendix 17 The EDiT study survey: piot questionnaire 121 Appendix 18 Fina questionnaire 135 Appendix 19 Expanded resuts tabes 149 Appendix 20 Quaitative anaysis of survey text comments 159 Appendix 21 Reviewer information sheet 163 Appendix 22 Screenshot of quaity data-coection too: front page 165 viii NIHR Journas Library

11 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 23 Head injury criteria and scoring 167 Appendix 24 Chronic obstructive pumonary disease criteria and scoring 171 Appendix 25 Hoistic review form 175 Appendix 26 Inter-rater reiabiity: consistency/rank scores 177 Appendix 27 Proportions of expicit review criteria met across a head injury and chronic obstructive pumonary disease case notes 181 Appendix 28 Distribution of hoistic review scores across a head injury and chronic obstructive pumonary disease case notes 187 Appendix 29 Protoco (phase 1) 189 Appendix 30 Protoco (phase 2) 197 Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. ix

12

13 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Gossary Anaysis of variance A psychometric test of the difference between two variabes. Chronic obstructive pumonary disease A coection of ung diseases incuding chronic bronchitis, emphysema and chronic obstructive airways disease. Cinica supervisor A consutant trained to supervise foundation doctors. A cinica supervisor is responsibe for monitoring and signing off those foundation doctors working in their department as part of their Foundation Programme. Comparative study A comparative study is one in which ony a sma body of evidence exists to benchmark the resuts of the study against. Conference of Postgraduate Medica Deans of the United Kingdom A quartery conference of Medica Deans. Cronbach s apha A measure of reiabiity with a higher correation suggesting a stronger reiabiity. Educationa supervisor A medica practitioner trained to supervise foundation doctors during a specific speciaty pacement or through a number of pacements across their Foundation Programme. Eectronic portfoio An onine record of progress maintained by a foundation doctors. Incudes the recording of scores from the Foundation Programme assessments of competency. Emergency department Department within a hospita where emergency or acute patients are taken for initia assessment and management. Emergency medicine Refers to the speciaty of emergency medicine (or choice of career). Evauating the Impact of Doctors in Training (EDiT) study A nationa research study being undertaken by the Heath Services Research section at the Schoo of Heath and Reated Research (ScHARR), University of Sheffied. The study is funded by the Heath Services and Deivery Research programme of the Nationa Institute for Heath Research as part of a programme of work examining the impact of staff motivation and we-being on patient care. Foundation doctor Generic term for a foundation doctors (foundation year 1 and foundation year 2). Foundation Programme assessments Assessments that must be competed by a foundation doctors to demonstrate competency against the Foundation Curricuum. Foundation schoo A body that administers the Foundation Programme within a ocaity within the deanery area. Foundation schoo administrator Administrative roe within the foundation schoo. Foundation schoo director The head of the foundation schoo. This individua reports to the postgraduate dean. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. xi

14 GLOSSARY Foundation training The term for the new structure for postgraduate medica training, introduced in It consists of a 2-year programme-based training structure giving trainees (foundation doctors) experience of a wide range of medica speciaties before they choose their career speciaty. Foundation year 1 doctor A postgraduate doctor in his or her first year of training foowing graduation from medica schoo. At the end of this year doctors achieve fu registration with the Genera Medica Counci. Foundation year 2 doctor A doctor in the second and fina year of his or her postgraduate training, now termed foundation training. Doctors at this eve are usuay expected to practice increasingy as independent and autonomous practitioners. Genera Medica Counci The statutory body/reguator of undergraduate and postgraduate medica training in the UK. Intracass correation A measure of inter-rater reiabiity. Junior doctor Previous tite of foundation doctor. Longitudina study A type of research design in which participants are foowed up for a specified period and data are coected at different time points. Normative data A substantia body of evidence or data on a specific variabe (e.g. job satisfaction) that can be used to benchmark the resuts of a study. Pacements The cinica speciaties where foundation doctors obtain their training. Typicay there are six pacements in the 2-year Foundation Programme. Postgraduate education stakehoders A study term to describe those individuas who work in postgraduate medica education, such as postgraduate deans and foundation schoo directors. Postgraduate medica deaneries Administrative regiona bodies responsibe for the impementation of postgraduate medica training in the UK, in accordance with the standard set out by the Genera Medica Counci. Postgraduate Medica Education and Training Board The authority for training throughout the NHS before its merger with the Genera Medica Counci in Speciaist training Period of speciaised training eading to consutant recognition. Trainee Generic term for a doctors in training, incuding foundation year 1 and foundation year 2 doctors. Training eads Generic term in our study for a staff with responsibiities for training foundation doctors. xii NIHR Journas Library

15 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 List of abbreviations ANOVA COPD CS CT DOPs ECG ED EDiT EDTL EM e-portfoio ES EWTD anaysis of variance chronic obstructive pumonary disease cinica supervisor computerised tomography direct observation of procedures eectrocardiogram emergency department Evauating the Impact of Doctors in Training emergency department training ead emergency medicine eectronic portfoio educationa supervisor European Working Time Directive F1/FY1 foundation year 1 F2/FY2 foundation year 2 FS foundation schoo FSA FSD FT GMC ICC mini-cex mini-pat MMC NICE PES PMD PMETB SD SHO TL foundation schoo administrator foundation schoo director foundation training Genera Medica Counci intracass correation mini-cinica evauation exercise mini-peer Assessment Too Modernising Medica Careers Nationa Institute for Heath and Care Exceence postgraduate education stakehoder postgraduate medica deanery Postgraduate Medica Education Training Board standard deviation senior house officer training ead Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. xiii

16

17 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Scientific summary Background There is a growing recognition amongst poicy-makers that the heath and we-being of NHS staff is a vita component of the continuing commitment to provide high-quaity care for patients. NHS trusts with better records in improving the we-being of staff, evidenced by reduced sickness days and turnover, have demonstrated higher rates of patient satisfaction and better performance. Postgraduate medica training has undergone major restructuring in recent years with the introduction of foundation training (FT), in part to improve the working conditions of postgraduate doctors. The new FT mode introduced a 2-year fixed programme of training for doctors, repacing the previous house officer structure. In the second year of FT (foundation year 2 or F2), doctors are expected to become increasingy independent members of heath-care teams. This is particuary true in busy, shift-driven speciaties such as emergency medicine (EM) where deivering patient care is chaenging in a fast-paced, performance-driven service. Litte is known about the impact of FT on the we-being of foundation doctors, particuary evauating change over the period of training and the impact of working in speciaties such as EM. The ink between doctor we-being and quaity of patient care is aso under-researched. Theoretica concepts of motivation suggest that there is an energy poo from which amounts of energy are drawn according to demand aocation. It is ikey that extra resources of energy wi be required in the emergency work environment and we woud anticipate variations in motivation and we-being of F2 doctors to be associated with emergency department (ED) pacements. Objectives This study aimed to evauate the we-being of F2 doctors in training and to examine associations with quaity of care provided to patients attending the ED. It was carried out in two phases. Phase 1 objectives To describe the nationa strategic view of the aims of deivering FT with a particuar focus on the roe of training in supporting the we-being of doctors. To assess how FT is impemented on a regiona basis and in particuar its impact on the speciaty of EM. Phase 2 objectives To undertake a ongitudina study using a structured survey to assess F2 doctors in terms of their we-being, motivation, confidence and competence at four time points over a 12-month period. To conduct a survey at four time points [at the end of foundation year 1 (F1) and then after each F2 pacement] to assess the eve of and change in F2 doctor we-being, motivation, confidence and competence. One of these pacements wi be in EM and the impact of this pacement can be assessed in reation to the study outcomes. Assess patient safety and quaity of care by F2 doctors by reviewing the cinica records of patients receiving emergency care from F2 doctors and evauating routine ED data to ink workoad and mean time with the patient for each of the participating F2 doctors. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. xv

18 SCIENTIFIC SUMMARY Methods Phase 1 Consutation exercise and scoping study A consutation exercise and scoping study was undertaken to describe the strategic aims and impementation of FT in Engand, with a particuar focus on the speciaty of EM. The foowing quaitative methods were used. Eighteen semistructured interviews with key stakehoders [nationa stakehoders, postgraduate deans, foundation schoo directors, training eads (TLs) in EDs] and four focus groups with F2 doctors in EM were undertaken to identify nationa structures, potentia variation in the impementation of FT ocay, the roe of doctors in training, the provision of training by other staff within the speciaty, the we-being of doctors in training and the quaity of patient care being provided by foundation doctors. Postgraduate education stakehoders (PESs), incuding postgraduate deans and foundation schoo directors, were recruited from four deaneries, and TLs and F2 doctors were recruited from four EDs in Engand between December 2008 and March Three researchers were invoved in conducting and anaysing the interviews and focus groups to gain mutipe perspectives and insights into the data coected. Overa themes vaidated by participants were derived for each stakehoder group. A summary tempate was then produced to bring together the simiarities and differences across the groups. Phase 2 Longitudina study A 12-month ongitudina study was undertaken with a sampe of F2 doctors in Engand between August 2010 and August 2011 to measure eves of and changes in we-being and motivation at four time points. Measures of work-reated outcomes (we-being, motivation, intention to quit, confidence in managing acute conditions and experience in performing practica procedures) and job-reated characteristics (e.g. work demands, task feedback, roe carity) were coected using an onine survey at four time points, one before and three during the 12 months of the study, covering a range of speciaties, one of which was a pacement in EM. A tota of 30 EDs in nine postgraduate medica deaneries participated in the ongitudina study. In tota, 654 F2 doctors had a pacement in the participating EDs in the study period and were eigibe to be incuded, with 217 doctors competing the study (33.2%). We anaysed the pattern of change in sampe mean scores over the four survey time points for each of the work-reated outcomes and job-reated characteristics. Variation in mean scores by time of pacement in the ED was aso compared with normative data. A cinica case notes review of foundation year 2 doctors quaity of care Quaity of care, as documented in the cinica records of F2 doctors during their pacement in the ED, was assessed using two we-estabished methods (criterion based and hoistic review). The F2 doctors were a participants in the ongitudina study and were working in 10 of the 30 participating EDs. In tota, 74 doctors were incuded in this part of the study and an average of 10 case notes per doctor were reviewed. The study assessed quaity of care deivered in reation to two cinica conditions, head injury and chronic obstructive pumonary disease (COPD). Higher speciaist trainees in EM were recruited from each of the xvi NIHR Journas Library

19 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 participating EDs and trained to review the cinica records reating to their own hospita using a standardised assessment process. For the criterion-based review, criteria were deveoped for the two cinica conditions using reevant nationa and oca cinica guideines and vaidated by two ED consutants. Scoring of criterion-based data invoved cacuating a tota score for each head injury and COPD patient record. To compare mean criterion scores for the two conditions, a score for the proportion of the criteria met was aso cacuated by dividing the tota score by the maximum potentia score. The hoistic review aowed reviewers to assess different eves of heath-care quaity in the notes and rate the quaity of care provided on a numerica scae (1 = unsatisfactory, 6 = very best care). Statistica anaysis examined inter-rater reiabiity and the quaity of care deivered by F2 doctors during ED pacements. Mean scores were cacuated for criterion-based and hoistic ratings across three eves of case compexity (ow, average and high). Anaysing the association between work-reated we-being and motivation, quaity of care and performance in the emergency department Associations between F2 doctor work-reated we-being and F2 doctor quaity of care were anaysed at one point in time, during their pacement in the ED. Doctors with both ED quaity-of-care data coected from the cinica case notes review and ED we-being data coected from the ongitudina study were incuded in the anaysis to measure the pattern and strength of associations. Additiona data on F2 doctor ED performance were aso obtained, measuring performance against the 4-hour ED target and compared with the ED we-being outcomes. Resuts Phase 1 Nationa and regiona PESs agreed that there was a cear nationa framework in pace for FT but that variation existed at the regiona eve in how FT was impemented. To an extent the variation refected oca NHS service needs; however, differences in the quaity and amount of supervision and feedback that trainees received was concerning. There was a ack of a ceary defined end point for the second year of FT, which meant inconsistency in the end points used (e.g. competion of FT assessments, demonstrating competence, successfuy moving into speciaty training). Three stakehoder groups [nationa and regiona stakehoders and emergency department training eads (EDTLs)] agreed that F2 doctor we-being was a focus ony for those F2 doctors aready shown to be in difficuty ; there were no systems in pace to identify periods of overwork or strain for the average trainee that coud cause detriment to their performance. A stakehoders agreed that the ED presented a chaenging but worthwhie earning environment requiring a significant amount of support from senior staff. In some cases this paced significant strain on aready stretched ED senior staff. There was disagreement about the performance and confidence eves of F2 doctors, with EDTLs seeing F2 doctors as underprepared for the demands of a performance-driven service. PESs and nationa and regiona stakehoders suggested that F2 doctors were fit for purpose, athough they acknowedged that there were often difficuties at the beginning of pacements. Trainees admitted to having anxieties over eements of patient care in the time-pressured ED environment. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. xvii

20 SCIENTIFIC SUMMARY Phase 2 Longitudina study F2 doctors reported a significant increase in confidence managing common acute conditions over the second foundation year. The biggest increase in confidence was associated with undertaking the ED pacement. Competence managing five common practica procedures improved significanty over the second foundation year. The biggest increase in competence in a five practica procedures was associated with undertaking the ED pacement. F2 doctors have comparabe or better eves of job satisfaction and anxiety/depression than other NHS workers and on average expend a ower eve of effort than those in manageria roes. However, the ED pacement was associated with a sight increase in anxiety and effort (not significanty different from eves in the comparison groups) and, for some groups, a decrease in extrinsic job satisfaction (e.g. issues of pay and working conditions). Summary of cinica case notes review of foundation year 2 doctors quaity of care Mean scores for the proportion of head injury and COPD criteria met were 50.8% and 54.9% respectivey. A detaied breakdown of these resuts identified weaknesses in reation to the extent of cinica information recorded in case notes. Findings from an anaysis of case mix for head injury and COPD cases reveaed no significant differences in reation to quaity and compexity of cinica presentation, indicating that any observed differences in quaity of care are not attributabe to case mix variation. A high eve of agreement was found among reviewers across the sites for the criterion-based review [intracass correations (ICCs) of ], but agreement for the hoistic review was ower (ICCs of ). Reationship between foundation year 2 doctors work-reated we-being and job-reated characteristics and quaity of care during emergency department pacements No statisticay significant associations were found between work-reated we-being and quaity of care or performance outcomes. There was evidence of sma- to medium-sized associations between anxiety and depression and two performance outcomes (with higher eves of anxiety or depression ikey to be associated with poorer performance outcomes). A simiar pattern of association was seen for motivation and two quaity-ofcare outcomes (with higher eves of effort ikey to be associated with better quaity-of-care scores). Concusions Our study was the first to systematicay examine a sampe of trainees at the end of their first year (F1) and throughout the second year (F2). We used a mutipe-perspective mixed-methods study to examine the current arrangements for the deivery of FT in Engand and to examine a group of 217 foundation doctors in 28 NHS trusts as they proceeded through their second year of training (from August 2010 to August 2011). We found a cear framework for FT with some variation in educationa phiosophy, impementation and views on assessment. There were disagreements over the outcomes of training and the ack of a cear end to F2 training. The ongitudina study showed an increase in confidence and competence of trainees xviii NIHR Journas Library

21 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 across F2, with the steepest rise occurring after the ED pacement. Trainees had simiar or better eves of we-being than other doctors and heath-care workers but reported a sight rise in anxiety and effort (not significanty different from eves in the comparison groups) and, for some, a decrease in extrinsic job satisfaction associated with the ED pacement. We have demonstrated that it is possibe to systematicay record eves of we-being of trainee doctors and compare these over time and with other normative studies, enabing appropriate interpretation. These measures coud be incorporated within trainees eectronic portfoios (e-portfoios) to faciitate monitoring of trainee we-being and to enabe any changes to be acted on. Limitations The study has imitations in the sampe providing stakehoder input, and focus group participants may have more experience in teaching hospitas than smaer NHS trusts. Further studies may benefit from incuding greater numbers of other staff who have more informa roes in supervising and supporting F2 doctors, such as senior nursing staff and nurse practitioners. Athough 217 F2 doctors from 28 acute trusts participated in the ongitudina study, this is a sma proportion of a F2 doctors training throughout the UK. This sampe made up approximatey one-third of the eigibe popuation undertaking ED pacements in the 28 trusts (654 doctors). It is possibe that the F2 doctors participating in the study had greater eves of we-being than those who chose not to participate. However, we achieved the target of 210 F2 doctors cacuated to be sufficient to show an effect on the measurement of we-being and motivation. Assessing quaity of care through case-note review is reiant on information being recorded in the notes, which may not refect every detai of the care provided. Future study Further studies examining quaity-of-care outcomes and junior doctors we-being and motivation. These woud need to be arge-scae, muticentre studies to provide sufficient power to examine possibe reationships. More arge-scae studies ooking at assessment of competence, feedback and case discussion conducted by a range of heath-care staff may yied further good practice that can be incorporated into the FT assessment programme. Impications for practice Disseminate the findings of this study to encourage more genera support for work-based earning and assessment as part of postgraduate medica education, especiay to organisations such as the UK Foundation Programme Board. We woud seek nationa communication of the findings so that participating trusts can earn of the findings through conferences such as the Heath Services Research Network annua symposium and the NHS Confederation conferences. Trainees eves of we-being and motivation can be measured accuratey over time and woud form an appropriate part of the e-portfoio, but this woud require timey feedback to supervisors to enabe appropriate work demands and roe carity to be determined within the pacement period. If this service cannot be provided within a usefu time frame a trainee report measure regarding their we-being, work demands and roe carity and use of their abiities shoud be communicated to the trainees and their supervisors, enabing oca changes in pacements to be made. There is a we-vaidated system for the recording of we-being amongst NHS staff [the Nationa NHS Staff Survey, URL: (accessed 22 November 2013)] and this woud be utiised to specificay identify and benchmark the we-being of Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. xix

22 SCIENTIFIC SUMMARY foundation doctors. However, it woud need to be acknowedged that this is an annua review and is not as accurate as pacement measures. This study offers cear evidence that a F2 doctors woud gain in confidence and competence from undertaking an ED pacement; however, this shoud be accompanied by additiona support for senior staff to enabe them to provide the eve of support that trainees need during this intense earning period. In addition, more consideration needs to be given to work ife baance issues during this pacement period. The success of workpace earning depends on the provision of adequate eves of supervision and support for trainees. The exact eve of support needs to be determined by working cosey with senior trust staff who support trainees in the workpace and their educationa supervisors. This by necessity wi not be one size fits a as it wi depend on a number of factors associated with service deivery and requires consutation with both the Foundation Programme and the trusts invoved. The espoused educationa phiosophy of medica training (as probem-based education supported by workpace experientia earning) shoud be debated to articuate a cear and understood purpose of FT, enabing the impementation of agreed earning outcomes with supervisors and trainees. Further work shoud be carried out on work-based assessments, with cose examination and deveopment of specific criteria that contribute to a ceary defined and measurabe endpoint for F2. Carefu consideration shoud be given to incorporating forma processes for careers advice at both the F1 and the F2 points in training to ensure that foundation doctors acquire the most appropriate training for their intended career track. Athough the benefits of ED pacements are acknowedged, this may not aways be the case when intended career tracks invove service speciaties such as aboratory medicine and radioogy. Funding The Nationa Institute for Heath Research Heath Services and Deivery Research programme. xx NIHR Journas Library

23 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Chapter 1 Background Introduction There have been a number of changes made recenty by the government to improve the working conditions of NHS staff based on evidence that improved working conditions can improve staff we-being and in turn improve the quaity of patient care. However, the evidence of a direct ink between the we-being of staff and the quaity of patient care within heath care is imited. More evidence is required about which characteristics of working in the NHS infuence staff we-being and which aspects of we-being infuence patient care. This study aims to evauate the we-being of foundation doctors and compare this with the quaity of care provided to patients attending the emergency department (ED). Key aspects of we-being that may infuence quaity of care incude motivation, job satisfaction and confidence. Measures of these factors wi have the potentia to be deveoped into a too that may be utiised more widey for doctors throughout the NHS. Poicy context and iterature review NHS poicy context The heath and we-being of NHS staff has been of great interest to poicy-makers in recent years, with a growing acknowedgement that good eves of heath and we-being are ikey to have benefits for organisations and patients. The roe of organisations in contributing to the heath and we-being of staff is recognised as key, with support structures aimed at improving the heath of staff ikey to positivey infuence staff retention, sickness absence, productivity and aso, potentiay, patient satisfaction and quaity of care. 1,2 The pubication of two major reports has increased the focus on heath and we-being in the UK workpace and NHS organisations. 3,4 The Back review 3 examined the heath of the UK working-age popuation with a focus on the arge-scae probem of sickness absence and reduced productivity (incuding the roe of common menta heath conditions). There is evidence that reduced we-being is one of the major causes of reduced productivity for individuas in work. Aongside this, a growing iterature inks morae and job satisfaction with heath outcomes and performance. Athough individuas may differ in the importance they attach to issues such as saary or eve of responsibiity, this review identified key job-reated characteristics that infuence we-being at work, such as empoyee autonomy and adequate socia support. Good management and eadership aso pay a vita roe in promoting we-being and improving performance. 3 Foowing on from the Back review, 3 the Boorman review 4 examined issues of heath and we-being in the NHS workforce. The focus on staff we-being is expained by the continuing high rates of sickness absence in the NHS, with over 10 miion sick days ost annuay, equivaent to 45,000 whoe-time equivaent staff, 5 with over one-quarter of absences caused by stress, depression and anxiety. The NHS review of heath and we-being 4 found inks between the we-being of staff and key performance indicators such as patient satisfaction and trust performance, with trusts with ower rates of sickness absence and turnover more ikey to score highy on indicators of patient satisfaction and quaity of care. The report 4 recommended that organisations deveop strategies and provide services to NHS staff to prevent and treat sickness, incuding work-reated stress, anxiety and depression, and that management be assessed on their contribution to staff heath and we-being. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 1

24 BACKGROUND These reports are consistent with pedges made in Lord Darzi s 2008 report 6 regarding the need for a broader commitment to heath and we-being in workpaces and the recognition that the heath and we-being of NHS staff was an important component of the commitment of the NHS to provide high-quaity care. Nationa Institute for Heath and Care Exceence (NICE) guidance has aso been produced, 7 which recommends a strategic approach by empoyers to the we-being of staff. This incudes ensuring that job design, seection, recruitment, training and appraisa promote we-being and that assessment of the we-being of empoyees is undertaken to identify areas for improvement. Athough the ink between staff deveopment, motivation and we-being and patient care is recognised as important, 8,9 the impact of staff stress, depression and other aspects of we-being on patient care has been generay under-researched. Evidence demonstrating a ink between indicators of we-being and indicators of patient safety, experience and quaity of care is rare and has primariy been coected in nurse settings in the USA. There is aso a ack of good-quaity evidence from data coected ongitudinay. 10 Training doctors in the NHS Training and appraisa have been identified in the iterature as important eements of appropriate peope management, impacting on knowedge and skis, job satisfaction and we-being, which in turn may infuence patient outcomes. 11 Previous studies have demonstrated reationships between the quaity and extent of training and appraisa and the we-being of staff and better patient care Recenty, postgraduate medica training has undergone changes in response to ong-standing criticism of its suitabiity in a modern, patient-centred NHS. A report by the Chief Medica Officer 17 highighted a number of perceived probems with the job structure, working conditions and training opportunities in postgraduate medica education, with the baance between medica training and service provision weighted too heaviy in favour of providing for service deivery at the expense of a we-structured and we-panned training programme for postgraduate trainees [senior house officers (SHOs)]. SHO training pacements were perceived as short term and stand-aone and not part of a ceary structured training programme. These issues caed into question whether doctors were being appropriatey trained to meet the demands of a modern, patient-centred NHS. 18 Postgraduate training was aso criticised for faiing to provide more trained speciaists for a consutant-ed NHS. 2 The report recommended the introduction of a new programme-based system of postgraduate training [foundation training (FT)] that woud provide broad-based speciaty experience and fexibe training arrangements. Foundation training The new mode of FT was pioted in 2004 and introduced nationay in The mode introduced a fixed 2-year Foundation Programme to address the perceived deficiencies of the previous postgraduate training grades (pre-registration house officer and SHO). Postgraduate training was structured around a forma programme with a nationa curricuum and structured assessment of cinica competencies (see Appendix 2). The first year of FT (foundation year 1 or F1) focused on deveoping the skis and competencies earned during undergraduate medica training. The competion of core competencies was required during F1 to achieve fu registration with the Genera Medica Counci (GMC). The second year of FT (foundation year 2 or F2) was designed to enabe doctors to become functioning members of the heath-care team, competent in the management of the acutey i patient and with key skis in team working, time management and communication with both professionas and patients. Foundation training generay consisted of 4-month pacements in a variety of speciaties to give postgraduate doctors sufficient experience of different areas of medicine. Modernising Medica Careers (MMC) aso provided a cear structure for post FT with run-through speciaist training foowing on from FT NIHR Journas Library

25 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 As we as introducing new structura arrangements, FT changed the deivery of training within pacements. For the first time doctors were expicity required to demonstrate competency to practise through the competion of a range of specific assessments (see Appendix 2). These assessments were based on a new Foundation Curricuum. 20 The Foundation Programme aso required the designation of educationa supervisors (ESs) and cinica supervisors (CSs), charged with ensuring that foundation doctors were meeting their educationa and training goas. The we-pubicised probems with seection processes in the eary stages of MMC reform ed to an inquiry into MMC, which aso examined FT. 21 The inquiry highighted a number of areas of concern with the FT mode, incuding the insufficient breadth of cinica experience in foundation pacements; a ack of fexibiity in programmes; and the ength of programme pacements. This contributed to a perception that foundation doctors were not reaching appropriate eves of cinica responsibiity compared with their SHO predecessors. The report aso recommended greater carity about expectations of the roe of F2 doctors in the heath-care team and what their service contribution shoud be. A more recent evauation of the Foundation Programme reported improvements incuding a we-defined curricuum, trainees exposed to a wider range of medica speciaties and impementation of a comprehensive programme of trainee assessment. 22 However, the report stated that the programme sti acked an articuated purpose, found that there was confusion over the roe of the F2 doctor and questioned the abiity of pacements to accuratey refect the current and future needs of the NHS. Further, the assessment process paced excessive oads on ESs and there were safety and quaity issues in the earning environment. Reduced hours of working and the European Working Time Directive As we as changes to the structure of postgraduate training, there has been a major change to the conditions of work for postgraduate foundation doctors. The New Dea for junior doctors, pubished in 1991, 23 highighted the need for improved working conditions for this workforce group primariy focused on working hours. It was widey acknowedged that excessive hours of practice amongst foundation doctors was a risk to patients. In 2003 the working week was imited to 56 hours and the European Working Time Directive (EWTD) 24 further imited the hours that medica staff coud work to a maximum of 48 hours. This was impemented in stages with the 48-hour imit enforced by aw in Heath service staff motivation and we-being High eves of stress among heath-care professionas has been recognised as a probem for some time. A review of doctors stress eves found that between 28% and 30% of doctors had above-threshod eves of stress compared with about 18% of the genera popuation. 16 A survey of over 11,000 NHS staff 5 found staff reporting high eves of stress and that they did not consider that senior managers took a positive interest in their heath. Some studies have aso reported high eves of depression amongst doctors. 25,26 Studies have aso reported higher eves of stress among doctors (both consutants and junior doctors) working in emergency medicine (EM), with above-threshod scores for around haf of the respondents from each group. 27,28 These eves of stress are again higher than might be expected among the genera popuation. 29 However, the impact of stress, depression and we-being on patient care has been generay under-researched. 10 Foundation doctor we-being There is a imited iterature examining the we-being of doctors in training. One study ooked at eves of psychoogica distress in SHOs working in the ED. 27 SHOs were seected from six EDs in London and received questionnaires to measure psychoogica outcomes and coping strategies. Over haf of respondents scored above the threshod for psychoogica distress on the Genera Heath Questionnaire. 30 Higher eves of anxiety and depression were reated to a venting stye of coping (such as expressing negative feeings) whereas ower scores for these outcomes were associated with a more active coping stye (such as devising strategies to cope with stressors). Another study foowed junior doctors for 3 years Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 3

26 BACKGROUND after their graduation from medica schoo finding that first-year postgraduates reported eves of depression of 29%, dropping to 10% by their third postgraduate year. 26 Foundation doctors and quaity of care Studies evauating the impact of foundation doctors on the quaity of patient care have evauated the foowing outcomes: (1) numbers of patients seen; (2) reattendance of patients in the ED; and (3) confidence and competence in managing conditions. A prospective observationa study compared the productivity (numbers of patients seen) of F2 doctors and SHOs working across two EDs in Scotand. 31 Both groups demonstrated a significant rise in productivity between the first and ast months of their attachments. There were no significant differences in productivity between the two groups of doctors over the 12-month study period. However, there were concerns about a reduction in the percentage of patients seen by junior doctors overa and an increased need for senior review of patients. Further anaysis by Armstrong et a. 32 investigated the number of patients seen by a junior doctors (SHO/F2) over a 3-year period. The study found a 4% decrease in the number of patients seen by junior doctors in this period. In addition, there was a significant reduction of 16.6% in the number of patients seen per hour (an indicator of work rate). A study at an inner-city ED in Engand aso found no significant differences in the mean number of patients seen by F2 doctors and SHOs over a 12-month period. 33 Individua doctor performance had a greater infuence on the number of patients seen than type of doctor (either F2 or SHO), with a sma number of F2 doctors seeing consideraby more patients than their SHO coeagues. A study by Whiticar et a. 34 compared reattendance rates of patients to the ED over a month in 2006 by grade of doctor assessing the first presentation. Junior doctors (SHOs and F2 doctors) had higher reattendance rates (2.83% vs. 2.32%; p = 0.52) than midde-grade doctors and nurse practitioners (athough the resut was not statisticay significant). Croft and Mason 35 assessed eves of confidence in foundation doctors management of common minor cinica presentations in an inner-city ED. Foundation doctors confidence in treating minor injury patients was identified as a probem, and a ack of exposure to minor injuries during daytime hours was cited by doctors as a possibe cause. A further study 36 evauated junior doctors experience in performing practica procedures in an ED. Two cohorts were measured: trainee doctors in the ED in June 2005 and June The study found that doctors in the ater cohort reported significanty ess experience in each procedure. One study 37 measured SHO and pre-registration SHO knowedge of basic acute care in 12 topics. A tota of 185 junior doctors from six UK hospitas were incuded in the study. This study found that knowedge was poor across a range of basic acute care topics and that junior doctors were poory prepared to identify and treat criticay i patients. Overa, these studies raise questions with regard to foundation doctors confidence and performance in the ED which require further investigation. NHS staff motivation, we-being and patient care There is imited evidence of a direct association between factors that affect performance and outcomes in heath care, which woud be important to take into account when studying a changing workforce. In one study, 15 which sought associations between organisationa practice and cinica outcomes, it was possibe to demonstrate a inkage between good human resources practice (such as appraisa and training) and effective teamwork and reductions in measures of patient mortaity. A further study in a non-heath care 4 NIHR Journas Library

27 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 setting demonstrated that organisationa cimate (e.g. ski deveopment, concern for empoyee wefare) was significanty associated with productivity and profitabiity, and that the reationship was mediated by empoyee job satisfaction. 38 There is an increasing iterature on inks between patient safety and organisationa cuture and cimate, with a range of toos and interview methods proposed. 39 The ink between staff deveopment, motivation and we-being and the infuence of these factors on patient care is recognised as important. 8,9 In one review of the iterature, 10 a significant inear effect was found between eves of nurse stress and burn-out and patient outcomes (patient satisfaction, medication errors and patient fas). However, the cross-sectiona designs of the studies and ack of contro of confounding variabes (such as doctor sickness absence) 40 imits the usefuness of these findings. 10 Few studies have evauated the consequences of we-being for foundation doctors in terms of confidence, competence or patient outcomes. A study of SHOs working in 27 hospitas evauated the reationship between psychoogica distress and confidence in performing cinica tasks. 41 The questionnaire was administered four times during the 6-month rotation. Overa, confidence eves in carrying out a range of practica and cinica tasks (recorded on a visua anaogue scae) increased significanty between the first and fourth months of the SHO training rotation. SHOs with higher psychoogica distress scores at the end of months 1 and 4 had ower confidence scores. Factors associated with greater psychoogica distress were organisationa, such as workoad, certain cinica presentations and consutation issues such as communication. Summary The changes to postgraduate medica training (incuding uncertainties over future direction) and restrictions on working hours impact directy on postgraduate foundation doctors in training. These changes have aso occurred at a time of rising demand for heath care, with greater demands on staff in terms of providing care in services that are increasingy performance driven. It is important to consider how these major changes have infuenced the we-being and motivation of foundation doctors and aso the consequent impact on quaity of care. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 5

28

29 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Chapter 2 Aims and objectives The study was conducted in two phases and used a mixed-methods approach across mutipe sites to achieve the foowing aims and objectives. Aims To describe the current arrangements for the deivery of FT in Engand. To identify how the experiences of F2 doctors training in EDs infuence their we-being and motivation. To evauate how the we-being and motivation of F2 doctors in EDs is associated with the quaity of patient care. To identify key measures of F2 doctor we-being and motivation that are associated with quaity of patient care. Key measures that are identified may underpin the deveopment of a too to monitor we-being and motivation during training. Objectives Phase 1 To conduct a nationa and regiona consutation exercise with training stakehoders to: describe the nationa strategic view of the aims of deivering FT, with a particuar focus on the roe of training in supporting the we-being of doctors assess how the nationa view is impemented on a regiona basis through the postgraduate deaneries and identify any regiona variation to impementation within the speciaty of EM undertake a scoping exercise to identify factors contributing to the we-being of F2 doctors in training within up to four EDs to deveop measures to inform a quantitative evauation of foundation doctors in phase 2 of this study. Phase 2 To undertake a ongitudina study using a structured survey to assess F2 doctors in terms of their we-being, motivation, confidence and competence at four time points over a 12-month period. To conduct a survey at four time points (at the end of F1 and then after each F2 pacement) to assess the eve of and change in F2 doctor we-being, motivation, confidence and competence. One of these pacements wi be in EM and the impact of this pacement can be assessed in reation to the study outcomes. Assess patient safety and quaity of care by F2 doctors by reviewing the cinica records of patients receiving emergency care from F2 doctors and evauating routine ED data to ink workoad and mean time with the patient for each of the participating F2 doctors. We wi examine the findings from phase 2 to: evauate whether there is a reationship between F2 doctor we-being and motivation and patient care identify best-practice modes of F2 doctor training, which might be generaised and impemented across the NHS to promote a heathy and productive foundation doctor workforce provide a starting point for the deveopment of a too that can be used to monitor the we-being, motivation and training of doctors in EM and other speciaties. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 7

30

31 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Chapter 3 Phase 1: consutation exercise and scoping study Introduction A consutation exercise with nationa and regiona postgraduate education stakehoders (PESs) and a scoping study with training eads (TLs) and F2 doctors in the ED were undertaken using quaitative methods to examine FT from mutipe perspectives at the nationa, regiona, trust and foundation doctor eves. Aim and objectives The aim of the consutation exercise and scoping study was to describe the current arrangements for the deivery of FT in Engand. The objectives were to: describe the nationa strategic view of the aims of deivering FT, with a particuar focus on the roe of training in providing for the we-being of F2 doctors assess how the nationa view was impemented on a regiona basis through the postgraduate medica deaneries (PMDs) and identify any regiona variation to impementation within the speciaty of EM identify factors contributing to the we-being of F2 doctors within up to four EDs. The data coected were to inform the deveopment of measures to be used in a quantitative evauation of F2 doctors in the phase 2 ongitudina study. Methods Ethica and governance arrangements Ethica approva for phase 1 was received in May 2009 (ref.: 09/H1307/27). Approvas from non-nhs organisations and research governance approvas from participating NHS trusts were obtained between June and November Consutation exercise To understand the strategic aims, nationa structure and impementation of FT, nationa and regiona stakehoders from key postgraduate educationa organisations invoved in the deivery and impementation of FT were interviewed by teephone or videoconference. A semistructured interview schedue was designed around the aims and objectives of the study, with a particuar focus on identifying potentia variation in the impementation of FT and provision within training of an appreciation of the we-being and motivation of F2 doctors (see Appendices 3 and 4 for interview schedues). A etter of invitation and information sheet about the Evauating the Impact of Doctors in Training (EDiT) study were given to interviewees in advance, aong with the interview questions. Written, informed consent was received from each participant before the interview (see Appendices 6 8 for the etter of invitation, information sheet and consent form respectivey). Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 9

32 PHASE 1: CONSULTATION EXERCISE AND SCOPING STUDY Scoping study A scoping study invoving interviews with TLs and focus groups with F2 doctors was carried out in four EDs to evauate the impact of training in the ED on both F2 doctors and other heath-care staff. The four EDs were seected from 15 EDs recruited for the phase 2 ongitudina study. There was a particuar focus on we-being and motivation of F2 doctors in order to deveop measures to inform the quantitative evauation in the phase 2 ongitudina study. Interviews took pace with TLs in EDs. A semistructured interview schedue was designed to assess the training roe of doctors, the impact of training on staff, the we-being of training doctors and the quaity of care (see Appendix 5). An Information sheet and interview questions were given to interviewees in advance (see Appendix 7). Written, informed consent was received from each participant before each interview. Focus groups were hed with four groups of foundation doctors in their ED pacement (mainy F2 doctors, athough some F1 doctors were present in two groups). A semistructured schedue (see Appendix 9) was aso designed for the focus groups, which incuded issues around we-being, confidence, competence and ED training experiences. Written, informed consent was received from each individua before the focus groups. Data anaysis (consutation exercise and scoping study) Three researchers were invoved in the anaysis of interview and focus group data in order to gain mutipe perspectives and insights into the data coected and ensure inter-rater reiabiity. Interviews and focus groups were not recorded to maintain anonymity of the participants; however, data coected from a participants in the consutation exercise and scoping study comprised thematic accounts and refections from each of the researchers, 42 the content of which was vaidated with the participants to gain a fu understanding of meaning. 43 One researcher produced tempates for the four participant groups, aggregating themes at the group eve for (1) nationa PESs, (2) regiona PESs, (3) emergency department training eads (EDTLs) and (4) F2 doctors in the ED primariy by using the interview/focus group questions as a priori coding. 44,45 Each tempate was then examined by the other two researchers, checking for inter-rater reiabiity to reduce any potentia bias. Agreement was high (92%), with issues of terminoogy being the main areas of correction; any areas of misunderstanding or possibe bias were corrected on each tempate. Individua tempates derived from the interviews and focus groups were aso reviewed by the three researchers for saience (agreement of themes across stakehoders) and difference (individua perspectives that add vaue to the enquiry) and key overa themes were derived for each group of participants. A summary tempate was then produced to bring together the simiarities and differences between key themes across the groups (Tabe 1). Resuts Sampe achieved Consutation exercise A tota of 10 interviews were undertaken, three with nationa PESs and seven with regiona PESs, between November 2008 and February Interviews asted for approximatey minutes. The nationa PES interviews were with representatives of three nationa bodies invoved with panning FT. The regiona PESs were from four deaneries and foundation schoos (FSs). 10 NIHR Journas Library

33 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 1 Main themes arising from comparison of perspectives of different phase 1 participant groups Nationa stakehoders Regiona stakehoders A cear nationa structure was deveoped in 2005 with 8000 trainees per annum in Engand Variation in impementation across the regions was seen, matching training to service needs. Pacements were simiar to previous schemes with few new or innovative pacements being created Attempts to assess the we-being of trainees were we meaning but confined to supporting the 1 2% of doctors described as in difficuty. The trainee survey (2007/8) described a higher number of doctors coping with experience outside their competence and intending to eave training Assessment processes were impemented in various ways with itte consistency across the regions. There was concern that trainees were not a getting constructive feedback on their work The main issue identified was that the second year of FT was acking in focus and did not achieve a miestone for trainees (compared with the first year, which ed to registration as a doctor) A cear nationa approach was described but NHS trusts were rarey invoved in the panning of pacements Athough foowing a simiar framework, variations in training deivery were noted, in particuar with regard to supervision and assessment There was no overa assessment of the we-being and motivation of trainees, athough there were cear procedures concerning the support given to doctors in difficuty Trainees emerging from the foundation scheme were fit for purpose and were better than before E-Portfoios were being used by most trainees, athough there were some chaenges with regard to the avaiabiity and use of information technoogy Three issues were noted: (1) difficuty in getting consutants to spend time as ESs; (2) pacements having different sign-off criteria at their end; and (3) no assessment of the impact of training on patient care TLs in NHS trusts F2 doctors No system to dea with trainees with we-being issues who ack motivation to earn. Informa mechanisms exist based on observation of support staff and feedback from ESs Training consisted of a mix of forma teaching and informa earning. Generic teaching sessions acked vaue. The ED was seen as a chaenging rotation where trainees needed much support from staff The 4-month rotations increased trainees experience but it was fet that students coud not achieve competence in their pacements within the 4-month period: they are just becoming competent when they eave The main issue was the variation in context of different pacements; the ED was chaenging but other roes were often supernumerary-type posts offering itte invovement in cinica decision-making Trainees were confident if (1) they received feedback on their decisions; (2) they had previous experience of the condition; and (3) the patient was satisfied with his or her treatment Trainees were anxious about their abiity to make decisions regarding patient discharge and how to dea with the associated risk Most trainees fet that they became competent in the ast month or two of their 4-month pacement period Teaching was not reevant to their practice and there were difficuties attending teaching sessions because of their shift patterns Baanced view on assessment; haf of them saw its vaue A enjoyed the ED context, finding it chaenging, and having the opportunity to make decisions about patient care e-portfoio, eectronic portfoio. Scoping study The scoping study took pace between November 2009 and February Eight interviews with EDTLs were carried out across four EDs. The group comprised six consutants (two F2 TLs, four CSs or ESs) and two nurse practitioner tutors. The interviews asted for approximatey minutes. Four focus groups were hed with F2 doctors in three EDs. Two focus groups were hed within a teaching hospita site (with seven and four participants) and the other two focus groups were hed in two separate arge acute trust EDs (with eight and six participants). The focus groups asted for approximatey 90 minutes. A further panned focus group in a smaer district hospita was canceed on two occasions because of staffing pressures. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 11

34 PHASE 1: CONSULTATION EXERCISE AND SCOPING STUDY Nationa stakehoder interview findings In the foowing sections the numbers in parentheses indicate the numbers of participants agreeing with a particuar statement. Statements without a number are the comments of one participant ony. Number of trainees and sufficiency Two of the three PESs agreed that there were 7000 Foundation trainees in Engand and a further 800 in Scotand (2). A participants fet that the voume was sufficient as there is open access for paces between trainees from the UK and trainees from other European countries (3). Additiona paces are required to aow non-european trainees to access the programme. Strategic aims and nationa structure A participants described the aims of FT as deivering a structured programme to provide a range of experiences to foundation doctors (3). The outcomes of the programme were designed to be common to a foundation doctors but these coud be achieved in a variety of ways. A cear nationa structure was described consisting of (1) a Nationa Curricuum, (2) a nationa structure of FSs headed by the UK Foundation Office, comprising the directors of FSs who met with education eaders to maintain an overa strategy, (3) an Assessment Framework and (4) a nationa conference to share best practice (with deanery, foundation doctor and organisationa representatives present) and a Foundation Programme website containing guides and review papers. Two participants described the reguator of medica education as having overa responsibiity for training (2). At the time of our interviews there was joint responsibiity for F1 training between the GMC and the Postgraduate Medica Education Training Board (PMETB), with F2 training reguated competey by the PMETB. Various review groups (such as the Curricuum Group) met with Academy of Medica Roya Coeges representatives to discuss standards and assessment toos and recommend changes to the PMETB in the form of review documents. Since competion of the interviews the PMETB has merged with the GMC and the atter is now responsibe for the reguation of a stages of postgraduate education and training. Participants described a variety of approaches that are currenty being used to manage the impementation of F2 training, such as stakehoder conferences (with deanery, foundation doctor and organisationa representatives present) (2) and a website containing guides and review papers. Variation in impementing foundation training at the regiona eve Pacements Participants agreed that foundation doctors have six 4-month rotations managed by the deaneries (3). Two of the three participants concuded that a uniform nationa impementation of FT was difficut to achieve, mainy because of regiona variation within the UK in terms of oca popuation heath needs and different operating structures (2). Some deaneries have pacements of varying ength (from 4 months to 1 year), combining a variety of roes (2), and participants noted that variation was encouraged within the nationa framework as a way of producing innovation and a variety of pacements (3): For exampe, combining GP and acute medicine pacements with eadership deveopment (was unpopuar with F2 doctors at first) can produce a rounded programme. It was noted that up to 15 trainees per year worked their F2 pacements outside the UK. Participants fet that foundation doctors were reuctant to try new earning opportunities that refected current job opportunities, preferring pacements refecting a more traditiona understanding of a doctor s roe in acute medicine. Some programmes offered good future career taster sessions whereas others did not (2). Overa, the innovative pacements woud seem to be in the minority and were often unpopuar with 12 NIHR Journas Library

35 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 trainees unti they were estabished in their use. However, athough the governing body (PMETB) encouraged varied training experiences, one participant fet that it did not evauate outcomes. Evauation of the we-being of foundation year 2 doctors in foundation training It was noted that, athough support for trainees was attempted and we meaning, itte systematic evauation of trainees we-being or motivation appeared to occur in practice. Participants fet that a good dea of time was spent discussing the few doctors in difficuty within the training system (1 2% in tota) and itte time was spent evauating the we-being or motivation of the average trainee. The former PMETB trainee survey asks questions of the trainees about themseves and the programme (e.g. how often they were forced to cope with probems beyond their competence or experience; their concern about the reporting of medica errors) (2). Further pacement feedback forms and mutisource feedback may offer some insights into whether probems had occurred in the workpace. A participant noted that medica training can be over nurturing and the idea of FT was to bring some sense of the reaity to trainees about work. Assessments A key part of achieving curricuum competencies is demonstrated by assessments (2) (e.g. observations, case-based discussions; see Appendix 2). Participants described the assessment of training outcomes as variabe, from both the trainees and the supervisors perspectives. PESs concerns were that deaneries vary in their understanding of the assessment process and constructive feedback is not aways being given. PESs fet that the trainees found the assessments to be just form fiing. Severa comments focused on the need to change, for exampe Assessments need to be tightened up and not be so wooy. In particuar, it shoud be ensured that assessments demonstrate[d] the achievement of competencies, assessments need[ed] to be refined and simpified for both the assessor and the trainee and they need to be thoughtfu; and focused. However, it was noted that the PMETB fee that work-based assessments are better than academic assessments. Evauation of foundation training Participants were asked how F2 training is evauated and the foowing comments were made: Deaneries are judged on how they meet their standards by QAFP [Quaity Assurance of the Foundation Programme] assessments and visits. Evidence of trainees competing their assessments in the eectronic portfoio. (2) Post pacement questionnaires. The Sign-off of the FY2 year. In response to whether postgraduate training of doctors was fit for purpose, two participants fet that it was moving in the right direction and one that there are a ot of things in the FT that work better than the od system. Much of this positive change was to do with work-based assessments (2). FT has reduced the variabiity in training by setting a specific curricuum, which has been the driver for earning. However, chaenges were acknowedged with regard to FT. A participants agreed that it was difficut to baance the F1 and F2 years (3), with F2 acking in carity in comparison to F1, which eads to GMC registration (2). Other issues were changing the view that teaching sessions are the ony earning opportunities, varying pacements across deaneries and in particuar getting enough community pacements. It was aso noted that not a pacements are fied by foundation doctors and that some F2 Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 13

36 PHASE 1: CONSULTATION EXERCISE AND SCOPING STUDY doctors are not empoyed after FT or do not get the job they want (this comment woud seem to refer to a greater number of generaist community roes avaiabe than speciaty pacements). Career deveopment of foundation year 2 doctors As FT is the step between medica schoo and speciaist training, we wanted to evauate the eve of support that existed for trainees to hep them decide on their future career. Career deveopment is aso an important facet of job satisfaction, 46 in particuar using your abiities. 47 Participants described a variety of career support resources provided in various programmes: taster sessions, career management sessions, speciaist training advisors and web resources. Recenty, a Nationa Education Advice Forum had been set up to improve careers advice. There was a comment that more sideways movement shoud be made in programmes, offering change and more choice. There was concern that trainees were sotted into work and miss the bigger picture. However, it was noted that taster sessions are not we taken up by trainees, possiby because it is difficut to get reease from their pacements. It was aso noted that career deveopment shoud be done in medica training: FY2 is too ate and that a more positive attitude was required of F2 doctors: there is more that we can do; but the trainee has been in the NHS for five to six years at this point and they shoud be abe to put some effort in themseves. Future deveopment and changes Foundation training had been under way for 3 years when this project started and there had been severa chaenges to its estabishment, most notaby the Tooke report. 21 We wanted to examine participants views on the future deveopments to this programme. Athough participants acknowedged criticisms of FT, they fet that there was poitica support for the training (and 4-month pacements) and that it woud remain reativey unchanged (2). However, participants acknowedged that there were opportunities for deveopment: There wi be a curricuum review in 2009 and this wi give the opportunity to refine and improve specific outcomes. The review wi give us chance to pu back and standardise things across the four countries. There is a need to resource trusts and recognise training and earning time... we need to understand the issues of pacements; whie they shoud be ess about service and more about earning. I do not want doctors to become supernumerary. The next focus wi be post FT; and this wi need to be done fexiby. Regiona stakehoder interview findings The seven regiona PESs in our study were invoved in programmes that offered paces for between 200 and 550 postgraduate doctors. The smaest programme started in 2008 and worked with nine trusts, expecting to expand in the foowing year. The argest programme worked with 24 trusts and 19 community pacements. Regiona impementation of Foundation Programmes A regiona stakehoder participants described a nationa structured programme for F2 (7), with three 4-monthy pacements (2). Each region had its own pattern of committees and iaison groups that managed deanery and pacement arrangements, for exampe there are 16 individua programmes in the oca heath economy for two years. Participants (2) observed that trainees were empoyed by trusts whie continuing in the privieged position of having deanery support to continue their medica education. 14 NIHR Journas Library

37 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Most participants described three 4-month pacements that were either constructed from scratch or were deveoped after some transition arrangements (2). A variety of pacements were described invoving primary care (2) and ED (but not in every rotation, 2). The use of primary care pacements varied from being in every track to descriptions of GP pacements that are poory supported (2). It was noted that the two foundation years were rarey considered concurrenty: there is no couping of F1/F2 imiting choice and fexibiity of the trainee if they want to do a different rotation that fits their career path. One participant noted that we have some FY2s who spend the whoe year abroad. Invovement of pacement organisations in the panning of training Six out of seven participants commented on forma (and informa) meetings between PMDs and trusts regarding the panning of pacements (6). In two cases FSs were based in the trusts and cose working reationships were described. However, this was not true in a cases: There is oca design of the rotation against minimum standards; we visit every two years. We don t hande this at a we; the education community is separate from service deivery distancing the Trusts. Trusts need to be invoved more: we try to isten to them; but this is often swept aside by demands of evauation. Poor transfer of information between departmenta procedures and foundation schoo procedures; such as poor prescribing. Quaity assurance A variety of mechanisms designed by the PMETB (e.g. end-of-pacement trainee survey, annua report) and oca mechanisms (e.g. trust reports, significant events) were described. Annua or biannua forma visits to trusts were carried out by postgraduate medica education representatives. A wide range of data were coected (e.g. induction, rotas, career outputs, job evauation survey too, end-of-pacement survey, serious incident reporting). Most participants fet that these mechanisms were adequate but that itte synthesis occurred with the information, commenting (4): QA [quaity assurance] systems are difficut. Yes, the mechanisms exist but there is a danger of information overoad. If quaity issues are indicated trusts produce an action pan. Postgraduate assessments and supervision Eectronic portfoio (e-portfoio) use was noted by five participants, with three commenting on widespread use and others noting difficuty in impementation needing to make them more user friendy and some trainees being more engaged with computer technoogy than others. Their use as a quaity assurance too aowing the deanery to track activity was wecomed. There were mixed views about the vaue of forma assessments. On the positive side participants fet that the toos were effective [especiay the mini-peer Assessment Too (mini-pat)] but that they needed to be appied effectivey and not used as tick-box exercises (2). Difficuties were noted, such as getting a fu spectrum of procedures and senior cinicians time (2). Trainee strategies were noted, such as eaving a the assessments to the end of a rotation and getting friendy staff member to sign-off competencies (2). Participants recognised the important supervisory roes of the ES and CS (5), noting: Consutants find it difficut to give their time as an ES as it is not recognised in their job pan. (2) Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 15

38 PHASE 1: CONSULTATION EXERCISE AND SCOPING STUDY A good ES knows their trainees we; others have ony one meeting. We rey on CS assessments to know about probems with i heath or progress. The ES is a key roe; we are working on this but it is a poory understood roe. (2) ES roe needs to be cose and specia reationship with a trainee to buid their competency we need time and resources to buid this capacity. The quaity of assessment and supervision varied: e-portfoios are ony used by some and the roe of the ES is poory understood. (3) Some trusts have difficuty providing ESs. Trainees and assessors do not understand the importance of the assessment process. Some areas are more engaged with computer processes than others. Induction programmes were criticised (2) either as acking (not containing sufficient information, typicay about how the hospita worked at night) or for being hed at inappropriate times: best in August and worse in December. Participants were asked what criteria were used to sign off the end of an F2 pacement. Athough participants (6) offered simiar criteria, emphases were different. Criteria isted were teaching attendance eves of at east 70% (2), satisfactory appraisas, mutisource feedback (2), competed e-portfoio of competencies (2), good reports from a pacements, deanery guideines for assessment and end-of-year review. Evauation of we-being of foundation year 2 doctors in foundation training Regiona stakehoders described the forma process of defining a F2 doctor in difficuty (3) when significant probems were noted and an action pan put in pace to manage heath issues ( often going back to what was happening in medica schoo ). Others (2) noted an informa monitoring process (from the ES/CS). A majority of participants (5) fet that there coud be better ways to monitor trainees and check to see if they are reaching their potentia. Regarding the question of who assessed we-being, no cear pattern emerged in the view of the participants. Two participants stated that the roe of identifying issues ay with the CS and reevant medica team, who then needed to inform the ES at the deanery. Other participants fet that identification of we-being issues shoud rest at the deanery eve, either with ESs or with postgraduate managers. There was a major concern regarding the degree to which probems coud be kept hidden from supervisors in genera, as serious issues of we-being known to a trainee s GP or occupationa heath woud not be routiney communicated to supervisors. In terms of motivation the majority of participants beieved that the foundation experience of practising medicine after a ong undergraduate tenure was a motivating factor. Further, the opportunity to be part of a functioning team in the ED, making decisions (2) and being given timey feedback (3), was aso a motivating factor for F2 doctors. When asked which aspects of work F2 doctors strugge with a variety of exampes were given: poor or itte supervision (2); antisocia shift patterns; having to prioritise and make quick decisions (2); ED workoad; not being invoved in decision-making in some supernumerary roes; and work that invoves a high eve of communication and deegation. 16 NIHR Journas Library

39 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 One participant gave an exampe of a oca patient safety group as a forma route by which an issue of F2 doctor we-being was identified, with the group identifying a serious incident reating to a particuar trainee. The majority of stakehoders (5) did not beieve that the impact of F2 doctor we-being on patient care was evauated formay: Not sure if this is considered. Currenty we get indirect reports from staff saying when they are not good with patients. Evauation of foundation training When asked about the extent to which FT was providing doctors who are fit for purpose there was a varied response from participants. Four of the seven participants beieved that F2 was deivering training at the eve required, offering proof of certain competencies and providing more rounded individuas who communicate we, and was better at identifying doctors in difficuty than the previous mode of training. Three participants had concerns about the variation in active supervision and assessment in the workpace (2) and the abiity of F2 to adequatey prepare trainees for speciaty training (2). In terms of the EM context, there was widespread recognition among the participants that the ED was a tough training speciaty for F2 doctors. However, the eements that made it difficut were aso its strengths, requiring doctors to see arge numbers of acutey i patients and make decisions about whether to send them home or admit them to hospita (3). Daunting for trainees, they have to work in a big team which is good, deas with a arge numbers of i patients; and they get a ot of peer support. Training is very good in ED; they are consistenty good at induction and teaching parameters. This was not the type of experience offered by many other speciaties. EDs were seen to have good induction processes for trainees, enabing them to understand what was expected of them (2). The confidence eves of trainees were ow at the beginning of their pacements and they were ikey to be exposed beyond their competence initiay. For the process to work smoothy good eves of senior and midde-grade support (2) were required in departments. Career deveopment of foundation doctors Participants noted that career deveopment in FT was under further deveopment in the regions where they were based (4). One region was investing heaviy in career deveopment, appointing an associate dean for careers, offering mandatory training in career deveopment, having speciaist careers advisors with open appointment times and offering podcasts about different speciaties. Another taked about ad hoc taster weeks for trainees to experience different workpaces. However, most activities (3) were described as being work in progress or under discussion. Some imitations in terms of career choice were acknowedged (3) but it was fet that some F2 doctors needed to have more reaistic aspirations based on their abiities. However, there was a sense that F2 was too ate to begin careers training and that the process shoud begin at medica schoo (2). There was disagreement between participants over whether F2 doctors were being asked to seect speciaist training posts too eary in their careers. One participant fet that F2 doctors were od enough to make good decisions, whereas another fet that decisions to enter a particuar speciaty were being made without sufficient experience in that area. However, there was agreement that careers advice needed to be given at an eary stage (2). Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 17

40 PHASE 1: CONSULTATION EXERCISE AND SCOPING STUDY Future deveopments and changes Participants were asked about any deveopments in FT that they thought woud happen in the future. It was acknowedged that there was a obby for change (2) to put F1 into the medica schoo curricuum and extend FT for a third year. Both were fet to be unnecessary as it was thought that the system needs time to bed down and that there was itte need to change too much as FT wi deveop we over the next 4 5 years (2). However, participants agreed that more (and better-quaity) supervision needs to be provided (2), assessments need to be improved and more work needs to be carried out around career deveopment for F2 doctors. Training ead interview findings The interviews with the eight EDTLs eicited information about the impementation of FT in the ED, incuding the roe of F2 doctors in the ED, changes in training provision, the evauation of the we-being and motivation of F2 doctors, preparation for the deivery of patient care, and assessments. Deivery of foundation training received in the emergency department Participants described the experiences of FT with five to 24 students in their departments. One of the TLs from a department that took a higher number of students (both Foundation Programme and undergraduate) described an increase in numbers that had happened 2 years before: there are too many trainees in the department; it is difficut to provide support for them a. Three of the eight participants described an induction to the ED asting around 3 days. The purpose of the induction was to introduce F2 doctors to the way that the departments were run and their protocos and procedures. One participant described the support avaiabe for induction incuding posters and CDs with earning materias and protocos (additionay on the intranet). Most commony, ED teaching was organised as 1- to 2-hour weeky sessions (4) focusing on specific aspects of ED work such as breaking bad news, deaing with domestic vioence, trauma training and training in specific cinica conditions, sometimes using externa speakers. One TL described a mandatory skis training day to earn specific procedures used in the ED such as intubation and the use of neck drains. Another TL described training that was supported with onine modues tracking teaching sessions. A third TL stated that F2 doctors were encouraged to attend 3-day ife support training courses (adut ife support and advanced trauma ife support). In addition, two participants described generic teaching days (or deanery days) hed twice a month deivering materia from the Foundation curricuum. These sessions were usuay consutant ed and covered areas of cinica governance and audit and were described as rather didactic, and not geared to the ED roe. Another TL described ongoing work in their centre to estabish mandatory training for F2 doctors. Roe in the emergency department and day-to-day working Training eads were asked about the roe of F2 doctors in their EDs. F2 doctors were described as being part of the ED, gaining hands-on experience (2) and working with a wide range of patients (2), and with various degrees of autonomy to make decisions. Activities were described as watching and progressing patients treatment with support and feedback from senior medica staff (2). One TL noted that in the first 4 weeks of an F2 s pacement every patient was seen by a senior doctor unti the F2 s confidence increased. Another TL noted: F2s may not be abe to work to the standard of junior doctors they are ess confident now. A variety of departmenta working arrangements were described by participants. ED work was considered as experientia earning (3) that was supported by discussion with midde grades (24 hours), senior registrars and consutants (2) and assisted with guideines and by reading. Learning on the job was supported in one centre by an aocated CS who hed appraisa meetings with F2 doctors at east twice in their training rotation. One participant described a nurse practitioner-ed service in which F2 doctors assessed the diagnoses made by nurse practitioners to give them experience of minor injuries cases. 18 NIHR Journas Library

41 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 One TL noted that a patients in the ED were seen by a consutant and that there was a service tension between what was done in training and what was done as the job. Changes in the way that the emergency department has provided training in recent years When asked about changes to training participants focused on teaching, supervision, increased workoad, issues caused by the EWTD and the reduction of pacement ength from 6 months to 4 months. Two TLs noted that training (teaching) was no onger geared to ED specificay and that previousy practica procedures had been taught more formay (on a one-to-one basis) foowed by supervision unti the trainee grew in confidence. It was noted that medica schoo training had changed its focus from teaching to probem-based earning; athough this was thought to be better it was acknowedged that there is a greater variation in earning as a resut. Gaps in trainee knowedge were noted (2) resuting in staff having to offer more support and deveopment in the ED than before (e.g. in anatomy and physioogy). One TL commented that some F2 doctors had never been exposed to acutey unwe patients before and that the trainees found this difficut to cope with. Centra to these issues of training was the need to provide additiona senior support to faciitate F2 doctor earning (5). Athough some TLs noted that there were more consutants in the ED now (2), enabing a F2 doctors to have supervisors, others noted that there were not enough senior staff to support their students (2) or that other medica staff and nursing staff were undertaking supervision. For exampe: There is insufficient senior staff for cose supervision; we are not giving what we shoud be giving in terms of education and training. increased strain on the team in ED to support trainees; this increases the stress on senior doctors. Couped with the issue of increased supervision for foundation doctors is the issue of increased workoad (2). This is summarised we by the foowing quote: We are incrediby busy; the workoad just gets higher and we are not broken yet but it is not far off we just need more senior staff. Changes in the working pattern of doctors to shift-based work (associated with the EWTD) were seen to be interfering with access to teaching and training opportunities for F2 doctors (3). For exampe: Some training time is ost; many miss two or three (teaching) sessions as they are off shift or on hoiday or study eave. They ony attend sessions if they are interested and if they miss them this eaves gaps in their knowedge. The TLs thought that career deveopment aso suffered from shift working, with trainees being asked to ook at different work options when they are off duty. Athough it was noted that F2 doctors were on empoyment contracts, it was fet that there was itte reward or monitoring of F2 doctor performance (2): there is no cear system to refect how we trainees work: good trainees get frustrated. One TL noted that the ED had productivity measures (noting the number of patients seen by a foundation doctor) so that staff were aware of the confidence eves of various trainees. There were severa negative comments about the change in rotation ength (from 6 months to 4 months). The shorter rotation was perceived as not being enough time for the trainee to earn how to do the job we (2): They are just becoming competent when they eave. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 19

42 PHASE 1: CONSULTATION EXERCISE AND SCOPING STUDY Lack of confidence in the cinica setting was particuary evident if this was the first F2 rotation. One TL noted that the reduction in rotation ength had required different ways of working with F2 doctors (supervising a of the patient episodes in the F2 doctors first 4 weeks of working, as noted above). Evauation of we-being and motivation There was some divergence in TLs views on how effectivey F2 we-being was evauated in the ED. Some participants fet that we-being was poory evauated (2) as interviews were no onger used in pacement aocation, whereas others beieved that systems such as mutidiscipinary team feedback (mini-pat) or more informa feedback from staff (such as nurses) identified doctors who were strugging or in difficuty (2). However, it was cear that there was no forma system by which foundation doctor webeing and motivation were evauated. Athough this was the case, TLs fet that if there were issues then staff working with the F2 doctor woud notice (3), through observation or by an incident occurring or unfavourabe reports being given in the mutisource feedback. It was fet that senior medica staff or ESs woud become aware of issues of we-being (3) during assessments (such as the mini-pat), when examining e-portfoio progress or in supervisory meetings. If workoad monitoring is used this woud aso offer a strong indicator of poor performance. Motivation The TLs were asked what aspects of ED work the F2 doctors are motivated by and about the areas in which they strugge. The profie of a keen F2 who wants to progress was ceary described (2): they have carried out audit projects, they turn up bright and eary, and they are more fexibe about working onger hours. Motivated F2 doctors usuay see acute medicine as part of their career pan. Less motivated F2 doctors were the ones who did not enjoy working out of hours with reay sick patients (2) or who found it difficut to work with non-medica management cases (such as with oder frai patients). Some ack motivation because of poor cinica knowedge ( this may be a ong-standing probem ) or because their previous experience of ED (as a medica student or an F1) was poor and/or they did not see this rotation as part of their career pan. The potentia impact of poor F2 we-being on patient care was ess considered. However, patient monitoring (e.g. waiting times) and workoad monitoring (2) were thought to be usefu in detecting issues. Is foundation year 2 training adequate preparation for the deivery of patient care? When asked whether F2 training adequatey prepares doctors for service-eve roes, TLs described variation in trainee performance and confidence (4), with some F2 doctors abe to meet the demands paced upon them but others unprepared and acking in confidence. F2 doctors were perceived (in comparison with SHOs) to take onger to become independent fuy functioning doctors, to be more reuctant to take decisions and to work at a sower rate. When asked to evauate the ED experience for F2 doctors there were rather more negative comments (16) than positive (5). From a positive perspective the ED offers probem-based earning, an intense experience where trainees have to think for themseves and take decisions, broad exposure to a range of medica discipines and chaenging shifts demonstrating service deivery ; in addition, senior cover is aways avaiabe. From a negative perspective, rotation patterns were often antisocia (5) and, couped with the EWTD, there are issues of missed teaching and other earning opportunities (2); foundation doctors have had itte prior experience of acutey i patients and are ess used to decision-making in their previous pacements ; and they have had ess hands-on experience with practica aspects of training, reducing their confidence (2). Trainees are not we prepared for service deivery (3), having previousy been encouraged to do their best, without taking into account how ong this makes a patient episode (2). In addition, TLs fet that, as 20 NIHR Journas Library

43 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 it was a busy environment, it was difficut to offer sufficient supervision in the ED (2), which the trainees were needy for. When asked how the training coud be improved, most TLs were mainy in favour of a return to 6-month rotations (5), whereas some coud see that a baance between breadth and depth of experience was needed (3). For exampe: A return to six months is needed, but this may have a negative impact on the breadth of understanding associated with ess service deivery. Participants comments about improvements to training mainy focused on teaching (5), in particuar the incusion of more specific training reated to each pacement (2) and ess generic training content. The change of emphasis to probem-based earning, requiring an active earner, was noted: big change from passive forma earning to active trainee seeking earning. We need a marker for the active earner. However, this approach was criticised as motivation to earn is not as high as in the past; they (trainees) expect to be taught. TLs wanted to offer more supervision (3) but this was fet to be impossibe in a service-ed speciaty. In summary, TLs were aware of the difference in context between the various Foundation pacements, in which many F2 doctors were not required to make decisions and were supernumerary to service deivery; however, in ED, trainees are vita to service deivery and we have no contro over our rotas. Assessments Participants were asked about the contribution of FT assessments to ensuring that doctors provided good-quaity patient care. Overa, haf of the participants (4) fet that the assessments were an improvement in the evauation of the deivery of patient care and haf (4) fet that there were issues that needed to be addressed with the assessments before they woud be of vaue. The strengths of the assessments incuded providing the opportunity for observation of trainees undertaking practica procedures (direct observation of procedures or DOPs), for exampe suturing, and providing the opportunity for trainees and supervisors to discuss eements of care that were worrying a trainee (case-based discussions) and for cinica assessments with direct feedback to the trainee (2). However, it was noted that the benefits were case dependent. The issues imiting the use of assessments were the need for open-ended assessment of experience (and ess ticking boxes or mundane superficia questions) (2) and ack of confidence that assessments (such as mutisource feedback in which the trainee chooses the assessors) refected abiity (2). It was fet that assessments were aso imited in their abiity to identify doctors who were strugging with service deivery. Focus groups with foundation year 2 doctors Focus groups were hed with F2 doctors who were on pacements in four of the EDs participating in this study (25 F2 doctors in tota). The focus groups discussed issues of confidence and competence, anxiety and genera experience in EM. Foundation year 2 doctor competence and confidence Participants were asked what gave them confidence to deiver good patient care. There was strong agreement across the focus groups that positive feedback on their performance from senior coeagues (consutants and registrars affirming that the appropriate cinica process had been foowed ), previous experience of the cinica situation ( previousy seeing how to manage a case ike a Coes fracture ) and apparent patient satisfaction ( patient saying that they fee better after treatment ) gave them confidence in their abiities. Further, participants from three of the four focus groups agreed that good earning experiences, such as earning from near misses, and appropriate teaching, such as acquiring knowedge that was specific and usefu in ED, such as what to do when a patient presents with a headache, increased their confidence. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 21

44 PHASE 1: CONSULTATION EXERCISE AND SCOPING STUDY In addition, participants from two of the four focus groups agreed that knowing that your skis work we, peer support ( checking questions with peers before approaching seniors ), teamworking ( working as part of a team when doing forma assessments ) and acknowedgement by the referra team that a correct referra was made a enhanced their confidence. F2 doctors were asked what things they worried about when they finished their shift. There was agreement that the main worry was about sending a patient home: was I right, shoud I have referred, I worry ess if they are referred because the patient is safe. In addition, participants from two of the focus groups worried about making decisions within the 4-hour target ( compared to other speciaties there is a sma space of time to make a decision whether to admit or discharge a patient ), their own sef confidence ( I am anxious about discharging patients; if I can discuss this with a consutant I am reassured ) and correcty diagnosing the condition ( did I get it right, was my judgement correct ). Improving training for foundation year 2 doctors Participants from a of the focus groups agreed that teaching coud be improved in terms of content ( more cinicay reevant topics rather than heath and safety ) and their avaiabiity ( EWTD cuts down teaching time, shift working does not aow me to compete the minimum teaching requirement ). Participants from three of the four focus groups fet that it was important that there were review sessions ( about patients that were seen on a shift after the event; taking about whether they discharged or not ), opportunities to address the poor work ife baance of working in the ED ( some rotas are awfu, doing a difficut shift every day is very demanding and I woud have to consider the ifestye impications if I took this job on as a career ) and consideration of specific difficuties of working in the ED ( not having enough staff on at peak times ike bonfire night, not having protocos to foow, repetition of the same questions: is the patient going to breach and what drugs are needed ). There were aso varying views on teamworking in the ED. One group fet that the ED experience was improved by the teamworking of doctors and nurses, whereas another group noted that teamworking varied depending on the time of day: teamworking was more ikey out of hours when there were ess seniors around. There were varying views of the appropriate ength of the ED rotation: two focus groups agreed that they were gad of the extra choice that a 4-month rotation gave within their FT and one group (with some participants doing a 6-month rotation) fet that the onger rotation was a good thing and that some trainees had made an active choice to do this ength of rotation, saying there was more opportunity to earn about acute care and it woud be considered a badge of honour competing a six-month rotation in ED. Vaue of assessments Participants from three focus groups gave a baanced view of the positive (3) and negative aspects (3) of the forma assessment process. From a positive perspective participants agreed that forma assessments buid on the type of assessment used in the ast 2 years of their undergraduate training (such as DOPs, which heped them begin to interact with senior medica staff); working as part of a mutidiscipinary team faciitated assessment; and they enjoyed the mini-cinica evauation exercise (mini-cex) as it offered the best feedback on their progress. Negative aspects were that assessments assume that a junior doctors are the same ; the process is fawed as you ony put in the e-portfoio things that you have done we; not where you may have earned more 22 NIHR Journas Library

45 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 from something that you have done bady ; and there seems itte point in doing DOPs as these are done as part of everyday practice. Emergency department environment When F2 doctors were asked whether they enjoyed the ED environment there was a positive yes response (4) because it is unpredictabe work where you need to pick up cues to understand the patient s condition (2); participants fet that it is a good teamworking environment where they did not fee isoated (as they did in some surgica rotations); and there are opportunities to carry out a arge variety of different sorts of work enabing a fast rate of earning: You earn so much by doing things at a fast rate having to make decisions it is great experience for the future. However, these views were baanced by the rubbish working hours. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 23

46

47 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Chapter 4 Longitudina study Aims and objectives Aims To measure eves of (and change in) F2 doctor we-being, motivation, confidence and competence over the period of their F2 training. To measure the impact of pacements in EDs on F2 doctor we-being, motivation, confidence and competence. Objectives To undertake a 12-month ongitudina study to assess F2 doctors experiences of working in terms of their we-being, motivation, professiona identity, confidence and competence. To impement a survey at four time points during the ongitudina study (at the end of the F1 period and then after each of the F2 pacements). One of these F2 pacements wi be in the ED and the impact of this pacement on F2 doctors experiences of working wi be assessed. Methods Ethica and governance arrangements Ethica approva for the phase 2 ongitudina study was received in September 2009 (ref.: 09/H1300/80). Approvas from non-nhs organisations and research governance approvas from participating NHS trusts were obtained between November 2009 and June Survey design We used a cosed onine survey design because in the phase 1 focus groups F2 doctors expressed a preference for onine surveys rather than paper versions. F2 doctors were eigibe for this study if they had a pacement in the ED during their F2 rotations. The onine survey was accessibe via a porta on the EDiT study website, 48 with eigibe doctors sent a ink taking them to the appropriate part of the website to access the survey (see Appendix 13 for a screenshot of the website). The EDiT website provided information about the study, such as news and updates, and aso incuded an interactive eement (a medica casebook quiz). The website was designed to be informative and attractive to potentia participants and thus increase recruitment and retention over the period of the survey. Visitors to the website coud examine the information provided without being obiged to compete the survey itsef. Participants As the focus of the study was examining the impact of pacements in the ED on the FT experience of F2 doctors, the samping frame (eigibe doctors) for the survey was a F2 doctors in Engand who had a pacement in the ED as part of their F2 training year. Eigibe doctors were identified foowing discussions with PMDs and EDs. Sampe identification In the first instance, (August September 2008) we identified a type 1 EDs (defined as consutant-ed 24-hour service with fu resuscitation faciities and designated accommodation for the reception of emergency patients) in Engand from the Department of Heath website. 49 We identified and contacted 176 NHS trusts in Engand with type 1 EDs for expressions of interest. After our approach, 45 trusts Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 25

48 LONGITUDINAL STUDY responded and expressed an initia interest in taking part in the study. Further contact took pace with ED eads after the initia approach and more detaied discussions about the requirements of the study were hed. Finay, 28 trusts with 30 EDs agreed to participate as the study sites. Study contacts, incuding ead consutants, foundation consutant eads and research nurses, were identified in the 30 participating EDs. Deaneries and foundation schoos A 14 PMDs were contacted (December 2008 January 2009) to identify expressions of interest in participation. The postgraduate medica dean and the foundation schoo director (FSD) were contacted in the first instance. Detais about the study were provided and agreement to participate in the study was sought from each deanery. Agreement to participate was obtained from nine deaneries. The FSD from each of these nine deaneries was asked to identify key contacts within the schoos (foundation schoo administrators; FSAs) to assist the study team with the identification of F2 doctors who woud have a pacement in the ED within their F2 training ( ). EDs of NHS trusts were incuded in the study ony when the deanery aso agreed to participate. Eigibe participants After recruitment of participating EDs and deaneries, our tota eigibe F2 doctor sampe consisted of 654 F2 doctors, training between August 2010 and August Each of these F2 doctors had a pacement in one of the 30 EDs participating in the study. Participant contact For data protection reasons and reasons stipuated by the approving ethics committee, we coud not be given the names/addresses of the eigibe F2 doctors by participating deaneries and FSs; instead, approaches to the doctors were made by study contacts in FSs and EDs on behaf of the study team. The initia approach to inform the eigibe F2 doctors of the study was made by the reevant FSAs. An e-mai with an attached etter of invitation and participant information eafet (see Appendices 14 and 15 respectivey) was sent to the F2 doctors before the start of the study (May 2010). The e-mai notified the doctors that they coud opt out of the study at any stage and receive no further contact regarding the study. Recruitment and consenting In Juy 2010 a further e-mai was sent by FSAs to those eigibe F2 doctors who had not opted out after the initia approach. This e-mai provided a ink to and study password for the onine survey. After accessing the survey, potentia participants were required to enter their e-mai address and the study password to proceed any further (ensuring that ony eigibe F2 doctors competed the survey). Participants were asked to enter an e-mai address that woud be current throughout the study time period to enabe the accurate matching of participants responses at further time points. When participants entered the correct password an onine survey consent form was generated (see Appendix 16). Participants were unabe to proceed with the survey unti they had competed the consent form (foowing consent, F2 doctors were abe to access the survey proper with their e-mai address generating a unique study ID that woud identify them throughout the study). Foowing the initia recruitment e-mai in Juy, the sampe of F2 doctors (with the exception of consenting participants or those who opted out) were contacted at two further time points (August and September 2010) by FSAs as a reminder to participate in the study. If F2 doctors had consented to take part at the first time point (T1) they were then contacted at subsequent time points directy by the study team. Participants coud enter the study at T2 if they wished (and were coded accordingy) but not at ater time points. Deveopment of survey measures In the first instance a piot questionnaire was designed to measure a range of work-reated outcomes and job-reated characteristics by adapting we-vaidated scaes. The questionnaire was pioted with a sma 26 NIHR Journas Library

49 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 sampe of F2 doctors and foowing feedback some minor amendments were made to the fina questionnaire (see Piot study of questionnaire). The piot questionnaire and the fina questionnaire are provided in Appendices 17 and 18 respectivey. The content of the fina questionnaire is detaied in the foowing sections. Background demographic information Baseine information was coected on sampe age, sex, pace of quaification, year of quaification, ethnicity and description of Foundation pacements (e.g. speciaty undertaken, trust). Detais of the pacements (speciaty and trust) were cross-checked with equivaent pacement information from the nine participating deaneries to ensure accuracy of reported information. Individua characteristics Baseine background information was coected on personaity and coping characteristics using vaidated scaes: Personaity. Afive-factor structure represents the most universa description of the dimensions of personaity 50 and from this the dimension of conscientiousness (feeing capabe) is an important personaity trait for F2 doctors to deveop during their FT. Meta-anaytic studies suggest that conscientiousness is a vaid predictor of job performance. 51 The scae consisted of eight items (e.g. Woud you describe yoursef as typicay organised? ) rated by the participant from 1 ( extremey inaccurate ) to9( extremey accurate ). Coping. A coping scae consisted of 16 items (e.g. When faced with a stressfu situation I try to figure out how to resove the probem ). This 16-item scae seeks to identify the strategies that individuas use to cope with difficut, upsetting soutions. Deveoped from the work of Tobin et a., 52 it examines a hierarchica structure of coping strategies stemming from probem-focused and emotiona-focused higher-order categories of strategies. 53 The scae divides into four positive strategies: persona proactivity (PPRO) items a, e, i, m seeking support (SSUP) items c, g, k, o not seeking revenge (NREV) items d, h,, p not keeping things to sef (NSLF) items b, f, j, n. Work-reated outcomes We were interested in how confidence and competence deveoped over the course of F2 training and this was measured using three scaes. Two of the scaes were adapted from a previous study. 35 The scae regarding confidence in decision-making was deveoped after discussions during the phase 1 focus groups with F2 doctors about infuences on we-being. Confidence in managing conditions. Participants were asked how they fet about managing 23 common acute medica conditions (such as edery fa, chest pain, stroke, overdose and back pain). Participants scored their confidence on a nine-point scae from 1 ( owest eve of confidence ) to9 ( highest eve of confidence ). The 23 conditions were seected by an ED consutant and a consutant in acute medicine who were members of the project steering group. The conditions were seected as a comprehensive range of acute presentations that F2 doctors were expected to manage independenty and competenty during the course of their F2 training. Box 1 detais the conditions incuded. Experience in performing practica techniques. This scae aimed to measure experience in performing five common practica medica techniques. The five procedures were defibriation, arteria bood gas anaysis, suturing, eectrocardiogram (ECG) interpretation and radiograph interpretation. Participants were asked how experienced they were in performing the techniques and they scored themseves from 1( no/itte experience ) to9( confident in performing aone ). The procedures were again seected by our steering group consutants as representing a comprehensive range of procedures that F2 doctors Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 27

50 LONGITUDINAL STUDY BOX 1 List of 23 acute conditions for assessment of F2 doctors confidence Diarrhoea and vomiting. Shortness of breath. Coapse unknown cause. Acute menta heath probem. Edery fa. Chest pain. Back pain. Cardiac arrest. Papitations. Abdomina pain. Acute painfu joint. Recta beeding. Acute aergic reaction. Left side pain. Acute stroke. Overdose paracetamo. Diabetic ketoacidosis. Acute confusion. Headache. Seizure. Ceuitis. Haematemesis. Rash. were expected to carry out independenty and competenty during the course of their F2 training and as part of their training curricuum. Confidence in decision-making. Confidence in decision-making was one of the key issues raised by F2 trainees in our phase 1 focus groups. As there were no vaidated scaes in this area, the study team deveoped a scae to examine the abiity of trainees to make appropriate decisions. This consisted of three statements on eements of decision-making (e.g. Thinking about the decision and judgements I made during the pacement, I am confident I made the appropriate decisions ). Participants scored their agreement with the statements from 1 ( strongy disagree ) to5( strongy agree ). We-being and motivation We were interested in eves of and change in F2 doctor we-being and motivation during F2 and specificay the impact of their pacement in the ED on we-being and motivation. Four we-estabished previousy vaidated scaes were used: Anxiety and depression. These scaes are derived directy from origina measures of two dimensions of job-reated we-being, from anxiety to contentment and from depression to enthusiasm. 54,55 The third dimension of we-being (from dispeasure to peasure ) is measured by the job satisfaction scae. These scaes are used in preference to more genera notions of we-being, for exampe the Genera Heath Questionnaire, 30 as they are specific to the work context, which may change over time with F2 doctor pacements. These scaes have been found to be sensitive for predicting absence, 40 job demands 56 and eader behaviour. 57 These scaes use six items, three reated to anxiety (e.g. In the ast month of your pacement, how much of the time did your roe make you fee worried ) and three to depression ( In the ast month of your pacement, how much of the time did your roe make you fee 28 NIHR Journas Library

51 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 miserabe ). There are five possibe responses ranging from not at a to a great dea. Responses are averaged across items with higher scores representing greater job-reated anxiety or depression. Job satisfaction. The questionnaire incorporated a 15-item job satisfaction scae. 46 The scae expores overa job satisfaction by combining subscaes of intrinsic satisfaction (affective reactions to job features that are integra to the work itsef) and extrinsic satisfaction (affective reactions to job features externa to work). Items expored satisfaction with areas such as physica working conditions, freedom to choose own method of work, recognition for good work, cinica supervision and chances of career progression. The scae has been found to be a vaid and reiabe measure of job satisfaction and there is a arge body of comparative data avaiabe (e.g. Muarkey et a., 58 Stride et a. 59 ). The wording of severa of the origina items was amended to make the scae more appropriate to the FT context. The 15-item job satisfaction scae has seven responses ranging from 1 ( extremey dissatisfied ) to7 ( extremey satisfied ). An exampe item reating to intrinsic satisfaction is How satisfied are you with the amount of responsibiity you are given? An exampe item reating to extrinsic satisfaction is How satisfied were you with your rate of pay? Scae scores are derived by averaging item scores, with higher vaues representing greater satisfaction. A copy of the scae can be found in Appendix 18. Motivation. In this study we wanted to expore what makes a motivating work environment for F2 doctors. Working with expectancy theory 60 we assume that peope have certain amounts of energy, which is used to satisfy our needs (such as achievement, safety), and motivation is the process which determines how that energy is used to satisfy needs. Motivation can be considered both as effort (the amount of time or energy put into work) and as direction (specific tasks that energy is appied to). One scae was incuded in the questionnaire. This scae focused on the effort that the participant woud expend on his or her work pacement. The scae incuded three items (e.g. How woud you rate the amount of effort you put into your job? ). Participants scored their effort from 1 ( owest effort ) to5 ( greatest amount of effort ). Intention to quit. This refers to the individua s intention to eave his or her roe and was measured using three items 61 incuding It is very ikey that I wi activey ook for a new job in the next year. These items have previousy been used with medica staff 62 and were adapted to the FT context (e.g. Thinking of your career in medicine so far, how true is the foowing statement: it is very ikey that I wi activey ook for a new job outside the medica profession in the next year? ). Participants scored their agreement with the items from 1 ( strongy disagree ) to5( strongy agree ). Teaching and training Teaching and training are important eements of a trainee doctors work pacement and we incuded two scaes reated to teaching and training received and the knowedge acquired from the work pacement. The two scaes were deveoped after key issues were raised during our phase 1 focus groups about infuences on we-being. Impact of teaching and training on management of conditions. This incuded three items that measured how the management of medica conditions may have been improved during the work pacement. Participants were asked if their management of the 23 acute conditions (see Box 1) woud have been improved with cearer guideines, better teaching or more supervision, with each item scored from 1 ( strongy disagree ) to5( strongy agree ). Deveopment of professiona knowedge. This examined the professiona knowedge acquired over the course of F2 training. It was measured using three items (e.g. Do you fee more abe to work as part of a cinica team? ) and participants scored agreement from 1 ( not at a ) to5( a great dea ). Roe characteristics This section incuded three we-vaidated scaes that examined the characteristics of the foundation pacement roes. Work demands. This scae was designed to measure the extent to which F2 doctors fee that they have the time and resources to carry out their job propery in their F2 pacements. Studies have shown that time pressures and workoad are major infuences of we-being for heath-care workers Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 29

52 LONGITUDINAL STUDY (e.g. Borri et a., 63 Borri et a., 64 Hipwe et a., 65 Richardsen and Burke 66 ). For exampe, a ongitudina study of heath-care workers 63,64 demonstrated medium to strong negative reationships between work demands and job satisfaction (r = 0.30). Based on a measure of subjective quantitative workoad, 67 the scae draws on the findings of Kahn et a. 68 This scae consists of six items with five responses ranging from 1 ( not at a ) to5( a great dea ). An exampe of an item from this scae is I coud not meet a the conficting demands made on my time at work. Task feedback. This scae expores the F2 doctors understanding of their own work performance and their coeagues perceptions of feedback. This is particuary reevant in the heath-care context as there is a strong degree of interdependence with coeagues in the profession. Feedback is a core job dimension defining the saient properties of work 69 and infuencing outcomes such as work motivation, performance, satisfaction and attendance behaviour Feedback from various sources eads to knowedge of individua performance, roe carity and appropriateness of work behaviour. This measure was based on a subscae of the Job Diagnostic Survey: feedback from the job itsef. 71 The scae has four items with five responses ranging from 1 ( strongy disagree ) to5( strongy agree ). Two of these items refer to the individua s understanding of his or her own work performance, for exampe I usuay know whether or not my work is satisfactory in this job. Two further items expore the understanding of others perceptions of feedback, for exampe Most peope on this job have troube figuring out whether they are doing a good or a bad job. Roe carity. This referred to how cear the individua fet in his or her roe in terms of what was expected of him or her. This was measured using five items (e.g. I had cear, panned goas and objectives for my job ), with agreement scored from 1 ( not at a ) to5( a great dea ). A ack of understanding of the tasks that form a part of the individua job roe is ikey to be stressfu. 72 Issues of roe carity are pertinent to the we-being of F2 doctors as their roes are often i-defined and encompass a wide range of behaviours. Severa arge-scae studies of heath-care staff have demonstrated that roe carity is reated to job satisfaction. 63,64,73 For exampe, Borri et a. 63 found strong positive reationships between roe carity and job satisfaction. These reationships were confirmed in a subsequent survey. 64 Reationa characteristics Socia support Two dimensions of socia support have been described: the dominant characteristic of emotiona/ psychoogica support, which invoves istening, caring, approva and sympathy, and instrumenta support, which invoves active behaviour. 74 However, socia support is a compex phenomenon and may vary with different types of work. 75,76 Socia support at work is provided by different peope such as co-workers (peers) or coeagues, supervisors or managers and team members. In addition, there are non-organisationa sources of support such as spouses, friends or reatives. A arge-scae survey of NHS staff 63 found strong positive reationships between socia support and we-being, which was vaidated by a simiar observation 2 years ater. 64 In this study we examine two forms of support saient to F2 doctors: Supervisor support. This referred to the avaiabiity of advice and support from supervisors and was measured by six items (e.g. To what extent did your CS encourage you to give your best effort? ). Extent of support was scored from 1 ( to a very itte extent ) to5( to a very great extent ). Coeague support. This referred to the avaiabiity of advice and support from coeagues and was measured by four items (e.g. To what extent were you abe to count on your coeagues to hep you with a difficut task at work? ). Extent of support was scored from 1 ( to a very itte extent ) to5( to a very great extent ). The first item examines emotiona support, 74,77,78 asking the recipient about the extent to which coeagues isten to you when you need to tak about probems at work. The other three items focus on instrumenta or tangibe socia support. 74,77,79 These ask the recipient the extent to which coeagues back you up, hep with a difficut task and hep in a crisis situation. 30 NIHR Journas Library

53 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Additiona job-reated characteristics. Further job characteristic scaes were derived from other potentia determinants of we-being identified during our focus groups discussions with F2 doctors in phase 1. These were extent of teaching and training, hours of work in a working week and extent (hours) of contact with CSs. Teaching and training received. This was measured by three items [e.g. Thinking of your F2 roes did you have any forma teaching sessions arranged in your department (in addition to the externa FT)? ]. Responses were rated from 1 ( none ) to5( agreat dea ). Hours of work. Two questions addressed the issue of the number of hours of work that F2 doctors undertook in their different pacements. The first question, In a typica working week in your roe how many hours did you work?, had an open response to record number of hours. The second question, In the ast four weeks of your Foundation Roe did you have a shift changed at short notice?, had a dichotomous ( yes or no ) response. CS contact hours. Two questions examined hours of contact with the CS and had open responses to record number of hours. ( In a typica week how many hours of one-to-one contact did you have with your CS? and In a typica week how many hours of cose working contact, e.g. ward round managing a patient, did you have with your CS? ). Survey free-text comments The survey incuded a space for further comments, aowing participants to offer further commentary. Participants were abe to highight issues reated to their working experiences that were not covered by our survey measures. They were aso abe to expand on eements in the survey measures that were of particuar pertinence to them (a fu description and resuts from this section are provided in Appendix 20). Sampe size We panned to incude a sampe size of 210 F2 doctors from across EDs in Engand. We hypothesised that the job-reated we-being of F2 doctors, measured using the six-item anxiety contentment and six-item depression enthusiasm dimensions of the Warr 54 job-reated we-being scae woud change over the 12-month study period. Assuming a correation of r = 0.4 between the baseine and 12-month test scores, a sampe size of 210 cases was required to detect a 0.2-unit change in anxiety contentment at the p < 0.05 eve of significance with 95% power. We envisaged an average of 24 F2 doctors in each participating ED during the 12-month period of the study. We assumed a 20% non-response rate within each ED and a further 20% oss of paired cases between pre- and post-pacement surveys across the sampe, giving us a fina required sampe of around 14 F2 doctors per ED. To satisfy the power anaysis above we aimed to recruit doctors from at east 15 EDs. Piot study of the questionnaire We conducted a piot study in June 2010, primariy to test the reiabiity of the survey measures, particuary those deveoped by the research team, but aso to examine: the usabiity of the questionnaire (e.g. issues of ength) the feasibiity of using FSAs to forward e-mais to F2 doctors on our behaf, with inks to the onine survey the functionaity of the onine web-based patform for the questionnaire. Participants Three deaneries took part in the piot study. These deaneries were seected on the basis that they had aready identified a FSA who was abe to forward the e-mais to F2 doctors, as proposed in the main study. The three deaneries contacted a tota of 57 doctors in seven EDs. The doctors were in their fina F2 pacement and therefore were not eigibe to be part of the main study sampe as they were eaving FT before the start of the study proper. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 31

54 LONGITUDINAL STUDY Anaysis We conducted a reiabiity anaysis for a of the scae measures (sets of attitudina items designed to measure a singe construct) incuded in the questionnaire, using Cronbach s apha statistic to determine the interna consistency of each scae. We aso assessed the item frequencies to check that a spread of responses was being coected (i.e. the fu range of experiences was identified by each item) and the response rate for each item to check that a were understandabe and reevant. Resuts Reiabiity The reiabiity scores (Cronbach s apha) for the majority of the scaes were > 0.7 (denoting acceptabe reiabiity) (Tabe 2). Two motivation measures scored ower for reiabiity and were omitted from the fina survey. These were motivation in terms of examining the direction (or prioritisation) of participants effort in the roe, measured by three items (e.g. I divide my time across tasks in the way that is most hepfu to the organisation ), and motivation in terms of the eve of reward that participants received for their effort, aso measured by three items (e.g. Working hard on this job is not rewarded ). For these two motivation scaes participants rated agreement from 1 ( strongy disagree ) to 5( strongy agree ). Confidence in decision-making was found to have a ower scae reiabiity, but anaysis showed that two items had good eves of reiabiity ( I am confident that I made the appropriate decisions and Considering the information avaiabe to me I made the best decisions possibe ). A third item ( I had sufficient time to make the number of decisions expected of me ) was introduced to the fina survey scae to maintain it as a three-item scae. A other scaes and items were retained for the main study. Usabiity The system for recruiting the F2 doctors worked we. E-mais were sent to them by the designated FSA in each deanery. This proved acceptabe to both the deaneries and the doctors. TABLE 2 Piot reiabiity anaysis Measure Number of items Reiabiity score (Cronbach s apha) Job satisfaction Depression Anxiety Intention to quit Motivation (effort) Motivation (direction of effort) Motivation (need satisfaction) Personaity (conscientiousness) Coping (persona proactivity) Coping (seeking support) Coping (not seeking revenge) Coping (not keeping things to sef) Confidence in decision-making Teaching/training received Deveopment of professiona knowedge or earning NIHR Journas Library

55 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Twenty-four F2 doctors ogged onto the survey via the website patform and did not report any probems with the usabiity/functionaity of the questionnaire. A tota of 21 F2 doctors competed the survey piot and were incuded in the anaysis. Three doctors did not compete any part of the questionnaire. These doctors were subsequenty e-maied to ask why they had not competed the questionnaire but they did not repy to the e-mai. Survey impementation Foowing the piot study the fina questionnaire was confirmed and administered identicay at the four time points (T1 T4), with the exception of background information, which was coected at T1, and the deveopment of professiona knowedge scae, which was added at T4. The survey utiised survey-design software (SurveyGizmo, Bouder, CO, USA) to enabe onine usage. The survey measures were grouped together in four sections, with the demographic data coected ony the first time that a participant received the survey. The initia survey consisted of 31 items presented over eight pages. A items presented on one page had to be competed to proceed to the next page of the survey. A back button aowed participants to go back through the survey and amend answers if they wished to do so. Cohort samping points We aimed to administer the survey measures at up to four time points (T1 T4), capturing F2 doctors working experiences in four Foundation Pacements, the fina F1 pacement and the subsequent three F2 pacements (Tabe 3). Doctors were recruited initiay at T1, athough a sma number did not join the study and respond to the survey unti T2. Anaysis Using ongitudina mutieve modeing, the pattern of change in mean sampe scores on each of our survey measures was assessed over the duration of the study. After the initia measurement of mean scores at the endoff1(seetabe 3), variation in mean sampe scores on the survey measures, by time of pacement in the ED, was aso assessed by categorising each member of the sampe as beonging to one of three groups, depending on whether they had competed their second-year pacement in the ED as their first, second or fina F2 pacement (Tabe 4). Grouping the sampe in this way aowed the measurement of the impact that a pacement in the ED had on doctors roe characteristics and work-reated outcomes. TABLE 3 Survey time points Survey time point Pacement captured Date of survey administration T1 Fina F1 pacement (Apri August 2010) Juy September 2010 T2 First F2 pacement (August December 2010) December 2010 February 2011 T3 Second F2 pacement (December 2010 Apri 2011) Apri June 2011 T4 Fina F2 pacement (Apri August 2011) August September 2011 TABLE 4 Emergency department groups for anaysis of the impact of pacement in the ED on study outcomes F1 F2 ED group T1 T2 T3 T4 1 Pacement in the ED 2 Pacement in the ED 3 Pacement in the ED Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 33

56 LONGITUDINAL STUDY Resuts Response rate Overa, 217/654 eigibe F2 doctors competed/partiay competed the questionnaire for at east one of the four time points, a response rate of 33.2%. Tabe 5 detais the number (%) responding at each of the four time points. In tota, 87 (40.1%) doctors competed the survey at a four time points, 26 (12.0%) competed the survey at three time points, 56 (25.8%) competed the survey at two time points and 48 (22.1%) competed the survey at one time point. Sampe characteristics The mean age of our study doctors was 27 years, 58.2% were femae and they had been quaified as a doctor for a mean of 1.5 years. A tota of 68.7% were white British, 8.5% were back, 10.9% were South Asian, 6.0% were East Asian and 6.0% were of mixed race. UK-quaified (at undergraduate eve) F2 doctors made up 83% of the sampe. Response by trust Detais of the number of F2 doctors recruited from each participating deanery and ED are provided in Appendix 19 (see Tabe 21). Emergency department rotation groups As detaied in the anaysis section, we cassified participants into three ED pacement groups depending on the timing of their F2 pacement in the ED (whether carried out as rotation one, two or three) (Tabe 6). In tota, 26 participants (12.0%) aso reported competing a pacement in the ED as part of their F1 training. We anaysed this subgroup to see whether they differed in any way from those without previous F1 ED experience. There were no statisticay significant differences between the three ED groups in terms of age, sex, ength of time quaified as a doctor and scores on either the trait personaity measures or the coping strategy measures (see Appendix 19, Tabes 22 and 23). TABLE 5 Number (%) of respondents competing the questionnaire at each time point (n=217) T1 T2 T3 T4 No. (%) responding 188 (86.6) 154 (71.0) 135 (62.2) 108 (49.8) No. (%) of non-responders 29 (13.4) 63 (29.0) 82 (37.8) 109 (50.2) Tota TABLE 6 Time point at which respondents competed their second-year pacement in the ED ED group (time of ED training) n % 1 (August December 2010) (December 2010 Apri 2011) (Apri August 2011) Tota NIHR Journas Library

57 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Work-reated outcomes Confidence in managing common conditions We measured eves of and change in mean sampe scores for: confidence in managing individua acute common conditions overa confidence in managing acute common conditions (cacuated across the 23 conditions). Confidence was measured from 1 (owest eve of confidence) to 9 (highest eve of confidence). Detais of the 23 conditions are provided in Box 1. Mean confidence scores for each of the 23 acute conditions varied significanty across the four time points, with a statisticay significant increase in confidence for managing each condition between T1 and T4 (see Appendix 19, Tabe 24). Mean overa confidence (across the 23 conditions) varied significanty across the four time points (F 3,309 = 86.0, p < 0.005), with an increase in mean overa confidence found over the course of F2 training (Tabe 7). Box 2 provides a quote from a free-text comment made by a F2 doctor on the questionnaire. Impact of pacement in the emergency department Participants were divided into three groups depending on the rotation in which they had competed their second-year pacement in the ED (see Tabe 4). For 14 of the 23 conditions (shortness of breath, coapse unknown cause, acute menta heath probem, edery fa, chest pain, back pain, abdomina pain, acute aergic reaction, acute stroke, overdose paracetamo, diabetic ketoacidosis, acute confusion, seizure and recta beeding) there was a statisticay TABLE 7 Overa confidence over time in managing patients with different inesses/conditions Time point of response ED group Mean SD Mean SD Mean SD Mean SD SD, standard deviation. BOX 2 Quote from a free-text comment made on the questionnaire I reay enjoyed this year; it was hard work but it heped me move on from stress of a the practica things to earn the bigger picture of patient-centred care and being part of a team. T4 participant Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 35

58 LONGITUDINAL STUDY significant interaction between change in confidence and group (i.e. time of pacement in the ED), with the biggest increase in mean confidence for each group of doctors associated with the pacement in the ED compared with other pacements (see Appendix 19, Tabe 24 for mean scores over time by condition and ED group). There was a significant interaction between change in confidence and group (i.e. time of pacement in the ED) when predicting mean overa confidence across the 23 conditions (F 6,309 = 9.3, p < 0.005), with the biggest increase in confidence for each group of doctors associated with their pacement in the ED compared with other pacements (Figure 1). Box 3 provides some quotes from the free-text comments made by F2 doctors on the questionnaires. Experience in performing practica procedures We assessed the eves of and change in mean sampe scores for: experience of doctors in performing five common practica procedures (defibriation, arteria bood gas anaysis, suturing, ECG interpretation and radiograph interpretation) overa experience across the five common practica procedures. Mean score overa confidence in managing patients with different inesses/conditions Time point at which you are carrying out your F2 EM pacement Rotation 1 Rotation 2 Rotation Time point of response 4 FIGURE 1 Interaction between time and time of ED pacement and mean overa confidence in managing patients with different conditions. BOX 3 Quotes from free-text comments made on the questionnaires A&E was a very good cinica experience, and I am gad that I had the opportunity to undertake this pacement. T4 participant ED is an exceent training ground for F2s for any future fied, given the sheer voume of patients, good support and reguar teaching. T4 participant 36 NIHR Journas Library

59 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 There was a statisticay significant increase in reported mean experience over time for defibriation (F 3,344 = 23.3, p < 0.005), suturing (F 3,323 = 61.4, p < 0.005), ECG interpretation (F 3,315 = 27.2, p < 0.005) and radiograph interpretation (F 3,362 = 19.1, p < 0.005). Mean scores across the four time points for overa confidence are shown in Tabe 8. The coefficient apha for mean overa experience in performing practica procedures ranged from 0.60 (T4) to 0.80 (T3). Mean overa experience (across the five different practica procedures) varied significanty across the four time points (F 3,367 = 61.1, p < 0.005), with a statisticay significant increase in experience over the course of F2. Impact of pacement in the emergency department There was a statisticay significant interaction between change over time and group (i.e. time of pacement in ED) when predicting the improvement in mean experience scores for arteria bood gas anaysis, suturing, ECG interpretation and radiograph interpretation. For each of these techniques the biggest increase in mean experience was associated with the pacement in the ED compared with other pacements (see Appendix 19, Tabe 25 for mean scores over time for each procedure by ED group). There was a significant interaction between change over time and group (i.e. time of pacement in the ED) when predicting mean overa experience across the five practica procedures (F 6,367 = 10.3, p < 0.005), with the biggest increase in perceived experience reported by each group of doctors associated with their pacement in the ED compared with other pacements (Figure 2). TABLE 8 Overa experience over time in managing patients with different inesses/conditions Time point of response ED group Mean SD Mean SD Mean SD Mean SD Mean score experience in performing practica procedures Time point at which you are carrying out your F2 EM pacement Rotation 1 Rotation 2 Rotation Time point of response 4 FIGURE 2 Interaction between time and time of ED pacement and mean overa experience in performing practica procedures. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 37

60 LONGITUDINAL STUDY There were many positive free-text comments made on the questionnaires about the earning experience of working in the ED (Box 4). We-being Anxiety and depression Leves of and change in mean sampe scores for anxiety and depression were cacuated. Scores for each ranged from 1 (owest eve) to 5 (highest eve). The coefficient apha for anxiety ranged from 0.85 (T4) to 0.91 (T2) and for depression ranged from 0.87 (T1) to 0.93 (T3). There were no statisticay significant variations in mean sampe scores for either anxiety or depression across the four time points. The mean scores for anxiety and depression over time are shown in Appendix 19 (see Tabe 26). Impact of pacement in the emergency department There was a significant interaction between time of pacement in the ED and mean anxiety (F 6,390 =7.9, p < 0.005) with a three groups of doctors showing the biggest rise in mean anxiety during their pacement in the ED (Figure 3). There was no difference BOX 4 Quotes from free-text comments made on the questionnaires I earned an incredibe amount cinicay in knowedge and practica skis. T2 participant Experience was rewarding as I gained skis and experiences that I coud not have got esewhere. T2 participant Going forward, I fee more confident cinicay for the experience. T3 participant A&E was a very good cinica experience, and I am gad that I had the opportunity to undertake this pacement. T4 participant 5 Mean score anxiety Time point at which you are carrying out your F2 EM pacement Rotation 1 Rotation 2 Rotation Time point of response 4 FIGURE 3 Interaction between time and time of ED pacement and anxiety. 38 NIHR Journas Library

61 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 between ED pacements and other pacements for scores on depression. Mean scores for each ED group by time point can be found in Appendix 19 (see Tabe 26). Severa free-text comments were made about the nature of the ED experience and the impact on participants we-being, with exampes provided in Box 5. Comparison with other occupationa groups We compared F2 doctors mean eves of anxiety [mean 2.10, standard deviation (SD) 0.61] and depression (mean 1.68, SD.59) at each of the time points with those of a comparison group of 2909 professiona and technica workers (mean anxiety 2.4, SD 0.83; mean depression 1.88, SD 0.87). F2 doctors reported significanty fewer anxious and depressive symptoms than the comparison group at T1, T3 and T4 and simiar eves of symptoms as the comparison group at T2. Job satisfaction Leves of and change in mean sampe scores for job satisfaction were cacuated. As we as an overa mean job satisfaction score, the 15-item overa score was divided into two separate scae mean scores for intrinsic job satisfaction and extrinsic job satisfaction. A items on the job satisfaction scae were scored from 1 (owest eve) to 7 (highest eve). The coefficient apha for mean overa job satisfaction ranged from 0.86 (T1) to 0.90 (T2), which compares favouraby with the reiabiity of 0.87 obtained in a study of 20,694 NHS trust empoyees. 59 The coefficient apha for mean intrinsic job satisfaction ranged from 0.80 (T1) to 0.86 (T2) and for mean extrinsic job satisfaction ranged from 0.72 (T1) to 0.78 (T2). A measures of mean job satisfaction increased over the time period of the study; this was significant for overa job satisfaction (F 3,397 = 7.2, p < 0.005) and intrinsic job satisfaction (F 4,410 = 9.5, p < 0.05). Mean scores for job satisfaction at each time point are shown in Appendix 19 (see Tabe 27). Impact of pacement in the emergency department There was no statisticay significant interaction between time of pacement in the ED and mean sampe scores for overa job satisfaction or intrinsic job satisfaction. There was a statisticay significant interaction between time of pacement in the ED and mean extrinsic job satisfaction score (F 6,380 = 3.72, p < 0.005), with two of the three ED groups reporting a decine in mean extrinsic job satisfaction immediatey after their pacement in the ED (Figure 4). Mean job satisfaction scores for each ED group at each time point are shown in Appendix 19 (see Tabe 27). BOX 5 Quotes from free-text comments made on the questionnaires ED was stressfu at the beginning. A&E is a stressfu job but the most rewarding I have done so far. A&E is stressfu and aways very busy. T2 participant T3 participant T3 participant Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 39

62 LONGITUDINAL STUDY Mean score extrinsic job satisfaction Time point at which you are carrying out your F2 EM pacement Rotation 1 Rotation 2 Rotation Time point of response 4 FIGURE 4 Interaction between time and time of pacement in the ED and mean extrinsic job satisfaction. Severa free-text comments were made about eves of satisfaction with working in the ED, with exampes given in Box 6. However, participants were not happy with the shifts and working hours (extrinsic aspects of the roe) (Box 7). Comparison with other occupationa groups We compared F2 doctors mean eves of job satisfaction (overa mean 5.13, SD 0.55) at each of the time points with those of a comparison group of 2616 doctors and NHS staff (mean 4.55, SD 0.80). F2 doctors were significanty more satisfied than the comparison group of NHS staff at a time points. Motivation (effort) Leves of and change in mean sampe scores for motivation were cacuated, with scores ranging from 1 (owest eve) to 5 (highest eve). The coefficient apha for motivation ranged from 0.83 (T1) to 0.91 (T3). There was no statisticay significant variation in mean motivation scores over the four time points of the survey (see Appendix 19, Tabe 28). BOX 6 Quotes from free-text comments made on the questionnaires Thoroughy enjoyed. T2 participant I woud thoroughy recommend this job to any FY2. Athough very demanding it was very rewarding. T2 participant I thoroughy enjoyed my second pacement and earned a great dea. T3 participant Thoroughy enjoyed my A&E pacement, reay grew up as a doctor. T3 participant 40 NIHR Journas Library

63 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 BOX 7 Quotes from free-text comments made on the questionnaires Objected to working 12 out of 16 weekends when others worked eight or nine. T2 participant Pretty antisocia shifts of mosty afternoon and evenings to cover service needs. T3 participant Excessive working hours. T3 participant Rota was tough and ife was put on hod. T3 participant Shift work was difficut and work ife baance was amost non existent. T4 participant Impact of pacement in the emergency department There was a statisticay significant interaction between time of pacement in the ED and mean score for motivation (F 6,384 = 4.1, p < 0.005), with a three groups of doctors showing the biggest rise in motivation during their pacement in the ED compared with other pacements (Figure 5). Mean motivation scores for each ED group at each time point are shown in Appendix 19 (see Tabe 28). Severa free-text comments were made about the effort that F2 doctors expended, with exampes given in Box 8. Intention to quit Leves of and change in mean scores for intention to quit were cacuated, with scores ranging from 1 (owest eve) to 5 (highest eve). The coefficient apha for the current sampe ranged from 0.79 (T2) to 5 Mean score effort Time point at which you are carrying out your F2 EM pacement Rotation 1 Rotation 2 Rotation Time point of response 4 FIGURE 5 Interaction between time and time of pacement in the ED and motivation. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 41

64 LONGITUDINAL STUDY BOX 8 Quotes from free-text comments made on the questionnaires I worked very hard to fuy cerk patients. T3 participant The pressure of 4 hour target made it difficut to fee you had competed everything before patient eft dept. T3 participant 0.86 (T4), which compares reasonaby we with the reiabiity of 0.91 obtained in a study of 110 EM staff (incuding junior doctors, emergency care practitioners and paramedics). 62 There was no significant variation in mean intention-to-quit scores across the four time points and there was aso no impact on intention to quit of pacement in the ED (see Appendix 19, Tabe 29). Comparison with other occupationa groups We found no difference between F2 mean scores for intention to quit (T1 T4) and scores from a previous study of 110 EM staff. 62 Job-reated characteristics Roe characteristics We measured eves of and changes in mean sampe scores for work demands, roe carity and task feedback, with scores ranging from 1 (owest eve) to 5 (highest eve). We aso measured mean hours worked in a typica working week and shift changed at short notice in the ast 4 weeks ( yes / no ). The coefficient apha for work demands ranged from 0.81 (T4) to 0.88 (T2), which compares we with the reiabiity of 0.80 quoted by Dwyer and Ganster. 80 The coefficient apha for task feedback ranged from 0.82 (T1) to 0.90 (T3, T4). This compares we with the reiabiities of with heath-care sampes in the study by Haynes et a. 81 The coefficient apha for roe carity ranged from 0.85 (T1, T4) to 0.87 (T2). This compares we with the reiabiities of with heath-care sampes in the study by Sziagyi et a. 82 There was evidence of statisticay significant variation across the four time points in mean scores for both work demands (F 3,393 = 10, p < 0.005) and roe carity (F 3,389 = 6.5, p < 0.005). There was no statisticay significant variation in mean hours worked or shifts changed at short notice between pacements. The mean scores at each of the four time points and overa for work demands, roe carity, task feedback, hours worked in a typica working week and shift changed at short notice in the ast 4 weeks are provided in Appendix 19 (see Tabes 30 and 31). Impact of pacement in the emergency department There was a statisticay significant interaction between time of pacement in the ED and task feedback, with each of the doctor groups reporting the biggest decine in mean task feedback score after their pacement in the ED (F 6,371 = 3.24, p < 0.005) (Figure 6). Box 9 shows some free-text quotes from comments made by F2 doctors on the questionnaires. 42 NIHR Journas Library

65 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO Mean score feedback Time point at which you are carrying out your F2 EM pacement Rotation 1 Rotation 2 Rotation Time point of response 4 FIGURE 6 Interaction between time and time of pacement in the ED and task feedback. BOX 9 Quotes from free-text comments made on the questionnaires I tried to earn but ack of feedback on whether management of cases were satisfactory or not was the biggest probem. T4 participant A weath of experience; did not get much feedback on what we were doing I m not sure the job can ever offer this. T3 participant There was aso a significant interaction between time of pacement in the ED and mean hours worked in a typica working week, with the biggest increase apparent for each group of doctors associated with their ED pacement (F 6,416 = 7.5, p < 0.005) (Figure 7). There were many free-text comments made on the questionnaires about working hours and shift patterns, with exampes given in Box 10. Reationa characteristics Leves of and change in mean sampe scores for support from CSs and coeagues were assessed on two scaes. Extent of support on each scae was scored from 1 ( to a very itte extent ) to5( to a very great extent ). The coefficient apha for CSs support ranged from 0.90 (T1) to 0.94 (T2, T3, T4). The coefficient apha for coeague support ranged from 0.90 (T1) to 0.94 (T2, T3). This compares we with quoted reiabiities for each reationa scae of with heath-care sampes. 81 There was no statisticay significant variation in mean sampe scores for reported support from either CSs or coeagues across the four pacement periods (see Appendix 19, Tabe 32). Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 43

66 LONGITUDINAL STUDY Mean hours worked in a typica working week Time point at which you are carrying out your F2 EM pacement Rotation 1 Rotation 2 Rotation Time point of response 4 FIGURE 7 Interaction between time and time of pacement in the ED and hours worked. BOX 10 Quotes from free-text comments made on the questionnaires Long hours and cose to EWTD imit of ess 70 hours per week; but not taking ruing at its vaue of avoiding exhausted staff. T2 participant Pretty antisocia shifts of mosty afternoon and evenings to cover service needs. T3 participant Excessive working hours. T3 participant Shift work was difficut and work ife baance was amost non existent. T4 participant Impact of pacement in the emergency department There was a statisticay significant interaction between time of pacement in the ED and mean support from coeagues (F 6,390 = 3.4, p < 0.005). This was mainy caused by a arge increase in mean support from coeagues reported by one ED group immediatey after their ED pacement (Figure 8). There were severa free-text comments made about support during the ED pacement, but mosty from staff and supervisors, with exampes given in Box 11. Emergency department-eve comparison of roe characteristics and work-reated outcomes We aso investigated whether non-trivia variation existed in respondents roe characteristics and workreated outcome measures across the participating EDs. This anaysis was cross-sectiona in nature as respondents roe characteristics and work-reated outcomes reative to the specific ED in which they did their ED training were typicay measured just once, immediatey after their ED pacement (i.e. respondents did not typicay do their other F2 pacements in the same department). 44 NIHR Journas Library

67 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO Mean score coeague support Time point at which you are carrying out your F2 EM pacement Rotation 1 Rotation 2 Rotation Time point of response 4 FIGURE 8 Interaction between time and time of pacement in the ED and coeague support. BOX 11 Quotes from free-text comments made on the questionnaires Fantasticay supportive staff. Medica staff were approachabe and supportive. Highy trained nurses who encouraged junior doctors a ot. I aways fet there was someone to ask if I needed to check a decision I made. We supervised. Good support and reguar teaching. T2 participant T2 participant T2 participant T3 participant T3 participant T4 participant The majority of the variabes showed itte or no variance that coud be expained by the ED in which the F2 doctors were working during their ED pacement. The percentage variance in scores for each of the roe characteristics and work-reated outcomes that existed at the ED eve exceeded 10% for CS support, number of hours of one-to-one contact with supervisor, number of hours of cose working contact with supervisor and job satisfaction. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 45

68 LONGITUDINAL STUDY Summary of resuts 1. F2 doctors reported a significant increase in confidence in managing 23 common acute conditions over the course of F2 training, with the biggest increase in confidence arising immediatey after their ED pacement. 2. Experience managing common procedures improved significanty over the four time points (with competence in interpreting arteria bood gases improving non-significanty between T1 and T4), with the biggest increase in competence in a five procedures arising immediatey after the ED pacement. 3. In terms of we-being, F2 doctors showed simiar (T2 comparison) or better (T1, T3, T4) eves of anxiety and depression than a comparison group of professiona and technica workers. Leves of anxiety and depression did not vary significanty over time but the biggest rise in anxiety was associated with an ED pacement. 4. Job satisfaction is another aspect of we-being and F2 doctors reported significanty higher eves of job satisfaction than a arge comparison group of doctors and other NHS staff. A significant improvement in overa and intrinsic job satisfaction was seen during F2. However, in two of the three groups studied the ED pacement was associated with significanty ower scores for the extrinsic aspects of job satisfaction (e.g. pay, hours of work). 5. Motivation was examined in terms of effort and this did not vary significanty across F2. Reported eves of effort were simiar to (T3) or ower than (T1, T2, and T4) those in a comparison group of managers. However, a steep increase in effort was associated with the ED pacement. 6. Examination of work characteristics showed that there was significant variation in work demands and roe carity across the various pacements. However, the ED pacement was associated with a significant reduction in task feedback and an increase in socia support from coeagues and hours worked in a typica week (work demands increased sighty and roe carity improved but not significanty). 7. There was some weak (cross-sectiona) evidence of variation in work characteristics across trusts associated with variation in hours of one-to-one contact and cose working contact with CS, work demands, job satisfaction, amount of teaching and support from CSs. 46 NIHR Journas Library

69 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Chapter 5 A cinica case notes review of foundation year 2 doctors quaity of care Introduction Reviewing quaity of care as documented in patient records has become an estabished method for assessing the quaity of care in heath-care organisations. 83 This approach has been used in a variety of heath-care settings, incuding emergency care The two main approaches currenty used for retrospectivey assessing quaity of care as recorded in cinica records are (1) hoistic review (aso referred to as impicit review), whereby reviewers use their cinica knowedge and professiona judgement to assess the quaity of care, and (2) criterion-based review (aso referred to as expicit review), in which there is a consensus or estabished standard of care that supports review against these expicit standards. The advantages and disadvantages of these two review methods are identified in Brown et a. 89 Hoistic review can be easier to conduct as it reies on professiona judgements of good and poor care that can be appied to any condition. The subjectivity of these judgements is regarded as a weakness of impicit review compared with criterion-based assessments of care, which rey on more expicit criteria (e.g. those derived from nationa cinica guideines) and have greater reproducibiity. Structured methods for hoistic review and for deveoping expicit review criteria that can be appied to assessing quaity of care from case notes have been revised in the UK. 90 Evidence-based cinica guideines pubished by NICE are now being accompanied by review criteria to support the assessment of cinica quaity. The criterion-based review method has been criticised for not being sensitive enough to identify unexpected factors potentiay infuencing outcomes of care, 91,92 whereas the reative advantage of hoistic review is that judgements can capture the fu extent of cinica decisions about care. The issue of poor to moderate eves of inter-rater reiabiity is acknowedged as a probem with the use of case-note review, and findings from a systematic review of inter-rater reiabiity show higher reiabiity vaues for studies that used criterion-based review than for studies that used hoistic review. 93 This might be expected given the subjective nature of impicit assessment in the atter approach and the review authors do highight the potentia drawback of this higher reiabiity being the possibiity that predefined review criteria may omit eements of care that can be considered when using the hoistic approach. On that basis, Liford et a. 93 argue that the two methods may be regarded as compementary and advocate a mixed hoistic and criterion-based approach. A more recent study by Hutchinson et a. 94 aimed to determine which of the two methods provides the most usefu and reiabe information for assessing quaity of care. The study entaied a arge-scae review of case notes across severa hospitas and reviewers using both hoistic and criterion-based review methods. Their study found a reasonabe eve of agreement between the two methods and the individua reviewers. The findings in reation to inter-rater reiabiity were broady consistent with those of other studies, showing ower scores for hoistic reviews. The review methods used in this study are consistent with those empoyed by Hutchinson et a. 94 Aims A review of the cinica records of patients treated by participating F2 doctors was undertaken to evauate the quaity of care and adherence to evidence-based guideines. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 47

70 A CLINICAL CASE NOTES REVIEW OF FOUNDATION YEAR 2 DOCTORS QUALITY OF CARE The aims of this review were to: assess the quaity of care provided by a F2 doctors during their ED pacement provide a measure of quaity of care that can be used to assess its reationship with job-reated characteristics, we-being and motivation. Methods Quaity of care provided by F2 doctors during their ED pacement was assessed using a combination of criterion-based and hoistic review methods. The study assessed quaity of care deivered in reation to two cinica conditions: head injury and chronic obstructive pumonary disease (COPD). Speciaist registrars in EM were recruited from each of the participating EDs to act as reviewers in their own hospita, as woud happen with cinica audit. Recruitment of emergency departments/sites and reviewers The aim was to recruit 12 EDs to the quaity-of-care study with the expectation that this woud provide access to 144 F2 doctors. Initiay, a 30 participating EDs were contacted by e-mai in November 2010 to assess whether their ED information systems recorded the identifiers of the primary caregivers (in this case F2 doctors) for cinica episodes in the ED. Identifying ED episodes for which the F2 doctors were the primary caregiver was a necessity in the records review of F2 doctor quaity of care. Nineteen of the 30 responded, detaiing that their ED information systems had this faciity. These 19 were then contacted by e-mai in March 2011 with an outine of the study and an invitation to take part. Fourteen of the 19 EDs contacted expressed interest in participating, but four of these were unabe to compete the reviews within the required time frame for the project. A tota of 10 EDs took part in the study and provided access to 74 F2 doctors. These 10 EDs represented six of the nine deaneries where F2 doctors carried out their pacements in EM. Recruitment of the EDs aso reied on the recruitment of sufficient numbers of midde-grade doctors (speciaist registrars) in EM to review the seected records at each site. Reviewers were recruited through the ead consutants at each site; a tota of 28 reviewers were recruited across the 10 participating EDs. A copy of the reviewer information sheet is provided in Appendix 21. Choice of cinica conditions for review The choice of cinica conditions for review invoved the initia identification of conditions that F2 doctors coud be expected to see with reasonabe frequency during their time in the ED, that are reasonaby common in presentation, that present in aduts, that wi differentiate performance and that have we-defined cinica guideines to support the criterion-based review. Foowing discussions between the project team and oca ED consutants, six potentia cinica conditions were identified: overdose head injury COPD asthma back pain gastrointestina beed. The consutant eads at the 14 EDs that had expressed an interest in participating in the quaity-of-care review were asked to rank the six conditions in reation to their suitabiity and ikeihood that F2 doctors at their site woud have responsibiity for the care of patients with these conditions. In tota, 10 of the 14 sites and two F2 doctors provided rankings for the provisiona ist of conditions. Three sites suggested chest pain as an additiona condition but it was fet that there woud be ess scope for variation in practice. Taking an average of a ranks, overdose, head injury and COPD were ranked highest across a sites. COPD 48 NIHR Journas Library

71 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 and head injury were the two conditions chosen for the study on the basis that they best met the criteria above. These two conditions aso had the advantage that they were quite ED specific and different from the range of conditions incuded in the ongitudina study assessment of confidence in managing conditions across a pacements. Deveopment of criterion-based review form Criterion-based review requires the reviewer to identify and record specific items of care that are consistent with estabished guideines on quaity of care. The review criteria in this study were deveoped using estabished methods for deveoping expicit evidence-based review criteria from cinica guideines as detaied in Hutchinson et a. 95 For each of the two conditions, the first drafts of the criteria were deveoped using reevant nationa cinica guideines 96,97 and oca ED guideines; these were subsequenty vaidated by the project team and ED cinicians. Cinicians in the project team in consutation with ED coeagues identified a subset of criteria that might be usefu in the study. Refinement of the criteria invoved three stages. Initiay, the criteria were reviewed to consider whether the necessary information was ikey to be avaiabe in the case notes. The draft ists of criteria were then sent to the consutant eads at participating EDs to cassify the criteria as essentia, desirabe or non-essentia and to eicit comments on the structure and wording of each item. A fina review by the project team sought to ensure that a criteria were cear and ogica. This process resuted in 15 head injury criteria and 20 COPD criteria. The criterion-based review items and scores for head injury and COPD are provided in Appendices 23 and 24 respectivey. Deveopment of the hoistic review form The structured review form used for the hoistic review was consistent with that in previous studies using this approach 88,94 in providing a framework to structure impicit judgements with a view to maximising inter-rater reiabiity. Unike the criterion-based approach, reviewers were not provided with any specific criteria for current best practice and were asked to use their professiona judgement. The hoistic review form provided reviewers with a imited structure to enabe different eves of heath-care quaity to be identified from exceent to poor care across different aspects of care (see Appendix 25). This approach has been used successfuy by the study team in previous research examining quaity in emergency care. 88 The reviewers were asked to rate the quaity of care actuay provided (as documented in the cinica notes) in reation to three key aspects of care (assessment of the cinica probem, investigations performed and patient management) and overa care on a numerica scae (1 = unsatisfactory, 6 = very best care). Reviewers were provided with written guidance to aid consistency in the interpretation of the numerica quaity-of-care scae (Tabe 9). In addition, reviewers were asked to provide textua comments regarding the overa quaity of care. TABLE 9 Guidance for reviewers in rating quaity of care 1 Care fe short of current best practice in one or more significant areas resuting in the potentia for, or actua, adverse impact on the patient 2 Care fe short of current best practice in more than one significant area but is not considered to have the potentia for adverse impact on the patient 3 Care fe short of current best practice in ony one significant area but is not considered to have the potentia for adverse impact on the patient 4 This was satisfactory care, faing short of current best practice in more than two minor areas 5 This was good care, faing short of current best practice in one or two minor areas ony 6 This was exceent care and met current best practice Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 49

72 A CLINICAL CASE NOTES REVIEW OF FOUNDATION YEAR 2 DOCTORS QUALITY OF CARE Assessing the quaity of recording in the case notes Assessing quaity of care using case-note review is dependent on the extent to which information is recorded in the patient record, the quaity of that information and the egibiity of the record. Furthermore, the patient record is itsef an indicator of quaity of care, as information recorded in cinica notes underpins decisions about patient care and continuity of care. A study examining the reationship between quaity of case notes and adverse events found that poor quaity (competeness, readabiity and adequacy) of the avaiabe patient information was associated with higher rates of adverse events. 98 Reviewers were therefore asked to assess the quaity of each record using a six-point rating scae (1 = inadequate, 6 = exceent) foowing the guidance in Tabe 10. Assessing case mix: compexity To assess the potentia variation in case mix across the case notes a question was incuded to determine the compexity of each case, consistent with that used by assessors as part of cinica evauation in the FT programme. Reviewers were asked to rate the compexity of the cinica presentation detaied in each patient record as ow, average or high. Deveoping the web-based data-coection too The project team deveoped a web-based data-coection too using the same survey-design software (SurveyGizmo) as the ongitudina study. The data-coection website enabed reviewers to record responses to each of the assessment categories on the review form and submit their data directy to the study team (see Appendix 22 for screenshot of the data-coection website). The presentation of the criterion-based and hoistic review forms was consistent with the format outined above. For each record, the reviewer was asked to identify the cinica presentation detaied in the patient record as either a head injury or COPD and was then directed to the reevant review form. Reviewers were required to compete the criterion-based form first, foowed by the hoistic form. Seection of records Much of the research on case-note review has focused on care provided at site eve rather than individua doctor eve, and Hofer et a. 99 recommend a reativey sma sampe of 5 10 reviews as sufficient to characterise care for a site where the condition has a good evidence base. The study aimed to seect 10 case notes per F2 doctor over a specified period during their ED pacement, to incude a mix of five head-injury and five COPD cases. It was not aways possibe to obtain five records per condition for each F2 doctor and some sites seected > 10 records when they were avaiabe. Across the 10 participating sites, between six and 17 records were seected for each doctor during their ED pacement. A persona identifiers were removed from the records. To minimise potentia bias resuting from harshness or eniency of individua reviewers, the case notes for each F2 doctor were distributed amongst the individua reviewers at that site. To assess inter-rater reiabiity at sites with more than one reviewer, a sma subset of records was assessed by a reviewers within each site. As the records were accessibe ony to those staff within each site, it was TABLE 10 Guidance for reviewers in rating quaity of cinica records 1 The patient record contains gaps in three or more significant areas 2 The patient record contains gaps in two significant areas 3 The patient record contains gaps in one significant area 4 The patient record is satisfactory and contains gaps in ony three or more minor areas 5 The patient record is good and contains gaps in ony one or two minor areas 6 The patient record is exceent 50 NIHR Journas Library

73 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 not possibe to assess reiabiity across the participating sites. The study team worked cosey with research staff and administrators at each of the participating sites, who assisted in identifying suitabe patient records for review, copying records and removing any persona identifying information. This incuded copying sets of common records required to assess inter-rater reiabiity. Tabe 11 shows the numbers of F2 doctors, reviewers, records to assess quaity of care and records to assess inter-rater reiabiity at each participating ED. Reviewer training The study provided a reviewers with standardised training in case-note review during a 3-hour session deivered at various participating sites. The aim of the training was to provide guidance on how to compete the criterion and hoistic review forms and the type of textua comments required, and to aow reviewers to practice conducting reviews with a set of anonymised case notes. The data-coection website was aso demonstrated. Reviewers were aware of the aims of the study. It was considered unikey that the reviewers woud know any of the F2 doctors because of rotation changes; however, the importance of making assessments soey on the basis of the information contained in the patient case notes was stressed. At the end of the training the reviewers were sent an e-mai providing them with access to the data-coection website. The reviewers were aso provided with a set of case notes for review, incuding any additiona records to support the assessment of inter-rater reiabiity. Data anaysis Data from the SurveyGizmo database were transferred to IBM SPSS version 19 (SPSS Inc., Chicago, IL, USA) for statistica anaysis. Statistica anaysis examined inter-rater reiabiity, the quaity of care deivered by F2 doctors during their ED pacements and the reationship between work characteristics, we-being and quaity of care. TABLE 11 Numbers of F2 doctors, reviewers, records to assess quaity of care and records to assess inter-rater reiabiity Site (ED no.) No. of F2 doctors No. of reviewers No. of records to assess quaity of care per F2 doctor (tota records) No. of records to assess inter-rater reiabiity (48) (20) NA (39) NA (132) (63) (106) (49) (76) (172) (40) 12 Tota NA, not appicabe. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 51

74 A CLINICAL CASE NOTES REVIEW OF FOUNDATION YEAR 2 DOCTORS QUALITY OF CARE Scoring of criterion-based data invoved cacuating a tota score for each head injury and COPD patient record based on the tota criteria met per record (see Appendices 23 and 24 for criterion scores for head injury and COPD respectivey). To compare mean criterion scores for the two conditions, a score for the proportion of the criteria met was aso cacuated by dividing the tota score by the maximum potentia score. Anaysis of hoistic scores for head injury and COPD cases used the actua ratings from each of the hoistic rating scaes, which are comparabe across the two conditions. Mean criterion-based and hoistic scores for the two conditions were cacuated for use in assessment of the reationship between quaity of care, job-reated characteristics, we-being and motivation. Intracass correations (ICCs) were cacuated to assess the reiabiity of ratings between pairs or groups of reviewers within participating EDs. The ICC gives the correation between reviewer ratings for the same set of case notes and is based on continuous data, unike kappa statistics, which require the data to be categorica. Descriptive statistics were cacuated for criterion-based and hoistic ratings to examine quaity of patient care deivered by F2 doctors during their ED pacements. Mean scores were cacuated for criterion-based and hoistic ratings across three eves of case compexity (ow, average and high) and statistica tests were conducted to assess the statistica significance of differences in scores [anaysis of variance (ANOVA), t-test]. Pearson s correation coefficients were cacuated to assess the reationship between criterion and hoistic ratings. Resuts Inter-rater reiabiity Tabe 12 provides the ICCs used to assess inter-rater reiabiity; these are based on the actua (absoute) scores rather than the ranked (consistency) scores as the actua ratings are used to examine performance. The singe-measure ICCs are presented as the quaity-of-care anaysis uses ratings from singe reviewers rather than averaging mutipe ratings for each record. In genera, these wi be ower than the reiabiity that might be expected from using the average of severa raters (Tabe 13). The correations for the criterion-based reviews range from 0.65 to 0.94 and represent strong agreement. The correations for the hoistic review are ower and more variabe ( ) but are reasonaby consistent with those from other studies. 93,94 These studies aso report ower ICC statistics for hoistic review than for criterion-based review. Athough an ICC of 0.8 is regarded as indicative of good agreement, this generay reates to data having a cear right and wrong answer and for which 100% agreement is possibe. Higher eves of agreement are more achievabe with criterion-based review using we-estabished criteria, whereas in the hoistic review approach, based on subjective judgements, such a high eve of agreement is ess ikey. Appendix 26 provides ICC anayses based on the ranked review scores rather than the actua (absoute) scores (Tabes 36 and 37). Quaity of care deivered by foundation year 2 doctors during emergency department pacements Tabe 14 presents the mean criterion-based and hoistic scores for a F2 doctors. The mean scores for the different aspects of care assessed by hoistic review are a > 4 on the six-point rating scae. There was a significant difference between the mean scores for investigations [t(739) = 2.04, p = 0.04], with head-injury case notes scoring higher than COPD case notes. 52 NIHR Journas Library

75 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 12 Inter-rater reiabiity between reviewers within each site: criterion-based and hoistic review of common head injury and COPD records (absoute score, singe measures) Records, reviewers and approach Site (ED code) a No. of records No. of reviewers No. of F2 doctors Inter-rater reiabiity: ICC (95% CI) Criterion tota 0.94 (0.82 to 0.98) 0.73 (0.52 to 0.88) 0.65 (0.07 to 0.89) 0.92 (0.62 to 0.98) 0.69 (0.35 to 0.90) 0.74 (0.49 to 0.93) 0.94 (0.83 to 0.98) 0.93 (0.57 to 0.98) Hoistic assessment 0.40 ( 0.19 to 0.79) 0.29 (0.07 to 0.61) 0.43 ( 0.09 to 0.79) 0.21 ( 0.25 to 0.75) 0.38 (0.03 to 0.75) 0.18 ( 0.01 to 0.60) 0.58 (0.21 to 0.87) 0.30 ( 0.14 to 0.71) Hoistic investigations 0.34 ( 0.18 to 0.74) 0.08 ( 0.01 to 0.32) 0.36 ( 0.29 to 0.77) 0.14 ( 0.27 to 0.70) 0.47 (0.10 to 0.80) 0.19 (0.01 to 0.58) 0.14 ( 0.17 to 0.61) 0.43 ( 0.08 to 0.78) Hoistic management 0.31 ( 0.33 to 0.74) 0.11 ( 0.01 to 0.37) 0.39 ( 0.18 to 0.77) 0.10 ( 0.06 to 0.53) 0.66 (0.27 to 0.89) 0.14 ( 0.09 to 0.41) 0.19 ( 0.10 to 0.63) 0.32 ( 0.14 to 0.72) Hoistic overa quaity 0.48 ( 0.13 to 0.82) 0.20 (0.00 to 0.39) 0.35 ( 0.16 to 0.74) 0.11 ( 0.11 to 0.60) 0.60 (0.18 to 0.87) 0.12 ( 0.05 to 0.52) 0.31 ( 0.02 to 0.72) 0.20 ( 0.18 to 0.62) Hoistic ED record 0.65 (0.17 to 0.88) 0.47 (0.22 to 0.75) 0.32 ( 0.16 to 0.72) 0.56 (0.10 to 0.90) 0.49 (0.09 to 0.82) 0.30 (0.05 to 0.71) 0.39 ( 0.05 to 0.79) 0.37 ( 0.15 to 0.76) CI, confidence interva. a Ony sites with more than one reviewer are incuded in the reiabiity anaysis; therefore, some sites do not appear in this tabe. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 53

76 A CLINICAL CASE NOTES REVIEW OF FOUNDATION YEAR 2 DOCTORS QUALITY OF CARE TABLE 13 Inter-rater reiabiity between reviewers within each site: criterion-based and hoistic review of common head injury and COPD records (absoute score, average measures) Records, reviewers and approach Site (ED code) a No. of records No. of reviewers No. of F2 doctors Inter-rater reiabiity: ICC (95% CI) Criterion tota 0.97 (0.90 to 0.99) 0.93 (0.84 to 0.98) 0.79 (0.07 to 0.89) 0.97 (0.83 to 0.99) 0.87 (0.61 to 0.97) 0.94 (0.85 to 0.99) 0.98 (0.94 to 0.99) 0.96 (0.73 to 0.99) Hoistic assessment 0.58 ( 0.48 to 0.88) 0.67 (0.28 to 0.89) 0.61 ( 0.09 to 0.79) 0.44 ( 1.46 to 0.90) 0.64 (0.10 to 0.90) 0.58 ( 0.06 to 0.90) 0.81 (0.44 to 0.95) 0.47 ( 0.33 to 0.83) Hoistic investigations 0.51 ( 0.43 to 0.85) 0.31 ( 0.08 to 0.70) 0.53 ( 0.84 to 0.87) 0.32 ( 1.79 to 0.88) 0.73 (0.24 to 0.92) 0.58 (0.07 to 0.89) 0.32 ( 0.78 to 0.82) 0.60 ( 0.17 to 0.88) Hoistic management 0.47 ( 0.99 to 0.85) 0.38 ( 0.03 to 0.74) 0.56 (0.45 to 0.87) 0.26 ( 0.21 to 0.77) 0.85 (0.52 to 0.96) 0.20 ( 0.98 to 0.81) 0.41 ( 0.39 to 0.84) 0.49 ( 0.32 to 0.84) Hoistic overa quaity 0.65 ( 0.31 to 0.90) 0.42 (0.01 to 0.76) 0.52 ( 0.38 to 0.85) 0.27 ( 0.43 to 0.82) 0.82 (0.40 to 0.95) 0.44 ( 0.40 to 0.87) 0.58 ( 0.05 to 0.89) 0.33 ( 0.43 to 0.76) Hoistic ED record 0.79 (0.29 to 0.94) 0.82 (0.59 to 0.94) 0.49 ( 0.37 to 0.84) 0.80 (0.25 to 0.96) 0.74 (0.23 to 0.93) 0.72 (0.26 to 0.94) 0.66 ( 0.17 to 0.92) 0.54 ( 0.36 to 0.86) CI, confidence interva a Ony sites with more than one reviewer are incuded in the reiabiity anaysis; therefore, some sites do not appear in this tabe. 54 NIHR Journas Library

77 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 14 Mean scores for criterion-based and hoistic review of case notes Review Head injury (n = 382), mean (SD) COPD (n = 359), mean (SD) Hoistic assessment 4.14 (1.15) 4.30 (1.12) Hoistic investigations 4.61 (1.29) 4.42 (1.23) Hoistic management 4.46 (1.21) 4.33 (1.20) Hoistic overa quaity 4.32 (1.24) 4.28 (1.17) Hoistic ED record 4.17 (1.15) 4.16 (1.15) Criterion tota (% of criteria met) (14.60) (16.00) Appendix 27 provides detais of the proportions of review criteria assessed as being met in the head injury and COPD patient case notes. This gives an indication of the reative strengths and weaknesses of F2 doctors performance and quaity of care for these two conditions. Appendix 28 shows the distribution of hoistic review scores for head injury and COPD case notes across the six-point rating scae. The more detaied information on proportions of review criteria being met (see Appendix 27) highights omissions in recording information. A common issue across both the head injury records and the COPD records appears to be the eve of detai provided. Criteria reating to documenting the mechanism and detais of head injury are met by 93% and 80% of case notes respectivey; however, information is acking in reation to oss of consciousness and the reason for organising a computerised tomography (CT) head scan. Simiary, for the COPD case notes, detais about shortness of breath and respiratory rate were recorded by most but there was variabe recording for other criteria. Case mix Tabe 15 shows the distribution of head injury and COPD case notes across the three categories of compexity of case presentation. The majority of cases across the two conditions were cassified as average. A higher proportion of head injury cases than COPD cases were categorised as ow compexity and conversey a higher proportion of COPD cases than head injury cases were cassified as high compexity. A one-way ANOVA was carried out to test for differences in mean quaity-of-care scores across the three compexity categories. No significant difference was found for head injury scores, but the COPD criterion scores were significanty higher for more compex cases (F 2,356 = 8.55, p = 0.00). A post hoc Tukey test showed that the difference was specific to the high compexity category, which had significanty higher scores than the ow or average compexity categories (p < 0.05). A further t-test anaysis was conducted to assess whether F2 doctors with more compex COPD cases achieved higher criterion scores than those with COPD cases of ow or average compexity ony. This anaysis found no significant differences in COPD criterion scores between F2 doctors with high compexity cases and those with ow and average compexity cases ony. Therefore, we are confident that any variation observed in the quaity of care is not attributabe to a variation in case mix. TABLE 15 Distribution of head injury and COPD case notes across the three categories of case compexity Compexity Head injury (n = 384), % (n) COPD (n = 361), % (n) Low 32.6 (125) 10.8 (39) Average 62.8 (241) 77.0 (278) High 4.2 (16) 11.6 (42) Missing data 0.5 (2) 0.6 (2) Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 55

78 A CLINICAL CASE NOTES REVIEW OF FOUNDATION YEAR 2 DOCTORS QUALITY OF CARE Reationship between criterion-based and hoistic quaity-of-care ratings Pearson s correation coefficients cacuated to expore the reationship between criterion-based and hoistic ratings for head injury and COPD case notes are presented in Tabes 16 and 17 respectivey. Associations between the various hoistic measures are sighty stronger than those between the hoistic and criterion-based measures. TABLE 16 Correations between criterion-based and hoistic ratings for head-injury case notes (n=382) Aspects of care Investigations Management Overa quaity ED record Criterion tota Assessment Pearson s correation Significance (two-taied) Investigations Pearson s correation Significance (two-taied) Management Pearson s correation Significance (two-taied) Overa quaity Pearson s correation Significance (two-taied) ED record Pearson s correation Significance (two-taied) TABLE 17 Correations between criterion-based and hoistic ratings for COPD case notes (n=359) Aspects of care Investigations Management Overa quaity ED record Criterion tota Assessment Pearson s correation Significance (two-taied) Investigations Pearson s correation Significance (two-taied) Management Pearson s correation Significance (two-taied) Overa quaity Pearson s correation Significance (two-taied) ED record Pearson s correation Significance (two-taied) NIHR Journas Library

79 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Summary Intracass correations cacuated to assess inter-rater reiabiity showed a high eve of agreement amongst reviewers for the criterion-based review ( ). The hoistic review ICCs were much ower ( ) and varied across sites. Taking an average measure of quaity across reviewers produced higher ICCs for criterion-based ( ) and hoistic ( ) review. Mean scores for the proportion of head injury and COPD criteria met were 50.78% and 54.88% respectivey. Mean scores for the different aspects of care assessed by hoistic review range from 4.14 to 4.61 on the six-point scae. Anaysis of the criteria met identifies strengths and weaknesses in reation to quaity of care, specificay the extent of information recorded in case notes. Findings from an anaysis of case mix for head injury and COPD cases found no significant differences in reation to quaity of care and compexity of cinica presentation, indicating that any observed differences in quaity of care are not attributabe to case mix variation. Correations between criterion-based and hoistic quaity-of-care ratings show associations between a review measures, but the intercorreations between the hoistic measures are somewhat higher ( ) than those between the hoistic and criterion-based measures ( ). Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 57

80

81 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Chapter 6 Anaysing the reationship between foundation year 2 doctors job-reated characteristics, work-reated we-being and motivation, quaity of care and performance during (emergency department) pacements Introduction and aims This part of the study aimed to expore possibe associations between the we-being of F2 doctors and the quaity of patient care being deivered. Therefore, data from the ongitudina study that reated to the we-being of our cohort during their ED pacement were extracted (see Chapter 4), as we as data from our cinica records review study reating to the quaity of care provided (see Chapter 5). The new data set aimed to: evauate how the job-reated characteristics, we-being and motivation of F2 doctors in the ED is associated with the quaity of patient care identify key measures of F2 doctor we-being and motivation that are associated with quaity of patient care and which wi underpin the deveopment of a too to monitor we-being and motivation during training. Methods As described in the previous chapter, quaity-of-care data for two conditions (head injury and COPD) were retrospectivey coected from the ED cinica records of 74 F2 doctors, from the time of their F2 pacement in the ED. These 74 F2 doctors had participated in the ongitudina study of F2 doctors. A subsampe of 55 of the 74 F2 doctors from the cinica case notes review who aso suppied data in the ongitudina study about their job-reated and work-reated outcomes during their pacement in the ED were subsequenty incuded in this anaysis. Anaysis Detaied information on how the work-reated outcomes and job-reated characteristics were cacuated is contained in Chapter 4. In summary, measures of work-reated outcomes (we-being and motivation, intention to quit, confidence in managing acute conditions) and job-reated characteristics (e.g. work demands, task feedback, roe carity) were anaysed for 55 F2 doctors from the time of their pacement in the ED. Detaied information on how the quaity-of-care outcomes were cacuated is contained in Chapter 5. In summary, two approaches were used, criterion-based review and hoistic review. Criterion-based review invoved cacuating a tota score for each head injury and COPD patient record based on the tota number of criteria met per record. The hoistic review provided reviewers with a numerica scae to rate the quaity of care actuay provided (1 = unsatisfactory, 6 = very best care). For this anaysis we used four outcomes two criterion-based scores for head injury and COPD and two hoistic overa care scores for the two conditions. Hoistic overa scores were cacuated by combining hoistic scores for assessment, investigations and management. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 59

82 ANALYSING FOUNDATION YEAR 2 DOCTORS DURING ED PLACEMENTS Correations between mean scores for the work-reated outcomes and job-reated characteristics reating to the time point at the end of each respondent s ED pacement and the four quaity-of-care outcomes were cacuated using Pearson s correations. A statisticay significant eve of association was set at p < The correated variabes are isted in Box 12. Resuts Quaity of care The ony statisticay significant association at the p < 0.05 eve was the association between task feedback and hoistic overa care. However, there was a cear pattern of sma- to medium-sized correations between eves of motivation and two quaity-of-care outcomes (head injury hoistic overa care and COPD hoistic overa care) such that higher eves of motivation were more ikey to occur amongst those doctors with higher scores for these two quaity-of-care outcomes. A simiar pattern of effects existed, abeit in the opposite direction, for work demands and these two quaity-of-care outcomes such that respondents with higher work demands were more ikey to have ower quaity-of-care outcomes for head injury hoistic overa care and COPD hoistic overa care (with the atter approaching statistica significance, p < 0.051). Indeed, when we examined the association with the singe hoistic measure of overa care, the association between work demands and overa care was statisticay significant. The correations between a job-reated characteristics/work-reated outcomes and quaity-of-care outcomes are provided in Tabe 18. BOX 12 List of work-reated variabes and job-reated characteristics and quaity-of-care outcomes (ED pacement ony) Work-reated variabes and job-reated characteristics Confidence in managing common conditions. Improvement in management. Work demands. Roe carity. Feedback. CS support. Coeague support. Job satisfaction. Depression. Anxiety. Intention to quit. Motivation. Professiona knowedge/earning. Teaching/training received. Quaity-of-care outcomes Head injury criterion-based score. Head injury hoistic overa care score. COPD criterion-based score. COPD hoistic overa care score. 60 NIHR Journas Library

83 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 18 Associations between job-reated characteristics/work-reated outcomes and quaity of care (n = 55) Pearson s correations Job-reated characteristic/work-reated outcome (at singe point in time post ED pacement) Head injury (criterion-based score) COPD (criterion-based score) Head injury (hoistic overa care) COPD (hoistic overa care) Confidence in managing common conditions Improvement in management Work demands Roe carity Feedback CS support Coeague support Job satisfaction Depression Anxiety Intention to quit Motivation Profession knowedge/earning Teaching/training received Routine performance data Participants To further expore inks between we-being and care provided to patients in the ED, we aimed to coect performance data on their ED pacements for a 217 F2 doctors who had participated in the ongitudina study. We approached the 30 participating EDs to provide the reevant data; data were suppied by 18 EDs (60%) on 116 F2 doctors (53.4%). A subsampe of 74 of the 116 (63.8%) F2 doctors had aso suppied data in the ongitudina study about their work-reated outcomes and job-reated characteristics during their pacement in the ED and they were subsequenty incuded in this anaysis. Data coected A data were routiney coected by the participating EDs. The foowing individua-eve data were requested from a participating EDs: The tota number of patients seen by each F2 doctor during their ED pacement. The percentage of patients seen by each F2 doctor within the 4-hour ED performance target. The number of unpanned patient reattendances within 7 days per F2 doctor. The mean doctor episode time per F2 doctor (over pacement in the ED). Episode time was defined as the time interva from when the patient first saw the F2 doctor to discharge/referra from the ED. The mean tota time per F2 doctor (over pacement in the ED). Tota time was defined as the time interva from patient arriva in the ED to discharge/referra from the ED. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 61

84 ANALYSING FOUNDATION YEAR 2 DOCTORS DURING ED PLACEMENTS In addition, data were coected at the ED eve as foows: Mean waiting time for each month during the period of the ongitudina study. Waiting time was defined as the time interva from when the patient arrived in the ED to when he or she eft the ED. The percentage of waiting times in the ED that were within 4 hours for each month during the period of the ongitudina study. Anaysis Pearson s correations between mean scores for the work-reated outcomes and job-reated characteristics reating to the time point at the end of each respondent s ED pacement and performance outcomes were cacuated. A statisticay significant eve of association was set at p < Resuts As in the anaysis of quaity of care, no statisticay significant associations existed between we-being and motivation and scores on the performance outcomes at the p < 0.05 eve (Tabe 19). There was a cear pattern of sma- to medium-sized correations between both anxiety and depression and two of the performance outcomes such that: respondents with reported higher eves of anxiety or depression had higher mean episode times and higher mean tota waiting times respondents with reported higher eves of anxiety or depression were more ikey to see a smaer percentage of patients within the 4-hour ED performance target. Summary There was ony one statisticay significant association at the p < 0.05 eve between work-reated we-being and motivation and job-reated characteristics and scores on either the quaity-of-care outcomes or the performance outcomes. This was unsurprising given the sma sampe sizes in both of the anayses. There was evidence of sma- to medium-sized associations between motivation and two quaity-of-care outcomes (with higher eves of effort associated with better quaity-of-care scores). A simiar strength of association was seen for anxiety and depression and two performance outcomes (with higher eves of anxiety or depression associated with poorer performance outcomes). Further research is required to coect ongitudina data on measures of both we-being and quaity/ performance to further investigate the nature and strength of these reationships. 62 NIHR Journas Library

85 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 19 Association between job-reated characteristics/work-reated outcomes and performance Job-reated characteristic/ work-reated outcome (at singe point in time post ED pacement) F2 doctor mean episode time F2 doctor mean tota waiting time %off2 doctor patients seen within 4 hours Mean ED waiting times %ofed patients seen within 4 hours Confidence in managing common conditions Pearson s correation n Improvement in management Pearson s correation n Work demands Pearson s correation n Roe carity Pearson s correation n Feedback Pearson s correation n CS support Pearson s correation n Coeague support Pearson s correation n Job satisfaction Pearson s correation n Depression Pearson s correation n Anxiety Pearson s correation n Intention to quit Pearson s correation n Motivation Pearson s correation n Professiona knowedge/eaning Pearson s correation n Teaching/training received Pearson s correation n Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 63

86

87 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Chapter 7 Discussion Introduction We used a mutipe perspective mixed-methods study to examine the current arrangements for the deivery of FT in Engand and to examine a group of 217 foundation doctors as they proceeded through their second year of training (F2, August 2010 August 2011) in 28 NHS trusts. We were particuary interested in exporing doctors confidence and competence during training as we as their we-being and motivation; the impact of an ED pacement on their we-being and abiities; and the quaity of care that they provided. Principa findings The main findings from our study are shown in Tabe 20. This chapter wi expore these findings and discuss them in reation to evidence from other studies. Postgraduate medica training: the theory and the reaity Postgraduate medica training has changed from the previousy unstructured and service-ed experience of the house officer years to a more structured programme of FT accommodating over 8000 trainees in Engand. Coins 22 undertook an evauation of this change and this report became avaiabe during the course of this study, aowing some comparisons. FT was estabished in 2005; our stakehoder study examined the position of the programme in its fifth year of operation and the ongitudina study examined the reaity for the trainees in its sixth year of operation. We found that there was a strong nationa framework for FT across Engand and Scotand which was we understood by nationa and regiona stakehoders, demonstrating the successfu change to a structured programme of postgraduate deveopment and agreeing with the Coins evauation. 22 However, examining the reaities of impementation at regiona and trust eve reveaed severa differences in terms of pacement quaity and the amount of supervision and assessment avaiabe for trainees. There was evidence that the training programme had good quaity-assurance processes in pace and the regiona and nationa stakehoder groups were aware of some of these differences, but issues of communication and invovement of NHS trusts in pacement panning and deivery were highighted as reasons for impementation difficuties. There was aso concern expressed at the trust eve in reation to the capacity of trainers to deiver the voume of training and assessments required by FT, and that this often conficted with service deivery commitments. We found that there was a variety of innovative pacements being empoyed (such as genera practice and paiative care or highy speciaised roes such as medica bioogy and neonata orthopaedics) and there was a sight variation in the ength of ED pacements, with some regions preferring the use of 6-month rotations to 4-month rotations; however, this variation occurred in a minority of cases. This agreed with the pattern reported in the Coins evauation. 22 We found that one of the main areas of variation was the educationa phiosophy of the FT programme hed by the educators and the programme impementers at trust eve. Nationa and regiona stakehoders communicated a view of the programme that buit on current probem-based medica education with experientia earning in the workpace, enabing deveopment of the competency and confidence of junior doctors. By contrast, severa TLs and some trainees emphasised the importance of specific training sessions on ED-reated topics to deveop trainee competency and confidence. These differences woud seem to undermine the purpose of FT, requiring the espoused needs of the programme to be fuy and Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 65

88 DISCUSSION TABLE 20 Summary of study findings Phase 1 stakehoder study Phase 2 ongitudina study of F2 doctors Nationa framework of FT in pace Variation in impementation Differences in educationa phiosophy Lack of cear end point to F2 Mixed views on forma assessments We-being focus ony on few in difficuty ED is a chaenging earning environment Disagreement on outcomes of training Increase in confidence managing common acute conditions Increase in competence performing common practica procedures Simiar a or better eves of anxiety and depression, stabe over year Higher a eves of job satisfaction with improvement over year Simiar a or ower eves of motivation, stabe over year Variation in work demands and roe carity across pacements Biggest increase in competence and confidence seen after ED pacement Increase in anxiety and effort and decrease in extrinsic job satisfaction associated with the ED pacement Outcome reviews suggest possibe reationship between trainee anxiety, onger patient episodes and seeing fewer patients; may offer simpe performance measure in the ED a Compared with normative and comparative data. widey articuated and debated both at trainee and at trust eve to move FT forward. This issue has been noted previousy 100 and woud appear not to have been resoved. Coins 22 concuded that the FT programme acked an articuated and accepted purpose, especiay in its second year, and we woud contend that differences in educationa phiosophy and impementation of agreed earning outcomes are a major reason that the purpose of FT is not ceary understood. This study shows that, according to the perceptions of F2 doctors, there is improvement in undertaking their work roe throughout the F2 period. The time spent in the ED had the greatest impact on their perceived confidence and competence compared with a other pacements that they experienced during the year. However, the infuence of the FT programme itsef is uncear given that we did not incude any contro areas not deivering FT for comparison. However, it is cear that, overa, F2 doctors perceived that their skis improved over time. Our study has moved the understanding of F2 doctors training forward by exporing deveopment across a range of pacements over time, in which F2 doctors experienced various work roes in diverse patterns (such as day, night, on-ca or shift working). By ooking at severa cohorts of F2 doctors we have been abe to enarge the scope of enquiry of postgraduate medica education compared with studies considering ony a singe speciaty or work pattern; this was caed for in the review by Scaan. 101 In addition, we have deveoped a simpe, sef-report measure to assess perceived confidence and competence during training, caed for by Mier and Archer. 102 Variation in impementation across organisations Our study demonstrated cross-sectiona evidence of variation between trusts in the number of hours of one-to-one and cose working contact with CSs, work demands and amount of teaching and support from CSs. These findings vaidate the comments of regiona stakehoders who noted variation in supervision and those of nationa stakehoders who were concerned that not a trainees were getting constructive feedback on their work. In their focus groups trainees demonstrated the importance of adequate supervision in enabing them to gain confidence in their decision-making abiities, especiay eary in a pacement, and comments about the quaity of support and supervision were frequenty incuded in the questionnaires with exampes of exceent and poor supervision being given. These data buid on reviews and studies (e.g. Scaan, 101 Marteau et a. 103 ) offering cear support for the need for and vaue of cose, supportive supervision, incuding feedback for trainees on their abiities, to enabe them to deveop confidence and competence as doctors. However, the study was not sufficienty powered to revea any objective evidence that reduced supervisor support impacted negativey on the perceived cinica competence and confidence of the doctors when directy questioned. It woud appear that two factors stand in the way of deveoping a strong framework of supervision within the workpace. The first is the difference in educationa phiosophy between medica educators, supervisors 66 NIHR Journas Library

89 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 and trainees that we have aready discussed. Second, but probaby the most important, is the fact that supervision and support are taking pace in the workpace and supervisors have to find the time and opportunity to undertake these activities, often during periods of busy service deivery. This is particuary an issue in time-pressured speciaties ike EM. This is an important resource issue associated with workpace education that seems to rest soey with trusts, who are aso responsibe for service deivery. It is hard to ignore the comment made by a busy consutant ooking at the impact that FT has had on his department: We are not broken yet but it is not far off we just need more senior staff. Ceary a more joined-up resource between medica training and service deivery needs to be expored, particuary with senior NHS trust staff, to enabe the appropriate eve of supervisory support to be deivered. Assessment of competence An obvious focus of the support and deveopment of trainees is the competion of their e-portfoio of competence. This study, aong with the evauation by Coins, 22 shows the deveopment of a nationa assessment programme. However, the stakehoder groups describe variation in impementation. Nationa and regiona stakehoders commented that the impementation of assessment was far from consistent across the regions and varied in quaity between departments and assessors. Regiona stakehoders noted that e-portfoios were being used by most trainees, athough there were some chaenges with regard to the avaiabiity and use of information technoogy in various trusts. TLs recognised the importance of assessment in experientia earning but noted that feedback required deivery on a 24-hour basis, necessitating the invovement of senior and midde-grade medica staff aong with senior and practitioner nursing staff. However, the vaue of actuay being abe to observe the trainee in structured situations was acknowedged. Trainees hed a baanced view of the nationa assessment process: some saw the vaue of gaining direct feedback on their competence and the opportunity to discuss individua cases with their supervisor; others fet that the format was too tick boxed and, athough aowing the examination of routine cinica cases, did not aow discussion of compex cases that might provide greater opportunities for earning. These findings are supported by the study by Hrisos et a., 104 which examined earning e-portfoio use in one deanery between 2004 and Athough direct comparisons with this study are difficut, it woud appear that trainees and supervisors have deveoped a itte more faith in the recording of experientia work-based earning in the e-portfoio. Ceary more deveopment work needs to be carried out with assessments to examine the baance between feedback and case discussion that is possibe in a busy working environment. One of the main criticisms of FT was the ack of a consistent framework to assess the competion of the F2 period of workpace training. Unike the end of F1 training, which is marked by registration with the GMC, a strong outcome as it invoves a shared responsibiity between the deanery and the trusts, 22 the end of F2 training reies on the sign off by TLs. Nationa stakehoders fear that this end stage acks focus and is not a sufficienty significant miestone for the junior doctors. Regiona stakehoders are concerned that there is not a consistent approach, as with assessment, and it is ikey that different sign-off criteria are required by different assessors. Quaitative comments from trainees woud encourage a view that they have sef-evauated their earning over the F2 period and have acknowedged the experiences that have benefitted their practice (see comments in The impact of the emergency department pacement). There was itte evidence from any of the stakehoder groups that F2 sign-off ed to the identification of specific issues or doctors in difficuty. It woud seem that more work is needed to deveop a nationay agreed scheme that marks the end of F2. We-being and motivation of foundation doctors Nationa and regiona stakehoders and TLs confirmed that there were no systems in pace to identify and support periods of overwork and strain for F2 doctors in genera but that there was provision for those few who had been designated as doctors in difficuty during their training programme. Our study was the first to systematicay examine a sampe of trainees at the end of their F1 training and throughout the F2 programme. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 67

90 DISCUSSION On average, trainees reported eves of anxiety and depression that were simiar to (T2) or better than (T1, T3, T4) those of a normative group of professiona and technica workers. Leves of anxiety and depression did not vary significanty over time. Reported eves of job satisfaction were, on average, significanty higher than those of a normative group of doctors and other NHS staff, and overa and intrinsic job satisfaction (associated with aspects of work such as freedom to choose their own method of working and opportunity to use their abiities) were seen to increase over the period of their FT. Trainees reported ower eves of motivationa effort than a group of genera managers and this did not vary significanty across their training. This study has demonstrated that it is possibe to systematicay record eves of we-being of trainee doctors and compare these with eves found in other normative studies, enabing appropriate interpretation. It is ikey that these measures coud be incorporated within trainees e-portfoios. Robust and stabe eves of we-being of foundation doctors from a number of NHS trusts are demonstrated in this study, describing a more positive picture of trainee we-being than has been found in previous singe-centre or singe-measure studies (e.g. Brennan et a., 105 Yates et a. 106 ). We examined various job-reated characteristics associated with we-being across the F2 period and found that there was significant variation in work demands and roe carity across the various time points and, therefore, pacements. These data were supported by quaitative comments from trainees who describe a variety of work roes: in some they are being stretched with a heavy workoad (e.g. being on-ca at night) whereas in others they have nothing to do and are not invoved in decision-making about patients. For exampe: I fee that my first F2 pacement was not representative of my experiences as a doctor to date, as it was a particuary difficut job. As the SHO on take for 3 busy admissions units I coud take up to 40 referras per 12 hour shift, and I spent a substantia amount of time doing paperwork, answering the beep and organising the ist rather than seeing patients. When the opportunity arose to earn a new ski (e.g. umbar puncture) I was constanty being caed away to do mundane tasks and training suffered as a resut. The consutants, registrars and other SHOs on the unit were exceent very approachabe and supportive it was just the nature of the job that ground me down it was a reentess stream of cerking without any feedback as to whether your initia management was correct. This was not the faut of anybody, it was just the nature of the work, but I did find it very dispiriting. I did earn from the 4 months, but I fet that my job was 98% service deivery with very itte training, hence why I have given quite negative feedback about it. By contrast, currenty I am working on POSU [postoperative surgica] and my job is exacty what I hoped for. I am earning new skis, I have time to do tasks propery and give patients high quaity care, the team are exceent, and I fee very we supported with penty of teaching and other opportunities for deveopment incuding ring-fenced training time. T2 participant I very much enjoyed my time in this pacement, but the hours were just very unsociabe. I think that more coud have been earned if the department was not so busy every patient that I saw, I fet I coud earn something from, but there was not the time for much teaching on the shop foor. T3 participant I had a rotation in XX was quite franky, dreadfu. I had three consutants (one working 50% for the trust) who I woud see for about an hour a day. I had no junior coeagues, hence no support. I had minima teaching, certainy no forma teaching in the job. I was argey on my own, in a meaningess roe (there is not much an F2 can do in a XX department and certainy very itte responsibiity or decisions you can take). I had no patients. T2 participant Simiary, in some pacements trainees are cear about what is expected of them within their work whereas in other pacements there is itte direction as to what they are expected to do and ambiguity over whether 68 NIHR Journas Library

91 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 or not they are part of the medica team responsibe for service deivery. Exampes show that some ongoing monitoring of the trainees work environment, as we have shown in this study, woud identify issues of work demands and roe carity and, if measured at an appropriate time, enabe changes to be made in pacements reporting ess than satisfactory earning outcomes. From this we can concude that the average junior doctor is ikey to have comparabe or better eves of we-being than other NHS workers and, on average, to expend a ower eve of effort than those in manageria roes. However, before we concude that trainees cope we with their FT, we shoud bear in mind that these are averaged data and within these groups there may be specific individuas who require additiona support at specific times and/or in certain pacements. A monitoring process using measures such as those within this study coud be incorporated into the e-portfoio, fufiing recommendations made by Borman 4 and Darsi. 6 Appication of these findings woud encourage reguar monitoring and review of junior doctor we-being (we note that there is a genera job satisfaction survey within the trainee annua survey used by deaneries; however, this woud seem to be concerned with specific training-reated outcomes). We see the vaue in two eves of assessment: at the pacement eve associated with the e-portfoio and at an organisationa eve within the NHS trust. This study has demonstrated an accurate and vaid method of assessing trainee we-being and motivation, the outcomes of which are required by both supervisors and trainees in order to adapt and deveop pacements as appropriate earning experiences. Therefore, these data shoud not be kept at deanery eve but communicated ocay and swifty to supervisors and trainees. A usefu method of assessing we-being is the trust s annua staff survey (provided by the Nationa NHS Staff Survey Co-ordination Centre). By adapting this survey to incude specific categories for F1 and F2 doctors, these doctors coud be identified at trust eve and their resuts compared with those of other doctors within that specific trust; in addition, nationa and between-trust comparisons woud aso be possibe. Incorporating assessment of junior doctor we-being within the human resource framework of each trust may increase oca ownership of FT. The impact of the emergency department pacement We were particuary interested in the impact of ED pacements on trainees competence and confidence and we-being and motivation. The ongitudina design of the study provided the opportunity to study a group of trainees who encountered the ED during their first, second or third F2 pacement. This study has shown that trainees dispay a significant increase in competence and confidence managing common conditions and performing routine procedures across F2, and the biggest increase in competence and confidence occurred immediatey after their ED pacement, regardess of when in the year the ED pacement occurred and what other pacements they experienced during the year. This finding was vaidated by quaitative comments from trainees who, athough noting difficuties reating to the high work demands, ong working hours and difficut shift patterns associated with the service deivery roe of EM, aso described the exceent supervision, fantastic earning experiences and exceent teamworking that increased their confidence as doctors. For exampe: Despite how stressfu I found A&E in the beginning, on the whoe it was a fantastic experience and I am definitey gad to have done it. The consutants and SpRs [speciaist registrars] at XX A&E are without exception briiant doctors to work with. T2 participant A&E was a very good cinica experience, and I am gad that I had the opportunity to undertake this pacement. It was difficut at times, particuary with regards to the rota/hours, but I have earnt a ot. T4 participant Emergency department pacements were aso associated with changes in we-being, motivation and the way that trainees worked. On average, the biggest increase in effort by trainees was shown in reation to Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 69

92 DISCUSSION their ED pacement compared with a other pacements during the year, indicating that they had to use up greater reserves from their energy poo during their ED pacement. Further, athough not significanty different, on average trainees biggest increase in anxiety was associated with their ED pacement. Quaitative commentary in the questionnaires and materia from trainees focus groups suggest that this anxiety stems from worry about making the right decision to admit or send home a patient, and it is this that trainees want the most feedback on and support for. In terms of work characteristics we found a significant reduction in task feedback during the ED pacement, which was not matched by the trainee need for feedback on their decisions from senior staff. This refects the busy and often unpanned nature of the workoad in EM, which often makes panning one-to-one time with trainees chaenging. Overa eves of job satisfaction and intrinsic job satisfaction were not seen to decrease during the ED pacement whereas some participants reported a significant decrease in eves of extrinsic job satisfaction at this time. Extrinsic satisfaction is reated to aspects such as pay, hours and conditions of work and these factors received a good dea of commentary from trainees, particuary with regard to ong shifts and antisocia hours of work, supported by data reveaing increased hours worked per week. The contrast between the demands of service deivery in EM and those in other pacements is often great, with some trainees coming into EM having never experienced working night shifts, for exampe. It is, therefore, perhaps not surprising that there are some reported changes in job satisfaction as a resut. Trainees ceary cope we with these difficuties as, with the exception of extrinsic satisfaction, there were no other significant differences in we-being, which remained favouraby comparabe with that of other heath-care workers. Increased socia support from coeagues during their ED pacement woud appear to hep trainees, as woud the carity of their roe in the ED, heping them know how to direct their efforts. Unfortunatey, these factors woud seem to go hand in hand with the nature of emergency working, which for some trainees woud have been their first taste of service deivery and a contrast to their previous year of training. This study has shown that there are considerabe benefits of the ED pacement in terms of competence and confidence gained by junior doctors. However, this does come at a cost of sighty increased anxiety and decreased extrinsic job satisfaction, athough the eves of these are comparabe to those of other doctors and heath-care workers. Carefu monitoring of trainees and good supervision with direct feedback on the quaity of referra decisions are vita at this time. However, it shoud be noted that this burden wi fa on hard-working staff with few additiona resources to offer. If the ED is adopted as a fixed rotation for F2 training, additiona senior resources in the ED shoud be carefuy considered. Quaity of care provided by foundation year 2 doctors The stakehoder study showed a eve of disagreement with regard to the quaity of care provided by F2 doctors. Regiona and nationa stakehoders suggested that F2 doctors were abe to provide adequate patient care and were better prepared in terms of their communication and procedura skis than postgraduates predating MMC. However, supervisors working in trusts found F2 doctors underprepared for service demands, having had itte previous experience of decision-making with acutey i patients. Trainees recognised their supervisors concerns, admitting to anxieties reating to decision-making, but agreed with the educationa stakehoders that they were abe to deiver adequate patient care at the end of their 4 months of training. Given these varying views of the adequacy of the quaity of care provided by F2 doctors we carried out a cinica case notes review of doctors invoved in the ongitudina study for their ED pacement. Foowing consutation, two conditions were seected for the case notes review: head injury and COPD. Overa, the quaity of care provided by the cohort of F2 doctors in this study during their ED pacements appears good, with mean scores from hoistic review ranging from 4.14 to 4.61 on a scae from 1 to 6. Examination of the spread of hoistic ratings across a records does highight some scope for improvement in a sma proportion of cases. The mean score for the proportion of review criteria met was > 50% for both head injury and COPD cases. Examination of the proportions of review criteria met highights aspects of care for which there is scope to improve the eve of detai recorded in patient case notes. Given the 70 NIHR Journas Library

93 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 importance of patient case notes for communication, audit and ega purpose, it is important that a reevant information is ceary documented. Assessment of quaity of care can focus on care outcomes or the process of care. Process measures such as case notes review are recommended as being more suitabe for judging quaity in heath care. 83 The iterature suggests that both criterion-based and hoistic review methods have strengths and weaknesses and shoud be regarded as compementary. 93 The findings show stronger correation coefficients for intercorreations between the various hoistic measures than for correations between criterion-based and hoistic review measures, despite the criterion-based review being carried out first for each record. This appears to support the view that hoistic review is assessing aspects of quaity not refected in criterion-based review 94 and a thorough assessment of performance shoud combine the two approaches. Furthermore, the inter-rater reiabiity resuts iustrate the benefit of enhanced reiabiity when using two reviewers and taking an average measure of quaity of care. The web-based data-coection too proved very efficient for coecting the review data such that the addition of automated data anaysis and output functions coud support performance review and feedback by CSs and potentiay peer and even sef-review by junior doctors. When we examined the reationship between we-being and motivation and the quaity-of-care outcomes there were no statisticay significant associations found, athough sma- to medium-sized correations were noted between motivation and work demands and hoistic care of head injury and COPD cases. Junior doctors reporting greater effort during their ED pacement were ikey to have better quaity-of-care outcomes for these two conditions. However, junior doctors who reported higher work demands were ikey to have ower quaity-of-care scores for these conditions. Detaied examination of performance data for 74 junior doctors working in the ED reveaed no statisticay significant associations with we-being; however, there was a suggestion of sma- to medium-sized correations between anxiety, time spent with patients and percentage of patients seen. This suggests a possibe reationship between anxious F2 doctors and onger patient episodes and seeing fewer patients within their ED pacement. Athough none of these reationships is statisticay significant, they are pausibe in busy working conditions and are in keeping with the comments of the supervisors during phase 1 of the study. This study indicates that these reationships shoud be investigated in more depth in arge-scae studies. However, these findings (ength of patient episode and number of patients seen) may offer a usefu and easy rubric for supervisors to use to identify trainees who are strugging with the EM working environment. There is aready anecdota evidence (S Mason, emergency medicine consutant, Sheffied Teaching Hospitas NHS Foundation Trust, 2011, persona communication) that such measures are aready being used informay to chart trainees progress in the ED. Limitations 1. The strength of this study is that it foowed a group of foundation doctors throughout their F2 training. However, this group consisted of 217 doctors and this may be considered a sma number considering the 8000 doctors currenty undertaking FT. Our sampe covered nine deaneries and 28 trusts in Engand. Because of confidentiaity constraints deaneries sef-seected to be part of this sampe, which may precude those that had particuar difficuties in we-being and motivation. However, the study achieved its intended sampe size of 210 doctors needed to address the primary outcome of the study (change in we-being over the year). The fact that these doctors were distributed over a range of EDs and NHS trusts probaby strengthens this study in terms of reporting these findings as generaisabe. 2. Participants expored the information about the study and sef-seected to be invoved. Therefore, it is possibe that the group of participants who made themseves avaiabe for this study had robust eves of we-being when joining the study. Nevertheess, the study was abe to track their we-being and motivation over a period of 12 months, exporing any changes that may have occurred during that time, and thereby fufiing one of its key objectives. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 71

94 DISCUSSION 3. The focus groups in the stakehoder study incuded F2 doctors from three arge EDs, two of which were teaching hospitas. There may have been differences between the views and experiences of F2 doctors in this study and the views and experiences of F2 doctors from smaer trusts. Further, our sampe of 10 PESs may be considered a sma sampe but it was sufficient for saturation of information. Our sampe of stakehoders from EDs mainy consisted of consutant TLs who were therefore at a senior grade and may have had specific work experiences. Further studies may benefit from incuding greater numbers of other staff who have a more informa roe in supervising and supporting F2 doctors, such as senior nursing staff and nurse practitioners. 4. Assessing quaity of care through case-note review is reiant on information being recorded in the notes, which may not refect every detai of the care provided. Competeness of the record is itsef a quaity issue and in this study poor recording is refected in ower quaity-of-care scores. This part of the study did not achieve its intended sampe of 144 doctors from 12 EDs. However, this is the first study of its kind to use case-note review to evauate quaity of care in the ED and aso to try and ink it with we-being and motivation amongst doctors. 5. The reiabiity anaysis found a high eve of agreement between reviewers in reation to the criterionbased review. The eve of agreement for the hoistic review of case notes was ower and was ess consistent across sites. Estabishing inter-rater reiabiity was chaenging as ethics and research governance constraints meant that reviewers coud review notes ony within their own hospita. Therefore, it was possibe to obtain a measure of inter-rater reiabiity ony within sites with more than one reviewer, to give a broad indication of consistency across reviewers. The number of records for review was seected to ensure that the reiabiity anaysis did not outweigh the main reviewing activity and was based on a previous study of quaity in emergency care using hoistic review that had achieved moderate to strong reiabiity ( ) using 14 records. 88 The higher eve of agreement in that study may have been achieved because the records examined were more concise than the ED records in this study. Future study 1. Further examination of quaity-of-care outcomes and junior doctors we-being and motivation. Future studies woud need to be arge-scae, muticentre studies to provide sufficient power to examine possibe reationships. 2. More arge-scae studies ooking at assessment of competence, feedback and case discussion conducted by a range of heath-care staff may yied further good practice that can be incorporated into the FT assessment programme. Impications for practice 1. Disseminate the findings of this study to encourage more genera support for work-based earning and assessment as part of postgraduate medica education, especiay to organisations such as the UK Foundation Programme Board. We woud seek nationa communication of the findings so that participating trusts can earn of the findings through conferences such as the Heath Services Research Network annua symposium and the NHS Confederation conferences. 2. Trainees eves of we-being and motivation can be measured accuratey over time and woud form an appropriate part of the e-portfoio, but this woud require timey feedback to supervisors to enabe appropriate work demands and roe carity to be determined within the pacement period. If this service cannot be provided within a usefu time frame a trainee report measure regarding their we-being, work demands and roe carity and use of their abiities shoud be communicated to the trainees and their supervisors, enabing oca changes to be made to pacements. There is a we-vaidated system for recording we-being amongst NHS staff (the Nationa NHS Staff Survey 107 ) and this woud be utiised to specificay identify and benchmark the we-being of foundation doctors. However, it woud need to be acknowedged that this is an annua review and not as accurate as pacement measures. 72 NIHR Journas Library

95 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO This study offers cear evidence that a F2 doctors woud gain in confidence and competence from an ED pacement; however, this shoud be accompanied by additiona support 17 for senior staff to enabe them to provide the eve of support that trainees need during this intense earning period. In addition, more consideration needs to be given to work ife baance issues during this pacement period. 4. The success of workpace earning depends on the provision of adequate eves of supervision and support of trainees. The exact eve of senior support needs to be determined by working cosey with senior staff and supervisors. 22 This by necessity wi not be one-size-fits-a as it wi depend on a number of factors associated with service deivery and requires consutation with both the Foundation Programme and the trusts invoved. 5. Consideration shoud be given to debating the espoused educationa phiosophy of medica training (as probem-based education supported by workpace experientia earning) to articuate a cear and understood purpose of FT, enabing the impementation of agreed earning outcomes with supervisors and trainees. 6. Further studies shoud be carried out on work-based assessment, with cose examination and deveopment of specific criteria that ead to a nationa scheme marking the end of F2 training. 7. Carefu consideration shoud be given to incorporating forma processes for careers advice at both the F1 and the F2 points in training to ensure that foundation doctors acquire the most appropriate training for their intended career track. Athough the benefits of ED pacements are acknowedged, this may not aways be the case when intended career tracks invove service speciaties such as aboratory medicine and radioogy. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 73

96

97 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Acknowedgements We woud ike to thank the foowing for their contribution to the research: staff at the participating research sites, particuary staff at PMDs, FSs and EDs, who agreed to participate and provided assistance with contacting the study participants and their recruitment the speciaist trainee reviewers who took part in the cinica case notes review the members of the steering group committee for their contribution to the management of the study. Contribution of authors Suzanne Mason, Coin O Keeffe, Angea Carter, Rache O Hara and Chris Stride were the grant hoders and managed the main parts of the study. For phase 1, Angea Carter, Coin O Keeffe and Suzanne Mason undertook the quaitative interviews and the anayses. For phase 2, Coin O Keeffe, Suzanne Mason, Angea Carter and Chris Stride designed and impemented the ongitudina study and Chris Stride undertook the anayses for the ongitudina study. Rache O Hara, Coin O Keeffe and Suzanne Mason designed and impemented the cinica case notes review and Rache O Hara and Chris Stride undertook the anayses for the cinica case notes review. Coin O Keeffe, Angea Carter and Rache O Hara wrote the drafts of the report and Suzanne Mason, Chris Stride and Angea Carter edited the fina report. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 75

98

99 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 References 1. Department of Heath. The NHS knowedge and skis framework (NHS KSF) and the deveopment review process. London: Department of Heath; URL: nationaarchives.gov.uk/+/ pubicationspoicyandguidance/dh_ (accessed 28 June 2013). 2. Department of Heath. The NHS pan. A pan for investment. A pan for reform. London: The Stationary Office; Back C. Working for a heathier tomorrow. London: The Stationary Office; Boorman S. NHS heath and we-being review fina report. London: Department of Heath; Van Stok C, Starkey T, Shehabi A, Hassan E. NHS workforce heath and we-being review: staff perception research. In NHS heath and we-being, the Boorman review. London: Department of Heath; Department of Heath. A high quaity workforce: NHS next stage review. London: Department of Heath; Nationa Institute for Heath and Cinica Exceence. Promoting menta webeing through productive and heathy working conditions: guidance for empoyers. London: NICE; Karasek R, Theore T. Heathy work: stress, productivity and the reconstruction of working ife. New York, NY: Basic Books; Parker S, Wa T. Job and work design: organizing work to promote we-being and effectiveness. Advanced topics in organizationa behavior. Thousand Oaks, CA: Sage Pubications; Hewitt C, Lankshear A, Maynard A, Shedon T, Smith K. Heath service workforce and heath outcomes: a scoping study. London: Nationa Co-ordinating Centre For Service Deivery and Organisation; Michie S, West MA. Managing peope and performance: an evidence based framework appied to heath service organizations. Int J Manag Rev 2004;5: j x 12. Gro R, Mokkink H, Hesper-Lucas A, Tieens V, Bute J. Effects of the vocationa training of genera practice consutation skis and medica performance. Med Educ 1989;23: Tharenou P, Burke E. Training and organizationa effectiveness. In Robertson I, Cainan M, Bartram D, editors. Organisationa effectiveness: the roe of psychoogy. Chichester: John Wiey; pp West E. Management matters: the ink between hospita organisation and quaity of patient care. Qua Heath Care 2001;10: West MA, Borri CS, Dawson J, Scuy J, Carter M, Aneay S, et a. The reationship between staff management practices and patient mortaity in acute hospitas: a ongitudina study. Int J Human Resource Manag 2003;13: Wa TD, Boden RI, Borri CS, Carter AJ, Goya DA, Hardy GE, et a. Minor psychiatric disorder in NHS trust staff: occupationa and gender differences. Br J Psychiatry 1997;171: Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 77

100 REFERENCES 17. Donadson L. Unfinished business: proposas for reform of the senior house officer grade. London: Department of Heath; Cough C. Modernising Medica Careers: effect on NHS service deivery. Hosp Med 2005; 66: Nationa Heath Service. Modernising Medica Careers. URL: (accessed 7 August 2013). 20. Academy of Medica Roya Coeges Foundation Programme Committee. Foundation programme curricuum London: Academy of Medica Roya Coeges; URL: (accessed 7 August 2013). 21. Tooke SJ. Aspiring to exceence: findings and recommendations of the independent inquiry into Modernising Medica Careers ed by Sir John Tooke. London: MMC Inquiry; Coins JF. Foundation for exceence an evauation of the foundation programme. London: Medica Education; NHS Management Executive. Junior doctors. The new dea. London: NHS Management Executive; Tempe J. Time for training: a review of the impact of the European Working Time Directive on the quaity of training. London: Medica Education; Firth-Cozens J. Emotiona distress in junior house officers. Br Med J (Cin Res Ed) 1987;295: Reuben DB. Depressive symptoms in medica house officers. Effects of eve of training and work rotation. Arch Intern Med 1985;145: McPherson S, Hae R, Richardson P, Obhozer A. Stress and coping in accident and emergency senior house officers. Emerg Med J 2003;20: Burbeck R, Coomber S, Robinson SM, Todd C. Occupationa stress in consutants in accident and emergency medicine: a nationa survey of eves of stress at work. Emerg Med J 2002;19: Capan RP. Stress, anxiety, and depression in hospita consutants, genera practitioners, and senior heath service managers. BMJ 1994;309: Godberg DP, Hiier VF. A scaed version of the Genera Heath Questionnaire. Psycho Med 1979;9: Armstrong PA, White AL, Thakore S. Senior house officers and foundation year doctors in emergency medicine: do they perform equay? A prospective observationa study. Emerg Med J 2008;25: Armstrong PA, White AL, Thakore S. Reduced productivity among junior trainees in the emergency department and the impact on senior cinicians. Emerg Med J 2010;27: Eager R, Banks M. Transition to the foundation programme: does it affect the numbers of patients seen by SHOs? Emerg Med J 2006;23: emj Whiticar R, Webb H, Smith S. Re-attendance to the emergency department. Emerg Med J 2008;25: Croft SJ, Kuhrt A, Mason S. Are today s junior doctors confident in managing patients with minor injury? Emerg Med 2006;23: NIHR Journas Library

101 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO Croft SJ, Mason S. Are emergency department junior doctors becoming ess experienced in performing common practica procedures? Emerg Med J 2007;24: /emj Smith G, Popett N. Knowedge of aspects of acute care in trainee doctors. Postgrad Med J 2002;78: Patterson M, Warr P, West M. Organizationa cimate and company productivity: the roe of empoyee affect and empoyee eve. J Occup Organ Psycho 2004;77: / Morey JC, Simon R, Jay GD, Wears RL, Saisbury M, Dukes KA, et a. Error reduction and performance improvement in the emergency department through forma teamwork training: evauation resuts of the MedTeams project. Heath Serv Res 2002;37: / Hardy GE, Woods D, Wa TD. The impact of psychoogica distress on absence from work. J App Psycho 2003;88: Wiiams S, Dae J, Gucksman E, Weesey A. Senior house officers work reated stressors, psychoogica distress, and confidence in performing cinica tasks in accident and emergency: a questionnaire study. BMJ 1997;314: Braun V, Carke V. Using thematic anaysis in psychoogy. Qua Res Psycho 2006;3: Bryman A, Be E. Business research methods. Oxford: Oxford University Press; Carter A, Wood S, Goodacre S, Sampson F, Stabes R. Evauation of workforce and organizationa issues in estabishing primary angiopasty in Engand. J Heath Serv Res Poicy 2010;15: King N. Tempate anaysis. In Symon G, Casse C, editors. Quaitative methods and anaysis in organizationa research: a practica guide. Thousand Oaks, CA: Sage Pubications; pp Warr PB, Cook JD, Wa TD. Scaes for the measurement of some work attitudes and aspects of psychoogica we-being. J Occup Psycho 1979;52: j tb00448.x 47. Warr P, Capperton G. The joy of work? Jobs, happiness and you. Sussex: Routedge; Heath Services Research, Schoo of Heath and Reated Research (ScHARR), University of Sheffied. The EDiT study. URL: (accessed 12 Juy 2013). 49. Department of Heath. A&E attendances. URL: / Performancedataandstatistics/AccidentandEmergency/DH_ (accessed 11 June 2013). 50. Costa PT, McCrae RR. Revised NEO personaity inventory (NEO PI-R) manua (UK edition). Oxford: Hogrefe; Ones DS, Dichert S, Viswesvaran C, Judge TA. In support of personaity assessment in organizationa settings. Person Psycho 2007;60: j x 52. Tobin DL, Horoyd KA, Reynods RV, Wiga JK. The hierarcha factor structure of the Coping Strategies Inventory. Cognit Ther Res 1989;13: Lazararus RS, Fokman S. Stress, appraisa and coping. New York, NY: Springer; Warr P. Work, unempoyment and menta heath. Oxford: Oxford University Press; Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 79

102 REFERENCES 55. Warr PB. The measurement of we-being and other aspects of menta heath. J Occup Psycho 1990;63: Warr PB. Decision atitude, job demands, and empoyee we-being. Work Stress 1990;4: Van Dierendonck D, Haynes C, Borri C, Stride C. Leadership behavior and subordinate we-being. J Occup Heath Psycho 2004;9: Muarkeys WT, Warr PB, Cegg CS, Stride C. Measures of job satisfaction, menta heath and job-reated we-being: a bench-marking manua. Sheffied: University of Sheffied; Stride C, Wa TD, Catey N. Measures of job satisfaction, organisationa commitment, menta heath and job reated we-being: a benchmarking manua. Chichester: John Wiey; Pritchard E, Ashwood EL. Managing motivation: a manager s guide to diagnosing and improving motivation. New York, NY: LEA/Psychoogy Press; Cammann C, Fichman M, Jenkins D, Kesh J. The Michigan organizationa assessment questionnaire. Ann Arbor, MI: University of Michigan; Mason SOKC, Coeman P, O Hara R, Dixon S, Rick J. A muti centre community intervention tria to evauate the cinica and cost effectiveness of emergency care practitioners. London: Nationa Co-ordinating Centre for NHS Service Deivery and Organisation (NCCSDO); Borri CS, Wa TD, West MA, Hardy GE, Shapiro DA, Carter A, et a. Menta heath of the workforce in NHS Trusts. Sheffied: Institute of Work Psychoogy, University of Sheffied; Borri CS, West MA, Wa TD, Shapiro DA, Haynes CE, Stride C, et a. Stress among staff in NHS trusts. Sheffied: Institute of Work Psychoogy, University of Sheffied; Hipwe AE, Tyer PA, Wison CM. Sources of stress and dissatisfaction among nurses in four hospita environments. Br J Med Psycho 1989;62: j tb02812.x 66. Richardsen AM, Burke RJ. Occupationa stress and job satisfaction among Canadian physicians. Work Stress 1991;5: Capan RD. Organizationa stress and individua strain: a socia-psychoogica study of risk factors in coronary heart disease amongst administrators, engineers and scientists. Ann Arbor, MI: University of Michigan, Institute for Socia Research; Kahn RL, Wofe DM, Quinn RP, Sneok JD, Rosentha RA. Organizationa stress: studies in roe confict and ambiguity. New York, NY: Wiey; Hackman J, Odham GR. Motivation through the design of work: test of theory. Organ Behav Hum Perform 1976;15: Cordery JL, Wa TD. Work design and supervisory practice: a mode. Hum Reat 1985;38: Hackman J, Odham GR. Deveopment of the job diagnostic survey. J App Psycho 1975; 60: Rizzo J, House RJ, Lirtzman SJ. Roe confict and ambiguity in compex organizations. Admin Sci Q 1970;15: Keoway EK, Baring J. Job characteristics, roe stress and menta heath. J Occup Psycho 1991;64: House J. Work stress and socia support. Reading, MA: Addison-Wesey Pubishing; NIHR Journas Library

103 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO Abde-Haim AA. Individua and interpersona moderators of empoyee reactions to job characteristics: a re-examination. Person Psycho 1982;32: Capan RD, Cobb S, French JRP, Van Harrison RV, Pinneau SR. Job demands and worker heath. Washington, DC: Nationa Institute for Occupationa Safety and Heath; Capan RD, Cobb S, French JRP, Van Harrison RV, Pinneau SR. Job demands and worker heath. Ann Arbor, MI: University of Michigan, Institute for Socia Research; Maakh-Pines A, Aronson E, Kafry D. Burnout. New York, NY: Free Press; Cohen S, Wis TA. Stress, socia support, and the buffering hypothesis. Psycho Bu 1985;98: Dwyer DJ, Ganster DC. The effects of job demands and contro on empoyee attendance and satisfaction. J Organ Behav 1991;12: Haynes CE, Wa TD, Boden RI, Rick JE. Measures of perceived work characteristics for heath service research: test of a measurement mode and normative data. Br J Heath Psycho 1999;4: Sziagyi AD Jr, Sims HP Jr, Keer RT. Roe dynamics, ocus of contro, and empoyee attitudes and behavior. Acad Manag J 1976;19: Liford RJ, Brown CA, Nicho J. Use of process measures to monitor the quaity of cinica practice. BMJ 2007;335: Hiatt HH, Barnes BA, Brennan TA, Laird NM, Lawthers AG, Leape LL, et a. A study of medica injury and medica mapractice. N Eng J Med 1989;321: NEJM Woff AM. Limited adverse occurrence screening: using medica record review to reduce hospita adverse patient events. Med J Aust 1996;164: Woff AM, Bourke J. Detecting and reducing adverse events in an Austraian rura base hospita emergency department using medica record screening and review. Emerg Med J 2002;19: Sari AB, Shedon TA, Crackne A, Turnbu A, Dobson Y, Grant C, et a. Extent, nature and consequences of adverse events: resuts of a retrospective casenote review in a arge NHS hospita. Qua Saf Heath Care 2007;16: O Hara R, O Keeffe C, Mason S, Coster JE, Hutchinson A. Quaity and safety of care provided by emergency care practitioners. Emerg Med J 2012;29: emj Brown C, Hofer T, Joha A, Thomson R, Nicho J, Frankin BD, et a. An epistemoogy of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement. Qua Saf Heath Care 2008;17: qshc Hutchinson A, McIntosh A, Anderson J, Gibert C, Fied R. Deveoping primary care review criteria from evidence-based guideines: coronary heart disease as a mode. Br J Gen Pract 2003;53: Camacho LA, Rubin HR. Assessment of the vaidity and reiabiity of three systems of medica record screening for quaity of care assessment. Med Care 1998;36: / Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 81

104 REFERENCES 92. Mohammed MA, Mant J, Bentham L, Stevens A, Hussain S. Process of care and mortaity of stroke patients with and without a do not resuscitate order in the West Midands, UK. Int J Qua Heath Care 2006;18: Liford R, Edwards A, Giring A, Hofer T, Di Tanna GL, Petty J, et a. Inter-rater reiabiity of case-note audit: a systematic review. J Heath Serv Res Poicy 2007;12: / Hutchinson A, Coster JE, Cooper KL, McIntosh A, Waters SJ, Bath PA, et a. Comparison of case note review methods for evauating quaity and safety in heath care. Heath Techno Assess 2010;14(10). 95. Hutchinson A, Coster JE, Cooper KL, McIntosh A, Waters SJ, Bath PA, et a. Assessing quaity of care from hospita case notes: comparison of reiabiity of two methods. Qua Saf Heath Care 2010;19:e Nationa Institute for Heath and Cinica Exceence. Head injury. Triage, assessment, investigation and eary management of head injury in infants, chidren and aduts. London: NICE; Nationa Institute for Heath and Cinica Exceence. Chronic obstructive pumonary disease. Management of chronic obstructive pumonary disease in aduts in primary and secondary care. London: NICE; Zegers M, de Bruijne MC, Spreeuwenberg P, Wagner C, Groenewegen PP, van der Wa G. Quaity of patient record keeping: an indicator of the quaity of care? BMJ Qua Saf 2011;20: Hofer TP, Asch SM, Hayward RA, Rubenstein LV, Hogan MM, Adams J, et a. Profiing quaity of care: is there a roe for peer review? BMC Heath Serv Res 2004;4: O Brien M, Brown J, Ryand I, Shaw N, Chapman T, Giies R, et a. Exporing the views of second-year foundation programme doctors and their educationa supervisors during a deanery-wide piot foundation programme. Postgrad Med J 2006;82: Scaan S. Education and the working patterns of junior doctors in the UK: a review of the iterature. Med Educ 2003;37: Mier A, Archer J. Impact of workpace based assessment on doctors education and performance: a systematic review. BMJ 2010;341:c Marteau TM, Wynne G, Kaye W, Evans TR. Resuscitation: experience without feedback increases confidence but not ski. BMJ 1990;300: Hrisos S, Iing JC, Burford BC. Portfoio earning for foundation doctors: eary feedback on its use in the cinica workpace. Med Educ 2008;42: j x 105. Brennan N, Corrigan O, Aard J, Archer J, Barnes R, Beakey A, et a. The transition from medica student to junior doctor: today s experiences of tomorrow s doctors. Med Educ 2010;44: Yates PJ, Benson EV, Harris A, Baron R. An investigation of factors supporting the psychoogica heath of staff in a UK emergency department. Emerg Med J 2012;29: /emj Nationa NHS Staff Survey. URL: (accessed 22 November 2013). 82 NIHR Journas Library

105 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 1 Schematic of current postgraduate training in the UK foowing the Tooke report 21 Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 83

106 APPENDIX 1 Process Quaifications Examinations Medica student Medica schoo 4 6 years Postgraduate training Generaist training Speciaist training Foundation program 2 years Singe appication from medica schoo Curricuum approved by reguator Core speciaist training Higher speciaist training F1 year F2 year Medica degree GMC icence after year 1 84 NIHR Journas Library

107 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 2 Summary of key components of the Foundation Programme Team assessment of behaviour CS end-of-pacement report ES end-of-pacement report Mini-CEX DOPs Case-based discussion Deveoping the cinica teacher Mini-PAT Mutisource feedback given to trainee from a number of mutiprofessiona coeagues. Previousy known as 360-degree feedback A summary of the progress and achievements of the foundation doctor foowing a meeting to discuss these with the CS Comprising information from both the CS s end-of-pacement report and the e-portfoio An evauation of an observed cinica encounter An observation of a doctor s interaction with a patient whist they carry out a specific cinica procedure A retrospective discussion between a supervisor and a trainee regarding a specific cinica case managed by the trainee An assessment too to deveop a trainee s skis in teaching and presentation The mini-pat provides feedback from a range of co-workers across the domains of good medica practice. Using Portfoio Onine a these can be mapped to the core objectives of the curricuum. The PMETB and the GMC have identified peer ratings as suitabe for postgraduate assessment and revaidation evidence a Accessed 22 November For more information see the Foundation Programme Curricuum ( foundation-doctors). Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 85

108

109 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 3 Nationa stakehoder interview schedue Interview questions for nationa stakehoders in postgraduate medica education (phase 1) Good morning/afternoon, my name is Angea Carter/Suzanne Mason. I woud ike to thank you for giving me some of your vauabe time for this interview. The interview is part of a nationa evauation of FY2 doctor we-being. The study is being carried out over 3 years and is examining different arrangements for the emergency departments. In particuar, we are interested in the infuence of we-being and motivation on patient care. We sent you an information sheet about the study when we arranged the interview woud you ike to see this again? I have sent/wi forward a consent form for you to sign and return to us. Are you happy to sign this? This particuar interview is one of a series we are conducting with stakehoders in postgraduate medica education at a nationa and regiona eve who are in a position to infuence decision-making, particuary the FY2 training agenda. The aim of the interview is to expore your perceptions and experiences of FY2 training nationay. I d ike to assure you that anything that you say to me/us today wi be treated in confidence. NO individua wi be identified. I ony have your name to note that I have competed my interviews as panned. The interview wi take about 60 minutes and I have a copy of the questions for you to have a ook at to hep you. Is there anything you woud ike to ask before we begin? (5 minutes) Background 1. Briefy describe your own roe in reation to FY2 training? (3 minutes) Prompt: Can you describe the input that you have in the panning and impementation of FY2 training. Approach to postgraduate medica education 2. Can you describe to what extent there is a nationa strategic approach to FY2 training in Engand/Waes/ Scotand? (5 minutes) Prompt: Who is responsibe? Prompt: What mechanisms exist for impementing a nationa approach? Prompt: Poicies, poicy documents, conferences and curricuum? Prompt: Nationay how many trainees are there? Is this sufficient? Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 87

110 APPENDIX 3 3. What are the chaenges that exist in the impementation of a nationa strategic approach for FY2 training? (5 minutes) Prompt: What are the ikey successes? How are these measured? Prompt: What are the ikey faiures? How are these measured? Prompt: Issues specificay for the ED? Variation in training 4. To what degree is there variation in the impementation of postgraduate training at the regiona eve? (5 minutes) Prompt: Is there variation at foundation schoo eve? Prompt: What do you think are the causes of variation in impementing training? Quaity 5. What are the criteria for success for FY2 doctors training? (7 minutes) Prompt: How are these measured? Prompt: How we are these met? (in terms of patient care and performance) Prompt: Are they sufficient? (in terms of patient care and performance) Prompt: Is training providing FY2 doctors that are fit for purpose? 6. What quaity assurance mechanisms exist for FY2 training at a nationa eve? (5 minutes) Prompt: e.g. Feedback from heath organisations, doctors? Prompt: Are these sufficient? If issues are noted what changes are made? (35 minutes to this point) (Sue takes over) We-being and motivation of FY2s 7. How is the we-being and motivation of FY2 doctors evauated in their training? (8 minutes) Prompt: How is this assessed, when and by whom? (Is it part of the assessment process?) Prompt: If there are issues of we-being or motivation how are these addressed? Prompt: Differentiate between oca and strategic responses. Prompt: Is the potentia impact on patient care considered? 88 NIHR Journas Library

111 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Career deveopment 8. Do you fee there is sufficient career deveopment buit in to the training so that FY2s can buid their careers on a firm basis? (7 minutes) Prompt: Is there imitation of choice for doctors? Prompt: Are they speciaising too eary? Prompt: Are doctors sotted in to operationa arrangements and miss out on the bigger picture? (50 minutes to this point) Future deveopments 9. Can you describe any deveopments that wi occur in the near future that may affect FY2 training? (5 minutes) Prompt: Regiona or nationa basis? Prompt: Are there any changes you woud personay ike to see made to postgraduate medica training? Ending Is there any information you woud ike to add that woud enabe us to understand the current and panned future state of postgraduate medica training? Thank you for your vauabe time we wi be sending you a copy of our fina report. (5 minutes) Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 89

112

113 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 4 Regiona stakehoder interview schedue Interview questions for regiona stakehoders in postgraduate medica education (phase 1) Good morning/afternoon, my name is Angea Carter/Suzanne Mason/Coin O Keeffe. I woud ike to thank you for giving me some of your vauabe time for this interview. The interview is part of a nationa evauation of FY2 doctor we-being. The study is being carried out over 3 years and is examining different arrangements for the emergency departments. In particuar, we are interested in the infuence of we-being and motivation of the FY2 doctors on patient care. We sent you an information sheet about the study when we arranged the interview woud you ike to see this again? I have sent/wi forward a consent form for you to sign and return to us. Are you happy to sign this? This particuar interview is one of a series we are conducting with stakehoders in postgraduate medica education at a nationa and regiona eve who are in a position to infuence decision-making, particuary around the FY2 training agenda. The aim of the interview is to expore your perceptions and experiences of FY2 training regionay. I d ike to assure you that anything that you say to me/us today wi be treated in confidence. NO individua wi be identified. I ony have your name to note that I have competed my interviews as panned. The interview wi take about 55 minutes and I have a copy of the questions for you to have a ook at to hep you. Is there anything you woud ike to ask before we begin? (5 minutes) Background 1. Briefy describe your own roe in reation to FY2 training? (2 minutes) Prompt: Can you describe the input that you have in the panning and impementation of FY2 training? Panning and impementation (regiona eve) 2. How is FY2 training panned and impemented at the regiona eve? (5 minutes) Prompt: Individuas responsibe, poicy? Prompt: Is this approach consistent with the nationa strategic approach or does it differ in any way? Prompt: How many trainees are there? How many trusts are invoved? Is this sufficient for your regiona area? 3. Are the pacement organisations invoved in the panning of training? (3 minutes) Prompt: If so, how? Prompt: Meetings, visits, joint bodies? Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 91

114 APPENDIX 4 Variation 4. Is there variation in the impementation of foundation training across the regions? (5 minutes) Prompt: What are the points of variation (e.g. pacement ength)? Prompt: What are the causes of variation? (20 minutes to this point) Quaity 5. What are the key training outcomes for FY2 doctors? (5 minutes) Prompt: How usefu is the mentor/educationa supervisor roe for you in making an assessment of the junior doctors? Prompt: Is there variation in educationa supervisors? Prompt: Are these the same criteria for signing off the successfu competion of a pacement? If not, why not? Prompt: Is the e-portfoio used? (Sue takes over) 6. To what extent is FY2 training providing doctors that are fit for purpose? (5 minutes) Prompt: How usefu are the forma assessments to assess FY2s abiity? Prompt: Are doctors sotted in to operationa arrangements and miss out on the bigger picture? This repeated ater in career deveopment? Prompt: Are there any specific issues for the ED? Prompt: What type of issues are you finding with FY2 doctor training? Did these happen before foundation training? (if a specific issue is mentioned ask them to describe the exampe fuy) 7. What quaity assurance mechanisms exist for foundation training in your region? (5 minutes) Prompt: e.g. Feedback from heath organisations, doctors? Prompt: Are these sufficient? If issues are noted what changes are made? (35 minutes to this point) We-being and motivation of FY2s 8. How is the we-being and motivation of FY2 doctors evauated in their training? (10 minutes) Prompt: How is this assessed, when and by whom? (Is it part of the assessment process?) Prompt: In your experience what motivates FY2 doctors? 92 NIHR Journas Library

115 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Prompt: What aspects of work do FY2s strugge with? Prompt: How is the potentia impact on patient care considered? Career deveopment 9. Is sufficient career deveopment buit in to the training so that FY2s can buid their careers on a firm basis? (5 minutes) Prompt: Is there imitation of choice for doctors? Prompt: Are they speciaising too eary? Prompt: Are doctors sotted in to operationa arrangements and miss out on the bigger picture? (50 minutes to this point) Future deveopments 10. What deveopments wi occur in the near future that may affect foundation training? (5 minutes) Prompt: Are there any changes you woud personay ike to see made? Prompt: Changes that may infuence patient care? Prompt: Changes that may infuence FY2s we-being and motivation Ending Is there any information you woud ike to add that woud enabe us to understand the current and panned future state of postgraduate medica training? Thank you for your vauabe time we wi be sending you a copy of the Executive Summary of our fina report. (5 minutes) Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 93

116

117 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 5 Trainer stakehoder interview schedue Interview questions for consutants/training eads in emergency departments (phase 1) Good morning/afternoon, my name is [Researcher]. I woud ike to thank you for giving me some of your vauabe time for this interview. The interview is part of a nationa evauation of FY2 doctor training. The study is being carried out over 3 years and is examining different arrangements for the impementation of training in emergency departments. In particuar, we are interested in the infuence of we-being and motivation on patient care. We sent you an information sheet about the study when we arranged the interview woud you ike to see this again? I have sent/wi forward a consent form for you to sign and return to us. Are you happy to sign this? This particuar interview is one of a series we are conducting with stakehoders in training at a regiona and oca eve who are in a position to infuence decision-making around postgraduate medica training, and particuary the FY2 training agenda. The aim of the interview is to expore your perceptions and experiences of FY2 training ocay. I d ike to assure you that anything that you say to me/us today wi be treated in confidence. NO individua wi be identified. I ony have your name to note that I have competed my interviews as panned. The interview wi take about 60 minutes and I have a copy of the questions for you to have a ook at to hep you. Is there anything you woud ike to ask before we begin? (5 minutes) Background 1. Briefy describe your own roe in reation to FY2 training pease? (3 minutes) Prompt: How are you brought into contact with FY2s working in your department? Training 2. Can you describe the training FY2 doctors receive in your ED? (4 minutes) Prompt: What is the roe of the junior doctors in the ED? Prompt: How many trainees are there? Is this sufficient or is it too many? 3. Has there been any changes to the way training in the ED has been provided in recent years? (3 minutes) Prompt: What is the impact of changes to training (e.g. changes to way you deiver care, workoad and supervision)? 4. Do you think the FY2 doctor training is adequate preparation for them to deiver the care they are expected to provide? (5 minutes) Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 95

118 APPENDIX 5 5. In what way do you think training can be improved? (5 minutes) (25 minutes to this point) Quaity 6. How we do assessments measure the abiity of FY2s to deiver good cinica care? (5 minutes) 7. How are FY2s assessed? (5 minutes) (35 minutes to this point) We-being and motivation of FY2s 8. How is the we-being and motivation of FY2 doctors evauated in their training? (10 minutes) Prompt: How is this assessed, when and by whom? (Is it part of the assessment process?) Prompt: In your experience what motivates FY2 doctors? Prompt: What aspects of work do FY2s strugge with? Prompt: Is the potentia impact on patient care considered? Prompt: Do FY2s integrate into the ED? 9. What cinica support or supervision do the FY2 doctors get in the ED? (5 minutes) Prompt: Do you consider the cinica support to be adequate or is too much support/supervision required? Prompt: What other types of support do FY2s receive (peers, occupationa heath)? Prompt: What support shoud they receive? Impact on care 10. To what extent do you fee FY2s in your ED contribute to the provision of a good quaity service? (5 minutes) Prompt: Issues such as workoad, work rate, independent working. Prompt: Are there any improvements that coud be made to the way FY2s work in your ED? Give exampes. Ending Is there any information you woud ike to add that woud enabe us to understand the current and panned future state of FY2 training? Thank you for your vauabe time we wi be sending you a copy of the Executive Summary of our fina report. (5 minutes) 96 NIHR Journas Library

119 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 6 Letter of invitation to postgraduate medica education stakehoders and emergency department consutants/training eads for participation in phase 1 interviews Sent to staff via NHS service address or via e-mai. Dear [name of individua], Tite of project: The EDiT Study: Evauation of Doctors in Training (phase 1: consutation and scoping study) I am writing to ask if you woud kindy consider taking part in an interview for a nationa research study evauating the experiences of foundation year 2 (FY2) doctors and how this impacts on their we-being, motivation and the quaity of care they provide. I am contacting you because the organisation where you work is invoved in the study and you have been identified because you are invoved in FY2 training and may be interested in participating. The study is funded for 33 months. Phase 1 asts for 5 months and incudes a consutation and scoping study in up to four postgraduate deaneries and EDs in Engand. Interviews with key stakehoders in these organisations are part of this consutation study and I am asking you to consider taking part in one of these interviews. I am attaching some further information about the research and what agreeing to take part in the interviews may invove for you. I woud be gratefu if you coud read this carefuy. I hope this is cear but if you have any further questions pease contact the project manager Coin O Keeffe on or emai him at c.okeeffe@sheffied.ac.uk. If after reading the information eafet and discussing this with others you fee happy to participate then pease emai the project manager who wi contact you regarding setting up the interview. If you woud rather not take part then thank you for your time and we wi not contact you again. Yours sincerey Signed Suzanne Mason Lead Investigator Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 97

120

121 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 7 Participant information sheet for phase 1 interviews The EDiT study: Evauation of Doctors in Training: phase 1 interviews with postgraduate medica education stakehoders/ ED consutants (training eads) Participant information sheet We woud ike to invite you to take part in a research study. The study is evauating the experiences of foundation year 2 (FY2) doctors and the impact on their we-being and the quaity of care they provide. The evauation is being carried out in Engand. Before you make any decision regarding participation you need to understand why the research is being done and what it woud invove for you. Pease take time to read the foowing information carefuy. Tak to others about the study if you wish. If you have any further questions, pease contact the project manager (contact detais provided at the end of this eafet). What is the purpose of the study? The purpose of the study is to understand the current arrangements for the impementation of FY2 training in Engand on a nationa and regiona basis. We are particuary interested in how the panning and impementation of training at the nationa, regiona and oca eve impacts on the we-being of FY2 doctors. A number of interviews are taking pace in Engand in order to better understand these issues. Who is conducting the study? The work has been funded by the NIHR Service Deivery and Organisation (SDO) Research Programme. It is being undertaken independenty by a research team, ed by a senior medica doctor based at the University of Sheffied. Why have I been seected? You have been seected because the organisation where you work is invoved in the panning and impementation of FY2 training. You have been identified as someone who deivers FY2 training and therefore may be interested in participating. Do I have to take part? A decision to take part in this study is entirey vountary. Any decision regarding participation wi be confidentia between you and the research team. You are aso free to withdraw from the study at anytime. What does agreeing to take part invove? Your invovement woud be to participate in one interview with a member of the research team. The main topic of discussion wi be how postgraduate medica education is panned and impemented by your organisation, particuary in regard to FY2s. The interview wi take pace at a convenient time for you, either at your pace of work or over the teephone. The discussion wi ast for around an hour. Data coected from the interview wi be anaysed independenty by the research team. If you agree to take part you woud participate in the study for 5 months during the consutation phase of the project (phase 1). The research study is funded for 33 months in tota. What about confidentiaity and data protection? A information you may give wi be treated in the strictest confidence. The interviewer wi take notes on the discussion but any information you give during the interview wi be fuy anonymised and combined with the views and experiences of other participants who agree to take part. No individua wi be identifiabe at any stage in the pubication or presentation of the findings. Data coected wi be stored securey in a manner consistent with the data protection act. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 99

122 APPENDIX 7 What are the risks of participating? We beieve that the risks are minima. We understand that there are many demands on your time and there is some inconvenience in taking part in the interview. You are free at any stage to withdraw from the interview or take time out if you wish. How wi I benefit from this study? We hope you wi find the experience of taking part in the interviews interesting and usefu. You wi have the opportunity to receive feedback from the study team in a short report of the overa interview findings if you wish to. What wi happen as a resut of the study? The data coected from you wi be aggregated with the data from other participants in the interviews and this wi be anaysed and used to produce a report which wi be made avaiabe for a participants. This report wi be pubished by the funders of the study and wi be avaiabe to inform poicy decisions around postgraduate training. Who has reviewed the study? A research in the NHS is ooked at by an independent group of peope, caed a Research Ethics Committee, to protect your safety, rights, we-being and dignity. This study has been reviewed and given a favourabe opinion by Research Ethics Committee. The study has aso been scientificay reviewed by independent peer reviewers prior to funding being given. What shoud I do now? You shoud take enough time as you fee you need to consider whether to take part. If you do wish to take part, there is a contact emai/number of the Project Manager to repy to beow. The research team wi then contact you to arrange a time for the interview and forward a consent form for you to sign. If you do not wish to take part then you are not required to do anything and we wi not contact you again. What wi happen if I don t want to carry on with the study? If after consenting to take part in the interview you subsequenty change your mind about participating, you can withdraw from the study at any time (incuding during or after the interview itsef). Any data coected from you woud not be incuded in the study. Further contact If you have any further questions then pease fee free to contact Coin O Keeffe, Project Manager. Thank you for your time 100 NIHR Journas Library

123 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 8 Consent form for phase 1 interviews CONSENT FORM Tite of Project: The EDiT Study: Evauation of Doctors in Training. [Phase 1 interviews Postgraduate Postgraduate education stakehoders/ed Consutants (training eads)] Name of Researcher: Ms Suzanne Mason Principa Investigator Pease initia box 1. I confirm that I have read and understand the information sheet dated 30/1/09 (version 3) for the above study. I have had the opportunity to consider the information, ask questions and have had these answered satisfactoriy. 2. I understand that my participation in the phase 1 interviews is vountary and that I am free to withdraw at any time without giving any reason and without being affected in any way. 3. I understand that reevant data coected from me during the study may be ooked at by authorised individuas such as reguatory authorities (for purposes such as monitoring the conduct of the research). I give permission for these individuas to have access to data coected during the study. 4 I agree to take part in the above study. Name of Person Date Signature Research Team Member Date Signature When competed, 1 for patient; 1 for researcher site fie Staff (Phase 1) consent form V4.0 Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 101

124

125 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 9 Foundation year 2 doctors focus group schedue (phase 1) Site no. Group: Date: Foundation year 2 doctors focus group schedue As peope come in ask them to compete the consent form. Introduction (5 minutes) Good morning/afternoon, my name is Angea Carter/Coin O Keeffe and Suzanne Mason and we are researchers working on the EDiT study. Many thanks for your interest in this work. I hope you wi have seen the information sheet about the study. If not here is another copy. If you are happy to take part in this discussion I woud be gratefu if you coud sign the consent form. This is one of a panned series of focus groups and interviews to discuss with those invoved in foundation training to find out what it is ike to do this work and training and what contributes to a successfu earning experience. The aim of the project is to appreciate the things that contribute to effective training and how this may be impemented esewhere. We are interested in your ideas of how this training can be done in the best way; in particuar what motivates you and gives you confidence when working with patients. This is a supportive study; no one is here to criticise what you do. We wi examine issues today and ask for your suggestions that wi be put in our fina report. Anything that you say to us today wi be treated in confidence and NO individua wi be identified. Identities wi be protected and individuas wi not be named. Further, no feedback wi be given to anyone in the trust or other organisations about what we discuss today. The study wi concude next year and the findings wi be made avaiabe. It is intended that we work together for 55 minutes. Are you happy to continue? Let anyone eave who is not happy to contribute to the study. (5 minutes) Start of group First estabish that the group are peope who are FY2 doctors. 1. Think of an experience of working practice that gives you confidence in your competence to deiver good patient care. Can you write down (for a few minutes keeping this to yoursef) the things that give you confidence Individua working for three or four minutes on successfu and unsuccessfu events. (5 minutes) 1a. What gives you confidence? 1b. What heps you fee competent? Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 103

126 APPENDIX 9 Go round the group recording materia on a fip chart exporing comments for specific detai. (10 minutes) (20 minutes) 2. Think about when you finish a shift what are the things that you worry about/keep you awake? Can you write down (for a few minutes keeping this to yoursef) the things that give you confidence Individua working for three or four minutes on successfu and unsuccessfu events. (3 minutes) Go round the group recording materia on a fip chart exporing comments for specific detai and differences. (15 minutes) Prompts: What cinica support do you get in your training? Who is this from (feow doctors, nursing staff, AHPs)? What part does the forma assessment pay in your training? (35 minutes to here) 3. What coud be done to improve your work experience as an FY2 doctor in the ED? Can you write down (for a few minutes keeping this to yoursef) the things that give you confidence Individua working for three or four minutes on successfu and unsuccessfu events. (3 minutes) Go round the group to get out ideas (encourage a debate and workabe soutions ooking for the resources that wi be required). (15 minutes) Consider ength of attachment, supervision, order of pacements; assessments, size of department, number of trainees; shifts; mentors; pressure of working environment; feedback on work performance. (50 minutes to here) 4. We are keen to use a questionnaire to examine the issues the next intake of FY2s have in more detai this woud be at the beginning, midde and end of pacements do you think this is feasibe and in what format woud you best ike to receive it eectronic via emai, paper, other? (7 minutes) (57 minutes to here) Concusion We have reached the end of our time now and woud ike to thank you for your participation. Many thanks for your time. A copy of the Executive Summary of our report wi be made avaiabe to a who have participated in this study. (3 minutes) End 104 NIHR Journas Library

127 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 10 Letter of invitation to foundation year 2 doctors for participation in phase 1 focus groups Distributed at deanery training event. Dear Sir/Madam, Tite of project: The EDiT Study: Evauation of Doctors in Training (Phase 1: consutation and scoping study) I am writing to ask if you woud kindy consider taking part in a focus group for a nationa research study, evauating your experiences as a foundation year 2 (FY2) doctor and how this impacts on your we-being, motivation and confidence. I am contacting you with the permission of the deanery and foundation schoo which organises your FY2 programme. The study wi ast for 5 months and incudes work in four deaneries in Engand. Focus group discussions with interested FY2 doctors are panned in each of the four deaneries. I am attaching some further information about the research and what agreeing to take part in the focus groups may invove for you. I woud be gratefu if you coud read this carefuy. I hope this is cear but if you have any further questions pease contact the project manager Coin O Keeffe. If after reading the information eafet and discussing this with others you fee happy to participate then pease emai the project manager (see above) who wi then contact you about taking part. If you do not fee as if you want to be invoved in the study then thank you for your time and you wi not be contacted again. Yours sincerey Signed Suzanne Mason Lead Investigator Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 105

128

129 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 11 Focus group participant information sheet The EDiT study: Evauation of Doctors in Training (Phase 1: focus group with foundation year 2 doctors) Participant information sheet We woud ike to invite you to take part in a focus group for a research study. The study is evauating the experience of foundation year 2 (FY2) doctors in Engand and the impact on their we-being, motivation and confidence in their roe. Before you make any decision regarding participation you need to understand why the research is being done and what it woud invove for you. Pease take time to read the foowing information carefuy. Tak to others about the study if you wish. You may wish to speak with your FY2 representative about participating. Aternativey, if you have any further questions, pease contact the project manager (contact detais provided at the end of this eafet). What is the purpose of the study? The purpose of the study is to understand what infuences the we-being, motivation and confidence of FY2 doctors. There have been a number of poicy initiatives aimed at improving the experiences of the NHS workforce (incuding postgraduate doctors) and evauation of these initiatives is aimed to measure their successes and faiures. Who is conducting the study? The work has been funded by the NIHR Service Deivery and Organisation (SDO) Research Programme. It is being undertaken independenty by a research team, ed by a senior medica doctor based at the University of Sheffied. Why have I been seected? You have been seected because your FY2 programme is organised by a deanery which has agreed for you to be approached regarding participation in the study. Do I have to take part? A decision to take part in this study is entirey vountary and it is entirey your decision whether to take part or not. Any decision regarding your participation wi be confidentia between you and the research team. You are aso free to withdraw from the study at anytime without any repercussions to yoursef. What does agreeing to take part invove? Your invovement woud be to participate in one focus group discussion with other FY2 doctors, faciitated by a member of the research team. The main topic of discussion wi be what contributes to a successfu earning and training experience. The focus group wi take pace at a convenient time for you. The discussion wi ast for around an hour. Data coected from the focus group wi be anaysed independenty by the research team. What about confidentiaity and data protection? A information you may give wi be treated in the strictest confidence. The researchers wi take notes during the discussion but any information you give during the focus group wi be fuy anonymised and combined with the views and experiences of other FY2 doctors who agree to participate. If an issue is raised by participants during the focus groups which is judged to have serious consequences for either the we-being of FY2 doctors, or serious impications for patients, then this issue may be discussed with senior staff in the postgraduate deanery and the ED. However, no individua FY2 doctor wi be identifiabe at any Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 107

130 APPENDIX 11 stage in discussions, pubication or presentation of the findings. Data coected wi be stored securey in a manner consistent with the Data Protection Act. What are the risks of participating? We understand that there are many demands on your time and there is some inconvenience in taking part in the focus group. We are aso aware you may have concerns about taking part in further evauation of your roe aong with your work-based assessments. The focus group wi be organised at a time which is convenient to you and is ikey to take pace during teaching time organised by the deanery which runs your Foundation Programme. We wi be asking you about both positive and negative experiences during your FY2 year. There is a very sma chance you may become upset if you vountariy discose an experience that was particuary stressfu or unhappy. You are free at any stage to withdraw from the focus group or take time out if you wish. The focus group is designed to be a supportive environment and the faciitator is an experienced researcher in this methodoogica approach. Ony other FY2 doctors and the research team wi be present at the focus group. How wi I benefit from this study? We hope you wi find the experience of taking part in the focus group interesting and usefu. You wi have the opportunity to receive feedback from the study team in a short report of the overa focus group findings if you wish to. What wi happen as a resut of the study? The data coected from you wi be aggregated with the data from other FY2 participants and this wi be anaysed and used to produce a report which wi be made avaiabe for a participants. This report wi be pubished by the funders of the study and wi be avaiabe to inform poicy decisions around postgraduate training. Who has reviewed the study? A research in the NHS is ooked at by an independent group of peope, caed a Research Ethics Committee, to protect your safety, rights, we-being and dignity. This study has been reviewed and given favourabe opinion by Leeds West Research Ethics Committee. The study has aso been scientificay reviewed by independent peer reviewers prior to funding being given. What shoud I do now? You shoud take enough time as you fee you need to consider whether to take part. If you do wish to take part, there is a contact emai/number for the study project manager to repy to beow. The research team wi then forward a consent form for you to sign aong with the detais of when and where it is suggested the focus group wi take pace. If you do not wish to take part then you are not required to do anything and we wi not contact you again. What wi happen if I don t want to carry on with the study? If you after consenting to take part in the focus group you subsequenty change your mind about participating, you can withdraw from the study at any time (incuding during or after the focus group itsef). Any data coected from you woud not be incuded in the study. Further contact If you have any further questions then pease fee free to contact Coin O Keeffe, Project Manager. Thank you for your time 108 NIHR Journas Library

131 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 12 Focus group consent form CONSENT FORM Tite of Project: The EDiT Study: Evauation of Doctors in Training. [Phase 1: focus group with Foundation year 2 doctors] Name of Researcher: Ms Suzanne Mason Principa Investigator Pease initia box 1. I confirm that I have read and understand the information sheet dated 29/04/09 (version 3.0) for the above study. I have had the opportunity to consider the information, ask questions and have had these answered satisfactoriy. 2. I understand that my participation in the focus group study is vountary and that I am free to withdraw at any time without giving any reason and without being affected in any way. 3. I understand that reevant data coected from me during the study may be ooked at by authorised individuas such as reguatory authorities (for purposes such as monitoring the conduct of the research). I give permission for these individuas to have access to data coected during the study. 4 I agree to take part in the above study. Name of Person Date Signature Research Team Member Date Signature When competed, 1 for patient; 1 for researcher site fie FY2 (Phase 1) consent form V4.0 Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 109

132

133 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 13 Website information/screenshot Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 111

134

135 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 14 Letter of invitation to foundation year 2 doctors for participation in phase 2 Distributed at deanery training event. Dear Sir/Madam, Tite of project: the EDiT study: Evauation of Doctors in Training (phase 2: ongitudina observationa study of foundation year 2 doctors) I am writing to ask you to consider taking part in a nationa research study evauating the we-being of foundation year 2 (FY2) doctors and the quaity of care they provide. The focus of this research is on the experiences of FY2 doctors in the emergency department (ED) and we are interested in your experiences in your ED pacement in particuar. I am contacting you with the permission of the deanery and foundation schoo which organises your FY2 programme. Part of the nationa evauation is a 12-month ongitudina observationa study of FY2 doctors. It is this particuar study I am asking you to consider taking part in. I am attaching some further information about the research and what agreeing to take part may invove for you. I woud be gratefu if you coud read this carefuy. I hope this is cear but if you have any further questions pease contact the project manager Coin O Keeffe. If after reading the information eafet and discussing this with others you fee happy to participate then pease sign and date the encosed consent form and return this in the pre-paid enveope provided. If you wish to discuss participation further then pease emai the project manager (see above), who wi then contact you again about taking part. If you do not want to be invoved in the study and you want no further contact about it, then pease emai this request to the project manager. Many thanks for your time Yours sincerey Signed Suzanne Mason Lead Investigator Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 113

136

137 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 15 Survey participant information sheet The EDiT study: Evauation of Doctors in Training (phase 2: ongitudina observationa study of foundation year 2 doctors) Participant information sheet We woud ike to invite you to take part in a research study. The study is evauating the experience of foundation year 2 (FY2) doctors in the emergency department (ED) and the impact on their we-being and the quaity of care they provide. The evauation is being carried out in a number of EDs in Engand. Pease take the time to read the foowing information which expains why the research is being done and what it woud invove for you. Tak to others about the study if you wish. You may wish to speak with your FY2 representative (contact detais to be incuded) about participating. Aternativey, If you have any further questions, pease contact the project manager (contact detais provided at the end of this eafet). What is the purpose of the study? The purpose of the study is to understand what infuences the we-being, motivation and confidence of FY2 doctors in the ED and how this is inked to the quaity of care they provide. There have been a number of poicy initiatives aimed at improving the experiences of the NHS workforce (incuding postgraduate doctors) and evauation of these initiatives is aimed to measure their successes and faiures. Who is conducting the study? The work has been funded by the NIHR Service Deivery and Organisation (SDO) Research Programme. It is being undertaken independenty by a research team, ed by a senior medica doctor based at the University of Sheffied. The research is being conducted independenty by the research team. Why have I been seected? You have been seected because your FY2 programme is organised by a deanery which has agreed for you to be approached regarding participation in the study AND because you have an FY2 pacement in the ED. Do I have to take part? We understand there are various expectancies paced upon you in your roes but pease be assured there is no pressure on you to participate. A decision to take part in this study is vountary and it is your decision entirey whether to take part or not. Any decision regarding your participation wi be confidentia between you and the research team. You are aso free to withdraw from the study at any time without any repercussions to yoursef. What does agreeing to take part invove? If you agree to take part you woud participate in the ongitudina observationa study for the fu 12 months of your FY2 year. The research study is funded for 33 months in tota. Initiay some imited contact detais (name and emai address) wi be entered onto a database in order that we may contact you about the study when necessary to do so. A study ID wi be assigned to each participating doctor which wi substitute for these contact detais as a means of ensuring anonymity of participants. Your main invovement woud be to compete a questionnaire up to a maximum of four times during your FY2 year. You wi be contacted by emai in order that a ink to an eectronic questionnaire can be provided at the appropriate time points. Aternativey you may receive a posta questionnaire if this is considered a more appropriate method in your department. The questionnaire wi use vaidated Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 115

138 APPENDIX 15 measures to assess your we-being, confidence and motivation at key time points in your FY2 year. It wi take in the region of around minutes to compete. In addition, a random seection of ED notes, in which you were the primary care giver during your ED pacement, wi be reviewed by experienced ED cinicians in order to assess the quaity and safety of care provided. This process wi take pace retrospectivey, after you have eft the ED. This process wi be carried out excusivey by the reviewers and the research team. A member of the cinica team in each ED wi identify the reevant notes for a participating FY2 doctors in each participating ED. The research team wi randomy seect a sma sampe of these notes for each doctor. Each set of notes wi be fuy anonymised and the reevant doctor study ID assigned. In this way the reviewers wi be binded to who has carried out the care. Data coected from you, from the survey and the notes review, wi be anaysed independenty by the research team. A data coected wi be entered onto a database which wi be fuy anonymised. Your persona detais wi not be stored on this database. Data coected from you wi be aggregated with the data of the other participating trainee doctors in your department in order that reationships between we-being and quaity can be measured at the department eve and variation across participating EDs can be measured. In this way no individua doctor s we-being or quaity-of-care scores wi be reported. There may be instances where a participating doctor s research scores may need to be examined on an individua basis. The quaity-of-care scores for an individua doctor may be such that there is evidence that patients are being put at serious risk. The processes to be foowed in such instances are described beow under confidentiaity and data protection. What about confidentiaity and data protection? A information you may give wi be treated in the strictest confidence. It is possibe that we wi hod imited eectronic information about you such as your name and emai address. This information wi be hed purey for the means of contacting you about the study where necessary to do so. After the study is compete a names and emai addresses wi be deeted. No individua FY2 doctor wi be identifiabe at any stage in the pubication or presentation of the findings. Data coected wi be stored securey in a manner consistent with the Data Protection Act. Exceptiona circumstances where breaching of confidentiaity may be necessary Uness exceptiona circumstances demand otherwise, individua identifiabe study data wi be kept confidentia at a times between yoursef and the study team. There are instances where data confidentiaity woud have to be compromised and information discosed to an individua outside the research team. They are as foows: 1. If incuded cinica notes were reviewed and detaied care which may have put a patient at serious risk, then the identity of the doctor who provided this care woud be discosed. The reevant ID number of the participating doctor woud be discosed to the ead consutant of the department by the doctor carrying out the case notes review. Any action taken woud be at the discretion of this ead cinician, but may incude contacting the reevant participating doctor to discuss the matter further and recommend any necessary support. The types of errors which may resut in this course of action are as foows: serious medication errors and unidentified missed conditions such as a missed fracture. 2. If you were interested in receiving feedback regarding your scores on the we-being survey or the quaity of care study then this coud ony be provided by accessing your persona identifiers in order to seect the reevant data and send a summary to you. What are the risks of participating? We understand that there are many demands on your time and there is some inconvenience in competing surveys. As described above there may be instances when confidentiaity has to be broken and your resuts from the study discussed with a third party. 116 NIHR Journas Library

139 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 How wi I benefit from this study? We beieve there are benefits to taking part in this study. If you specificay request the information from the research team, you wi be abe to receive feedback on your individua scores from the we-being survey and the quaity of care study. As stated above, in order to provide this information your confidentiaity woud have to be breached. What wi happen as a resut of the study? The data coected from you wi be aggregated with the data from other FY2 participants and this wi be anaysed and used to produce a report which wi be made avaiabe for a participants. This report wi be pubished by the funders of the study and wi be avaiabe to inform poicy decisions around postgraduate training. Who has reviewed the study? A research in the NHS is ooked at by an independent group of peope, caed a Research Ethics Committee, to protect your safety, rights, we-being and dignity. This study has been reviewed and given favourabe opinion by Leeds East Research Ethics Committee on 30 September The study has aso been scientificay reviewed by independent peer reviewers prior to funding being given. What shoud I do now? You shoud take enough time as you fee you need to consider whether to take part. If you do wish to take part, there is a contact emai/number for the study project manager to repy to beow. The research team wi then forward a consent form for you to sign aong with the detais of when the study wi begin. What wi happen if I don t want to carry on with the study? If you agree to participate but then decide to withdraw from the study, then there wi be no repercussions for you. Any data we have coected wi be anaysed as panned but you wi not receive any additiona questionnaires nor wi your notes be incuded in the review of quaity of care. Further contact If you have any further questions then pease fee free to contact Coin O Keeffe. Thank you for your time Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 117

140

141 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 16 Survey consent form CONSENT FORM Tite of Project: The EDiT Study: Evauation of Doctors in Training. [Phase 2: Longitudina Observationa Study of Foundation Year 2 Doctors] Name of Researcher: Ms Suzanne Mason Principa Investigator Pease initia box 1. I confirm that I have read and understand the information sheet dated 29/09/2009 (version 5.) for the above study. I have had the opportunity to consider the information, ask questions and have had these answered satisfactoriy. 2. I understand that my participation in the study is vountary and that I am free to withdraw at any time without giving any reason and without being affected in any way. 3. I understand that reevant data coected from me during the study may be ooked at by authorised individuas such as reguatory authorities (for purposes such as monitoring the conduct of the research). I give permission for these individuas to have access to data coected during the study. 4. I agree to take part in the above study Name of Person Date Signature Research Team Member Date Signature When competed, 1 for patient; 1 for researcher site fie Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 119

142

143 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 17 The EDiT study survey: piot questionnaire A survey on working as a foundation doctor Once you have read the foowing information pease enter the survey password and your emai address (at the bottom of this page) and press Next to begin the survey. What is this survey? This is a survey of your views and opinions of your current F2 PLACEMENT and of the department as a whoe where you are undertaking this pacement. This is not a test. There are no right or wrong answers. We want to know your persona views on the issues raised in the questionnaire. You wi have received an information sheet about why this questionnaire is being administered. If you want further information then pease visit the study website at Who wi see my answers? The information you give is totay confidentia. Findings wi be made avaiabe to a who participate, but in such a way that it is not possibe for individuas to be identified. The research team at the University of Sheffied wi be the ony organisation to have access to the questionnaires competed by individuas. How do I fi in this survey? Pease compete the questionnaire for your current roe. The survey wi take about minutes to compete. How shoud I respond? For each statement you are asked to seect one response that best fits your views. Pease answer a the questions as openy and honesty as possibe. Respond according to your first reaction. Do not spend too ong on one question. Pease enter your emai address.* Consent form Tite of project: The EDiT Study: Evauation of Doctors in Training (Phase 2: Longitudina Observationa Study of Foundation Year 2 Doctors) Name of researcher: Ms Suzanne Mason, Principa Investigator I confirm that I have read and understand the information sheet dated 29/09/2009 (version 5) for the above study. I have had the opportunity to consider the information, ask questions and have had these answered satisfactoriy. *[ ] Yes I understand that my participation in the study is vountary and that I am free to withdraw at any time without giving any reason and without being affected in any way. *[ ] Yes Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 121

144 APPENDIX 17 I understand that reevant data coected from me during the study may be ooked at by authorised individuas such as reguatory authorities (for purposes such as monitoring the conduct of the research). I give permission for these individuas to have access to data coected during the study. *[ ] Yes I agree to take part in the above study. *[ ] Yes Your confidence in managing common conditions The foowing section is designed to identify how confident you fee in managing common medica conditions, many of which you wi come across throughout your Foundation Years. (*Indicates that a question is compusory and must be competed before moving to the next page.) 1. How do you fee about managing patients with the foowing presenting compaints?* Seect a response from 1 9 and mark beow with 1 = owest eve of confidence and 9 = highest eve of confidence Diarrhoea and vomiting ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Shortness of breath ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Coapse unknown cause ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute menta heath probem ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Edery fa ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Chest pain ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Back pain ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Cardiac arrest ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Papitations ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Abdomina pain ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute aergic reaction ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Left side pain ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute stroke ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Overdose paracetamo ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Diabetic ketoacidosis ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute confusion ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Headache ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Seizure ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Ceuitis ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Haematemesis ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Rash ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute painfu joint ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Recta beeding ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 122 NIHR Journas Library

145 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO How experienced are you at performing the foowing practica techniques?* 1 2 = no/itte experience, 3 4 = some experience with support, 5 = moderate experience, 6 7 = good experience, 8 9 = confident aone Defibriation ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Arteria bood gas anaysis ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Suturing ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ECG interpretation ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Radiograph interpretation ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 3. Woud your management of the conditions shown be improved with:* Strongy disagree Disagree Neither disagree nor agree Agree Strongy agree Cearer guideines ( ) ( ) ( ) ( ) ( ) Better teaching ( ) ( ) ( ) ( ) ( ) More supervision ( ) ( ) ( ) ( ) ( ) Your job The foowing questions ask you to describe your job. Pease answer a the questions ticking the answer which best describes your current roe. 4. How often do you find yoursef meeting the foowing probems in carrying out your job* Not at a Just a itte Moderate amount Quite a ot A great dea I do not have enough time to carry out my work ( ) ( ) ( ) ( ) ( ) I cannot meet a the conficting demands made on my time at work I never finish work feeing I have competed everything I shoud I am asked to do work without adequate resources to compete it () () () () () () () () () () () () () () () I cannot foow best practice in the time avaiabe ( ) ( ) ( ) ( ) ( ) I am required to do basic tasks, which prevent me competing more important ones () () () () () Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 123

146 APPENDIX How true are the foowing of your job?* Not at a Just a itte Moderate amount Quite a ot A great dea I have cear, panned goas and objectives for my job () () () () () I know what my responsibiities are ( ) ( ) ( ) ( ) ( ) I know that I have divided my time propery ( ) ( ) ( ) ( ) ( ) Expanation is cear of what has to be done ( ) ( ) ( ) ( ) ( ) I know exacty what is expected of me ( ) ( ) ( ) ( ) ( ) 6. The foowing statements concern the information you and others get about your work performance.* Strongy disagree Disagree Neither disagree nor agree Agree Strongy agree I usuay know whether or not my work is satisfactory in this job I often have troube figuring out whether I m doing we or poory on this job Most peope on this job have a pretty good idea of how we they are performing their work Most peope on this job have troube figuring out whether they are doing a good or bad job () () () () () () () () () () () () () () () () () () () () Working reationships The foowing questions ask you about the reationships you have in your current roe. Pease answer a the questions, seecting the answer which best describes how you fee. 7. How much does your Cinica Supervisor:* To a very itte extent To a itte extent To some extent To a great extent To a very great extent Encourage you to give your best effort? ( ) ( ) ( ) ( ) ( ) Set an exampe by working hard him/hersef? Offer new ideas for soving job-reated probems? Encourage those who work for him/her to work as a team? () () () () () () () () () () () () () () () 124 NIHR Journas Library

147 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO To what extent can you count on your Cinica Supervisor:* Not at a To a sma extent Neither great nor sma extent To a great extent Competey To isten to you when you need to tak about probems at work? () () () () () To hep you with a difficut task at work? ( ) ( ) ( ) ( ) ( ) 9. In a typica week how many hours do you have contact with your Cinica Supervisor?* 10. To what extent can you:* Not at a To a sma extent Neither great nor sma extent To a great extent Competey Count on your coeagues to isten to you when you need to tak about probems? Count on your coeagues to back you up at work? Count on your coeagues to hep you with a difficut task at work? Reay count on your coeagues to hep you in a crisis situation at work, even though they woud have to go out of their way to do so? () () () () () () () () () () () () () () () () () () () () Your we-being and motivation The foowing questions ask you to describe things you ike and disike about your current roe and your genera we-being. Pease answer a the questions, ticking the answer which best describes what you do most of the time. 11. The statements beow concern how satisfied you fee with different aspects of your roe. How satisfied are you with:* Extremey dissatisfied Very dissatisfied Moderatey dissatisfied Not sure Moderatey satisfied Very satisfied Extremey satisfied The physica work conditions? The freedom to choose your own method of working? () () () () () () () () () () () () () () Your feow workers? ( ) ( ) ( ) ( ) ( ) ( ) ( ) The recognition you get for good work? Your Cinica Supervisor? () () () () () () () () () () () () () () Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 125

148 APPENDIX 17 Extremey dissatisfied Very dissatisfied Moderatey dissatisfied Not sure Moderatey satisfied Very satisfied Extremey satisfied The amount of responsibiity you are given? () () () () () () () Your rate of pay? ( ) ( ) ( ) ( ) ( ) ( ) ( ) Your opportunity to use your abiities? Reations between doctors and other heath-care professionas? Your chance of career progression? The way the department is managed? The attention paid to suggestions you make? () () () () () () () () () () () () () () () () () () () () () () () () () () () () () () () () () () () Your hours of work? ( ) ( ) ( ) ( ) ( ) ( ) ( ) The amount of variety in your job? () () () () () () () Your job security? ( ) ( ) ( ) ( ) ( ) ( ) ( ) The foowing questions ask you about your genera we-being and motivation. Pease answer a the questions. It is possibe that competing some questions may draw your attention to probems you experience. If you are worried that these are serious, we woud advise you to contact your GP. 12. In the ast month, how much of the time has your current roe made you fee:* Not at a Just a itte A moderate amount Quite a ot A great dea Goomy ( ) ( ) ( ) ( ) ( ) Uneasy ( ) ( ) ( ) ( ) ( ) Worried ( ) ( ) ( ) ( ) ( ) Tense ( ) ( ) ( ) ( ) ( ) Depressed ( ) ( ) ( ) ( ) ( ) Miserabe ( ) ( ) ( ) ( ) ( ) 13. Thinking of your current roe, how true are the foowing:* Strongy disagree Disagree Neutra Agree Strongy agree I often think about eaving medicine? ( ) ( ) ( ) ( ) ( ) It is very ikey that I wi activey ook for a new job outside medicine in the next year? I am starting to ask my friends/contacts about other job possibiities outside medicine? () () () () () () () () () () 126 NIHR Journas Library

149 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Pease answer the foowing questions about your overa motivation in your current roe by marking the most accurate answer. 14. How woud you rate the amount of effort you put into your job?* ( ) Very ow ( ) Low ( ) Moderate ( ) High ( ) Very high 15. I consistenty put forth the maximum effort possibe at work.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 16. How much of your tota, maximum possibe effort do you put into your job?* ( ) 50% or ess ()51 75% ()76 85% ()86 95% ()96 100% Pease answer each question in reation to your current roe by marking the most accurate answer. 17. It is not cear to me how much effort to put into different parts of my job.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 18. I divide my time across tasks in the way that is most hepfu to the organisation.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 19. My supervisor and I agree on what tasks are most and east important.* ( ) Never ( ) Rarey ( ) Sometimes ( ) Usuay ( ) Aways Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 127

150 APPENDIX 17 Pease answer each question by marking the box that gives your opinion. We want to know how much effort in your roe infuences how good or bad your job outcomes are (such as criticisms, feeings of accompishment, friendships). 20. I get better job outcomes if I increase my eve of effort.* ( ) Never ( ) Rarey ( ) Sometimes ( ) Usuay ( ) Aways 21. There is a strong tie between how hard I work and how good my job outcomes are.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 22. Working hard on this job is not rewarded.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 23. Woud you describe yoursef as typicay:* Extremey inaccurate Very inaccurate Moderatey inaccurate Sighty inaccurate Not sure Sighty accurate Moderatey accurate Very accurate Extremey accurate Careess ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Disorganised ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Efficient ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Inefficient ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Organised ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Practica ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Soppy ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Systematic ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 128 NIHR Journas Library

151 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO Peope cope with difficut, stressfu or upsetting situations in a variety of ways. Pease read each statement and seect the response that best indicates how much you tend to react in that way when faced with a difficut, stressfu or upsetting situation. When faced with a difficut/stressfu situation:* Never Sedom Sometimes Often Very often I try to figure out how to resove the probem ( ) ( ) ( ) ( ) ( ) I act as though nothing happened, hoping it wi go away ( ) ( ) ( ) ( ) ( ) I seek the support and guidance of other peope ( ) ( ) ( ) ( ) ( ) I get upset or angry with the peope who cause the probem ( ) ( ) ( ) ( ) ( ) I change something so the situation wi improve ( ) ( ) ( ) ( ) ( ) I avoid the probem by seeping, watching TV, engaging in diversionary activities more () () () () () I ask someone I respect for advice ( ) ( ) ( ) ( ) ( ) I try to get back at those who created the troube ( ) ( ) ( ) ( ) ( ) I come up with a coupe of strategies to make the situation better I keep my concerns and emotions about the situation to mysef () () () () () () () () () () I tak to friends or famiy about my circumstances ( ) ( ) ( ) ( ) ( ) I figure out who was responsibe for what happened ( ) ( ) ( ) ( ) ( ) I doube my efforts to correct the situation and achieve my objective () () () () () I put off deaing with the matter ( ) ( ) ( ) ( ) ( ) I get sympathy and understanding from someone ( ) ( ) ( ) ( ) ( ) I make sure that those responsibe for the probem receive their due punishment () () () () () Thinking of the decisions (or judgements) about patient care that you have made during your current roe assess the foowing statements: 25. I am confident that I made the appropriate decisions.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 26. Considering the information avaiabe to me I made the best decisions possibe.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 129

152 APPENDIX How much assistance did you need in making these decisions?* ( ) None at a ( ) Just a itte ( ) A moderate amount ( ) Quite a ot ( ) A great dea 28. Thinking of your current roe did you have:* None Just a itte A moderate amount Quite a ot A great dea Any forma teaching offered in your department (in addition to the generic teaching)? Any informa training offered in your department (in addition to the generic training), e.g. one-to-one shop foor teaching from a senior member of the medica staff, informa mentoring by senior medica staff? () () () () () () () () () () Any feedback on the quaity of your work? ( ) ( ) ( ) ( ) ( ) 29. Thinking about your current roe:* Not at a Just a itte A moderate amount Quite a ot A great dea Has your knowedge of medica conditions increased? Do you fee more abe to work as part of the cinica team? Do you understand more about how heath-care professionas work together? () () () () () () () () () () () () () () () Background detais It is important we know some of your background detais to represent the views of different groups of peope. About your job: 30. Year of quaification:* 31. Pace of quaification:* ()UK ( ) Non-UK 130 NIHR Journas Library

153 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 About you: 32. Age:* 33. Are you:* ()Mae? ( ) Femae? 34. What is your ethnic background? (Pease seect the answer that best describes your ethnic background)* ( ) White British ( ) White Irish ( ) White Other ( ) Back British ( ) Back Caribbean ( ) Back African ( ) Any other Back background ( ) Asian British ( ) Asian Chinese ( ) Asian Indian ( ) Asian Pakistani ( ) Asian Bangadeshi ( ) Any other Asian background ()Mixed White and Back British ()Mixed White and Back Caribbean ()Mixed White and Back African ()Mixed White and Asian ( ) Any other mixed background ( ) Any other ethnic group (pease specify): Hours of work: 35. In a typica working week in your current roe, how many hours are you contracted to work?* 36. What is the tota number of extra hours worked in a typica week?* 37. In the ast four weeks of your current roe have you had a shift changed at short notice (ess than five days)?* ( ) Yes ()No Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 131

154 APPENDIX 17 Foundation training experience: 38. Pease indicate your first, second and fina F1 pacements:* Genera practice Hospita medicine Hospita surgery Emergency medicine Obstetrics and gynaecoogy Radioogy Psychiatry Paediatrics and chid heath Trauma and orthopaedics Medica education Academic based Laboratory medicine Your first F1 pacement: () () () () () () () () () () () () Your second F1 pacement: () () () () () () () () () () () () Your fina F1 pacement: () () () () () () () () () () () () 39. Pease indicate your first, second and fina F2 pacements:* Genera practice Hospita medicine Hospita surgery Emergency medicine Obstetrics and gynaecoogy Radioogy Psychiatry Paediatrics and chid heath Trauma and orthopaedics Medica education Academic based Laboratory medicine Your first F2 pacement: () () () () () () () () () () () () Your second F2 pacement: () () () () () () () () () () () () Your fina F2 pacement: () () () () () () () () () () () () 132 NIHR Journas Library

155 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Future pans: 40. What do you pan to do after your F2 year?* ( ) Take a short-term contract in medicine ( ) Work abroad in medicine ( ) Take some time out from medicine ( ) Don t know yet ( ) Go onto speciaist training What speciaty wi you be moving into? ( ) Genera practice ( ) Hospita medicine ( ) Hospita surgery ( ) Emergency medicine ( ) Obstetrics and gynaecoogy ( ) Radioogy ( ) Psychiatry ( ) Paediatrics and chid heath ( ) Trauma and orthopaedics ( ) Medica education ( ) Academic based ( ) Laboratory medicine ( ) Other (pease state): Further comments 41. If you have any further comments to make, pease fee free to write them beow. Thank you for your co-operation. Dr A Carter, Dr CB Stride, Ms S Mason, Mr C O Keeffe Copyright 2010 the Authors. A rights reserved. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 133

156

157 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 18 Fina questionnaire A survey on working as a foundation doctor Once you have read the foowing information pease enter the survey password and your emai address (at the bottom of this page) and press Next to begin the survey. What is this survey? This is a survey of your views and opinions of your current F2 PLACEMENT and of the department as a whoe where you are undertaking this pacement. This is not a test. There are no right or wrong answers. We want to know your persona views on the issues raised in the questionnaire. You wi have received an information sheet about why this questionnaire is being administered. If you want further information then pease visit the study website at Who wi see my answers? The information you give is totay confidentia. Findings wi be made avaiabe to a who participate, but in such a way that it is not possibe for individuas to be identified. The research team at the University of Sheffied wi be the ony organisation to have access to the questionnaires competed by individuas. How do I fi in this survey? Pease compete the questionnaire for your current roe. The survey wi take about minutes to compete. How shoud I respond? For each statement you are asked to seect one response that best fits your views. Pease answer a the questions as openy and honesty as possibe. Respond according to your first reaction. Do not spend too ong on one question. Pease enter your emai address.* Consent form Tite of project: The EDiT Study: Evauation of Doctors in Training (Phase 2: Longitudina Observationa Study of Foundation Year 2 Doctors) Name of researcher: Ms Suzanne Mason, Principa Investigator I confirm that I have read and understand the information sheet dated 29/09/2009 (version 5) for the above study. I have had the opportunity to consider the information, ask questions and have had these answered satisfactoriy. *[ ] Yes I understand that my participation in the study is vountary and that I am free to withdraw at any time without giving any reason and without being affected in any way. *[ ] Yes I understand that reevant data coected from me during the study may be ooked at by authorised individuas such as reguatory authorities (for purposes such as monitoring the conduct of the research). I give permission for these individuas to have access to data coected during the study. *[ ] Yes Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 135

158 APPENDIX 18 I agree to take part in the above study. *[ ] Yes Your confidence in managing common conditions The foowing section is designed to identify how confident you fee in managing common medica conditions, many of which you wi come across throughout your Foundation Years. (*Indicates that a question is compusory and must be competed before moving to the next page.) 1. How do you fee about managing patients with the foowing presenting compaints?* Seect a response from 1 9 and mark beow with 1 = owest eve of confidence and 9 = highest eve of confidence Diarrhoea and vomiting ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Shortness of breath ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Coapse unknown cause ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute menta heath probem ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Edery fa ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Chest pain ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Back pain ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Cardiac arrest ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Papitations ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Abdomina pain ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute aergic reaction ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Left side pain ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute stroke ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Overdose paracetamo ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Diabetic ketoacidosis ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute confusion ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Headache ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Seizure ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Ceuitis ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Haematemesis ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Rash ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Acute painfu joint ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Recta beeding ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 136 NIHR Journas Library

159 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO How experienced are you at performing the foowing practica techniques?* 1 2 = no/itte experience, 3 4 = some experience with support, 5 = moderate experience, 6 7 = good experience, 8 9 = confident aone Defibriation ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Arteria bood gas anaysis ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Suturing ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ECG interpretation ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Radiograph interpretation ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 3. Woud your management of the conditions shown be improved with:* Strongy disagree Disagree Neither disagree nor agree Agree Strongy agree Cearer guideines ( ) ( ) ( ) ( ) ( ) Better teaching ( ) ( ) ( ) ( ) ( ) More supervision ( ) ( ) ( ) ( ) ( ) Your job The foowing questions ask you to describe your job. Pease answer a the questions ticking the answer which best describes your current roe. 4. How often do you find yoursef meeting the foowing probems in carrying out your job* Not at a Just a itte Moderate amount Quite a ot A great dea I do not have enough time to carry out my work ( ) ( ) ( ) ( ) ( ) I cannot meet a the conficting demands made on my time at work I never finish work feeing I have competed everything I shoud I am asked to do work without adequate resources to compete it () () () () () () () () () () () () () () () I cannot foow best practice in the time avaiabe ( ) ( ) ( ) ( ) ( ) I am required to do basic tasks, which prevent me competing more important ones () () () () () Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 137

160 APPENDIX How true are the foowing of your job?* Not at a Just a itte Moderate amount Quite a ot A great dea I have cear, panned goas and objectives for my job ( ) ( ) ( ) ( ) ( ) I know what my responsibiities are ( ) ( ) ( ) ( ) ( ) I know that I have divided my time propery ( ) ( ) ( ) ( ) ( ) Expanation is cear of what has to be done ( ) ( ) ( ) ( ) ( ) I know exacty what is expected of me ( ) ( ) ( ) ( ) ( ) 6. The foowing statements concern the information you and others get about your work performance.* Strongy disagree Disagree Neither disagree nor agree Agree Strongy agree I usuay know whether or not my work is satisfactory in this job I often have troube figuring out whether I m doing we or poory on this job Most peope on this job have a pretty good idea of how we they are performing their work Most peope on this job have troube figuring out whether they are doing a good or bad job () () () () () () () () () () () () () () () () () () () () Working reationships The foowing questions ask you about the reationships you have in your current roe. Pease answer a the questions, seecting the answer which best describes how you fee. 7. How much does your Cinica Supervisor:* To a very itte extent To a itte extent To some extent To a great extent To a very great extent Encourage you to give your best effort? ( ) ( ) ( ) ( ) ( ) Set an exampe by working hard him/hersef? ( ) ( ) ( ) ( ) ( ) Offer new ideas for soving job-reated probems? ( ) ( ) ( ) ( ) ( ) Encourage those who work for him/her to work as a team? () () () () () 8. To what extent can you count on your Cinica Supervisor:* Not at a To a sma extent Neither great nor sma extent To a great extent Competey To isten to you when you need to tak about probems at work? () () () () () To hep you with a difficut task at work? ( ) ( ) ( ) ( ) ( ) 138 NIHR Journas Library

161 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO In a typica week how many hours do you have contact with your Cinica Supervisor?* 10. To what extent can you:* Not at a To a sma extent Neither great nor sma extent To a great extent Competey Count on your coeagues to isten to you when you need to tak about probems? Count on your coeagues to back you up at work? Count on your coeagues to hep you with a difficut task at work? Reay count on your coeagues to hep you in a crisis situation at work, even though they woud have to go out of their way to do so? () () () () () () () () () () () () () () () () () () () () Your we-being and motivation The foowing questions ask you to describe things you ike and disike about your current roe and your genera we-being. Pease answer a the questions, ticking the answer which best describes what you do most of the time. 11. The statements beow concern how satisfied you fee with different aspects of your roe. How satisfied are you with:* Extremey dissatisfied Very dissatisfied Moderatey dissatisfied Not sure Moderatey satisfied Very satisfied Extremey satisfied The physica work conditions? The freedom to choose your own method of working? () () () () () () () () () () () () () () Your feow workers? ( ) ( ) ( ) ( ) ( ) ( ) ( ) The recognition you get for good work? () () () () () () () Your Cinica Supervisor? ( ) ( ) ( ) ( ) ( ) ( ) ( ) The amount of responsibiity you are given? () () () () () () () Your rate of pay? ( ) ( ) ( ) ( ) ( ) ( ) ( ) Your opportunity to use your abiities? Reations between doctors and other heath-care professionas? Your chance of career progression? The way the department is managed? () () () () () () () () () () () () () () () () () () () () () () () () () () () () Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 139

162 APPENDIX 18 Extremey dissatisfied Very dissatisfied Moderatey dissatisfied Not sure Moderatey satisfied Very satisfied Extremey satisfied The attention paid to suggestions you make? () () () () () () () Your hours of work? ( ) ( ) ( ) ( ) ( ) ( ) ( ) The amount of variety in your job? () () () () () () () Your job security? ( ) ( ) ( ) ( ) ( ) ( ) ( ) The foowing questions ask you about your genera we-being and motivation. Pease answer a the questions. It is possibe that competing some questions may draw your attention to probems you experience. If you are worried that these are serious, we woud advise you to contact your GP. 12. In the ast month, how much of the time has your current roe made you fee:* Not at a Just a itte A moderate amount Quite a ot A great dea Goomy ( ) ( ) ( ) ( ) ( ) Uneasy ( ) ( ) ( ) ( ) ( ) Worried ( ) ( ) ( ) ( ) ( ) Tense ( ) ( ) ( ) ( ) ( ) Depressed ( ) ( ) ( ) ( ) ( ) Miserabe ( ) ( ) ( ) ( ) ( ) 13. Thinking of your current roe, how true are the foowing:* Strongy disagree Disagree Neutra Agree Strongy agree I often think about eaving medicine? ( ) ( ) ( ) ( ) ( ) It is very ikey that I wi activey ook for a new job outside medicine in the next year? I am starting to ask my friends/contacts about other job possibiities outside medicine? () () () () () () () () () () Pease answer the foowing questions about your overa motivation in your current roe by marking the most accurate answer. 14. How woud you rate the amount of effort you put into your job?* ( ) Very ow ( ) Low ( ) Moderate ( ) High ( ) Very high 140 NIHR Journas Library

163 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO I consistenty put forth the maximum effort possibe at work.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 16. How much of your tota, maximum possibe effort do you put into your job?* ( ) 50% or ess ()51 75% ()76 85% ()86 95% ()96 100% Pease answer each question in reation to your current roe by marking the most accurate answer. 17. It is not cear to me how much effort to put into different parts of my job.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 18. I divide my time across tasks in the way that is most hepfu to the organisation.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 19. My supervisor and I agree on what tasks are most and east important.* ( ) Never ( ) Rarey ( ) Sometimes ( ) Usuay ( ) Aways Pease answer each question by marking the box that gives your opinion. We want to know how much effort in your roe infuences how good or bad your job outcomes are (such as criticisms, feeings of accompishment, friendships). 20. I get better job outcomes if I increase my eve of effort.* ( ) Never ( ) Rarey ( ) Sometimes ( ) Usuay ( ) Aways Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 141

164 APPENDIX There is a strong tie between how hard I work and how good my job outcomes are.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 22. Working hard on this job is not rewarded.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 23. Woud you describe yoursef as typicay:* Extremey inaccurate Very inaccurate Moderatey inaccurate Sighty inaccurate Not sure Sighty accurate Moderatey accurate Very accurate Extremey accurate Careess ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Disorganised ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Efficient ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Inefficient ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Organised ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Practica ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Soppy ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Systematic ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 24. Peope cope with difficut, stressfu or upsetting situations in a variety of ways. Pease read each statement and seect the response that best indicates how much you tend to react in that way when faced with a difficut, stressfu or upsetting situation. When faced with a difficut/stressfu situation:* Never Sedom Sometimes Often Very often I try to figure out how to resove the probem ( ) ( ) ( ) ( ) ( ) I act as though nothing happened, hoping it wi go away () () () () () I seek the support and guidance of other peope ( ) ( ) ( ) ( ) ( ) I get upset or angry with the peope who cause the probem () () () () () I change something so the situation wi improve ( ) ( ) ( ) ( ) ( ) I avoid the probem by seeping, watching TV, engaging in diversionary activities more () () () () () I ask someone I respect for advice ( ) ( ) ( ) ( ) ( ) I try to get back at those who created the troube ( ) ( ) ( ) ( ) ( ) 142 NIHR Journas Library

165 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Never Sedom Sometimes Often Very often I come up with a coupe of strategies to make the situation better I keep my concerns and emotions about the situation to mysef () () () () () () () () () () I tak to friends or famiy about my circumstances ( ) ( ) ( ) ( ) ( ) I figure out who was responsibe for what happened ( ) ( ) ( ) ( ) ( ) I doube my efforts to correct the situation and achieve my objective () () () () () I put off deaing with the matter ( ) ( ) ( ) ( ) ( ) I get sympathy and understanding from someone ( ) ( ) ( ) ( ) ( ) I make sure that those responsibe for the probem receive their due punishment () () () () () Thinking of the decisions (or judgements) about patient care that you have made during your current roe assess the foowing statements: 25. I am confident that I made the appropriate decisions.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 26. Considering the information avaiabe to me I made the best decisions possibe.* ( ) Strongy disagree ( ) Disagree ( ) Neutra ( ) Agree ( ) Strongy agree 27. How much assistance did you need in making these decisions?* ( ) None at a ( ) Just a itte ( ) A moderate amount ( ) Quite a ot ( ) A great dea Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 143

166 APPENDIX Thinking of your current roe did you have:* None Just a itte A moderate amount Quite a ot A great dea Any forma teaching offered in your department (in addition to the generic teaching) Any informa training offered in your department (in addition to the generic training), e.g. one-to-one shop foor teaching from a senior member of the medica staff, informa mentoring by senior medica staff? () () () () () () () () () () Any feedback on the quaity of your work? ( ) ( ) ( ) ( ) ( ) 29. Thinking about your current roe:* Not at a Just a itte A moderate amount Quite a ot A great dea Has your knowedge of medica conditions increased? ( ) ( ) ( ) ( ) ( ) Do you fee more abe to work as part of the cinica team? ( ) ( ) ( ) ( ) ( ) Do you understand more about how heath-care professionas work together? () () () () () Background detais It is important we know some of your background detais to represent the views of different groups of peope. About your job: 30. Year of quaification:* 31. Pace of quaification:* ()UK ( ) Non-UK About you: 32. Age:* 33. Are you:* ( ) Mae? ( ) Femae? 144 NIHR Journas Library

167 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO What is your ethnic background? (Pease seect the answer that best describes your ethnic background)* ( ) White British ( ) White Irish ( ) White Other ( ) Back British ( ) Back Caribbean ( ) Back African ( ) Any other Back background ( ) Asian British ( ) Asian Chinese ( ) Asian Indian ( ) Asian Pakistani ( ) Asian Bangadeshi ( ) Any other Asian background ()Mixed White and Back British ()Mixed White and Back Caribbean ()Mixed White and Back African ()Mixed White and Asian ( ) Any other mixed background ( ) Any other ethnic group (pease specify): Hours of work: 35. In a typica working week in your current roe, how many hours are you contracted to work?* 36. What is the tota number of extra hours worked in a typica week?* 37. In the ast four weeks of your current roe have you had a shift changed at short notice (ess than five days)?* ( ) Yes ()No Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 145

168 APPENDIX 18 Foundation training experience: 38. Pease indicate your first, second and fina F1 pacements:* Genera practice Hospita medicine Hospita surgery Emergency medicine Obstetrics and gynaecoogy Radioogy Psychiatry Paediatrics and chid heath Trauma and orthopaedics Medica education Academic based Laboratory medicine Your first F1 pacement: () () () () () () () () () () () () Your second F1 pacement: () () () () () () () () () () () () Your fina F1 pacement: () () () () () () () () () () () () 39. Pease indicate your first, second and fina F2 pacements:* Genera practice Hospita medicine Hospita surgery Emergency medicine Obstetrics and gynaecoogy Radioogy Psychiatry Paediatrics and chid heath Trauma and orthopaedics Medica education Academic based Laboratory medicine Your first F2 pacement: () () () () () () () () () () () () Your second F2 pacement: () () () () () () () () () () () () Your fina F2 pacement: () () () () () () () () () () () () 146 NIHR Journas Library

169 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Future pans: 40. What do you pan to do after your F2 year?* ( ) Take a short-term contract in medicine ( ) Work abroad in medicine ( ) Take some time out from medicine ( ) Don t know yet ( ) Go onto speciaist training What speciaty wi you be moving into? ( ) Genera practice ( ) Hospita medicine ( ) Hospita surgery ( ) Emergency medicine ( ) Obstetrics and gynaecoogy ( ) Radioogy ( ) Psychiatry ( ) Paediatrics and chid heath ( ) Trauma and orthopaedics ( ) Medica education ( ) Academic based ( ) Laboratory medicine ( ) Other (pease state): Further comments 41. If you have any further comments to make, pease fee free to write them beow. Thank you for your co-operation. Dr A Carter, Dr CB Stride, Ms S Mason, Mr C O Keeffe Copyright 2010 the Authors. A rights reserved. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 147

170

171 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 19 Expanded resuts tabes TABLE 21 Numbers of doctors recruited from each ED and NHS trust Deaneries Site (ED code) No. of F2 doctor participants No. of eigibe F2 doctors in trust Response rate (%) North West Northern East Midands Yorkshire and Humber London Mersey Severn West Midands Kent, Surrey and Sussex Tota Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 149

172 APPENDIX 19 TABLE 22 Characteristics of each ED group: age and ength of time quaified as a doctor n Mean SD SE 95% CI for mean Minimum Maximum Age at start of study (2010) (years) ED group to ED group to ED group to Tota to Log-age at start of study (2010) ED group to ED group to ED group to Tota to Years before 2010 that respondent quaified ED group to ED group to ED group to Tota to Log of years before 2010 that respondent quaified ED group to ED group to ED group to Tota to CI, confidence interva; SE, standard error. TABLE 23 Characteristics of each ED group: personaity traits n Mean SD SE 95% CI for mean Minimum Maximum Organisationa skis/abiity ED group to ED group to ED group to Tota to Coping via positive strategies ED group to ED group to ED group to Tota to CI, confidence interva; SE, standard error. 150 NIHR Journas Library

173 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 24 Mean scores for confidence in managing common acute conditions over time by ED group (time of pacement) Time point of response Condition ED group 1 Mean SD Mean SD Mean SD Mean SD TREAT1 Diarrhoea and vomiting TREAT2 Shortness of breath TREAT3 Coapse unknown causes TREAT4 Acute menta heath probem TREAT5 Edery fa TREAT6 Chest pain TREAT7 Back pain TREAT8 Cardiac arrest TREAT9 Papitations TREAT10 Abdomina pain TREAT11 Acute aergic reaction TREAT12 Left side pain TREAT13 Acute stroke TREAT14 Overdose paracetamo TREAT15 Diabetic ketoacidosis TREAT16 Acute confusion TREAT17 Headache TREAT18 Seizure TREAT19 Ceuitis TREAT20 Haematemesis TREAT21 Rash TREAT22 Acute painfu joint TREAT23 Recta beeding ED group 2 TREAT1 Diarrhoea and vomiting TREAT2 Shortness of breath TREAT3 Coapse unknown cause TREAT4 Acute menta heath probem TREAT5 Edery fa TREAT6 Chest pain TREAT7 Back pain TREAT8 Cardiac arrest TREAT9 Papitations TREAT10 Abdomina pain TREAT11 Acute aergic reaction continued Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 151

174 APPENDIX 19 TABLE 24 Mean scores for confidence in managing common acute conditions over time by ED group (time of pacement) (continued) Time point of response Condition Mean SD Mean SD Mean SD Mean SD TREAT12 Left side pain TREAT13 Acute stroke TREAT14 Overdose paracetamo TREAT15 Diabetic ketoacidosis TREAT16 Acute confusion TREAT17 Headache TREAT18 Seizure TREAT19 Ceuitis TREAT20 Haematemesis TREAT21 Rash TREAT22 Acute painfu joint TREAT23 Recta beeding ED group 3 TREAT1 Diarrhoea and vomiting TREAT2 Shortness of breath TREAT3 Coapse unknown cause TREAT4 Acute menta heath probem TREAT5 Edery fa TREAT6 Chest pain TREAT7 Back pain TREAT8 Cardiac arrest TREAT9 Papitations TREAT10 Abdomina pain TREAT11 Acute aergic reaction TREAT12 Left side pain TREAT13 Acute stroke TREAT14 Overdose paracetamo TREAT15 Diabetic ketoacidosis TREAT16 Acute confusion TREAT17 Headache TREAT18 Seizure TREAT19 Ceuitis TREAT20 Haematemesis TREAT21 Rash TREAT22 Acute painfu joint TREAT23 Recta beeding NIHR Journas Library

175 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 25 Mean scores for experience in performing practica techniques over time by ED group (time of pacement) Time point of response Tota Practica technique ED group 1 Mean SD Mean SD Mean SD Mean SD Mean SD EXPT1 Defibriation EXPT2 Arteria bood gas anaysis EXPT3 Suturing EXPT4 ECG interpretation EXPT5 radiograph interpretation ED group EXPT1 Defibriation EXPT2 Arteria bood gas anaysis EXPT3 Suturing EXPT4 ECG interpretation EXPT5 radiograph interpretation ED group EXPT1 Defibriation EXPT2 Arteria bood gas anaysis EXPT3 Suturing EXPT4 ECG interpretation EXPT5 radiograph interpretation Tota EXPT1 Defibriation EXPT2 Arteria bood gas anaysis EXPT3 Suturing EXPT4 ECG interpretation EXPT5 radiograph interpretation Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 153

176 APPENDIX 19 TABLE 26 Mean scores for anxiety and depression over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD Depression Anxiety ED group 2 Depression Anxiety ED group 3 Depression Anxiety Tota Depression Anxiety TABLE 27 Mean scores for job satisfaction over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD Job satisfaction Extrinsic job satisfaction Intrinsic job satisfaction ED group Job satisfaction Extrinsic job satisfaction Intrinsic job satisfaction ED group Job satisfaction Extrinsic job satisfaction Intrinsic job satisfaction Tota Job satisfaction Extrinsic job satisfaction Intrinsic job satisfaction NIHR Journas Library

177 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 28 Mean scores for motivation (effort) over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD Effort ED group 2 Effort ED group 3 Effort Tota Effort TABLE 29 Mean scores for intention to quit over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD Intention to quit ED group 2 Intention to quit ED group 3 Intention to quit Tota Intention to quit Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 155

178 APPENDIX 19 TABLE 30 Mean scores for roe characteristics (work demands, roe carity and feedback) over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD Work demands Roe carity Feedback ED group 2 Work demands Roe carity Feedback ED group 3 Work demands Roe carity Feedback Tota Work demands Roe carity Feedback TABLE 31 Mean scores for hours worked over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD Hours worked ED group 2 Hours worked ED group 3 Hours worked Tota Hours worked NIHR Journas Library

179 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 TABLE 32 Mean scores for reationa characteristics (CS support and coeague support) over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD CS support Coeague support ED group 2 CS support Coeague support ED group 3 CS support Coeague support Tota CS support Coeague support TABLE 33 Mean scores for reationa characteristics over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 One-to-one contact hours with CS Cose working contact hours with CS ED group 2 One-to-one contact hours with CS Cose working contact hours with CS ED group 3 One-to-one contact hours with CS Cose working contact hours with CS Tota One-to-one contact hours with CS Cose working contact hours with CS Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 157

180 APPENDIX 19 Impact on management of conditions for teaching and training We incuded a three-item scae on impact of teaching and training to assess whether F2 doctor management of the 23 medica conditions may have been improved by cearer guideines, better teaching or more supervision (e.g. During you most recent training period woud your management of the conditions have been improved with cearer guideines? ) Responses to the three items were on a five-point scae ranging from strongy disagree to strongy agree. The reiabiity of the scae ranged from 0.67 (T1) to 0.82 (T4). There was no significant variation across the four time points for any of the improvement in management variabes. TABLE 34 Impact of teaching and training on management of conditions over time by ED group (time of pacement) Time point at which F2 ED pacement carried out ED group 1 Time point of response Tota Mean SD Mean SD Mean SD Mean SD Mean SD Cearer guideines Better teaching More supervision ED group 2 Cearer guideines Better teaching More supervision ED group 3 Cearer guideines Better teaching More supervision Tota Cearer guideines Better teaching More supervision NIHR Journas Library

181 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 20 Quaitative anaysis of survey text comments Incuded in the survey was a space for further comments and a number of participants took the opportunity to offer further commentary. The proportion of F2 doctors providing comments at each time point is shown in Tabe 35. These comments were anaysed using tempate anaysis with the researcher bind to the quantitative findings of the survey. Comments made at the end of the foundation year 1 training period Participants made sighty more negative than positive comments on the overa F1 experience. Positive comments were mainy about the good supervision and support and the opportunities for earning, for exampe: An exceent year, I enjoyed a my rotations. ED was the hardest but the most rewarding pacement; it was the most varied and supportive one of the year. Some participants commented that they had had an unpeasant time or that pacements made them fee miserabe. A poor environment for earning (because of either dysfunctiona dynamics within the team or the environment being isoated from mainstream medicine), poor supervision (remote CS, having to make decisions aone), the heavy workoad or the ack of resources to support their work within the trust were the main reasons cited by participants for ack of enjoyment. A coupe of participants described mixed positive and negative experiences and ony one described a pattern of working (four pacements) that was not the reguar three pacements within a year. Two further participants described career choices that they were abe to make at the end of the F1 training period. Comments made at the end of the first foundation year 2 pacement Comments were made about both ED and non-ed rotations. The ED was described fairy equay as a chaenging and a rewarding experience, for exampe I fet miserabe most of the time because of the time pressure and antisocia working hours, it was a fantastic experience and I am gad that I have done it and I gained skis and experiences that I coud not have got esewhere. In fact, many of the comments considered both the chaenging and the rewarding aspects of the ED pacement. Positive experiences were mainy the resut of good support and supervision, good earning opportunities, supportive teaching, exceent teamworking and earning that increased participants confidence in their cinica practice. Chaenging aspects of the ED were the hours of work and rotas that incuded shifts that eft itte time for ife outside the ED, the high work demands and feeing stressed, with some (a minority) finding the teaching poor and a ack of support from consutants. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 159

182 APPENDIX 20 TABLE 35 The proportion of F2 doctors providing comments at each time point T1, n/n (%) T2, n/n (%) T3, n/n (%) T4, n/n (%) 35/188 (18.6) 31/154 (20.1) 41/135 (30.4) 27/108 (25) A variety of pacements other than the ED were described, such as genera practice and paiative care or speciaised roes such as medica bioogy and neonata orthopaedics. Severa participants commented that their pacement was not what they had thought it woud be from the advertisement or that changes had occurred since their appication (e.g. a department cosing). Experiences were seen as negative mainy because of poor support and supervision, itte training or teaching and having to carry out what were considered to be mundane or administrative tasks. Genera practice rotations were seen positivey, for exampe I have earned vauabe skis in being abe to identify probems in short periods of time. This rotation was seen as being a good match with an ED rotation. Ony one participant described a pattern of working that was not the reguar three pacements within a year (two pacements). Comments made at the end of the second foundation year 2 pacement Most of the comments about ED and non-ed pacements were made at this time point. The ED was described as being rather more chaenging than rewarding, but many of the negative comments were baanced with positive aspects of the roe/earning. For exampe, A&E is a stressfu job but the most rewarding I have done so far and a good experience with penty of independence; much better than when I was here as an F1. Comments about the chaenging experiences in the ED mainy reated to the high work demands, rotas that incuded unsocia hours of work, ess than good supervision and difficuties in attending training because of shift working. Positive experiences incuded exceent supervision and good earning experiences that increased trainees confidence in their cinica practice. The pacements other than those in the ED varied from academic pacements to those in genera medicine, with critica care and genera practice pacements being enjoyed and offering exceent training opportunities. Positive and negative experiences were described fairy equay, athough there were rather more factors associated with negative experiences than with positive experiences. Negative non-ed experiences were associated with poor support and supervision with itte feedback, high eves of expected service provision (sometimes in excess of their work experience to date) and itte teaching. Positive experiences were associated with a good eve of supervision and support with an emphasis on training. One participant commented on the stresses of appying for future jobs during the second pacement. Comments made at the end of the third foundation year 2 pacement and at the end of foundation training Unsurprisingy, this was the time point with the east commentary; however, some participants did describe their ED and non-ed experiences. Again, there was a mixed pattern of commentary regarding the ED pacement and, athough more chaenges than rewards were mentioned, the comments showed maturity. For exampe, ED is an exceent training ground for F2 doctors for any future fied, given the sheer voume of patients, good support and reguar teaching. 160 NIHR Journas Library

183 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Negative comments described high work demands, difficut rotas and imited supervision and teaching. Positive comments described exceent earning experiences and penty of support, athough the opportunity to refect on the quaity of referra was sometimes requested but rarey avaiabe. Comments about non-ed pacements were more positive than negative. A range of pacements was described and some participants took the opportunity to refect on their F2 Year. Positive experiences were characterised by good earning opportunities, being supported by a range of supervisors and a positive working environment. Less positive experiences were associated with a range of work demands, from having to organise one s own time to having itte to do, and training being provided out of step with earning opportunities. Refections over the F2 Year noted the effort required I reay enjoyed this year but it was hard work (A&E, Acute med, POSU [paedriatic observation unit]) and the overa vaue heped me move on from stress of a the practica things to earn to the bigger picture of patient-centred care and being part of a team. Ony two participants commented on their future work, with one staying on in a department that they had enjoyed working in and the other withdrawing from patient care to work in pathoogy. What was remarkabe about this was how few participants commented about the next steps in their medica careers. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 161

184

185 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 21 Reviewer information sheet The EDiT study: evauation of doctors in training: a notes review study evauating the quaity of care of junior doctors in the emergency department Reviewer information sheet We woud ike to invite you to assist us in a research study. The study is evauating the experience of foundation year 2 (FY2) doctors in the emergency department (ED) and the impact on their we-being and the quaity of care they provide. The evauation is being carried out in a number of EDs in Engand. Pease take the time to read the foowing information which expains why the research is being done and what it woud invove for you. Tak to others about the study if you wish. What is the purpose of the study? The purpose of the study is to understand what infuences the we-being, motivation and confidence of FY2 doctors in the ED and how this is inked to the quaity of care they provide. There have been a number of poicy initiatives aimed at improving the experiences of the NHS workforce (incuding postgraduate doctors) and evauation of these initiatives is aimed to measure their successes and faiures. Who is conducting the study? The work has been funded by the NIHR Service Deivery and Organisation (SDO) Research Programme. It is being undertaken independenty by a research team, ed by a senior medica doctor based at the University of Sheffied. Why have I been seected? You have been seected because you are a midde grade doctor in a participating ED and have the required experience to undertake an assessment of the quaity of care of junior doctors in your department. We understand there are various demands paced upon you in your roe and a decision to take part in this study is entirey vountary. What does agreeing to take part invove? If you agree to take part you woud participate in a Quaity of Care Notes Review Study over a 4- to 6-week period. Essentiay you wi review and rate the ED notes of seected FY2 doctors in your ED using a structured eectronic proforma. This work is designed to be a earning opportunity which wi enhance your career by participating activey in a research study and earning a usefu technique which you can utiise in your daiy practice in the future. You wi undergo training in a aspects of conducting a notes review. This wi be a singe day of training and wi incude demonstrations of the proforma software, how to review ED notes for quaity of care incuding working through exampes. You wi be aocated a number of ED notes to review. The review process is straightforward and we have experience of carrying out simiar studies in this setting. Previous reviewers have found the task interesting and fairy straightforward. Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 163

186 APPENDIX 21 How wi I benefit from this study? We beieve there are benefits to taking part in this study: It is an opportunity to take part in a nationa research study that may contribute to your continuing professiona deveopment. This piece of research can form part of your study time and be added to your CV. You wi receive a certificate of participation in research which can be added to your portfoio. Your invovement wi be acknowedged on pubications foowing on from the study. You wi be trained in a technique of notes review which wi be of benefit to your future day-to-day practice in emergency medicine, especiay in reation to handing compaints and undertaking notes review for assessing trainees. What about confidentiaity and data protection? Your identity wi not be discosed during the research process nor the names of any other staff or the hospita where you work. A data coected wi be fuy anonymised. What shoud I do now? You shoud take enough time as you fee you need to consider whether to take part. If you do wish to take part, there is a contact emai/number for the study project manager to repy to beow. The research team wi then contact you in order to take this work forward. Further contact If you have any further questions then pease fee free to contact Coin O Keeffe. Thank you for your time 164 NIHR Journas Library

187 DOI: /hsdr01150 HEALTH SERVICES AND DELIVERY RESEARCH 2013 VOL. 1 NO. 15 Appendix 22 Screenshot of quaity data-coection too: front page Queen s Printer and Controer of HMSO This work was produced by Mason et a. under the terms of a commissioning contract issued by the Secretary of State for Heath. This issue may be freey reproduced for the purposes of private research and study and extracts (or indeed, the fu report) may be incuded in professiona journas provided that suitabe acknowedgement is made and the reproduction is not associated with any form of advertising. Appications for commercia reproduction shoud be addressed to: NIHR Journas Library, Nationa Institute for Heath Research, Evauation, Trias and Studies Coordinating Centre, Apha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 165