Evaluation of Project ECHO (Extension for Community Healthcare Outcomes) Northern Ireland programme Report for Health and Social Care Board

Transcription

1 Evaluation of Project ECHO (Extension for Community Healthcare Outcomes) Northern Ireland programme Report for Health and Social Care Board May

2 Contents TITLE PAGE NUMBER Contents 2 Abbreviations 3 Executive summary 4 Acknowledgements and Contributors 7 Chapter 1 Introduction 9 Chapter 2 - Methods for Evaluation of ECHO with Healthcare Professionals 12 Chapter 3 - Diabetes ECHO 15 Chapter 4 - Optometry ECHO 33 Chapter 5 - Nursing Homes ECHO 55 Chapter 6 - Dermatology for GP trainees ECHO 72 Chapter 7 - Carers support ECHO 92 Chapter 8 - Hub feedback 112 Chapter 9 Overall Discussion 118 Chapter 10 Recommendation Summary 124 References 125 2

3 Abbreviations AMD DHSSPS ECHO GP GPST GPwSI HCP HSC ICP NH NI NIH NIMDTA OCT PIL TYC USA Age-related Macular Degeneration Department of Health, Social Services and Public Safety Extension for Community Healthcare Outcomes General Practitioner GP Speciality Trainee GP with a special interest Healthcare Professional Health and Social Care Integrated Care Partnership Nursing Home Northern Ireland Northern Ireland Hospice Northern Ireland Medical and Dental Training Agency Optical Coherence Tomography Participant Information Leaflet Transforming Your Care United States of America 3

4 Executive Summary Introduction Project ECHO (Extension for Community Healthcare Outcomes) uses tele-conferencing technology to improve access to specialised care through supporting and training primary health care professionals (HCPs) remotely (at spokes ) from a centralised hub of experts. ECHO has been proven to improve care across the United States, and a pilot study in community hospice nurses in Northern Ireland (NI) showed an improvement in knowledge and self-efficacy of HCPs.(1) In order to determine if ECHO would be effective in other contexts in NI, the HSC funded five ECHOs knowledge networks: Diabetes, Optometry, Nursing Homes, Dermatology for GP trainees, and one for supporting carers of palliative care patients. This report includes the results of the evaluation of the project and recommendations for the future use of ECHO in NI. Methods The four ECHOs with Healthcare Professionals (HCPs) were evaluated using similar methods. A pre ECHO knowledge and self-efficacy assessment was undertaken, along with collection of demographic data for spoke participants. At the end of each ECHO program, a post ECHO knowledge and self-efficacy assessment was undertaken by spoke participants, to determine if there was a change over the period of the project. They were also asked to complete a retrospective-pretest evaluation of self-efficacy (i.e. how competent and confident do they feel they were before the ECHO project with the benefit of hindsight). A survey of all participants (hub and spokes) views on ECHO in general was performed, and focus groups were held with spoke members of each ECHO network. The carers support ECHO was evaluated using a questionnaire of all participants, and a focus group of hub members. Results Two networks (optometry and dermatology for GP trainees) demonstrated a statistically significant improvement in knowledge and self-efficacy. The other two clinical networks (nursing homes and diabetes), while they demonstrated an improvement in knowledge and self-efficacy, the response rates were too low for statistical analysis. All four networks with HCPs demonstrated very positive views towards ECHO and the education and support that it provided, both through a questionnaire and through focus groups of spoke members. Both hub and spoke members valued being involved in ECHO networks, and all would participate again in ECHO if given the opportunity. All said it had improved the care they provided for patients. 4

5 Due to difficulties with recruitment and retention, only four carers participated in the carers program. While feedback from the carers was positive, and all said they would participate again and recommend the program to others, half stated that they would prefer a face-toface carers service. Hub members of the carers ECHO identified that it was a good forum for information sharing, but that it lacked the face-to-face support that carers can particularly benefit from. They identified issues with recruitment and some strategies that may be helpful for future carers ECHOs. Conclusion The quantitative and qualitative findings from this study support the use of ECHO in Northern Ireland for healthcare professionals by demonstrating statistically significant improvements in knowledge and self-efficacy in two networks, and positive feedback across all the networks replicating findings in other ECHO knowledge networks in the US setting,(2-4) and one pilot study in Northern Ireland.(1) Due to low numbers secondary to issues with recruitment and retention of carers, it is not possible to draw firm conclusions on the usefulness of ECHO for carers and further research is needed. As a low-cost high-impact model, ECHO can be adapted to meet the needs and resources of different communities and populations. At a time when health care providers are under mounting pressure to do more and spend less, this model provides an affordable solution to addressing growing need in the UK in training and supporting healthcare professionals. Further research is needed to look at the impact on patient care and service delivery. Suggestions from evaluation ECHO should continue to be developed and implemented to help educate healthcare professionals across Northern Ireland in a cost effective manner Adequate funding is necessary to allow protected time for all to participate, both at the hub and the spokes The importance of the skills of the facilitator was a recurring theme, and emphasises the need for good quality facilitation training and supervision to ensure networks exhibit fidelity to the ECHO model Future ECHO networks should continue to be evaluated to ensure they meet the needs of the population they are trying to educate and support To improve the response rates in future evaluations, funding for individual practices or participants could be dependent on participating in the evaluation process. This should therefore reduce the bias of a partial response, and improve the generalisability and hence usefulness of the results 5

6 Further studies are required to determine if the ECHO model improves patient care and reduces costs e.g. through reduced referral rates, improved quality of life, and better patient and staff outcomes Minor technological issues should be addressed to make connectivity easy and not detract from the learning environment. Issues of not being able to connect from some sites due to HSC security policies also need addressed Ensuring that preparatory work is available in good time for the spoke members, and that additional online resources are easily accessible were highlighted as contributing to the overall benefits associated with being part of an ECHO network More research is needed to determine the usefulness of a carers ECHO network The suggestions made in this report could contribute to designing a future Carers ECHO and through further evaluation determine more conclusively if the ECHO format is a useful methodology in providing information and support to carers in a cost effective manner. 6

7 Acknowledgements This project could not have been completed without the enthusiastic support and commitment of all the teams who were involved in the setting up and running of the five ECHO clinics in a short period of time. They, along with the Education Department at the Northern Ireland Hospice, established and maintained the ECHO clinics throughout the project. Their commitment to providing excellent education to others and hence improving patient care was outstanding, and the evaluation of the clinics would not have been possible without their expert involvement in the evaluation assessments. We are particularly indebted to all the participants who so willingly gave of their time and effort to participate both in Project ECHO and in its evaluation, and who throughout the project demonstrated a commitment to patient care which was truly inspiring. Administration support for the ECHO evaluations through Patricia Marshall, Áine McMullan, Rebecca Donnelly, Ciara McClements, Tracey McTernaghan and Claire Armstrong was invaluable. Our sincere thanks also go to the Health & Social Care Board for funding this study, and in particular to the HSC ECHO project Board, to Dr Sanjeev Arora and the ECHO team at the University of New Mexico for ongoing support and encouragement and to the Senior Management Team at the Hospice for having the vision to invest in the ECHO way of working. Contributors to the report Dr Clare White (Principle Investigator), Consultant Palliative Medicine, N. Ireland Hospice all aspects of the study Dr Clare McVeigh, Nurse Research Lead, N. Ireland Hospice focus groups, carers ECHO evaluation and write up Prof Max Watson, Medical Director, N. Ireland Hospice set up and running of Project ECHO NIH Dr Lynn Dunwoody, School of Psychology, Ulster University, N. Ireland statistical analysis Administration Support Patricia Marshall, Information Systems Administrator, N. Ireland Hospice - data entry Áine McMullan, Project manager, Project ECHO NI - Coordination of ECHO information 7

8 Tracey McTernaghan, N. Ireland Hospice coordination of ECHO data collection and marking Claire Armstrong, N. Ireland Hospice coordination of ECHO data collection and marking Rebecca Donnelly, Regional ICP Administrative Support, Health and Social Care Board coordination of ECHO data collection Ciara McClements - Project Manager Transforming Your Palliative and End of Life Care (TYPEOLC) Delivering Choice Programme, Northern Ireland - coordination of ECHO data collection and marking Dermatology ECHO Dr Nigel Hart, Senior lecturer in General Practice, Queens University Belfast and Associate Director in General Practice, NIMDTA Dr Siobhan McEntee, GP in Glengormley Practice, GP Trainer, GP Program Director with NIMDTA Claire Loughrey, Director of General Practice, NIMDTA Optometry ECHO Mr Raymond Curran, Head of Ophthalmic Services, Health and Social Care Board, Belfast Mrs Margaret McMullan, Clinical Ophthalmic Adviser, Health and Social Care Board, Belfast Mr Michael Williams, Senior Lecturer, Centre for Medical Education, Queen's University Belfast Honorary Consultant, Medical Ophthalmology, Belfast Health and Social Care Trust Prof Augusto Azuara-Blanco, Clinical Professor Centre for Vision and Vascular Science, Queen s University Belfast Consultant Ophthalmologist, Belfast Health and Social Care Trust Diabetes ECHO Dr Roy Harper, Consultant Endocrinologist, South Eastern Trust/ HSCB Dr Neil Black, Consultant Endocrinologist, Western Trust Nursing Home ECHO Sue Foster, Head of Education, Northern Ireland Hospice Corrina Grimes, Allied Health Professinal Consultant, Public Health Agency 8

9 Chapter 1 - Introduction Accessible education for healthcare professionals (HCPs) is essential if they are to be enabled to provide high quality care in a rapidly evolving health care environment. This is particularly challenging within a community context as practitioners may work in isolated settings, with varied access to educational opportunities and peer review of practice. With the constraints of limited resources and the isolated setting in which many HCPs work, new innovative or creative approaches to education and mentorship that facilitate individual and collective learning and changes in practice are required. One such approach is to use the structure of ECHO (Extension for Community Healthcare Outcomes), which was developed by University of New Mexico and uses point to point video-conferencing technology involving a central specialist hub linking with primary care providers remotely at spokes. ECHO uses a collaborative model of medical education and clinical support, and aims to empower and equip HCPs to provide better care to more people, right where they live.(5) Participants in the primary care setting (at the spokes) receive evidence-based or best practice guidance from specialists at the hub, case-based learning from peer-presentations and have opportunity for live questions and answers. The ECHO model does not provide direct patient care, but through training using real life cases provides front-line HCPs with the knowledge and support to manage similar patients with complex conditions. It does this by engaging HCPs in a sustained learning system and partnering them with specialist mentors to form a community of practice.(5) It is therefore different from telemedicine (picture 1), which aims to treat patients directly. Instead it aims to multiply knowledge through educating others to treat more patients than could ever be directly cared for by one individual HCP. Through ECHO there is opportunity to quickly translate new knowledge into practice, and thus improve outcomes for patients in more remote settings. ECHO is now used across the United States of America (USA) for 45 different disease and health conditions including hepatitis C, diabetes, asthma, pain management and rheumatology, and has been shown to improve patient outcomes.(2-4) The impact on such outcomes, and the reduction in waiting lists at central specialist clinics have been key in promoting its widespread uptake in health systems which are struggling to meet the needs of patients living far from central services.(2-4) Indeed the Senate in the USA has just passed a bill supporting the widespread use of ECHO across the country. 9

10 Picture 1 ECHO vs. Telemedicine ECHO uses cloud based Zoom software which is compliant with encryption standards and which allows staff to connect with ECHO sessions using standard computers, laptops, tablets and hand held devices. Zoom allows for up to 100 spokes to join a single ECHO session and does not require expensive dedicated telemedicine equipment though is dependent on a reliable broadband connection. In 2014 the Northern Ireland Hospice (NIH) undertook a six month pilot project of ECHO with its community hospice nurses, and the evaluation showed statistically significant improvements in knowledge and self-efficacy(1); 96% recorded gains in learning, and 90% felt that ECHO had improved the care they provided for patients; 83% would recommend ECHO to other healthcare professionals (HCPs); 70% stated the technology used in ECHO had given them access to education that would have been hard to access due to geography.(1) This was the first evaluation of ECHO in the UK and Europe. In 2015 the Health and Social Care (HSC) Board received funding of 403k from the Executive Change Fund in and worked in partnership with NIH to pilot the use of the ECHO model across the HSC to determine if the successes of ECHO in the USA could be replicated in Northern Ireland. The project pilot period was from October 2015 until 31st March

11 There were approximately 40 ECHO sessions set up (November 2015 March 2016) across the following Knowledge Networks using the hub and spoke model: Diabetes for GPs Optometry for Optometrists Palliative Care and Quality Improvement for Nursing Homes (26 Nursing Homes involving over 90 healthcare professionals) Dermatology for GP Trainees Support for Carers of Patients with Palliative Care needs Each of these ECHO knowledge networks was evaluated to help inform decisions around the future of ECHO in Northern Ireland. The ECHO knowledge networks are described in turn along with the results of their evaluation. 11

12 Chapter 2 - Methods for Evaluation of ECHO with Healthcare Professionals These methods relate to the evaluation for each of the four clinical ECHO networks with HCPs- Diabetes, Optometry, Nursing Homes and Dermatology for GP trainees. The carers support evaluation is described separately as it required a different evaluation strategy. Aims and Objectives The aim of the study was to evaluate the use of ECHO Knowledge Networks for HCPs working in Northern Ireland. In order to do this, the study objectives were: To determine if the use of ECHO improves HCPs knowledge and self-rated skills and confidence (self-efficacy) in the management of patients To explore the HCPs experience and perceived usefulness of ECHO in meeting knowledge and support needs To determine if hub participants benefited from participating in ECHO Study Design A prospective longitudinal cohort study was undertaken for each ECHO, using a combination of qualitative and quantitative methods. The study comprised of two stages: Stage 1: Baseline assessment prior to ECHO commencement for spoke participants Stage 2: End of ECHO assessment for hub and spoke participants. All ECHO evaluations, with the exception of the Dermatology for GP trainees, were run by an independent evaluation team from the NIH research department. Assessment tools were specifically written for each particular ECHO, but adapted from a master format. The creation of the assessment tools appropriate to each network was undertaken by the clinical lead running that network. The Dermatology ECHO for GP trainees evaluation was run by the hub members (Dr Hart and Dr McEntee) and the evaluation team from NIH was only involved in obtaining general feedback on ECHO from participants and through running the focus group. STAGE 1 Baseline data collection prior to ECHO commencement: Spoke participants: Each HCP participating in the spokes (or community) was asked to complete the following tools, based on the research aims for the study: 1. A confidential self-efficacy tool focusing on participant confidence in managing different situations that they face in delivering the relevant service in their particular clinical area. 12

13 2. A written knowledge assessment. This was developed by each individual ECHO clinical leads team with input from the research team as required. 3. Demographic data for all participating HCPs included gender, age, profession, area worked in (rural / urban), and length of time working in their particular area. The tools took a maximum of 60 minutes to complete and were undertaken prior to commencement of the ECHO network, either at a training day or they were sent out to participants and returned once completed. Participants names were not recorded on the tool; instead each participant was allocated a unique code that was only identifiable to the administration team. Each participant was provided with information about the evaluation using a participant information leaflet (PIL). Participation in the evaluation was considered an essential requirement of being involved in the ECHO, and hence if participants did not wish to participate in the evaluation the intention was that they would be unable to participate in the ECHO network. Consent to take part in the evaluation was presumed on completion and return of the questionnaires. STAGE 2- After ECHO Knowledge Network completed: Spoke participants - at the end of the ECHO project all HCPs at the spokes were asked to complete the following survey data: Questionnaire of self-efficacy to determine if there was a change over the period of the project. They were also asked to complete a retrospective-pretest evaluation of self-efficacy (i.e. how competent and confident do they feel they were before the ECHO project with the benefit of hindsight). This was used to try to reduce the bias of self-evaluation and facilitate HCPs to be more objective in assessing the impact of ECHO on self-efficacy.(6, 7) The pre-test, post project and retro-pretest questionnaires were all compared using a unique identifier code Knowledge assessment (the same as pre-echo) with results being compared with their pre-echo assessment Questions relating to participants overall views on ECHO and the network they were involved in. Hub participants - All hub participants were asked to complete a questionnaire via Survey Monkey (8) of their experiences of participating in an ECHO network. Focus Groups To address the second objective of exploring the HCPs experience and perceived usefulness of ECHO in meeting knowledge and support needs, focus groups were 13

14 undertaken with the HCPs who had attended the ECHO programme at the spokes and who indicated a willingness to participate in a focus group discussion. All spoke participants who participated in ECHO received an invitation letter via requesting if they would be willing to participate as well as a PIL. The intention of this stage of the study was to provide greater insight into the perceptions and experiences of participating in the ECHO knowledge networks, problems encountered and benefits, and whether or not it would be useful to continue with the programme. A focus group schedule, based on the aims and objectives of the project was used to elicit information. There was a cooling off period of a minimum seven days between receiving the letter inviting participation, the PIL and participating. With the participant s permission, the discussion was audio-taped and supplemented by field notes. Participants names were not recorded on the tape; instead each participant was allocated a unique code that was only be identifiable to the researcher. The groups were facilitated by a researcher who was not involved in the running of ECHO using a structured framework. Focus groups were conducted using the ECHO technology following the final ECHO session or at another pre-arranged time. Inclusion Criteria All HCPs participating in the ECHO programme were invited to participate in the relevant aspects of the study. Governance and Consent Approval was granted from N. Ireland Hospice / Ulster University research governance committee. Consent to take part in the study was presumed on completion and return of the questionnaires. Formal consent was taken for the focus groups. No patients were involved in this aspect of the evaluation, only HCPs who were assured that this evaluation was about evaluating ECHO, not about evaluating HCPs and their individual knowledge and practice. Data analysis - Due to the small sample, descriptive statistics and where possible, nonparametric tests (Friedman and Wilcoxon) were used to summarise the participant characteristics and survey data. The data from the focus group was analysed by adopting a thematic analysis framework described by King and Horrocks (2010). (9) This thematic analysis approach allowed the investigator to be flexible in their interpretation and development of the themes, and not to be confined by a rigid analytical framework. 14

15 Chapter 3- Diabetes ECHO Background There are over 83,000 people living with diabetes in Northern Ireland with rates of approximately 5% of the general population. Approximately 90% of people living with diabetes have type two diabetes, the prevalence of which is rising rapidly. Diabetes care is delivered in both primary and secondary settings, and in NI approximately two-thirds of people are looked after solely by primary care through Practice Nurses and GPs. Historically a larger proportion of people in Northern Ireland are looked after in secondary care when compared to other areas in the UK. GPs and Practice Nurses are experienced professionals, but their confidence and knowledge base in diabetes care varies from practice to practice. It is estimated that 10% of the healthcare budget is spent on delivering diabetes care including the management of complications from diabetes. Diabetes is therefore a key target for the HSC and is one of the priority conditions on which Transforming your Care (TYC) and Integrated Care Partnership (ICP) initiatives focus. Diabetic patients need multidisciplinary input to deliver all the care that is needed. This is appreciated to varying degrees in different care settings. It is also a complex therapeutic area which continues to change rapidly, so it is hard to keep up to date particularly if it is not the clinician s main clinical area. These factors can decrease confidence in using new therapies to manage patients. This can contribute towards therapeutic inertia or encourage premature referral to speciality care, which results in overloading of such services when the problem could be dealt with without attending a secondary care clinic. The converse is that delay in an appropriate referral can lead to an increased complication risk. Primary care professionals therefore need supported and educated to manage patients appropriately in the primary care setting, and to know when to refer to secondary care. The Intervention The Diabetes ECHO trained and supported primary care providers, GPs and practice nurses, to improve their knowledge and skills in the management of patients with diabetes. The ECHOs were held weekly on a Wednesday afternoon from pm and covered a curriculum devised by spoke participants including diagnosis, appropriate use of Type 2 agents, use of insulin, and management of diabetes at the end of life. For detail see table 1. Hub - The hub included diabetologist s from the regional diabetes strategy group who expressed an interest in being involved in the ECHO network. There was also an ICP Clinical Lead with an interest in diabetes, a Trust dietician, a Trust Special Diabetic Pharmacist and a National Care Advisor from Diabetes UK. 15

16 Spokes - Involvement in ECHO was opened to all GP s in the region. Thirteen GP s, who were joined by some of their practice nurses and practice based pharmacists took part. Table 1 Diabetes ECHO Network Diabetes ECHO Network Clinical Dr Roy Harper Diabetologist South Eastern Trust Champion/Lead Dr Neil Black - Diabetologist Western Trust Dr. Glynis Magee - Diabetologist Southern Trust Facilitator Admin Support Florence Findlay White (Diabetes UK) Tracey McTernaghan Frequency of clinics Weekly and then bi-weekly. 9 sessions in total. Dates: Dec 9th,16th/Jan 13th,20th,27th/Feb 3rd,17th/Mar 2nd,16 th Start date Wednesday 9 th December Training Date 25 th November 2015 at Jennymount Hub members Barney McCoy (ICP Clinical Lead South) Lesley Hamilton (Western Trust) Neil Black (Western Trust) Roy Harper (SE Trust/HSCB) Brid Farrell (PHA) Magee, Glynis (diabetologist Southern Trust) Lynne Thomas (dietician) SE Trust Rosemary Donnelly - Specialist diabetic Pharmacists (SE Trust) Hub costs As per costing schedule Based on 12 weeks of clinic and 4 hours clinic time (inc prep) and one session of training 11,481 paid to the 5 Trusts on 1 st December 2015 Spoke members 13 GP s confirmed (Representing all ICP areas across the region except West) Spoke costs 170 for GP practices per session, backfill. Equipment needed Only 4 webcams required as most using their own equipment, for spokes laptop/ipad 16

17 Curriculum Curriculum developed at training day. Currently, what is the best way to diagnose diabetes in both adults and children? Communicating the diagnosis news to patients and inspiring change. When & how do I start insulin in Type 2 diabetics? What Type 2 agents should I be using and when? (Part 1) What Type 2 agents should I be using and when? (Part 2) The practicalities of Insulin including the sick day rule. What has my diabetic patient been told in their structured education programme? What do I do when my diabetic patient is dying? How do I recognise this and manage the last year of life? Rules Diabetes & driving Evaluation Methods Evaluation was undertaken as described in Chapter two. Participants completed their pretest evaluations at the training day, and were ed their post-test evaluations which were returned by post or . Results Diabetes ECHO There were 13 GP sites who participated in ECHO, and while practice nurses joined intermittently, only the GPs were invited to take part in the evaluation as they were the primary target of the ECHO and attended consistently. Eight spoke participants completed the pre ECHO assessments (response rate 61.5%), with six completing the post ECHO assessments (response rate 46.2%). Demographic data for all participants in the pretest evaluation are shown in table 2. For the rest of the results, only the participants who completed the pre and post evaluations are included. 17

18 Table 2 Diabetes Demographic data Pre ECHO Evaluation Range Totals Age Gender Male 6 Female 2 Work area Urban 3 Rural 3 Mixed 2 Years in practice < Knowledge and Self-efficacy Assessments The mean scores of knowledge improved slightly between the pre ECHO and post ECHO assessments for the six participants. Average knowledge scores improved from to (out of a possible 40 marks); from 54% to 57%. Two participants score dis-improved, one was the same and three improved. Means and standard deviations are shown in table 3. Due to the low response rate no further statistical analysis was possible. Error bars are shown in figure 1. Table 3 Diabetes ECHO Knowledge and total self-efficacy results Pre- ECHO Post-ECHO Retro-Pre ECHO Outcome Mean SD Mean SD Mean SD Knowledge Total efficacy Self

19 Figure 1 Error bars representing the mean and two standard deviations either side of the mean for the diabetes knowledge assessment Self efficacy results improved (table 3), and a higher average score for the post-test evaluation in all areas compared with the pretest and retro-pretest evaluation. The different domains are shown in table 4. Due to the low response rate no further statistical analysis was possible. Table 4 Diabetes ECHO Self efficacy assessment - Participants self-rated confidence in each area (1 - not confident at all, 5 - very confident) Question Diagnose and classify patients with diabetes Treat patients with diabetes using optimised treatment regimens Understand the possible side-effects and limitations of most pharmacological treatments used in diabetes Initiate and support patients as they use injection therapies Assess and expertly manage diabetes complications and co-morbidities Educate and motivate patients with diabetes Serve as a local expert within my practice and area for diabetes questions and issues Pre ECHO average (Range) 4.38 (3-5) 4.00 (2-5) 3.88 (2-5) 2.63 (1-5) 3.50 (2-5) 3.75 (3-5) 3.38 (1-5) Retropre ECHO Average (Range) 4.00 (3-5) 3.67 (2-5) 3.33 (2-5) 3.17 (1-5) 3.33 (2-5) 4.00 (3-5) 3.67 (1-5) Post ECHO average (Range) 4.83 (3-5) 4.50 (2-5) 4.33 (2-5) 4.17 (1-5) 4.50 (2-5) 4.83 (3-5) 4.50 (1-5) 19

20 General Feedback of Diabetes ECHO Box 1 demonstrates participant s views on ECHO in general. The six respondents were very positive, with 100% having learnt a lot through ECHO participation and enjoyed it, 100% felt it helped translate knowledge into practice more than other teaching sessions they had been involved in and had improved the care they provided for patients, and 100% would participate again. Box 1 General ECHO Diabetes Results 1. Rating on a scale of 1-5 the quality of learning / usefulness from each area (1- poor, 5- excellent) Review of previous session Presentations Case based discussions Overall do you feel you have learnt through participating in ECHO? A lot 6 A little 0 NO 0 3. Did you find participating in ECHO enjoyable? A lot 6 A little 0 No 0 4. Do you think that participating in ECHO has improved the care you provide for patients? A lot 6 A little 0 No 0 20

21 5. Do you think the format of ECHO helps translate knowledge from teaching into practice more than other teaching sessions you have been involved in? Yes 6 No 0 6. Would you recommend ECHO to other healthcare professionals in your area? Yes 5 No 0 7. Have you used any of the online resources via Moodle, and if yes have you found these useful? Used and found useful Power point presentations 4 0 Video of the teaching sessions 4 0 Video of case presentations 4 0 Other supporting materials Regarding ECHO technology Used and found NOT useful Agree Disagree Unsure It has given me access to education that would have been hard to access due to geography It was a good medium to access teaching / education at a different location from where I work Any technical difficulties were acceptable and did not put me off participating in ECHO 5 0 No issues Any technical difficulties did not significantly reduce my learning How do you rate your overall ECHO experience? (1- poor, 5- excellent) No issues 21

22 10. Would you participate in ECHO sessions in the future if the opportunity arose? Yes 6 NO 0 Focus Group Results The focus group was conducted with five of the spoke GPs. Analysis of the focus group data uncovered two overarching themes, each with their own descriptive and interpretative themes that are outlined in thematic diagrams (figure 2 and 3). Overarching theme 1, ECHO Enhanced Clinical Knowledge and Skills, explored the reasons why participants perceived that ECHO optimised their clinical practice. Interpretative themes 1A and 1B highlighted the contributing factors that resulted in participant s perceived enhancement of clinical practice. Overarching theme 2, Consideration for the Future of ECHO, displayed the key conceptions that participants perceived should be considered for future ECHOs. Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills This theme encompassed how participants perceived ECHO to be an education platform that enhanced their clinical knowledge and skills. Interpretative theme 1A (figure 2) depicted how participants perceived that ECHO optimised the care they delivered their patients through the knowledge they had gained, and also through their increased confidence in caring for patients with diabetes. Interpretative theme 1B (figure 2) illuminated how ECHO also created a safe learning environment for participants. Findings highlighted that this was perceived to have been achieved by the encouragement of the hub and the sharing of knowledge. The learning environment was also enhanced as ECHO was perceived as providing a platform which met the differing learning styles of participants. 22

23 Figure 2 Theme 1 ECHO Enhanced Clinical Knowledge and Skills for Diabetes ECHO Descriptive Themes Interpretative Themes Overarching Themes Increased GPs confidence in their skills when caring for patients with diabetes Increased knowledge and skills regarding diagnosing diabetes 1A Optimised Patient Care Increased knowledge regarding new treatments 1 ECHO Enhanced Clinical Knowledge and Skills Importance of the Hub Encouraged sharing of knowledge 1B Created an effective learning environment Highlighted areas for further learning Adapted to participants learning styles 23

24 Figure 3 Theme 2, Consideration for the Future of ECHO for Diabetes ECHO Descriptive Themes Interpretative Themes Overarching Themes Funding allows protected time Funding ensures quality teaching 2A There needs to be funding Team approach at the spokes Future hub members 2B Furture ECHO participants 2 Consideration for the Future of ECHO "Technical hitches" What worked well 2C Technological considerations Interpretative Theme 1A: Optimised Patient Care The majority of participants described how they perceived that the ECHO program had a positive impact on patient care. Many felt this was due to the impact of ECHO on increasing their confidence when providing care to patients with diabetes in the primary care setting: I think my confidence in dealing with some patients with diabetes has risen significantly. (DIA/12) For some, this was achieved through collaborative working at the ECHO clinics: 24

25 It really raised my confidence in dealing with diabetic patients. I really enjoyed sharing cases with other GPs and realising that I wasn't on my own when I got stuck as to where to go next, and it just reassured me and made me feel more confident in taking the next step in treating my diabetic patients. (DIA/14) It's a confidence thing. It's really improving everybody's confidence in how they manage these patients and bringing our expertise up a little bit more and helping to understand how other people deal with their problem cases that we have, which is extremely important for all of us. (DIA/15) One participant felt that ECHO would also be beneficial to his colleagues in helping build their confidence in managing patients with diabetes: Some of my other partners tend to be very reluctant (regards diabetes). anybody who has a blood sugar above 7 tends to get landed at my door, but I would encourage my partners to take part in something like this, which would help. (DIA/15) One participant perceived that the ECHO program had increased their knowledge and skills in relation to diagnosing the type of diabetes a patient has: There were quite a few things I learned from this. I thought it was useful to go through the diagnosis of diabetes, which at first glance seems relatively straightforward but in a small number of cases can be difficult and different, and looking at things outside of the normal Type 1 Type 2 diabetes was useful and made me think again about categorising people with diabetes. (DIA/05) Some participants also illuminated that the ECHO clinics had increased their awareness and confidence in relation to considering new treatment methods: I thought all the presentations were useful but I thought it was particularly useful, the discussions around some of the newer drugs and how they can be used effectively. (DIA/05) I think the thing for me was where to go after you d started a couple of different types of drugs and you re thinking about referring the patient for consideration of insulin and becoming more comfortable with some of the newer drugs, the sodium-glucose cotransporter 2 (SGLT2) inhibitors, and things like that, and certainly it has helped with that significantly. (DIA/12) 25

26 Overall, these findings have highlighted how the learning gained through the ECHO platform enhanced the confidence and clinical skills of the participants. This was perceived to enhance patient care due to raised awareness amongst participants on how to optimally diagnose and treat patients with type 1 and type 2 diabetes. Interpretative Theme 1B: Created an effective learning environment Many participants expressed that the enthusiasm and encouragement at the hub added to the positive learning environment provided through the ECHO platform. Many felt that this positively attributed to the success of the program: The enthusiasm at the hub is really, really important and I think on this occasion was extremely successful. (DIA/15) There was great enthusiasm from the hub, which sort of encouraged and motivated everybody, and I think that was extremely useful and helpful. (DIA/15) Findings additionally illuminated that participants found the multi-professional presence at the hub beneficial: The other point I would add, which I thought was particularly good, was the multi professional aspect to it, the fact that we had not just GPs and consultants but there were diabetic specialist nurses, dieticians and others there. So there was a good breadth of experience and complementary knowledge sets and experience sets there. (DIA/05) I think just to emphasise what was said earlier about the multidisciplinary approach, I think having groups of people who are coming at it from a different angle really means that you're sharing a lot more information. Having the diabetic specialist nurses and the dieticians and the consultants and GPs and the practice nurses, all in the one group, really enriched the experience and really helped us all to appreciate where others are coming from. (DIA/05) Findings also highlighted how participants perceived ECHO as providing an effective learning environment through encouraging the sharing of knowledge and experiences: One of the big benefits for me has been hearing what my colleagues [the spokes] are doing and realising that I can be a little bit more adventurous with some things and that I can be a little less adventurous with other things, and actually realising that we re pretty much doing much the same stuff, maybe approaching it in slightly different ways but very definitely doing very similar things. (DIA/12) 26

27 One participant highlighted that the ECHO experience had highlighted areas for further learning: For me it s highlighted a lot I don't know, really it s highlighted areas within diabetes that I realise actually I need to do a lot more reading in that area, and then it s reinforced other areas where I feel more comfortable. But it certainly highlighted a lot of things that I didn t know, that I didn t know. (DIA/05) The majority of participants conveyed that ECHO provided and effective learning environment due to the formats ability to adapt to their learning styles. This perception resulted from the mixture of didactic and experiential learning provided at the clinics: In terms of the content, I thought the content was very good. It was well presented and there were lots of new information and lots of experiential learning, as well as factual learning. (DIA/05) The presentations, I thought, were very good, very thought provoking and encouraged me to read around them. But I got the most out of our discussions. Our case discussions, I think, were extremely well thought through. (DIA/15) One participant also illuminated that they benefited from the interaction with other clinicians: The way I like to learn is interaction with people, so this is very easily accessible interaction with other colleagues and with consultants and other specialists. So I was very grateful for the experience. (DIA/14) Accounts demonstrated how the ECHO platform created an effective learning environment for GPs with a special interest in diabetes. This was achieved by the presence of an encouraging hub and an environment that nurtured the sharing of knowledge amongst clinicians. The learning environment was also optimised as the platform lended itself to meeting the varying learning styles of participants and providing them with the opportunity to further consider their clinical practice. 27

28 Overarching Theme 2: Consideration for the Future of ECHO Findings were indicative of the need for further considerations when providing future ECHOs. This was conveyed though participant s views on the need for appropriate funding to participate in ECHO and who should participate in ECHO, both at the hub and the spokes. Findings also illuminated the technological considerations that may be needed to enhance the ECHO experience. (Figure 3) Interpretative Theme 2A: There needs to be funding Participants conveyed that for ECHO to successful in the future, there needs to be adequate funding. The majority of participants highlighted the need for protected time to be funded for future ECHOs: For this particular set of 10 [current ECHO clinics] we all got payment for it so we were able to protect the time, we were able to set aside the time to do it. But if this was an ongoing thing and we were looking at doing it once a fortnight, I suspect that the enthusiasm might wane as people realise that life is very busy and if I'm not being funded to provide backfill for this how am I possibly going to keep it going? (DIA/12) One of my concerns for the future of it, is how do we manage to get protected time in what is a very busy day, a very busy week, in order to get a group of people together? Time is a big factor here for all of us in general practice and as it is obviously in hospital as well, and the funding does give us protected time. (DIA/12) One participant also highlighted the need for appropriate funding to ensure the teaching remains of a good quality: What it necessitates though [funding] is that the teaching is of high quality and it is really worthwhile. (DIA/15) In summary, the presence of appropriate funding is needed to facilitate future ECHOs. This is perceived to be needed not just to protect the time of clinicians to participate, but also to ensure the high standard of education provided is being maintained. Interpretative Theme 2B: Future ECHO Participants Findings also conveyed perceptions of who should participate in future ECHOs to optimise the learning gained. One participant highlighted that having more than one team member at each spoke would be of benefit to clinical practice: 28

29 There was one session where my practice nurse joined me, and that was useful because we were able to follow and complete the discussion outside of the ECHO session and it meant that the learning was much more widespread within the practice. And I think that is something I would encourage for the future, is we try and have teams of people meeting together, not just one individual from the team, and I think that if you can create enthusiasm in our practice then the standard of care will rise much higher. (DIA/12) Participants also illuminated their thoughts on who should be present at the hub for future ECHOs. One participant conveyed how the presence of other specialist clinicians who advise on conditions related to diabetes, would benefit the hub: Maybe also to consider other specialties with regards, for example... there's so much, foot care, renal disease etc. obviously as part of the programme for a future date would be helpful as well. (DIA/15) Accounts displayed how membership at the hub and spokes must be carefully considered to ensure optimal multi-professional collaboration. It is also perceived that increased membership at each spoke will increase the impact of learning and thus optimise patient care. Interpretative Theme 2C: Technological considerations Findings were indicative of the need to consider the potential technological needs of participants for future ECHOs. Some participants conveyed that technical issues from the hub were often encountered: I thought the volume sometimes at the hub, or hearing what they were saying clearly at the hub was difficult at my end, that may have been our problem at our computer end, and every time the speaker [facilitator or teacher] moved or turned their head I seemed to lose the sound quite a bit. (DIA/15) The sound in the hub, occasionally it would drop, again, as people were turning their head, or it wasn t the person directly in front of the camera, if it was someone else who was contributing. (DIA/12) One participant expressed that they found it difficult to connect to ECHO through Wi-Fi in their workplace: 29

30 One day I had to work on Wednesday and I tried to link in from work, but we don't have Wi- Fi and the work computer wasn't set up, I didn't have the speaker and the headphones. It was my own fault. I tried to bring my laptop in but couldn't connect up because the laptop can't connect to work. So issues with NHS or GP connection that was all. (DIA/14) However, the technology was also considered as promoting optimal participation in ECHO due to the flexibility it provided: The other thing just to say is I set myself a target of trying to be in a different location for every session, and the car was parked in different car parks around Northern Ireland, so 3G, 4G signal was perfectly adequate, which I didn't expect it to be. I actually expected it to drop out a lot more than it did. So I think the technology worked really well. (DIA/14) One participant perceived however that the success of the technology can often be attributed to initial installation of the equipment and the ongoing administrative support: I think just the initial setting up of the various places where you're going to do ECHO is very, very important and have shadow runs prior to the actual meetings to make sure everybody is up and running would be helpful and less stressful. The ECHO administrator did a great job [in coordinating these], I have to say. (DIA/15) Discussion While the knowledge assessment for the diabetes ECHO did not demonstrate a substantial improvement in knowledge scores and no statistical analysis was possible due to low numbers, self-efficacy scores improved following the completion of the ECHO knowledge network, and the feedback received from participants was extremely positive. When asked about their overall views of participating in the network, 100% stated they had learnt a lot through participation and enjoyed it, 100% felt it helped translate knowledge into practice more than other teaching sessions they had been involved in and had improved the care they provided for patients, and 100% would participate again. A potential reason for the lack of improvement in knowledge score when compared with the perception of learning was the assessment itself- it was difficult (average scores were 54% and 57%) and may not have fully assessed the topics that were being taught in the hub. This was likely due to the tight timeframe of setting up the evaluation before the ECHO started when the curriculum had not been finalised, and could easily be addressed in future evaluations, with knowledge assessments being written based on the curriculum that will be covered in the upcoming 30

31 ECHOs sessions. It also highlighted the disconnect that existed before the ECHO between what consultants in Diabetes perceived as the diabetic knowledge level among GPs. The focus group findings highlighted the safe and effective learning environment that the ECHO platform can provide for GPs working with patients with diabetes in the primary care setting. ECHO optimised the delivery of care to patients through effectively enhancing the knowledge and skills of GPs in relation to a variety of areas such as: diagnosing diabetes and utilising new and innovative treatment options. Participants also illuminated the importance of the hub and how effective it had been in motivating and encouraging the spokes. For future ECHOs it has been suggested that specialists in other areas should be invited to the hub, and spokes should consist of multiple members of the care delivery team. ECHO was conveyed as an effective way to deliver clinically relevant education that promotes patient care and it provided a key network of support for all involved. The response rate to the evaluation was low and future evaluations should have mechanisms in place to encourage participation, for example funding being dependant on participation in the evaluation process or protected time being made available through additional funding. This could help prevent bias and improve the generalisability of results. While spoke members commented that they felt that participating in ECHO had improved the care they provided for patients, it was not within the scope of this evaluation to look at the impact on service delivery or patient care, and future studies should look to address this to consider the cost effectiveness of ECHO through potential reductions in referral rates to secondary care, improvements in diabetic control and reduction in long term complications. Suggestions from evaluation - The diabetic ECHO network should continue and be made available to more GPs and other primary care professionals - Technological issues need addressed including sound quality from the hub and connectivity from NHS / GP sites - Funding needs to be continued to allow protected time for participation and ensure high quality teaching from specialists at the hub is maintained - More than one participant at each site should be encouraged to enhance the community of practice locally and to allow spoke participants to continue the discussions and learning after the ECHO session is finished - Future knowledge assessments should be based on the curriculum being taught 31

32 - Participation in the ECHO network and receipt of funding could be dependent on participation in the evaluation process, or funding should be made available to allow participants to take part in the evaluation. This could improve the generalisability of results and prevent bias - Future studies should look at impact on service delivery and patient care, for example referral rates to secondary care and diabetic control of patients managed in the primary care setting, to determine if participation in the ECHO knowledge network has a direct impact on these areas. 32

33 Chapter 4 - Optometry ECHO Background Ophthalmology is a high demand specialty, typically accounting for 7-8% of all outpatient appointments, both regionally and nationally, each year. In the Northern Ireland context, this results in an excess of 100,000 out-patient appointments annually. Many ophthalmic conditions are age-related, and many are long-term conditions (LTC s) where cure may not be possible, and management to maintain useful vision is the goal. Glaucoma and macular degeneration are two such conditions. The Optometry ECHO had the objective of, providing primary care optometrists a safe space to improve their knowledge base, and in turn helping them to better manage patients with glaucoma or macular eye disease (age-related macular degeneration - AMD). By telementoring and case-sharing, ECHO aimed to enhance the available knowledge and clinical diagnostic skills in primary care, thus helping to improve case-handling and referral patterns. The Intervention The Optometry ECHO knowledge networks were on Friday mornings from am and covered topics relating to AMD and Glaucoma. Twenty-one Optometrists from across the region took part in 12 ECHO sessions to gain additional knowledge, skill and confidence in the diagnosis and management of patients with these long term ophthalmic conditions. The ECHO curriculum supported these primary care providers in elements of care, such as diagnosing macular disease at the margins and how and when to refer. For more details see table 5. Hub - The hub members included academic clinicians who were specialists in the identified sub-specialties of AMD and Glaucoma. These included the HSCB Ophthalmic Service lead, a Clinical Senior Lecturer and Consultant Ophthalmologist, a Professor of Ophthalmology and Consultant Ophthalmologist, a Biomedical Scientist, a Primary Care Optometrist, a university research Optometrist and a Clinical Optometric Adviser from the HSCB. Spokes - The spokes were selected because they are all holders of the College of Optometrists Professional Certificate in either (i) glaucoma or (ii) medical retina. The ECHO programme was alternately around these sub-specialities. 33

34 Table 5 Optometry / Ophthalmology ECHO Optometry/Ophthalmology ECHO Clinical Raymond Curran Champion Facilitator Educators Margaret McMullan Prof Agusto Azuaura-Blanco (Glaucoma) Mr Michael Williams (AMD) Dr Ruth Hogg: Lecturer QUB Dr Linda Knox Dr Gerry Mahon: Biomedical Scientist QUB/BHSCT Admin support Claire Armstrong Frequency of Weekly (12) clinics Fridays at am Training day Friday 13 th November at Jennymount Sessions 20, 27 November 11, 18 December 8, 15, 22, 29 January , 12, 19, 26 February 2016 Hub members Mr Michael Williams:Clinical Senior Lecturer, Honorary Consultant Ophthalmologist QUB/BHSCT Prof Agusto Azuara-Blanco: Clinical Professor/Consultant Ophthalmologist QUB/BHSCT Dr Ruth Hogg: Lecturer QUB Dr Gerry Mahon: Biomedical Scientist QUB/BHSCT Mr Alan Rundle: Primary Care Optometrist Raymond Curran: AD of Integrated Care, Head of Ophthalmic Services, HSCB Mrs Margaret McMullan: Clinical Optometric Adviser, HSCB ECLO s (Eye care liaison officer) invited as required pertinent to curriculum Hub costs Spoke members Spoke costs Based on 12 weeks of clinic and 4 hours clinic time (inc prep) and one session of training 11,481 paid to the 5 HSC Trusts on 1 st December (for participation in all ECHO networks as required) 21 Primary care optometrists to allow hours 165 per ECHO session 34

35 Spoke 8 webcam & microphones/headsets Equipment Majority using their own equipment Equipment costs 8 webcams & microphones/headsets x 50 = 400 Curriculum Developed. Alternating between glaucoma and AMD with a session on Optical Coherence Tomography AMD - Presentations, photos and images to make decisions on Glaucoma - The optic disc and glaucoma: Importance of disc size. Recognising a normal disc. The ISN'T rule. Difficult discs. Tilted discs. Glaucoma suspects AMD - Diagnosing macular disease at the margins. How and when to refer Glaucoma - Fields. Some dots are missing: Is it glaucoma? What do I do? And: is the patient fit for driving? Glaucoma - IOP: Ocular hypertension - OHTS study - central corneal thickness - refractive surgery AMD - What should be done for patient with early AMD? Are supplements worth it? Glaucoma - Systemic medications and glaucoma. Complianceadherence: what to do? AMD - OCT Machines: Believers, Non-Believers and Doubters why buy one/why not buy one/what to do with one? AMD - Neovascular AMD: diagnosis, monitoring and management. What to tell patients about it? Glaucoma - Rare Discs AMD - Dry AMD and the Eye Clinic Liaison Officer: how patient s needs are addressed Glaucoma Gonioscopy Evaluation Methods Evaluation was undertaken as described in Chapter two. All participants completed their pre-test evaluations at the training day, and were ed their post-test evaluations which were returned by post or . Results Twenty one spoke participants completed the pre-echo assessments (response rate 100%), with 11 (response rate 52.4%) completing the post ECHO assessments (only 10 completed the retro-pretest evaluation of self-efficacy). Demographic data for all participants are shown in table 6. For the rest of the results only the participants who completed the pre and post evaluations are included. 35

36 Table 6 Optometry Demographic data Pre ECHO Evaluation Range Totals Age Gender Male 4 Female 17 BSC Honours Qualifications Optometry 18 Other 3 Do you already hold Prof Cert in Yes 20 Glaucoma? No 1 Do you already hold Prof Cert in Yes 3 Medical Retina? No 18 Years in profession < Belfast 8 LCG area of practice Southern 2 South Eastern 2 Northern 7 Western 1 North & West 2 Current Area of Work Rural 5 Urban 10 Mixed 6 Current area of work Independent Practice 14 Corporate / Multiple (>3 practices) 6 Do you have access to OCT in your Yes 6 practice? No 15 Do you regularly use OCT in your Yes 5 clinical practice? No 16 Is the Optometry practice you work in Yes 17 connected to FPS BSO web portal? No 4 Knowledge and Self-efficacy Assessments The mean scores of knowledge improved between the pre-echo and post-echo assessments. Average knowledge scores improved from 23.7 to 27.6 (out of a possible 38 marks); from 62% to 73%. Only one participants score dis-improved, one was the same and the rest improved. Due to the small sample, non-parametric tests were used to analyse the 36

37 data. Wilcoxon results indicated a significant improvement in knowledge (p=0.008) pre and post-echo. More details are shown in table 7 and figure 4. Eleven participants completed the pre and post-test self-efficacy evaluations, with ten the retro-pretest evaluation. Self-efficacy results improved (table 7), with a lower average retropretest evaluation score than pretest score (in all but one area), and a higher average score for the post-test evaluation in all areas. The different domains are shown in table 8. Due to the small sample, non-parametric tests were used to analyse the data. A Friedman test showed that overall self-efficacy improved significantly across the three (pre, retro-pre and post) time points (p=0.006); post hoc tests using a Bonferroni adjusted alpha (p=0.017) to control for Type I errors revealed statistically significant improvement in post ECHO selfefficacy (p=0.007) and between Retro-Pre and Post ECHO self-efficacy (p=0.008). More details are shown in table 7 and figure 5. Table 7 Optometry ECHO Knowledge and total self-efficacy results Pre-ECHO Retro-pre ECHO Post ECHO Centiles 25th Median 75th 25 th Median 75th 25th Median 75th Knowledge Self-efficacy Figure 4 Box Plot for Optometry Knowledge Scores 37

38 Table 8 Optometry ECHO Self efficacy assessment - Participants self-rated confidence in each area (1 - not confident at all, 5 - very confident) Question Pre test Retro Pretest Post test Average Average Average (Range) (Range) (Range) To examine and assess optic nerve head appearance (3-5) (3-4) (4-5) Identify signs of glaucomatous damage to the optic nerve head (3-5) (3-5) (4-5) Interpret visual field plots (2-5) (2-5) (4-5) Identify signs of macular disease (both types dry and wet) (3-5) (2-4) (4-5) Differentiate the clinical appearance of wet AMD and Dry AMD (2-5) (2-5) (4-5) Interpret OCT scans with respect to macular changes (1-4) (1-4) (3-5) Overall I am confident in my ability to do my job well and provide a safe and quality (3-5) (3-4) (4-5) service to patients Figure 5 Box Plot for Optometry Self Efficacy Scores 38

39 General Feedback of Optometry ECHO Box 2 demonstrates participant s views on ECHO in general. Participants were very positive about their experience of ECHO, with 100% having learnt through participation, 100% felt it helped translate knowledge into practice more than other teaching sessions they had been involved in, and 100% would recommend it to others and participate again. Box 2 General ECHO Optometry Results 1. Please rate each on a scale of 1-5 the quality of learning / usefulness from each area (1- poor, 5- excellent) Review of previous session Presentations Case based discussions Overall do you feel you have learnt through participating in ECHO? A lot 10 A little 1 No 0 3. Did you find participating in ECHO enjoyable? A lot 10 A little 1 No 0 4. Do you think that participating in ECHO has improved the care you provide for patients? A lot 7 A little 3 No 0 5. Do you think the format of ECHO helps translate knowledge from teaching into practice more than other teaching sessions you have been involved in? Yes 11 No 0 39

40 6. Would you recommend ECHO to other healthcare professionals in your area? Yes 11 No 0 7. Have you used any of the online resources via Moodle, and if yes have you found these useful? Used and found Used and found NOT useful useful Power point presentations 4 1 Video of the teaching sessions 6 1 Video of case presentations 4 1 Other supporting materials Regarding ECHO technology Agree Disagree Unsure It has given me access to education that would have been hard to access due to geography It was a good medium to access teaching / education at a different location from where I work Any technical difficulties were acceptable and did not put me off participating in ECHO Any technical difficulties did not significantly reduce my learning How do you rate your overall ECHO experience? (1- poor, 5- excellent) Would you participate in ECHO sessions in the future if the opportunity arose? Yes 11 No 0 40

41 Focus Group Ten optometrists participated in the focus group from the spokes for the Optometry ECHO. Analysis of the focus group data uncovered three overarching themes, each with their own descriptive and interpretative themes that are outlined in thematic diagrams. Overarching theme 1, ECHO Enhanced Clinical Knowledge and Skills, explored the reasons why participants perceived that ECHO optimised their clinical practice (figure 6). Interpretative themes 1A, 1B and 1C highlighted the contributing factors that resulted in participant s perceived enhancement of clinical practice. Overarching theme 2, ECHO Exceeded Expectations and Changed Misconceptions, highlighted the preconceptions participants had before ECHO commenced and how these altered throughout the project (figure 7). Interpretative themes 2A and 2B illuminated the intricacies of why and how perceptions changed in a positive way. Overarching theme 3, Consideration for the Future of ECHO, displayed the key conceptions that participants perceived should be considered for future ECHOs (figure 8). Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills This theme encompassed how participants perceived ECHO to be an education platform that enhanced their clinical knowledge and skills (figure 7). Interpretative theme 1A depicted how ECHO provided an environment that facilitated the development of interdisciplinary relationships and communication, within the field of optometry and ophthalmology. Participants also perceived that ECHO optimised the care they delivered to their patients through the knowledge they had gained, and also through their increased confidence in making appropriate referrals. Interpretative theme 1C illuminated how ECHO also created a safe learning environment for participants. Findings highlighted that this was perceived to have been achieved by the hub members encouraging learning and treating all queries with respect. The learning environment was also enhanced by the relationships that were built between members of the spokes. 41

42 Figure 6 Theme 1 ECHO Enhanced Clinical Knowledge and Skills for Optometry ECHO Descriptive Themes Interpretative Themes Overarching Themes Strengthened the link between primary and secondary care providers "Relationship between ophthalmologists and optometrists has increased because of ECHO." 1A Nurtured interdisciplinary relationships and communication Increased optometrist confidence in their skills Reduced unnecessary referrals 1B Optimised Patient Care 1 ECHO Enhanced Clinical Knowledge and Skills Expertise at the Hub Encouraged sharing of knowledge Built relationships between optometrists 1C Created a safe learning environment "No questions are silly." 42

43 Figure 7 Theme 2 ECHO Exceeded Expectations and Changed Misconceptions for Optometry ECHO Descriptive Themes Interpretative Themes Overarching Themes Would it be too formal? How would the technology work? Addressed concerns about technology "Afraid that knowledge may not be up to par." 2A "Didn't know what to expect." 2B ECHO challenged misconceptions 2 ECHO Exceeded Expectations and Changed Misconceptions Figure 8 Theme 3 Consideration for the Future of ECHO for Optometry ECHO Descriptive Themes Interpretative Themes Overarching Themes "Cutting it fine to have your homework done." Accessibility of resources Potential for Enhanced relationships between optometrists and GPs Relevant to Other professionals in the community 3A Changes to be made 3B Furture ECHO participants 3 Consideration for the Future of ECHO 43

44 Interpretative Theme 1A: Nurtured interdisciplinary relationships and communication The majority of participants described how they perceived the relationships that were fostered within the ECHO environment. Many felt that the ECHO platform provided a forum which strengthened the links between primary and secondary care providers. Participants perceived that this was optimised through inter-professional learning about the role of optometrists in the community, and ophthalmologists in the secondary care setting: They [ECHO Hub] also learnt a lot from us [Spokes], what it's like in the front line [Community setting] and the decisions you have to make. So I think ECHO has been great for understanding, we understand more about how they work [Secondary Care Setting], what happens with patients but they also understand more about what it's like in the community. (OP/15) Participants perceived that through raising awareness of professional s roles, ECHO actively participated in facilitating interdisciplinary relationships: Realistically what it's [ECHO s] doing is building relationships, where beforehand there was this barrier of a hospital that stops us [Community Optometrists] from building that. (OP/1) Findings also illuminated that ECHO enhanced relationship building between optometrists and ophthalmologists: I think everybody would agree that certainly the relationship between ophthalmologists and optometrists has increased because of ECHO. I'm sure that the ophthalmologists would probably think a lot more about our knowledge and certainly understand what we're looking at in practice, and we also understand more about what they're looking at and what they would want to see and that relationship has been fantastic. (OP/1) Overall, these findings suggested that the unique learning environment that was nurtured by the ECHO platform facilitated the building of relationships between professionals across the primary and secondary care setting. This led to direct enhancement of interdisciplinary awareness of the key role various clinicians have in patient care. 44

45 Interpretative Theme 1B: Optimised Patient Care Participants also envisaged that the ECHO program had a positive impact on patient care. Many felt this was due to ECHO increasing the perceived confidence they had in relation to their clinical skills. This resulted from the education provided at ECHO: I felt so much more confident on the glaucoma case about what to look out for and what way to manage it. (OP/14) I think we've done macular and glaucoma in depth and it was fantastic, what we've learnt out of it and certain learning points, I definitely feel more confident managing this in practice. (OP/15) For many this was a very positive outcome from ECHO as they had hoped taking part would improve patient care: I definitely went into ECHO hoping that it would be able to improve the care that I provide to patients, and I definitely feel that I have. (OP/12) Findings additionally illuminated the perceived impact of ECHO on referrals to secondary care services. The majority perceived that the learning they had achieved through ECHO enabled them to make more appropriate referrals in relation to various conditions: Our disc assessment, I think, is better, our ability to reason and to look and decide whether we want to keep the patient in practice or refer, I think it's better. I think it'll reduce, certainly for me it will reduce some unnecessary referrals but it will also help me pick up on things that I really need to refer. (OP/1) I think I feel more confident now, when I'm doing things with patients, to look close, especially looking at discs or looking at the macula more closely and thinking 'what else could be going on that we can't see?' that it's perfectly okay to send somebody for further investigation without necessarily knowing exactly what you can see or what you can't see, but also being a bit more confident about what you can see and thinking 'yeah, this is definitely something that needs sorted out' rather than being a bit woolly about what you're sending in. (OP/13) The majority of participants also perceived that the learning gained had helped them to feel more confident in dealing with particular conditions. This resulted in participants reducing unnecessary referrals: 45

46 I found that knowing the pathway and knowing the treatment that our patients were going to be receiving in the hospital helped me to recognise when I'm actually more happy and confident to keep them in practice and when it's safe to do that, and that's been a really useful tool. So of course, it's lovely to be able to refer them when I'm sure and I have the confidence to do that, but also the reverse, to keep the patients here and to have confidence in our own ability and skill set. (OP/11) Findings also highlighted perceptions that reducing the number of unnecessary referrals would result in decreased pressure on secondary care services. One participant expressed the perception that more education through ECHO on anterior segment conditions would help decrease the demand for acute eye service referrals: If we were more confident about dealing with these [anterior segment conditions] the pressure on acute eye services would be lessened. (OP/9) In summary, these findings have highlighted how the learning gained through the ECHO platform enhanced the confidence and clinical skills of the participants. This resulted in improved patient care due to raised awareness amongst participants of how to act on and manage specific conditions, and appropriately refer to other services. Awareness of appropriate referral criteria also resulted in the perception that ECHO could reduce referrals to secondary care services. Thus, enhancing the provision of care in the primary care setting and ultimately reducing the demand on secondary care services. Interpretative Theme 1C: Created a Safe Learning Environment Many participants expressed that the expertise at the hub added to the positive learning environment provided through the ECHO platform. Findings illuminated how the ophthalmology presence at the hub was beneficial to the spokes: I personally found it really beneficial, especially the two ophthalmologists that sat in on a lot of the sessions, you were getting feedback from them when they were receiving our referrals, and local protocols were particularly beneficial. So I certainly definitely felt I had benefitted from it. (OP/12) The majority also expressed that alongside the hub, the spokes provided encouraging peer support and advice at the sessions: 46

47 Just the fact that you had an ophthalmologist there and you have your peers, that you can get an immediate answer, it's just brilliant, because quite often you're just left wondering about things, 'am I doing the right thing?' and so it s just to have that immediate response was just great. (OP/14) The majority of participants also referred to the important role of the facilitator at each session. Findings indicated that participants perceived the facilitators to be key in optimising the learning gained at the ECHO session: Both the facilitators had listened to what we were looking for them to talk about at ECHO and they really stuck to that, they really looked at it, and they listened to that and then they addressed them, and it was incredibly useful. (OP/1) The ECHO hub was also regarded by participants as providing a safe learning environment through the support provided by the hub when asking questions: I think every question that was brought up, it was made to feel significant and you felt you could bring up any issue, which I think is very important. (OP/14) Alongside the perceived benefits of ECHO to those directly involved, participants also expressed that the learning they gained through ECHO was shared amongst other professional colleagues. Many expressed that colleagues would come to them for guidance due to their involvement in ECHO: Colleagues that weren't involved in the ECHO session, they would sometimes come to me with cases that they're not so sure about and I definitely feel more confident in providing answers to their queries as well. (OP/12) This resulted in a perceived benefit to the other colleagues patients: I found other colleagues asking me things and it was something that I could maybe bring as a question in ECHO, so the whole team did learn, and because I work in different practices, that was spread among different practices too. So yeah, it's helped in the care of my patients and in the patients of my colleagues too. (OP/15) The majority of participants highlighted that the ECHO program allowed for relationship building between the spokes. Many expressed that they felt less isolation within their field 47

48 due to building relationships with the various optometrists who were partaking in the program: So I think, from a relational point of view, it's been fantastic as well, as well as getting to know the spokes. I had never met X [member of the spoke] and yet last week I walking into a room and chatted to him as if I had known him for months. So, at the end of the day, I've got to know another 20 optometrists that I would feel very confident going and talking to if I was to meet in a meeting. So I think that's been fantastic. (OP/1) I think with meeting all the spoke optometrists too, you know, we know names but we never see each other really, so it was great to put a face to a name. (OP/09) Accounts demonstrated that participants viewed ECHO as a safe learning environment that was enhanced through the expertise, and skilled facilitation, present at the hub. Key learning was also nurtured from the peer support provided by the spokes and the ability to build relationships with other colleagues. Evidence also displayed how ECHO can often reach wider than the participants themselves. Participants shared the learning gained with other colleagues and thus perceived the impact of ECHO to go beyond them and their patients to the patients of others. Overarching Theme 2: ECHO Exceeded Expectations and Changed Misconceptions Overarching theme 2 encompassed how participants perceived ECHO as exceeding their expectation and altering the preconceptions they held before participating (figure 7). Interpretative theme 2A conveyed how participants were wary of ECHO before it started due to a lack of awareness of what it specifically entailed. However, interpretative theme 2B encapsulated how the misconceptions participants held were changed throughout the program. Interpretative Theme 2A: Didn t know what to expect. Many participants conveyed how they were concerned that ECHO would be too formal. However this perception changed: I think you probably did think it might be a little bit more formal than it was, and I think it was really, like the others have said, because it was quite informal, even with the ophthalmologists in there and their lectures and things, it never felt so formal that you couldn't interact and ask questions, (OP/13) 48

49 This participant expressed that the more comfortable environment was directly related to the expert communication skills of the hub members: The ophthalmologists that we had, seemed to be very good communicators. The level they were pitching things at, the things they were saying, the informality of the whole thing, I think they were extremely good at communicating. (OP/13) Findings also highlighted that many had concerns about the influence of technology on the ECHO program. Many were unsure if this would negatively impact on the interaction between the spokes: At the beginning I was unsure about the technology and I thought we would be very detached from one another. (OP/9) However, for this particular participant this concern was unfounded as the program went on: But as the weeks went on, I think the spokes started to talk more to one another and ask each other questions. (OP/9) Some were concerned about the use of technology due to their own computer skills: I was a bit apprehensive, first of all, before using it because I'm not a very computer person. (OP/20) Overall, the concerns regarding the use of technology and the format of the sessions were reduced as participants partook in the program. This was facilitated by the ECHO equipment, and also the members of the hub and spokes. Interpretative Theme 2B: ECHO Challenged Misconceptions The misconceptions held by participants before ECHO were reduced throughout the program. The reasons for this are illuminated within interpretative theme 2B. Accounts highlighted how initial fears regarding technology through the IT support given prior to commencing the program: But they came out and installed the camera for me in my office, and you literally just go in, type in the code and up it pops, and it's been fantastic, no bother. So very good. (OP/20) 49

50 I couldn't find any fault with it at all. It was as easy as anything. I literally pushed the button and the thing, the camera comes on, the sound works. I never had any bother with it at all. Once you put your code in it just turns everything on by itself, it was great. (OP/13) Participants also found it useful to use alternative tablet and Smartphone devices to participate in the ECHO clinics: I think it was very easy to use. I just used it on an ipad mini and I'm just amazed how easy it is to use, and I think people have been even accessing it on a phone, so it's been fantastic technology. (OP/15) Using mobile devices also allowed participants to have flexibility in where they joined the clinic: Because it's on the ipad, and I do locum and I've done it in three other practices and at home, so it's been amazing, you take your ipad and connect to the Wi-Fi, and you can really do it anywhere and that's the amazing thing about it. (OP/15) It was expressed by some participants that they held the misconception that ECHO would not be aimed at a high enough level to meet their learning needs. However one participant highlighted that this was not the case: I think my concern going into ECHO was that it was going to be pitched at a very low level and we would end up listening to an awful lot of stuff that we already knew, and it turned out to be not like that at all. (OP/1) On the other hand, some participants expressed that prior to partaking in ECHO many may feel they do not have the knowledge required. However it was perceived that ECHO does provide a safe environment for learners at various levels: Others may be afraid that their knowledge may not be up to par, and I think it's important that people do realise that ECHO is a safe environment and that you're not going to be wrong, as such, and you can get your questions answered without anybody judging you, which I think is very important. (OP/14) In summary, findings conveyed that misconceptions regarding ECHO were soon altered after the program commenced. This was facilitated by the ease of use of IT equipment and the safe learning environment provided by the ECHO platform. 50

51 Overarching Theme 3: Consideration for the Future of ECHO Findings were indicative of the need for further considerations when providing future ECHOs (figure 8). This was conveyed though participant s views on changes to be made that were discussed in interpretative theme 3A. Findings also illuminated the role of ECHO for other participants and this was demonstrated in interpretative theme 3B. Interpretative Theme 3A: Changes to be made Participants conveyed that in order to effectively repeat the ECHO program, more consideration may be needed regarding the amount of time participants are given to review the case presentations. Many expressed that they did not feel they were given sufficient time to prepare for the session: For me personally, the information was usually ed, but it tended to come out on the Thursday, just the day before ECHO, and I work late on a Thursday and it was just you were getting home and by the time you were looking at it, it was maybe 9:30, 10 o'clock at night, which was when you were a bit more tired. So if we had it maybe... it did sometimes comes out on the Wednesday, but if it did come out a couple of days beforehand it would be useful to have a bit more time to look over the information. (OP/20) There were some weeks that you were cutting it quite fine to have done your homework, so it would be useful. It's not always possible but it would have been useful to have had it a bit earlier. (OP/11) Even 24 hours earlier, if we had the information 24 hours earlier. I, as well, struggled and I was looking at the case presentations at 9 o'clock before ECHO started and it would have been better to have it a day earlier. (OP/9) Many participants also conveyed that they had difficulty accessing online resources that were made available to those participating in ECHO: I found a little bit of difficulty getting onto the site with the resources. (OP/09) I did have a look on the online resource site, it did take me a while, I did find some resources but it was a wee bit of hoking around to find it. (OP/20) I did find it difficult to get on to the online resource site and some of the links were difficult to open. (OP/19) 51

52 In summary, findings highlighted the need for future ECHOs to consider the time participants are giving to prepare for the session. Accessibility of resources that are made available throughout the program also needs to be addressed to ensure participants receive optimal information and learning. Interpretative Theme 3B: Future ECHO Participants Findings indicated that ECHO is perceived to be of benefit across multiple professions, and healthcare settings. Participants conveyed that ECHO may be a beneficial platform to use to enhance interdisciplinary working between optometrists and GPs: So communication can only be good, and you could even imagine this [ECHO] between us [optometrists] and GPs, perhaps, or within different professions, which would perhaps be beneficial as well. (OP/15) Ultimately this could promote partnership working in the primary care setting: I think GPs don't necessarily understand what we know, some of the time. The GPs in the rural area where I work, send me a lot of patients but I'm not sure that GPs in general understand how much more useful we can be to them than they think we are. (OP/13) Participants also envisaged that the ECHO model would be beneficial across other disciplines: I would imagine this model could be shared out among any profession. In any healthcare profession there would be cases that could be brought and discussed [at ECHO] with perhaps somebody, maybe a consultant with more junior doctors, in the same situation, we would have peers and also those who can provide answers. So I imagine it would be quite easily shared out. (OP/14) I was saying to a dentist a few months ago, he's not long qualified but he's taken a different route than most dentists, he's gone in to hospital dentistry and he's seeing a lot of very interesting cases, and one comment he made was I'm glad I'm not out in the community setting. So it sounded to me like oh dear, that's quite like our profession. So I think dentistry, yeah, a general dentist would probably gain the same way that we have from ECHO. (OP/9) 52

53 Interpretative theme 3B illuminated the benefits of ECHO across disciplines and how it could promote partnership working, especially within the primary care setting. Discussion Optometrists who participated in the ECHO knowledge network demonstrated statistically significant improvements in knowledge and self-efficacy following the 12 ECHO sessions. When asked about their views on ECHO in general, 100% stated they had learnt through participation, 100% felt it helped translate knowledge into practice more than other teaching sessions they had been involved in, and 100% would recommend it to others and participate again. The focus group findings highlighted the safe and effective learning environment that the ECHO platform provided for optometrists working within the primary care setting. ECHO optimised the delivery of care to patients through effectively enhancing the interdisciplinary relationships between optometrists and ophthalmologists. Patients also benefited from the enhanced knowledge, skills and clinical confidence that participant s experienced as a result of ECHO. Although participants often had their doubts about the effectiveness of ECHO due to technological concerns, or a desire to maintain credibility amongst their peers, the ECHO pilot soon changed these perceptions by the reassuring and encouraging environment provided by the hub, and by the hub facilitators. ECHO was perceived as an effective way to deliver clinically relevant education that will promote safe and quality patient care and reduce unnecessary referrals to secondary care services. It provided a key network of support for all involved and their colleagues. Irritants such as the tight timeframe of receiving homework 24 hours before the ECHO session leaving little time to prepare and the accessibility of resources on-line should be easily addressed in future networks. The response rate to the evaluation was lower than expected (52.4%) and future evaluations should have mechanisms in place to encourage participation, for example; funding being dependant on participation in the evaluation process, protected time being made available through additional funding and processes in place to encourage non-responders. While spoke participants felt that participating in ECHO had improved the care they provided for patients and had improved the appropriateness of referrals to secondary care, it was not within the scope of this evaluation to look at the impact on service delivery or patient care, and future studies should look to address this to consider the cost effectiveness of ECHO through potential reductions in referral rates to secondary care and improvements in optometry care for patients in the community. 53

54 Suggestions from evaluation - The optometry network should continue and be made available to more optometrists with consideration for involvement of other primary care professionals where relevant. - Case presentations and homework should be made available earlier to allow more time for preparation. - Training on how to access online resources needs to be improved. - Funding needs continued to allow protected time for participation. - Participation in the ECHO network and receipt of funding should be dependent on participation in the evaluation process, or funding should be made available to allow participants to take part in the evaluation. This should improve the generalisability of results and prevent bias. - Future studies should look at impact on service delivery and patient care, for example referral rates to secondary care and management of patients in the primary care setting, to determine if participation in the ECHO knowledge network has a direct impact on these areas. 54

55 Chapter 5 - Nursing Homes ECHO The Intervention The Nursing Home ECHO knowledge network supported nursing home staff throughout Northern Ireland to improve their knowledge and skills in the care and management of patients with a wide range of healthcare needs. Issues such as renal failure, palliative and end of life care, drug management, and heart failure were discussed with a team of multidisciplinary professionals. To enable the Nursing Homes to embed changes in their systems, the HSC Safety Forum also supported the nursing home staff to build knowledge in quality and safety improvement tools and techniques during the ECHO clinics. This included the Model for Improvement, the Plan Do Study Act (PDSA) Cycle, and measuring for improvement, along with a range of other ideas for change. The sessions were on a Tuesday afternoon from 2-4pm. This network completed 10 ECHO sessions. More information on the set up of this ECHO is shown in table 9. Hub - In the ECHO hub there were a number of HCPs including a Palliative Medicine Consultant, a Lecturer in Palliative Care, the HSC Safety Forum Clinical Director and Patient Safety Advisor, a Pharmacist, a Dietician and other HCP s as required including a Physiotherapist, Occupational Therapist, Speech and Language Therapist, Chaplain and Social Worker. Spokes - Recruitment was via three methods 1. Nursing Homes were recruited through the existing HSC Safety Forum Nursing Home collaborative. 2. Invitations were also send to homes within the Four Seasons and Care Circle organisations. 3. Expression of interest were sought through members of the Regional Transforming Your Palliative Care Programme Board. There were a total of 26 nursing homes involved with up to 70 staff participating at the spokes in any one session. 55

56 Table 9 Palliative Care in Nursing Homes ECHO Palliative Care Nursing Homes Clinical Champion/ Max Watson & Sue Foster Facilitator Admin support Frequency of clinics Session dates Training Hub members Hub costs Spoke members Spoke costs Equipment needed for spokes Total Spoke Equipment costs Curriculum development update Ciara McClements/ Rebecca Donnelly 10 sessions 2 hours each (Tuesdays 2pm-4pm) Weekly initially (November) and then every other week November 3 rd, 10 th, 17 th, 24 th December 8 th January 12 th, 26 th February 9 th, 23 rd March 15th 15 th September and 29 th October Facilitator/Educator Max Watson / Sue Foster and other NIH staff as required (Consultant / GP, Nurse, Pharmacist, Dietitian, Physiotherapist, Occupational Therapist, Speech and Language Therapist, Chaplain, Social Worker) Corrina Grimes - PHA Janet Haines Woods - HSC QI Safety Forum Gavin Lavery HSC QI Safety Forum Hospice staff costs Staff from 26 Nursing Homes across Northern Ireland. Nursing Homes will be paid an Education Grant in February of approximately webcams required (9 for Four Seasons) 12 installed Extra equipment required for Four Seasons: 2nd hand pc plus vat KVM switch - 37 plus vat USB wireless adapter (may be required) - approx 12 So approximately 200 x 9 homes = 1800 Approx 5, Communication 2. Symptom Management in Palliative & EOL Care 3. Pain Management 4. Ethical Issues 5. Drug Management 6. Recognising Death & Dying 7. Delirium 8. Behavioural & Psychological Symptoms in Dementia (BPSD) 9. Heart Failure 56

57 Evaluation Methods Evaluation was undertaken as described in Chapter 2. There were different assessments for registered and unregistered staff in both knowledge and self-efficacy. All participants completed their pretest evaluations at the training day or in their individual nursing homes under the supervision of their manager, and managers were ed the post-test evaluations to administer to staff and then return them by post or . Results Nursing Home ECHO Sixty two registered and 30 unregistered spoke participants completed the pre ECHO assessments (92 total), with eight registered nurses (response rate 12.9%) and two unregistered staff (response rate 6.7%) completing the post ECHO knowledge assessments. Seven registered and three unregistered staff completed the post ECHO self-efficacy assessments. Demographic data for all participants are shown in table 10 (registered staff) and 11 (unregistered staff). For the rest of the results only the participants who completed the pre and post evaluations are included. Table 10 Nursing Home Demographic data Pre ECHO Evaluation for Registered Staff Range Totals Age Gender Male 2 Female 60 Profession Registered Nurse 36 Team Leader/ Deputy Manager 12 Manager 13 Other 1 Years in profession < Current Area of Work Rural 8 Urban 17 Mixed 29 Not stated 8 57

58 Table 11 NH Demographic data Pre ECHO Evaluation for Unregistered Staff Range Totals Age Not stated 2 Gender Male 3 Female 27 Profession Healthcare assistant 23 Other 7 Years in profession < Current Area of Work Rural 8 Urban 6 Mixed 9 Not stated 7 Knowledge and Self-efficacy Assessments The mean scores of knowledge improved between the pre-echo and post-echo assessments. For registered staff average knowledge scores improved from 41.8 to 45.6 (out of a possible 50 marks); from 84% to 91%. No participants score dis-improved, two were the same and the rest improved. For unregistered staff one participants score improved from 76% to 84%, the other participant dis-improved by 1 mark from 90% to 88%. Due to the low response rate no statistical analysis was possible. Seven registered and three unregistered participants completed the three self-efficacy evaluations. Self-efficacy results improved (table 12 for registered and table 13 for unregistered staff), with a lower average retro-pretest evaluation score than pretest score in all areas for registered staff and in all but one area (heart failure patients) in unregistered staff, and a higher average score for the post-test evaluation in all areas. For registered staff, overall confidence, happiness and feeling of support improved, however, so did stress levels. Due to the low response rate no statistical analysis was possible. 58

59 Table 12 NH ECHO Self efficacy assessment for Registered Staff- Participants self-rated confidence in each area (1 - not confident at all, 5 - very confident) Question Pre ECHO Average (Range) Retropre Average (Range) Post Average (Range) Understand the principles of palliative care and apply them in my daily work 3.93 (1-5) 3.50 (2-5) 4.43 (3-5) Break bad news to clients/residents and their families 4.04 (2-5) Communicate effectively with clients/residents and their families Recognise when a client/resident is approaching the dying phase and manage the dying phase appropriately Know what medications will help a client/resident to control their symptoms and to give these appropriately Play a role in providing good symptom management to clients/residents 4.61 (3-5) 4.29 (2-5) 4.00 (1-5) 4.05 (1-5) Assessing and managing a client/resident s pain 4.13 (2-5) Provide appropriate bereavement support to clients /residents /their families Make decisions about clients/resident s care within an ethical framework 4.16 (2-5) 4.02 (1-5) Support and facilitate advance care planning 3.89 (1-5) 3.50 (2-5) 4.0 (3-5) 3.83 (2-5) 3.67 (3-5) 3.50 (2-5) 3.50 (2-5) 3.33 (2-5) 3.17 (2-5) 3.67 (1-5) 4.57 (4-5) 4.71 (4-5) 4.71 (3-5) 4.29 (4-5) 4.43 (3-5) 4.29 (3-5) 4.29 (2-5) 4.14 (2-5) 4.43 (3-5) Manage clients/residents with dementia 4.27 (2-5) 3.83 (2-5) 4.57 (3-5) Manage clients/residents with heart failure 3.84 (2-5) 3.50 (2-5) 4.00 (2-5) Manage clients/residents with renal failure 3.05 (0-5) 2.83 (2-4) 4.14 (3-5) Manage clients/residents with COPD Overall my confidence in my ability to do my job well and provide an excellent service to clients/residents is 3.32 (0-5) 3.46 (0-5) 3.17 (2-5) 3.83 (2-5) 4.14 (3-5) 4.57 (3-5) Overall my happiness in my job is 3.57 (0-5) Overall my stress level in relation to my job is 2.61 (0-5) Overall I feel supported in doing my job (0-5) 3.67 (2-5) 3.17 (2-4) 4.00 (3-5) 4.43 (3-5) 3.43 (3-5) 4.43 (3-5) 59

60 Table 13 Nursing Home ECHO Self efficacy assessment for Unregistered Staff- Participants self-rated confidence in each area (1 - not confident at all, 5 - very confident) Pre ECHO Retropre Post ECHO Question Average (Range) Average (Range) Average (Range) Understand the principles of palliative care and apply them in my daily work 3.60 (2-5) 2.67 (2-3) 4.00 (4) Comfort clients/residents/families who have received bad news 3.74 (2-5) 2.00 (2-3) 4.67 (4-5) Communicate effectively with clients/residents and their families Recognise when a client/resident is approaching the dying phase and play my role in helping with the dying phase appropriately 4.17 (1-5) 3.54 (1-5) 2.33 (2-3) 2.00 (1-4) 4.33 (4-5) 4.67 (4-5) Know when a client/resident needs medications to control their symptoms and to advocate for them 3.43 (1-5) 2.33 (1-4) 4.67 (4-5) Play a role in providing good symptom management 3.34 (1-5) 2.00 (1-3) 3.67 (3-4) Helping to monitor and help with a client s/resident s pain 3.60 (1-5) 2.00 (1-3) 4.00 (3-5) Provide appropriate bereavement support to clients/residents/ their families 3.46 (1-5) 1.67 (1-3) 4.00 (4) Make decisions about clients/residents care within an ethical framework 3.00 (1-5) 1.67 (1-3) 3.67 (3-4) Support and facilitate advance care planning 2.60 (1-5) Care for clients/residents with dementia 4.26 (2-5) Care for clients/residents with heart failure 3.40 (1-5) Care for clients/residents with renal failure 3.09 (1-5) Care for clients/residents with COPD 3.49 (1-5) Overall my confidence in my ability to do my job well and provide an excellent service to 4.11 clients/residents is (2-5) Overall my happiness in my job is 4.43 (0-5) Overall my stress level in relation to my job is 3.34 (0-5) Overall I feel supported in doing my job (0-5) 2.00 (2) 2.67 (2-3) 1.33 (2) 2.00 (1-3) 2.00 (1-3) 3.67 (3-4) 2.67 (2-3) 2.00 (2) 2.33 (2-3) 3.67 (3-4) 4.67 (4-5) 3.00 (0-5) 4.00 (3-5) 4.33 (4-5) 4.67 (4-5) 3.00 (3) 2.00 (2) 2.67 (2-3) 60

61 General Feedback of NH ECHO Box 3 demonstrates participant s views on ECHO in general. Due to low numbers of responses the registered and unregistered staff have been combined. Participants were very positive about their experience of ECHO, with 100% having learnt through participation and found it enjoyable, 100% felt it helped translate knowledge into practice more than other teaching sessions they had been involved in and improved the care they provided for patients, and 100% would recommend it to others and participate again. Box 3 General feedback from NH ECHO- registered and unregistered staff 1. Please rate each on a scale of 1-5 the quality of learning / usefulness from each area (1- poor, 5- excellent) Review of previous session Presentations Case based discussions Overall do you feel you have learnt through participating in ECHO? A lot 9 A little 1 No 0 3. Did you find participating in ECHO enjoyable? A lot 10 A little 0 No 0 61

62 4. Do you think that participating in ECHO has improved the care you provide for patients? A lot 9 A little 1 No 0 5. Do you think the format of ECHO helps translate knowledge from teaching into practice more than other teaching sessions you have been involved in? Yes 10 No 0 6. Would you recommend ECHO to other healthcare professionals in your area? Yes 10 No 0 7. Have you used any of the online resources via Moodle, and if yes have you found these useful? Used Used and found useful and found NOT useful Power point presentations 7 0 Video of the teaching sessions 6 1 Video of case presentations 6 1 Other supporting materials

63 8. Regarding ECHO technology Agree Disagree Unsure It has given me access to education that would have been hard to access due to geography It was a good medium to access teaching / education at a different location from where I work Any technical difficulties were acceptable and did not put me off participating in ECHO Any technical difficulties did not significantly reduce my learning How do you rate your overall ECHO experience? (1- poor, 5- excellent) Would you participate in ECHO sessions in the future if the opportunity arose? Yes 10 NO 0 Focus Group Five registered nurses and one nursing home manager who participated at the spokes for the nursing home ECHO participated in the focus group. Analysis of the focus group data uncovered two overarching themes, each with their own descriptive and interpretative themes that are outlined in thematic diagrams. Overarching theme 1 (figure 9), ECHO Enhanced Clinical Knowledge and Skills, explored the reasons why participants perceived that ECHO optimised their clinical practice. Interpretative themes 1A and 1B highlighted the contributing factors that resulted in participant s perceived enhancement of clinical practice. Overarching theme 2, Consideration for the Future of ECHO, displayed the key conceptions that participants perceived should be considered for future ECHOs (figure 10). 63

64 Figure 9 Theme 1 ECHO Enhanced Clinical Knowledge and Skills for Nursing Home ECHO Descriptive Themes Interpretative Themes Overarching Themes Enhanced relationships between the Nursing Homes and GPs Increased knowledge, confidence and skills regarding palliative and end of life care Importance of the Hub 1A Optimised Patient Care 1 ECHO Enhanced Clinical Knowledge and Skills Healthcare assistants benefited from ECHO Benefited from interaction with other homes 1B Created an effective learning environment Resources were excellent 64

65 Figure 10 Theme 2 Consideration for the Future of ECHO for Nursing Home ECHO Descriptive Themes Interpretative Themes Overarching Themes Time to participate Not on a weekly basis Too much in one session 2A Challenges Staff enthusiasm lacking at times 2 Consideration for the Future of ECHO Seperate sessions for staff without a medical or nursing background 2B Future ECHOs ECHO for GPs and Nursing Homes Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills This theme (figure 9) encompassed how participants perceived ECHO to be an education platform that enhanced their clinical knowledge and skills. Interpretative theme 1A depicted how participants perceived that ECHO optimised the care they delivered to their patients through the knowledge and confidence they had gained, and also through enhancing relationships with GPs. Interpretative theme 1B illuminated how ECHO also created an effective learning environment for participants. Findings highlighted that this was perceived to have been achieved by the effectiveness of the hub and the interaction between the spokes. The learning environment was also enhanced as ECHO was perceived as providing a platform which benefited healthcare assistants, alongside registered nurses. 65

66 Interpretative Theme 1A: Optimised Patient Care The majority of participants described how they perceived that the ECHO program had a positive impact on patient care. Many felt this was achieved by enhancing the relationship they had with GPs. ECHO aided in increasing participants confidence when discussing patients with the GP, and helping the GP to value their opinions on patient care: I think it s definitely made me feel more confident dealing with the likes of the doctors. After our last presentation, regarding our client with the pain in the hands, I was able to go back to his GP and have quite a lengthy discussion, and they were asking where I had got these suggestions from and once I mentioned this ECHO project and the consultant facilitator, she was straightaway on board with giving out the treatment. You re not just seen as a nurse trying to tell the doctor what to do; you ve a bit more evidence to back up what you re saying, I think, which has helped. (NH/3) We did our case study on a resident in her pain relief, we discussed what we had discussed on the case presentation with the GP and her medication has been reviewed as a result of that. So yes, it s been very productive. (NH/86) Many also felt that ECHO increased their knowledge, confidence and skills regarding palliative and end of life care: I think it s [ECHO] given a level of confidence and also I think the discussions have given the nurses something to check against, particularly say, around some drugs and their interactions and things like that, and I think it s just how people think a little more, which is just great. (NH/50) After the heart failure session it promoted us to have a discussion about monitoring and blood results for our resident that was given high doses of diuretics. So it just helped us to focus as well, rather than just blindly follow the treatment plan without any questioning or checking. So it was helpful for that resident as well. (NH/1) Overall, ECHO was perceived as optimising patient care through enhancing the relationship between nursing homes and GPs and increasing the knowledge, skills and confidence of participants. 66

67 Interpretative Theme 1B: Created an effective learning environment Participants conveyed that ECHO created an effective learning environment for the spokes. Accounts portrayed that this was contributed to by the effectiveness of the hub and the facilitator: The consultant facilitator was amazing, very respectful of what s in nursing homes and what we re doing and always constantly very positive about the work that we re already doing. It was very much supported. (NH/86) I believe they [the hub] were very supportive and it was clear how respectful everybody was to each other in the hub. The respect when it was you re turn and not making you feel silly, any questions or queries that you raised, because sometimes it can be very isolating in a nursing home, trying to lead and drive forward quality improvement. But nobody was made to feel stupid about any queries they raised. It was treated that it was supportive for learning. (NH/1) Participants also conveyed that healthcare assistants also benefited from the ECHO program: The carers that attended did feel it was very, very useful, and it was all carers at that session. (NH/1) For some participants ECHO was perceived as being beneficial for healthcare assistants in the future as well: I do feel that it has opened our minds to they [healthcare assistants] need as much information as we require as nurses, because they re the ones that are doing the hands on care every day, they re the ones that are assessing the residents every day and reporting back to the nurses. So definitely I think the care assistants will benefit from this programme greatly. (NH/86) The majority of participants perceived that they benefited from the interaction with the other spokes that they got at ECHO: Overall perceptions have been very positive, that s my personal opinion. I feel I have benefited from it, plus I've really enjoyed networking with everybody. (NH/1) But I think the interaction between all the care homes was brilliant. (NH/50) 67

68 This interaction also helped to dispel participants concerns regarding ECHO: I suppose, going into it, I was apprehensive but that was totally cleared very quickly, and it was lovely, the whole camaraderie between everybody, because we re all in the one boat, we re all having the same problems and it s great to share knowledge and information with each other. So I really thought it was very good. (NH/86) I suppose with us being in a nursing home we re quite isolated so the beauty of this is that it is reachable to everybody and we can all share information and we are part of the group, we ve been talking about this outside of the session as well, so it s been a great resource. (NH/86) The resources were also viewed as being extremely beneficial to all participants: I think the resources that have been supplied have been fantastic. The information, there's a hell of a lot of printing to be done, even yet! So it is definitely valuable material and I thought we were going to get some really good material out of it. (NH/1) Plus, a lot of the stuff on the website, I could print that off and if staff weren t able to attend these sessions we were able to disseminate all the information. So they got a lot of information out of it. (NH/86) Overarching Theme 2: Consideration for the Future of ECHO Findings were indicative of the need for further considerations when providing future ECHOs (figure 10). This was conveyed though the challenges participant s perceived regarding getting the appropriate time to be able to dedicate to ECHO and also the timing of the sessions themselves. Accounts also illuminated the challenges of getting staff enthusiastic regarding ECHO. Interpretative theme 2B depicted perceptions of how the future of ECHO may be considered. Interpretative Theme 2A: Challenges Accounts conveyed the challenges that participants perceived in relation to ECHO. Findings suggested that getting time to participate in ECHO was an issue for participants: I think at the start I thought a lot more staff would be involved and I was very keen for that. It was very difficult with staffing issues and that to get staff off the floor, so that would be the only thing that I personally was disappointed in. (NH/86) 68

69 Many perceived that having the ECHO session on a weekly basis was challenging: I would like things spaced out a little bit more and not on a weekly basis. So maybe fortnightly. (NH/50) I think weekly for a period was a bit of a challenge because time is always precious to everybody. (NH/1) Some also felt that there was a lot covered in one session which made it difficult to engage fully in the ECHO and cover all the information: I felt sometimes maybe the second presentation wasn't starting until maybe 3:50, 3:45 and so we d maybe spent a lot of time on one that didn t have an awful lot of issues and then maybe the second case presentation had a lot more that we could have discussed but because the presentation was done first. (NH/86) I think we probably tried to squeeze too much into a session and I do think that having the quality framework going on as well was probably overcrowding the session and sometimes that meant we didn t get all of the case studies done etc. (NH/50) Some felt challenge by staff s lack of enthusiasm to take part in ECHO: We didn t have the enthusiasm as much from the staff, the three of us seemed to be the Three Musketeers wanting to do everything but we didn t get as much enthusiasm from the staff. (NH/1) Well, as I said, I thought that the staff would have been more participating in it. I was the lone ranger over here [laughter] I really, really hoped that the staff would have got involved because I thought they would have got a lot out of it, they would have really enjoyed it, as I did. (NH/86) Interpretative Theme 2B: Future ECHOs Findings highlighted that for future ECHOs participants perceived that separate sessions for staff without a medical or nursing background may be required: It s better to have a separate one [ECHO], a simpler session, because they [non-registered nursing staff] did enjoy it and the amount they learned from it but there are areas which they couldn t quite understand. So probably better split the sessions. (NH/10) 69

70 I think some of the sessions were a little bit kind of medical based, I do think sometimes you get more benefit by segregating groups of staff but I also think there is a place for mixing staff as well. So I would go for both. (NH/50) I feel that a lot of the information was very nurse led and I had my activity therapist in on some of the sessions and it just looked like it was completely over her head. (NH/86) Many also felt that future ECHOs would be of benefit between GPs and Nursing Homes: ECHO could really be used between nursing homes and GPs as well I think that would be of great benefit to a nursing home. (NH/1) Discussion The response rates for the post ECHO evaluation were very low despite repeated and persistent efforts to encourage participation. There was an improvement in knowledge and self-efficacy, but no statistical analysis was possible due to the low numbers. General feedback on the ECHO experience was positive, and from the low number of responses, 100% stated they had learnt through participation and found it enjoyable, 100% felt it helped translate knowledge into practice more than other teaching sessions they had been involved in and improved the care they provided for patients, and 100% would recommend it to others and participate again. The focus groups confirmed this- the ECHO platform was viewed as a positive learning experience that promoted the awareness of palliative and end of life care amongst nursing, healthcare assistants and other staff within the nursing home setting. Of particular benefit was the role ECHO had in nurturing the relationships between GPs and nursing home staff and the confidence and skills it gave staff in the delivery of palliative care. Nursing homes also conveyed that the interaction between spokes, that the ECHO platform lends itself to, also enhanced the ECHO experience. For future ECHOs with this group of participants it may be useful to consider the timing of the sessions to allow more staff to join and get the most out of each session. Funding for protected time may have helped this issue, as staff reported that with the busyness of the job it was hard to get time to attend due to patient care needs. A weakness is the low response rate that is likely to have led to a biased sample of more enthusiastic HCPs participating. This was reinforced in the focus group where participants commented on the lack of enthusiasm of some other staff. The ability to generalise these 70

71 results is therefore limited. More research is necessary to determine the benefits of ECHO in this setting, and whether it improves staff knowledge, self-efficacy and ultimately patient care. Suggestions from evaluation - Due to the very low response rate further research is needed in this area to determine how useful ECHO is in nursing homes for healthcare professionals. While the responses from participants were very positive, the response rate of approximately 10% makes the results ungeneralisable as it was likely a biased sample. - Consideration should be given to incentives for participating in the evaluation for staff e.g. funding dependent on this or funding for protected time to participate. - Funding needs to be considered for protected time for staff to participate. - The timing and frequency of any future sessions would need to be considered closely with the involved spokes. - Research looking at the direct impact on patient care and analysis of evidence of using the quality improvement learning would be likely to provide very valuable information as to the cost effectiveness and direct outcomes of ECHO investment. 71

72 Chapter 6 - Enhanced Dermatology for GP Trainees ECHO Background Skin disease is common and distressing. Around 24% of the population consult their GP with a skin problem in any 12-month period.(10) It is estimated that approximately 14% of all GP consultations are in relation to disorders or concerns about the skin.(9) Establishing and maintaining competence in this area of medicine is therefore essential for any GP. There is variable (and often limited) training in dermatology at undergraduate level leading to confidence and competence gaps at the post-graduate stage.(11-13) Most skin disease can be appropriately and efficiently managed in primary care. Of the nearly 13 million people presenting to GPs with a skin problem each year in England and Wales, around 6.1% (0.8 million) are referred for specialist advice.(14) While there are well over 1000 dermatological diseases, 10 of them (eczema, psoriasis, acne, urticaria, rosacea, infections/infestations, leg ulcers and stasis eczema, lichen planus and drug rashes) account for 80% of consultations for skin disease in General Practice and speciallycollected data from four specialist Dermatology departments in England show that specialists most commonly see people with skin lesions (35-45%), eczema, psoriasis and acne. Within one health trust area in Northern Ireland demand for secondary care dermatology services has been seen to grow year on year with a growth of 4.5% for 13/14 on the previous year.(15) This growth in demand contributes to greater waiting times, delays in effective clinical management and an increase in patient distress. Additionally, new initiatives around models of care are being developed. Prominent among these is Transforming Your Care (TYC) which aims to increase the proportion of care delivered within community settings.(16) The Kings Fund suggests a number of approaches to demand management among these being Schemes to manage GP referrals.(17) It is likely that progress in respect of confidence and competence among GPs and the adoption of best practice guidelines/ frameworks is likely to yield the greatest dividends. GP Trainees follow a well-defined curriculum(18) which is broad in its reach and completed within a short 3 year time period. Care of People with Skin Problems is one of the core competencies of this curriculum for the Membership exam of the Royal College of General Practitioners (MRCGP).(18) Some GP Trainees will get the opportunity to gain deeper experience in the management of dermatology problems through secondary-care clinical attachments but the majority will not. An opportunity to increase knowledge and competence 72

73 during training in respect of the management of dermatological conditions has the potential to have a career-long impact for future cohorts of GPs. Trainees at the GP ST2 level have 100+ consultations per week. These consultations are unselected and undifferentiated and will cover the full range of clinical areas for which anyone might consult their GP. On the basis of average consultation rates it is therefore expected that each GPST2 will see and manage 14+ patients with dermatological issues each week. The Northern Ireland Medical and Dental Training Agency (NIMDTA) is an organisation which operates on behalf of the Department of Health, Social Services and Public Safety (DHSSPS) to train medical and dental professionals. NIMDTA commissions, promotes and oversees postgraduate medical and dental education and training throughout Northern Ireland. NIMDTA organises and delivers the recruitment, selection and allocation of doctors and dentists to foundation, core and specialty training programmes and rigorously assesses the performance of trainees through annual review and appraisal. It works in close partnership with local education providers to ensure that the training and supervision of trainees support the delivery of high quality, safe, patient care. NIMDTA now recruit approximately 85 trainees each year to the GP Training programme. These GP specialist trainees (GPST) have a 3 year run-through training programme (GPST1, GPST2 and GPST3) of which 18 months takes place within hospital attachments and 18 months in General Practice (6 months in GPST2 and 12 months during GPST3). The Intervention Twenty-eight GP Trainees (GPST2 level) from across the region took part in this ECHO over 5 sessions to learn about the common dermatological conditions of eczema, psoriasis and acne. For more details see table 14. The Programme Lead (a GP Training Programme Director with NIMDTA who is also a GP with enhanced dermatology training developed a Programme Resource Pack comprising a range of written material, pictures and videos that were used in the sessions. (These resources were based on her work previously done with the Northern Area Primary Care Dermatology Pathway and were reviewed by local consultants). The ECHO sessions were scheduled to take place on a 2-weekly basis and ran for 1.5 hours around lunchtime to facilitate maximum participation and minimum disruption to the working day. The meetings were conducted via video-conference technology that allowed GP Trainees to join from any location that provided a webcam and internet connection thus avoiding the need to journey to a central location. 73

74 Participants were asked to prospectively identify Cases from their GP surgeries and to use the agreed Case Proforma setting out the anonymised details of a non identifiable patient, the challenges faced, the treatment modalities used and the outcomes achieved. The hub team provided feedback and guidance on management options and advice on refinements to the management options available. It was expected that in each meeting up to 3 cases would be discussed. Most of the GP Trainees joined the ECHO Clinics via videoconference principally from one of two locations 1. One of the seven community offices of the NIH for those GPST2s in close proximity or 2. From the nearest GP Practice which had a webcam facility Some of the GP Trainees joined using their own tablet or laptop devices from home as access to the GP Practice Wi-Fi network was forbidden by the local Health and Social Care security policies. A map of the locations of trainees, Northern Ireland Hospice community offices and locations of the GP Webcams can be seen in Picture 2. Picture 2 - Locations of trainees, NIH community offices and locations of the GP Webcams Hub - The hub team was chaired by the Programme Lead (GP with enhanced Dermatology Skills) and a variety of invited guest team members (e.g. Consultant or Specialist Nurse Dermatologist and a Pharmacist with enhanced knowledge of prescribing for dermatological conditions). Spokes - The spokes were identified as the half of the GPST2 cohort who are placed in General Practice during the first 6 months of their ST2 year. There were 28 GP trainees involved in total. Some based at NIH community sites (16) and some at training practices (12). 74