After Francis Policy Commentary Over the last two decades, the collection and use of patient experience information has become commonplace in England s NHS and many other international health systems. Early efforts to collect systematic data on people s experiences have tended to use traditional postal survey methodologies: these offer high coverage at a low cost, but with the disadvantage that the process of data collection and reporting is time consuming. This often means a delay of several months between people s experiences and reporting of these, which has been cited as a barrier to the effective use of feedback for improvement (Reeves & Seccombe, 2008). Another reported barrier is that results are often presented at whole organisation level, rather than being specifically attributed to smaller units like hospital wards (op cit). Overall, there has been disappointingly little change in the quality of people s reported experiences of hospital services (Raleigh et al., 2015), and there is a need to advance the use of patient feedback for service improvement (Coulter, Locock, Ziebland, & Calabrese, 2014). Collecting and reporting feedback in near to real-time and at a localised level has the potential to address these barriers. There have been numerous attempts to introduce near real-time feedback into the NHS, starting with the launch of commercial data capture products in 2006. In 2008, the then Secretary of State for Health, Alan Johnson, announced plans for all hospitals to collect immediate feedback from patients within twelve months (Snow & Kendall-Raynor, 2008), and by 2010 the NHS was collecting more than 300,000 response per month in near real-time (Robert & Cornwell, 2011, p. 18). The NHS Friends and Family Test, announced in 2012 and launched in 2013, took real-time feedback to a new volume, eventually requiring all patients to be offered the chance to answer a single question after a wide range of types of care episode (Cabinet Office & Department of Health, 2013; Department of Health, 2012). The perceived benefit of the real-time approach was further reinforced by a recommendation in the Francis Inquiry into Mid-Staffordshire NHS Trust that patient feedback should be recorded and reported in as near to real-time as possible (Francis, 2013). Despite the widespread use of and high hopes for near real-time feedback, there has been little evaluation of the effectiveness of this kind of evidence for improving service quality. There is a particular lack of information on the use of feedback to measure and improve relational, as opposed to transactional elements of care, and it is these relational aspects such as compassion that are those most central to the findings of the Francis Inquiry (Robert & Cornwell, 2011, 2013). This study, completed by the Picker Institute and University of Oxford, sought to address this with the development and evaluation of a new tool for measuring and improving people s experiences of relational aspects of care. Through this, we have developed and validated a new toolkit and instrument that can be freely used by NHS organisations. About the study Development and testing of the real-time approach and instrument was completed over a thirty month period with funding from the National Institute for Health Research, NIHR. We sought to establish whether it was possible first to measure and secondly to improve people s experiences of P2878 03.01..2017 CG SK FINAL 1
relational aspects of care whilst in hospital, and to investigate the factors that influenced the success of this approach across a number of sites. Relational care in hospital was selected as a focal area to address the findings of the Francis Inquiry into Mid Staffordshire Hospital NHS Foundation Trust and contemporary policy concerns about levels of compassion in acute care. We chose to focus specifically on relational care in emergency departments, as these are high volume, high demand services, and in the care of older inpatients, who are disproportionately represented in hospitals and who may particularly benefit from good relational care. We collected data using electronic devices (tablet computers and touchscreen kiosks) so as to minimise the length of time between data being collected and available, and to avoid the marginal costs associated with printing and data entry. To make the administration of the questionnaires practicable, we recruited volunteers at each of six participating hospital trusts and train them to gather survey data from current patients. Methods Initial stages of the study included mixed methods research to develop a questionnaire covering relational aspects of care as experienced by patients in hospital. This involved: A literature review to identify previous studies and instruments looking at relational aspects of care and interpersonal relationships in hospital. Focus groups and interviews with recent emergency department attendees and older people recently discharged from hospital after an inpatient episode. These sought to explore what patients understood by relational or emotional care and to prioritise components of this for measurement in a relatively concise questionnaire. Secondary quantitative analysis, including factor analysis, of data from previous national surveys of inpatient and emergency department care. This sought to identify existing questions that measure relational aspects of care in the national surveys. Based on this research, we developed a longlist of 62 questions to potentially use. These were condensed via review with our expert advisory group into a shorter list of twenty items, which were assessed via cognitive interviewing, a questionnaire testing process where researchers observe patients completing the questionnaire and seek (through questions and inviting respondents to think aloud ) to determine whether questions used can be consistently understood and responded to. Cognitive testing was undertaken on two wards and in emergency departments at three NHS trusts, and showed that the items selected were effective in measuring what they were intended to. Once the questionnaire was completed, we provided training for volunteers at each of the six participating hospital trusts, providing support to allow volunteers to feel confident in asking patients for feedback and using the tablets. Volunteers and staff members then collected feedback from patients over a period of ten months, with detailed results reported weekly throughout this time. Two participatory workshops were completed at each trust first midway through the data collection period, to review results and plan improvements, and second after the close of data collection, to investigate how the data had been used and what (if any) improvements had been 2
made. To support the evaluation of the instrument and near real-time feedback approach, we also collected feedback from hospital staff via surveys and through one-to-one interviews. Findings We found that a short (20 item) questionnaire presented via a tablet computer can feasibly be used to gather bedside feedback from older patients on hospital wards. Although we observed a definite preference for further shortening of the questionnaire which has led to the development of a condensed short-form version as part of the toolkit most patients were comfortable with completing the full survey. The use of kiosks in emergency departments was, by contrast, markedly less successful; it proved difficult to encourage patients to complete these at appropriate times in their stay, and this greatly limited the volume and value of data collected. Overall, the use of the real-time feedback was associated with a small but statistically significant improvement in people s reported experiences of relational aspects of care. The large majority of differences for individual wards were positive over the study period, but few differences were statistically significant which reflects the small sample sizes available. Collecting large volumes of feedback from individual wards proved difficult, both because of the numbers of patients available and the amount of volunteer resource that could be used. This did not prevent the feedback collected from being valuable, but it demonstrates the need to be realistic in setting expectations about how much data can be collected and how this can be used statistically: in short, some pragmatism is needed. The use of volunteers to support data collection proved effective, producing unexpected benefits albeit with some significant caveats. Recruitment and retention of appropriately skilled volunteers was a challenge; obtaining the relevant clearances for volunteers was time consuming, and even with training and practice not all volunteers felt comfortable approaching patients to complete the survey. Consequently, we recommend that a person specification be used to support recruitment of appropriately skilled volunteers. Amongst the volunteers who were comfortable with approaching and talking to patients, however, there were major benefits. We found that collecting feedback became only part of the role fulfilled by the volunteers; their presence, and they time they spent discussing relational aspects of care with patients, also became an intervention in its own right. Hospital staff frequently reported this as a major benefit of the study. Volunteers also reported benefits to themselves that resulted from their participation, including increased confidence in speaking with others and the establishment of informal mentoring relationships. Staff valued the regular, localised feedback provided by the real-time measure, including both written comments and numeric data. The reporting of the data, via a dynamic, interactive Excel document, was not always as well received. Amongst busy front-line staff there was a preference for a static single page summary results which could be quickly reviewed and easily digested. We also found that there were occasions where results did not make it to front line staff because they were not shared within trusts, and this limited their use for improvement. 3
Conclusions and recommendations The study demonstrated that it is possible to measure and to improve people s experiences of relational care in hospital via the use of real-time feedback and it highlighted barriers and enablers associated with effective improvement. As part of the study, we have developed a toolkit consisting of the survey instruments, guidance, and case studies. These documents, including the validated questionnaire developed by the Picker Institute and the University of Oxford, are available for free use by NHS organisations and will be made available at www.pickereurope.org in February 2017. The aim of the toolkit is to fill a gap in the existing resources available for organisations to measure and improve people s experiences of care. Simultaneously, the toolkit can help organisations showcase how they are addressing current policy directives and demonstrate their organisation s commitment to improving compassionate care. We recognise that NHS organisations will already have access to data from national surveys, the friends and family test, and other collections. These each have their benefits and their limitations. National surveys are systematic and robust, but taken time for results to be ready and have limited granularity. The friends and family test is simple and immediate, but there are significant limitations to the use of the quantitative data collected. And few or no existing collections provide in-depth feedback about people s experiences of relational aspects of care, such as compassion and emotional support areas that have proven stubbornly resistant to change in the past and that remain a policy priority. It is these issues that the toolkit seeks to address, and our aim is that it should be useful as part of an organisation s overall suite of feedback methods not necessarily as a replacement or silver bullet for measuring and improving patient experience. We want users to be able to get the most out of these tools to capture useful information and drive real improvement in line with current health policy directives, and with that in mind we recommend: Using volunteers is an effective way of gathering feedback and improving people s experiences of care on wards but the right volunteers need to be used, and they need plenty of support. Person specifications should be used to recruit appropriately skilled volunteers and they must have regular communication with a responsible member of staff at the trust. Support and engagement is needed from the whole ward team, including nursing staff, consultants, and others. Without this, it can prove difficult for volunteers to access patients, and there may be limited appetite to use the results for improvement. Collecting feedback in near real-time can be cost effective, but this doesn t mean there are no costs. To make use of this approach, there is a need for investment in tablet computers and software, and time must be devoted to setting up and testing the questionnaire. Once established and in use the ongoing and marginal costs are very limited, though, so this kind of approach is best suited where collections are intended to continue for a reasonably long period of time to offset the initial costs. Results should be reported back to front-line staff as often as possible in a direct, accessible format that highlights areas for potential improvement. Teams should consider the use of this information for action planning by finding opportunities to discuss and prioritise findings. At the 4
same time, measurement of patient experience should continue with the same questionnaire to assess the impacts of changes made. If you d like more information about the study or about the toolkit, please contact Susanne Käsbauer, PhD Picker Buxton Court 3 West Way Oxford OX2 0JB Tel. +44 (0) 1865 208105 Email: Susanne.Kaesbauer@PickerEurope.ac.uk Funding This project was funded by the National Institute for Health Research Health Services and Delivery Research Programme (project number 13/07/39). The views and opinions therein are those of the authors and do not necessarily reflect those of the HS&DR Programme, NIHR, NHS or the Department of Health. This article presents independent research funded by the National Institute for Health Research (NIHR). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. 5