Deep-Dive Review: Evaluating the CCG s use of Patient Experience Information. Omar Al-Ramadhani Planning and Assurance Manager ENC B

Transcription

1 Deep-Dive Review: Evaluating the CCG s use of Patient Experience Information Omar Al-Ramadhani Planning and Assurance Manager 1

2 1. Aim Introduction Sources of Patient Experience Information and Themes Section Summary: Current Information Overview of Data Nationally-Generated Patient Experience Data Locally-Generated Patient Experience Data How the CCG Currently uses Patient Experience Information and how we can Improve our Approach Section Summary The Use of Patient Experience Information Strategic Planning Procurement Monitoring and Evaluation Recommendations...24 Appendix A - Friends and Family Test...25 Appendix B - Sources of Patient Experience Information...28 Appendix C HRWD Tracker

3 1. Aim This report aims to review how NHS Southwark CCG uses patient experience information to inform service design and improve the quality and safety of NHS services used by Southwark patients. The review looks at patient experience information obtained from national and local sources and additionally considers the gaps in the CCG s approach. In summary, the review focuses on answering the following questions: Is the information available to the CCG fit for purpose and are there any gaps? How does the CCG collate and analyse patient experience data and to what extent are we effective in using this information to improve services? What are the most frequently identified patient experience issues that require improvement? 2. Introduction Commissioning services that offer a good standard of patient experience has consistently been a key priority for the CCG. Improving patient experience has been a key objective in the CCG s operating, strategic plans and recently published CCG Quality Framework. The CCG also recognises that good patient experience is a fundamental component of good quality care. Understanding patients views of commissioned services is an essential feature of monitoring the services we commission. The CCG s commitment to improving patient experience is also reflected in national policy. In the white paper, Equity and Excellence: Liberating the NHS, published in July , the government made clear its commitment to improving patient experience and creating an NHS that is genuinely centred on patients and carers. Patient experience has gained increased recognition as a core component of good quality care 2 with the NHS Outcomes Framework making clear that the provision of a good experience of care for patients is a central goal for both NHS commissioners and providers. Patient experience has also been linked to improved patient outcomes. A systematic review of 55 studies in primary care and hospitals found consistent positive associations between patient experience, patient safety and clinical effectiveness for a wide range of disease areas. This supports the case for including patient experience as one of the central measurements of quality in healthcare and underlines its importance 3. In response to heightened national focus on patient experience following the reports into Mid- Staffordshire; Winterbourne View and Keogh s review into hospital mortality, the NHS has increased the production of patient experience information in order to ensure both providers and commissioners monitor and improve performance. NHS organisations now generate and have access to a significant volume of data. However, evidence tells us that data is not yet systematically 1 Department of Health Equity and Excellence: Liberating the NHS. 2 Department of Health High Quality Care for All - NHS Next Stage Review Final Report. 3 The Health Foundation Measuring Patient Experience. 3

4 used by commissioners of NHS services. A recent paper published in the British Medical Journal 4 highlights that, whilst multiple sources of patient experience information is available to healthcare organisations, not enough is being done to use that information to monitor and improve services. The following sections of the review sections 3 and 4 look in further detail at some of the information that is regularly available to the CCG. These sections look at some of the key themes in these data and also review how the CCG currently analyses and utilises this information over the course of the commissioning cycle. Section 5 concludes the review with a number of practical recommendations for the CCG to take forward. Note: This review reflects on and incorporates recommendations made in the CCG Quality Framework. 4 A. Coulter, L. Locock, S. Ziebland and J. Calabrese Collecting data on patient experience is not enough: they must be used to improve care. The British Medical Journal. 4

5 3. Sources of Patient Experience Information and Themes This section looks at the patient experience information currently available to the CCG and highlights areas where the CCG could make further use of these data sources. Useful data on patient experience should have the following characteristics: a) Provides adequate coverage over its commissioned activity b) Is broad enough to highlight areas that may require improvement and c) Granular enough to make it possible to drill down to understand issues in more depth. 3.1 Section Summary: Current Information a) The CCG regularly focuses on national data sources to provide an overview of patients experience of acute, mental health, primary care and social care services. b) The CCG needs to develop an effective way of collecting informal/soft data to identify trends and recurring issues (refer to recommendations 1 and 2 in Section 5). c) The CCG should make more use of patient focus groups in certain service areas. Focus groups can uncover unanticipated issues that surveys fail to identify and they can be a good way of engaging people who do not respond to surveys; and the most disadvantaged patients (The Health Foundation, 2013). The CCG should access information being shared at hospital patient groups. This can be done by attending meetings or asking for minutes/notes of meetings (refer to recommendations 1 and 5b in Section 5). d) For both of the above the CCG should consider an approach to conducting qualitative surveys of patients or the completion of focus groups. This approach could consider the role of local voluntary sector groups and/or Healthwatch Southwark as they have links to patients and the community (refer to recommendation1 and 5 in Section 5). e) There are some gaps where information is not routinely collected/analysed, these include some out of hospital pathways, small providers and out of hours doctors services (refer to recommendation 4 in Section 5). f) The CCG does not have access to much data that measures patient experience at the interface between services except for data on discharge from acute care. The data we have is mainly restricted to activity at specific providers rather than across pathways. An effective way of gathering this information could be through patient interviews and focus groups. Continuing care could be a good area to start to gather this information as the team have good links with patients and carers (refer to recommendation 5 in Section 5). 3.2 Overview of Data This Section provides an overview of the types of patient experience information available to the CCG and highlights a number of themes. 5

6 England has been a pioneer in measuring patient experience information and was the first country to introduce a mandatory survey of hospital inpatients in National survey reporting has expanded considerably to now include many aspects of care (box 1) 5. There is therefore a significant volume of intelligence and information available to CCGs. Indeed one challenge faced by commissioners has been to establish systems for the routine collations and interpretation of patient experience data. Box 1: Regular national health and social care surveys carried out in England Care Quality Commission (CQC) national patient experience surveys General Practice Patient Survey National Cancer Patient Experience Survey VOICES survey of bereaved people Patient reported outcomes of selected surgical procedures (PROMs) Adult Social Care Survey Adult Social Care Carers Survey CQC social care user surveys The CCG commissions a wide array of services from multiple providers and so it must be able to draw upon a varied approach to capturing patient experience information. All data capture methods have pros and cons, so a combination of styles may be needed to understand issues at a level which can be used to inform and drive improvements. 5 A. Coulter, L. Locock, S. Ziebland and J. Calabrese Collecting data on patient experience is not enough: they must be used to improve care. The British Medical Journal. 6

7 The diagram below presents a spectrum of patient experience information. The means of collecting data are grouped into how specific or general the information is and how descriptive it is (see figure below). Source: Measuring Patient Experience, The Health Foundation, Nationally-Generated Patient Experience Data There are a number of nationally directed patient experience surveys that generate data to be used by NHS providers, commissioners and regulators of services. These data and the results at local providers are summarised below and reflect the most recently available data: Maternity Survey 2013 King s College Hospital NHS Foundation Trust (KCH) achieved scores that were comparable with the national average on every question except for being involved enough in decisions about their care during labour and birth which the Trust scored below the national average. Guy s & St. Thomas NHS Foundation Trust (GSTT) achieved scores that were comparable with the national average on every question except for two questions relating to staff behaviour. The trust performs below the national average for leaving patients alone at a time that worried them and responding quickly to the call button. Accident and Emergency Survey 2012 KCH achieved scores that were comparable with the national average on every question. GSTT achieved scores that were comparable with the national average on every question. The Trust performed better than the national average regarding doctors and nurses listening to what patients had to say, however the Trust performed less favourably against the national average for feeling threatened by other patients or visitors. 7

8 Cancer Survey 2012/13 KCH under-performed in most areas of the survey, achieving scores in the lowest 20%. The Trust performed particularly poorly in explaining what was wrong to patients and taking into account patients views while discussing treatment. GSTT performed better than KCH but the trust does have a number of areas that require improvement. Inpatient Survey 2013 KCH achieved scores that were comparable with the national average on most of the questions, however, the Trust underperformed in explaining how patients would be put to sleep and how to control pain and the time spent on waiting lists. The Trust performed particularly poorly for giving patients enough notice about when they were going to be discharged. GSTT did not achieve any scores below the national average and performed well in a number of areas. Patient Experience Data Sources Used in the Keogh Review The review carried out by Sir Bruce Keogh into hospital mortality outliers 6 focussed on a range of domains that are indicative of the level of quality and safety of care; one of the domains was patient experience. The review focused on a number of publically available sources of information which when analysed together provide a valuable insight into the general performance of providers highlighting areas that could require further investigation. The review examined the following sources of information: National data sources CQC Inpatient survey NHS England Cancer survey PEAT: Privacy and dignity PEAT: Environment PEAT: Food Ombudsman s rating Friends and Family test Patient voice comments PEAT = Patient Environment Action Team Note: KCH and GSTT were not included in the review Other National Surveys The Care Quality Commission (CQC) co-ordinate surveys to collect feedback on the experiences of people using a range of health care services provided by the NHS. One of the surveys carried out is 6 NHS England, Review into the quality of care and treatment provided by 14 hospital trusts in England: overview report. 8

9 annual; the adult inpatient survey, however the rest are carried out every 2-3 years. A cancer annual survey is carried out by Quality Health on behalf of NHS England. Other national surveys include: Outpatient (2003, 2004/5, 2009 and 2011) A&E (2003, 2004/5, 2008 and 2012) Maternity (2007, 2010 and 2013) GP Patient Survey (collected every 6 months, published annually) Patient Environment Action Team The Patient Environment Action Team (PEAT) programme has been superseded by the Patient-Led Assessments of the Care Environment (PLACE) programme. These self-assessments are undertaken by teams of NHS and private/independent health care providers and include at least 50 per cent members of the public (known as patient assessors). They focus on cleanliness, food, hydration, and the extent to which the provision of care with privacy and dignity is supported in clinical and non-clinical services. Although the assessments are comprehensive, scoring does not differentiate between levels of performance well enough. The vast majority of hospitals achieve ratings of good/excellent (8 or 9 out 10), which makes it difficult to assess performance and track improvement. As with most national surveys PLACE surveys are carried out annually. Friends and Family Test The Friends and Family Test (FFT) is a single question survey which asks patients whether they would recommend the NHS service they have received to friends and family who need similar treatment or care. The Friends and Family test is discussed in detail in appendix A. General Practice Patient Survey Although the CCG does not commission primary care, it has a statutory duty to assist and support NHS England in securing continuous improvement in the quality of primary care. A source of patient experience information is the national GP Patient Survey which assesses patients experiences of the access to and quality of care they receive from their local GPs, dentists and out-of-hours doctor services. Southwark s overall survey response rates and satisfaction with GP services and out-of-hours services were similar to the London average. Patient Voice Comments Patient Voice Comments measure counts of negative comments adjusted by counts of positive comments about services at a trust. For any source of patient feedback, there will be an expected relationship between the number of positive comments and the number of negative comments across all trusts. For example, the number of negatives would be expected to increase at a given rate as the number of positives increase. Using a statistical procedure called Poisson Regression, results from all trusts are taken to work out what that specific relationship is (i.e. the slope of the regression indicates how quickly negative comments increase relative to positive comments). Then, for each trust, this relationship is applied 9

10 to the number of positive comments to predict what the number of negatives would be. The departure of the observed number of negatives from the expected (predicted) number determines how much of an outlier a given trust is. If a given trust s observed count of negative comments is much higher than expected, it can potentially be flagged as a risk by the CQC (if the disparity is extreme and statistically significant). The table below shows that GSTT s observed negative comments is below expected but King s negative comments are slightly above expected. The CQC has not identified King s score as a risk. Expected Observed King s GSTT Source: CQC Intelligent Monitoring Report, 13 March 2014 The information available from the CCG s main providers is included in appendix B. Limitations of National Surveys 1. Frequency. The annual surveys can be an insightful source of information on a number of areas of care; however by the time responses are gathered and results published, issues may have already been addressed by the provider or new issues may have occurred which are not captured by the results. Running more regular and focused local surveys would help to monitor services more closely and highlight emerging trends. 2. Granularity. National survey questions are fairly general and do not drill down into specialty or ward level. This makes the information useful for identifying general issues and trends which could require further investigation. An example of this occurred after a national survey showed that London hospitals were performing poorly in maternity care which Southwark CCG followed up by developing a Commissioning for Quality and Innovation (CQUIN) payment to improve certain areas of maternity care at King s. 3.4 Locally-Generated Patient Experience Data Patient Experience Data Generated by Foundation Trust Providers GSTT produce an annual patient experience report which appraises the trust s performance in recent national surveys, performance in the trust s own near time feedback surveys, performance against patient experience CQUINs and a summary of patient advice and liaison service performance. The report also reviews findings from the trust s patient engagement activities. There is a good mix of quantitative and qualitative information. The report highlights areas of good performance and also areas that require improvement. The issues are discussed generally, however the report could form a good basis for drilling down into areas of concern with the trust and identifying specific wards or specialties that require improvement. King s produce monthly how are we doing (HRWD) surveys (refer to Appendix C for the HRWD tracker) which ask patients questions on patient experience (including those covered by CQUINs) and certain points of delivery. The report used to only present quantitative information; however since August 2013 the report has published all patient comments made at the Trust as well as 10

11 comments made about the trust at NHS Choices and Patient Opinions. The quantitative data is broken down by specialty which helps provide a good level of detail to the data. The trust also produces a quarterly patient experience report which provides an update on the trust s performance in national surveys, the HRWD survey, complaints, PALS and a review of the trust s rating on NHS Choices. GSTT has replaced its annual community services patient experience report with near time surveying. This provides more regular data and is currently reported quarterly for ten services. The trust is trying to increase the accessibility of the survey in a number of ways including; enhancing the domiciliary telephone process; making surveys more user friendly for children and those with learning disabilities and providing support to help capture the views of more vulnerable patients. GSTT Community services District Nursing Foot Health Health Visiting Intermediate Care Occupational Therapy Paediatrics Physiotherapy Reproductive and Sexual Health Speech and Language Therapy Specialist Regional Rehabilitation There is a large variation in the number of questionnaires submitted for each service with the most being around 1,000 for Foot Health and as little as nine for Specialist Regional Rehabilitation. The surveys do not provide any qualitative data, however they could be effective at identifying high level problem areas. So far the information has not resulted in service improvements because the data has not been able to illuminate any areas that require improvement. Options being considered to improve the surveys include; themed questions on specific areas and bespoke questions for each service. Commissioners are currently designing a survey for district nursing with bespoke questions that will attempt to cover the integration of services. SLaM s patient experience reporting is centred around the trust s Patient Experience Data Information Centre (PEDIC) reports which present quantitative data for inpatients, outpatients and each clinical academic group (CAG). The questions in the report have been agreed by commissioners with the trust. The last report (Q3 13/14) is detailed enough to monitor performance over time and initiate discussions with the trust on areas that require improvement. A patient experience survey was carried out by the Southwark IAPT service between January 2013 and December 2013 receiving over 1,000 responses. The survey asked five questions and allowed patients to make comments and so provides both quantitative and qualitative information. On the whole, patients rated the service highly. 11

12 Local Provider Themes The table below presents areas of patient experience which have been highlighted as under performing from the trust reports mentioned above. GSTT Acute Inpatients Outpatients King s All areas SLaM Receipt and explanation of the ward pack Quality of the food Medication side effects Noise at night Cleanliness of toilets and bathrooms Choice of appointment times Being kept informed of appointment wait in clinic Addressing worries and fears Knowing who to contact Delays in appointment starting time Being told how long you would wait in outpatients Complaints response times Noise at night from staff Explanation of possible medication side effects Inpatients Offered a meeting with the ward pharmacist Restraint and injected medications used as a last resort with minimum use of force? GSTT Community Services NA Receiving a copy of care/recovery plan and jointly producing the plan Receiving a copy of care/recovery plan and jointly producing the plan Knowing how to make a complaint Offered a crisis plan for emergency mental health situations Being given a full assessment after accessing crisis support at SLaM 12

13 CCG Engagement NHS Southwark CCG regularly captures patient experience information from the Engagement and Patient Experience Committee (EPEC), a sub-committee of the Governing Body and local Patient Participation Groups (PPGs). The CCG also captures information from large ad hoc patient engagement events and through a number of service user groups (Discussed further in Section 4). EPEC and PPG Feedback Concerns raised at EPEC and locality PPGs are recorded in meeting minutes and are also communicated to lead officers and the CCG Governing Body through the CCG s quarterly quality report. Issues are also raised directly by the Head of Membership and Engagement with commissioners, however this is currently done informally. Issues that have been captured include: Waiting times for inpatient treatment Outpatient clinic waiting times Cancelled outpatient and inpatient appointments Discharge delays from hospital Referrals being lost Quality of follow up and after care particularly after cancer treatment. Timeliness of discharge information Communication between secondary and primary care. The CCG can potentially collect large amounts of informative soft data through engagement events and patient forums, however soft data is not always formally published and so the CCG s access and use of it can be limited. Feedback collected from engagement events or patient forums are not always written up and analysed, which risks information being lost. Out of Hospital Pathways Most out of hospital pathway services are contracted to carry out annual patient experience surveys. The dermatology service returns an annual survey and King s and GSTT are contracted to survey patients in the Musculoskeletal Clinical Assessment and Treatment Service (MCATS). Some out of hospital pathway services (for example the diabetes service) return monthly patient surveys, however commissioners have highlighted that they don t have the capacity to review the data on a regular basis but will analyse the data during a service review. Feedback indicates that beneficial data has come from conversations with patients during visits (for example the diabetes quality visits) and by engaging patient groups and forums. The Southwark and Lambeth diabetes forum has been a good source of information as it represents a diverse and fairly representative group of patients. Interviewing patients and running focus groups can be 13

14 resource intensive so it has been suggested that collaborating with voluntary services who already have links with patients may be an effective way of gathering a rich perspective on patient experience. Patient experience information across pathways The CCG lacks access to patient experience information that tracks patient experience across pathways and the transfer between health and care services. This gap in information has been recognised nationally and prompted five leading healthcare organisations to initiate an 18 month project aimed at capturing older people s experiences of care coordination. The project will develop a survey tool to measure how older people and service users experience care when it is delivered by multiple organisations. It is hoped that the final tool set will be available for use in the summer of Sources of Information the CCG does not routinely utilise There are a number of sources of information that the CCG does not routinely use, these are listed below: Hospital patient groups There are many patient groups at King s that meet to share feedback on services; these include King s In Conversation King s In Your Shoes King s Maternity Services Liaison Committee King s Cancer Voices Online chat forums and social media These can be a useful source of patient comments about services in Southwark. The CCG has used the SE1 online forum to ask Southwark people for their experiences of using the urgent care centre in June The discussion generated 3 pages of comments and allowed the CCG to engage with members of the forum to ask follow up questions. The CCG has also used the Myhealth London chat forum facility as part of the Call to Action campaign but the topic did not generate any discussion. The CCG has been using Twitter to publicise its engagement events and activities. Tweets have mainly focused on drawing peoples attention to the CCG s website and not for direct engagement. The Twitter account has 3,766 followers. 7 The Nuffield Trust: New project launched to explore how well health and social care services are coordinated. Accessed 14 August

15 4. How the CCG Currently uses Patient Experience Information and how we can Improve our Approach This section will review how patient experience information has been used by the CCG to commission services and how the use of information could be improved. The CCG s work can be summarised by the commissioning cycle below: Involving patients in the commissioning cycle and ensuring services are commissioned to improve patient experience can lead to more efficient and cost effective pathways and ones that lead to better outcomes. 15

16 4.1 Section Summary The Use of Patient Experience Information a) The CCG has been effective at conducting large scale engagement events like those that inform the CCG s commissioning intentions and the recent Call to Action event. However, we are aware that some feedback indicates that engagement events are not held early enough in the planning process to ensure that decisions are genuinely informed by patient views. There is a feeling that some engagement events and forums have been used to endorse decisions that have already been made. b) When carried out in good time, engagement events have been an effective way of gathering information to inform strategic decisions. It is essential that Commissioners work with the Communications and Engagement team to feedback to patients and participants in our engagement to demonstrate where their involvement has helped to shape our plans. c) The CCG should undertake a different approach to focussed engagement. Our experience is that engagement in small forums and discussion groups work best when it includes expert patients or current and ex-users. Wherever possible the CCG should focus engagement on issues that patients are able to offer valuable insights and opinions (refer to recommendation 1 in Section 5). d) The CCG has been effective in driving service improvements through the use of CQUINs. However without the use of CQUINs (and quality accounts at King s) it is difficult to engage providers to measure additional areas of patient experience and implement improvement plans. Clinical Quality Review Groups (CQRGs) are used as a way of exerting influence to encourage providers to improve on certain areas of patient experience (refer to recommendation 3 in Section 5). e) There are good examples of how triangulation of national and local data (King s maternity care) and the triangulation of patient experience information and complaints information (mental health) has been effective in driving improvements. This approach should be enhanced as part of the CCG s assurance of commissioned providers (refer to recommendation 2 in Section 5). 16

17 4.2 Strategic Planning Examples of Good Practice The CCG has been effective at carrying out large engagement events to gather patient experience information. The recent Call to Action event for example was attended by around 70 people and produced a wealth of information that was captured in a well written report. Recently, the CCG used a Survey Monkey to gather information on the experience of users of the talking therapy services to inform the development of the new service. The survey returned over 60 responses and asked a number of questions across a range of themes. The survey also provided free text boxes to allow for more detailed feedback. Widely accessible electronic surveys like this can provide valuable benefits particularly if they are coupled with survey methods aimed at hard to reach and the most vulnerable patient groups. Services like Survey Monkey allow for: 1) Wide coverage and easy access. 2) Eliminates manual input of survey responses. 3) Real time data charts and analysis. The findings from the survey confirmed that patients wanted access to the service after 6 pm, they wanted the choice of a range of languages and wanted a more even number of male and female counsellors. There are a number of additional examples of when patients have made a valuable contribution to more focussed pieces of work, like in the review of muskuloskeletal physiotherapy services. Engagement here involved a workshop of current users to inform potential changes to the service (box 2). Box 2: Provision of physiotherapy and osteopathy services. In early 2013 the provision of physiotherapy and osteopathy in Southwark was reviewed. The review found that as well as the provision of physiotherapy at hospitals, several GP surgeries also provided physiotherapy or osteopathy services. A consultation engagement event was held with patients on Wednesday 25 September. 14 patients attended, most of who had used a non-core funded Osteopathy Service at their local practice and/or Physiotherapy Services at various sites. Discussions at the event focused on osteopathy in particular as a number of patients were concerned about changes to their current practice based osteopathy services. Although the CCG had decided that a change to the service was necessary to provide a more equitable and evidence based service; patients were engaged on a number of options on how the future service may look. 17

18 The Continuing Care team has worked to support patient representatives during the development of adult personal health budgets (Box 3). Box 3: Adult Personal Health Budgets The continuing care team tried to overcome these issues when inviting patients to sit on a steering group to oversee implementation of adult personal health budgets (PHBs). They chose one patient who was representative of wider groups and one patient who had lived experience of holding a PHB. To ensure the lay members made useful contributions they were kept well informed of relevant issues outside of steering group meetings and additional effort was made to ensure communication was always in a clear format. Areas for Improvement The CCG has generally been less effective at smaller scale engagement of patients at the planning stage of the commissioning cycle. CCG leads have highlighted that on occasion small scale engagement events are not organised at the very beginning of the planning cycle and are carried out to justify strategic decisions that have already been made. It can be difficult for lay people to make valuable contributions at this stage because: They are sometimes not well informed about the case for change. Strategic topics are not presented in a way that are easily understood. The scope of some service changes are sometimes limited by centrally mandated specifications and the need to achieve national targets which limits the scope of engagement. A number of CCG leads have suggested that asking patients to focus on one or two areas that really benefit from a patient perspective, like the production of information leaflets, may be more beneficial than engaging people on high-level technical issues. 18

19 4.3 Procurement Examples of Good Practice There are examples of where the CCG has used patient experience information and engagement to inform procurement decisions. The Talking Therapies procurement will involve two service users on its steering group and a representative from Healthwatch Southwark. The process will require a significant time commitment from those involved as there are monthly meetings (potentially moving to fortnightly) and the interview process has been planned to take place over a week. The service users are being supported to take part by receiving technical briefings and receiving compensation. For mental health services, patients involvement in procuring services has been limited. There have been a couple of examples of children being involved at this stage. Children fed back their views as part of the 1000 voices initiative and were also involved in helping select early health providers. The Southwark Youth Council (SYC) and the looked after children user group have been involved in designing and procuring services (Box 4). Box 4: Southwark Youth Council Southwark Youth Council (SYC) is a voice for young people run by young people. Young people are redesigning public services and influencing how neighbourhoods are improved. The SYC has been proactive in various decision making activities across the borough. The main members of SYC are young people living, attending school, and working in Southwark. They hold events and workshops as a way for young people to express their views and for SYC to take those views to the decision makers. Areas for Improvement For out of hospital pathways, the general approach is to engage patients through workshops to discuss procurement options. However, for some services a patient representative from EPEC has sat on procurement panels even though they are not a current or ex-user of the service (Box 5). Box 5: Procurement of Out of Hospital Pathways Commissioners tried to encourage service users of the ENT and dermatology services to be involved in the procurement of the new services. However, it was difficult to generate interest from patients because involvement could require a level of commitment that was beyond most patients particularly as meetings take place during working hours. In February 2013, the procurement processes of the ENT and dermatology services were presented at EPEC. It was agreed at the meeting that there should be a patient representative present on both procurement panels and both of these representatives were members of EPEC. 19

20 4.4 Monitoring and Evaluation Examples of Good Practice There are a number of good examples of where patient experience information is being used to monitor and evaluate services. At King s, commissioners monitor patient experience performance through monthly HRWD reports and quarterly Quality and Governance reports and discuss issues further at monthly CQRG meetings. Working jointly with King s, commissioners have used patient experience data to initiate service improvements through the development of CQUINs. Box 6: King s Maternity service In 2011/12, a study of maternity services in England showed that London had one of the highest maternity death rates; was double the national average and that women in London were found to be the least satisfied with the care that they received during labour and birth. King s HRWD surveys also showed that patient experience was below target for maternity services. This prompted commissioners to work with the trust to develop a CQUIN for 2012/13 to improve performance, which did result in an improvement in performance and the achievement of most of the CQUIN targets. Box 7: King s Worst Wards In 2011/12 commissioners and King s used the data that informs the HRWD surveys to identify the worst performing wards. A CQUIN was introduced in 2012/13 to incentivise performance improvement in those wards and resulted in improvement in patient experience and achievement of all CQUIN targets. Box 8: King s Outpatients In 2012/13, a CQUIN was developed with the trust to measure and improve patient experience in specific areas of outpatients that had been identified as requiring improvement from local HRWD survey results. The areas were identified from survey results in 2011/12 and a number of indicators were developed to track and improve performance in areas identified as needing the most improvement. Commissioners have been able to identify areas for service improvement at King s through patient experience data and the trust has worked jointly with commissioners to develop CQUINs to improve performance. CQUINs can be an effective way of improving performance temporarily, however if the improvements are primarily driven by incentives then there is a risk that performance could 20

21 deteriorate following removal of the incentive. Inserting penalties into contracts is another potential approach; however it has proven difficult to insert penalties for indicators that are not nationally mandated making it difficult to embed quality improvements contractually. The CSU works with King s to improve areas of patient experience by discussing patient experience issues covered in quarterly quality reports at CQRG meetings. It is difficult to quantify the effectiveness of this approach on improving patient experience. Quality Accounts King s annual Quality Accounts summarise their performance and improvements against quality priorities and objectives set by themselves for It also outlines those they have agreed for the coming year The main domains of the report cover patient safety, clinical effectiveness and patient experience. The two main areas of development under patient experience for 2013/14 were improving outpatient experience and improving patient experience of discharge. 1. Improving outpatient experience in Suite 1 The aims of 13/14 were to agree which 5 questions are most in need of improvement within Suite 1, develop action plans and agree improvement target percentage for Q2, 3 and 4. The trust also aimed to achieve a decrease in patient complaints relating to the outpatient department. 2. Improving patient experience of discharge The aims of 13/14 were to improve trust-wide patient satisfaction in two key areas; providing information about medication after discharge and providing patients with information on what to do and who to contact if they have a concern after discharge. These are questions in the HRWD local surveys and so improvement could be tracked on a monthly basis. Mental Health - Examples of effective use of patient experience information The CCG Mental Health Commissioning Team regularly engage with patient forums and use the information to feedback to services for improvement. Some of the common issues that have been raised include day centre provision, inpatient ward food, inpatient ward cleanliness and staff attitude. a) Day centre provision has been a particularly common topic of discussion over the last 12 months coupled with the introduction of personal budgets. Council members have voiced frustration that they have not been kept up-to-date with latest information which led to project leads organising a news letter for service users. b) Comments on the quality and quantity of food on wards are fed directly back to the SLaM ward manager who in turn picks up the issue with the hospitality contract holders. 21

22 Any improvements are fed back to the group at the following meeting. Box 9: Southwark Mind NHS Southwark CCG commission Lambeth and Southwark Mind to run the Southwark User Council. All members are current or previous service users and represent commissioned services and visit their service on a regular basis asking for feedback from current service users. Feedback is then discussed at the Council monthly meetings (closed for the first 2 hours) and then open with Commissioners and Service providers in attendance for the next 1.5 hours. Two members of the council sit on the Mental Health Programme Management Group. The CCG Mental Health Commissioning Team monitor performance at SLaM through its PEDIC reports. Areas of concern highlighted by PEDIC survey results are raised with the trust through commissioner responses and are discussed at quality meetings with the trust. The CCG Mental Health Commissioning Team also tries to triangulate patient experience information with complaints reports and again feeds back concerns in commissioner responses. Commissioners have also conducted ward inspections at SLaM and have picked up a number of issues that can affect patient experience from ward staff. These have been raised at quality meetings with the trust. Box 10: SLaM patient experience CQUIN In 2013/14, a patient experience CQUIN was included in the SLaM contract. The CQUIN involved developing patient focus groups to feedback on areas that require improvement. The issues highlighted were staff attitude, feeling respected by staff and the environment. SLaM then developed action plans to improve these areas and this was monitored by the CCG mental health team. In Q4, a patient focus group met to review progress against improvement plans and acknowledged that there had been improvement in all the areas highlighted at the start of the year. In 2014/15, commissioners requested that SLaM become more pro-active in asking for patient feedback. It was agreed that the trust be more proactive in asking patients if they would like to feedback on their experiences at discharge on an ipad. The ipad also gave patients the opportunity to fill in PEDIC surveys and write comments on their experiences. Areas for Improvement The triangulation of patient experience information with other quality and safety information has been improving. The quarterly CCG Quality Report has included some information on patient experience and information from the CCG s engagement structure and public engagement events; this is a good start, however future reports should include more patient experience information to allow for more consistent and wider triangulation. 22

23 CQRGs review patient experience information and this provides an ideal opportunity to triangulate patient experience information with other quality and safety information. CQRG leads report back to the Integrated Governance and Performance Committee (IGP) and Governing Body on the main quality and safety concerns at providers, however there isn t always a specific focus on patient experience information. If more patient experience information is reported back, commissioners could then add their insights to the data and enhance the performance monitoring process. Organising and Managing the Data Commissioners have fed back that there is a lack of capacity and a system to collect, analyse and triangulate data effectively. Quality Analysis Database Birmingham CCG has been developing a system that they hope will revolutionise the way they use patient experience information. They are developing a Quality Analysis Database (QUAND) which will eventually form a substantial central repository for all the CCG s patient experience data. They are designing the system to automatically extract information from national databases like NHS Choices and Patient Opinion and from local databases like the primary care patient experience module, Datix. They are also trying to triangulate this data with quality and safety indicators like falls and pressure ulcers and so the system will be able to automatically extract data from the Strategic Executive Information System (STEIS). The system will also allow users to input information manually making it capable of capturing soft data. Data will be broken down to a granular level allowing users to search at provider, specialty and even ward level to try and identify trends. Insights Dashboard London CCGs are considering procuring a London based patient experience dashboard that will bring together a wide array of publically available information covering hospitals and GP practices. There are three different subscription levels available for the CCG to choose from - subscription levels offer varying degrees of detail and bespoke information. Southwark CCG managers will be attending the London CCG Engagement Leads meeting on 29 September to review subscription options. 23

24 5. Recommendations 1) The CCG could further benefit from the use of patient experience information by triangulating national and local information and by triangulating patient experience information with complaints and quality and safety information. This could provide a more comprehensive level of intelligence which could help commissioners identify trends. Commissioners should consider broadening the source of information to include more frequent use of online forums; and reports and intelligence from organisations that work closely with patients like charities and Healthwatch Southwark. To allow for effective triangulation, the CCG needs to capture more qualitative data to add greater depth to the quantitative data we currently collate. This can be through issues identified in the CCG Quality Report, in the form of written feedback, local soft intelligence captured from engagement with patients. The CCG should consider the resource implications of this development. 2) CCG officers should develop quarterly briefings to CQRG leads, with intelligence gained from a triangulated approach to quality and safety. This will allow CQRG leads to monitor performance with providers more effectively and support them to improve services. 3) Commissioners should further encourage providers (particularly large providers) to design and implement improvement plans to enhance patient experience in specific areas of underperformance. Providers are not keen to engage on this in the absence of financial incentives so a collaborative approach overseen by CQRG meetings could be developed. 4) The CCG should work with services that do not currently provide patient experience information to develop appropriate means of capturing information. The regular capture of patient experience and the implementation of improvement plans should be included in all provider contracts. 5) Commissioners should consider ways of understanding in greater detail patients, carers and their families experience of care across pathways and not just at provider sites. a. The CCG should consider commissioning broad-based surveys of patients to understand from a wider sample more about their experience of the care they have recently received. This would need to look at specific issues that we know are important (e.g. the discharge process; compassion of nursing care etc.). b. A project should be initiated to understand in greater depth some of the issues facing patients more likely to use health services. Patients with Continuing Healthcare packages of care may be a useful cohort to work with in the first instance. The CCG s continuing care team works closely with individual patients and their carers and could conduct in-depth interviews to understand in more detail patients experiences. 6) The CCG should ensure that patient representatives are involved in all clinical procurements, including procurement panels. Patient representatives can help the CCG identify innovation and good practice and identify providers who meet the needs of patients. For this to happen; they should be experienced patients or carers in the specific area and be offered support by the CCG to fulfil this role. 24

25 Appendix A - Friends and Family Test The Friends and Family Test (FFT) is a single question survey which asks patients whether they would recommend the NHS service they have received to friends and family who need similar treatment or care. The FFT was rolled-out across the NHS following recommendations made by the Nursing and Care Quality Forum (NCQF) to the Prime Minister. Initially, the test had two main aims as defined by the Department of Health: To provide a comparable performance measure across providers An improvement tool to enhance understanding about where and how improvements could be made. A recent review of the FFT by NHS England endorsed the tests potential to facilitate service improvement; however it did recognise its limitation in providing a comparable performance measure across providers. FFT was initially rolled out for providers of NHS funded acute services for inpatients (including independent sector organisations that provide acute NHS services) and patients discharged from A&E (type 1 & 2) from April As of 1 October 2013 the survey was extended to include all women of any age who use NHS funded maternity services and from 1 December 2014 it will be available in GP practices. The test will be rolled out even further next year; from January 2015 it will be rolled out in mental health and community services and from 1 April 2015, it will be expanded to NHS dental practices, ambulance services, patient transport services, acute hospitals outpatients and day cases. The 2014/15 National CQUIN FFT is part of the 2014/15 national CQUIN which incentivises providers to improve performance. The 2014/15 CQUIN will incentivise the following: 1. Early implementation of the test in outpatient and day case departments by 1 October Increasing or maintaining response rates in acute inpatient services. i. A baseline response rate in Q1 of at least 25 per cent and by Q4 a response rate that is both (a) higher than the response rate for Q1 and (b) 30 per cent or over; or ii. Maintaining a response rate that is over 30 per cent. 3. Increasing or maintaining response rates in A&E. i. A baseline response rate of at least 15 per cent and by Q4 a response rate that is both (a) higher than the response rate for Q1 and (b) 20 per cent or over; or ii. Maintaining a response rate that is over 20 per cent. 25

26 The Question In its current form, the test asks the following standardised question: How likely are you to recommend our ward/a&e department to friends and family if they needed similar care or treatment? Patients will use a descriptive six-point response scale to answer the questions with the following response categories: 1 Extremely likely 2 Likely 3 Neither likely nor unlikely 4 Unlikely 5 Extremely unlikely 6 Don t know Friends and Family Test results are calculated using underlying Net Promoter Score methodology. All patients within the target groups will be given the opportunity to respond to the Friends and Family Test. Patients must be surveyed at, or within 48 hours of, discharge. Scoring Methodology The following example explains how the FFT score is calculated. Assume that a trust records the following results. Total number of responses Extremely likely 114 Likely 54 Neither/nor 24 Unlikely 2 Extremely unlikely 6 Total

27 The FFT score is calculated by the following formula: Proportion of respondents who would be extremely likely to recommend (response: extremely likely ) (x 100) = 57 Minus Proportion of respondents who would not recommend (response: neither likely nor unlikely, unlikely & extremely unlikely ) ) = (x 100) = 16 Friends and Family Test Score: = 41 A common query regarding the scoring methodology is, why those that respond "likely" are excluded from the calculation? FFT is meant to drive excellence within patient experience so, if a patient responds with likely then they will not have had the best experience. However, they are not excluded from the calculation, because they are included in the denominator. If a patient chooses to respond likely instead of a less positive category, that will lead to a higher score. If they choose to respond likely instead of extremely likely that will lead to a lower score. Likely responses therefore have an important influence on the resulting score. 27

28 Appendix B - Sources of Patient Experience Information Source/Report Detail Source National Voices King s How Are We Doing (HWRD) King s In Conversation King s In Your Shoes King s Maternity Services Liaison Committee King s Cancer Voices National coalition of health and social care charities in England. Focus on patient centred care Opportunity for patients to fill in surveys on many services areas: Outpatients, Emergency Department, Emergency Dental, Early Pregnancy Unit, Maternity Services, Dental Outpatient Services, Inpatients and Day surgery patients. Carers, relatives, friends or visitors can fill in How are we doing? Comments and Suggestions cards. King s produce monthly reports detailing results. Listening events with patients regarding the following questions: Are patients always our first priority? Would you recommend our hospitals as places to receive care? What do you think we could do better? Opportunity for King s staff to listen to patient views. The MSLC is a group that includes mums as well as midwives, doctors and other health professionals. The group meets about every two months. A group made up of patients, relatives and carers affected by cancer and health professionals. Jessica Bush King s Head of Engagement and Patient Experience. jessica.bush@nhs.net King s Have Your Say Repository of patient comments 28

29 Source/Report Detail Source GSTT Feedback Opportunity for patients to fill in surveys on Friends and Family test, general surveys, service-specific surveys and Community services. s-and-visitors/feedback/your-feedback.aspx GSTT annual Patient Experience and Engagement Report The GP Patient Survey CQC National Maternity Survey 2013 CQC National A&E Survey 2012 CQC National Inpatient Survey 2012 Provides an overview of the work that has taken place during GP survey results available at CCG and individual practice level. Key findings from the 2013 national survey of women s experiences of maternity care. Asks people about their experiences of A&E departments in Asks patients who were admitted to hospital services about their experiences in GSTT Board papers ccg&index=0 rveys/maternity-services-survey-2013 CQC National Outpatient Survey 2011 Asks patients about their views on their most recent visit to an outpatient department

30 Source/Report Detail Source CQC Community Mental Health Survey 2013 NHS England National Cancer Patient Experience Programme Asks patients about their experiences of receiving community mental health services. Asks patients about their experiences of receiving cancer care. rveys/community-mental-health-survey Local quarterly Community surveys Friends and Family Test SLaM PEDIC surveys and reports Cancer Patient Experience Survey: Insight Report and League Table Includes; Children occupational therapy, children physiotherapy, children speech and language therapy, District nursing, foot health, health visiting, paediatrics and reproductive and sexual health The Friends and Family Test is a single question survey which asks patients whether they would recommend the NHS service they have received to friends and family who need similar treatment or care. Currently the test covers; i) A&E ii) Inpatients iii) Maternity There are a number of patient experience surveys carried out by SLaM which cover; outpatients, inpatients and trustwide. The National Cancer Patient Experience Survey (CPES) captures a picture of progress among hospital trusts. London trusts are amongst the worst performing in the country Jean Young, Meridian Desktop al-work-areas/friends-and-familytest/friends-and-family-test-data/ Southwark CCG Mental Health team outus/research/keystats/2013cpesinsight BriefingFINAL.pdf 30

31 Appendix C HRWD Tracker 31