Quality Assurance Accreditation Scheme Assignment Report 2016/17
Contents 1. Introduction 2. Executive Summary 3. Findings, Recommendations and Action Plan Appendix A: Terms of Reference Appendix B: Assurance Definitions and Risk Classifications P a g e 1
1. Introduction, Background and Objective MIAA undertook a review of the effectiveness of the Trusts developed Quality Assurance Accreditation Scheme (QAAS) in accordance with the requirements of the Internal Audit Plan 2016/2017, as approved by the Audit Committee. The Health and Social Care Act 2008 (Regulated Activities) Regulations 2010 stipulates that any provider of regulated activities, including the NHS, must be registered with the Care Quality Commission (CQC) in order to legally provide services. NHS Trusts must be able to demonstrate that they comply with the legally enforceable essential standards of quality and safety regulations as set out in the Act. This registration system places emphasis on ongoing compliance rather than snap-shot evaluation. It is designed to provide a rigorous method of monitoring and should help to ensure greater consistency and higher standards in the quality of care. Organisations are developing and adapting their own internal monitoring systems in which to monitor, report and provide assurances of its performance and regulatory requirements. It is essential that all healthcare organisations regularly review their current evidence framework and associated assurance activities to ensure that there is a robust and grounded approach in place. 2. Executive Summary There are some weaknesses in the design and/or operation of controls which could impair the achievement of the objectives of the system, function or process. However, either their impact would be minimal or they would be unlikely to occur. Significant Assurance As agreed with the Trust, MIAA undertook an unannounced review of 9 randomly selected clinical areas identified below; Emergency Department Intensive Care Unit Helme Chase Ward 5 Ward 17 KEOSC Ward 9 Ward 37 Day Surgery P a g e 2
have developed and implemented an effective system for the Quality Assurance Accreditation Scheme with identified clear lines of accountability. The review by MIAA has identified areas for further review in relation to the role of the Quality Assessor Team, ward engagement, and revisit timescales, retrospective amendments to Guru and governance reporting. The recommendations made do not pose a high level risk to the Trust but once implemented, will strengthen the systems and processes that the Trust have established. The following provides a summary of the key themes; Has the Trust has established an effective system to provide self-assessments which are fit for purpose? have established an effective framework for the Quality Assurance Accreditation Scheme (QAAS) using internal data intelligence to identify priority clinical areas for inspection. The Inspections and visit schedule, at ward level, are led by the Quality Matron for the Trust who is supported by named Quality Champions who undertake a collective approach to the ward visits. This is good practice. It was identified that from the 9 wards that were reviewed by MIAA there were 5 occasions, where 3 assessors were in attendance at the QAAS visit, 2 occasions where there was 2 assessors and also 2 occasions where there was 1 assessor. MIAA were verbally advised that the rationale for the differences in the number of assessors was due to resource availability. This was not documented and therefore MIAA are unable to provide assurance in this regard. Undertaking inspections with only 1 assessor is a business continuity risk, should the staff member require unplanned time away from the office, the QAAS visits may not be sustainable. To overcome this, the Trust should consider implementing a framework which identifies, by ward size, the minimum number of assessors required to undertake the QAAS visit. (Recommendation 2 Medium) Where it is not possible to proceed with the QAAS due to lack of resource, this should be escalated through the Trust s governance structure. (Recommendation 2 Medium) When reviewing the completed QAAS reports for the clinical areas MIAA did identify that from the 9 most recent ward reports there were 2 errors captured within the results which related to training compliance within the Emergency Department and Day Surgery/Westmoreland Surgical Centre. Whilst this did not affect the overall assurance opinion of ward performance, this has highlighted an area for further review in the P a g e 3
control design of the framework. The QAAS system on Guru does not have the functionality to amend the QAAS data once it has been populated. There is a function within Guru that will support deletion of the QAAS report in its entirety however this approach removes auditable evidence and causes rework. The Trust should consider options on how information entered on the QAAS system can be amended retrospectively without deleting the original report. (Recommendation 5 low) The Trust has developed a framework for re-inspection of the wards following their QAAS inspection. Where wards was have been identified as; Inadequate, a revisit is scheduled for completion in 2 months. Requires improvement, a schedule revisit is set for 3 months. Good within 6 months. In benchmarking the Trust s QAAS framework against assessment frameworks used within other acute providers, MIAA would recommend that the Trust reconsiders the revisit visit schedule. This would support the Trusts to target its resources in the high risk areas. (Recommendation 3 medium) Following each inspection, the Quality Matron collates the findings and Guru apportions the overall assurance opinion from the answers provided. Verbal feedback is provided to the ward by the Quality Matron at the time of the visit and a full copy of the report is available for Ward Managers shortly after the inspection. Providing feedback in this manner is good practice and will support wards in taking immediate action if required. Wards are involved in the collation of data and evidence to populate the QAAS MIAA can provide assurance that from the 9 wards reviewed, all were involved in the collation of data and evidence. All were aware of QAAS and each ward Manager/Matron had a good knowledge and understanding of the role of the Quality Matron and the purpose of the visits. All welcomed the visits being unannounced and ward staff advised MIAA that they felt supported by the Quality Matron. Most of the clinical areas were engaged with the scheme, albeit some areas were more engaged that others. All wards identified improvement in practices in areas which had been highlighted as non-compliant within their individual QAAS. Visits. This is good practice and provides assurance that clinical areas are taking action against the recommendations made during the inspections. Wards did not always proactively feedback their completed actions and progress against the recommendations directly in to Guru. Wards verbally advised MIAA that P a g e 4
they would await their next QAAS visit as this would highlight their improvement. The Trust should ensure that all senior ward staff take a proactive approach in reporting in to the QAAS. This will ensure that Guru contains real time information. (Recommendation 5 low) MIAA reviewed the route in which the findings from the QAAS visits are shared with ward staff. Communicating to the Team fosters a proactive approach, avoiding reoccurrence of non-compliance and provides staff with the rationale for any changes in practice. From the 9 wards reviewed, MIAA found documented evidence that the information had been shared with staff within 6 wards. This was achieved through Newsletters, Team Meeting minutes, 3 minute Briefs and Safety Brief meetings. Whilst not part of this review, MIAA did identify that not all wards met regularly as a team. Ensuring that wards meet on a regular basis will provide staff with a forum in which to raise any queries. In the main, senior ward staff understood their roles and responsibilities for QAAS however this was not reflected within the Intensive Care Unit at Royal Lancaster Infirmary. There were gaps in knowledge on Ward 17 regarding QAAS however the Matron had taken proactive steps to meet with the Quality Matron in order to clarify the wards requirements which is good practice and provides assurance regarding good engagement. Is the evidence collation process for QAAS is effective and provide assurance to of current performance The evidence collation process for QAAS was found to be effective although could be strengthened following the implementation of the recommendations within this report. In providing assurance of current performance this is undertaken through a number of forums. Matrons Trust Matrons meet on a monthly basis. Within the meeting, a Matrons Dashboard is submitted which provides ward performance information. A copy of the Dashboard for August 2016 was submitted for review. The first QAAS visit for Ward 5 was undertaken on 23 rd March 2016, however information is documented from November 2015. Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep P a g e 5
FGH Ward 5 General Surgery 3 5 5 8 2 1 2 2 1 This approach is reflective of other wards identified within this review. (Recommendation 6 low) Quality Committee From July 2016 the Quality Committee has been in receipt of a QAAS report. The report is informative and provides a dashboard overview of the level of assurance provided to the wards that have been reviewed by the Quality Matron and Quality Leads. MIAA reviewed the report from July 2016 (next due for submission in September 2016) which found that ward 37 is identified as green however the detailed report submitted to MIAA for review is 'amber'. (Recommendation 1 Medium) The remaining information was accurately reflective. Board The first QAAS update report was submitted to the Quality Committee in July 2016, at the time of the audit undertaken by MIAA, a Board meeting had not occurred. MIAA are therefore unable to provide assurance that the QAAS data is presented to the Board for assurance. Patient Medical Records MIAA were asked to review a number of patient records within the clinical areas identified above to assess if each of the patient records identified a clear medical management plan and where tests were required that the results were actioned. MIAA reviewed 36 patient records which identified that 2 from the 36 did not have documented evidence that referrals / testing that was required, had been undertaken. Within ward 17, a patient had been identified as requiring referral to the Mental Health Midwife as a safeguarding concern. The ward was unable to locate documented evidence to provide assurance that this was actioned. MIAA raised this with the Matron on the day of the audit as an urgent action. A patient within ward 9 was identified for an ECG on 8 th September 2016. On the day of the audit by MIAA, 12 th September 2016, a copy of the patients ECG could not be located. This was raised with the Ward Manager at the time of the audit as an urgent action. It must be noted that at the time of the audit, wards were transitioning from paper patient records to Lorenzo and some staff were unfamiliar with the system. Further P a g e 6
training would be advantageous in order to ensure that staff are able to locate patient referral and test requirements. (Recommendation 4 Medium) 3. Findings, Recommendations and Action Plan The review findings are provided on a prioritised, exception basis, identifying the management responses to address issues raised through the review. To aid management focus in respect of addressing findings and related recommendations, the classifications provided in Appendix B have been applied. The table below summarises the prioritisation of recommendations in respect of this review. Critical High Medium Low Total 0 0 4 2 6 Other detailed findings and recommendations are set out below. P a g e 7
Detailed Recommendations 1. Governance Reporting Quality Committee Risk Rating: Medium Control design Issue Identified - The Update on the Quality Framework Report that was submitted to the Quality Committee contained an error relating Ward 37. Within the same Committee papers say Ward 37 is identified as 'green' however the report submitted is 'amber'. Specific Risk Failure to accurately report within the Governance structure can lead to organisational risk. Recommendation - The Trust should ensure that quality assurance measures are implemented to ensure accuracy of reporting. Management Response (Remedial Action Agreed) - Error acknowledged Further proof reading and cross checking against dashboard reports will now take place Responsibility for Action QA Matron / Visit teams Deadline for Action ongoing 2. Business Continuity Risk Rating: Medium Control design Issue Identified - On occasion the Trust has 1 assessor to undertake the QAAS visits. Specific Risk QAAS. Failure to ensure robust business continuity plans to ensure the delivery of Recommendation - The Trust should consider implementing a framework which identifies, by ward size, the minimum number of assessors required to undertake the visit. Where it is not possible to proceed with the QAAS due to the resources available the Trust should ensure that these are formally recorded within the Trust s governance structure. P a g e 8
Management Response (Remedial Action Agreed) - No visit now goes ahead with just QA Matron minimum of two people per visit required otherwise it is rescheduled. If visits are cancelled this is escalated verbally to DCN / Matrons / CAN S or via numerous email to teams asking for additional support on rescheduled date and to ECN / Matrons meetings and visits rescheduled to reflect this. A big conversation is being arranged to review the whole QA framework and support will from part of the discussion Ongoing meetings with PMO and information discussed and workbook updated to reflect this. Responsibility for Action QA Matron Deadline for Action Big conversation is scheduled for November 2016 ( date tbc) other actions will be ongoing 3. Timescales for Revisit Risk Rating: Medium Control design Issue Identified - In benchmarking the Trust s QAAS framework against assessment frameworks used within other acute providers, MIAA would recommend that the Trust reconsiders the revisit visit schedule. This would support the Trusts to target its resources in the high risk areas. Specific Risk failure to target resources in the high risk areas could put patient safety at risk. Recommendation - MIAA would recommend that the Trust reconsiders the revisit schedule. Management Response (Remedial Action Agreed) - No new visits will be undertaken until all revisits are scheduled in calendar (this date will still be dependent on the visit scores. The visit schedule reflects this. All re visits will be completed in timescales depending on score Responsibility for Action QA Matron Deadline for Action November 2016 4. Lorenzo Risk Rating: Medium P a g e 9
Operating effectiveness Issue Identified - Staff were unfamiliar with the functionalities within Lorenzo. Specific Risk Failure to access patient medical records putting patient safety at risk. Recommendation - Further training would be advantageous in order to ensure that staff are able to locate patient referral requirements and test results. Management Response (Remedial Action Agreed) - to address and support Referred to Nursing Design Authority Responsibility for Action QA Matron / Nursing Design Authority Deadline for Action - referred 5 th October 2016 subsequent meetings and actions to follow 5. Guru Risk Rating: Low Control design Issue Identified - The Guru system does not support retrospective editing of completed QAAS records. When errors occur these cannot be amended, the record must be deleted and the QAAS results entered in their entirety. The review evidenced that wards were not proactively updating Guru with completed actions. Specific Risk Failure to provide clear performance data regarding Ward performance resulting in a breach of regulatory requirements. Recommendation - The Trust should consider processes where results can be quality assured prior to data entry or alternatively permit retrospective editing. Ward staff should be reminded of their roles and responsibilities regarding follow on actions from QAAS. Management Response (Remedial Action Agreed) - Teams reminded of the need to do this via various email and support guide shared Information on QA Webpage for all staff made available of how to do this I3 support offered to support teams Meetings to discuss GURU issues verbal ongoing as issues arise P a g e 10
Staff shown what and how to do this on QAAS visits Responsibility for Action All teams involved in QAAS process Deadline for Action ongoing 6. Matrons Dashboard Report Risk Rating: Low Control design Issue Identified - The Matrons Dashboard provides ward performance information. A copy of the Dashboard for August 2016 was submitted for review. The first QAAS visit for Ward 5 was undertaken on 23 rd March 2016, however information is documented from November 2015. This is reflected of other wards reviewed. Specific Risk Failure to provide clear performance data regarding Ward performance resulting in a breach of regulatory requirements. Recommendation - The Trust should consider adopting a uniformed approach to reporting. Management Response (Remedial Action Agreed) Regular Meetings arranged with I3 development team to share issues as they arise Issue above resolved further report sent to demonstrate compliance Responsibility for Action QA Matron / I3 dashboard development team Deadline for Action ongoing Follow-up In light of the findings of this audit we would recommend that follow-up work to confirm the implementation of agreed management actions is conducted within the next 12 months. P a g e 11
Appendix A: Terms of Reference The overall objective of the review was to provide an opinion on the effectiveness of the Trusts developed Quality Assurance Accreditation Scheme (QAAS), reviewing the operating effectiveness from Board to ward. The following sub objectives were agreed with the Trust; The Trust has established systems which are fit for purpose The Trust has established an effective system to provide self-assessments which are fit for purpose. Wards are involved in the collation of data and evidence to populate the QAAS. Feedback from QAAS reports allow corrective actions to be implemented at Ward level. The evidence collation process for QAAS is effective and provide assurance to of current performance MIAA undertook a review of the objectives and risks identified above for the established systems and processes and reviewed the operational controls against expected corporate controls. MIAA undertook unannounced compliance testing on 3 clinical areas at each of the hospital sites idenitfed below. Furness General Hospital Royal Lancaster Infirmary Westmoreland General Hospital Limitations of Scope: The review was limited to the systems defined in the objectives and to the locations outlined above. The review does provide assurance of compliance against all actions identified in QASS which are attributable to the clinical areas for review or the Trusts compliance with CQC standards. A p p e n d i x A 1
Limitations inherent to the internal auditor s work We have undertaken the review of Quality Assurance Accreditation Scheme process, subject to the following limitations. Internal control Internal control, no matter how well designed and operated, can provide only reasonable and not absolute assurance regarding achievement of an organisation's objectives. The likelihood of achievement is affected by limitations inherent in all internal control systems. These include the possibility of poor judgement in decision-making, human error, control processes being deliberately circumvented by employees and others, management overriding controls and the occurrence of unforeseeable circumstances. Future periods The assessment of controls relating to the Quality Assurance Accreditation Scheme process is that at September 2016. Historic evaluation of effectiveness is not always relevant to future periods due to the risk that: The design of controls may become inadequate because of changes in the operating environment, law, regulation or other; or The degree of compliance with policies and procedures may deteriorate. Responsibilities of management and internal auditors It is management s responsibility to develop and maintain sound systems of risk management, internal control and governance and for the prevention and detection of irregularities and fraud. Internal audit work should not be seen as a substitute for management s responsibilities for the design and operation of these systems. We shall endeavour to plan our work so that we have a reasonable expectation of detecting significant control weaknesses and, if detected, we shall carry out additional work directed towards identification of consequent fraud or other irregularities. However, internal audit procedures alone, even when carried out with due professional care, do not guarantee that fraud will be detected. The organisation s Local Counter Fraud Officer should provide support for these processes. Data Protection and Freedom of Information All documents acquired or created by us during the course of this assignment remain the property of the client. A p p e n d i x A 2
MIAA are, thus, considered as a data processor rather than a data controller and are not, therefore, directly subject to the requirements of the Data Protection Act. No information relating to this, or any other, assignment will be directly disclosed to a third party by MIAA in response to a subject access request. Any requestor will be advised that they should approach the client. These principles will also be applied in respect of any request for information relating to this, or any other, assignment under the Freedom of Information Act. A p p e n d i x A 3
Appendix B: Assurance Definitions and Risk Classifications Level of Assurance High Significant Limited No Description Our work found some low impact control weaknesses which, if addressed would improve overall control. However, these weaknesses do not affect key controls and are unlikely to impair the achievement of the objectives of the system. Therefore we can conclude that the key controls have been adequately designed and are operating effectively to deliver the objectives of the system, function or process. There are some weaknesses in the design and/or operation of controls which could impair the achievement of the objectives of the system, function or process. However, either their impact would be minimal or they would be unlikely to occur. There are weaknesses in the design and / or operation of controls which could have a significant impact on the achievement of the key system, function or process objectives but should not have a significant impact on the achievement of organisational objectives. There are weaknesses in the design and/or operation of controls which [in aggregate] have a significant impact on the achievement of key system, function or process objectives and may put at risk the achievement of organisational objectives. Risk Rating Assessment Rationale Critical High Medium Low Control weakness that could have a significant impact upon, not only the system, function or process objectives but also the achievement of the organisation s objectives in relation to: the efficient and effective use of resources the safeguarding of assets the preparation of reliable financial and operational information compliance with laws and regulations. Control weakness that has or is likely to have a significant impact upon the achievement of key system, function or process objectives. This weakness, whilst high impact for the system, function or process does not have a significant impact on the achievement of the overall organisation objectives. Control weakness that: has a low impact on the achievement of the key system, function or process objectives; has exposed the system, function or process to a key risk, however the likelihood of this risk occurring is low. Control weakness that does not impact upon the achievement of key system, function or process objectives; however implementation of the recommendation would improve overall control. A p p e n d i x B 1
Key Contacts and Report Distribution Name Title Report Distribution Aaron Cummins Director of Finance Final Lynne Wyre Deputy Chief Nurse Draft & Final Mary Aubrey Director of Governance Draft & Final Sue Smith Executive Chief Nurse Draft & Final Sally Young Quality Assurance Matron Draft & Final Acknowledgement and Further Information MIAA would like to thank all staff for their co-operation and assistance in completing this review. This report has been prepared as commissioned by the organisation, and is for your sole use. If you have any queries regarding this review please contact the Audit Manager. To discuss any other issues then please contact the Director. MIAA would be grateful if you could complete a short survey using the link below to provide us with valuable feedback to support us in continuing to provide the best service to you. https://www.surveymonkey.com/r/miaa_client_feedback_survey
MIAA Key Contacts Name: Hannah Dreelan Title: Healthcare Quality Facilitator Telephone: 0151 285 4575 Email: hannah.dreelan@miaa.nhs.uk Name: Title: Pete Nowell Internal Audit Manager Telephone: 0151 285 4768 Email: pete.nowell@miaa.nhs.uk Name: Debbie Rimmer Title: Head of Healthcare Quality Telephone: 0151 285 4528 Email: debbie.rimmer@miaa.nhs.uk A p p e n d i x B 1