Data Quality and Clinical Coding for Improvement What happens when the data are wrong? The key responsibilities for Clinicians and Managers

Similar documents
The Royal Wolverhampton Hospitals NHS Trust

Frequently Asked Questions (FAQ) Updated September 2007

Monitoring hospital mortality A response to the University of Birmingham report on HSMRs

Scottish Hospital Standardised Mortality Ratio (HSMR)

Learning from Deaths; Mortality Review Policy

Learning from Deaths Policy LISTEN LEARN ACT TO IMPROVE

Focus on hip fracture: Trends in emergency admissions for fractured neck of femur, 2001 to 2011

Reducing In-hospital Mortality

Care Quality Commission (CQC) Inspection Briefing

Using the structured judgement review method

NHS performance statistics

O U T C O M E. record-based. measures HOSPITAL RE-ADMISSION RATES: APPROACH TO DIAGNOSIS-BASED MEASURES FULL REPORT

SUMMARY OF INDICATOR CHANGES FOR VERSION 3 INTELLIGENT MONITORING REPORTS Acute and Specialist NHS Trusts 23 June Final Draft, Subject to Change

Measuring for improvement The new CQC hospital programme. Professor Sir Mike Richards Chief Inspector of Hospitals King s Fund 6 th November 2013

NHS performance statistics

Special measures: one year on. A report into progress made at 11 NHS trusts that were put into special measures in July 2013

Overview of a new study to assess the impact of hospice led interventions on acute use. Jonathan Ellis, Director of Policy & Advocacy

April Clinical Governance Corporate Report Narrative

Assessing Quality of Hospital Services - the importance of national clinical audits

Learning from Patient Deaths: Update on Implementation and Reporting of Data: 5 th January 2018

Appendix 1 MORTALITY GOVERNANCE POLICY

Hospital Standardised Mortality Ratios

NHS Performance Statistics

Monthly and Quarterly Activity Returns Statistics Consultation

London CCG Neurology Profile

Is the quality of care in England getting better? QualityWatch Annual Statement 2013: Summary of findings

EuroHOPE: Hospital performance

NHS Patient Survey Programme 2016 Emergency Department Survey

Emergency readmission rates

My Discharge a proactive case management for discharging patients with dementia

National Trends Winter 2016

The non-executive director s guide to NHS data Part one: Hospital activity, data sets and performance

Improving Patient Outcomes

Review of Follow-up Outpatient Appointments Hywel Dda University Health Board. Audit year: Issued: October 2015 Document reference: 491A2015

Operational Focus: Performance

Researcher: Dr Graeme Duke Software and analysis assistance: Dr. David Cook. The Northern Clinical Research Centre

Increased mortality associated with week-end hospital admission: a case for expanded seven-day services?

Indicator Specification:

NHS LANARKSHIRE QUALITY DASHBOARD Board Report October 2011 (Data available as at end August 2011)

FT Keogh Plans. Medway NHS Foundation Trust

4 CM/02/18/04. Chief Executive s report to the Board David Behan, Chief Executive Kate Eisenstein, Special Policy Advisor to the Chief Executive

A new integrated model for Care Homes from Walsall CCG/Healthcare NHS Trust

Hospital Mortality Monitoring. May 2015

Finalised Patient Reported Outcome Measures (PROMs) in England Data Quality Note

Society for Cardiothoracic Surgery in Great Britain and Ireland

The Royal Wolverhampton NHS Trust

The Francis Report: Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry 1 A carer s response

Hospital Standardized Mortality Ratios, Edmonton, Canada: A Tale of Two Sites Lessons Learned from the UK

Learning from Deaths Policy. This policy applies Trust wide

Ambulatory emergency care Reimbursement under the national tariff

Boarding Impact on patients, hospitals and healthcare systems

Results of censuses of Independent Hospices & NHS Palliative Care Providers

Kentucky Sepsis Summit. August 2016

BOLTON NHS FOUNDATION TRUST. expansion and upgrade of women s and children s units was completed in 2011.

Delivering a Paperless NHS by Dr Masood Nazir General Practitioner Clinical Informatics Advisor NHS England

Health Care Quality Indicators in the Irish Health System:

Case-mix Analysis Across Patient Populations and Boundaries: A Refined Classification System

NHS Outcomes Framework 2014/15:

National Cancer Patient Experience Survey National Results Summary

Supplemental materials for:

National Clinical Audit & Patient Outcome Programme: An update

Mortality Report Learning from Deaths. Quarter

Pricing and funding for safety and quality: the Australian approach

Tele Stroke ( Telemedicine in Practice)

Commissioning for Value insight pack

Patient safety in the NHS in England and the development of the Healthcare Safety Investigation Branch (HSIB)

North Central London Sustainability and Transformation Plan. A summary

Executive Summary 10 th September Dr. Richard Wagland. Dr. Mike Bracher. Dr. Ana Ibanez Esqueda. Professor Penny Schofield

TRUST BOARD SEPTEMBER Surgical Services Reconfiguration

Patient Experience Strategy

North West COPD Report Nov 2011

Do quality improvements in primary care reduce secondary care costs?

Using mortality data to improve the quality and safety of patient care December 2015

Board of Director s Meeting

National Cancer Patient Experience Survey National Results Summary

Improvement and Assessment Framework Q1 performance and six clinical priority areas

Predict, prevent & manage AKI: A UK collaboration to detect a devastating condition AKI

Birmingham Solihull and the Black Country Area Team

SOUTHAMPTON UNIVERSITY HOSPITALS NHS TRUST. Quarterly Clinical Effectiveness and Outcomes Report:

Implementation of the right to access services within maximum waiting times

Community Pharmacy in 2016/17 and beyond

Implementing NHS Services Seven Days a Week

Current policy context of safe staffing in A&E Departments

Improving Health Services for Carers

Reference costs 2016/17: highlights, analysis and introduction to the data

SHEFFIELD TEACHING HOSPITALS NHS FOUNDATION TRUST EXECUTIVE SUMMARY REPORT TO THE HEALTHCARE GOVERNANCE COMMITTEE HELD ON 25 NOVEMBER 2013

Quality Strategy. CCG Executive, Quality Safety and Risk Committee Approved by Date Issued July Head of Clinical Quality & Patient Safety

Innovation Series Move Your DotTM. Measuring, Evaluating, and Reducing Hospital Mortality Rates (Part 1)

Aneurin Bevan Health Board. Living Well, Living Longer: Inverse Care Law Programme

National Mortality Case Record Review Programme. Using the structured judgement review method A guide for reviewers (England)

Prof. Helen Ward Profesora clínica de Salud Pública y Directora PATIENT EXPERIENCE RESEARCH CENTRE (PERC) IMPERIAL COLLEGE

Hospital at home or acute hospital care: a cost minimisation analysis Coast J, Richards S H, Peters T J, Gunnell D J, Darlow M, Pounsford J

The PCT Guide to Applying the 10 High Impact Changes

Recommendations of the NH Strategy

MORTALITY REVIEW POLICY

Continuously Measuring Patient Outcome using Variable Life-Adjusted Displays (VLAD)

New foundations: the future of NHS trust providers

NHS Wiltshire PCT Programme Budgeting fact sheet /12 Contents

Unscheduled care Urgent and Emergency Care

Hospital Generated Inter-Speciality Referral Policy Supporting people in Dorset to lead healthier lives

Transcription:

Data Quality and Clinical Coding for Improvement Data Quality and Clinical Coding for Improvement What happens when the data are wrong? The key responsibilities for Clinicians and Managers 19 th November 2012 10.15-10:45 am Hallam Conference Centre Brian Jarman Imperial College, London b.jarman@ic.ac.uk @Jarmann

Subjects I have been asked to cover learning from Mid Staffordshire clinical coding: understanding the data and what the data is telling you how Dr Foster use the data to identify outliers and produce alerts case studies and examples in practice

Mid Staffs Independent Inquiry, 24 Feb 2010, p366, 46 Furthermore, the maintenance, certification and submission of accurately recorded data to the regulatory authorities are ultimately the responsibility of Trust leadership.

Data covered Hospital Standardised Mortality Ratios (HSMRs) methodology from Dr Foster unit (DFU) at Imperial College HSMR published annually from 2001 by Dr Foster Intelligence (DFI) main national press Data Quality Summary from DFI Monthly mortality alerts - used the Dr Foster Unit (DFU) at Imperial College to identify outliers and produce alerts

Variables for HSMR logistic regression adjustment Age group (<1, then 5-year bands to 90+) Sex Admission method/type (emergency, elective etc) Admission source (home, transfers etc) Deprivation quintile (based on postcode) Diagnosis subgroup (CCS sub-groups within each CCS group) Comorbidity (Charlson score) Emergency admissions in previous 12 months Palliative care (any episode that has a treatment function code 315 or any Z515 ICD10 diagnosis code) Month of admission Year of discharge (Day cases are excluded from the risk models ~ 70 deaths/year) http://www.drfosterintelligence.co.uk/newspublications/hsmrmethodology.asp

Data Quality Summary DFU monthly checks on reords: Duplicate record Admission/Appointment date Episode order Admission method/appointment type Sex Date of birth Provider Episode start Episode end Episode duration Postcode NHS Number GP Practice Local patient ID Age Day case Deprivation Provider + Local patient ID Consultant Ethnicity Elective/Referral date Discharge/Outcome Diagnosis Procedure Procedure date Spell overlaps Missing first episode Missing last episode Inconsistent bed-days/los Age group Sex (spell) Admission (spell) Episode order (spell) Invalid LOS Invalid HRG

Effect of coding of variables in model Direct vs indirect standardisation Comorbidity (Charlson Index) Palliative care Admission vs patient All in-hospital deaths vs 30-days inhospital deaths HSMR covering 80% deaths (56 CCS) vs HSMR all IP deaths (259 CCS gps)

Indirectly standardised HSMR Comparison of indirect vs direct standardisation 2005 HSMR (simple model) 150% 140% 130% 120% 110% 100% Trusts Identical match 25% more than directly standardised 10% more than directly standardised 10% less than directly standardised 25% less directly standardised 90% 80% 70% 60% 60% 70% 80% 90% 100% 110% 120% 130% 140% 150% Directly standardised HSMR

HSMR without Charlson Charlson co-morbidity index 150 140 Comparison of HSMR calculated with and without Charlson English Acute Trusts 2008/9 130 120 110 100 90 y = 0.9043x + 9.6201 R² = 0.8918 Salford Royal NHS Foundation Trust Trusts Identical match 25% more than standard HSMR 10% more than standard HSMR 10% less than standard HSMR 25% less than standard HSMR 80 70 60 50 50 60 70 80 90 100 110 120 130 140 150 HSMR

HSMR excluding paliative care Palliative care 150 140 130 120 110 100 90 y = 1.0367x - 3.4837 R² = 0.9001 Comparison of HSMR calculated using adjustment or exclusion to account for palliative care English Acute Trusts 2008/9 Trusts Identical match 25% more than standard HSMR 10% more than standard HSMR 10% less than standard HSMR 25% less than standard HSMR 80 70 Medway NHS Trust 60 Walsall Hospitals NHS Trust 50 50 60 70 80 90 100 110 120 130 140 150 HSMR adjusting for palliative care

HSMR based on last admission in financial year Admissions versus Patients 150 140 130 120 110 100 y = 0.8741x + 12.626 R² = 0.9182 Comparison of HSMR calculated using admissions and based on last admission in financial year English Acute Trusts 2008/9 Trusts Identical match 25% more than standard HSMR 10% more than standard HSMR 10% less than standard HSMR 90 Trafford Healthcare NHS Trust 25% less than standard HSMR 80 70 60 50 50 60 70 80 90 100 110 120 130 140 150 HSMR

HSMR in-hospital 30 day deaths HSMR (all in-hospital deaths) vs. all in-hospital 30 day deaths 150 140 y = 0.9774x + 1.7673 R² = 0.9941 Comparison of HSMR calculated using 30 day in-hospital deaths with HSMR using all in-hospital eaths English Acute Trusts 2008/9 130 Trusts 120 110 100 Identical match 25% more than in-hospital deaths HSMR 10% more than in-hospital deaths HSMR 10% less than in-hospital deaths HSMR 25% less than in-hospital deaths HSMR Linear (Trusts) 90 80 70 60 60 70 80 90 100 110 120 130 140 150 HSMR (all in-hospital deaths)

HSMR all 30 day deaths HSMR 30 day in-hospital deaths vs 30 day all deaths 150 140 130 y = 0.8756x + 12.671 R² = 0.9219 Comparison of HSMR calculated using 30 day in-hospital deaths with HSMR using all 30 day deaths English Acute Trusts 2004/5 120 Trusts Identical match 110 100 90 Bradford Teaching Hospitals NHS Foundation Trust 25% more than in-hospital deaths HSMR 10% more than in-hospital deaths HSMR 10% less than in-hospital deaths HSMR 25% less than in-hospital deaths HSMR 80 70 60 50 50 70 90 110 130 150 HSMR in-hospital 30 day deaths

HSMR based on all diagnosis groups 56 diagnosis group HSMR vs. all 259 CCS diagnoses 150 140 Comparison of HSMR calculated using 56 diagnoses compared with all 259 diagnoses English Acute Trusts 2008/9 130 120 110 100 y = 0.9743x + 2.2451 R² = 0.9666 Trusts Identical match 25% more than standard HSMR 10% more than standard HSMR 10% less than standard HSMR 25% less than standard HSMR Linear (Trusts) 90 80 70 60 60 70 80 90 100 110 120 130 140 150 HSMR

The changes of palliative care coding by CfH rules. Changes at Medway, Mid Staffs and Basildon = effect on HSMRs March 2007 - Medway From March 2007 CfH Coding Clinic permitted non-curable illnesses to be coded as palliative care secondary diagnosis Z51.5 (not just terminal illnesses under consultant) Imperial College was persuaded to introduce a palliative care adjustment Medway increased % deaths coded Z51.5 from <1% to 37% and HSMR value decreased (advised by CHKS). Only trust in England at that time to make those changes CHKS, Dec 2008: Our review of coding at Medway NHS Foundation Trust found that end-of-life care was recorded for only eight per cent of deaths, when the actual proportion should have been 37 per cent. We believe that the hospital mortality index should exclude these cases and, by making this adjustment, Medway's mortality index reduced by just over a third. March 2008 - Mid Staffs Mid Staffs made similar changes and HSMR reduced by ~7% Two other West Midlands SHA trusts made similar changes later in 2008 We publicised the effect of Z51.5 changes on HSMR values ( doesn t save lives ) Mid 2009 - Basildon & Thurrock Made similar changes to starting mid 2009 June 2010 CfH Coding Clinic reverted to ~pre 2007

Changes in coding at Medway, Mid Staffs and Basildon Mid Staffs and Basildon In 2005/06 had the highest HSMR (lower 95% CI above 100) In 2007 had the largest number of mortality alerts from Imperial College Had had significantly high HSMRs for about 10 years First major palliative care coding change was at Medway NHS Foundation Trust in 2007 - was advised would reduce its HSMR value.

CHKS 11 Dec 2008 press release re HSJ article (exh BJ87)

2004-Q2 2004-Q3 2004-Q4 2005-Q1 2005-Q2 2005-Q3 2005-Q4 2006-Q1 2006-Q2 2006-Q3 2006-Q4 2007-Q1 2007-Q2 2007-Q3 2007-Q4 2008-Q1 2008-Q2 2008-Q3 2008-Q4 2009-Q1 2009-Q2 2009-Q3 2009-Q4 2010-Q1 2010-Q2 2010-Q3 2010-Q4 2011-Q1 2011-Q2 2011-Q3 2011-Q4 2012-Q1 % deaths coded as palliative care HSMR Medway coding of palliative care vs HSMR 60% 50% MEDWAY HSMR 160 140 40% 120 100 England % deaths coded as palliative care 30% 20% MEDWAY %deaths Z51.5 ENGLAND % deaths Z51.5 80 60 40 Medway NHS Foundation Trust (RPA) % deaths coded as palliative care HSMR Medway NHS Foundation Trust 10% 20 0% 0

Happenings re Mid Staffs in February & March 2008 1 Feb 2008 Mid Staffs was formally awarded Foundation Trust status by Monitor 1 Feb 2008 A group of local patients told the local PCT about complaints from patients that had come to the HCC independently through its helpline 14 Feb 2008 Healthcare Commission Investigations Committee met to discuss concerns about the Trust 15 Feb 2008 Mid Staffs increased the % cases coded as palliative care and coder started coding all deaths herself. Large increase % deaths coded palliative care Z51.5 from March 2008. 18 March 2008 Healthcare Commission investigation

Healthcare Commission investigation at Mid Staffordshire NHS Foundation Trust - press release 18 March 2008 Healthcare Commission press release 18/3/08 The Healthcare Commission has also recently received a number of concerns from individual patients and relatives, about standards of care at ward level. The Healthcare Commission is therefore launching an investigation at Mid Staffordshire NHS Foundation Trust. The trust s data on outcomes for patients has also recently caused the Dr Foster Unit at Imperial College London to bring concerns to the attention of the trust and the Healthcare Commission.

2004-Q2 2004-Q3 2004-Q4 2005-Q1 2005-Q2 2005-Q3 2005-Q4 2006-Q1 2006-Q2 2006-Q3 2006-Q4 2007-Q1 2007-Q2 2007-Q3 2007-Q4 2008-Q1 2008-Q2 2008-Q3 2008-Q4 2009-Q1 2009-Q2 2009-Q3 2009-Q4 2010-Q1 2010-Q2 2010-Q3 2010-Q4 2011-Q1 2011-Q2 2011-Q3 2011-Q4 2012-Q1 % deaths coded as palliative care HSMR Mid Staffs coding of palliative care vs HSMR 40% 35% 30% 25% 160 140 120 100 England % deaths coded as palliative care 20% 15% 10% 5% 80 60 40 20 Mid Staffordshire NHS Foundation Trust (RJD) % deaths coded as palliative care HSMR Mid Staffordshire NHS Foundation Trust 0% 0

2004-Q2 2004-Q3 2004-Q4 2005-Q1 2005-Q2 2005-Q3 2005-Q4 2006-Q1 2006-Q2 2006-Q3 2006-Q4 2007-Q1 2007-Q2 2007-Q3 2007-Q4 2008-Q1 2008-Q2 2008-Q3 2008-Q4 2009-Q1 2009-Q2 2009-Q3 2009-Q4 2010-Q1 2010-Q2 2010-Q3 2010-Q4 2011-Q1 2011-Q2 2011-Q3 2011-Q4 2012-Q1 % deaths coded as palliative care HSMR Basildon & Thurrock coding of palliative care vs HSMR 40% 35% 160 140 England % deaths coded as palliative care 30% 120 25% 20% 15% 10% 5% 100 80 60 40 20 Basildon and Thurrock University Hospitals NHS Foundation Trust (RDD) % deaths coded as palliative care HSMR Basildon and Thurrock University Hospitals NHS...tion Trust 0% 0

37. Mid Staffordshire NHS Hospitals Trust HSMRs follow-up after mortality reduction programme (as at 16 Dec 2010) Healthcare Commission first report 18/03/2008

John Holden, Director of system regulation Department of Health; former head of foundation trust team. Mid Staffs Public Inquiry 14-Sep2011, Exhibit JH6 DH email 04/03/2009 Subject: Re: Mid staffs lines to take etc During the period of SHA and DH assessment (ie up to SoS support) the first Dr Foster report had only just been published, April 2007). The SHA had set in place a process for the findings to be validated and supported the trust in developing an action plan. At this time, the issue was thought to be largely one of clinical coding. It was only after further Dr Foster reports were published, from July2007, focusing on specific patient groups was it recognised (including by HCC) that there was potentially greater cause for concern. Even so the HCC did not launch its investigation until March 2008.

John Holden email 09 March 2009 exhibit JH9 DH email 09/03/2009: Subject Urgent-Mid Staffs. "Miranda, further to our discussion, the Secretary of State is tomorrow briefing the Cabinet on the issues arising at Mid Staffs. The key issue being considered is how regulation is working in the NHS, and whether the difficulties at Mid Staffs are evidence that more needs to be done. In order to brief Secretary of State we need to establish the extent to which standardised mortality rates feature in Monitor s compliance regime and application assessment process. Looking at Monitor s Board Minutes for January 2008 re consideration of Mid Staffs, it is noted that the Trust received a 127 mortality rate for2005/06 from Dr Foster. This has reduced to c. 101 between May and August 2007/08 as a result of significant improvements to coding for co-morbidities; This figure 101 is incorrect (exhibit BJ83). The HSMR at Mid Staffs May-Aug 2008 = 108.7 (96.0-122.6)

Mid Staffs Public Inquiry, 20-Sep-2011 PROFESSOR SIR BRUCE KEOGH, NHS Medical Director, Department of Health from November 2007 DH briefing to Minister on Mid staffs lines to take etc Exhibits BK49, JH6 "In April 2007, the Dr Foster s Good Hospital Guide classified the Trust as having a High Hospital Standardised Mortality Rate. (The Good Hospital Guide was first published in 2001) [DH witnesses to Mid Staffs Public Inquiry said HSMRs were only available from 2007]

HEALTHCARE COMMISSION, INVESTIGATION INTO MID STAFFORDSHIRE NHS FOUNDATION TRUST, 18 MARCH 2009 The trust assumed that the raised mortality was primarily due to poor coding of information, but could not substantiate its claim... The default position appeared to be that the deaths had been inevitable and there were no problems with the care of the patients.

Mid Staffs Public Inquiry oral evidence observed minus expected deaths 2005/6-2007/8 was not 400-1200 Approx 500, range 400 to 600, over 3 years, 95% Conf Ints Exact 492, range 391-595, over 3 years, 95% Conf Intervals 2023 mentions of mortality 1018 mentions of HSMR 458 mentions of hindsight 799 mentions of coding

Some of the problems at Mid Staffordshire. Professor Sir George Alberti. 29 April 2009 Understaffing of A&E - too few consultants, middle-grade doctors and nurses Initial patient assessment by untrained receptionists Poor supervision of junior doctors Weak leadership of nurses and inadequate nurse training Poor equipment in A&E Long delays and tendency to move patients to the Emergency Assessment Unit (EAU), Clinical Decision Unit (CDU) and assess and treat area in order to meet the 4 hour target before they had been investigated or any diagnosis made Lack of protocols and clear pathways Chaotic, large, understaffed EAU with little training for the nurses Poorly equipped EAU. Poor handover from EAU to medical and surgical wards Insufficient beds for coronary care or strokes Major delays for emergency operations Inadequate numbers of experienced surgeons with poor 24/7 cover Poor post-operative care Very poor patient care on the medical and surgical wards Inadequate handling of patient complaints

Healthcare Commission report on Mid Staffordshire NHS Hospitals Trust 18 March 2009 http://www.rcn.org.uk/ data/assets/pdf_file/0004/234976/healthcare_commission_report.pdf The SHA was not aware of any concerns regarding the quality of services provided by the trust before Dr Foster Intelligence published its Hospital Guide in April 2007. [Note: HSMRs published annually in the Dr Foster Good Hospital Guide had shown Mid Staffs to have significantly high HSMRs for 9 years before 2007/8. Mortality alerts were sent by Imperial College to the hospital and the Care Quality Commission from July 2008]

Mid Staffs Public Inquiry 26-Sep-2011 Sir Hugh Taylor's statement, ex-permanent Secretary, DH, Statement Paragraph 54: "In terms of the role of the SHA in relation to the situation at the Trust, at the time of publication of the report, my view is that it probably became too defensive, having been drawn too far into the issue of poor coding rather than seeing the mortality information as a basis for further investigation."

Bill Moyes, ex-chair Monitor Evidence to the HoC Health Select Committee, 5 March 2009 "We also relied on the extensive work that had been done by the trust itself to try and understand its high mortality rates. It was not that we did not know about them, but we were persuaded by the external scrutiny of SHAs, by the work that the SHA did, by the PCT's view that this was a coding problem and not an indicator of really seriously deficient care."

Mid Staffs Public Inquiry 20-Sep-11 PROFESSOR SIR BRUCE KEOGH, NHS Medical Director, Department of Health from November 2007 The trust..."had little focus on outcomes before the publication [of the HSMRs]..." It considered that poor coding was a likely explanation.'

Mid Staffs Public Inquiry, 2-Mar-11 DR PHILIP COATES, Consultant physician at Mid Staffs Trust from 1996, NICE guideline and Clinical Governance lead Q. We'll come on to this in a little more detail later, but you, I think, are fairly frank in your statement in saying that it was wrong to latch on to this coding explanation, rather than fully exploring whether there were clinical deficiencies. A. It was -- it was absolutely wrong to focus on it as a sole cause for the abnormal HSMR, and it was only much later that we began to realise that we had -- we should be looking at clinical quality. Q. Was Monitor wrong to accept what you were saying? A. I would have to say yes.

Mid Staffs Public Inquiry Dr Suarez, Medical Director Mid Staffs, statement, Paragraph 122 "CKS [sic] offered to look at our data to see if they could demonstrate the same high mortality figures that Dr Fosters did; however they could not. CKS identified that there were coding issues, but I believe they were very reassuring in that they said we did not have a problem with mortality."

Telegraph publication 24 April 2007 showed Mid Staffs HSMR significantly high The Department of Health asked for the following statement from them to go at the end of the publication: We would strongly advise against patients using these figures to make decisions about the relative safety of hospitals. Inquiry Counsel oral Closing Submission 9 December 2011: "What of the HSMR statistics published in April 2007? Why, we ask rhetorically, wasn't more attention paid to them? May 2007 University of Birmingham review of Imperial College methodology commissioned by West Midlands SHA.

Mid Staffs Independent Inquiry, 24 Feb 2010. Harvard external experts opinion (endorsed in Inquiry report) " the University of Birmingham reports, though probably wellintentioned, were distractions. They used the Mid Staffordshire issue as a context for discrediting the Dr. Foster methodology. the [HSMR] rates published in the Good Hospital Guide alone obviously required extremely serious consideration,

Mid Staffs Independent Inquiry, published 24 Feb 2010, pp 336 and 446 Section G Mortality statistics Harvard independent experts opinion We also agree that every statistical quality monitoring algorithm, including Dr Foster, should be critically examined by experts to determine its validity. However, we believe that in the case of Mid-Staffordshire, there were so many different warning flags from different entities, using different approaches, and over multiple time periods, that it would have been completely irresponsible not to aggressively investigate further.

Sir Bruce Keogh, Medical Director of the NHS in England, statement to the Mid Staffs Public Inquiry, para 219: "I am reasonably confident that the process which is now in place would have picked up the situation at the trust before it was granted Foundation Trust status. The HSMR information and the question in the staff survey about whether the member of staff would be happy for someone in their family to be treated at the hospital are two examples..."

Criticisms of HSMRs ranking and league tables Sir Bruce Keogh statement para 139: I have major reservations with the presentation of simplistic league tables of HSMRs (such as those presented in the 2007 Dr Foster Hospital Guide) Professor David Spiegelhalter [MSInq D135/P88/15] "I had considerable misgivings about Dr Foster's work for the Good Hospital Guide, particularly their insistence on ranking hospitals and their use of HSMR." The HSMRs were in fact published in 2007, (as Sir Bruce Keogh suggested that they should be in paragraph 133 of his statement), in alphabetical order in three bands, 'low', 'average' and 'high'.

INQUIRY REPORT PRESENTATION TO SoS BY ROBERT FRANCIS, QC http://www.midstaffsinquiry.com/assets/docs/report-presentation-by-robert-francis-qc.doc Finally and perhaps of most concern, I found a widespread culture of denial. The then Board s reaction to the HCC report was individually and collectively one of denial. It should have been one of searching self-criticism. The high HMSR at Mid Staffs, however provided compelling grounds for an investigation of the type carried out by the HCC. It became apparent throughout the inquiry that many staff during the period under investigation did express concern about the standard of care being provided to patients. The tragedy was that they were ignored. The staff evidence persuaded me that a culture of bullying and fear was prevalent in the Trust among staff. I heard of a fear of bullying being a possible explanation for more staff not coming forward with concerns. What this investigation has uncovered is a scale of failure which I believe to be greater than that revealed by the HCC report.

Monthly mortality alerts NHS hospital trusts are sent a confidential letter when an alert occurs on a series of statistical process control charts (cumulative sum analysis, CUSUM) that are run each month designed to detect a doubling, over the preceding 3 months, of the odds of death for a number of diagnosis and procedure groups that cover all IP deaths in England. Alerts are filtered to include only signals with a probability of a false alarm less than 0.1% and other restrictions are also applied to exclude some diagnoses including cancer, and vague symptoms and signs. Diagnostic procedures such as endoscopies are also excluded. The mortality alerts are copied to the Care Quality Commission (before 01 April 2009 to the Healthcare Commission) First alerts sent to a trust in May 2007 Excluding other diagnoses and cancer we currently use 43 diagnoses, 79 procedures for the alerts Our letter to the CE of a Trust notes that alerts could be due to: Poor coding Inadequate case-mix adjustment Quality of care

Monthly alerts sent to hospitals: Example diagnosis = Acute MI

Anonymised version of a monthly alert letter Sent to trust Chief Executive (copied to the CQC)

Imperial DFU (& HCC) Mortality alerts Jul to Nov 2007 Operations on jejunum Aortic, peripheral and visceral artery aneurysms Peritonitis and intestinal abscess Other circulatory disease HCC alerts Diabetes (HCC) Epilepsy and convulsions (HCC) Repair abdominal aortic aneurysm (HCC)

Mortality alert from Imperial College to CE Mid Staffs 29 June 2007

Royal College of Surgeons invited review of Mid Staffs surgical department 2007 & 2009 (Ch 18 Closing Submission 2/12/11] The Royal College of Surgeons conducted two invited reviews of the Trust, one in 2007 and one in 2009. The review team was asked to advise whether the complication rate following colorectal surgery was acceptable and whether any change of practice needed to be adopted.

Royal College of Surgeons invited review of Mid Staffs surgical department 2007 & 2009 (Ch 18 Closing Submission 2/12/11] There was reference within the [2009] report to the Trust providing care that was "grossly negligent". Notwithstanding the seriousness of that finding Mr Black gave evidence that in line with usual practice the College did not report it to the GMC. The Case Review Report refers to so many badly managed cases that it would be difficult to single out any particular surgeon. The review team found that the service provided by the general surgical unit was inadequate, unsafe and at times, dangerous.

Royal College of Surgeons invited review of Mid Staffs surgical department 2007 & 2009 (Ch 18 Closing Submission 2/12/11] Mr Black was asked about the College s failure to follow up the recommendations in his oral evidence: Q. How did you follow up that the trust had acted upon the recommendations that the reviewers made? A. I believe we wrote to them, and we never subsequently made sure that the recommendations had been carried out. Q. Do you recognise that for a member of the public listening to that, that is to say unacceptable, to put it mildly? A. I would accept that. The strong recommendations were made to the trust. The college has no statutory power to enforce those recommendations. We did follow up by letter. We did not follow it up any more than that, and I wish we had done.

Richard Hamblin, Director of Intelligence CQC, Slide 58 of PPT Institute of Health Economics 9 April 2010 as example of CQC alert

Imperial College alert for complications of surgical procedures or medical care we sent to the Healthcare Commission on 31/08/2007

The CQC website - Mortality outliers programme example (accessed 16/04/2010):- http://www.cqc.org.uk/_db/_documents/cqc_mortality_outliers_programme[1].doc Our data is therefore more specific and more timely than HSMR.

Imperial College alert for Excision of the lung we sent to the Healthcare Commission on 17/01/2009

HCC: Following up mortality 'outliers, Dec 2008 We have established a new programme of work to identify and follow up concerns about apparently high rates of mortality in the NHS. Over the 12 month period considered by this report, we have considered 85 alerts, relating to 56 acute trusts (one out of three acute trusts in England). After analysing the data, we concluded that 42 of these alerts (relating to 29 or 17% of acute trusts in England), needed to be pursued by the Commission. There is clear evidence that our follow-up action has led directly to improvements in clinical practice, and in the use of clinical data in NHS trusts. We expect this to have led to direct improvements in the care provided to patients, but acknowledge that further evaluation is necessary to understand the extent to which this may have happened.

50. Example for Royal Bolton: #nof Apr 2004 Aug 2008 (1899 admissions) Intervention reduce adm to op time & appoint orthopaedic geriatrician

2004-Q3 2004-Q4 2005-Q1 2005-Q2 2005-Q3 2005-Q4 2006-Q1 2006-Q2 2006-Q3 2006-Q4 2007-Q1 2007-Q2 2007-Q3 2007-Q4 2008-Q1 2008-Q2 2008-Q3 2008-Q4 2009-Q1 2009-Q2 2009-Q3 2009-Q4 2010-Q1 2010-Q2 2010-Q3 Crude death rate 51. Royal Bolton: fractured neck of femur Apr 2004 Jul 2010 Reduction of crude death rate after intervention Intervention 30% 25% 20% Crude death rate 15% 10% 5% 0%

2004-Q3 2004-Q4 2005-Q1 2005-Q2 2005-Q3 2005-Q4 2006-Q1 2006-Q2 2006-Q3 2006-Q4 2007-Q1 2007-Q2 2007-Q3 2007-Q4 2008-Q1 2008-Q2 2008-Q3 2008-Q4 2009-Q1 2009-Q2 2009-Q3 2009-Q4 2010-Q1 2010-Q2 2010-Q3 Cumulative sum obs - exp deaths 52. Royal Bolton: #nof Apr 2004 Jul 2010 Cumulative sum of actual deaths expected deaths if 2004-Q3 SMR had persisted Intervention 0-20 -40-60 -80-100 -120-140 -160-180 Cusum of Observed -Expected deaths if 2004-Q3 SMR had persisted -200

2004-Q2 2004-Q3 2004-Q4 2005-Q1 2005-Q2 2005-Q3 2005-Q4 2006-Q1 2006-Q2 2006-Q3 2006-Q4 2007-Q1 2007-Q2 2007-Q3 2007-Q4 2008-Q1 2008-Q2 2008-Q3 2008-Q4 2009-Q1 2009-Q2 2009-Q3 2009-Q4 2010-Q1 2010-Q2 2010-Q3 % LoS values > 75th %ile for England (for #nof) 53. Change of length of stay (leads to reduced costs % LoS values > 75th %ile for England (for #nof) Intervention 35 30 25 20 15 10 5 0 Long LoS = % LoS values greater than 75th percentile patient in England for #nof Linear (Long LoS = % LoS values greater than 75th percentile patient in England for #nof)

Fig 3 Hospital standardised mortality rates (HSMR) in acute hospitals in England, 2007-8, and change in position of North West London Hospitals NHS Trust (NWLH) Robb, E. et al. BMJ 2010;340:c1234 Copyright 2010 BMJ Publishing Group Ltd.

Hero of the Hudson Capt. Chesley Sullenberger s From 1967 to 1976, the chances of dying in a large U.S. jetliner crash were roughly 1 in 2 million. Today it s closer to 1 in 20 million thanks to tightened safety standards and a massive culture change in the industry...

Hero of the Hudson Capt. Chesley Sullenberger s We still think of these [patient deaths] as an unavoidable consequence of providing care and sometimes that s true, but not always, Sullenberger said. We must stop thinking about them as unavoidable and start thinking about them as unthinkable. www.ucsf.edu/news/2012/11/13085/herohudson-applies-lessons-aviation-improvepatient-safety

Paediatric cardiac surgical mortality in England after Bristol: BMJ 2004; 329 : 7 October 2004

Paediatric cardiac surgical mortality in England after Bristol: BMJ 2004; 329 : 7 October 2004 External inspection Intervention

Paediatric Cardiac Surgery in England age 0-5 From 1991/95 to 1999/02 PCS mortality at Bristol reduced from 29% to 3.5% ie 8 times From 1991/95 to 2008/12 PCS mortality in England reduced from 12% to 2% ie 6 times From 1967 to 1976, mortality in large U.S. jetliner crashes reduced from 1 in 2 million to 1 in 20 million i.e. 10 times.

Nobel prizes vs chocolate consumption - correlation