Data Quality and Clinical Coding for Improvement What happens when the data are wrong? The key responsibilities for Clinicians and Managers

Transcription

1 Data Quality and Clinical Coding for Improvement Data Quality and Clinical Coding for Improvement What happens when the data are wrong? The key responsibilities for Clinicians and Managers 19 th November :45 am Hallam Conference Centre Brian Jarman Imperial College,

2 Subjects I have been asked to cover learning from Mid Staffordshire clinical coding: understanding the data and what the data is telling you how Dr Foster use the data to identify outliers and produce alerts case studies and examples in practice

3 Mid Staffs Independent Inquiry, 24 Feb 2010, p366, 46 Furthermore, the maintenance, certification and submission of accurately recorded data to the regulatory authorities are ultimately the responsibility of Trust leadership.

4 Data covered Hospital Standardised Mortality Ratios (HSMRs) methodology from Dr Foster unit (DFU) at Imperial College HSMR published annually from 2001 by Dr Foster Intelligence (DFI) main national press Data Quality Summary from DFI Monthly mortality alerts - used the Dr Foster Unit (DFU) at Imperial College to identify outliers and produce alerts

5 Variables for HSMR logistic regression adjustment Age group (<1, then 5-year bands to 90+) Sex Admission method/type (emergency, elective etc) Admission source (home, transfers etc) Deprivation quintile (based on postcode) Diagnosis subgroup (CCS sub-groups within each CCS group) Comorbidity (Charlson score) Emergency admissions in previous 12 months Palliative care (any episode that has a treatment function code 315 or any Z515 ICD10 diagnosis code) Month of admission Year of discharge (Day cases are excluded from the risk models ~ 70 deaths/year)

6 Data Quality Summary DFU monthly checks on reords: Duplicate record Admission/Appointment date Episode order Admission method/appointment type Sex Date of birth Provider Episode start Episode end Episode duration Postcode NHS Number GP Practice Local patient ID Age Day case Deprivation Provider + Local patient ID Consultant Ethnicity Elective/Referral date Discharge/Outcome Diagnosis Procedure Procedure date Spell overlaps Missing first episode Missing last episode Inconsistent bed-days/los Age group Sex (spell) Admission (spell) Episode order (spell) Invalid LOS Invalid HRG

7 Effect of coding of variables in model Direct vs indirect standardisation Comorbidity (Charlson Index) Palliative care Admission vs patient All in-hospital deaths vs 30-days inhospital deaths HSMR covering 80% deaths (56 CCS) vs HSMR all IP deaths (259 CCS gps)

8 Indirectly standardised HSMR Comparison of indirect vs direct standardisation 2005 HSMR (simple model) 150% 140% 130% 120% 110% 100% Trusts Identical match 25% more than directly standardised 10% more than directly standardised 10% less than directly standardised 25% less directly standardised 90% 80% 70% 60% 60% 70% 80% 90% 100% 110% 120% 130% 140% 150% Directly standardised HSMR

9 HSMR without Charlson Charlson co-morbidity index Comparison of HSMR calculated with and without Charlson English Acute Trusts 2008/ y = x R² = Salford Royal NHS Foundation Trust Trusts Identical match 25% more than standard HSMR 10% more than standard HSMR 10% less than standard HSMR 25% less than standard HSMR HSMR

10 HSMR excluding paliative care Palliative care y = x R² = Comparison of HSMR calculated using adjustment or exclusion to account for palliative care English Acute Trusts 2008/9 Trusts Identical match 25% more than standard HSMR 10% more than standard HSMR 10% less than standard HSMR 25% less than standard HSMR Medway NHS Trust 60 Walsall Hospitals NHS Trust HSMR adjusting for palliative care

11 HSMR based on last admission in financial year Admissions versus Patients y = x R² = Comparison of HSMR calculated using admissions and based on last admission in financial year English Acute Trusts 2008/9 Trusts Identical match 25% more than standard HSMR 10% more than standard HSMR 10% less than standard HSMR 90 Trafford Healthcare NHS Trust 25% less than standard HSMR HSMR

12 HSMR in-hospital 30 day deaths HSMR (all in-hospital deaths) vs. all in-hospital 30 day deaths y = x R² = Comparison of HSMR calculated using 30 day in-hospital deaths with HSMR using all in-hospital eaths English Acute Trusts 2008/9 130 Trusts Identical match 25% more than in-hospital deaths HSMR 10% more than in-hospital deaths HSMR 10% less than in-hospital deaths HSMR 25% less than in-hospital deaths HSMR Linear (Trusts) HSMR (all in-hospital deaths)

13 HSMR all 30 day deaths HSMR 30 day in-hospital deaths vs 30 day all deaths y = x R² = Comparison of HSMR calculated using 30 day in-hospital deaths with HSMR using all 30 day deaths English Acute Trusts 2004/5 120 Trusts Identical match Bradford Teaching Hospitals NHS Foundation Trust 25% more than in-hospital deaths HSMR 10% more than in-hospital deaths HSMR 10% less than in-hospital deaths HSMR 25% less than in-hospital deaths HSMR HSMR in-hospital 30 day deaths

14 HSMR based on all diagnosis groups 56 diagnosis group HSMR vs. all 259 CCS diagnoses Comparison of HSMR calculated using 56 diagnoses compared with all 259 diagnoses English Acute Trusts 2008/ y = x R² = Trusts Identical match 25% more than standard HSMR 10% more than standard HSMR 10% less than standard HSMR 25% less than standard HSMR Linear (Trusts) HSMR

15 The changes of palliative care coding by CfH rules. Changes at Medway, Mid Staffs and Basildon = effect on HSMRs March Medway From March 2007 CfH Coding Clinic permitted non-curable illnesses to be coded as palliative care secondary diagnosis Z51.5 (not just terminal illnesses under consultant) Imperial College was persuaded to introduce a palliative care adjustment Medway increased % deaths coded Z51.5 from <1% to 37% and HSMR value decreased (advised by CHKS). Only trust in England at that time to make those changes CHKS, Dec 2008: Our review of coding at Medway NHS Foundation Trust found that end-of-life care was recorded for only eight per cent of deaths, when the actual proportion should have been 37 per cent. We believe that the hospital mortality index should exclude these cases and, by making this adjustment, Medway's mortality index reduced by just over a third. March Mid Staffs Mid Staffs made similar changes and HSMR reduced by ~7% Two other West Midlands SHA trusts made similar changes later in 2008 We publicised the effect of Z51.5 changes on HSMR values ( doesn t save lives ) Mid Basildon & Thurrock Made similar changes to starting mid 2009 June 2010 CfH Coding Clinic reverted to ~pre 2007

16 Changes in coding at Medway, Mid Staffs and Basildon Mid Staffs and Basildon In 2005/06 had the highest HSMR (lower 95% CI above 100) In 2007 had the largest number of mortality alerts from Imperial College Had had significantly high HSMRs for about 10 years First major palliative care coding change was at Medway NHS Foundation Trust in was advised would reduce its HSMR value.

17 CHKS 11 Dec 2008 press release re HSJ article (exh BJ87)

18 2004-Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q1 % deaths coded as palliative care HSMR Medway coding of palliative care vs HSMR 60% 50% MEDWAY HSMR % England % deaths coded as palliative care 30% 20% MEDWAY %deaths Z51.5 ENGLAND % deaths Z Medway NHS Foundation Trust (RPA) % deaths coded as palliative care HSMR Medway NHS Foundation Trust 10% 20 0% 0

19 Happenings re Mid Staffs in February & March Feb 2008 Mid Staffs was formally awarded Foundation Trust status by Monitor 1 Feb 2008 A group of local patients told the local PCT about complaints from patients that had come to the HCC independently through its helpline 14 Feb 2008 Healthcare Commission Investigations Committee met to discuss concerns about the Trust 15 Feb 2008 Mid Staffs increased the % cases coded as palliative care and coder started coding all deaths herself. Large increase % deaths coded palliative care Z51.5 from March March 2008 Healthcare Commission investigation

20 Healthcare Commission investigation at Mid Staffordshire NHS Foundation Trust - press release 18 March 2008 Healthcare Commission press release 18/3/08 The Healthcare Commission has also recently received a number of concerns from individual patients and relatives, about standards of care at ward level. The Healthcare Commission is therefore launching an investigation at Mid Staffordshire NHS Foundation Trust. The trust s data on outcomes for patients has also recently caused the Dr Foster Unit at Imperial College London to bring concerns to the attention of the trust and the Healthcare Commission.

21 2004-Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q1 % deaths coded as palliative care HSMR Mid Staffs coding of palliative care vs HSMR 40% 35% 30% 25% England % deaths coded as palliative care 20% 15% 10% 5% Mid Staffordshire NHS Foundation Trust (RJD) % deaths coded as palliative care HSMR Mid Staffordshire NHS Foundation Trust 0% 0

22 2004-Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q1 % deaths coded as palliative care HSMR Basildon & Thurrock coding of palliative care vs HSMR 40% 35% England % deaths coded as palliative care 30% % 20% 15% 10% 5% Basildon and Thurrock University Hospitals NHS Foundation Trust (RDD) % deaths coded as palliative care HSMR Basildon and Thurrock University Hospitals NHS...tion Trust 0% 0

23 37. Mid Staffordshire NHS Hospitals Trust HSMRs follow-up after mortality reduction programme (as at 16 Dec 2010) Healthcare Commission first report 18/03/2008

24 John Holden, Director of system regulation Department of Health; former head of foundation trust team. Mid Staffs Public Inquiry 14-Sep2011, Exhibit JH6 DH 04/03/2009 Subject: Re: Mid staffs lines to take etc During the period of SHA and DH assessment (ie up to SoS support) the first Dr Foster report had only just been published, April 2007). The SHA had set in place a process for the findings to be validated and supported the trust in developing an action plan. At this time, the issue was thought to be largely one of clinical coding. It was only after further Dr Foster reports were published, from July2007, focusing on specific patient groups was it recognised (including by HCC) that there was potentially greater cause for concern. Even so the HCC did not launch its investigation until March 2008.

25 John Holden 09 March 2009 exhibit JH9 DH 09/03/2009: Subject Urgent-Mid Staffs. "Miranda, further to our discussion, the Secretary of State is tomorrow briefing the Cabinet on the issues arising at Mid Staffs. The key issue being considered is how regulation is working in the NHS, and whether the difficulties at Mid Staffs are evidence that more needs to be done. In order to brief Secretary of State we need to establish the extent to which standardised mortality rates feature in Monitor s compliance regime and application assessment process. Looking at Monitor s Board Minutes for January 2008 re consideration of Mid Staffs, it is noted that the Trust received a 127 mortality rate for2005/06 from Dr Foster. This has reduced to c. 101 between May and August 2007/08 as a result of significant improvements to coding for co-morbidities; This figure 101 is incorrect (exhibit BJ83). The HSMR at Mid Staffs May-Aug 2008 = ( )

26 Mid Staffs Public Inquiry, 20-Sep-2011 PROFESSOR SIR BRUCE KEOGH, NHS Medical Director, Department of Health from November 2007 DH briefing to Minister on Mid staffs lines to take etc Exhibits BK49, JH6 "In April 2007, the Dr Foster s Good Hospital Guide classified the Trust as having a High Hospital Standardised Mortality Rate. (The Good Hospital Guide was first published in 2001) [DH witnesses to Mid Staffs Public Inquiry said HSMRs were only available from 2007]

27 HEALTHCARE COMMISSION, INVESTIGATION INTO MID STAFFORDSHIRE NHS FOUNDATION TRUST, 18 MARCH 2009 The trust assumed that the raised mortality was primarily due to poor coding of information, but could not substantiate its claim... The default position appeared to be that the deaths had been inevitable and there were no problems with the care of the patients.

28 Mid Staffs Public Inquiry oral evidence observed minus expected deaths 2005/6-2007/8 was not Approx 500, range 400 to 600, over 3 years, 95% Conf Ints Exact 492, range , over 3 years, 95% Conf Intervals 2023 mentions of mortality 1018 mentions of HSMR 458 mentions of hindsight 799 mentions of coding

29 Some of the problems at Mid Staffordshire. Professor Sir George Alberti. 29 April 2009 Understaffing of A&E - too few consultants, middle-grade doctors and nurses Initial patient assessment by untrained receptionists Poor supervision of junior doctors Weak leadership of nurses and inadequate nurse training Poor equipment in A&E Long delays and tendency to move patients to the Emergency Assessment Unit (EAU), Clinical Decision Unit (CDU) and assess and treat area in order to meet the 4 hour target before they had been investigated or any diagnosis made Lack of protocols and clear pathways Chaotic, large, understaffed EAU with little training for the nurses Poorly equipped EAU. Poor handover from EAU to medical and surgical wards Insufficient beds for coronary care or strokes Major delays for emergency operations Inadequate numbers of experienced surgeons with poor 24/7 cover Poor post-operative care Very poor patient care on the medical and surgical wards Inadequate handling of patient complaints

30 Healthcare Commission report on Mid Staffordshire NHS Hospitals Trust 18 March data/assets/pdf_file/0004/234976/healthcare_commission_report.pdf The SHA was not aware of any concerns regarding the quality of services provided by the trust before Dr Foster Intelligence published its Hospital Guide in April [Note: HSMRs published annually in the Dr Foster Good Hospital Guide had shown Mid Staffs to have significantly high HSMRs for 9 years before 2007/8. Mortality alerts were sent by Imperial College to the hospital and the Care Quality Commission from July 2008]

31 Mid Staffs Public Inquiry 26-Sep-2011 Sir Hugh Taylor's statement, ex-permanent Secretary, DH, Statement Paragraph 54: "In terms of the role of the SHA in relation to the situation at the Trust, at the time of publication of the report, my view is that it probably became too defensive, having been drawn too far into the issue of poor coding rather than seeing the mortality information as a basis for further investigation."

32 Bill Moyes, ex-chair Monitor Evidence to the HoC Health Select Committee, 5 March 2009 "We also relied on the extensive work that had been done by the trust itself to try and understand its high mortality rates. It was not that we did not know about them, but we were persuaded by the external scrutiny of SHAs, by the work that the SHA did, by the PCT's view that this was a coding problem and not an indicator of really seriously deficient care."

33 Mid Staffs Public Inquiry 20-Sep-11 PROFESSOR SIR BRUCE KEOGH, NHS Medical Director, Department of Health from November 2007 The trust..."had little focus on outcomes before the publication [of the HSMRs]..." It considered that poor coding was a likely explanation.'

34 Mid Staffs Public Inquiry, 2-Mar-11 DR PHILIP COATES, Consultant physician at Mid Staffs Trust from 1996, NICE guideline and Clinical Governance lead Q. We'll come on to this in a little more detail later, but you, I think, are fairly frank in your statement in saying that it was wrong to latch on to this coding explanation, rather than fully exploring whether there were clinical deficiencies. A. It was -- it was absolutely wrong to focus on it as a sole cause for the abnormal HSMR, and it was only much later that we began to realise that we had -- we should be looking at clinical quality. Q. Was Monitor wrong to accept what you were saying? A. I would have to say yes.

35 Mid Staffs Public Inquiry Dr Suarez, Medical Director Mid Staffs, statement, Paragraph 122 "CKS [sic] offered to look at our data to see if they could demonstrate the same high mortality figures that Dr Fosters did; however they could not. CKS identified that there were coding issues, but I believe they were very reassuring in that they said we did not have a problem with mortality."

36 Telegraph publication 24 April 2007 showed Mid Staffs HSMR significantly high The Department of Health asked for the following statement from them to go at the end of the publication: We would strongly advise against patients using these figures to make decisions about the relative safety of hospitals. Inquiry Counsel oral Closing Submission 9 December 2011: "What of the HSMR statistics published in April 2007? Why, we ask rhetorically, wasn't more attention paid to them? May 2007 University of Birmingham review of Imperial College methodology commissioned by West Midlands SHA.

37 Mid Staffs Independent Inquiry, 24 Feb Harvard external experts opinion (endorsed in Inquiry report) " the University of Birmingham reports, though probably wellintentioned, were distractions. They used the Mid Staffordshire issue as a context for discrediting the Dr. Foster methodology. the [HSMR] rates published in the Good Hospital Guide alone obviously required extremely serious consideration,

38 Mid Staffs Independent Inquiry, published 24 Feb 2010, pp 336 and 446 Section G Mortality statistics Harvard independent experts opinion We also agree that every statistical quality monitoring algorithm, including Dr Foster, should be critically examined by experts to determine its validity. However, we believe that in the case of Mid-Staffordshire, there were so many different warning flags from different entities, using different approaches, and over multiple time periods, that it would have been completely irresponsible not to aggressively investigate further.

39 Sir Bruce Keogh, Medical Director of the NHS in England, statement to the Mid Staffs Public Inquiry, para 219: "I am reasonably confident that the process which is now in place would have picked up the situation at the trust before it was granted Foundation Trust status. The HSMR information and the question in the staff survey about whether the member of staff would be happy for someone in their family to be treated at the hospital are two examples..."

40 Criticisms of HSMRs ranking and league tables Sir Bruce Keogh statement para 139: I have major reservations with the presentation of simplistic league tables of HSMRs (such as those presented in the 2007 Dr Foster Hospital Guide) Professor David Spiegelhalter [MSInq D135/P88/15] "I had considerable misgivings about Dr Foster's work for the Good Hospital Guide, particularly their insistence on ranking hospitals and their use of HSMR." The HSMRs were in fact published in 2007, (as Sir Bruce Keogh suggested that they should be in paragraph 133 of his statement), in alphabetical order in three bands, 'low', 'average' and 'high'.

41

42 INQUIRY REPORT PRESENTATION TO SoS BY ROBERT FRANCIS, QC Finally and perhaps of most concern, I found a widespread culture of denial. The then Board s reaction to the HCC report was individually and collectively one of denial. It should have been one of searching self-criticism. The high HMSR at Mid Staffs, however provided compelling grounds for an investigation of the type carried out by the HCC. It became apparent throughout the inquiry that many staff during the period under investigation did express concern about the standard of care being provided to patients. The tragedy was that they were ignored. The staff evidence persuaded me that a culture of bullying and fear was prevalent in the Trust among staff. I heard of a fear of bullying being a possible explanation for more staff not coming forward with concerns. What this investigation has uncovered is a scale of failure which I believe to be greater than that revealed by the HCC report.

43 Monthly mortality alerts NHS hospital trusts are sent a confidential letter when an alert occurs on a series of statistical process control charts (cumulative sum analysis, CUSUM) that are run each month designed to detect a doubling, over the preceding 3 months, of the odds of death for a number of diagnosis and procedure groups that cover all IP deaths in England. Alerts are filtered to include only signals with a probability of a false alarm less than 0.1% and other restrictions are also applied to exclude some diagnoses including cancer, and vague symptoms and signs. Diagnostic procedures such as endoscopies are also excluded. The mortality alerts are copied to the Care Quality Commission (before 01 April 2009 to the Healthcare Commission) First alerts sent to a trust in May 2007 Excluding other diagnoses and cancer we currently use 43 diagnoses, 79 procedures for the alerts Our letter to the CE of a Trust notes that alerts could be due to: Poor coding Inadequate case-mix adjustment Quality of care

44 Monthly alerts sent to hospitals: Example diagnosis = Acute MI

45 Anonymised version of a monthly alert letter Sent to trust Chief Executive (copied to the CQC)

46 Imperial DFU (& HCC) Mortality alerts Jul to Nov 2007 Operations on jejunum Aortic, peripheral and visceral artery aneurysms Peritonitis and intestinal abscess Other circulatory disease HCC alerts Diabetes (HCC) Epilepsy and convulsions (HCC) Repair abdominal aortic aneurysm (HCC)

47 Mortality alert from Imperial College to CE Mid Staffs 29 June 2007

48 Royal College of Surgeons invited review of Mid Staffs surgical department 2007 & 2009 (Ch 18 Closing Submission 2/12/11] The Royal College of Surgeons conducted two invited reviews of the Trust, one in 2007 and one in The review team was asked to advise whether the complication rate following colorectal surgery was acceptable and whether any change of practice needed to be adopted.

49 Royal College of Surgeons invited review of Mid Staffs surgical department 2007 & 2009 (Ch 18 Closing Submission 2/12/11] There was reference within the [2009] report to the Trust providing care that was "grossly negligent". Notwithstanding the seriousness of that finding Mr Black gave evidence that in line with usual practice the College did not report it to the GMC. The Case Review Report refers to so many badly managed cases that it would be difficult to single out any particular surgeon. The review team found that the service provided by the general surgical unit was inadequate, unsafe and at times, dangerous.

50 Royal College of Surgeons invited review of Mid Staffs surgical department 2007 & 2009 (Ch 18 Closing Submission 2/12/11] Mr Black was asked about the College s failure to follow up the recommendations in his oral evidence: Q. How did you follow up that the trust had acted upon the recommendations that the reviewers made? A. I believe we wrote to them, and we never subsequently made sure that the recommendations had been carried out. Q. Do you recognise that for a member of the public listening to that, that is to say unacceptable, to put it mildly? A. I would accept that. The strong recommendations were made to the trust. The college has no statutory power to enforce those recommendations. We did follow up by letter. We did not follow it up any more than that, and I wish we had done.

51 Richard Hamblin, Director of Intelligence CQC, Slide 58 of PPT Institute of Health Economics 9 April 2010 as example of CQC alert

52 Imperial College alert for complications of surgical procedures or medical care we sent to the Healthcare Commission on 31/08/2007

53 The CQC website - Mortality outliers programme example (accessed 16/04/2010):- Our data is therefore more specific and more timely than HSMR.

54 Imperial College alert for Excision of the lung we sent to the Healthcare Commission on 17/01/2009

55 HCC: Following up mortality 'outliers, Dec 2008 We have established a new programme of work to identify and follow up concerns about apparently high rates of mortality in the NHS. Over the 12 month period considered by this report, we have considered 85 alerts, relating to 56 acute trusts (one out of three acute trusts in England). After analysing the data, we concluded that 42 of these alerts (relating to 29 or 17% of acute trusts in England), needed to be pursued by the Commission. There is clear evidence that our follow-up action has led directly to improvements in clinical practice, and in the use of clinical data in NHS trusts. We expect this to have led to direct improvements in the care provided to patients, but acknowledge that further evaluation is necessary to understand the extent to which this may have happened.

56 50. Example for Royal Bolton: #nof Apr 2004 Aug 2008 (1899 admissions) Intervention reduce adm to op time & appoint orthopaedic geriatrician

57 2004-Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q3 Crude death rate 51. Royal Bolton: fractured neck of femur Apr 2004 Jul 2010 Reduction of crude death rate after intervention Intervention 30% 25% 20% Crude death rate 15% 10% 5% 0%

58 2004-Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q3 Cumulative sum obs - exp deaths 52. Royal Bolton: #nof Apr 2004 Jul 2010 Cumulative sum of actual deaths expected deaths if 2004-Q3 SMR had persisted Intervention Cusum of Observed -Expected deaths if 2004-Q3 SMR had persisted -200

59 2004-Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q3 % LoS values > 75th %ile for England (for #nof) 53. Change of length of stay (leads to reduced costs % LoS values > 75th %ile for England (for #nof) Intervention Long LoS = % LoS values greater than 75th percentile patient in England for #nof Linear (Long LoS = % LoS values greater than 75th percentile patient in England for #nof)

60 Fig 3 Hospital standardised mortality rates (HSMR) in acute hospitals in England, , and change in position of North West London Hospitals NHS Trust (NWLH) Robb, E. et al. BMJ 2010;340:c1234 Copyright 2010 BMJ Publishing Group Ltd.

61 Hero of the Hudson Capt. Chesley Sullenberger s From 1967 to 1976, the chances of dying in a large U.S. jetliner crash were roughly 1 in 2 million. Today it s closer to 1 in 20 million thanks to tightened safety standards and a massive culture change in the industry...

62 Hero of the Hudson Capt. Chesley Sullenberger s We still think of these [patient deaths] as an unavoidable consequence of providing care and sometimes that s true, but not always, Sullenberger said. We must stop thinking about them as unavoidable and start thinking about them as unthinkable.

63 Paediatric cardiac surgical mortality in England after Bristol: BMJ 2004; 329 : 7 October 2004

64 Paediatric cardiac surgical mortality in England after Bristol: BMJ 2004; 329 : 7 October 2004 External inspection Intervention

65 Paediatric Cardiac Surgery in England age 0-5 From 1991/95 to 1999/02 PCS mortality at Bristol reduced from 29% to 3.5% ie 8 times From 1991/95 to 2008/12 PCS mortality in England reduced from 12% to 2% ie 6 times From 1967 to 1976, mortality in large U.S. jetliner crashes reduced from 1 in 2 million to 1 in 20 million i.e. 10 times.

66 Nobel prizes vs chocolate consumption - correlation