Case study: how reliable are our healthcare systems?

Case study: how reliable are our healthcare systems? CMSSQ Centre for Medication Safety & Service Quality Professor Bryony Dean Franklin Centre for Medication Safety and Service Quality Imperial College Healthcare NHS Trust / UCL School of Pharmacy

Questions Would you still use email if 1 in 7 messages did not meet the recipient? Would you accept your luggage being lost on 1 in 4 flights? How would you feel if your car didn t start one day per week?

Reliability What do we mean by reliability of healthcare systems? Is reliability important?

What is reliability? Reliability is the probability of a component, or system, functioning correctly over a given period of time under a given set of operating conditions Where functioning correctly implies a given specification

What is reliability? Failure free operation over time (Resar, 2006) It works every time

Is reliability important?

Is reliability important? YES... Outcomes based studies Preventable adverse events affect 1.5-4.4% inpatients Process based studies Prescribing errors in 1.5 to 15% inpatient medication orders, or 5% items prescribed in general practice Patients received scientifically recommended care in only 55% of cases (McGlynn et al., 2003) Quality indicators for melanoma adherence from 11.8 to 96.5% at patient level (Bilimoria et al, 2009) Patient experience

Is reliability important? BUT... Reliability contributes to safety, but is concerned only with the probability of failure (not the severity of its consequences) Sometimes there is no standardised specification The specification may be inappropriate due to individual patient needs Resilience also important being able to identify and rectify system failures

Objectives To identify and describe a selection of common but important processes within healthcare in which to study defects in reliability To explore the extent, nature and any variation between and within organisations To identify the systems factors involved in poor reliability To make recommendations for improving system reliability

5 Topics Providing information at the point of clinical decision making Prescribing for hospital inpatients Providing operating theatre equipment Providing equipment for the insertion of intravenous lines Handover within acute medicine

General approach Each topic studied in three organisations Documentation of processes involved using process mapping Measurement of reliability using quantitative data collection for each topic Exploration of causes of poor reliability using qualitative interviews and framework analysis

Overview of findings The five processes studied had overall reliability of 81-87% However, significant variation between organisations for most topics Reliability ranged from 63% for equipment availability in organisation D, to 96% for availability of clinical information in organisation A

Prescribing for hospital inpatients

Methods Selected medical admissions and surgical wards in each of 3 organisations Ward pharmacists collected data on inpatient and discharge items screened each day, and any prescribing errors identified in these Standard data collection form Verbal and written briefing for data collectors

Prescribing for inpatients 1,289 drug chart screenings 6,237 newly written medication orders across 10 wards on 3 sites. 368 omissions, giving denominator of 6,605 Data related to estimated 1,771 patient days. 1,025 errors in 974 of 6,605 medication orders (14.7%; 95% CI 13.8 to 15.6%). Reliability of 85.3% An estimated 58 errors per 100 patient days Mean clinical importance = 5.3 (0 to 10 scale)

Variation between specialties Specialty Erroneous orders Admissions 16.3% (CI: 15.2 to 17.4%) Surgery 12.2% (CI: 10.9 to 13.5%) Errors per 100 patient days 80 37 TOTAL 14.7% 58

Prescribing errors identified as a percentage of all medication orders Variation between specialties 9.0% 8.0% 7.0% 6.0% 5.0% 4.0% Admissions Surgical 3.0% 2.0% 1.0% 0.0% Omission No Indication Duplication Incomplete Prescription Allergic Choice of drug Inappropriate dose Inappropriate frequency or dosing schedule Incorrect route Incorrect formulation Inappropriate abbreviation Illegible No Instructions for administration Unknown

Variation among organisations Site Erroneous orders (95% confidence interval) Errors per 100 patient days A 13.6% (12.3 to 14.9%) 53 B 12.2% (10.7 to 13.7%) 47 C 18.4% (16.7 to 20.1%) 74 TOTAL 14.7% (13.8 to 15.6%) 58

Incidence of this error type (% of all medication orders) Variation among organisations 8.0% 7.0% 6.0% 5.0% 4.0% 3.0% Site A Site B Site C 2.0% 1.0% 0.0% Omission No Indication Duplication Incomplete Prescription Allergic Choice of drug Inappropriate dose Inappropriate frequency or dosing schedule Incorrect route Incorrect formulation Inappropriate abbreviation Illegible No Instructions for administration Unknown

Multivariate analysis After taking into account the effect of medication history taking (p<0.001), there were still significant differences between: Wards (p=0.01) Sites (p<0.001) (but not specialty) Suggests that there is an impact of local practice / teams / culture

Resilience Safety I when nothing goes wrong Safety II The ability of a system to sustain required operations under both expected and unexpected conditions (Erik Hollnagel, 2011)

Defences Pharmacists recorded the number of doses given before the error was corrected for 904 of 1,025 errors (88.2%). For these 904 errors: a mean of 0.9 doses were given (or omitted) before the error was corrected (range 0-11) In 57.7% of cases, the error was corrected before any doses administered Overall, 69.4% of errors resulted in intervention by the pharmacist.

Systems factors Lack of feedback to doctors about errors...sometimes that phenytoin prescription is written by Doctor X, Doctor X has gone home so I have to go to Doctor Y and get them to change it and that s fine, they learn something new, but Doctor X who wrote the prescription doesn t know anything about it. Pharmacist 3, organisation A

Systems factors Variation among doctors in how certain drugs prescribed Drugs to be stopped on a future date Drugs given less often than once daily Drug chart design Site B no box for maximum dose of drugs prescribed to be given when required

Systems factors Lack of information from primary care about patients medication histories It s Chinese whispers, isn t it? I get given a list of drugs from the GP or from the ambulance man who s copied it down from the daughter, he gives it to me and then I copy it down. And then [...] we change it and I write it down, then I write a letter [...] back to the GP Doctor 3, organisation C

Reliability of equipment availability in operating theatres

Data Collection Data collection form designed for theatre staff to complete after each procedure. Scrub nurses and surgeons asked to complete the forms after each procedure. Theatres from following specialties recruited from each site: trauma, orthopaedics, general surgery and paediatric surgery

Overall Results Site Total operatio ns studied Number of operations with equipment problems Number of equipment problems Percentage operations with one or more equipment problems A 258 50 56 19% D 67 25 28 37% F 165 19 19 12% Total 490 94 103 19% Reliability = 81%

Equipment problem scenario During an orthopaedic knee ligament repair, the special instrument used to harvest tendon for the repair was faulty. No other replacement was available - the surgeon had to work without the instrument. Surgery became technically difficult leading to increased duration of procedure and also the tendon harvested was inappropriate which could impact on patient s quality of life and cause failure of repair.

we always need a colposcope with that list and time and time again it isn t there or it s broken or it isn t back or nobody knows where it is Because the individual sites do tend to work slightly different, and one of our sites was a merger only two years ago, so they re very used to working on that site, and almost make the expectation when they go to one of the other sites it s going to be exactly the same, and we haven t got that consistency across the board in all specialities yet.

Say you open a tray and the diathermy lead s there but with no forceps, you can t use it. You cannot use the diathermy without the forceps, they go together. But on the outside it ll say diathermy forceps missing. What you want is a system in place that if the diathermy forceps are damaged or missing,replace them, set out, it s simple to me.

Potential impact for the NHS Around 8 million operations are performed per year in the UK (data from Hospital Episode Statistics) If 19% of operations affected by equipment problems, this equates to 1,520,000 operations per year About half of these delayed to some extent

Common Factors Lack of feedback mechanisms, both for individuals and systems Lack of standardisation Poor communication, both written and verbal A perception of over complexity of processes Staff come to accept poor reliability as normal, thus not reporting problems Lack of ownership of issues

Recommendations Improve feedback mechanisms. Standardisation (with customisation where needed). Standardising and improving communication. Developing a culture of challenge. Encouraging a sense of ownership. Ongoing measurement needed for key areas

Conclusions Routine systems that support clinical care have low reliability Prevalence of problems likely to be even higher than reported here, due to under-reporting Systems failed on 13-19% of occasions

Questions Would you still use email if 1 in 7 messages did not meet the recipient? Would you accept your luggage being lost on 1 in 4 flights? Would you be happy if your car didn t start one day per week?

Acknowledgment Material used in this presentation is based on a Research Report commissioned in 2009 by the Health Foundation (registered charity number 286987)