A Workshop for the SPSP Fellows. Better Quality Through Better Measurement: Worksheets. Robert Lloyd, PhD Executive Director Performance Improvement

A Workshop for the SPSP Fellows Better Quality Through Better Measurement: Worksheets Robert Lloyd, PhD Executive Director Performance Improvement 4April 2014 Paris

Measurement Plan Worksheet 1. 2. 3. 4. 5. 6. 7. 8. Measure Name Type (Process, Outcome or Balancing) Operational Definition Source: R. Lloyd 2010

Operational Definition Worksheet Team name: Date: Contact person: WHAT PROCESS DID YOU SELECT? WHAT SPECIFIC MEASURE DID YOU SELECT FOR THIS PROCESS? OPERATIONAL DEFINITION Define the specific components of this measure. Specify the numerator and denominator if it is a percent or a rate. If it is an average, identify the calculation for deriving the average. Include any special equipment needed to capture the data. If it is a score (such as a patient satisfaction score) describe how the score is derived. When a measure reflects concepts such as accuracy, complete, timely, or an error, describe the criteria to be used to determine accuracy. Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004.

Operational Definition Worksheet (cont d) Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004. DATA COLLECTION PLAN Who is responsible for actually collecting the data? How often will the data be collected? (e.g., hourly, daily, weekly or monthly?) What are the data sources (be specific)? What is to be included or excluded (e.g., only inpatients are to be included in this measure or only stat lab requests should be tracked). How will these data be collected? Manually From a log From an automated system BASELINE MEASUREMENT What is the actual baseline number? What time period was used to collect the baseline? TARGET(S) OR GOAL(S) FOR THIS MEASURE Do you have target(s) or goal(s) for this measure? Yes No Specify the External target(s) or Goal(s) (specify the number, rate or volume, etc., as well as the source of the target/goal.) Specify the Internal target(s) or Goal(s) (specify the number, rate or volume, etc., as well as the source of the target/goal.)

Dashboard Worksheet Name of team: Date: Measure Name (Provide a specific name such as medication error rate) Operational Definition (Define the measure in very specific terms. Provide the numerator and the denominator if a percentage or rate. Indicate what is to be included and excluded. Be as clear and unambiguous as possible) Data Source(s) (Indicate the sources of the data. These could include medical records, logs, surveys, etc.) Data Collection: Schedule (daily, weekly, monthly or quarterly) Method (automated systems, manual, telephone, etc.) Baseline Period Value Goals Short term Long term Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004.

Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004. Measure Name (Provide a specific name such as medication error rate) NON-SPECIFIC CHEST PAIN PATHWAY MEASUREMENT PLAN Operational Definition (Define the measure in very specific terms. Provide the numerator and the denominator if a percentage or rate. Indicate what is to be included and excluded. Be as clear and unambiguous as possible) Data Source(s) (Indicate the sources of the data. These could include medical records, logs, surveys, etc.) Data Collection: Schedule (daily, weekly, monthly or quarterly) Method (automated systems, manual, telephone, etc.) Baseline Period Value Goals Short term Long term Percent of patients who have MI or Unstable Angina as diagnosis Numerator = Patients entered into the NSCP path who have Acute MI or Unstable Angina as the discharge diagnosis Denominator = All patients entered into the NSCP path 1.Medical Records 2.Midas 3.Variance Tracking Form 1.Discharge diagnosis will be identified for all patients entered into the NSCP pathway 2.QA-URwill retrospectively review charts of all patients entered into the NSCP pathway. Data will be entered into MIDAS system 1.Currently collecting baseline data. 2.Baseline will be completed by end of 1 st Q 2010 Since this is essentially a descriptive indicator of process volume, goals are not appropriate. Number of patients who are admitted to the hospital or seen in an ED due to chest pain within one week of when we discharged them Operational Definition: A patient that we saw in our ED reports during the call-back interview that they have been admitted or seen in an ED (ours or some other ED) for chest pain during the past week All patients who have been managed within the NSCP protocol throughout their hospital stay 1.Patients will be contacted by phone one week after discharge 2.Call-back interview will be the method 1.Currently collecting baseline data. 2.Baseline will be completed by end of 1 st Q 2010 Ultimately the goal is to have no patients admitted or seen in the ED within a week after discharge. The baseline will be used to help establish initial goals. Total hospital costs per one cardiac diagnosis Numerator = Total costs per quarter for hospital care of NSCP pathway patients Denominator = Number of patients per quarter entered into the NSCP pathway with a discharge diagnosis of MI or Unstable Angina 1.Finance 2.Chart Review Can be calculated every three months from financial and clinical data already being collected 1.Calendar year 2010 2.Will be computed in June 2010 The initial goal will be to reduce the baseline by 5%within the first six months of initiating the project.

Exercise: Data Collection Strategies (frequency, duration and sampling) The need to know, the criticality of the measure and the amount of data required to make a conclusion should drive the frequency, duration and whether you need to sample decisions. Measure Vital signs for a patient connected to full telemetry in the ICU Blood pressure (systolic and diastolic) to determine if the newly prescribed medication and dosage are having the desired impact Percent compliance with a hand hygiene protocol Cholesterol levels (LDL, HDL, triglycerides) in a patient recently placed on new statin medication Patient satisfaction scores on the inpatient units Frequency and Duration Pull a sampling or take every occurrence? Central line blood stream infection rate Percent of inpatients readmitted within 30 days for the same diagnosis Percent of surgical patients given prophylactic antibiotics within 1 hour prior to surgical incision

Elements of a Run Chart The centerline (CL) on a Run Chart is the Median Measure ~ X (CL) Time Four simple run rules are used to determine if non-random patterns are present

% of patients with Length of Stay shorter than six days

3.5 Average Length of Stay for DRG 373 3 2.5 Touch time in minutes 2 1.5 1 Median ALOS 0.5 0 Oct Nov Dec Jan Feb March April May June July August Sept Oct Nov Dec Jan Feb March April May June July Month

Year Number of Acute Surgical procedures Month Number of Acute Surgical Procedures 2012 Jan 130 Feb 137 March 113 April 122 May 148 June 102 July 98 August 116 Sept 119 Oct 106 Nov 125 Dec 104 2013 Jan 130 Feb 97 March 115 April 118 May 107 June 108 July 105 August 138 September 121 October 110 Exercise Number of Acute Surgical Procedures Source: Peter Kammerlind, (Peter.Kammerlind@lj.se), Project Leader Jönköping County Council, Jonkoping, Sweden. Make a run chart with the data shown in the table to the left. Decide how you want to lay out the X (horizontal) axis and Y (vertical) axis. Plot the data points. Calculate the median. Hint: use the (n + 1)/2 formula to find the median position first. Then determine the median value. Finally, determine the number of runs on the chart. DO NOT USE YOUR CALCULATOR OR EXCEL!

Concept Measure Value Harm to patients Defective orders Likelihood of errors (defects) What Measure? (Percent, Count or Rate?) Of 1000 patients discharged last month, 60 experienced some degree of harm from medication errors. An audit of their records revealed a total of 10,000 medication orders. Of those orders, 800 were shown to have at least 1 error. A total of 1200 separate errors were identified overall.

You Make the Call! This is a 2 minute quiz! Find a buddy and decide if each measure is a defective (classification) or a defect (count) Measure Number of accidents per 1000 employee days Number errors per 25 food trays Percent of AMI patients who received aspirin within 24 hours of arrival in ER Percent of deaths per month Number of surgical complications per 1000 surgeries performed Proportion of pneumonia patients who get antibiotics appropriately at time of admission Number of falls per 1000 patient days Number of medication errors per 10,000 doses dispensed Defective (classification) Defect (count)

The Control Chart Decision Tree Source: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. ASQ Press, Milwaukee, WI, 2001. Variables Data Decide on the type of data Attributes Data Yes More than one observation per subgroup? No No Occurrences & Nonoccurrences? Yes < than 10 observations per subgroup? Is there an equal area of opportunity? Yes No Yes No No Are the subgroups of equal size? Yes X bar & R X bar & S XmR Average and Range Average and Standard Deviation Individual Measurement c-chart The number of Defects u-chart p-chart np-chart The Defect The percent of The number of Rate Defective Units Defective Units

Is it X bar & R, X bar & S or XmR chart? Measure Waiting time for STAT radiology test for each patient X bar & R XmR X bar & S Patient satisfaction scores for subgroups of five patients admitted to the WeCare ER each day Avg. waiting time for all STAT labs stratified by shift The turnaround time for a random sample of 15 CBCs is pulled each day and stratified by shift Cost for each hip replacement A diabetic patient s daily AM blood sugar Each week the length of stay for subgroups of five patients using treatment A is placed on a chart Each week the anesthesia time for 20 outpatient surgery patients is recorded 15 Copyright 2010 Institute for Healthcare Improvement/R. Lloyd

You Make the Call! Measure Subgroup? Type of Data? Type of Chart? Each day you record the number of films processed in the radiology department. Each day you record the number of films requested and the number that cannot be found in the radiology library. The number of inpatient restraints per month is placed over the total patient days each month. Each day you pull a stratified random sample of 15 CBCs and record the turnaround time (in minutes) for each CBC. The number of minutes it takes to get a stat med order administered to the patient (order time to administration time). 16 Copyright 2010 Institute for Healthcare Improvement/R. Lloyd

You Make the Call! Measure Subgroup? Type of Data? Type of Chart? Every two weeks you pull a sample of 30 medication orders and count the total number of orders that have 1 or more errors. The wait time in the ED (door to discharge) is tracked for each patient. The clinic receptionist notes the time of checkin for each patient. The physician notes the time when he/she first sees the patient in the exam room. An analyst compiles the data daily and reports the total number of patients who had to wait more than 30 minutes. The director of surgery keeps track of the total number of surgical procedures performed each day. The dietary department records the number of food trays that come back uneaten each day and the number of trays they produced. 17 Copyright 2010 Institute for Healthcare Improvement/R. Lloyd

You Make the Call! Measure You are interested in the average time patients spend in your waiting area, so every day a student randomly picks 5 patients and measures their actual waiting time in whole minutes. The ICU nurses want to evaluate the ventilatorassociated pneumonia (VAP) rate. They have data on the total number of pneumonia episodes and the total number of vent days each month. Weekly patient satisfaction scores are recorded for each inpatient unit You know the number of people who come to the A&E complaining of chest pain and the number who are actually diagnosed with an AMI or unstable angina. 18 Subgroup? Type of Data? Type of Chart? Copyright 2010 Institute for Healthcare Improvement/R. Lloyd

Selecting the Most Appropriate Chart Measure Name (Process or Outcome measure?) Subgroup? Type of Data? Chart of Choice? 19 Copyright 2010 Institute for Healthcare Improvement/R. Lloyd

How prepared is your Organization? Key Components* Will (to change) Ideas Execution Self-Assessment Low Medium High Low Medium High Low Medium High *All three components MUST be viewed together. Focusing on one or even two of the components will guarantee suboptimized performance. Systems thinking lies at the heart of CQI! 20