Advanced SPC for Healthcare December 5, 20 Brent James, MD, Intermountain Healthcare James Benneyan, PhD, Northeastern University Victoria Jordan, PhD, UT MD Anderson Cancer Center Introductions Who are you? Who are we? minute each: Name Where from Why here, what hope to learn Level of SPC experience 2
Meet the Faculty Brent James, MD Chief Quality Officer and Executive Director, Institute for Health Care Delivery Research, Intermountain Health Care and founder of ATP Faculty appointments at the University of Utah School of Medicine, Harvard School of Public Health, University of Sydney, Australia Member, National Academy of Science s Institute of Medicine Fellow, American College of Physician Executives Numerous awards and honors in healthcare including the Deming Cup, C. Jackson Grayson Medal, Distinguished Quality Pioneer, Joint Commission Earnest A. Codman Award, and the National Committee for Quality Assurance Quality Award Meet the Faculty Jim Benneyan, PhD Director, Healthcare systems engineering program, Northeastern University (Boston) Faculty, Northeastern University (systems engineering and operations research) Director, National Science Foundation Center for Health Organization Transformation (CHOT) PI and senior engineer, New England VA Engineering Resource Center (VERC) IHI faculty, improvement advisor, fellow Healthcare Quality and Productivity Incorporated, Partner Areas of expertise: Healthcare systems engineering Statistical quality control, probability, optimization 2
Meet the Faculty Victoria Jordan, PhD Director, Quality Measurement and Engineering, UT MD Anderson Cancer Center University of Texas Chancellor s Heath Fellow for Systems Engineering University of Texas McCombs School of Business Research Fellow PhD Industrial and Systems Engineering (Applied Statistics), MS Industrial & Systems Engineering, MBA, BS-Statistics > years Experience in Quality Improvement ASQ Certified Six Sigma Master Black Belt 5 Session Objectives Important concept of detection power to assess effectiveness of control charts Appropriate chart design and sample sizes More advanced charts (EWMA, Cusum) to improve detection ability Measurement error and impact on process improvement Common errors in practice 6 3
Agenda Introductions / Review Agenda SPC : Review of SPC Basics Assessing Performance Improving Performance Lunch Case study Assessment of Measurement Systems Top Common Pitfalls and g charts Wrap up, Q&A Agenda 8:30 8:50 Introductions, Agenda review Exercise 8:50 9:30 SPC : Review of basics 9:30 :5 Assessing performance 2 :5 break :5-:5 SPC 20 : Improving performance 3 2:00 - :00 lunch :00 :5 Case studies :5-3:00 SPC 30 : Measurement as a system 5 2:5 2:30 break 3:00 :00 More advanced topics, Pitfalls to avoid 8
Handouts You should have: Agenda SPC Excel macro Slides electronically Data sets electronically > laptop per table (relocate if needed) Optional: Calculator or Excel coffee 9 References. Benneyan J (2008), Design, Use, and Performance of Statistical Process Control Charts for Clinical Process Improvement, International Journal of Six Sigma, (3): 29-239 2. Montgomery D (985). Introduction to Statistical Quality Control, Wiley 3. Benneyan J (998), Statistical Quality Control Methods in Epidemiology. Chart Use, Statistical Properties, and Research Issues, Infection Control Hospital Epidemiology, 9():265-3. Jordan V, Benneyan J (202), Common Errors in Using Healthcare SPC, in Statistical Methods in Healthcare, Wiley, to appear 5. Benneyan J (200), Number-Between g-type Statistical Control Charts for Monitoring Adverse Events, Health Care Management Science, :305-38 5
Section I: SPC Basic Methods (Dr Brent James) Learning Objectives Review basic concepts of SPC Different types of variability Different types of data and charts o Binomial: np, p charts o Poisson: c, u charts o Normal: Xbar & S charts (or Xbar & R) (or XmR??) Chart construction and interpretation 2 6
Review of SPC Slides and case study from Brent? 3 Applications of Control Charts. Detect and monitor process Variation over time. 2. Distinguish Special Cause variation from Common Cause Variation. 3. Common language for discussing process performance.. Determine process capability 5. Develop a plan for process improvement
Definitions. Common Cause Variation: Causes that are inherent over time and affect everyone in the process and the process outcome. 2. Special Cause Variation: Causes that arise from specific circumstances and are not part of the process all of the time. 3. Stable Process: Implies that the variation is predictable within common bounds.. Unstable Process: A process that is affected by both special cause variation and common cause variation. The variation from one time period to the next is unpredictable. Attribute Control Charts Attribute control charts are used when it is necessary to classify or count a particular characteristic of a process instead of measuring it. There are four types of Attribute control charts: ) NP-Chart: For the number defective, where each item is either go/nogo, good/bad, yes/no, etc., use with constant subgroup size. 2) P-Chart: For the proportion defective, where each item is either go/nogo, good/bad, yes/no, etc., and changing or constant subgroup size. 3) C-Chart: For counting defects with a constant area of opportunity where the defects are drawn from. There is no upper limit on the number of defects that could occur. ) U-Chart: For counting defects per changing area of opportunity. There is no upper limit on the number of defects that could occur. Which chart should I use? Ask: ) Is there a maximum count for each group? Yes or No? 2) Is each subgroup the same size? Or changing? Note: A defective may be caused by more than one defect. 6 8
Understanding Variation Choosing the right control chart Group Exercise Which Control Chart Should be Used? For each of the following, identify different ways that the variable could be measured and the appropriate control chart(s) for each: o Patient wait time o Patient and family complaints o Falls o Medication Errors o SCIP bundle delivered o Glucose level o Percent of patients receiving genetic counseling o Time to next available appointment 8 9
Control Charts Tests for patterns in data are based on the laws of probability, assuming a normal distribution CENTER LINE UPPER CONTROL LIMIT 99.3% 95.% 68.36% Zone A B C C B A LOWER CONTROL LIMIT 9 Understanding Variation Non Random Patterns That Should be Investigated TEST : POINT OUTSIDE CONTROL LIMIT (p = 0.002) TEST 3: FOUR OF FIVE POINTS ON ONE SIDE OF THE CENTER LINE IN ZONE B OR BEYOND (p = 0.002) TEST 2: TWO OUT OF THREE POINTS ON ONE SIDE OF THE CENTER LINE IN ZONE A OR BEYOND (p = 0.005) TEST : RUN OF EIGHT POINTS ON THE SAME SIDE OF THE CENTER LINE (p = 0.0039) 20
Understanding Variation Non Random Patterns That Should be Investigated Test 5. SIX POINTS IN A ROW STEADILY INCREASING (OR DECREASING) (p = 0.002) Test. STRATIFICATION - 5 POINTS HUGGING THE CENTERLINE (p = 0.0033) Test 6. FOURTEEN POINTS IN A ROW ALTERNATING UP & DOWN Test 8. EIGHT POINTS IN A ROW ON BOTH SIDES OF THE CENTER LINE WITH NONE IN ZONE C (p = 0.000) 2 Table Exercise Assessment of Control For the following slides, determine what type of control chart is used and whether or not the process is in control. If the process is not in a state of control, explain why (what rule is violated) and what that means in terms of the specific process.
Control Chart Example 0.95 0.90 Handwashing Compliance UCL=0.92 Proportion 0.85 0.80 0.5 P=0.80 0.0 LCL=0.68 3 6 9 22 3 3 Control Chart Example 0.95 0.90 Handwashing Compliance UCL=0.92 Proportion 0.85 0.80 0.5 P=0.80 0.0 LCL=0.68 3 6 9 22 3 3 2 2
Control Chart Example 0.95 0.90 Handwashing Compliance UCL=0.92 Proportion 0.85 0.80 0.5 P=0.80 0.0 LCL=0.68 3 6 9 22 3 3 Control Chart Example 2 Available Beds 0.09 UCL=0.09 Proportion 0.08 0.0 0.06 0.05 0.0 P=0.06 0.03 LCL=0.03 0.02 3 6 9 22 Day of Week 3 3 3
Control Chart Example 2 Available Beds 0.09 UCL=0.09 Proportion 0.08 0.0 0.06 0.05 0.0 P=0.06 0.03 LCL=0.03 0.02 3 6 9 22 Day of Week 3 3 Control Chart Example 2 Available Beds 0.09 UCL=0.09 Proportion 0.08 0.0 0.06 0.05 0.0 P=0.06 0.03 LCL=0.03 0.02 3 6 9 22 Day of Week 3 3
Control Chart Example 3 Amount Spent on Clinic Supplies 5000 UCL=5060 Dollars 500 000 3500 X=066 3000 3 6 9 22 3 3 LCL=302 200 UCL=22 Moving Range 900 600 300 MR=3 0 LCL=0 3 6 9 22 3 3 Control Chart Example 3 Amount Spent on Clinic Supplies 5000 UCL=5060 Dollars 500 000 3500 X=066 3000 3 6 9 22 3 3 LCL=302 200 UCL=22 Moving Range 900 600 300 MR=3 0 LCL=0 3 6 9 22 3 3 5
Control Chart Example Percentage of Barcodes Containing Errors.00% UCL=.% Proportion 6.00% 5.00%.00% P=5.% 3.00% LCL=3.0% 3 6 9 22 Time (Days) 3 3 Control Chart Example Percentage of Barcodes Containing Errors.00% UCL=.% Proportion 6.00% 5.00%.00% P=5.% 3.00% LCL=3.0% 3 6 9 22 Time (Days) 3 3 6
Control Chart Example Percentage of Barcodes Containing Errors.00% UCL=.% Proportion 6.00% 5.00%.00% P=5.% 3.00% LCL=3.0% 3 6 9 22 Time (Days) 3 3 Control Chart Example 5 Patient Complaints UCL=.5 Number of Complaints 20 5 5 C=.23 LCL=2.9 0 3 6 9 22 3 3
Control Chart Example 5 Patient Complaints UCL=.5 Number of Complaints 20 5 5 C=.23 LCL=2.9 0 3 6 9 22 3 3 Control Chart Example 5 Patient Complaints UCL=.5 Number of Complaints 20 5 5 C=.23 LCL=2.9 0 3 6 9 22 3 3 8
Section II: Assessing Chart Performance (Jim Benneyan) 3 9