All rights reserved. No part of this publication may be reproduced in any form or by any means without written permission from the publisher.

Transcription

1

2 Senior Editor: Maria R. Aviles, M.A. Project Manager: Bridget Chambers Manager, Publications: Paul Reis Associate Director, Production: Johanna Harris Executive Director: Catherine Chopp Hinckley, M.A., Ph.D. Joint Commission/JCR Reviewers: Maureen Carr, Caroline Christensen, Linda Faber, Linda Hanold, Kimo Hemmes, Cathy Hinckley, Helen Hoesing, Barbara Holland, Jerod Loeb, Paul Reis, Sharon Sprenger, Paul vanostenberg, John Wallin Joint Commission Resources Mission The mission of Joint Commission Resources (JCR) is to continuously improve the safety and quality of health care in the United States and in the international community through the provision of education, publications, consultation, and evaluation services. Joint Commission Resources educational programs and publications support, but are separate from, the accreditation activities of The Joint Commission. Attendees at Joint Commission Resources educational programs and purchasers of Joint Commission Resources publications receive no special consideration or treatment in, or confidential information about, the accreditation process. The inclusion of an organization name, product, or service in a Joint Commission Resources publication should not be construed as an endorsement of such organization, product, or service, nor is failure to include an organization name, product, or service to be construed as disapproval. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. Every attempt has been made to ensure accuracy at the time of publication; however, please note that laws, regulations, and standards are subject to change. Please also note that some of the examples in this publication are specific to the laws and regulations of the locality of the facility. The information and examples in this publication are provided with the understanding that the publisher is not engaged in providing medical, legal, or other professional advice. If any such assistance is desired, the services of a competent professional person should be sought The Joint Commission Joint Commission Resources, Inc. (JCR), a not-for-profit affiliate of The Joint Commission, has been designated by The Joint Commission to publish publications and multimedia products. JCR reproduces and distributes these materials under license from The Joint Commission. All rights reserved. No part of this publication may be reproduced in any form or by any means without written permission from the publisher. Printed in the U.S.A Requests for permission to make copies of any part of this work should be mailed to Permissions Editor Department of Publications and Education Joint Commission Resources One Renaissance Boulevard Oakbrook Terrace, Illinois U.S.A. permissions@jcrinc.com ISBN: Library of Congress Control Number: For more information about Joint Commission Resources, please visit For more information about Joint Commission International, please visit

3 (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) +... a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0 any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) Table of Contents ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx d (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y Introduction Content of the Book viii Target Audience x Acknowledgements x vii Chapter 1: Benchmarking: A Tool for Performance Improvement 1 What Is Benchmarking? Types of Benchmarking Internal Benchmarking External Benchmarking External Benchmarking in the Era of Health Care Reform A Combination Approach: Internal and External Benchmarking Benchmarking and the Transition to emeasures Keys to Benchmarking Success A Standardized Approach to Benchmarking Leadership Support and Involvement Organization Resources A Well-Planned Approach to Benchmarking Joint Commission Standards Related to Benchmarking Joint Commission International Standards Related to Benchmarking Chapter 2: Planning a Benchmarking Project 19 Choosing a Topic to Study Understanding the Benchmarking Topic Deciding What to Benchmark Prioritizing Benchmarking Topics Obtaining Leadership Support Engaging Clinical Leaders Providing Access to Information Organizing a Benchmarking Team Deciding Who Should Be on the Benchmarking Team The Role of the Team Leader The Role of the Facilitator iii

4 Establishing Ground Rules Training Team Members Developing a Plan Chapter 3: Selecting Appropriate Benchmarking Partners 35 Traits of Appropriate External Benchmarking Partners Finding Appropriate Benchmarking Partners The Screening Process Avoiding Problems During Partner Selection Establishing Ground Rules Case Study 1: Texas Children s Hospital, Houston Potential Benchmarking Databases and Partners Benchmarking Sources and Partners in the United States The Joint Commission Core Measure Sets Accountability Measures ORYX Quality Reports and Quality Check Joint Commission Library of Other Measures Strategic Surveillance System Center for Transforming Healthcare and the Targeted Solutions Tool Using Joint Commission Initiatives for Benchmarking Centers for Medicare & Medicaid Services Hospital Inpatient Quality Reporting Program Hospital Outpatient Quality Reporting Program Physician Quality Data Reporting System Hospital Compare Medicare Provider Analysis and Review Quality Care Finder Agency for Healthcare Research and Quality AHRQ Quality Indicators National Quality Measures Clearinghouse Healthcare Cost and Utilization Project Health.Data.gov Health Indicators Warehouse National Quality Forum Institute for Healthcare Improvement Collaboratives Improvement Map Improvement Tracker Tool Physician Consortium for Performance Improvement National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set Quality Compass National Healthcare Safety Network Surgical Care Improvement Project iv

5 The Leapfrog Group Leapfrog Hospital Survey Comparative Performance Assessment for Hospitals International Benchmarking Sources and Partners Joint Commission International International Library of Measures Using the International Library of Measures for Benchmarking Joint Commission International Cardiac Surgery Benchmarking Project World Health Organization Global Health Observatory Database The Global InfoBase Communicable Disease Global Atlas European Health for All Database Targeted Benchmarking Interventions Eurostat The Australian Council on Healthcare Standards Clinical Indicator Program Essence of Care Organization for Economic Co-Operation and Development Health Care Quality Indicators Project Chapter 4: Selecting Benchmarks and Collecting Data 105 Selecting Specific Performance Measures Characteristics of Effective Measures Using Predefined Measures or Creating Customized Measures Understanding Selected Performance Measures Types of Data The Structure of a Measure Inclusion and Exclusion Criteria Sampling Stratification Determining Where to Collect Performance Measure Data Determining How to Collect Performance Measure Data Supporting an Effective Data Collection Process Creating an Operational Definition Identifying Who Will Collect Data Training Data Collectors Supporting Data Security Verifying Data Quality and Validity Case Study 2: Kaiser Permanente, Oakland, California Chapter 5: Analyzing Benchmarking Data 137 Planning for Data Analysis Performing a Risk Adjustment Why Adjust for Risk? What Types of Measures Should Be Risk Adjusted? Risk Adjustment Methods Limitations of Risk Adjustment v

6 Compiling Benchmarking Data Data Analysis Tools Boxplot Radar Chart Control Chart Histogram Line Graph Run Chart Pareto Chart Determining Data Distribution Determining Central Tendency Calculating Variation and Standard Deviation Presenting Comparative Data Determining Reasons for a Gap in Performance Presenting Analyzed Data Making It Visual Balanced Scorecards Dashboards Providing Clear Explanations of Data Case Study 3: Ministry of Health Singapore Chapter 6: Using Benchmarking Data to Improve Performance 165 Setting a Goal for Improvement Obtaining Leadership Support Creating and Participating in a Performance Improvement Team Allocating Resources Identifying Possible Interventions for Improvement Interventions for Performance Improvement Redesigning an Existing Process Implementing an Intervention Plan-Do-Study-Act Lean Thinking Six Sigma Combining Lean and Six Sigma Robust Process Improvement Empowering Staff to Support Change and Implement Interventions Educating Staff on a New Process Ensuring Effective Communication About a New Process Measuring Improvement Case Study 4: Bangkok Hospital Medical Center, Bangkok, Thailand Conclusion 186 Index 189 vi

7 (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d) ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y Introduction = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d)ny(n) = ky. y(6) + ay = 0. any(n) + an 1y(n 1) a1y + a0y = 0. (x a)3(x b)3y cy = 0. (ax + b)n(cx + d To measure is to know. This famous quote by Lord Kelvin underscores the importance of measuring performance. To expand on this idea, one could say, To know is to improve. An organization cannot improve its performance if it does not know first how it is performing. Without first measuring, organizations cannot gauge whether their efforts are adequate or exceptional or whether things could be improved. One effective and frequently used method for measuring and analyzing performance is benchmarking. This proven process involves comparing oneself to a benchmark (or target) or to other performance standards or organizations whether internally or externally particularly those organizations that are the best at what they do. Almost anything can be benchmarked to establish thresholds of what constitutes best performance or practice, and benchmarking can be performed within or outside the health care industry. Over the past 20 years, benchmarking has gone from being a little-known performance measurement philosophy to a standard of practice across numerous industries. Health care organizations are employing benchmarking in many aspects of their performance improvement programs to compare internal and external performance, identify best practices, and reveal opportunities for improvement. Organizations can use benchmarking anywhere along the performance improvement continuum, from launching a program to measuring its final outcome. This dynamic tool can be helpful particularly in meeting the fluid regulatory requirements, which stipulate that health organizations must quantify their performance. Regardless of how organizations use benchmarking, the ultimate goal of this process should be to improve performance. Although some individuals view benchmarking as focusing only on understanding individual problems (for example, mortality rates, infection rates, patient satisfaction rates), others understand that it presents opportunities to solve larger issues, such as providing better care, increasing customer satisfaction, and gaining a competitive edge. The concepts behind benchmarking are deceptively simple identify a problem, collect data, compare data, and respond to those comparisons. However, the intricacies involved in each of these steps can present challenges. Benchmarking in Health Care, second edition, offers potential solutions to these challenges by presenting step-by-step instructions for creating and implementing benchmarking projects, as well as improving performance. This book demonstrates how benchmarking is a continual Benchmarking: A Tool Planning a Selecting Appropriate Selecting Benchmarks Analyzing Using Benchmarking Data Introduction for Performance Improvement Benchmarking Project Benchmarking Partners and Collecting Data Benchmarking Data to Improve Performance vii

8 Benchmarking in Health Care process improvement activity that must be embedded in an organization s culture and that benchmarking is applicable to any type of health care organization anywhere in the world. Content of the Book Each chapter of this book covers a critical step in a benchmarking project. Chapter 1 establishes a foundation by defining benchmarking and the different types of benchmarking (internal and external, both within and outside health care) and lists the keys to success in benchmarking projects, particularly standardization, leadership support and involvement, adequate resources, and planning. It also explains Joint Commission standards related to benchmarking, including the new accountability measures standard, and Joint Commission International benchmarking standards, including the standard that requires organizations to choose clinical measures from the International Library of Measures. Readers will find the discussion on health care reform requirements related to benchmarking such as pay-for-performance, value-based purchasing, hospital-acquired conditions, and reducing readmissions particularly useful and applicable. Chapter 2 outlines the planning phase of a benchmarking project. When planning a benchmarking project, it is important for organizations to clearly define and understand the topic they are going to benchmark, and then to select the variables they ultimately decide to benchmark by prioritizing topics using a variety of tools (such as Pareto charts, cause-and-effect diagrams, affinity diagrams, multivoting, and prioritization matrices). After selecting an appropriate topic to benchmark, it is critical for an organization to create a benchmarking team with clearly defined roles and responsibilities and for that team to obtain leadership support and to develop a project plan that will realize the benchmarking project s goals and objectives. Chapter 3 provides a discussion of factors that organizations need to take into consideration when selecting appropriate partners for external benchmarking projects, including identifying salient characteristics of partners, determining appropriate places to locate partners, screening partners, and establishing ground rules among partners for a mutually beneficial benchmarking project. The first half concludes with a case study from Texas Children s Hospital, Houston, Texas, which implemented a project to benchmark pediatric patient falls. The second half of Chapter 3 provides a comprehensive list of 44 databases and partners that organizations can use in their benchmarking projects. These are divided according to databases and organizations that are located in the United States and those located in other regions of the world. U.S. based sources include the Centers for Medicare & Medicaid Services, the Agency for Healthcare Research and Quality, the National Quality Forum, and the Institute for Healthcare Improvement, to name a few. Joint Commission benchmarking initiatives, such as accountability measures, ORYX, and The Center for Transforming Healthcare, are included in this group. Among the most notable international databases and partners are Joint Commission International, the World Health Organization, Eurostat, and the Australian Council on Healthcare Standards Clinical Indicator Program. Chapter 4 covers the next steps in a benchmarking project: selecting benchmarks and collecting data. These steps require organizations to have viii

9 Introduction consistent, well-defined, and reliable measures and data collection processes. Organizations can learn about how to select appropriate performance measures, how to decide whether to use predefined or customized measures, and how to differentiate among the different types of data (process measures, outcomes measures, and structural measures). Organizations also need to take numerous factors into consideration when collecting data, including how the data are structured (that is, the numerator and denominator) and whether to use inclusion or exclusion criteria, sampling, and stratification. Tips on where organizations can find appropriate benchmarking data are also provided; these include various types of records, forms, and electronic health records. Of central importance in any benchmarking project is ensuring that organizations collect valid, reliable, and useful data that will achieve the project goals. Chapter 4 discusses effective tools that organizations can use to collect data as well as appropriate data collection strategies, including using operationalized and standardized definitions, identifying data collection personnel and ensuring that they receive necessary training, and securing data against privacy breaches. And although all these concepts are important, ensuring that data collected are valid and reliable is a key data collection strategy covered in Chapter 4. Also included in the chapter is a case study focusing on Kaiser Permanente in Oakland, California. The case study describes how the organization identified and benchmarked against best practices in performance improvement to develop a systematic performance improvement approach that could be used across the entire Kaiser Permanente organization. The next chapter, Chapter 5, discusses the importance of analyzing benchmarking data to transform them into information useful to improve performance. Important steps in the data analysis phase of a benchmarking project include adequate planning and performing a risk adjustment, if necessary, prior to data compilation followed by data analysis and data presentation. The chapter describes numerous tools organizations can use to analyze benchmarking data, including box plots, radar charts, control charts, histograms, line graphs, run charts, and Pareto charts. After data have been compiled and analyzed, these results need to be presented in a clear, concise, and informative manner to a variety of audiences. A discussion of some of the most effective tools organizations can use to display data is presented, including balanced scorecards and dashboards. A case study from the Ministry of Health Singapore is presented, which describes the agency s project to benchmark the quality of health care in Singapore. Chapter 6 discusses the culmination of an organization s benchmarking project: translating the data collected, analyzed, and displayed to real performance improvement. Organizations realize this last step in the benchmarking journey by setting realistic improvement goals, obtaining leadership support, identifying targeted interventions aimed at improving performance, successfully implementing these interventions using a variety of performance improvement methodologies (including Lean, Six Sigma, and Robust Process Improvement ), and empowering staff to implement the interventions. After implementation of the interventions when the wheels of change and improvement have been set in motion it is imperative for organizations to continually measure and monitor them to verify ix

10 Benchmarking in Health Care that improvement is indeed occurring. A continuous feedback cycle is thereby established in which interventions are monitored, any necessary adjustments are made to the processes, and the results of the revised processes are measured again, all of which lead to performance improvement. Therefore, benchmarking and performance improvement are continuous and constant parts of an organization s day-to-day operations. The book concludes with a case study from Bangkok Hospital Medical Center, Thailand, which outlines the efforts of two hospitals to benchmark rates of breast conserving therapy and share best practices for breast cancer treatment. Target Audience Benchmarking in Health Care is designed to appeal to the novice benchmarker as well as researchers with more experience, covering both the most basic concepts and more advanced practices. Health care professionals who may find this book to be useful include performance improvement directors, quality managers, risk managers, patient safety officers, nursing and other clinical leaders, medical staff leaders, and others interested in using benchmarking in their work. Hospital staff will find this book particularly applicable because they perform the bulk of benchmarking projects in their clinical activities, through participation in clinical research activities, and due to health care reform legislation that requires them to carry out numerous performance measurement related activities. This book can be of value to all types of health care organizations as they continuously engage in performance improvement projects, which necessitates their knowledge of benchmarking practices. Acknowledgements Every book represents the collaboration of a great team of individuals who contribute their knowledge, expertise, and personal experience to the final products, and this book is no exception. This book would not have been possible without the hard work and talent of a large number of individuals, and Joint Commission Resources (JCR) would like to express its gratitude to them. JCR gratefully acknowledges the following organizations that served as case studies for this book. Their willingness to participate, desire to share their knowledge, and eagerness to contribute various resources and materials have greatly enhanced the quality of this book. These organizations serve as role models and provide important benchmarking lessons that other facilities can learn from. Bangkok Hospital Medical Center, Bangkok, Thailand Virginia Maripolsky, B.S.N., M.S.W., assistant chief executive officer, Nursing Affairs Chanpong Tangkanakul, M.D., medical director, Bangkok Neuroscience Center Children s Hospitals and Clinics of Minnesota, Minneapolis and St. Paul Laure K. Orgon, M.A., R.N., clinical education specialist Children s Medical Center, Dallas Kathy Speer, Ph.D., R.N., P.N.P.-B.C., director, Evidence Based Practice and Research Institute for Healthcare Improvement, Cambridge, Massachusetts Robert Lloyd, Ph.D., executive director, Performance Improvement x

11 Introduction Kaiser Permanente, Oakland, California Lisa Schilling, R.N., M.P.H., vice president, Healthcare Performance Improvement, Department of Care and Service Quality Alide Chase, senior vice president, Quality and Safety Ministry of Health, Singapore Lee Chien Earn, M.B.B.S. M.Med. (PH), F.A.M.S., deputy director of medical services (Health Care Services and Health Care Performance), Singapore Ministry of Health Lim Eng Kok, M.B.ChB., MSc.H.S.M., deputy director, Clinical Benchmarking St. Louis Children s Hospital, St. Louis Heidi W. Fields, M.S.N., R.N., C.P.N.P.-P.C., advance practice nurse, Professional Practice and Systems Texas Children s Hospital, Houston Francine Kingston, Dr.PH., M.S.N., R.N.- B.C., director Angela R. Jones, M.N., R.N., C.N.A.-B.C., nursing quality specialist Wattanosoth Hospital/Bangkok Cancer Hospital and Siriraj Hospital, Bangkok, Thailand Kris Bhothisuwan, M.D., patient care unit director, Breast Center, Member of Breast Surgery International; Emeritus Clinical Professor, Siriraj Hospital Thanks are also due to the following reviewers at The Joint Commission, JCR, and health care organizations around the world who reviewed multiple drafts of the manuscript, provided invaluable comments and edits, and greatly enhanced the quality of the book: Maureen Carr, M.B.A., project director, Department of Standards and Survey Methods, The Joint Commission Caroline Christensen, project director, Department of Standards and Survey Methods, The Joint Commission Lee Chien Earn, M.B.B.S. M.Med. (PH), F.A.M.S., deputy director of medical services (Health Care Services and Health Care Performance), Ministry of Health, Singapore Linda Faber, Ph.D., R.N., M.N., F.A.C.H.E., senior consultant, Joint Commission Resources Helen Glenister, R.N., M.B.A., Ph.D., chief operating officer, The Learning Clinic, London Linda Hanold, M.H.S.A., C.P.H.Q., director, Quality Measurement, Division of Healthcare Quality Evaluation, The Joint Commission Kimo Hemmes, M.S., associate director, Department of Standards and Survey Methods, The Joint Commission Catherine C. Hinckley, M.A., Ph.D., executive director, Publications and Education, Joint Commission Resources Helen Hoesing, R.N., M.P.H., Ph.D., senior consultant, Joint Commission International Barbara Holland, R.N., M.S.N., C.P.H.Q., C.C.M., associate director, Measurement and Quality Monitoring, Joint Commission International Claudia Jorgenson, R.N., M.S.N., associate director, Standards Development and Interpretation, Joint Commission International Jerod M. Loeb, Ph.D., executive vice president, Division of Healthcare Quality Evaluation, The Joint Commission Virginia Maripolsky, B.S.N., M.S.W., assistant chief executive officer, Nursing Affairs, Bangkok Hospital Medical Center, Bangkok, Thailand Paul Reis, manager, Publications and Education, Joint Commission Resources Sharon Sprenger, M.P.A., R.H.I.A., C.P.H.Q., director, External Measurement Relations, Division of Healthcare Quality Evaluation, The Joint Commission xi

12 Benchmarking in Health Care Paul R. vanostenberg, D.D.S., M.S., vice president, International Accreditation, Standards and Measurement, Joint Commission International John Wallin, M.S., R.N., Division of Healthcare Improvement, The Joint Commission Patricia A. Craig, M.S. M.I.S., associate project director, Division of Healthcare Quality Evaluation, The Joint Commission, provided knowledge, expertise, and invaluable assistance in writing the section on emeasures and their relationship to benchmarking. Thanks so much for fitting this book into your busy schedule. And last, but certainly not least, the author of this book, Kathy Vega, deserves a big thank you for her hard work, dedication, and mastery of everything benchmark. Thanks for being committed to this project through the ups and downs. xii