Assessing the Performance of Operating Rooms: What to Measure and Why?

original article Assessing the Performance of Operating Rooms: What to Measure and Why? Hong Choon Oh 1, MEng, PhD (Chem Eng), Tien Beng Phua 2, BSc (Dietetics), MSc (Health Science Management), Shao Chuen Tong 3, BSc (Biomedical Engineering), Master (Health Administration), Jeremy Fung Yen Lim 4, MBBS, MPH 1 Centre for Health Services Research, Singapore Health Services, Singapore 2 Department of Emergency Medicine, Singapore General Hospital, Singapore 3 Health Services Research and Evaluation Division, Ministry of Health, Singapore 4 Lien Centre for Palliative Care, Duke-NUS Graduate Medical School, Singapore Abstract There are multiple indicators that measure different aspects of operating room (OR) performance such as OR productivity, satisfaction of patients and staff involved. The choice of these indicator(s) used for monitoring the performance of ORs often impacts how the processes of ORs are organised and managed. However, there is still no consensus in literature on which indicator(s) should be used for monitoring OR performance. In an effort to promote consensus within the healthcare community, this paper discusses potential performance metrics which may be employed for evaluation of ORs, their rationale and their limitations, and explains why a multidimensional approach is critical in assessment of OR performance. Keywords: operating theatre, performance indicators, productivity INTRODUCTION Regardless of the types of patients, purpose and level of urgency for surgery, most present-day surgical procedures are carried out in operating rooms (ORs) which are designed and equipped to provide care to patients during surgical procedures. ORs are critical assets to all modern tertiary hospitals. This is because ORs contribute to almost two-thirds of a hospital s total revenue 1. Moreover, ORs also account for about 40%of the hospital s total expenses which include manpower costs (i.e. salaries of surgeons, anaesthetists, nurses, etc.) 2, the fixed and operating costs of the surgical facility. Thus, many hospitals that offer surgical services seek to improve the design and operations of their ORs to establish, restore and boost profitability, while retaining the quality of surgical care 3,4. Disclaimer: The views and opinions presented in this article are solely those of the authors and do not necessarily represent those of their respective organisations. However, any effort to improve the design or operation of ORs is a major undertaking due to the inherent complexity of the processes involved. This complexity can be attributed to several factors. First, determination of a schedule of patient arrival times that balance patient waiting with resource utilisation (e.g. OR, surgeons, nurses, etc.) is a complex problem which includes decisions such as sequencing of patient arrivals, allocation of patients to ORs, and matching of patients with surgical teams. Second, ORs are usually part of a surgical suite where they share common resources required in both reception of arriving patients and post-surgery recovery of patients. In the disposal of post-operative patients, Intensive Care Beds are a vital resource which is unfortunately prone to much utilisation uncertainty due to their multiple users, and this uncertainty is exacerbated in hospitals with Emergency Departments. Therefore managing the patient traffic through a surgical suite requires a holistic approach with account of both upstream and downstream resource requirements. Third, 105

Original Article there is significant uncertainty in several activities involved in the delivery of surgical care, such as the uncertainty related to arrival times of patients, OR personnel availability, duration of the surgical procedure, etc. Inevitably, this makes advanced planning of OR utilisation very difficult. Given the underlying complexity in operation management of ORs, it is not surprising that a multitude of performance measures that can be used to assess the ORs of a hospital have been reported in the literature. From the perspective of hospital management, the choice of OR performance measure(s) to track is critical since it defines the organisational goals of the departments or divisions that oversee the ORs. These goals inevitably have direct impact on how the processes of ORs are organised and managed. Due to differences in the emphasis that management teams place on productivity, revenue and patient satisfaction, different hospitals employ different indicators to assess the performance of their ORs. In an effort to promote consensus within the healthcare community and stimulate debate on what metrics would be appropriate for different settings, we describe potential performance metrics which may be employed for the evaluation of ORs, and explain the basis for the selection of these metrics that could be adopted, depending on hospital type. Operating Room Performance Measures The phenomenal technological progress accomplished over the years has enhanced the surgical processes that take place in ORs. For example, several surgical procedures can now be done with the aid of technologies such as robots 5 8 and patients involved get to enjoy the benefits of shorter surgical duration, better clinical outcomes or patient safety. In addition, new technologies such as radio frequency identification (RFID) tags have also been employed by hospitals to track OR assets and movement of OR patients as part of OR performance management process 9,10. Despite all these technological breakthroughs and their applications in ORs to improve the OR performance, it is interesting to note that there is still no consensus in the literature on measure(s) to be used for monitoring OR performance 11. To compound the non-consensus situation, a number of measures are available for OR performance assessment as accounted in a review article on OR planning and scheduling where Cardoen et al identified several OR performance measures that have been reported in more than 100 papers published after 2000 12. These include OR utilisation, throughput, waiting time of surgeons, overtime costs, contribution margin, makespan, waiting time of patients and number of cancelled surgeries. It is critical to have a good overview of known OR performance measures before a sound decision can be made with regards to the choice of measure(s) to use. Essentially, all reported OR performance measures can be broadly classified into 2 key groups, namely hospital-centric metrics and patient-centric metrics, depending on whether the quality of patient experience is considered or not. Hospital-centric Metrics There are several performance measures of ORs, which are of special interest to the hospital involved due to their impact on productivity or revenue. Since these measures do not take into consideration the experience of patients who have undergone surgeries, we collectively define them as hospital-centric metrics. Some of the commonly cited hospital-metric metrics include OR utilisation, throughput, waiting time of surgeons, overtime costs, contribution margin, and makespan as defined and illustrated in Table 1. Patient-centric Metrics There are also OR performance measures which explicitly account for the experience of patients who have undergone surgeries and have a direct impact on both patient satisfaction and safety. We define these measures as patient-centric metrics. Essentially, there are 2 main customer-centric metrics in the literature, namely waiting time of patients and number of cancelled surgeries. Similar to the waiting time of surgeons, waiting time of patient refers to the length of time spent by patients waiting prior to the start of their scheduled or urgent surgeries. Cancelled surgeries refer to scheduled surgeries that have been cancelled due to non-clinical reasons like non-availability of ORs, surgeons or manpower. Clearly, long waiting time and cancellation of surgeries are undesirable as they potentially have an adverse impact on patient satisfaction. In addition, unnecessarily long waiting time of patients for their urgent surgeries 106

Assessing the Performance of Operating Rooms: What to Measure and Why? Metric Table 1: Hospital-centric OR performance metrics. Description OR utilisation Ratio of time spent* by patients in OR to total OR time available 13 16 Throughput Number of surgical cases per unit time 17 21 Waiting time of surgeons Overtime costs Contribution margin Length of time spent by surgeons waiting prior to the start of their scheduled surgeries 22 25 Additional costs incurred due to performance of surgeries beyond the standard operating hours (8 to 12 hours per day) of ORs 26 30 Contribution margin is typically computed in terms of dollar per unit OR time and is the revenue generated by a surgical case less all the hospitalisation variable labour and supply costs 31 35 Makespan The time at which the last patient of a day leaves the OR 36 38 *Generally, there are two definitions of time spent by a patient in OR in the literature. It is either the interval from the instant when a patient is wheeled into OR till the point where patient leaves OR or from the point an incision is made on the patient to the point the patient s surgical opening is closed. may also pose a health or safety concern to the patients concerned since any delay in surgical procedure may prevent them from receiving timely intervention that is necessary to attain the desirable clinical outcome. Discussion Hospitals should strive for both operational efficiency and quality patient care and should not rely on just one of these metrics or dimensions to assess the performance of its ORs. For example, the OR utilisation indicator is a ratio of time spent by the patient in an OR to the time allocated for the surgical procedure involved (see Table 1). Since a high utilisation indicator (i.e. close or equal to 1) is desirable, surgeons may be discouraged from completing their surgical procedures earlier than they can. A utilisation indicator that exceeds one does not reflect favourably on the planning competence of the OR operational managers. As a result, the latter may also favour surgical procedures with longer allocated time since the risk of the utilisation indicator exceeding 1 tends to increase as the total time allocated for surgical procedures decreases. Thus, a hospital that monitors the performance of its ORs solely based on the OR utilisation indicator may stifle productivity since the OR utilisation metric tends to favour slow surgeons and long surgical cases. It also fails to evaluate the quality of service experienced by OR patients. This may have a detrimental effect on the repute of the hospital that in turn may discourage patients from having their surgeries performed in that hospital. Therefore, a hospital has to take a multiprong approach to evaluating the performance of its ORs where all operational and financial aspects of running ORs as well as the experience of patients involved are taken into consideration. Such an approach has been advocated in a paper by Macario who proposed a scoring system using 8 performance indicators to assess the performance of ORs 46. These performance indicators, which consist of both hospital-centric and patient-centric measures, are excess staffing costs, start-time tardiness, case cancellation rate, recovery room admission delays, contribution margin per OR hour, turnover times (i.e. set-up and clean-up turnover times for all cases), prediction error (i.e. error in case duration estimates per OR hour) and percentage of prolonged turnovers (i.e. turnovers which require more than 60 minutes). What should be done after appropriate OR performance measures are identified? Upon identification of the appropriate multidimensional OR performance measures (which include both patient-centric and hospitalcentric metrics) to track, it is critical that the managers or department heads of ORs are made aware of the importance of the selected metrics and their limitations. Operational managers of OR resources are typically clinicians who place greater emphasis on staff and patient factors while hospital 107

Original Article performance management departments tend to be staffed by non-clinicians who are focused on increasing productivity and reducing costs. A concerted effort has to be made to align the goals of the hospital management team with those of OR managers in the selection of OR performance measures. For example, OR productivity cannot be measured and compared across different surgical specialties using the utilisation indicator since they differ in terms of procedure complexity and revenue levels. The hospital management team and OR managers must work out a case mix adjustment formula in the utilisation indicator so that it reflects the OR productivity more accurately and fairly. The Singapore Ministry of Health s Table of Surgical Procedures is one possible, albeit crude, case mix adjustment mechanism that may be employed. Essentially, this table lists out withdrawal limits of surgical procedures which patients can claim using their respective national medical savings accounts 47. It is also equally important to ensure that the tracking of selected OR performance measures does not require extensive manual tracking and recording. OR resource management is complex and requires real-time data on a daily basis and the hospital management should ensure strategic deployment of information technology so that all selected OR performance measures can be monitored and reported in a timely manner without adding significant administrative workload to the OR team involved. Conclusion What gets measured gets done the operating efficiency and service quality of ORs of a hospital are governed by the measure(s) that is(are) chosen to evaluate the performance of ORs. Since there is no one-size-fit-all measure that can comprehensively assess OR performance, multiple indicators that encompass different operational domains of OR performance should be employed to enable a dynamic tension between different priorities. Two points merit emphasising. Firstly, identification of appropriate OR performance measures should be conducted jointly by OR operational managers and hospital management so that the manner by which ORs are run and managed are aligned to organisational goals. Secondly, the eventual basket of indicators selected should reflect the uniqueness of the deciding hospital s priorities and organisational goals pertinent to productivity, profitability and patient satisfaction. It is crucial that the decision as to which OR performance measures to use is made holistically with adequate consideration on several aspects of OR management like service quality, patient satisfaction as well as long-term financial sustainability. It is also important that adequate technical guidance and infrastructural support are made available to the OR teams involved so that (1) they appreciate the importance of tracking the selected OR performance measures, and (2) the performance indicators can be tracked without adding significant administrative workload to the teams. References 1. Jackson R. The business of surgery. Health Manag Technol. 2002;23:20 2. 2. Macario A, Vitez TS, Dunn B, McDonald T. Where are the costs in perioperative care? Analysis of hospital costs and charges for inpatient surgical care. Anesthesiology. 1995;83(6):1138 44. 3. Gabel RA, Kulli JC, Lee BS, Spratt DG, Ward DS. Operating Room Management. Boston: Butterworth-Heinemann; 1999. 224 p. 4. Harris AP, Zitzmann WG Jr. Operating Room Management: Structure, Strategies and Economics. St Louis: Mosby; 1998. 312 p. 5. Füchtmeier B, Egersdoerfer S, Mai R, Hente R, Dragoi D, Monkman GJ, et al. Reduction of femoral shaft fractures in vitro by a new developed reduction robot system RepoRobo. Injury. 2004;35 Suppl 1:S-A113 9. 6. Dharia SP, Falcone T. Robotics in reproductive medicine. Fertil Steril. 2005;84(1):1 11. 7. Pott PP, Scharf HP, Schwarz ML. Today s state of the art in surgical robotics. Comput Aided Surg. 2005; 10(2):101 32. 8. Lorincz A, Langenburg S, Klein MD. Robotics and the pediatric surgeon. Curr Opin Pediatr. 2003;15(3):262 6. 9. Albright B. RFID tags survive hospital sterilization [Internet]. 2009 Apr 7 [cited 2009 Sep]. Available from: http://www.rfidupdate.com/articles/index.php?id=1775. 10. Greenville Hospital System University Medical Center, ThingMagic. Greenville Hospital deploys integrated RFID solution for operating room asset tracking: case study [Internet]. 2009 [cited 2009 Sep]. 3 p. Available from: http://www.industrialportals.com/case_studies/ ThingMagic_GreenvilleHospital_CaseStudy_3.2009.pdf. 11. Marjamaa RA, Kirvelä OA. Who is responsible for operating room management and how do we measure how well we do it? Acta Anaesthesiol Scand. 2007;51(7):809 14. 12. Cardoen B, Demeulemeester E, Beliën J. Operating room planning and scheduling: A literature review. Eur J Oper Res. 2010;201(3):921 32. 13. Dexter F. Operating room utilization: information management systems. Curr Opin Anaesthesiol. 2003;16(6):619 22. 14. Jebali A, Alouane ABH, Ladet P. Operating room scheduling. Int J Product Econ. 2006;99:52 62. 15. Lamiri M, Xie X, Zhang S. Column generation for operating theatre planning with elective and emergency patients. IIE Trans. 2008;40:838 52. 108

Assessing the Performance of Operating Rooms: What to Measure and Why? 16. Ogulata SN, Erol R. A hierarchical multiple criteria mathematical programming approach for scheduling general surgery operations in large hospitals. J Med Syst. 2003;27(3):259 70. 17. Harper PR. A framework for operational modeling of hospital resources. Health Care Manag Sci. 2002; 5(3):165 73. 18. Santibáñez P, Begen M, Atkins D. Surgical block scheduling in a system of hospitals: An application to resource and wait list management in a British Columbia health authority. Health Care Manag Sci. 2007;10(3):269 82. 19. Sciomachen A, Tanfani E, Testi A. Simulation models for optimal schedules of operating theatres. Int J Simul. 2005;6(12 13):26 34. 20. Testi A, Tanfani E, Torre G. A three-phase approach for operating theatre schedules. Health Care Manag Sci. 2007;10(2):163 72. 21. VanBerkel PT, Blake JT. A comprehensive simulation for wait time reduction and capacity planning applied in general surgery. Health Care Manag Sci. 2007; 10(4):373 85. 22. Denton B, Gupta D. A sequential bounding approach for optimal oppointment scheduling. IIE Trans. 2003;35(11):1003 16. 23. Denton B, Viapiano J, Vogl A. Optimization of surgery sequencing and scheduling decisions under uncertainty. Health Care Manag Sci. 2007;10(1):13 24. 24. Gupta D. Surgical suites operations management. Prod Oper Manag. 2007;16(6):689 700. 25. Marcon E, Kharraja S, Simonnet G. The operating theatre planning by the follow-up of the risk of no realization. Int J Product Econ. 2003;85(1):83 90. 26. Bowers J, Mould G. Managing uncertainty in orthopedic trauma theatres. Eur J Oper Res. 2004;154(3):599 608. 27. Dexter F. A strategy to decide whether to move the last case of the day in an operating room to another empty room to decrease overtime labor costs. Anesth Analg. 2000;91(4):925 8. 28. Hans E, Wullink G, van Houdenhoven M, Kazemier G. Robust surgery loading. Eur J Oper Res. 2008;185(3):1038 50. 29. McIntosh C, Dexter F, Epstein RH. The impact of servicespecific staffing, case scheduling, turnovers, and firstcase starts on anesthesia group and operating room productivity: a tutorial using data from an Australia hospital. Anesth Analg. 2006;103(6):1499 516. 30. Tyler DC, Pasquariello C, Chen CH. Determining optimum operating room utilization. Anesth Analg. 2003;96(4):1114 21. 31. Dexter F, Ledolter J. Managing risk and expected financial return from selective expansion of operating room capacity: mean-variance analysis of a hospital s portfolio of surgeons. Anesth Analg. 2003;97(1):190 5. 32. Dexter F, Lubarsky DA, Blake JT. Sampling error can significantly affect measured hospital financial performance of surgeons and resulting operating room time allocations. Anesth Analg. 2002;95(1):184 8. 33. Dexter F, Macario A, Lubarsky DA. The impact on revenue of increasing patient volume at surgical suites with relatively high operating room utilization. Anesth Analg. 2001;92(5):1215 21. 34. Lovejoy WS, Li Y. Hospital operating room capacity expansion. Manag Sci. 2002;48(11):1369 87. 35. O Neill L, Dexter F. Tactical increases in operating room block time based on financial data and market growth estimates from data envelopment analysis. Anesth Analg. 2007;104(2):355 68. 36. Hsu VN, de Matta R, Lee CY. Scheduling patients in an ambulatory surgical center. Nav Res Logist. 2003;50(3):218 38. 37. Marcon E, Dexter F. Impact of surgical sequencing on post anesthesia care unit staffing. Health Care Manag Sci. 2006;9(1):87 98. 38. Pham DN, Klinkert A. Surgical case scheduling as a generalized job shop scheduling problem. Eur J Oper Res. 2008;185(3):1011 25. 39. Bhattacharyya T, Vrahas MS, Morrison SM, Kim E, Wiklund RA, Smith RM, et al. The value of the dedicated orthopedic trauma operating room. J Trauma. 2006;60(6):1336 41. 40. Everett JE. A decision support simulation model for the management of an elective surgery waiting system. Health Care Manag Sci. 2002;5(2):89 95. 41. Guinet A, Chaabane S. Operating theatre planning. Int J Product Econ. 2003;85(1):69 81. 42. Kim SC, Horowitz I. Scheduling hospital services: the efficacy of elective-surgery quotas. Omega. 2002;30(5):335 46. 43. Arenas M, Bilbao A, Caballero R, Gomez T, Rodriguez M, Ruiz F. Analysis via goal programming of the minimum achievable stay in surgical waiting lists. J Oper Res. 2002;53(4):387 96. 44. Van Klei WA, Moons KG, Rutten CL, Schuurhis A, Knape JT, Kalkman CJ, et al. The effect of outpatient preoperative evaluation of hospital inpatients on cancellation of surgery and length of hospital stay. Anesth Analg. 2002;94(3):644 9. 45. Pollard JB, Olson L. Early outpatient preoperative anesthesia assessment: Does it help to reduce operating room cancellations? Anesth Analg. 1999;89(2):502 5. 46. Macario A. Are your hospital operating rooms efficient? A scoring system with eight performance indicators. Anesthesiology. 2006;105(2):237 40. 47. Ministry of Health. Withdrawal limits [Internet]. 2007 [cited 2009 Sep]. Available from: http://www.moh.gov. sg/mohcorp/hcfinancing.aspx?id=324. 109