University of Michigan Health System MiChart Department Improving Operating Room Case Time Accuracy Final Report

Transcription

1 University of Michigan Health System MiChart Department Improving Operating Room Case Time Accuracy Final Report Submitted To: Clients Jeffrey Terrell, MD: Associate Chief Medical Information Officer Deborah Laubach: Administrative Senior Manager Coordinators Andrew Gutting: Systems Analyst, MiChart-Op Time Josh Pigula: Senior Financial Specialist Supervising Instructors Mary Duck: Senior Management Consultant, UMH Program & Operations Analysis Dr. Mark P. Van Oyen: Professor, IOE Submitted By: IOE481-F8 Christine Dolikian, Senior IOE Student Jacob Landuyt, Senior IOE Student Esmond Loke, Senior IOE Student Date Submitted: December 15, 2015

2 EXECUTIVE SUMMARY The University of Michigan Health System (UMHS) has 108 operating rooms (ORs) across seven locations, which are used by various medical departments. Each surgery is scheduled for an expected duration, or case time. Because operating room time is a valuable resource, it is important to hospitals both financially and for patient care that scheduled case times are accurate so the operating rooms are utilized efficiently and procedures start on time. The MiChart OpTime department believes that averaging historical case times could be a useful way to predict future case times, but the department is unsure for which procedures this approach would be appropriate and how much improvement could be expected. As a result, a team of IOE 481 students from the University of Michigan was asked to lead a project with the goal of improving the accuracy of scheduled case times for operating room, Cath/EP, and IR (interventional radiology) cases. Background The operating room locations include University Hospital (UH), Cardiovascular Center (CVC), Mott Children s Hospital (MH), Von Voigtlander Women s Hospital (VH), Kellogg Eye Center (KEC), East Ann Arbor Surgery Center (EAA), and Livonia Surgery Center (LSC). According to the client, the opportunity cost per minute for the operating room is about $42, not including anesthesia. It is crucial to the health system that OR time is managed efficiently. In June of 2014, UMHS transitioned to MiChart, an electronic medical records system designed by EPIC Systems Corporation specifically for UMHS. MiChart offers UMHS a sophisticated system to manage scheduling and to track patient and procedure information. For each procedure performed, MiChart collects data on when procedures actually started and ended. MiChart has approximately 2,800 procedure codes representing more than 10,000 distinct procedures. For each procedure code, MiChart has a default case time. Despite all the data available in MiChart on the duration of prior procedures, the default case times were transferred from the old computer system to MiChart during the transition in June of 2014 and have not been updated. MiChart has a function, Procedure Time Averaging, which uses recent procedure records to continually update estimated case times. This function is not currently a formal part of scheduling cases. Methods and Findings The project team used five methods to determine how to improve scheduled case time accuracy: employee interviews, surgeon surveys, literature research, swim-lane diagrams, and data analysis. For this project case times are deemed accurate based on this definition: Procedures scheduled for less than 2 hours: scheduled case time is accurate if actual case time is within 15 minutes of scheduled case time Procedures scheduled for greater than or equal to 2 hours: scheduled case time is accurate if actual case time is within 30 minutes of scheduled case time i

3 Employee Interviews The team interviewed staff from MH, UH, CVC, KEC, and EAA to understand the scheduling processes at each of the locations and the factors perceived to affect scheduled case time accuracy. The team found that no standardized process exists and the process varies from location to location. Different types of staff members are responsible for adjusting case times from location to location. Many factors are perceived to influence case time accuracy. Surgeon Surveys Surgeons perform the surgeries and often begin the scheduling process, so it is important to understand the factors they perceive to influence case time accuracy. The team distributed an electronic survey to 170 surgeons across various surgical departments. The survey received 62 responses. The team found the following from the survey responses: Procedure complexity, faculty experience, and time of day of surgery were perceived to have the most influence on case time accuracy, while patient socioeconomic status was perceived to have very little influence on case time accuracy Anesthesia experience, OR staff and nursing experience, setup and cleanup, PACU availability, nursing breaks and shift changes, and complications were additional factors that were perceived to affect case time accuracy Surgeons do not receive feedback on accuracy and surgeons are open to basing case times off historical data. Seven surgeons mentioned that they do not get enough feedback and they do not see data on how long procedures actually take them. Surgeons perceive that they are more accurate than they actually are. Literature Search The team performed a literature search to gain insight on how to improve case time accuracy by studying three health system research studies pertaining to operating rooms and scheduling. The literature search revealed that variation in case time exists between surgeons performing the same procedure. Swim-Lane Diagrams To establish the current state of the case scheduling process, the team created swim-lane diagrams for five OR locations. By creating the swim-lane diagrams, the team understood the process used to schedule varies by location. Depending on the location, varying staff roles adjust the schedule. The swim-lane diagrams show that the processes used to schedule are complex, especially at Mott. Data Analysis The team was provided MiChart data from June 2014 through September First, the data was stratified by location and by service to see how many cases are underbooked, overbooked, and scheduled accurately in the current state. Next, the data was analyzed to see which factors influence procedure time accuracy. The team created a method to compute the optimal percentile at which to schedule procedure times, but found it was impractical when dealing with large data ii

4 sets. In addition, to determine the best method to predict scheduled case times, the team split the data in half. Half of the data was used to calculate the scheduled times based on various methods, and the other half of the data was used to test the effectiveness of the methods. The analysis examined how each of the alternative scheduling methods performed on case times that were distributed normally compared to non-normal distributions. The analysis also examined how different coefficients of variation affected the potential improvement opportunity for each alternative scheduling method. From the data analysis, the team found the following: 45% of cases were scheduled accurately between June 14 and October 15 Case time accuracy varies by location and service Procedure time averaging does not benefit procedures with normal distributions or low coefficients of variation more than procedures with non-normal distributions or high coefficients of variation Surgeons were the most accurate in estimating case time in case requests Eight of the factors analyzed are statistically significant to case time accuracy Conclusions Using the findings from both the qualitative and quantitative methods the team formed several conclusions. The major conclusions related to case time accuracy are below. The details can be found in the report. 1. Historical median is the most accurate way to schedule cases 2. Surgeons most accurately predict case times 3. The significant factors identified only explain a small portion of case time variation 4. Currently surgeons and schedulers do not receive feedback on scheduling performance 5. Surgeons work best with familiar anesthesiologists and nurses Recommendations The conclusions made by the team were used to develop recommendations that aim to improve the accuracy of OR, Cath/Ep, and IR cases. These recommendations are briefly listed below. 1. Set default case time in MiChart to the historical median 2. Consider requiring surgeons to submit all case requests 3. Do not use the factors analyzed in this report as the sole predictors of case time, as they account for little of the variation in case times 4. Provide a dashboard to surgeons and schedulers that displays recent scheduling accuracy performance 5. Explore ways to pair surgeons with familiar anesthesiologists and nurses The team predicts an improvement to overall accuracy of at least 11% if the recommendations are implemented. This will improve patient care, reduce staff overtime, and improve OR efficiency. iii

5 Table of Contents INTRODUCTION... 1 BACKGROUND... 2 Key Issues... 2 Goals and Objectives... 3 Project Scope... 3 METHODS AND FINDINGS... 3 Employee Interviews: Methods... 4 Employee Interviews: Findings... 4 Surgeon Surveys: Methods... 5 Surgeon Surveys: Findings... 5 Literature Search: Methods... 8 Literature Search: Findings... 8 Analysis of OR Case Times at UMMC... 8 Room Scheduling Analysis for Interventional Radiology... 9 Machine Learning at the Operating Room of the Future: A Comparison of Machine Learning Techniques applied to Operating Room Scheduling... 9 Swim-lane Diagrams: Methods... 9 Swim-lane Diagrams: Findings... 9 Data Analysis: Methods Data Analysis: Findings Current Situation By Location Current Situation By Service Factor Analysis Alternative Scheduling Methods Coefficient of Variation Normality Regression CONCLUSIONS Qualitative Conclusions Quantitative Conclusions RECOMMENDATIONS

6 Qualitative Recommendations Quantitative Recommendations EXPECTED IMPACT REFERENCES APPENDIX A: SUMMARY FROM EMPLOYEE INTERVIEWS APPENDIX B: SWIM-LANE DIAGRAMS APPENDIX C: CURRENT ACCURACY SITUATION BY SERVICE APPENDIX D: PATIENT AGE VS. ACCURACY APPENDIX E: CASE TIME ACCURACY BY STAFF ROLE AND SERVICE APPENDIX F: DESCRIPTIONS OF ASA SCORES APPENDIX G: SIMULATED USE OF CASE TIME AVERAGING (CATH/EP AND IR) APPENDIX H: COEFFICIENT OF VARIATION VS. SCHEDULING METHODS APPENDIX I: EVALUATION OF SCHEDULING METHODS USING REGRESSION

7 Table of Figures Figure 1: List of staff members interviewed... 4 Table 1:Surgeon perceptions on factors affecting case time accuracy... 6 Figure 2: Surgeon Perceptions of Other Factors that Affect Case Time Accuracy... 7 Figure 3: Long Procedures are perceived to be less accurate than short procedures... 7 Table 2: Comparison of actual and perceived accuracy by location... 8 Figure 4: Mott swim-lane diagram Figure 5: Method for simulating alternative scheduling methods Table 3: Accuracy By Location Figure 6: Accuracy with and without residents Figure 7: Accuracy by patient class Figure 8: Accuracy by anesthesia type Table 4: Staff Performance in terms of scheduled case time accuracy Table 5: Scheduler Performance in terms of scheduled case time accuracy Figure 9: Accuracy of scheduled vs. add-on cases Figure 10: Accuracy by case class Table 6: Top 50 surgeons in terms of case time accuracy Figure 11: Accuracy by ASA Score Table 7: Accuracy of scheduled case times during various parts of the day Table 8: Simulated use of procedure time averaging on OR Data Table 9: Simulated use of procedure time averaging by location

8 INTRODUCTION The University of Michigan Health System (UMHS) has 108 operating rooms (ORs) across seven locations, which can be reserved by various medical departments. The operating room locations include University Hospital (UH), Cardiovascular Center (CVC), Mott Children s Hospital (MH), Von Voigtlander Women s Hospital (VH), Kellogg Eye Center (KEC), East Ann Arbor Surgery Center (EAA), and Livonia Surgery Center (LSC). According to the client the opportunity cost per minute for the operating room is about $42 not including anesthesia. Each surgery is scheduled for an expected duration, or case time. It is critical to schedule case times accurately because of the value of OR time. Because operating room time is such a valuable resource, it is important that scheduled case times are accurate so the operating rooms are utilized efficiently and procedures start on time. The MiChart OpTime department believes that averaging historical case times could be a useful way to predict future case times. There is a function within MiChart, Procedure Time Averaging, which uses recent cases to update predicted procedure times. The department is unsure for which procedures this approach would be appropriate, and if it would be effective. As a result, a team of IOE 481 students from the University of Michigan was asked to use historical data from June 2014 through September 2015to explore potential improvement to the accuracy of the scheduled case times at the operating room locations. The team was also asked to investigate Cath/EP and IR cases and their improvement potential. To better understand the current situation, the team interviewed eight key employees involved in the scheduling process and surveyed 62 surgeons. The team conducted a literature search to gain insight from similar research studies and created swim-lane diagrams to document the current scheduling process at five locations. With the historical data, the team stratified the data in different ways to identify trends and analyzed which factors affected case time accuracy. Next, the team split the data in half. The team used one half of the data to calculate recommended case times for each procedure using various methods, then evaluated the methods of the various methods using the other half of the data. This report contains the team s findings, conclusions, recommendations, and supporting documentation. Lastly, the team has worked with the project clients and coordinators to develop a definition of accuracy to be applied to case times. It was found from an industry expert that a common definition of accurate is within 15 minutes of scheduled case time, but the team expanded the definition to account for the high variation in procedure lengths. The definition of accuracy is: Procedures scheduled for less than 2 hours: scheduled case time is accurate if actual case time is within 15 minutes of scheduled case time Procedures scheduled for greater than or equal to 2 hours: scheduled case time is accurate if actual case time is within 30 minutes of scheduled case time If cases end before the accurate threshold they are classified as overbooked, and if they end later than the acceptable range they are deemed underbooked. This definition is of case time designation is used consistently throughout the entire report and project. 1

9 BACKGROUND UMHS performed over 52,000 surgical cases in fiscal year 2015 [1]. Each surgery department is allocated a certain about of booking time at the various OR locations. To schedule procedures at the seven operating room locations, the various surgery departments usea centralized computer system. MiChart, the UMHS electronic medical records system, was officially implemented in June of Since implementation, MiChart has been used to manage scheduling for all cases. As a result, extensive data exists for all procedures performed at the various OR, Cath/EP, and IR sites.for each procedure performed, MiChart collects data on when procedures actually start and end. MiChart has approximately 2,800 different procedure codes, though in reality there are greater than 10,000 distinct procedures that can be performed. Each procedure code has a default case time and a Doctor Preference Card, which specifies the equipment the surgeon needs to complete the procedure. The default case times in MiChart were transferred from the old computer system to MiChart during the transition in June of 2014 and have not been updated since. MiChart has the capacity to use the detailed data collected during every case to update predicted case times on a surgeon level. For example, using the Procedure Time Averaging function takes recent cases performed by a surgeon, all the same procedure, and can calculate the average time the cases took as well as the median time. Currently, this function is not widely used in the scheduling process. Many people are involved in creating and modifying operating room schedules in MiChart. While the process varies by operating room location, the following four-step, high level process is essentially the same across all locations: 1. A surgeon, surgeon scheduler, resident, or nurse submits a case request in MiChart with the procedure type, case time, and Doctor Preference Card. 2. The surgeon s scheduler looks at the allocation of the operating rooms and adds the procedure to an open time slot on the MiChart schedule. 3. The central scheduler at each operating room location oversees the schedule, flags any potential problems with the schedule, and makes any necessary adjustments. 4. The front desk staff and charge nurse at each OR location swap rooms or rearrange the schedule on the day of the surgery, if necessary. The process can be complex and varies at each location. There is no universal way that case times get determined. Key Issues The procedure scheduling process has several issues. The major problems below were brought to the team as part of the reason this project is so important 2

10 Procedures are often delayed. Surgical departments frequently underbook cases to fit more procedures into their allocated block time, causing a backlog of cases toward the end of the working day. Default case times are not based on recent data and are often inaccurate. The default case times for each procedure code do not take into account the surgeon, location, or exact procedure being performed. Not accounting for these factors when determining case time can lead to underbooking (where too little time is allotted) or overbooking (where too much time is allotted) for a procedure and inefficient use of the operating rooms. No feedback is given to employees involved in the scheduling process. Surgeons, surgeon schedulers, and central schedulers are not given feedback on how accurate their scheduled case times are and are not able to make necessary adjustments in the future. The team will target these key issues areas to improve the accuracy of the scheduled OR case times. Goals and Objectives The primary goal of the project is to improve the accuracy of scheduled operating room case times. To achieve this goal, the team did the following: Understood the process used to schedule procedures at the various OR locations Identified which factors are perceived to influence scheduled case time accuracy and compared these perceptions to historical data Analyzed if procedure scheduling would benefit from case time averaging or alternative methods for predicting case times Developed a method to assist in predicting case times Project Scope The project focused on single panel, single procedure OR cases at UH, CVC, MH, KEC, EAA, and LSC. Single panel, single procedure cases account for 86.6% of OR cases from June 2014 through September The team gathered process information from surgeons, central schedulers, and charge nurses. This project also analyzed Cath/EP and IR cases separately from OR cases when gauging possible accuracy improvement. This project defined case time as wheels in to wheels out. The project excluded multiple panel and multiple procedure cases. Also out of scope is setup and cleanup time. The project also excluded cases at the VH OR and anesthesia cases. Anesthesia providers and clinic schedulers were out of scope. METHODS AND FINDINGS To improve the accuracy of scheduled case times, the team collected data through employee interviews and surgeon surveys to better understand the current scheduling process and staff perceptions. In addition, team conducted a literature search to gain insight from similar studies. This project also did an in-depth analysis on existing MiChart data from June 2014 through 3

11 September 2015 to identify trends and determine which method for predicting case times performs the best. Employee Interviews: Methods To understand the scheduling process, the team conducted several interviews with key stakeholders in the process at Mott Hospital, University Hospital, Cardiovascular Center, Kellogg Eye Center, and East Ann Arbor Surgery Center. These interviews with Charge RN s (Registered Nurse) and central schedulers captured the OR scheduling process. A list of the staff members interviewed is shown in Figure 1. Figure 1: List of staff members interviewed Speaking to the people who are involved with scheduling procedures enabled the team to document similarities and differences between locations. The primary purpose of these interviews was to identify how procedure times are determined and adjusted up until when the procedure is performed. Questions were used to identify the factors that the employees believe affect case times and the accuracy of scheduled procedure times. Collecting the perceptions of the people involved with establishing procedure times and managing the OR schedule is crucial to educating the scheduling and OR staff on what factors are influential to case times based on data analysis. It also ensured that the people performing the process had a voice in determining a better method. Employee interviews added key qualitative analysis to the project. Employee Interviews: Findings The employee interviews were very successful in providing an understanding of the different roles within the scheduling process and the part each plays in determining procedure times. The findings of these interviews are included here. A primary takeaway is that the process varies greatly location to location. This shows there is no standard process that is used to establish procedure times across all of UMHS. The detailed findings from each location are summarized in Appendix A. Also, a frequent comment was that the schedulers really do not have any way of knowing how long a case actually took. With no feedback on scheduling accuracy, schedulers are not aware is they are doing well when estimating case times. Similarly, nurses feel that surgeons are not conscious of scheduled case time when performing a procedure or even completing a case request. It is the nurses and schedulers opinion that surgeons assume they are estimating case times accurately because they too, do not receive feedback on actual case time compared to the scheduled time. 4

12 In summary, the team found that no standardized process exists, variation exists between locations, schedulers and surgeons are not aware of how well or poor they schedule cases, and many factors are perceived to influence case time accuracy by employees. It is also important to note that the staff responsible for adjusting case times is different location to location. Surgeon Surveys: Methods The surgeons performing a procedure play the largest role in how long a case takes. Also, surgeons often are the person estimating the case time. Therefore it was crucial to collect their thoughts and perceptions to determine how to improve case time accuracy. The team surveyed surgeons electronically to gather this information as well as offer an opportunity for surgeons to share their thoughts on case times and how they are scheduled. The survey consisted of seven questions, two of which were open-ended questions. The survey asked the following questions: 1. What is your service? 2. Where do you perform most of your surgeries? 3. Indicate if each of the factors has no effect, some effect, or significant effect on case time accuracy: Faculty experience Procedure complexity Operating room location Presence of resident Experience of resident Patient age Patient BMI Patient ASA score Patient class (inpatient vs. outpatient vs. surgery admit) Patient socioeconomic status Time of day of surgery Whether surgeon personally fills our case request form 4. For cases shorter than 2 hours, what percent of your cases do you believe are scheduled accurately (within 15 minutes of scheduled case duration)? 5. For cases longer than 2 hours, what percent of your cases do you believe are scheduled accurately (within 30 minutes of scheduled case duration)? 6. What other factors affect the gap between scheduled case times and actual case times (open ended)? 7. What other thoughts do you have about scheduling accuracy (open ended)? The survey was distributed electronically to the surgeons with the highest case volume (June 2014-September 2015) within each service. The survey was distributed to 170 surgeons total across 28 services and the team received 62 responses in total. Surgeon Surveys: Findings The 62 surgeon survey responses resulted in four significant findings. First, procedure complexity, faculty experience, and time of day of surgery were perceived to have the most influence on case time accuracy, and patient socioeconomic status was perceived to have very 5

13 little influence on case time accuracy. A weighted average was computed using the survey responses, where a score of 1 signified no effect, 2 signified some effect, and 3 signified significant effect. The results are seen in Table 1. Table 1: Surgeon perceptions on factors affecting case time accuracy Factor Score Procedure Complexity 2.74 Faculty Experience 2.40 Time of Day of Surgery 2.31 Experience of Resident 2.19 Patient BMI 2.02 Presence of Resident 2.00 Patient ASA Score 2.00 Operating Room Location 1.73 Who fills out case request 1.71 Inpatient vs. outpatient vs. surgery admit 1.69 Patient Age 1.56 Patient Socioeconomic Status 1.08 Furthermore, anesthesia experience, OR staff and nursing experience, setup and cleanup, PACU availability, nursing breaks and shift changes, and complications were additional factors that were perceived to affect case time accuracy. Half of the surgeons (31) listed anesthesia experience as a major factor affecting case time accuracy in an open ended question and suggests that anesthesia experience is one of the factors that affects case time most. Figure 2 below shows the most frequent comment topics in the free response questions. 6

14 N = 50 Figure 2: Surgeon Perceptions of Other Factors that Affect Case Time Accuracy Anesthesia and nursing were the most common factors believed to affect case time accuracy. Both of these factors emphasize that surgeons perform best when working with a familiar team. Third, surgeons do not receive feedback on scheduling accuracy and surgeons are open to basing case times off historical data. Seven surgeons mentioned that they do not get enough feedback and they don t see data on how long procedures actually take them. Fourth, surgeons believe that for longer procedures scheduled case times are significantly less accurate than short procedures scheduled case times. The team performed a paired t-test using responses from the 53 of 62 surgeons that reported performing both long and short procedures to compare the perceived accuracy for each classification of procedure length (Figure 3). Figure 3: Long Procedures are perceived to be less accurate than short procedures 7

15 These results indicate a significant difference in perception of case time accuracy between long and short procedures. In brief, surgeons believe long procedures are scheduled less accurately than short procedures. This is true, however, in reality 48% of short procedures are accurate and 41% of long procedures are accurate. In both cases surgeons overestimate the accuracy of scheduled case times. The team stratified by location to determine if surgeons believe they are more accurate than they actually are. The results are shown below in Table 2. Table 2: Comparison of actual and perceived accuracy by location Short Procedures Long Procedures Location Actual Perceived N Actual Perceived N CVC OR 24% 61% 3 29% 39% 3 EASC OR 53% 75% 8 61% 52% 8 KEC OR 58% 87% 3 45% 55% 2 LSC OR 59% 75% 1 58% N/A 0 MH OR 46% 73% 16 42% 55% 12 UH OR 37% 57% 30 40% 44% 29 Grand Total 48% 66% 61 41% 48% 54 Although the sample size is small, the surgeons perceive they are more accurate than they actually are at each location. Therefore, surgeons believe that scheduled times are more accurate than they actually are. Literature Search: Methods The team performed a literature search to gain insight on how to improve case time accuracy. Two research studies performed at the University of Michigan and one from MIT were reviewed. The articles and their relation to this project are listed below. Analysis of OR Case Times at UMMC (UMich) to understand the causes of variation in procedure time and if variation exists among surgeons performing the same procedure [2] Room Scheduling Analysis for Interventional Radiology from (UMich) to understand how optimization can be used to improve scheduled case time accuracy [3] Machine Learning at the Operating Room of the Future: A Comparison of Machine Learning Techniques applied to Operating Room Scheduling (MIT) to understand the effectiveness of machine learning techniques on scheduled case time accuracy [4] Literature Search: Findings From the three studies examined in literature search, the team gained valuable insight from prior studies on how to improve case time accuracy. Analysis of OR Case Times at UMMC First, the findings of this project state that variation exists between surgeons performing the same procedure. The results suggest that the cause of this variation is complex. Some of the influencing factors include where the surgeon trained, surgeon familiarity with the procedure, as 8

16 well as the time a surgeon has a resident perform part of the surgery. Variation between surgeons is important to OR case times. Second, the study found there are several other factors that influence case time beyond the surgeon performing the procedure. Patient characteristics were found to be significant in determining the case time. Operations on more critical condition patients took longer. The condition of the patient was based on metrics such as BMI and preexisting medical conditions. Another significant factor was anesthesia type. This study found that spinal anesthesia resulted in quicker procedures and general anesthesia procedures typically took longer [2]. Room Scheduling Analysis for Interventional Radiology This IOE 481 project consisted of an in-depth study of 34 outpatient cases prior to the transition to MiChart. In the project, the group developed a scheduling tool in Excel that calculates the optimal percentile to minimize the idle time and delay time for the surgeon. This was a potential tool that the team discussed reapplying to OR cases. However, it was decided that this would not be appropriate for this project [3]. Machine Learning at the Operating Room of the Future: A Comparison of Machine Learning Techniques applied to Operating Room Scheduling This MIT study evaluated the effectiveness of five machine-learning algorithms in predicting OR case times. The MIT study found that linear regression outperformed all five machine-learning algorithms in predicting operating room case times, though it is much simpler to implement. From this study, the team found that linear regression could be used to more accurately predict case time [4]. Swim-lane Diagrams: Methods To help in establishing the current state of the case scheduling process the team created swimlane diagrams. The swim-lane diagrams are on a location level. This served as a visual tool to identify who is involved in the process and what their responsibilities are in relation to determining and scheduling case times. Swim-lane diagrams are useful in highlighting potential issues in a process as well as showing key differences in a process between locations. Swim-lane Diagrams: Findings The swim-lane diagrams developed for each OR location flag key differences in the process between locations. Figure 4 is the swim-lane diagram developed for MH OR scheduling. 9

17 Figure 4: Mott swim-lane diagram This swim-lane diagram visualizes the complexity of the scheduling process. The timeline located at the top of the diagram shows the rough timing for each of the steps in the process. Cases are scheduled one to three months in advance but the case time may be adjusted in the days leading up to the procedure. Swim-lane diagrams for the other locations are all significantly different from each other. The key differences between locations are listed below. At EAA and KEC, the central scheduler determines the case time based on the case request At UH, case time is strictly determined by the case request At CVC and MH, the case time may be adjusted by either the scheduler or charge RN Add-on cases are handled by the OR front desk, charge RN, or central scheduler depending on location These differences are relevant to the overall process. The scheduled case time is how all procedures are scheduled. Accuracy of scheduled case time influences the entire day of procedures and can be a source of delays and overtime in OR s. The swim-lane diagrams for the other four locations are located in Appendix B. 10

18 Data Analysis: Methods The team was provided MiChart data from June 2014 to September The team first stratified the data by location and by service to see how many cases are underbooked, overbooked, and scheduled accurately in the current state. Next, the team performed a thorough analysis of possible factors that influence procedure time accuracy. The purpose of the analysis was to determine what factors significantly impact case time. Factors that are deemed influential to case time can be used to more accurately predict case times, and therefore result in more cases being scheduled accurately. This would decrease OR delays and overtime as well as improve patient satisfaction. To perform the analysis, the team used the MiChart data to compare case times across each factor. The percent time deviation from the scheduled time (or percent off ) was analyzed at different levels of each factor. This calculation captured the difference between scheduled and actual case time while eliminating the issue that total time different would be subject to the procedure length. These results identify which factors can influence case time accuracy. The factors analyzed are listed below. If a resident is present Patient class (Outpatient, Inpatient, Surgery admit) Anesthesia type Patient age Class of user creating case request Add-on or scheduled procedure Location of procedure Case class (Emergent, Scheduled, Urgent) Service performing procedure Case type ASA score Time of day These potential factors were collected through discussions with the client, review of prior projects, the project coordinators, surgeon survey responses, and interviews with OR nursing staff and OR schedulers. Next, the team formulated an optimization model. The goal was to find the optimal percentile to schedule procedures that would minimize delta (the difference between scheduled and actual case time). In addition, to determine the best method to predict scheduled case times, the team split the data in two eight month segments. The team used the more recent half of the data ( build data ) to calculate the average, median, 60 th percentile, and 70 th percentile for each combination of OR location, surgeon service, procedure code, and surgeon. The team rounded the values to the nearest 15 minutes. Only combinations that occurred at least five times in the build data and at least 10 times in the complete data were examined. The team then used the calculated procedure times and examined how case time accuracy would be affected if the average, median, 60 th percentile, or 70 th percentile had been used instead of the original scheduled case times for the 11

19 other half of the data ( test data ). The same analysis was done separately on Cath/EP data and Interventional Radiology data. When the analysis was done on Interventional Radiology data, case times were not analyzed on an individual surgeon level, because many surgeons had insufficient sample sizes. Figure 5 showed how the data was segmented. Figure 5: Method for simulating alternative scheduling methods Next, the team examined how each of the alternative methods performed on OR case times that were distributed normally vs. non-normally. The team also examined if coefficient of variation was a predictor of accuracy improvement when using one of the alternative methods. Last, the team examined the effectiveness of each model by using linear regression. The team created a regression model for each method, where the predictor was the proposed scheduled case time under the method and the response variable was the actual case duration. The team examined R-sq values to see how much variation in actual case time the proposed models explain. Data Analysis: Findings For the data analysis, the team used the following definition: Procedures scheduled for less than 2 hours: scheduled case time is accurate if actual case time is within 15 minutes of scheduled case time Procedures scheduled for greater than or equal to 2 hours: scheduled case time is accurate if actual case time is within 30 minutes of scheduled case time 12

20 Current Situation By Location The results of stratifying by location are shown below in Table 3. Table 3: Accuracy By Location Location Accurate Underbooked Overbooked # Cases CVC OR 28% 50% 22% 3365 EASC OR 55% 30% 15% 6890 KEC OR 57% 33% 11% 7144 LSC OR 59% 15% 26% 4892 MH OR 45% 40% 15% UH OR 39% 44% 17% Overall 45% 38% 17% This analysis indicated that overall 45% of cases are scheduled accurately. For cases scheduled inaccurately, the majority of cases are underbooked. There was variation among locations. Accuracy ranges from only 28% at the CVC OR to 59% at LSC. The team noticed that KEC and LSC are the most accurate, and also have relatively low case volumes. However, while the CVC OR has low case volume, the CVC OR is the least inaccurate, most likely due to high procedure complexity. Current Situation By Service Next, a similar analysis stratified the data by service. Services with case volumes less than 150 in the last 16 months were removed. Results are shown in Appendix C. From the table in Appendix C, the team noticed that a lot of variation exists among services. Accuracy ranged from 26% in Thoracic Surgery to 78% in Physical Medicine and Rehab. Factor Analysis The factor analysis identifies which of the factors analyzed are statistically significant to case time accuracy. Factor analysis was performed using only OR case data from MiChart. Data from June 14 through September 15 was used. Cases performed at the VH OR and cases with anesthesia or radiology services were not included in the analysis. The sample size ranged from approximately 55 to 60 thousand records after removing any incomplete records with regards to the factor being analyzed. If Resident Present The project team analyzed whether the presence of a resident had an effect on the accuracy of the scheduled case time. This was done by comparing the "percent off" between cases that had a resident and those that did not. The interval plots comparing cases with and without residents is shown in Figure 6. 13

21 Figure 6: Accuracy with and without residents The results showed a statistically significant difference between cases with residents compared to those without. Therefore, the presence of a resident is a factor that affects accuracy of scheduled case times. Those cases with a resident are more accurate than those without. Patient class (Outpatient, Inpatient, Surgery admit) Patient class is an important characteristic of each case. It can be an indicator of the amount of recovery required for a particular case. The interval plots from the analysis of patient class can be seen in Figure 7 below. 14

22 Figure 7: Accuracy by patient class The findings from the patient class analysis indicate that there is a significant difference in accuracy based on the patient class. Surgery admit cases are typically scheduled more accurately than either outpatient or inpatient cases. Anesthesia type Anesthesia usage varies greatly depending on procedure and patient. Figure 8 shows the analysis of the different types of anesthesia. 15

23 Figure 8: Accuracy by anesthesia type Epidural and spinal anesthesia are typically the anesthesia used on cases that were scheduled less accurately. However, both of these anesthesia types have lower sample sizes. The two most common forms of anesthesia are general and MAC. There is not a significant difference in case time accuracy for these forms of anesthesia. The findings show that anesthesia type is not a significant factor in accuracy of procedure times. Patient Age Patient age was analyzed by comparing the patient age to the percent difference between scheduled and actual time for each case. The results showed that age did not influence the accuracy. This does not mean that older patients take longer or shorter than young patients, it simply means that they are scheduled just as accurately. The scatter plot for patient age vs. accuracy can be found in Appendix D. Results from patient age analysis show it is not a significant factor. User Creating Case Request To determine which staff roles schedule cases most accurately, the team examined the users creating case requests. The findings suggested that 63% of case times changed after the case request was submitted. Also, it was determined that, if the initial case request times were never changed, only 34.4% of cases would have been accurate. Table 4 shows how accurate the various staff roles would have been if the initial case request times had been used instead of the final scheduled times. 16

24 Table 4: Staff Performance in terms of scheduled case time accuracy Provider N % Accurate % Cases Requested Faculty 19,692 49% 38% PA/NP 6,460 44% 13% Resident 8,616 39% 17% RN 3,550 36% 7% OR Scheduler 3,440-7% Clinic Scheduler 9,243-18% N=51,001; June 14 through Sept 15; Filters: no UH IR, no VH OR, no anesthesia, no radiology From Table 4, the project team determined that faculty were most accurate in requesting case times, with 49% of case times being accurate. Furthermore, Table 4 shows that faculty only input the case requests 38% of the time. The project team also determined that OR and clinic schedulers do not usually input an initial case time when creating a case request, hence the accuracy values for OR and Clinic Schedulers were ignored for Table 4. To determine an acceptable way of comparing the accuracy of case times provided by OR and Clinic Schedulers, the project team used the assumption provided by the client to equate the scheduled case time as the case request time for schedulers. Based on this assumption, Table 5 shows the accuracy of case times provided by OR and Clinic Schedulers. Table 5: Scheduler Performance in terms of scheduled case time accuracy Provider N % Accurate OR Scheduler 3, % Clinic Scheduler 9, % The number of case requests by each scheduler in Table 5 slightly differs from the corresponding values in Table 4, due to incomplete data points that were filtered out when determining scheduler performance. Also, the findings seen in Appendix E show that, if the default time was used by schedulers for case requests, 59.9% of cases scheduled by OR and Clinic Schedulers would be accurate. Faculty requested cases more accurately than other staff roles in general and requested 38% of cases. This does vary slightly by service and those findings are in Appendix E. From this study, the project team determined that faculty is the best performing staff overall, in terms of requesting accurate case times. However, other staff roles request case times more accurately than faculty within some individual services. The service where faculty is least accurate with case requests is in neurosurgery, with 22% of initial case request times being accurate. Add-on or scheduled procedure Analysis of add-on versus scheduled procedures examines if there is a difference in accuracy for the two scenarios. The results indicate that add-on cases are scheduled more accurately. Figure 9 shows the percent difference between scheduled and actual case times for add-on cases. 17

25 Figure 9: Accuracy of scheduled vs. add-on cases Statistically, there is a significant difference between add-on and scheduled cases. However, the difference is small and not practically significant. Case class (Emergent, Scheduled, Urgent) Case class is a significant indicator of the urgency of a procedure. Emergent cases are add-on cases that have to be performed immediately. The team hypothesized that emergent patients are often in more critical condition with a higher chance of complications, so emergent cases may be more accurate. However, the findings do not support this thought. The only statistically significant difference between among case classes is between urgent and scheduled, with urgent cases being scheduled 3% closer to actual case time on average. This is shown in Figure 10 below. 18

26 Figure 10: Accuracy by case class The findings indicate that case class is a significant factor in case time accuracy. Service performing procedure The project team investigated which service had the most accurate surgeons in terms of case time accuracy. Table 6 shows the services with the most number of surgeons in the top 50 most accurate surgeons for scheduled case time. Table 2: Top 50 surgeons in terms of case time accuracy Service # of Surgeons Ophthalmology 14 Obstetrics and gynecology (OBGYN) 11 Pediatric Hematology/Oncology 5 Otolaryngology 4 Physical Medicine and Rehab 3 Urology Surgery 3 Orthopedics 3 From Table 6, Ophthalmology and OBGYN had the most accurate surgeons, accounting for half of the top 50 surgeons. Surgeons within these two services generally complete procedures within the accuracy thresholds. This also indicates that the scheduling process within these two departments may be more consistent and tailored to suit the surgeons needs and preferences. 19

27 Case type Analysis of case type revealed that over 95% of cases in the scope of this project (excluding IR) are OR cases. Therefore case type is not a significant factor in case time accuracy. There is no additional analysis of case type. ASA score ASA score is a way to rate the patient s physical condition. There is a rating scale, 1-6 with 6 being the most severe. Patients with a score of 1 are healthy and 6 is brain dead. For the purposes of this analysis, the team removed cases with a rating of 5 or 6 because of the small sample sizes as well as the unpredictable nature of any of these cases. A complete description of all ASA scores is included in Appendix F. Below in Figure 11 is the comparison of cases rated 1-4. Figure 11: Accuracy by ASA Score The results show that patients with an ASA score of 4 tend to be scheduled most accurately, but only by 0.5% to 3%. These findings also suggest that aside from a score of 4, there is very little difference in accuracy between ASA scores one through three. ASA scores are subjective and depend on the person giving the score. Therefore the team feels that although ASA score is not a significant factor in determining case time accuracy due to the small difference and the subjective process of assigning scores. Time of Day The team stratified the data by looking at the case time accuracy of cases scheduled for various times of the day. First, the project team defined an inaccurate case as: a) If scheduled case time is less than or equals to 2 hours, the absolute difference between scheduled and actual case times exceeds 15 minutes, OR 20

28 b) If scheduled case time exceeds 2 hours, the absolute difference between scheduled and actual case times exceeds 30 minutes. Next, the project team determined the number and accuracy of all cases scheduled during various parts of the day, as seen in Table 7. Table 7: Accuracy of scheduled case times during various parts of the day Time of Day Number of Cases % Inaccurate OVERNIGHT (00:00 06:59) :00-8:59 19, :00-10:59 13, :00-12:59 12, :00-14:59 9, :00-17:00 3, NIGHT (17:01 24:00) 1, From Table 7, the findings showed that the case time accuracy was worst at the start of a day, gradually improving towards noon, before gradually worsening until night. Alternative Scheduling Methods By formulating an optimization problem to determine the optimal percentile to schedule cases to for several different scenarios, the team found that it was impractical due to large data sets. In order to identify the best method for scheduling cases, the team used the historical data to calculate how accurate cases would have been scheduled if an alternative method was used. The alternatives are the historical average, median, 60 th percentile, and 70 th percentile. Improvements could then be measured and compared to the current state. The team used the most recent eight months to calculate the new scheduled times, and applied these times to the previous eight months (June 14 to January 15). The results for OR cases are shown below in Table 8. Table 8: Simulated use of procedure time averaging on OR Data Method Accurate Overbook Underbook Original Scheduled 48% 15% (2,883 h/yr or 10 min/case) Average 56% 25% (5,087 h/yr or 17 min/case) Median 59% 19% (3,932 h/yr or 13 min/case) 60th Percentile 56% 25% (5,522 h/yr or 19 min/case) 70th Percentile 51% 34% (7,889 h/yr or 27 min/case) 37% (9,472 h/yr or 32 min/case) 19% (5,262 h/yr or 18 min/case) 22% (6,076 h/yr or 21 min/case) 19% (4,988 h/yr or 17 min/case) 15% (3,897 h/yr or 13 min/case) Build data: >=5 cases per location, service, procedure, surgeon combination in build and >10 over 16 months; N=31,013; Date: 2/1/15-10/2/15; Filters: no UH IR, no VH OR, no anesthesia, no radiology, no IR case type. Test data: N=17,716; Date: 6/7/14-1/31/15; Filters: no UH IR, no VH OR, no anesthesia, no radiology, no IR case type. 21

29 Table 8 showed that the greatest improvement to accuracy occurs when scheduling to the historical median. When using median accuracy improves from 48% to 59%. Similar accuracy improvements are reflected in both Cath/EP and IR cases. Summary tables for these cases are included in Appendix G. The team investigated if the accuracy improvements shown above in the overall OR data was echoed on a location level. The breakdown of accuracy on a location level is shown in Table 9. Table 9: Simulated use of procedure time averaging by location Location N Accurate Underbook Overbook CVC OR 720 Current 23% 53% 24% Average 41% 24% 35% Median 41% 27% 31% 60th Percentile 40% 21% 39% 70th Percentile 33% 17% 50% EASC OR 1,988 Current 57% 30% 13% Average 61% 16% 23% Median 62% 19% 19% 60th Percentile 61% 15% 24% 70th Percentile 58% 12% 30% KEC OR 2,747 Current 58% 34% 8% Average 68% 16% 16% Median 71% 19% 10% 60th Percentile 68% 17% 15% 70th Percentile 64% 13% 24% LCSC OR 1,974 Current 64% 13% 23% Average 81% 12% 7% Median 82% 13% 5% 60th Percentile 81% 13% 6% 70th Percentile 81% 9% 10% MH OR 4,140 Current 48% 40% 12% Average 56% 18% 26% Median 60% 22% 19% 60th Percentile 57% 18% 25% 70th Percentile 51% 14% 35% UH OR 6,147 Current 39% 45% 17% Average 43% 24% 33% Median 46% 28% 26% 60th Percentile 42% 23% 34% 70th Percentile 37% 18% 45% The results shown in Table 9 prove that average and median yielded a greater accuracy percentage than the current situation at each location. Median performed the same or slightly better than average at all locations. The fact that accuracy improves at all locations when using median supports the use of procedure time averaging across all locations. It is important to note that both LSC and CVC (the best and worst performing locations) showed significant improvements. 22

30 Coefficient of Variation An expert in the healthcare-scheduling field suggested that the team consider using coefficient of variation to determine which method would be most appropriate for a certain procedure. To differentiate between the current situation, mean, and median, the team found that there was a weak correlation between coefficient of variation and the absolute value of percent difference between scheduled and actual case times. The correlation values can be found in Appendix H. Because of the weak correlation, the coefficient of variation does not predict what procedures could benefit from procedure time averaging. Normality The team found that normally and non-normally distributed procedures had similar improvement under each of the methods for predicting scheduled case time. Therefore, all procedures are likely benefit from procedure time averaging, regardless of normality of case times. Regression The team also evaluated the performance of each of the scheduling methods using regression. The team put the scheduled time as the predictor and the actual time as the response value. The R-sq values and total squared errors for each alternative scheduling method were similar to the current state. These results are included in Appendix I. The major findings from both the qualitative and quantitative analyses led to the conclusions detailed in the following section. CONCLUSIONS Based on the findings from all of the various parts of the project, the team formed several conclusions. These are segmented into conclusions based on the qualitative methods and then the quantitative methods. Qualitative Conclusions The employee interviews found that schedulers and surgeons are unaware of their performance in regard to scheduling accuracy. It is a lack of feedback that drives this. OR schedulers understand the importance of cases being completed on time. However, they do not if the cases they schedule are completed in the allotted time. Similarly, nurses feel that surgeons are not usually conscious of how long procedures take. With the proper feedback both surgeons and schedulers could gain an understanding of their scheduling accuracy and the impact that inaccurate cases have on the OR and patient care. Surgeon perception and thoughts were captured in the surgeon survey. These results support these conclusions. First, surgeons believe that case time is accurate more often than it actually is. This confirms the nurse and schedulers thoughts that surgeons are not always aware of how long procedures actually take. Perception of the surgeons does not represent the reality of case time accuracy. Also, surgeons work best when they have a team of familiar anesthesiologists and nurses. It could improve procedure times if surgeons were able to work with the same team on a regular basis. This would lead to improved scheduling accuracy as case times become more predictable. Variation in setup and cleanup time was also a common complaint of the surgeons. 23

31 This falls outside of the defined scope of this project but it is an important factor to note. Additionally, nurse shift changes were identified as a factor that can affect case time. Finally, surgeons feel that there are many factors that can influence case time with procedure complexity being the most significant factor to case time. These factors are not always predictable and that explains some of the variability in procedure times. The swim-lane diagrams of the case scheduling process illustrates the variation in the current state. Current state findings led to several conclusions that are key to improving the process moving forward. Each location varies greatly from the others. This shows the lack of standardization. However, the needs of each location differs, as well as the volume of cases varies greatly. These two factors mean that a uniform standard process across all locations is not practical. Also, it can be concluded that the people responsible for adjusting case time varies location to location. The data analysis shows that CVC has the lowest percent of cases that are completed accurately. Combining this knowledge with the swim-lane diagrams, at CVC the OR central schedulers adjust case times. The results of this at CVC are not good. However, at EASC the central scheduler also adjusts case times and the cases are accurate 55% of the time compared to 28% at CVC. This shows that there is an opportunity to improve scheduling at CVC in particular, as well as overall. Swim-lane diagrams also show that most of case reviews in reference to time take place one to three days before the procedure. This is important because if cases are flagged as having an abnormal case time, there is little flexibility in the schedule to make major adjustments at the busier locations. For example at Mott or UH, cases are scheduled months in advance and if a case is identified as having an odd procedure time by the charge nurse the day before it is scheduled to take place, there is little flexibility in the OR to accommodate changes without disrupting other cases. These findings support the need to improve accuracy of case times. Quantitative Conclusions Data analysis findings were crucial to establishing the current state, determining what procedures could benefit from procedure time averaging, identifying factors significant to case time, and quantifying the improvement opportunity in using procedure time averaging. Each method of data analysis yielded significant findings that translate to the conclusions included below. The current state of case time accuracy was found to be 45%. It can be concluded that the health system has significant improvement opportunity across all locations. The best performing location was LSC where 59% of case times are accurate. Also, 38% of cases are being underbooked. This means cases are taking longer than they are scheduled for. Patient care, delays, and OR staffing issues are three major areas impacted by underbooked cases. These components of the OR can significantly improve by increasing accuracy of case times and minimizing underbooked cases. Additionally, the analysis of case time accuracy by service showed that there is significant variation in accuracy depending on the service performing the procedure. This finding supports the conclusion that type of procedure is a driving factor in case time variability and complex cases are more likely to be scheduled inaccurate. Factor analysis showed that eight of the factors considered are statistically significant to case time accuracy. However, the findings as a whole suggest that little of case time variation is explained by these factors. The team concludes that these factors are not a strong predictor of 24

32 case time and consequently should not be used as such. In addition, the factor analysis exposed that surgeons only complete the initial case request 38%, though they input case times most accurately. This leads to the conclusion that surgeons should be completing case requests and predicting case time. The team concluded that formulating an optimization model to minimize the difference between scheduled and actual case times was impractical when dealing with large datasets spanning various locations, services, procedures, and surgeons. Finally, the improvement opportunity of procedure time averaging was determined in the data analysis. The team found that overall accuracy could improve by at least 11% while reducing underbooked cases by 15%. This improvement would come while improving accuracy at each location. Median, as the scheduled time, shows the greatest improvement opportunity. Analysis of both coefficient of variation and normality were attempts to identify procedures that would benefit more than others from procedure time averaging. However, the findings show that neither coefficient of variation or normality are indicators of case time predictability. Improvement opportunity was similar between procedures regardless of coefficient of variation and normality. Therefore it can be concluded that no specific procedures benefit more from procedure time averaging than others. RECOMMENDATIONS Using the conclusions as a basis, the team has developed recommendations aimed to improve accuracy of scheduled OR, Cath/EP, and IR case times. These recommendations are segmented into qualitative and quantitative and included below. Qualitative Recommendations As a way of improving understanding of case time accuracy, the team recommends providing a dashboard that will display scheduling accuracy performance. This would be used by both schedulers and surgeons to monitor their performance and progress. Also, this would serve to make schedulers and surgeons more aware of the impact of cases being over and underbooked. This combats the surgeon perception that they consistently feel that they complete cases within the scheduled time range more often than the data suggests. It was concluded that surgeons work best with familiar anesthesiologists and nurses. Knowing this, exploring more ways to better pair surgeons with familiar teams could improve case times, making them more accurate and predictable. Pairing surgeons with familiar teams is difficult and presents many scheduling difficulties but the benefits would be seen across the OR and in patient care. Also, future investigations into factors that were identified as impacting case times but fell outside the scope of this project would be appropriate future work. The out of scope factors identified were room setup and cleanup, nursing shift changes, and PACU availability. A recommendation addressing the lack of a standard scheduling process across locations is to work with schedulers and OR staff to clarify responsibilities related to determining case times. Currently, most locations do not have a formal review process for case times. The team does not believe it is appropriate to implement a single standard scheduling process across all locations because of the varying needs for each location. However, increasing education and awareness 25

33 about case times and how to review and adjust them would improve accuracy and understanding. This paired with the recommended dashboard can take advantage of the experiences and data when reviewing OR schedules improving accuracy. Quantitative Recommendations Based on the current state and improvement opportunity, the team recommends using historical median as the default case time for all single procedure, single panel cases. Also, the team recommends using the historical median for IR and Cath/EP cases. In addition, turning on procedure time averaging functionality within MiChart will allow surgeons to gain a frame of reference when predicting case times. By coupling the median with faculty expert opinion there is potential to increase accuracy to levels beyond those predicted with just using median. This combination of expert experience and data analytics will also allow for tracking of scheduling performance that can be used to educate and improve the process. Median also effectively mitigates bias towards outliers, while not eliminating them from consideration. Conclusions from factor analysis supports the recommendation for surgeons to input all case requests. Surgeons are most accurate in predicting case times but only create 38% of case requests. However, it is not practical to require surgeons to complete all case requests. Based on this reality, the team recommends allowing some other roles, PA s, Residents, and Nurses to complete case requests. Related, do not have clinic or central schedulers perform case requests because of the low accuracy results for these roles. Finally, the factor analysis indicated that eight of the factors are significant, but the team recommends not using these factors in predicting case times. This is because of the complexity of medical procedures, the factors while significant, only explain a small portion of the variation in case time. These recommendations are based on the goal to improve accuracy of scheduled case times. It is important to note that this project did not focus on minimizing overtime costs or maximizing OR utilization. It is reasonable to suggest that these improvements may have positive impacts in these areas, but the analysis was not dedicated to these metrics. It is with confidence that the team s recommendations can be made in order to improve accuracy of scheduled case times. EXPECTED IMPACT The expected impact of implementing these recommendations would impact three major areas within the OR. First, patient satisfaction would increase with case time accuracy improvements. More accurate scheduled case times would reduce patient wait times thus improving patient care and satisfaction. Second, more accurate case times could improve staff morale. With more accurate OR schedules, unplanned overtime and the corresponding stress of OR staff would decrease. Finally, more accurate case times would improve OR efficiency. This would be recognized with more on-time starts for procedures and greater flexibility in day of OR schedule changes. With more cases ending before scheduled, more operating rooms will be available to shift cases into when a preceding case runs over the scheduled time. Currently this would cause delays in all subsequent cases scheduled for that room. The increase in procedures ending earlier 26

34 than scheduled could also make rooms available for more add-on cases. These are three areas that could see positive impacts if scheduled case times become more accurate. REFERENCES [1] University of Michigan Health System, Patient Care at the University of Michigan Health System. [Online]. Available: [Accessed : 14 Dec. 2015]. [2] K. Bukowski and P. Mehta, "Analysis of OR Case Times at UMMC. University of Michigan Health System: Ann Arbor, Michigan. Report. 5 Dec [3] Kaniz, Pigula, Dentrinos, and Spindell. Room Scheduling Analysis for Interventional Radiology. University of Michigan Health System. Report. 17 Apr [4] S. Davies. Machine Learning at the Operating Room of the Future: A Comparison of Machine Learning Techniques applied to Operating Room Scheduling. Massachusetts Institute of Technology EECS Dept: Cambridge, Massachusetts. 26 May

35 APPENDIX A: Summary From Employee Interviews Location Process Factors Affecting Accuracy UH OR OR personnel rarely adjust case times Scheduler does not directly deal with surgeons Front desk handles add-on cases Schedule locks at 11AM day before Charge RN reviews next day schedule to prepare equipment Will flag odd case times and notify either the floor runner, anesthesia, or the surgeon Room swapping is common day of procedure based on equipment needs and room availability EASC OR OR Scheduler adjusts case times based on past experience OR Scheduler handles add-on cases Room swaps driven by available equipment Schedule is finalized 2 days before MH OR Room is based on equipment Charge RN will clinic scheduler and surgeons when there are potential issues with case time Scheduler believes it is best when the surgeon personally inputs case request Schedule is finalized 11AM day before Believes clinic schedulers do not know if cases are scheduled accurately Charge RN will adjust bone marrow procedure times (scheduled for too long) Charge RN handles add-on cases Kellogg OR RN OR Manager runs OR schedule Will add time for new surgeons until they prove they can complete it faster Charge RN/Anesthesia handle addons Surgeon Scheduler Wrong DPC Patients arriving late Time for anesthesia to find vein Surgeon Kids are more prone to not follow instructions such as eating before and operation Surgeon Complications during procedure Complexity of case Complications 28

36 CVC OR Clinical Care Coordinators input case request and update case times Central Scheduler places case into schedule Central Scheduler will adjust case times and send a note to the clinical care coordinator to verify that it is appropriate Limited room shuffling Charge RN does not adjust schedule OR front desk handles add-ons Size of patient New procedure techniques IR cases more difficult to predict 29

37 APPENDIX B: Swim-lane Diagrams 30

38 31

39 32

40 33

41 APPENDIX C: Current Accuracy Situation by Service Service Accurate Underbook Overbook # Cases Acute Care Surgery 35% 40% 25% 1891 Colorectal Surgery 41% 43% 16% 599 Dental 36% 35% 29% 527 Dermatology 87% 13% 0% 114 General Surgery Endocrine 54% 40% 6% 1998 General Surgery Oncology 52% 28% 19% 1918 General Surgery Transplant 39% 41% 20% 1034 Hepatopancreatobiliary Surgery 36% 50% 14% 757 Internal Medicine Rheumatology 54% 4% 42% 52 Minimally Invasive Surgery 49% 39% 12% 927 Neurosurgery 28% 53% 19% 3178 Obstetrics and Gynecology 48% 29% 23% 3235 Ophthalmology 56% 32% 12% 9631 Oral Surgery 37% 40% 22% 1379 Orthopaedics 42% 48% 10% 8112 Otolaryngology 50% 34% 16% 6618 Pediatric GI Medicine 36% 63% 1% 1077 Pediatric Hematology/Oncology 61% 9% 30% 282 Pediatric Nephrology 63% 20% 17% 59 Pediatric Pulmonary Medicine 41% 14% 45% 176 Pediatric Surgery 37% 39% 23% 2389 Physical Medicine and Rehab 77% 16% 7% 713 Plastic Surgery 44% 36% 20% 4165 Thoracic Surgery 26% 50% 24% 498 Urology Surgery 47% 32% 21% 6295 Vascular Surgery 28% 54% 18% 987 Grand Total 45% 38% 17% N=60,127; Dates: June 14 through September 15 MiChart Data; Only OR Locations and only services with > 50 cases 34

42 APPENDIX D: Patient Age vs. Accuracy Scatterplot of Delta (%) vs Patient Ag e Delta (% ) Patient Age 35

43 APPENDIX E: Case time accuracy by staff role and service If default case times were used by schedulers Provider Accurate Cases Cases Scheduled Percent Accurate Clinic Scheduler 1,623 2, % OR Scheduler 848 1, % Total 2,471 4, % Service Provider Accurate Cases Percent Cases Requested Accurate General Surgery Endocrine PA/NP % Urology Surgery PA/NP % Minimally Invasive Surgery PA/NP % Colorectal Surgery PA/NP % Ophthalmology Faculty % Physical Medicine and Rehab Faculty % General Surgery Oncology Faculty % Pediatric Surgery PA/NP % Obstetrics and Gynecology PA/NP % Pediatric GI Medicine Resident % Otolaryngology Faculty % Plastic Surgery Faculty % Orthopaedics Faculty % Cardiac Surgery Faculty % General Surgery Transplant Faculty % Oral Surgery Resident % Vascular Surgery PA/NP % Hepatopancreatobiliary Surgery Faculty % Acute Care Surgery Resident % Thoracic Surgery RN % Neurosurgery Clinic Scheduler % 36

44 APPENDIX F: Descriptions of ASA Scores ASA PS Classification Definition Examples, including, but not limited to: ASA I A normal healthy patient Healthy, non-smoking, no or minimal alcohol use ASA II A patient with mild systemic disease Mild diseases only without substantive functional limitations. Examples include (but not limited to): current smoker, social alcohol drinker, pregnancy, obesity (30 < BMI < 40), well-controlled DM/HTN, mild lung disease ASA III A patient with severe systemic disease Substantive functional limitations; One or more moderate to severe diseases. Examples include (but not limited to): poorly controlled DM or HTN, COPD, morbid obesity (BMI 40), active hepatitis, alcohol dependence or abuse, implanted pacemaker, moderate reduction of ejection fraction, ESRD undergoing regularly scheduled dialysis, premature infant PCA < 60 weeks, history (>3 months) of MI, CVA, TIA, or CAD/stents. ASA IV A patient with severe systemic disease that is a constant threat to life Examples include (but not limited to): recent ( < 3 months) MI, CVA, TIA, or CAD/stents, ongoing cardiac ischemia or severe valve dysfunction, severe reduction of ejection fraction, sepsis, DIC, ARD or ESRD not ASA V ASA VI A moribund patient who is not expected to survive without the operation A declared brain-dead patient whose organs are being removed for donor purposes undergoing regularly scheduled dialysis Examples include (but not limited to): ruptured abdominal/thoracic aneurysm, massive trauma, intracranial bleed with mass effect, ischemic bowel in the face of significant cardiac pathology or multiple organ/system dysfunction Table taken from the American Society of Anesthesiologists Website referenced below: American Society of Anesthesiologists, ASA Physical Status Classification System, 2014 Oct. 15. [Online]. Available: [Accessed: 13 Nov. 2015]. 37

45 APPENDIX G: Simulated use of case time averaging (Cath/EP and IR) Cath/EP: Method Accurate Overbooked Underbooked Original Scheduled 48% 25% (845 h/yr or 15 min/case) Average 53% 23% (787 h/yr or 14 min/case) Median 58% 18% (579 h/yr or 10 min/case) 60th Percentile 51% 25% (859 h/yr or 15 min/case) 70th Percentile 48% 30% (1,083 h/yr or 20 min/case) 27% (2,157 h/yr or 39 min/case) 23% (1,455 h/yr or 26 min/case) 24% (1,582 h/yr or 29 min/case) 23% (1,465 h/yr or 26 min/case) 22% (1,395 h/yr or 25 min/case) Build Set: N>=5 overall; Calculated for each location, service, procedure combination; Date: 2/1/15-10/2/15; N=5,198. Test Set: N=3,329; Date: 7/1/14-1/31/15 Interventional Radiology: Method Accurately Overbooked Underbooked Original Scheduled 47% 18% (682 h/yr or 9 min/case) Average 49% 27% (1,154 h/yr or 16 min/case) Median 52% 18% (676 h/yr or 9 min/case) 60th Percentile 50% 27% (1,060 h/yr or 14 min/case) 70th Percentile 42% 42% (1,860 h/yr or 25 min/case) 34% (2,288 h/yr or 31 min/case) 24% (1,468 h/yr or 20 min/case) 30% (1,784 h/yr or 24 min/case) 23% (1,461 h/yr or 24 min/case) 16% (1,092 h/yr or 15 min/case) Build Set: N>=5 for each service, location, procedure combination; Date: 2/1/15-10/2/15, N=6136; Filters: Service = IR, case type = IR. Test Set: Date: 6/9/14-1/30/15; N: 4401; Filters: Service = IR, case type = IR 38

46 APPENDIX H: Coefficient of Variation vs. Scheduling Methods Method R-Sq % Off Scheduled (Current) 3.16% % Off Scheduled (Average) 4.88% % Off Scheduled (Median) 5.16% Build data: >=5 cases per location, service, procedure, surgeon combination in build and >10 over 16 months; N=31,013; Date: 2/1/15-10/2/15; Filters: no UH IR, no VH OR, no anesthesia, no radiology, no IR case type. Test data: N=17,716; Date: 6/7/14-1/31/15; Filters: no UH IR, no VH OR, no anesthesia, no radiology, no IR case type. Test data shown in graph. 39