Dynamic optimization of chemotherapy outpatient scheduling with uncertainty

Health Care Manag Sci (2014) 17:379 392 DOI 10.1007/s10729-014-9268-0 Dynamic optimization of chemotherapy outpatient scheduling with uncertainty Shoshana Hahn-Goldberg & Michael W. Carter & J. Christopher Beck & Maureen Trudeau & Philomena Sousa & Kathy Beattie Received: 15 August 2013 /Accepted: 15 January 2014 /Published online: 30 January 2014 # Springer Science+Business Media New York 2014 Abstract Chemotherapy outpatient scheduling is a complex, dynamic, uncertain problem. Chemotherapy centres are facing increasing demands and they need to increase their efficiency; however there are very few studies looking at using optimization technology on the chemotherapy scheduling problem. We address dynamic uncertainty that arises from requests for appointments that arrive in real time and uncertainty due to last minute scheduling changes. We propose dynamic template scheduling, a novel technique that combines proactive and online optimization and we apply it to the chemotherapy outpatient scheduling problem. We create a proactive template A portion of this research was presented at the IIE SHS conference in February 2012, where it won the graduate student paper prize and appears in their non-published proceedings as: Hahn-Goldberg S, Carter MW, Beck JC. Dynamic Template Scheduling To Address Uncertainty in Complex Scheduling Problems: A Case Study on Chemotherapy Outpatient Scheduling. It can be downloaded from IIEnet2.org S. Hahn-Goldberg (*): M. W. Carter : J. C. Beck Centre for Research in Healthcare Engineering, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada e-mail: shoshana@mie.utoronto.ca M. W. Carter e-mail: carter@mie.utoronto.ca J. C. Beck e-mail: jcb@mie.utoronto.ca M. Trudeau: P. Sousa : K. Beattie Odette Cancer Centre, Sunnybrook Health Sciences Centre, Toronto, ON, Canada M. Trudeau e-mail: Maureen.Trudeau@sunnybrook.ca P. Sousa e-mail: Philomena.Sousa@sunnybrook.ca K. Beattie e-mail: Kathy.Beattie@sunnybrook.ca of an expected day in the chemotherapy centre using a deterministic optimization model and a sample of appointments. As requests for appointments arrive, we use the template to schedule them. When a request arrives that does not fit the template, we update the template online using the optimization model and a revised set of appointments. To accommodate last minute additions and cancellations to the schedule, we propose a shuffling algorithm that moves appointment start times within a predefined time limit. We test the use of dynamic template scheduling against the optimal offline solution and the actual performance of the cancer centre. We find improvements in makespan of up to 20 % when using dynamic template scheduling compared to current practice. Keywords Optimization. Health care. Chemotherapy. Scheduling. Uncertainty 1 Introduction In recent years many health care systems have focused on wait times and efficient provision of services. In Canada, this is evident in the work promoted by the Canadian Institute for Health Information, an organization that has worked with governments to improve information on access and wait times [1]. As the population ages, there will be an increased demand for medical care, including cancer care and, specifically, chemotherapy. The Canadian Press [2] reported that from 2003 to 2008, Ontario had more than doubled its chemotherapy budget, yet wait times had not decreased. During the time measured, there was an increase in patients being treated and retreated. While the system was able to absorb more patients without increasing the wait time, as the population ages, there will be even more patients requiring chemotherapy and the system will need to adjust to accommodate the increase.

380 S. Hahn-Goldberg et al. The Odette Cancer Centre, a large cancer centre in Toronto, Canada and the case study used in this paper, has a growing demand for chemotherapy services. The chemotherapy centre wants to be able to service its demand with its existing resources. The complexity of the chemotherapy outpatient scheduling problem makes it an interesting problem from the perspective of optimization research as there are many aspects of the problem that present a challenge to traditional optimization techniques. First, there is large variability among treatments. A given chemotherapy treatment consists of a regimen: a particular set of drugs, dosages, methods of administration, and timetable for administration. The Odette Cancer Centre treats patients who follow over hundreds of different regimens. Treatment durations canrangefrom15minto8hforasingletreatment.second, chemotherapy has a unique property where nurses are required to attend to a single patient during parts of their treatment, yet can be shared among patients during other parts. Third, appointment requests arrive in real time and must be scheduled before all appointments for a given day are known. Due to the high variability in appointment durations, it is difficult to produce an efficient schedule a priori. Fourth, there is uncertainty due to last minute additions and cancellations. Fifth, the chemotherapy centre must provide patients with sufficient notice of their appointment time, meaning that appointment times cannot be changed at the last minute. Lastly, the chemotherapy scheduling problem at the Odette Cancer Centre is large, with around 100 patient appointments each day, 29 treatment chairs, and 13 to 15 nurses. Optimization techniques that take into account some of the complex features of the chemotherapy problem are generally tested on smaller problem instances. This paper defines the chemotherapy outpatient scheduling problem and solves it using dynamic template scheduling, a novel technique for incorporating the dynamism in the problem domain into the scheduling of chemotherapy appointments. By using an optimization model of the deterministic version of the chemotherapy outpatient scheduling problem and a sample of the future, a day s expected appointments are scheduled into an optimal template of open slots, which is then used for booking appointments. When a request for an appointment arrives that does not have a matching slot in the template, the template is dynamically updated using a new sample and the same deterministic optimization model. As an optional addition to dynamic template scheduling, this paper also details a shuffling algorithm used to move scheduled appointment times by a pre-determined time interval to accommodate additions and cancellations to a day that already has scheduled appointments. 2 Background In this section, we present relevant research in the areas of outpatient appointment scheduling and in particular, chemotherapy outpatient scheduling, and the general category of scheduling under conditions of uncertainty. We also present an overview of constraint programming, the optimization technique used in this research. 2.1 Outpatient scheduling Outpatient appointment scheduling involves scheduling patients into a primary care or outpatient clinic. The standard method of scheduling outpatient appointments is to use an appointment rule that is comprised of two parts: the appointment block size refers to how many patients are scheduled at a time and the appointment interval that refers to the time between appointment start times [3]. Appointment rules may also specify an initial block size, the number of patients that are given the first appointment in a session, which may differ from the general block size. Patients are typically scheduled for an appointment with a single server [3]. Although the allotted time per patient is equal for all patients, the actual duration of an appointment is unknown. There is also uncertainty due to patients requiring urgent appointments, walk-ins, and patients not showing for their appointments, no-shows. Approaches for appointment scheduling either include service time distributions in a stochastic model [4, 5]or test appointment rules in various situations using simulation [6, 7]. On the surface, the chemotherapy appears to be an appointment scheduling problem that requires the optimal scheduling of outpatient appointments. Both the outpatient appointment scheduling problem and the chemotherapy problem are dynamic; however, the chemotherapy problem is much more complex because it has multiple stages, many servers, and known appointment durations that differ significantly. Therefore, the queuing models that work well for appointment problems would not be able to handle the complexity of the chemotherapy problem. 2.2 Chemotherapy scheduling research Research on scheduling and sequencing chemotherapy appointments is limited. Recently, Santibanez et al. [8] developed a scheduling system for a large chemotherapy clinic in British Columbia, Canada. The paper is the culmination of a three year study including a review of the system they developed using simulation and mixed integer programming models. The simulation addressed the problem of assigning a date to each request for a first chemotherapy appointment in order to shorten the wait list. The mixed integer programming model looked at scheduling 1 day of appointments with the objective of finding a schedule that minimized the maximum nurse workload with an even workload among the nurses. The mixed integer programming model only considered the treatment stage; however, optimal solutions to the problem were found. They achieved significant operational improvement in

Dynamic optimization of chemotherapy outpatient scheduling with uncertainty 381 both the lead time of when patients are notified of their appointments, with a 57 % reduction in the number of patients exceeding the target of 1 week; and the wait list for first appointments, with an 83 % reduction in waitlist size. Turkcan et al. [9] created an integer programming model to solve a version of the deterministic chemotherapy outpatient scheduling problem. With problems of 50 patients a day, they judged the integer programming solve time as too long for their requirements. They therefore used a longest treatment time first heuristic to decrease the solution time. They focused on the broader scheduling task of deciding which appointments to service on which day and included a model to solve the problem of sequencing the appointments on a given day. However, their main objective was to balance the acuity of patients serviced by a particular nurse as opposed to maximizing utilization. As well, pharmacy constraints were not considered, only four types of cancers were serviced, and the problem instances were smaller than those considered here. 2.3 Scheduling under uncertainty In the last 15 years, under the heading of robust scheduling, techniques for solving problems with uncertainty have been a focus of many researchers [10 17]. Scheduling for uncertainty approaches fall into three categories: proactive, where an initial schedule takes into account uncertainty; reactive, where jobs are rescheduled after an uncertain event occurs; and predictive-reactive, a combination of the two [18 20]. Techniques for proactive scheduling include adding idle time into the schedule [10, 16, 21] and including a measure of problem variability in the schedule [14]. The approaches used are both exact [14, 22] as well as heuristic approaches [10, 16]. Reactive scheduling involves making decisions as information is revealed dynamically. Also termed online scheduling, online algorithms use whatever information is available at the time along with past assignments to make decisions online [23]. Approaches for reactive scheduling may use rules to decide which jobs to schedule [11] and may include exact techniques in order to make optimal decisions [24]. Predictive-reactive approaches are also a combination of exact [25] and heuristic [12, 13, 19, 26] methods. In response to the dynamic nature of many real world problems, Van Hentenryck and Bent [17] developed a class of algorithms that combine proactive and online optimization for a variety of problems. Decisions are made online at each point by solving a deterministic optimization model of a sample of the future. Online stochastic combinatorial optimization is very similar to the approach taken in this paper. Although there is a large amount of research on scheduling with uncertainty, as far as we are aware such techniques have never been applied to the chemotherapy outpatient scheduling problem. 2.4 Constraint programming Constraint programming is described as the study of computational systems based on constraints [27]. A constraint satisfaction program is a set of variables each with a finite domain and a set of constraints that restrict which values variables can take. Standard constraint programming solving is a combination of search, looking for an assignment of values to variables, with propagation at each node, checking to be sure the current values in the domains of each variable are viable. One of the advantages of using constraint programming is the ability to use global constraints, constraints with propagation algorithms developed specifically for them so that the propagation does more pruning or is less costly than if the same constraint was expressed using several smaller constraints [27]. The disjunctive and cumulative constraints [28] are global constraints that can be used to model resource constraints. The disjunctive constraint ensures that activities allocated to the same resource do not overlap in time. The cumulative constraint accomplishes the same objective for a resource with capacity greater than one. There is significant research within constraint programming that looks at creating propagation and search algorithms for scheduling problems and constraint programming is acknowledged as a technique well suited for scheduling problems [29, 30]. The specially designed propagation and search algorithms that make constraint programming a good choice for scheduling problems are complemented by the frameworks that exist for implementing constraint programming models. Comet, the framework used in this dissertation, has a library of global constraints, has algorithms designed for scheduling problems built into a specific scheduling module, separates modelling and search making it easy to use, and contains a powerful language that can capture the complex structures of our scheduling problem [31]. 3Methods Dynamic template scheduling uses an optimization model of the chemotherapy problem to create and update a template of a day s expected appointments. A template is a schedule of open slots of specific durations into which appointments can be scheduled. As requests for appointments arrive, the user assigns start times by matching appointments to slots in the template. In our context, a slot matches an appointment when the durations are exactly equal. When a request arrives that does not match any empty slots, the template is dynamically updated. Any appointments that have already been given slots are not changed. The remaining available time in the template is reallocated to a set of empty slots using an updated sample of the day s expected appointments and the optimization model. The user now has a new set of appointment slots to

382 S. Hahn-Goldberg et al. use for scheduling requests. If more requests arrive than expected, the algorithm will treat them as conflicts and each additional request will be scheduled individually by the optimization model. The remainder of this section is organized as follows. First we describe the offline problem and solution, where all the appointments that need to be scheduled are known beforehand and are scheduled using an optimization model. Second, we describe the empirical distribution of appointments and the statistical analysis conducted to determine the best use of the provided data. Third, we describe the online problem and the dynamic template scheduling algorithm designed to solve the problem where appointments are scheduled in real time. Fourth, we describe the shuffling algorithm added to accommodate last minute scheduling changes. Finally, we provide experiments evaluating dynamic template scheduling and the shuffling algorithms. 3.1 The offline problem Dynamic template scheduling uses an optimization model of the offline chemotherapy problem to create and update a template of a day s expected appointments. A template is a schedule of open slots of specific durations into which appointments can be scheduled. Following the notation outlined by Pinedo [32], the offline chemotherapy outpatient scheduling problem at the Odette Cancer Centre can be classified as FF 3:1,5,1;2,15(29),1(1);2,15(29),.25(1) prec, temp, r j C max,where the first grouping describes the resource environment, the second describes the processing characteristics, and the third describes the objectives. The chemotherapy outpatient scheduling problem has a resource environment that is a three stage flexible flow shop, FF 3. The first stage, pharmacy, has one resource, pharmacy technicians, with five units available and each patient requires one unit, represented in the notation by 1,5,1. The second stage, the first part of treatment, has two resources, nurses and chairs. There are 13 or 15 nurses and 29 chairs available and each patient requires one from each group, represented in the notation by 2, 15(29), 1(1). The third stage, the second part of treatment, has two resources, nurses and chairs. Each patient requires a chair and monitoring by a nurse. At this stage, every nurse can monitor up to four patients at a time, thus each patient requires a quarter of a nurse and one chair. There are 13 or 15 nurses and 29 chairs available, represented in the notation by 2, 15(29), 0.25(1). The processing characteristics include precedence constraints between stages, temporal constraints that limit the time between stages, and ready times for patients with other appointments earlier in the day; represented in the notation by prec, temp, and r j respectively. The objective of the problem is to assign start times to all the appointment stages while minimizing makespan, C max. In this research, the chemotherapy outpatient scheduling problem has a horizon of 1 day. As requests for appointments arrive, they are scheduled for a particular day in the future. Another version of the chemotherapy outpatient scheduling problem would be concerned with assigning appointments on a waiting list to treatment start dates and would have the objective of minimizing the indirect waiting time. In this paper, we assume that an external process decides the day for each appointment and focus on scheduling each day. Each calendar day is a separate problem instance. Makespan was minimized because the main issue facing the chemotherapy centre was expense, staff working overtime, and an increasing patient demand. As a result, minimizing makespan was chosen as the objective to ensure that the chemotherapy centre would be able to service the increasing demand without requiring additional resources. The optimization of resource use and the desired increase in throughput can also help to deal with access and shortening the indirect waiting time for patients beginning chemotherapy for the first time. The offline chemotherapy scheduling problem is defined in terms of a set of start times, S. The set of start times contains a possible entry every fifteen-minute from zero through the horizon. The model receives as an input a set of appointment requests, R, which need to be serviced on a given day. R is a set of integers of size q. Eachrequest,r, iscomposedofthree stages; pharmacy or drug preparation, treatment requiring direct nurse supervision, and the full treatment, which encompasses the portion of treatment requiring direct supervision. Each stage, i, is characterized by its duration, D i [r]. The chemotherapy problem requires that all requests be serviced. The goal is to find an assignment of all requests to start times, satisfying the problem constraints described below and minimizing the makespan, C max. The optimization model used for this problem is a constraint programming model, named CP3. Itproducesanoptimal solution to the theoretical offline problem with perfect information. The assignment is a schedule servicing all of the requests in the set R. In order to create a template of time-slots, we take each assignment and replace it with an open time slot. The duration of the open slot matches the full treatment duration of r. The constraint model formulation was developed in constraint programming using COMET 2.1.0, Dynadec software, and uses constraints and search techniques built into the software. Using constraint programming, many of the constraints are global constraints and as such are not linear constraints as is typical in a mixed integer programming model. A mixed integer programming was also created and tested in [33].

Dynamic optimization of chemotherapy outpatient scheduling with uncertainty 383 While the focus of this paper is not the optimization model itself, but rather its use as part of dynamic template scheduling, for completeness we include the full model definition. The notation for the constraint model, CP3, isasfollows: & R: set of patient requests. & C: set of chairs a maximum of 29 chairs and beds. & N: set of nurses available for stage two, the first stage of treatment there are 13 or 15 nurses. Each test day used for experiments contains the exact number of nurses that worked in the clinic on that day. & N4: set of nurses for stage three each 0.25 of a nurse is a resource. N and N4 represent different activities that the nurses can perform; setting up and monitoring respectively. Each nurse can only set up a single patient at a time during stage two, but they can monitor up to four patients at once in stage three. There are 52 or 60 units of nurses available because eachofthe13or15nursescanserviceuptofour patients during stage three. & T: set of times times range from zero through 40. Each time unit represents a fifteen-minute timeslot. & DR: set of servers for drug preparation there are five servers at all times. & D ir :durationofstagei for patient request r. & B r : The ready time, when patient request r can begin activity 1. If a patient has no previous appointments on the day of treatment, the ready time will be zero. & t: a constant. It represents a maximum time allowed between stages one and two. & F: set of already scheduled appointments. In many cases, we need to schedule a set of appointments in the presence of already scheduled appointments. & S * : an already assigned start time. The start times from the set F have already been fixed, when they are passed into the model. & G: the complete set of appointment requests and already scheduled appointments, defined as R F. When a constraint applies to both patient requests and already scheduled appointments, it applies to each appointment g from the set G. For each patient request, r R, we have two decision variables: the start times of the drug preparation and treatment stages. The final stage of treatment is scheduled to start at the same time as stage two: & & i=1,2; r R,S rp : The start time of stage i for patient request r. It takes a value from 0 through T -1. S H : The start time of Horizon. Horizon is an activity that is scheduled after the final appointment is complete. It represents the end time of the day. The optimization model of the offline problem that we created is a constraint programming model. The model objective and constraints are as follows: mins H Subject to: g G; cumulative S 1g ; D 1g ; 1; jdrj g G; cumulative S 3g ; D 3g ; 1; jcj g G; cumulative S 2g ; D 2g ; 1; jnj g G; cumulative S 3g ; D 3g ; 1; jn4j r R; S 2r S 1r þ D 1r r R; S 1r B r r R; S 3r ¼ S 2r r R; S H jtj g G; S 3g þ D 3g S H r R; S 2r S 1r þ D 1r þ t f F; i; i ¼ 1; 2; 3; S if ¼ S if ð1þ ð2þ ð3þ ð4þ ð5þ ð6þ ð7þ ð8þ ð9þ ð10þ ð11þ The objective of the model is to minimize makespan, that is, the start time of Horizon. Constraint (1) ensures the number of patients whose drugs are prepared at any time does not exceed the number of drug preparation servers. Each stage-one activity is constrained to require one unit of the set of drug preparation servers. The cumulative constraint [28] is a global constraint that ensures capacity of a discrete resource is respected. Constraint (2) ensures that the number of patients using chairs is less than or equal to the number of chairs available.

384 S. Hahn-Goldberg et al. Constraint (3) ensures that in stage two, at most one patient is assigned to a nurse at a time. Constraint (4) ensures that the number of patients assigned to each nurse in stage three does not exceed the maximum number of patients that the nurse is allowed to monitor by requiring that each patient has a nurse from N4. Since stage three represents the full treatment time, together with constraint 3, this ensures that the capacity of the nurses is respected and a nurse cannot be monitoring four patients while setting up a fifth. The two sets of nurses, N and N4, represent the same physical nurses in the application, yet different nursing roles, set up during stage two and monitoring during stage three. Nurses can be involved in both roles (i.e. setting up and monitoring) at the same time as long as they do not set up more than one patient at the same time and the total number of patients they are responsible for does not exceed four. In our model, each patient requires a nurse from N during stage two and a nurse from N4 during stage three. In addition, stage three and four are scheduled to start at the same time in order to ensure that a nurse will not be monitoring four patients, using the maximum capacity of N4, while setting up another, using the capacity of N. Indoingso,0.25ofa nurse required for the direct nursing component of treatment is accounted for by having stage three encompass the time of stage two and by the requirement of a nurse from N4 during that time. As a result, 1.25 of a nurse is required during stage two. Constraint (5) ensures that treatment cannot start before the end time of the preceding stage. Constraint (6) ensures the start time for each patient is greater than or equal to their ready time. Constraint (7) ensures that stage three starts at the same time as stage two. This is important because stage four encompasses both parts of the same treatment and the section of time representing stage two must occur at the exact same time as the actual stage three. Additionally, this ensures that the nursing requirement is set up properly. Constraint (8) ensures no patient finishes treatment after the end of the working day. Constraint (9) ensures Horizon does not start until all other activities have completed. There is a maximum time, t=30 min between stages one and two (constraint 10). This constraint says that the start time of stage two must start before the end time (start time plus duration) or stage one plus an added time of 30 min. This ensures that patients will not have to wait for extra time between stages, even if it creates a more efficient schedule. For each already scheduled appointment, f F, there are additional constraints added to the optimization model (constraint 11) that ensures the start time of each stage of that appointment will remain unchanged. CP3(R, F) denotes the optimal template for the input set R and F and ensures that any already scheduled appointments do not change. All appointments in R that are scheduled by CP3 will be replaced by empty slots, while the appointments in F are not. 3.2 Empirical distribution of appointments There are hundreds of chemotherapy regimens that can be grouped into buckets of appointments ranging in length from 15 to 510 min, in increments of 15 min. Regimens refer to a particular protocol of drug dosages and frequencies that is prescribed by an oncologist. Each regimen has a unique treatment length and the list of regimens treated on a given day at the chemotherapy centre provides us with the list of appointment treatment lengths that need to be scheduled by our model. A detailed description of the problem instances used is provided in Section 3.5. One year of data from the Odette Cancer Centre, from April 1st, 2009 through March 31st, 2010, was available to create a distribution depicting the expected number of appointments of each bucket or treatment length. Statistical analysis was utilized to determine the amount of data to use so that the empirical distribution is a statistical match to the current state of the system that we are trying to represent. In chemotherapy, research on new drugs is ongoing and the mix of treatment continuously evolves. Using data from too far in the past could result in an inaccurate approximation of the future. Kruskal Wallis, non-parametric, testing was used to determine if different sets of random data come from the same distribution [34]. The tests were performed using OriginPro 8.6. First, the data was divided into 12 sets, one for each month and Kruskal-Wallis was conducted to test the hypothesis that all the data could come from the same distribution, within 95 % confidence. With a test statistic of 5.48 10 13, the hypothesis was determined to be false. Then, tests were performed on two sets of data at a time. Figure 1 shows a grid of the year, where months that are statistically significant are shaded. As the months from which data is drawn become more widely separated, the likelihood of their appointments being from the same distribution falls. April is statistically the same as May and June, but not the same as July. Based on the statistical results, 3 months of data is determined to be an appropriate amount of data to use for creating the empirical distribution. Therefore the empirical probability distribution was created using the average number of appointments of each length over a three-month period.

Dynamic optimization of chemotherapy outpatient scheduling with uncertainty 385 3.3 The online problem and dynamic algorithm Asampleofsizeq is taken from the empirical distribution, where q is the maximum number of appointment requests that are expected for a given day. The initial set of expected appointments is denoted by E 0. Each expected appointment, e E 0, has duration, D[e], which represents the full treatment length. In this problem, every pharmacy stage is assumed to be 30 min in length and the direct nursing requirement is assumed to be 15 min. Only the treatment length differs for each appointment. E 0 is input for an offline problem that is optimally scheduled by the optimization model. The set of treatment durations, E 0 is translated into a full set of appointment requests, including pharmacy time and the direct nursing requirement as part of the full treatment duration before it is passed to the optimization model. The initial set of scheduled appointments, F 0 is empty. CP3(E 0,F 0 ) provides as output the initial template composed of TS time-slots. Each time-slot ts TS has a length D[ts] and can be represented by its start time s S. The set of timeslots available will change only when a conflict occurs and the template needs to be updated. In the online problem, the real set of requests, R, is revealed online, one request at a time. When each new request, r i,isrevealed, the online algorithm has available the previous assignment of requests, F, and the template created by the optimization model. The online algorithm is formalized below. Dynamic Template Scheduling (q, R) q - sample size representing the maximum number of appointments expected on a typical day R set of appointment requests The dynamic template scheduling algorithm assigns requests to time slots with the objective of minimizing the makespan. In lines 1 3 of the pseudo code, the algorithm takes a sample from the empirical distribution and creates a template using the constraint programming model. The set of requests in our experiments will be taken from the actual data of appointments serviced at the chemotherapy centre and then randomized and set to arrive one at a time. This will be described further in Section 3.5. For each request that arrives, the dynamic template scheduling algorithm checks the current template and sees if there is a matching slot available for the request (line 5). In line 7, if there are multiple slots available, the appointment is assigned to the earliest slot. In line 8, that appointment request becomes part of the set of scheduled appointments, denoted by F, and in line 9, q is reduced by 1. If there is no matching slot available, the algorithm must get an updated template. In line 11, the empirical distribution is used to generate a new list of expected appointments, E. The

386 S. Hahn-Goldberg et al. Fig. 1 Months that are statistically the same Apr May Jun Jul Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Jan Feb Mar appointment that caused the conflict is added into the sample, E, and is passed to the optimization model. In line 12, the optimization model then creates a new template. The dynamic template scheduling algorithm continues with the updated template until it receives another request that causes a conflict. If more than q requests are received, each additional request after the q th request will be treated as a conflict and a new template will be made with a spot for the additional request. Once all the requests are assigned start times, the day is considered complete. 3.3.1 An example The following example illustrates the dynamic template scheduling algorithm. In this example, there are two chairs available to treat patients. The typical maximum number of appointments that are expected, q, is 6. There will be six requests serviced. First, a sample of size six is taken from the empirical distribution, SAMPLE(6), to get E 0. For this example, E 0 =<15,15,30,30,60,120,>, where E 0 is a list of treatment durations given in minutes. This sample of treatment durations is translated into a set of full appointment requests, including pharmacy time and the direct nursing requirement. That set of appointment requests is then passed to the optimization model, CP3, and CP3(E 0, F 0 ={}) schedules these appointments optimally and gives a set of time-slots that can be used to schedule the actual requests that arrive. In this simple example, the optimal set of time-slots to minimize makespan is S=[{30,60,15,30},{120,15}], where the first chair has open slots of 30 min, 60 min, 15 min, and 30 min, and the second chair has open slots of 120 min and 15 min. The makespan is 135 min. In the diagram, bold lines outline the time slots and normal lines separate each fifteenminute time period, S can be represented as Then the actual requests for appointments arrive one at a time and we schedule them into the template as they arrive. When an appointment is scheduled into S, it will be shaded. The first request to arrive has a duration of 15 min. S becomes The assigned appointment is added to F. The next request to arrive is also 15 min. S becomes The assigned appointment is added to F. The third request to arrive is also 15 min. There is no match in the set of timeslots, so the template must be updated. Since two appointments have already been scheduled, q has been reduced to 4. To get our next sample size, we reduce q by one more and it becomes 3. A new sample of requests, SAMPLE(3) is taken

Dynamic optimization of chemotherapy outpatient scheduling with uncertainty 387 from the distribution and added to the request that caused the conflict to get E=<15,45,60,105>. This time, when scheduling the requests, they have to be scheduled around the already scheduled appointments. CP3(E,F)= S=[{15,60,15,45},{105,15}], where the bold numbers are already scheduled appointments. S can be represented by We schedule the final appointment request into S and we get our final schedule. Due to the already scheduled appointments and the expected appointment durations sampled, the resulting template has empty places, such as a 15 min hole left in the schedule after the 15 min and 60 min appointments on the first chair. The optimization model only creates slots for appointments that are listed in the sample and it does so around the already scheduled appointments. If an appointment request arrives that does not fit into the template, holes such as these may become filled. We return to the third request of 15 min and schedule it into S The assigned appointment is added to F. The fourth request is a 45-minute appointment. It is scheduled into S The assigned appointment is added to F. The fifth request is for a 105-minute appointment and it is scheduled into S The last request is for 45 minutes. Again, we need to get a new template. This time, we are only looking for a spot for one appointment, the appointment that caused the conflict. E= <45> and CP3(E,F)=S=[{15,15,45,15},{105,15}], where the bold numbers are already scheduled appointments. S can be represented by The scheduling procedure may leave empty slots in the schedule. This is a result of it being online scheduling. We do not know the durations of the later requests and the actual set of requests is not likely to match exactly with what was expected. Furthermore, we are assuming here that once an appointment is scheduled, it cannot be moved. 3.4 The shuffling algorithm Emergency chemotherapy appointments are not very common, but appointments are added to the schedule a day or two before execution, when the schedule is essentially complete. Appointments are cancelled for many reasons, including patients who have blood work results that are not acceptable for continuing with treatment and patients whose protocols change. On average, 20 % of appointments are cancelled at the Odette Cancer Centre. In the dynamic template scheduling algorithm described above, once an appointment is scheduled, the start time is fixed. Now, start times of appointments are permitted to move by a predetermined maximum amount. This allows for more flexibility when searching for a new optimal template. When an appointment has to be added to a complete or almost complete schedule, without allowing shuffling, this appointment would just be added to the end time of the chair with the earliest end time. With shuffling, appointments can be moved to make room for the new appointment and result in a more efficient schedule. When appointments are cancelled without shuffling, the resulting schedule can be full of holes, where chairs are left idle unnecessarily. By allowing shuffling, the already scheduled appointments can be moved to better utilize available resources. In the chemotherapy problem, extra work would occur if patients required notification of changed appointment times and patients would be unhappy to have their appointment times changed at the last minute; therefore, movement of an appointment s start time by as little as possible is preferred. At the Odette Cancer Centre, all patients are required to arrive to the centre an hour before their scheduled appointment time to allow sufficient time for their drugs to be prepared. As such, patients need not be notified of a change to their appointment time if there is a shift of 30 min or less. To implement shuffling, constraint (11) in the constraint model is altered and for each scheduled appointment, the constraint ensures that the new start time remains within a certain distance from the original start time. A constant, k, is

388 S. Hahn-Goldberg et al. given as the number of time slots in the schedule by which appointments can be shuffled. The new constraint (11a) is identical to the original constraint (11) with k equal to zero. The altered constraint is, f F; i; i ¼ 1; 2; 3; S if k S if S if þ k 3.5 Design of experiments ð11aþ The experiments are run using 30 problem instances, each representing an actual day at the Odette Cancer Centre from January 16th through March 5th, 2010. The list of patients serviced on each day as well as the regimens and corresponding treatment durations were extracted from the scheduling software in use at the centre. The direct nursing requirement and drug preparation time were determined by taking the typical time requirement for those stages at the Odette Cancer Centre. The times used were 30 min for each drug preparation time and 15 min for each direct nursing requirement time. The average times were validated by cross-checking the list of regimens treated at the Odette Cancer Centre with a database provided by Cancer Care Ontario containing estimated time requirements for all chemotherapy regimens. The times were further approved by oncologists and nurses at the chemotherapy centre. For the 30 problem instances used in this research, the average number of appointments per instance was 90.48 with a standard deviation of 17.49 appointments. The average number of total appointment minutes per instance was 10909.09 with a standard deviation of 1699.38 min. Each individual appointment treatment length is known and can range from 15 min to 510 min in duration. All the experiments were conducted on a Pentium Dual-Core CPU computer E5200 @ 2.50 GHz and 3.25 GB of RAM running 32 bit Windows XP. Several sets of experiments were conducted to test the dynamic template scheduling framework. First, the test sets were solved using the online algorithm. The set of appointments scheduled on a given day at the chemotherapy centre were randomized and set to arrive to the online algorithm one at a time. Each time the template has to be updated, the model was given 90 s to solve and the best solution at that time is used. The number of times that a template has to be updated is directly related to how close the sample is to the actual set of appointments that needs to be scheduled. In our problem sets, the average number of updates required to schedule a day or appointments was 17. During the execution of the algorithm, samples are randomly drawn from the empirical distribution. Therefore, each test set was solved five times and the average makespan was recorded. We then compared the performance of dynamic template scheduling to the theoretical optimal offline solution and the actual day that occurred at the Odette Cancer Centre. In the theoretical optimal offline solution, the list of appointment requests is known and scheduled using the optimization model. The corresponding makespan is referred to as the optimal makespan. For the actual day that occurred at the Odette Cancer Centre, the actual makespan is the time that the day ended at the chemotherapy centre. Next, we designed two sets of experiments to test the effect of implementing the shuffling algorithm: tests using shuffling to accommodate added appointments, and tests to adjust after cancellations occur. Due to the amount of data available on cancelled appointments; for the shuffling experiments, we used data from a subset of five problem instances at the Odette Cancer Centre. The five instances represented March 1st through 5th, 2010. Table 1 provides the number of appointments, total treatment minutes, and nurses available, for each data set. To test the effect of shuffling when appointments are added to the schedule, we included three sets of experimental conditions: the starting schedule, the number and length of added appointments, and the permitted start-time change between a previously scheduled appointment and its new time, k. Experiments were conducted using two sets of completed schedules: dynamic template schedules, the schedules resultant from using the dynamic template scheduling system; and real schedules, the schedules used by the Odette Cancer Centre. Four sets of added appointments are used: one appointment with duration of two hours, four appointments with durations of two hours each, one appointment with duration of four hours, and two appointments with durations of four hours each. Three values for k were used: 15 min, 30 min, and 60 min. In the experiments we began with a completed schedule as a set of already scheduled appointments, F, and used the optimization model to schedule the new appointments that were to be added. For each value of k, each set of experimental conditions was solved twice, once with shuffling prohibited, k was set to 0, and once with shuffling permitted, k was set to the corresponding limit. The difference between the two makespans was calculated to determine the effect of shuffling. To study the effect of shuffling after appointments are cancelled, we used the data from the same five days as above; however, including appointments that were booked and then cancelled, which are kept in a separate database. For these data Table 1 Experimental data Test set Number of appointments Total treatment minutes Day 1 63 6210 13 Day 2 83 9435 13 Day 3 81 11760 15 Day 4 99 10725 15 Day 5 90 11190 13 Nurses available

Dynamic optimization of chemotherapy outpatient scheduling with uncertainty 389 sets, the appointments that were serviced on a given day are combined into one list with appointments that were booked for that day and then cancelled. The experiments testing the effect of shuffling on cancellations were conducted as follows: 1. The complete set of appointments was scheduled using the optimization model. The complete set of appointments is larger than the set of appointments from the previous experiments because it includes appointments that were booked and then cancelled in addition to the set of appointments that were serviced. The makespan of the full schedule, the theoretical optimal offline solution for the complete set of appointments, was recorded as the full makespan. 2. The appointments that ended up being cancelled were removed from the schedule. The makespan of the schedule with the cancelled patients removed was recorded as the makespan after cancellations. 3. The start times of appointments were rescheduled using the optimization model and the start times were permitted to be shuffled by 30 min. The makespan was recorded as the makespan after shuffling. 4. The makespan after shuffling was compared to the makespan before shuffling. 4Results Table 2 provides the results of the experiments comparing the dynamic makespan resulting from dynamic template scheduling to the theoretical optimal makespan and the actual makespan. In total 150 problem instances were run by running each of 30 problem instances five times. The dynamic template does not perform as well as the theoretical optimal offline solution and performs better than Table 2 Dynamic template scheduling versus the theoretical optimal offline solution and the actual schedule. The values in the table are given in minutes and represent statistics of the difference between the dynamic makespan and the theoretical optimal offline and actual makespan. A positive difference means that the dynamic makespan was shorter and thus represents an improvement Test set Compared to optimal Compared to actual Average 48.45 85.80 Standard deviation 38.10 47.40 Median 45.00 75.00 Minimum 165.00 15.00 Maximum 0.00 225.00 95 % Confidence interval ( 120,0) (15,180) the current practice. There is an average difference between the theoretical optimal offline makespan and the dynamic makespan of just over 48 treatment minutes, indicating a reduction compared to the theoretical optimal offline makespan of 13 %. However, the results show the large benefit of employing the dynamic template approach. There is an average difference between the dynamic makespan and the actual makespan of just over 85 treatment minutes, indicating an improvement over the actual schedule of 15 %. Although there is a large improvement shown between the dynamic makespan and the actual makespan at the cancer centre, the authors acknowledge that there are assumptions made in the dynamic problem instances including using fixed durations for pharmacy and the direct nursing portion of treatment. In reality, these times differ depending on the regimen. Therefore, the results presented here are indicative of potential improvement. Table 3 shows the results of the shuffling experiments for adding appointments. The average difference in makespan when shuffling is permitted compared to when shuffling is not permitted is provided. By shuffling appointments, the makespan of the day can be made shorter by a maximum of the permitted time gap, k. Each appointment can only deviate by k from its original start time, so in order for the makespan to be reduced by that maximum value, every appointment that ended at the makespan in the original schedule has to be moved forward by the permitted time gap, k. In our experiments, the average makespan is calculated for each starting schedule and allowed shuffling time, over all cases of added appointments. The difference in makespan between when shuffling is prohibited and when it is permitted is calculated using the following equation: Difference in makespan ¼ makespan ðno shufflingþ makespan ðshufflingþ A positive difference in makespan corresponds to an improvement by allowing shuffling. The entries in the table are the differences in makespan, given in minutes. By permitting shuffling of appointment start times, the centre may be able to accommodate additional appointments. There is a larger difference in makespan when starting from Table 3 Differences in makespan when adding in appointments. The entries in the table represent the average improvement in makespan for a given starting schedule and time gap when a variety of appointments are added in. The improvement is given in minutes Starting model Time gap k (min) 15 30 60 Dynamic template 10.5 13.5 18.0 Real 15.0 30.0 60.0

390 S. Hahn-Goldberg et al. the real schedule because there is more room for improvement in the schedule. The real schedules contain more time where resources are unnecessarily idle and as a result more improvement can be made by shuffling when starting from a real schedule. Our experiments showed that permitting shuffling does not increase the run time of the optimization model or the run time of the dynamic template scheduling algorithm. Table 4 provides the results of the shuffling experiments for accommodating cancellations. The makespans are presented in minutes. Note that days two through five have the maximum possible reduction in makespan as each appointment can only move by at most 30 min. When appointments are cancelled from the schedule, the makespan does not change unless the last slot is cancelled. In order for the makespan to change, an appointment would need to be cancelled from the last slot of any chair that ends at the makespan and in our data, there was no such collection of cancellations. By shuffling appointments, the centre can better adjust their schedule after cancellations occur. 5 Discussion In this paper, chemotherapy outpatient scheduling was defined and solved using dynamic template scheduling, a novel method for combining proactive and online optimization to address uncertainty in scheduling problems. The chemotherapy outpatient scheduling problem is a complex problem and although it would appear that the problem could benefit from the application of optimization techniques, it has not been studied by many researchers. In this paper, dynamic template scheduling was tested against the theoretical optimal offline schedule, and the current actual practice. Plans are underway to develop a version of the dynamic template scheduling application that can be tested within CHARM, the scheduling system currently in use at the Odette Cancer Centre. Table 4 Cancellation experiments. The full makespan is the theoretical optimal offline solution of the complete set of appointments, including those that were scheduled and then cancelled. The makespan after cancellations is the makespan after the appointments are cancelled, but shuffling is not permitted. The makespan after shuffling is the makespan when the appointments are cancelled and shuffling is permitted. All values are given in minutes Test set Full makespan Makespan after cancellations Day 1 360 360 345 Day 2 510 510 480 Day 3 585 585 555 Day 4 555 555 525 Day 5 555 555 525 Makespan after shuffling The created model of the chemotherapy outpatient scheduling problem is a general model that can be used for other cancer centres, which may have differing amounts of pharmacy, nurse, and chair resources. In addition, the direct nursing requirement as a portion of the complete treatment has been included in the model as well. In our experiments, we used the actual data and resources available from the Odette Cancer Centre. Through the experiments, we were able to identify what the bottleneck in the system was for the Odette Cancer Centre, namely the pharmacy stage of the process. Further sensitivity analysis of the different resources confirmed that point and is presented in [34]. While a reduction of one pharmacy resource affects the makespan, the same effect requires a reduction of 4 nurses and 10 chairs. Having a general model with all resources and stages included ensures that the model is usable by other cancer centres and by Odette, if they move or make changes to their resource levels. Moving appointment times is a difficult topic at the Odette Cancer Centre for many reasons. There are concerns that patients will be unhappy as other arrangements or appointments may be impacted if their appointment times get moved. Through the experiments described in this paper, it can be shown that shuffling a small amount can improve the makespan when appointments are added or cancelled. In our opinion, shuffling should be added to the dynamic template system as an optional feature. The users should be able to enter the time limit and then shuffle only if they feel it is necessary. By keeping the feature optional, the users can decide by exactly how much to shuffle and when to shuffle. If they are happy with the schedule the way it is, they can choose not to shuffle, to avoid distressing any patients; however if they receive a cancellation or addition, they can take advantage of the shuffling feature. We anticipate that by integrating optimization into the scheduling system, the Odette Cancer Centre will be able to service more patients with the same or fewer resources. At present, the cancer centre is not using any optimization while scheduling and they do not take all their resources into account when booking appointments. They book patients based on chair capacity and only indirectly account for pharmacy and nurse capacity. As a result, resources are not being used as efficiently as they could be. The potential for improvement can be seen by studying the schedules obtained using dynamic template scheduling compared to the real schedules that were executed at the cancer centre. Figure 2 provides a diagram of test day one scheduled using dynamic template scheduling with shuffling next to the schedule that was executed at the cancer centre. Each row represents a separate chair used for treatment and each change in colour or shade represents a different patient being treated. Only the full treatment lengths are shown in this figure, so there is no visual of the drug preparation stage. The direct nursing requirement is included

Dynamic optimization of chemotherapy outpatient scheduling with uncertainty 391 Fig. 2 Test day 1 scheduled using dynamic template scheduling (left) and the actual schedule executed at the Odette Cancer Centre (right) as part of the full treatment length, but is not represented as a separate colour. Time runs along the x-axis, with each square of the grid being a fifteen-minute timeslot. For example, a 15- minute treatment is represented by one square shaded in one colour and a 4-hour treatment is represented by 16 consecutive squares of the same colour. The schedules show the potential advantage of integrating optimization into current practice. With optimization, the improvement in makespan and chair utilization is significant. In the real schedule, there are many more holes in the schedule and many chairs are not being used effectively. This is in part because the schedule was highly effected by the many cancellations and last minute changes that were made to their original schedule, which was likely much more efficient than the schedule they ended up using, highlighting the importance of allowing some shuffling in appointment start times as changes to the schedule occur. Chair utilization, defined as the percentage of time that the chair is in use before the last patient on that chair finishes, is only 64 % in the real schedule, but is 80 % in the schedule using dynamic template scheduling. Dynamic template scheduling can be easily incorporated into existing scheduling systems, resulting in minor changes for users. The system would provide them with optimal options of where to place appointments and users can apply their own knowledge and judgment to choose from the given options. Besides chemotherapy scheduling, there are many other applications that exhibit real-time uncertainty; e.g. health care applications where appointments are booked in advance, a factory where stock is made to order, a courier service where requests arrive at various times and decisions must be made on when to service them. Dynamic template scheduling can be applied to any problem where requests need to be assigned a start time before all requests are known. 6 Conclusion This paper demonstrated the potential for integrating optimization into the scheduling of chemotherapy outpatient appointments. Additionally, this paper showed that chemotherapy centres can use records of their past appointments to inform their future schedules and thereby integrate optimization in such a way that the dynamism and uncertainty inherent in the chemotherapy outpatient scheduling problem is accommodated. Nonetheless, we believe there are a number of limitations of this research: & & & There were several features omitted from the chemotherapy outpatient scheduling. Due to patient confidentiality, information involving any patient identification was not available to us. Such information includes which patients have multiple treatments in the same week, which patients receive more than one drug at a single treatment visit, and patient complexity. There may be restrictions on the number of complex patients that can be assigned to a nurse on the same day or at the same time. Treatment length is assumed to be deterministic in each regimen. Although, the estimates of treatment length are considered to be very accurate, there are patients that take longer due to complications during set up and allergic reactions. We assume that all patients will arrive on time for their appointments. In reality, patients may be late due to delays in previous appointments, traffic, and other reasons. There are many directions for future work that stem from the research presented in this paper in the form of extensions that address the weaknesses listed above, adaptations to make the system more generalizable, and further experimentation and implementation of the system. Extensions could include