A Preliminary Study into the Use of an Evolutionary Algorithm Hyper-heuristic to Solve the Nurse Rostering Problem

A Preliminary Study into the Use of an Evolutionary Algorithm Hyper-heuristic to Solve the Nurse Rostering Problem Christopher Rae School of Mathematics, Statistics & Computer Science University of KwaZulu-Natal Pietermaritzburg, South Africa stingrae789@gmail.com Nelishia Pillay School of Mathematics, Statistics & Computer Science University of KwaZulu-Natal Pietermaritzburg, South Africa pillayn32@ukzn.ac.za Abstract This paper reports on an initial attempt to solve the nurse rostering problem using an evolutionary algorithm selection perturbative hyper-heuristic. The main aim of this study is to get a feel for the potential of such a hyper-heuristic in solving the nurse rostering problem. This will be used to direct future extensions of this work. This study identifies lowlevel perturbative heuristics for this domain as well as a representation, initial population generation method, evaluation and selection methods, and genetic operator for the evolutionary algorithm hyper-heuristic. The approach was tested on six problems from the first international nurse rostering competition. The performance of the hyper-heuristic was found to be comparable to that of other methods applied to the same problems. The study has shown the potential of this approach and also identified future extensions of this work. Keywords- nurse rostering; hyper-heuritics; perturbation; evolutionary algorithms I. INTRODUCTION The nurse rostering problem (NRP) is an NP-hard problem which involves the scheduling of nurses to shifts so as to ensure the problem constraints are met or the number of constraints violated is minimized. Various methods have been evaluated for solving this problem including tabu search, simulated annealing, integer programming, constraint programming and genetic algorithms [1] amongst others. A more recent approach for solving combinatorial optimization problems is hyper-heuristics which aims at producing a more general solution to a problem rather than the best result for one or two data sets [2]. There has not been much research into the use of hyper-heuristics for solving the nurse rostering problem. Furthermore, the use of an evolutionary algorithm (EA) selection perturbative hyper-heuristic which caters for both heuristic selection and move acceptance has not been conducted. This paper reports on a preliminary study into the use of an evolutionary algorithm selection perturbative hyperheuristic (EA-SPHH) for solving the nurse rostering problem. The aim of the study was to identify low-level perturbative heuristics for this domain and a representation, method for initial population generation, methods for evaluation and selection and genetic operators for the EA hyper-heuristic. An evaluation of the hyper-heuristic on six nurse rostering problems revealed the effectiveness of hyper-heuristics for this domain. The following section describes the nurse rostering problem. An overview of hyper-heuristics and the application of hyper-heuristics to solve the nurse rostering problem is presented in section 3. Section 4 presents the EA-SPHH for solving the NRP. The experimental setup used to test the EA SPHH is described in section 5. Section 6 discusses the performance of the EA hyper-heuristic in solving the NRP. An overview of the findings of the study and future work is provided in section 7. II. THE NURSE ROSTERING PROBLEM The nurse rostering problem is a personnel scheduling problem that falls under the category of NP-complete in difficulty. This field of study has potential real world applications due it being applicable to other personnel scheduling problems and as such has been studied for over 40 years. The problem itself is to assign a set of shifts, defined as working periods, to the available nursing staff subject to the rules and regulations or better known as constraints of the employer and some of the preferences the nursing staff may have [3]. It has been said that this is the most challenging of the personnel scheduling problems [4]. Typically the nurse rostering problem defines the number of nurses and their skills, availability and shift preferences. These problem requirements are usually specified in terms of hard and soft constraints. Hard constraints must be met for a roster to be feasible. Soft constraints on the other hand may be violated as they are not mandatory but rather preferential requirements. Examples of hard constraints include a nurse may only work a single shift per day, a nurse must have a specific skill in order to work a certain shift and all shift requirements must be met. The aim is to minimize the soft constraint cost of a roster while keeping it free of hard constraint violations. A roster meeting all hard constraints is referred to as a feasible roster [5]. The quality of a roster is determined by its soft constraint cost. Soft constraints include minimum and maximum assignments per scheduling period, minimum or maximum consecutive days and nurse preferences like requests to work or not work on certain days amongst others. Many approaches have been applied to this domain including constraint programming [6], expert systems [6], 978-1-4673-4769-3/12/$31.00 c 2012 IEEE 156

genetic algorithms [3, 6, 7], heuristic techniques [6], integer programming [6], simulated annealing [6] and tabu-search [6, 8]. III. HYPER-HEURISTICS AND THE NURSE ROSTERING PROBLEM Hyper-heuristics provide a more generalized solution to the problem and search a heuristic space instead of a solution space which is typical of most optimization techniques [9]. Hyper-heuristics are divided into four main categories, namely, selection constructive, selection perturbative, generation constructive and generation perturbative [2]. In this paper a selection perturbative hyper-heuristic is investigated for solving the nurse rostering problem. Selection perturbuative hyper-heuristics explore a space of low-level perturbative heuristics. Perturbative low-level heuristic are essentially move operators, e.g. swap the shift of two nurses, and are used to improve the feasibility and/or quality of a roster. All the research in the domain of hyper-heuristics for the nurse rostering problem have focused on selection perturbative hyper-heuristics. A few studies have combined the use of a selection peturbative hyper-heuristic with another optimization technique to solve the NRP. Bilgin et al. [10] used a random selection hyper-heuristic to solve the Belgian nurse rostering problem which used simulated annealing or great deluge for move acceptance. It was found that simulated annealing performed best. This study used 6 low-level heuristics to assign, delete and change shifts. Burke et al. [11] implemented a tabu-search selection perturbative hyper-heuristic to generate a nurse roster for a UK hospital. Nine low-level perturbation heuristics, which change nurse shifts, are used. Among the hybrid approaches is that employed by Bilgin et al. [12] which combines a greedy shuffle heuristic and a random selection hyperheuristic with simulated annealing for move acceptance. This was one of 14 entries to the first international nurse rostering competition and placed within the top 5. This study used 12 low-level perturbative heuristics. The greedy shuffle is applied after the hyper-heuristic and swaps components of rosters for two nurses. Only swaps that produce feasible rosters of just as good or better quality are accepted. Bai et al. [13] use a hybrid of a genetic algorithm and a selective perturbative hyper-heuristic. The genetic algorithm uses the crossover and mutation operators on the solution space. The hyper-heuristic is embedded in the genetic algorithm and is used to improve each individual. Heuristics are selected based on their performance and simulated annealing is used for move acceptance. The low-level perturbative heuristics are those used in the study by Burke et al. [11] described above. Evolutionary algorithms have previously been used to search heuristic spaces of both constructive and perturbative low-level heuristics. Han et al. [14, 15] employ the use of a genetic algorithm hyper-heuristic to explore a space of perturbative heuristics to solve the staff trainer scheduling problem. The work done by Pillay [16] examines the use of an evolutionary algorithm selection constructive hyperheuristic to provide a solution to the examination timetabling problem. A similar approach is taken in this study in employing an evolutionary algorithm to search a space of low-level perturbative heuristic combinations to solve the nurse rostering problem. IV. EVOLUTIONARY ALGORITHM HYPER-HEURISTIC This section presents the evolutionary algorithm hyperheuristic used to solve the nurse rostering problem. The algorithm is generational with a set number of generations being implemented. An initial population is firstly created. This population is then iteratively refined during each generation by means of evaluation, selection and regeneration (using genetic operators). The overall algorithm implemented is outlined in Figure 1. The following sections describe each of these processes. Create initial population Repeat Evaluate population Select parents using tournament selection Apply genetic operators Until a maximum number of generations has been reached or the solution has been found Figure 1. Generational EA implemented A. Chromosome representation and initial population generation An individual in the population is represented by a string formed by characters representing low-level perturbative heuristics. Various low-level heuristics, based on those reported in the literature and human intuition, were tested during trial runs and the following performed well and were hence included in the heuristic set: Swap day first improving or equal accept first (t) - Selects two nurses and a day randomly. The shifts on the day are swapped for both nurses. Accepts the first equal or improving solution. Swap day first improving or equal accept best found (s) - Selects two nurses and a day randomly. The shifts on the day are swapped for both nurses. Accepts all improving or equal but will keep trying until a limit of tries is reached. Swap subset of days first improving or equal accept first (y) - Selects two nurses randomly and then selects a random number of days on which the shifts for those nurses are swapped. Accepts the first equal or improving solution. Swap subset of days improving or equal accept best found (x) - Selects two nurses randomly and then selects a random number of days on which the shifts for those nurses are swapped. Accepts all improving or equal but will keep trying until a limit of tries is reached. 2012 Fourth World Congress on Nature and Biologically Inspired Computing (NaBIC) 157

Swap subset of weekdays first improving or equal accept first (w) - Selects two nurses randomly and then selects a random number of weekdays. The shifts on Accepts the first equal or improving solution. Swap subset of weekdays improving or equal accept best found (v) - Selects two nurses randomly and then selects a random number of weekdays. The shifts on Accepts all improving or equal but will keep trying until a limit of tries is reached. Swap subset of weekends first improving or equal accept first (u) - Selects two nurses randomly and then selects a random number of weekends. The shifts on Accepts the first equal or improving solution. Swap subset of weekends improving or equal accept best found (c) - Selects two nurses randomly and then selects a random number of weekends. The shifts on Accepts all improving or equal swaps but will keep trying until a limited number of tries is reached. Blank move (b) - This heuristic has no semantic effect and is an explicitly defined intron that in a way creates noise and aims to prevent good building blocks from being divided. Each element of the initial string is randomly selected from the set of low-level perturbative heuristics. A character is used to represent each heuristic. These have been specified in brackets as part of the definition of each heuristic above. A limit, which is a function of the number of shifts to be assigned, is placed on the length of each individual. An example of an individual is ssttttvvc. B. Evaluation and Selection Prior to the application of the EA-SPHH an initial roster is created by randomly allocating shifts to nurses. Each element of the population is evaluated by using it to improve a roster. For example sttv will be used to improve a roster applying each of the low-level heuristics in order to the roster. In successive generations the best roster created from the previous generations is retained and each individual is applied to this roster. Thus, a best roster is stored and adapted throughout a run of n generations. This approach is different from that taken in similar studies employing evolutionary algorithm hyper-heuristics for rostering. The fitness of an individual is the sum of the hard and soft constraint violations of the roster adapted using the individual and this value is minimized. Tournament selection is used to choose parents of the next generation. This selection method returns the fittest individual in a tournament of t randomly selected individuals. C. Genetic Operators The population of each generation is created by applying the mutation, crossover and permutation operators to selected parents. Trial runs indicated the benefit of the permutation operator in addition to mutation and crossover. This was verified by performing ten runs with and without the use of the permutation operator. This is illustrated in Table 1 below. The following information is displayed: 1 - Permutation rate 2 - Mutation rate 3 - Crossover rate 4 - Generation on which the solution with the minimum soft constraint cost was found. 5 - Time taken, in seconds, to find a solution with the minimum cost 6 - Percentage of runs producing the minimum soft constraint cost 1. Table 1. Comparison of Genetic Operator Application Rates 1 2 3 4 5 6 10% 30% 60% 10 250.62 70% 0% 40% 60% 12 324.06 50% 0% 30% 70% 17 551.52 50% There is no limit placed on the size of the offspring created. The mutation operator randomly chooses a mutation point in an individual and replaces the low-level heuristic at this point with a randomly chosen heuristic. An example of mutation is illustrated in Figure 2. In this example the mutation point has randomly selected to be 4 and the heuristic at this point has been replaced by t. Individual before mutation: s s t v v u v t t s Individual after mutation: s s t t v u v t t s Figure 2. Example of mutation The crossover operator combines two individuals selected using tournament selection. This is achieved by selecting two random points, one in each individual. The substrings created by the crossover points are swapped to create two offspring. The fitter of the two offspring is returned as the result of the process. An example of crossover is depicted in Figure 3. 1 Is the best known soft constraint cost for each problem. 158 2012 Fourth World Congress on Nature and Biologically Inspired Computing (NaBIC)

Selected individual one: s s t v v u v t t s Selected individual two: s t s b t u v Best Known Result First International Nurse Rostering Problem Instances 01 06 04 Late 10 56 54 58 32 66 43 First child: s s t t u v Second child: s t s b t v v u v t t s Evaluate each individual and return the best newly created individual. The permutation operator performs a shuffle of the elements of the string thereby changing the position of each heuristic. This process is illustrated in Figure 4 where it can be seen that some elements have been moved randomly e.g. t is now at position 3. Individual before permutation: t s s u v v t v s t Individual after permutation: s s t v v u v t t s V. EXPERIMENTAL SETUP The EA hyper-heuristic was evaluated on six randomly chosen problems that were used for the track of the first International Nurse Rostering competition [8]. Six problems were randomly selected for testing purposes. The details for these problems are listed in Table 2. Table 2. Description of problem instances First International Nurse Rostering Problem Instances 01 06 04 Late 10 Nurses 10 10 10 10 10 10 Skills 1 1 1 1 1 1 Shift Types 4 4 4 3 4 4 Cover Size 152 152 152 136 168 152 Unwanted Patterns 3 3 3 4 8 0 Contracts 4 4 4 3 3 3 Day Off Requests Shift Off Requests Figure 3. Example of crossover Figure 4. Example of permutation 0 100 100 100 100 100 50 50 50 50 50 50 Due to the stochastic nature of genetic algorithms, ten runs, each using a different random number generator seed, was performed for each problem. The parameter values used by the evolutionary algorithms are listed in Table 3. These have been obtained empirically by performing trial runs. Table 3. EA Parameter values Parameter Value Population Size 100 Initial String Limit Tournament size 5 Crossover percentage 60% Mutation percentage 30% Permutation percentage 10% ½ number of shifts The EA-SPHH was implemented in Java 1.7 and all simulations were run on an Intel I7 (34 GHz) with 8 GB of RAM and Windows 7. VI. RESULTS AND DISCUSSION EA-SPHH was able to produce feasible rosters for all the problems it was evaluated on. Table 4 displays the minimum and average soft constraint cost and runtimes over ten runs for each problem. Table 4. Costs and Runtimes Problem Cost Runtime (in seconds) Min Avg. Min. Avg. 01 56 56.8 219.1 1713.48 06 58 59.3 322.47 1238.76 Spring 54 55 230.16 1335.06 33 38.9 395.33 3986.73 04 67 69.3 2696.33 13343.68 SpringLate10 46 49.8 295.96 2741.97 For each problem the individual producing the best solution quality roster differed from one run to another. Different combinations also proved to be more effective for the different problems. Hyper-heuristics aim to generalize well over a set of problems instead of producing the best solution for one ot two problems. Furthermore, the runtimes of hyper-heuristics can be expected to be higher than standard optimization given that the evaluation of each heuristic combination involves creating a solution using the combination. However, for completeness and to get a feel of the effectiveness of the EA hyper-heuristic, the results obtained by the hyper-heuristic was compared to the methods cited in 2012 Fourth World Congress on Nature and Biologically Inspired Computing (NaBIC) 159

the literature as producing the best results for the same set of problems. These include: 1. The perturbative selection hyper-heuristic using random selection and simulated annealing [10]. 2. The ejection chain and branch and price algorithm hybrid implemented by Burke et al. [17]. 3. The constraint optimization approach taken by Konobe [18]. 4. Integer programming hybrid [19]. 5. Adaptive local search [20]. 6. Integer programming [21]. 7. Harmony search [22]. 8. Adaptive neighbourhood search [23]. The best soft constraint cost produced by these methods and EA-SPHH are listed in Table 4. Note that a hyphen indicates that the method was not able to find a feasible solution. The best results are highlighted in bold. 01 Table 5. Comparison of Results 06 04 1 57 59 54 32-46 2 56 58 54 - - 43 3 56 58 54 32 67 44 4 56 58 54 33 67 44 5 56 58 54 - - - 6 56 58 54 32 66 43 7 62 63 60 55 94 77 8 56 58 54 32 66 43 EA- SPHH 56 58 54 33 67 46 Late 10 It is clear from Table 3 that not all methods were able to find feasible solutions for all of the problems. EA-SPHH has performed well as it was firstly able to produce feasible solutions for all six nurse rostering problems. Furthermore, the best soft constraint costs obtained by the EA-SPHH are either the known minimum or close to the know minimum for all six problems. VII. CONCLUSION AND FUTURE WORK This paper reports on a preliminary study conducted to ascertain the potential of an evolutionary algorithm selection perturbative hyper-heuristic in solving the nurse rostering problem. The study identified the low-level perturbative heuristics for this problem and the overall architecture for the EA hyper-heuristics. The study introduced a new genetic operator, namely, permuation, which improved the performance of the EA for this domain. The study also revealed the effectiveness of a shared memory approach which involved maintaining the best timetable over a run and improving this timetable during each generation. The EA- SPHH was evaluated on six nurse rostering problems. EA- SPHH was able to produce feasible rosters for all six problems. Furthermore, the soft constraint cost of the rosters were comparable to and in some cases better than other methods applied to the same set of problems. This study has clearly shown the potential of the EA- SPHH in solving the nurse rostering problem. Future work will involve conducting an analysis of the low-level heuristics used to evaluate their performance and will possibly examine the use of other perturbation heuristics such as allocate and de-allocate as well as testing the EA- SPHH on additional nurse rostering problems. REFERENCES [1] E. K. Burke, P. De Causmaecker, G. Vanden Berghe, and H. Van Landeghem, " The State of the Art of Nurse Rostering", Journal of Scheduling, Vol. 7, 2004, pp. 441-499. [2] E. K. Burke, M. Hyde, G. Kendall, G. Ochoa, E. Ozcan, and J. Woodward, "A Classification of Hyper-Heuristic Approaches", Handbook of Metaheuristics, Vol. 146, 2010, pp. 449-468. [3] U. Aickelin, and K. Dowsland, Exploiting Problem Structure in a Genetic Algorithm Approach to a Nurse Rostering Problem. Journal of Scheduling, 3 (3), 2000, pp. 139-153. [4] P. De Causmaecker, and G. Vanden Berghe, A Categorization of Nurse Rostering Problems. Journal of Scheduling, 2011, pp. 14: 3-16. [5] B. Cheng, H. Li, A. Lim, and B.Rodrigues, Nurse Rostering Problems - A Bibliographic Survey. European Journal of Operational Research, 151, 2003, pp. 447-460. [6] E. Burke, P. De Causmaecker, G. Vanden Berghe, and H. Van Landeghem, The State of the Art of Nurse Rostering. Journal of Scheduling,7, 2004, pp. 441-499. [7] G. Ochoa, M. Hyde, T. Curtois, J.A. Vazquez-Rodriguez, J.Walker,, M. Gendreau, G. Kendall, B. McCollum, A. Parkes, S. Petrovic, and E.K. Burke, HyFlex: A Benchmark Framework for Cross-Domain Heuristic Search, Proceedings of the European Conference on Evolutionary Computation in Combinatorial Optimization (EvoCOP 2012), 7245, 2012, 136-147. [8] S. Haspelagh, P. De Causmaecker, M. Stolevik, and A. Schaerf, The First International Competition on Nurse Rostering 2010, Annals of Operations Research, 2012 DOI: 10.1007/s10479-012-1062-0. [9] E. Burke, E. Hart, G. Kendall, J. Newall, P. Ross, and S. Schulenburg. Hyper-Heuristics: An Emerging Direction in Modern Research. Handbook of Metaheuristics. Chapter 16, 2003, pp. 457-474. [10] B. Biligan, P. De Causmaecker, and G. Vanden Berghe, A Hyper- Heuristic Approach to Belgian Nurse Rostering, Proceedings of the Multidisciplinary International Conference on Scheduling: Theory and Applications (MISTA 2009),2009, 683-689. [11] E.K. Burke, G. Kendall, and E. Soubeiga, A Tabu-Search Hyper- Heuristic for Timetabling and Rostering. Journal of Heuristics, 6, 2003, pp. 451-470. [12] B. Biligan, P. Demeester, M. Misir, W. Vancroonenburg, G. Vanden Berghe, and T. Wauters, A Hyper-Heuristic Combined with a Greedy Shuffle Approach to the Nurse Rostering Problem", Proceedings of PATAT 2010, 2010, https://www.kuleuvenkulak.be/~u0041139/nrpcompetition/abstracts/ l3.pdf. Accessed 31 August 2012. [13] R. Bai, E. K. Burke, G. Kendall, J. Li, and B. McCollum, A Hybrid Evolutionary Approach to the Nurse Rostering Problem. IEEE Transactions on Evolutionary Computing, 14(4), August 2012, pp. 580-590. [14] L. Han, G. Kendall, and P. Cowling, An Adaptive Length Chromosome Hyperheuristic Genetic Algorithm for a Trainer Scheduling Problem, Proceedings of the fourth Asia-Pacific Conference on Simulated Evolution And Learning, (SEAL'), Orchid Country Club, Singapore, 20, 18-22. [15] L. Han and G. Kendall, An Investigation of a Tabu Assisted Hyperheuristic Genetic Algorithm", Proceedings of IEEE Evolutionary Computing, CEC 03, 2003, 2230 2237. [16] N. Pillay, Evolving Hyper-Heuristics for the Uncapacitated Examination Timetabling Problem. Journal of the Operational Research Society, 63, 2012, 47-58. 160 2012 Fourth World Congress on Nature and Biologically Inspired Computing (NaBIC)

[17] E. K. Burke and T. Curtois, An Ejection Chain Method and a Branch and Price Algorithm Applied to the Instances of the First International Nurse Rostering Competition. Technical Report, 2012, School of Computer Science, University of Nottingham. [18] K. Nonobe, INCR2010: An Approach Using a General Constraint Problem Solver, Proceedings of PATAT 2010, 2010, https://www.kuleuven-kulak.be/~u0041139/nrpcompetition/abstracts/ m3.pdf. Accessed 31 August 2012. [19] C. Valouxis, C. Gogos, G. Goulas, P. Alefragis, and E. Housos, A Systematic Two-Phase Approach for the Nurse Rostering Problem. European Journal of Operational Research, 219(2), June 2012, pp. 425-433. [20] Z. Lu, J.K. Hao, Adaptive Local Search for the First Nurse Rostering Competition, Proceedings of PATAT 2010, 2010, https://www.kuleuven-kulak.be/~u0041139/nrpcompetition/abstracts/ m4.pdf. Accessed 31 August 2012. [21] H.G. Santos, T. Toffolo, S. Ribas, and R. Gomes, Integer Programming Techniques for the Nurse Rostering Problem, Proceedings of PATAT 2012, August 2012, 256-283. [22] M. A. Awadallah, A. T. Khader, and M. A. Al-Betar, Nurse Rostering Using Modified Harmony Search, Proceedings of the Second International Conference on Swarm, Evolutionary and Memetic Computing (SEMCCO 11), Vol. II, 2011, 27-37. [23] Z. Lu and J. Hao, Adaptive Neighbourhood Search for Nurse Rostering. European Journal of Operational Research, 218 (3), 2012, pp. 865-876. 2012 Fourth World Congress on Nature and Biologically Inspired Computing (NaBIC) 161