An Improvement Technique for Simulated Annealing and Its Application to Nurse Scheduling Problem

An Improvement Technique for Simulated Annealing and Its Application to Nurse Scheduling Problem Young-Woong Ko, DongHoi Kim, Minyeong Jeong, Wooram Jeon, Saangyong Uhmn and Jin Kim* Dept. of Computer Engineering Hallym University Chuncheon, Gangwondo 200-702 Republic of Korea {yuko,kdh,jmy,jwr,suhmn,jinkim}@hallym.ac.kr Abstract The simulated annealing was perceived as a useful method for many intractable problems. However, it needs additional strategies to cope with time complexity due to an initial state and search space reduction. In this work, we suggested an efficient transition rule and applied it to a nurse scheduling problem. It uses a cost matrix to reduce a set of candidates, which results in performance improvement. The experimental results showed that the suggested method generated a feasible solution for the nurse scheduling problem faster in time and better in quality compared to traditional simulated annealing. Keywords: nurse scheduling problem, simulated annealing, cost matrix 1. Introduction Daily work in any organization providing 24/7 services is divided into shifts. Shifts are defined as periods of time during which different groups of employees are in the line of duty. In such organizations, employees are assigned into a set of shifts in such a way that several constraints are satisfied. The constraints may be set up by staffing requirements, the rules by the administration and labor contract clauses. In nurse scheduling problem (NSP), nurses are employees to be assigned into a set of shifts and rest days in a timetable called nurse roster. Each shift must meet these constraints. It was proven to be NP-hard even if it was with only a subset of real world constraints [1]. In early days, the problem was formulated as a selection of a timetable that minimized an objective function that balanced the trade-off between staffing coverage and nurses' schedule preferences [2, 3]. They focused on the simplified versions of the problem which were too small or ignored too many constraints to be practical. Abdannadher et al., tackled the problem using the Constraint Logic Program (CLP) framework [4]. Jan et al. and Aickelin et al., applied genetic algorithm (GA) [5, 6] and Li et al., employed Bayesian optimization algorithm [7]. Kundu et al., applied genetic algorithm and simulated annealing (SA) and compared their performances with others [8]. The nurse scheduling problems covered in these researches are categorized by certain criteria [9]. In this paper, we solved the cyclic nurse scheduling problem with the following components [8]: Firstly, the preference of each nurse on a day and a shift. Secondly, minimal coverage constraints of required nurses per shift and per day like any shift scheduling problem. Thirdly, case-specific constraints by personal time requirements, 269

specific workplace conditions, etc. Hence, the objective of the NSP is to satisfy nurses' requests as much as possible while fulfilling the employers' concerns. In this paper, we suggest simulated annealing with a cost matrix (CMSA) to find a schedule that optimizes a cost function and satisfies a set of constraints. In the next section, we will briefly introduce NSP and its cost function and in Section 3, SA and the cost matrix and the transition rule in CMSA. Section 4 is concerned with experiments and results. Finally, conclusion and further work are discussed in Section 5. 2. Problem Description 2.1. Nurse Scheduling Problem NSP is to create weekly or monthly schedules for n nurses by assigning one out of a number of possible shift patterns to each nurse. These schedules have to satisfy working contracts and meet demands for a number of nurses of different grades on each shift, while being seen to be fair by the staff concerned. Therefore NSP is essentially a scheduling problem which suits a number of constraints. Constraints are usually classified as soft or hard constraints. Hard constraints must be satisfied fully in any working schedule and do not allow any violation. That is, a schedule which does not satisfy hard constraints cannot be a feasible solution. Possible examples include restrictions on the number of nurses for each shift, maximum number of shifts in a week or a month, etc. Soft constraints may be broken, but of which breaches must be minimized. In other words, soft constraints are desired to be satisfied, but violation does not make it an infeasible solution. Some examples are demands for a desired day-off or doing certain shift on a certain day with a certain nurses, etc. Generally, every nurse works on three shifts, morning, evening, and night, and rests for some holidays. There are various kinds of hard and soft constraints we have to consider. However, because the main objective of this study is to show fast and efficient SA approach, we confined constraints as follows. (a) Hard constraints (i) There are constraints on the number of nurses for each working shift per day. The number of nurses for morning, evening, and night shift should be within a range between the maximum and the minimum values. (ii) There are constraints for the working patterns. Morning after night shift, evening after night, morning after evening shift and three consecutive night shifts should be avoided. (b) Soft constraints There are constraints for the total number of off-days (o), night (n), morning (m) and evening (e) shifts during a certain periods for each nurse. 2.2. Cost Function We have to define a cost function to evaluate schedules for each nurse. Let N and D be the number of nurses and days, and s be one of the three shifts or an off-day required to be scheduled. Then, NSP is represented as a problem to decide an N D matrix, so that each element x ij of the matrix express that nurse i works on day j where x ij = {m, e, n, o}. 270

(a) To evaluate the violation of hard constraint (i), we define m j, e j, n j be the total number of nurses for morning, evening, and night shift on day j. If any of these numbers are not between minimum and maximum number of each shift (m min, m max ), (e min, e max ), (n min, n max ), cost c 1 will be increased by 1. (b) To evaluate the violation of hard constraint (ii), working patterns will be examined. Any violation of working pattern will increment cost c 2 by 1. Figure 1 shows the description of calculation of c 1 and c 2. C1(){ for(j=1;j<=n;j++){ if ((m j <m min ) (m j >m max )){c 1 =c 1 +1;} if ((e j <e min ) (e j >e max )){c 1 =c 1 +1;} if ((n j <n min ) (n j >n max )){c 1 =c 1 +1;} if (cost increased) {for(i=1; i<=d;i++){v ij =1;}} } C2(){ For (i=1;i<=d;i++){ For (j=1;j<=n;j++){ if ((x ij-1 ==n)&&(x ij ==m)){c 2 =c 2 +1;} if ((x ij-1 ==n)&&(x ij ==e)){c 2 =c 2 +1;} if ((x ij-1 ==e)&&(x ij ==m)){c 2 =c 2 +1;} if ((x ij-2 ==n)&&(x ij-1 ==n)&&(x ij ==n)){c2=c 2 +1;} if (cost increased){v ij =1;} } } } Figure 1. Description of Cost Function c1() and c2() and Cost Matrix v (c) To evaluate the violation of soft constraint, we define M i, E i, N i, O i be the total number of m, e, n, o for nurse i during the period of D and M req, E req, N req, O req be the required number of m, e, n, o for all nurses during the period of D. If any of these numbers M i, E i, N i, O i are not matched to required numbers M req, E req, N req, O req respectively, cost c 3 will be increased by 1. Figure 2 shows the description of calculation of c 3. Figure 2. Description of Cost Function c3() and Cost Matrix v 271

Different weights can be assigned for the costs c1, c2, and c3. Then, the final cost function f is f = c 1 w 1 + c 2 w 2 + c 3 w 3 where w 1, w 2 and w 3 are weight values for c 1, c 2 and c 3. Our goal is to minimize the cost function f so as to find an optimal nurse schedule. The simplest method to find an optimal nurse schedule is a brute force approach to evaluate all possible nurse schedules and find a feasible one with the minimum cost. It guarantees that the solution is feasible with the minimum cost. The number of all possible nurse schedules is 4 DxN. If D and N increase, this approach is intractable. This is a class of problems for which it is believed that no efficient algorithm exists, called NPhard (Papadimitriou, 1993). In other words, the algorithms that are guaranteed to find an optimal solution in reasonable time with the size of D and N may not exist. To overcome this problem, we use simulated annealing which is an approximation algorithm. SA provides an acceptable good solution in a reasonable amount of time, rather than an optimal solution. 3. Algorithmic Flow of CMSA 3.1. Simulated Annealing SA is a probabilistic approach that can be used to find a global minimum of a function for combinatorial optimization problems. To apply this algorithm, a set of state spaces S={s 1,, s n } and a cost function C:S R, where R is the set of real numbers, should be defined. A real value C(S) should be assigned to each state s S. The goal of the optimization problem is to find an optimal state s opt whose score is min(max){s i 1 }. SA continuously generates a new candidate state s new from a current state s current by applying transition rules and acceptance rules [10, 11]. The criteria of the acceptance rules are: (a) If, accept a new state s new. (b) If, accept a new state s new with probability ( ) where T is a temperature and ( ) ( ) is a cost difference. Probability ( ) prevents the system from fixation at a local minimum. A state s current is called a local minimum if there is no new state s new in S that is generated from the state s current by applying the transition rules and has a lower cost than that of s current. Temperature T controls a probability to accept a new state s new. Initially, T starts from a high temperature and after each iteration T decreases based on an annealing schedule and becomes zero eventually. The probability of accepting a new state with a higher cost than that of the current also decreases as temperature T decreases. If a careful annealing schedule and a number of iterations are given, SA converges to a global minimum state s opt. Because of efficient performance by this characteristic, SA has been applied to many combinatorial problems [12, 13]. The main disadvantage of SA is its requirement of a large amount of computation time [1]. This is because SA is based on Monte Carlo methods, which allows a new state with a higher cost than that of a current state. To reduce this computation time, a speedup strategy using a cost matrix is used in this paper. 272

3.2. Cost Matrix and Transition Rule for CMSA The very first nurse schedule s first which is an N D matrix is obtained by randomly assigning each nurse to one of the three shifts or day-off on each day. The cost E first is calculated by applying cost function c 1, c 2 and c 3. This cost becomes current cost E current. And new schedule is generated by applying a transition rule to current schedule. A traditional transition rule is that the value of randomly chosen cell from the current sche d- ule is changed randomly to make a new nurse schedule s new. In this study we use a cost matrix v which is also an N D matrix to apply a transition rule to the current schedule to make a new state more efficiently. The value of each cell in the cost matrix v is assigned to 0 or 1 when the cost of new schedule is calculated. Initially the value in each cell of v is set to 0. When the cost functions c1(), c2(), and c3() are applied to the new schedule and if the cost is increased by 1, the value of the corresponding cell of v is set to 1. Figure 1 shows how to set the value of v. The value 1 in a cell of the cost matrix v shows the violation of constraints. Now we apply a tradition rule to the cell x ij with certain probability p only if the corresponding cell v ij =1. Figure 3 shows the description of our transition rule. Figure 3. The Transition Rule for SA 3.3. Cost Matrix and Transition Rule Temperature Scheduling for CMSA The schedule implemented in SA is where is a constant defining the rate of annealing, i is the iteration number, T i the initial temperature and T the current temperature. The value of can be easily calculated from the total number of iterations, k, the final temperature, T f, and T i ( ) Figure 4 shows the pseudo code of SA for the experiments in this paper. 4. Experiments and Results The proposed CMSA and traditional SA(TSA) were implemented in C and run on a PC with an Intel Core(TM) i5-2520m 2.5 Ghz CPU and 4GB of memory. All the constraints described in this paper were incorporated in both methods with exact same conditions. The goal was to check whether the algorithm could actually generate an acceptable NSP and to compare proposed SA with traditional SA. The random number generator rand() was used. Identical problem instances were solved by both methods. 100 problems were generated randomly for each set. 15 nurses for N and from 1 to 4 weeks for D were used and w 1 =5, w 2 =5 and w 3 =1 were assigned. Hard constraints are same for all the problems (m min =4, m max =6), (e min =3, e max =5) and (n min =3, n max =5), and 273

soft constraints are proportional to the periods (M req =2, E req =2, N req =2 and O req =1 for 1 week). The probability p in Figure 3 was 2 %. Figure 4. Pseudocode for SA The methods were compared on the basis of four criteria: number of problem solved with cost=0 (E opt =0), the average cost of the solution obtained, and the average number of iterations to reach the final state s opt and the time T opt. The CMSA outperformed TSA in every aspect. Table 1 shows the performance of two methods. Table 1. Comparison of CMSA and TSA Period Method E opt =0 E opt I opt / k T opt (sec) 1week 2 weeks 3 weeks 4 weeks CMSA 79/100 0.42 305612 / 1x10 6 1.1 TSA 4/100 5.06 546054 / 1x10 6 1.9 CMSA 54/100 1.26 2900782 / 5 x10 6 23.9 TSA 0/100 8.88 2312737 / 5 x10 6 30.6 CMSA 72/100 0.66 12192435 / 20 x10 6 155 TSA 0/100 13.14 9588709 / 20 x10 6 121 CMSA 97/100 0.06 51631978 / 100 x10 6 926 TSA 0/100 17.28 56574940 / 100 x10 6 1607 Both methods solved all of problem instances with which did not offend any of hard constraints in all periods. CMSA generated schedules with optimal cost (E opt =0) in all periods whereas TSA could not generate schedules with optimal cost except 1 week. Also the average costs E opt from CMSA were smaller than TSA and cpu time T opt of CMSA was faster than that of TSA. CMSA was very effective compared to TSA. In all the quality of the solution from CMSA was very impressive due to its powerful transi- 274

tion rule with a cost matrix. In Figure 5, we presented the differences in costs and number of iterations for CMSA and TSA. It showed the average of 100 problem instances. It is clearly shown that the cost of the solution approached the optimum faster by CMSA than TSA. Figure 5. Comparison of Number of Iterations and Cost for CMS and TSA 5. Conclusion and Future Work In this paper, we proposed an efficient method for NSP using SA. In this method, a cost matrix is used for transition rule. This approach generated a nurse schedule faster in time and better in quality than traditional SA. Although we have presented this work in terms of nurse scheduling, it should be noticed that the main idea of the approach 275

could be applied to many other scheduling problems. Future research is aimed at experiments on the real hospital data with more constraints and diversity of requirements. Acknowledgements This research was supported by Hallym University Research Fund, 201 1(HRF- 201208-004) and Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education, Science and Techno l- ogy (No. 2009-0077545). References [1] T. Osogami and H. Imai, Classification of Various Neighborhood Operations for the Nurse Scheduling Problem, in Proceedings of the 11th International Conference on Algorithms and Computation(ISAAC '00), Springer-Verlag, (2000), pp.72-83. [2] H. E. Miller, W. P. Pierskalla and G. J. Rath, Nurse Scheduling using Mathematical Programming, Operations Research, vol. 24, no. 5, (1976), pp. 857-870. [3] D. M. Warner and J. Prawda, A Mathematical Programming Model for Scheduling Nursing Personnel in a Hospital, Management Science, vol. 19 (4-Part-1), (1972) December, pp. 411-422. [4] S. Abdennadher and H. Schlenker, Nurse Scheduling using Constraint Logic Programming, in Proc. AAAI 99/IAAI 99, (1999), pp.838-843. [5] A. Jan, M. Yamamoto and A. Ohuchi, Evolutionary Algorithms for Nurse Scheduling Problem, in Proc. The 2000 Congress on Evolutionary Computation, (2000), pp.196-203. [6] U. Aickelin and K. A. Dowsland, An indirect Genetic Algorithm for a Nurse-Scheduling Problem, Computers & Operations Research, vol. 31, no. 5, (2004) April, pp. 761-778. [7] J. Li and U. Aickelin, A Bayesian Optimization Algorithm for the Nurse Scheduling Problem, in Proc. The 2003 Congress on Evolutionary Computation, (CEC 03), vol.3, (2003) December, pp. 2149-2156, Canberra, Canada. [8] S. Kundu, M. Mahato, B. Mahanty and S. Acharyya, Comparative Performance of Simulated Annealing and Genetic Algorithm in Solving Nurse Scheduling Problem, in Proc. Int'l Multi Conference of Engineers and Computer Scientists 2008, vol. 1, (2008) March, pp. 1-5, Hong Kong. [9] P. Causmaecker and G. Berghe, A Categorisation of Nurse Rostering Problems, Journal of Scheduling, vol. 14, no. 1, (2011) Feburary, pp. 3-16. [10] N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller and E. Teller, Equation of State Calculations by Fast Computing Machines, Journal of Chemical Physics, vol. 21, no. 6, (1953) June, pp.1087-1092. [11] D. Kim, S. Uhmn and J Kim, Finding Relevant SNP Sets and Predicting Disease Risk Using Simulated Annealing, Int l J. Software Eng. And Its Appl., vol. 6, no. 3, (2012) July, pp.81-88. [12] E. Mirsadeghi and M. S. Panahi, Hybridizing Artificial Bee Colony with Simulated Annealing, Int l J. Hybrid Info. Tech., vol. 5, no. 4, (2012) October, pp.11-18. [13] M. Benaddy and M. Wakrim, Simulated Annealing Neural Network for Software Failure Prediction, Int l J. Software Eng. and its Appl., vol.6, no.4, (2012) October, pp.35-46. Authors Young-Woong Ko received the B.S., M.S. and Ph.D. degrees in Department of Computer Engineering from Korea University, Korea, in 1997, 1999 and 2003 respectively. Since 2003, he has been with the Department of Computer Engineering, Faculty of Engineering, Hallym University, where he is currently a professor. His research interests are in the areas of storage system, operating systems, and embedded systems. 276

Dong-Hoi Kim received MS and Ph.D degree in Computer engineering from Hallym University in 2002 and 2006, respectively. Since then he has been working as a lecturer on computer engineering at Hallym University. His research includes bioinformatics and data mining techniques. Minyeong Jeong received bachelor's degree in Computer Engineering from Hallym University in 2012. Since then she is in Master course in Computer Engineering at Hallym University. Her research interests include bioinformatics and algorithm. Wooram Jeon received bachelor's degree in Computer Engineering from Hallym University in 2012. Since then he is in Master course in Computer Engineering at Hallym University. His research interests include bioinformatics and pattern recognitition. Saangyong Uhmn received an MS, Ph.D degree in Computer Engineering at Hallym University in 1997, 2012 respectively. He worked as a lecturer at Hallym University. His research interests include distributed computing, algorithms and bioinformatics. Jin Kim received an MS degree in computer science from the college of Engineering at Michigan State University in 1990, and in 1996 a PhD degree from Michigan State University. Since then he has been working as a professor on computer engineering at the Hallym University. His research includes Bioinformatics and data mining. 277

278