Solving the chemotherapy outpatient scheduling problem with constraint programming

Journal of Applied Operational Research (2014) 6(3), 135 144 Tadbir Operational Research Group Ltd. All rights reserved. www.tadbir.ca ISSN 1735-8523 (Print), ISSN 1927-0089 (Online) Solving the chemotherapy outpatient scheduling problem with constraint programming Shoshana Hahn-Goldberg 1, *, J. Christopher Beck 1, Michael W. Carter 1, Dr. Maureen Trudeau 2, Philomena Sousa 2 and Kathy Beattie 2 1 Centre for Research in Healthcare Engineering, University of Toronto, Toronto, ON, Canada 2 Odette Cancer Centre, Sunnybrook Health Sciences Centre, Toronto, ON, Canada Abstract. Chemotherapy outpatient scheduling is a complex problem that has not been widely studied. In this paper, we use constraint programming to model and solve a deterministic chemotherapy outpatient scheduling problem. We address the question of what information from the problem domain to include in order to create the best model. Our evaluation of mathematically different models is based on a combination of the time to generate a solution and the quality of that solution when executed in a simulated chemotherapy clinic. Several constraint models are developed, each based on abstractions of the problem domain. A simulation is used to evaluate them based on their solvability and the quality of their solutions. We then compare our best constraint model to a mixed integer programming (MIP) model. Our experimental results demonstrate that constraint programming out-performs the MIP model and, when applied to an appropriate problem abstraction, can lead to a potential improvement in makespan of 14% to 37%. Keywords: optimization; healthcare; chemotherapy; scheduling; constraint programming * Received September 2013. Accepted May 2014 Introduction The delivery of chemotherapy at the Odette Cancer Centre in Toronto, is a four-stage processes consisting of chart review, drug preparation, treatment set-up, and drug delivery. The stages require the use of shared nursing, pharmacy, and clinic resources, resulting in a complex scheduling problem. In this paper, we undertake the development of a number of constraint programming (CP) formulations of the deterministic version of this problem, using the Odette Cancer Centre as a case study. We compare our chosen CP model to an equivalent mixed integer program (MIP) and demonstrate that the latter is unable to solve problems of the required scale. When we scale-down the problem instances, the CP model out-performs the MIP model by almost two orders of magnitude in terms of solution time. Evaluation of solution quality via simulation of the chemotherapy clinic shows a substantial reduction in daily clinic length compared to current practice without an increase in resources. * Correspondence: Shoshana Hahn-Goldberg, Centre for Research in Healthcare Engineering, Mechanical and Industrial Engineering, University of Toronto, 5 King's College Road, Toronto, Ontario M5S 3G8 Canada. E-mail: shoshana@mie.utoronto.ca

136 Journal of Applied Operational Research Vol. 6, No. 3 The chemotherapy outpatient scheduling problem is an interesting application for operations research as a number of aspects of the problem that present a challenge to traditional optimization techniques. First, there is large variability among treatments. Treatment durations can range from 15 minutes to 8 hours. Second, chemotherapy has a unique property where nurses are required to attend to a single patient at the beginning of treatment in order to connect them, but they can monitor multiple patients during for most of the treatment. Last, the chemotherapy problem at the Odette Cancer Centre is large, with around 100 patient appointments each day, 29 primary resources (chairs), and 13 to 15 nurses. In this paper we use constraint programming to model the deterministic chemotherapy outpatient scheduling problem. Constraint programming is the study of computational systems based on constraints (Rossi et al. 2006). A constraint satisfaction program is a set of variables each with a finite domain and a set of constraints that restrict which values variables can take. Although constraint programming is well suited for scheduling problems (Baptiste et al. 2001, Beck and Fox 1998), it has never been used for chemotherapy scheduling. Interested readers are referred to Baptiste et al. (2001) for a detailed description. In constraint programming, modelling refers to developing different, yet logically equivalent, models for a mathematically well-defined problem with the goal of creating a representation that will be most efficiently solved (Frisch et al. 2005). Here, the focus is on the fundamentally different question of the definition of a constraint model for a real problem, an important step that occurs in order to create a mathematically well-defined problem. That step, which will be referred to as the problem definition, is deciding what simplifications can be made to the real problem so that the optimization model can be solved in a reasonable amount of time and will result in a solution that is useful in the real world. By creating and comparing several problem definitions, we study how they compare in solvability and solution quality. Our primary contribution is the application of constraint programming to solve the deterministic chemotherapy outpatient scheduling problem. Chemotherapy outpatient scheduling problems have had very little exposure to optimization. Additionally, constraint programming has never been applied to this problem. In addition, this paper contributes to the literature on applying optimization to real world applications by investigation of the creation and evaluation of a set of problem definitions. Experimenting with different problem definitions is unusual in the literature, yet very common in practice. One contribution of this paper is to encourage more investigation of different problem definitions, so we can understand trade-offs that are typically not explored in the optimization literature. Background Outpatient appointment scheduling involves scheduling patients into a primary care or outpatient clinic. The standard scheduling method is to use an appointment rule comprised of two parts: the appointment block size (how many patients are scheduled at a time) and the appointment interval (the time between appointment start times) (Cayirli and Veral 2003). Approaches for solving appointment scheduling problems typically include stochastic modeling (Erdogen and Denton 2011) or simulation (Cayirli et al. 2006). On the surface, the chemotherapy problem appears to be an appointment scheduling problem that requires the optimal scheduling of outpatient appointments. However, the chemotherapy problem is much more complex because it has multiple stages, many servers, and very wide range of known appointment durations for different treatments. Therefore, techniques that work well for appointment problems are not be able to handle the complexity of the chemotherapy problem. Research on scheduling and sequencing chemotherapy appointments is limited. Recently Santibanez et al. (2012) developed a scheduling system for a large chemotherapy clinic in British Columbia, Canada, using simulation and mixed integer programming. The simulation addressed the problem of assigning a date to each request for a first chemotherapy appointment (as opposed to scheduling the start time of an appointment on a particular day, as is done in this paper) in order to shorten the wait list. The mixed integer programming model looked at scheduling one day of appointments to balance the workload among the nurses. They achieved significant operational improvement in both the delay in informing patients of their appointments, with a 57% reduction in the number of patients exceeding the target of one week; and the wait list for first appointments, with an 83% reduction in wait list size. Turkcan et al. (2012) created an integer programming model to solve a version of the deterministic chemotherapy outpatient scheduling problem that assigned days and start times to appointments. Their main objective was to balance the acuity of patients serviced by a particular nurse as opposed to maximizing utilization. As well, pharmacy constraints were not considered, only four types of cancers were serviced, and the problem instances were smaller than those considered here, with only 20 patients receiving treatment each week.

S Hahn-Goldberg et al 137 Methods In this section, a high-level method for developing and evaluating problem definitions is proposed and applied to the chemotherapy outpatient scheduling problem. Several constraint formulations are developed, each containing different simplifications, corresponding to possible definitions of the problem. We also develop a mixed integer programming (MIP) model and present our experimental design. A Method to Evaluate Problem Definitions In order to determine how different definitions of the chemotherapy outpatient scheduling problem differ in solvability and quality, the following iterative method is proposed: 1. Create a Base model that represents an initial definition of the problem. Any assumptions made in the base model should be discussed with the end users of the system and then documented. 2. Determine a list of potential elements that can be added, removed, or modified in the base model. These elements can be chosen for a variety of reasons including, but not limited to, intuitions for reducing the computational effort. 3. Create the problem definitions and models. When a modification is applied to the base model, a new problem definition is created. Each definition should then be transformed into a mathematical model. 4. Evaluate the models on solvability, the time it takes to deliver a solution. 5. Evaluate the models on quality, the usefulness of the solutions in the real world. The objective functions resultant from different definitions cannot be directly compared because the problems being solved are not equivalent. However, the solutions of the models can be compared in the real world or in a simulation that represents the real world. 6. Iterate as necessary. Collect the experimental results from the previous steps and perform an audit of the problem definitions and models. At this point, a model may be judged to meet all the user requirements and to perform well in both solvability and quality. In such a situation, that definition and model can be chosen. It is also possible that missing requirements are discovered or that none of the models perform acceptably in both solvability and quality. In that case, a new base model should be constructed in the subsequent iteration of step 1. In the following sections, the above method is demonstrated using the chemotherapy outpatient scheduling problem at the Odette Cancer Centre. Base Model Formulation Through interviews with system users, site observations, and an analysis of current scheduling practice, we determined what elements might influence the scheduling of chemotherapy appointments. We then created a definition that encompasses those elements. The following is a description of our Base model. The formulation was developed in constraint programming using COMET 2.1.0 (Dynadec Decision Technologies, Providence RI). Using constraint programming as the modeling technique made it natural to express the constraints, resulting in a model that is easy to understand. Constraint programming was also advantageous because COMET has built-in constraints and search techniques designed specifically for scheduling applications. Additionally, constraint programming makes it simple to experiment with different problem definitions because for our problem, a model for a different definition can be created through the addition or deletion of a global constraint (constraints over multiple variables with special propagation algorithms to make them more efficient). The chemotherapy treatment process at the clinic can be divided into four main stages: (1) chart review, (2) pharmacy or drug preparation, and two stages of treatment, (3) the first stage of treatment or setting up the patient and (4) the second stage of treatment or delivery of drugs. Each patient that arrives at the clinic, must progress through all four stages. The notation for the Base model is as follows: R: set of appointment requests. End: a dummy appointment request whose start time represents the end time of the last appointment of the day. S End : the start time of End. C: set of chairs a maximum of 29 chairs.

138 Journal of Applied Operational Research Vol. 6, No. 3 N3: set of nurses available for stage three, the first stage of treatment there are 13 or 15 nurses; the total capacity is N3. Each test day used for experiments contains the exact number of nurses that worked in the clinic on that day. N4: set of nurses for stage four each 0.25 of a nurse is a resource for a total of 52 or 60 units; the total capacity is N4. N3 and N4 represent the nurses in different roles, setting up and monitoring respectively. Our constraint model ensures that workload of each actual nurse is never exceeded across all roles (see below). Each of the 13 or 15 nurses can monitor up to four patients during stage four. T: set of times ranging from zero through 40. Each time is a 15-minute timeslot for a maximum of ten hours of daily treatment time. DR: set of servers for drug preparation there are five pharmacy technicians at all times; the total capacity is DR. D ir : duration of stage i for patient request r. Stage one represents chart review and has a duration of one time slot. Stage two represents drug preparation and has duration determined by the chemotherapy regimen. Stage three represents setting up the patient for treatment and has duration determined by the chemotherapy regimen. Stage four represents delivery of drugs and monitoring the patient during treatment. Stage four has duration of the complete treatment length determined by the regimen including both the setting up and drug delivery portions of the treatment. The duration of the stage four activity spans both parts of the treatment in order to ensure nursing capacity is respected (see the description of constraint 8). B r : the ready time, when patient request r can begin the first stage. If a patient has no previous appointments on the day of treatment, the ready time is zero. If a patient does have previous appointments, his/her ready time will be the known end time of the previous appointments. l: a constant representing the maximum time allowed between stages one and two, as well as between stages two and three. This time will ensure that any single patient does not wait an excessive amount of time. For each patient request r R, we have three decision variables: the start times of the chart review, drug preparation, and treatment stages. The final stage of treatment is scheduled to start at the same time as stage three., S ir : The start time of stage i for patient request r. It takes a value from 0 through T -1. The constraint programming model of the Base definition is as follows: min S End Subject to:, (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11), (12) The objective of the model is to minimize makespan, equivalently the start time of the End activity. However, the real application has two objectives: maximizing resource utilization and minimizing patient wait times. One abstraction in this model is to use a single objective to reduce complexity. Of the two objectives of the problem, maximizing resource utilization in order to service more patients by way of minimizing the makespan is chosen. Increasing the number of patients that can be serviced is the key objective for the chemotherapy centre because their demand is growing. Also, logically, by minimizing makespan, wait times will tend to be lower, indirectly accounting for the objective of minimizing wait times. However, minimizing makespan does not guarantee a reduction in wait times, so wait times are evaluated when the definitions are tested for quality.

S Hahn-Goldberg et al 139 Constraint (1) ensures only one patient receives chart review at any time. Each stage-one activity is constrained to require one chart review nurse. The disjunctive constraint (Beldiceanu and Demassey 2010) is a global constraint that ensures activities subject to the constraint do not overlap. Constraints (2) through (5) are cumulative constraints (Beldiceanu and Demassey 2010), global constraints that ensure that the capacity of a discrete resource is respected. Cumulative constraints have four parameters: the start times, durations, and resource requirements of each activity and the capacity of that resource. These cumulative constraints represent the pharmacy technicians, nurses, and chairs, respectively. In constraint (5) each nurse is really 0.25 of a nurse. Together with constraints (4) and (8), constraint (5) ensures that at any time a nurse can be setting up or monitoring at most four patients. Constraint (6) ensures the start time for each patient is greater than or equal to his/her ready time. Constraint (7) ensures that stages two and three cannot start before the end time of the preceding stage. Constraint (8) ensures that stage four starts at the same time as stage three. The stage four activity spans both parts of the treatment and the portion of the stage four activity representing stage three must occur at the same time as the actual stage three activity. This approach accounts for the two roles of a nurse: setting-up a patient (stage three) and monitoring (stage four). In having stages three and four start at the same time, 0.25 of a nurse for the monitoring component of treatment is required throughout the entire treatment (constraint 5) and, in addition, one nurse is required during stage three (constraint 4). As a result, 1.25 of a nurse is required during stage three. These constraints ensure that a nurse will not be monitoring four patients, using the maximum capacity of N4, while at the same time setting up another, using the capacity of N3. Constraint (9) ensures no patient finishes treatment after the end of the working day. Constraint (10) ensures End does not start until all other activities have completed. There is a maximum time, l = 2 timeslots, the equivalent of 30 minutes, between stages one and two (constraint 11). The same limit is imposed between stages two and three (constraint 12). These constraints ensure that patients will not have to wait for extra time between stages, even if such a delay would create a shorter schedule. Modifications and Corresponding Problem Definitions For step two in the proposed method, three modifications to the Base model are developed. Each modification results in a different problem definition and corresponding constraint model. 1. Chart Review Modification (CRS) The chart review stage is omitted. Removing a stage should reduce computational effort. Chart review is not included in the current scheduling process and so excluding it should mean the new solution is no worse than current practice. CRS is the same as Base except for the omission of constraints (1) and (12) and the modification of constraints (6) and (7) by replacing stage 1 with stage 2. 2. The Pharmacy Modification (CRS-PS) The chart review and pharmacy stages are omitted. Removing the pharmacy stage should reduce computational effort. Pharmacy time is not directly included in how the process is currently scheduled at the hospital. CRS-PS is the same as CRS except for the omission of Constraints (2), (7), and (13) and the modification of constraint (6) by replacing stage 2 with stage 3. 3. The Ramp Up Modification (CRS-RUS) Rather than completely omitting the pharmacy stage, we represent it implicitly by ramping up the chair capacity. At the start of the day, 21 chairs are available. After an hour, 24 chairs are available. After two hours, all 29 chairs are available. By reducing the number of chairs available, we control how many patients can be serviced at a given time. The number of chairs available at each time point reflects the number of chairs available on a typical day at the cancer centre. CRS- RUS corresponds to the model that includes the chart review simplification and the ramp up modification. It is the same as CRS-PS except for the specified limits on the number of chairs available in each time slot. MIP Model Constraint programming researchers often point to scheduling as a domain where their approaches are superior to mixed integer programming (MIP) (Bartak et al. 10, Beck and Fox 98). However, MIP is still a more generally dominant and widespread technology and is more commonly applied to scheduling problems in operations research (Demir and Kursat Isleyen 2012, Fattahi et al. 2007). Therefore, it makes sense to compare the solvability of the constraint model to that of a MIP model.

140 Journal of Applied Operational Research Vol. 6, No. 3 The MIP model presented here has the chart review stage omitted and so is similar, but not equivalent (see below), to the constraint model CRS. Due to the non-unary capacity of the nurses and chairs, we implement a time-indexed model. The binary decision variable x irt, is set to one if the start time of stage i of request r is time t. The model can be written as follows: min S End subject to: = 1, (13) (14) (15) (16) (17) (18) (19), (20) (21) (22) The objective function is to minimize the makespan. Constraint (13) ensures that each activity starts exactly once. Constraint (14) ensures that the makespan is at least the latest end time of any activity. Constraints (15), (16), and (17) ensure that the capacity of the pharmacy technicians, nurses, and chairs are respected in stages two, three, and four respectively. Constraint (18) is the precedence constraint between the first and second stages. Constraint (19) says that the start time of the second stage of treatment must equal the start time of the first stage of treatment. Constraint (20) says that no appointment can start before its ready time. Constraint (21) says that there is a maximum time, l, allowed between the start times of the drug preparation and treatment stages. Constraint (22) ensures x irt is a binary variable. Experimental Design To choose a problem definition (steps four and five of the proposed method) and understand the impact of the different formulations, three sets of experiments were conducted: (1) solvability experiments, (2) quality experiments, and (3) an experiment comparing constraint programming to MIP. All experiments are run using five problem instances, each corresponding to an actual day at the Odette Cancer Centre. The five sets used are March 1 st through 5 th, 2010 (see Table 1). The list of patients on each day as well as the treatment durations and regimens were extracted from the scheduling software in use at the centre. Treatment duration ranged from 15 minutes to 510 minutes (8.5 hours), with majority of the treatments lasting under 240 minutes (4 hours). The direct nursing requirement and drug preparation time was determined by matching each patient s regimen to a database provided by Cancer Care Ontario. Direct nursing times were between 15 and 60 minutes, with majority being 15 minutes. Drug preparation times were between 15 and 60 minutes, with majority being 30 minutes in length. The duration of the chart review stage was assumed to be 15 minutes for each patient. All the experiments were conducted on a Pentium Dual-Core CPU computer E5200 @ 2.50 GHz and 3.25 GB of RAM running 32 bit Windows XP. The constraint programming models were solved using comet (Dynadec Decision Technologies, RI) and the MIP was solved using CPLEX Optimization Studio 12.5.1 (IBM, NY). Table 1. Experimental Data Test Set Number of Appointments Total Treatment Minutes Nurses Available Day 1 63 6210 13 Day 2 83 9435 13 Day 3 81 11760 15 Day 4 99 10725 15 Day 5 90 11190 13

S Hahn-Goldberg et al 141 To evaluate solvability, each data set is solved using each constraint model and the time to solve and percentage of instances solved to optimality are measured. The ultimate goal is that the chosen model will be incorporated into the current scheduling software and will be used in real time, so the time to solve is an important measure. However, to compare the models for solvability, we chose a time in which all the constraint models could find a feasible solution. Each model was given a maximum run-time of 3600 seconds. At that point, the best solution found is taken as the resultant solution. To evaluate quality, the schedules produced by the constraint models were run through a simulation model of patient flow through the chemotherapy clinic. The average and maximum wait times were measured. Wait time is defined as time in the system, not being serviced at any of the four stages. Based on the results of the experiments on solvability and quality, one of the constraint models was chosen and compared to the MIP model. First, the five problem instances were solved by both the chosen constraint model and the MIP model. Then, six new test problems were solved by both models. Using the same number of resources, the number of patient appointments was reduced from 63 to 50, 40, 30, 20, 10, and 5. These problem sets are referred to as mip_50, mip_40, mip_30, mip_20, mip_10, and mip_5. Each model was given a maximum run-time of 3600 seconds. Results In this section we present the results from the three sets of experiments. Solvability and Quality Experiments Since the intention is for the model to be used in real time, it is important to note that for the majority of the runs, a feasible solution is found very quickly and if an optimal solution is found, it is found early on the process. A feasible solution was found 100% of the time, and a feasible solution was found within one second 96.67 % of the time. Table 2. Average Solvability and Quality Results across All Constraint Programming Models Measure Value Average time to solve (s) 864.76 Percent solved to optimality (%) 76.67 Average wait (min) 78.19 Maximum wait (min) 138.22 Table 2 summarizes the results for the Odette Cancer Centre. The average results were used to create a rating scale to qualitatively compare the models as presented in Table 3. The description refers to the percentage relative to the results of all models. For example, a time to solve value that is worse than 75% of all the time to solve values warrants a rating of poor. In our experiments, 75% of all the time to solve values were better (lower) than 446.41 seconds. Any model that takes longer than 446.41 seconds to solve is given a rating of poor for that metric. Additional ratings of Fair and Good are used where the solvability and quality measures fall into different rating groups. Fair refers to a mix of Poor and Average and Good refers to a mix of Average and Very Good. A rating of Average is given if there is a mix of Poor and Very Good. Table 3. Rating Scale Values Rating Description Time to Solve (s) Optimality (%) Average Wait (min) Maximum Wait (min) Poor Worse than 75% > 446.41 < 50 > 200 > 309 Average 25-75%.063-446.41 50 70 11 200 23 309 Very Good Better than 25% <.063 > 70 < 11 < 23

142 Journal of Applied Operational Research Vol. 6, No. 3 Table 4 shows the results and ratings of the four constraint models. The values are the results for the particular model averaged over all five problem instances. The results show the trade-off between models that have high solvability, which can be solved in a reasonable amount of time; and models that have high quality, yielding solutions that work well in a simulation model of the real world. Based on the values and ratings, we selected CRS as the best model. Table 4. Model Comparison Solvability Quality Model CPU time (s) Optimality (%) Rating Average Wait Max Wait (min) (min) Rating Base 2162.28 40 Poor 11.3 31 Average CRS 720.13 80 Average 9.0 22 Very Good CRS-PS 0.03 100 Very good 201.2 310 Poor CRS-RUS 89.44 100 Good 173.8 304 Average Comparison with MIP The MIP model was used to solve the five test sets presented in Table 1 with a maximum run-time of 3600 seconds. It was not able to find any feasible solutions in the allotted time. Therefore, we present the results to the six additional problem instances, mip_5 through mip_50. Table 5 presents the comparison between the MIP model and the constraint model CRS. If no solution was found within 3600 seconds, the CPU time is denoted by the notation -. Table 5. Comparison of MIP and CRS models Test Set Time to first feasible solution (s) MIP Time to optimal solution (s) Time to first feasible solution (s) CRS Time to optimal solution (s) mip_5 0.36 1.45 0.04 0.06 mip_10 21.03 39.39 0.12 9.00 mip_20 123.86 238.60 11.43 20.87 mip_30 1729.42 3305.04 33.47 49.20 mip_40 3138.80-45.92 59.64 mip_50 - - 46.00 78.20 Discussion By experimenting with different problem definitions, we were able to see the trade-off between solvability and quality. The most complete definition, Base, was hard to solve. CRS-PS, the definition easiest to solve, resulted in solutions with poor quality. If Base were to be implemented at the cancer centre, the scheduler would not be able to provide patients with appointment times in a timely manner. If CRS-PS were to be implemented into the scheduling system at the cancer centre, the patients would experience high wait times. We were able to choose a definition that rated well in both solvability and quality, CRS. CRS performed considerably better than the MIP model of the same problem. A likely explanation for these results is the known weakness of the time-indexed model that requires a binary variable for each activity and time-point on each resource. Such models scale poorly however, to our knowledge, there are no other MIP models that can represent non-unary capacity resources such as the nurses and chairs. We anticipate that by integrating optimization into the scheduling system, the Odette Cancer Centre will be able to service more patients with the same resources. At present, the cancer centre is not using optimization when booking appointments and, as a result, resources are not being used as efficiently as they could be. The potential

S Hahn-Goldberg et al 143 for improvement can be seen by comparing the schedules obtained using CRS to the current schedules at the cancer centre. Figure 1 provides a diagram of test day one scheduled using CRS next to the schedule that was executed at the cancer centre. Each row represents a separate chair used for treatment and each change in colour or shade represents a different patient being treated. Time runs along the x axis, with each square of the grid being a 15-minute timeslot. With optimization, the improvement in makespan and chair utilization is obvious and significant. In the real schedule, many chairs are not being used effectively. Chair utilization, defined as the percentage of time that the chair is in use before the last patient on that chair finishes, is only 64% in the real schedule, but is 85% in the CRS schedule. Fig. 1. Test day 1 scheduled using CRS (up) and the actual schedule executed at the Odette Cancer Centre (down).

144 Journal of Applied Operational Research Vol. 6, No. 3 Conclusion We developed and tested a number of constraint programming models for the deterministic chemotherapy outpatient scheduling problem at the Odette Cancer Centre in Toronto Canada. The best model (CRS) is able to be solved in a reasonable amount of time while achieving low patient wait times, as evaluated in a simulation. A preliminary comparison of the produced schedules with the current schedules at the cancer centre demonstrates a substantial improvement over current practice. We are continuing work with the Odette Cancer Centre to integrate CRS into their current scheduling software. A full solution will necessarily accommodate the dynamism of needing to schedule appointments before all appointments for a given day are known and the uncertainty arising from appointment cancellations. Dynamism and uncertainty is addressed in our subsequent work (Hahn-Goldberg 2014). This paper demonstrates the value for in-depth study of the problem definition stage when creating real applications. Our four constraint models represent different definitions of the deterministic chemotherapy outpatient scheduling problem and result in very different algorithm behaviour and solution quality. We provided a methodology for choosing and evaluating problem definitions of complex real-world problems. Additionally, this paper highlights the potential for using constraint programming to address the chemotherapy scheduling problem by comparing the best constraint model developed to a MIP model. The MIP model was unable to find feasible solutions to problem instances where the constraint models were able to find optimal solutions to four of five instances and a good feasible solution to the fifth. On scaled down problems the CRS constraint model out-performed the MIP model by almost two orders of magnitude. References Baptiste P, LePape C, and Nuijten W (2001). Constraint- Based Scheduling. Kluwers editions. Bartak R, Salido M, and Rossi F (2010). Constraint Satisfaction Techniques in Planning and Scheduling. Journal of Intelligent Manufacturing 21: 5-15. Beck JC and Fox MS (1998). A Generic Framework for Constraint-Directed Search and Scheduling. AI Magazine 19(4): 103-132. Beldiceanu N and Demassey S (2010). Global Constraint Catalogue. http://www.emn.fr/z-info/sdemasse/gccat/. Accessed 12 April 2014. Cayirli T and Veral E (2003). Outpatient Scheduling in Health Care: A Review of Literature. Production and Operations Management 12(4): 519. Cayirli T, Veral E, and Rosen H (2006). Designing Appointment Scheduling Systems for Ambulatory Care Services. Health Care Management Science 9: 47-58. Demir Y and Kursat Isleyen S (2012). Evaluation of Mathematical Models for Flexible Job-Shop Scheduling Problems. Applied Mathematical Modelling. Erdogen S and Denton B (2011). Dynamic Appointment Scheduling of a Stochastic Server with Uncertain Demand. Informs Journal on Computing, Articles in Advance. 1-17. Fattahi P, Mehrabad M, and Jolai F (2007). Mathematical Modeling and Heuristic Approaches to Flexible Job Shop Scheduling Problems. Journal of Intelligent Manufacturing 18(3): 331-342. Frisch AM, Jefferson C, Hernandez BM, and Miguel I (2005) The Rules of Constraint Programming. In the Proceedings of the 19th International Joint Conference on Artificial Intelligence: 109-116. Hahn-Goldberg S (2014). Dynamic Optimization Addressing Chemotherapy Outpatient Scheduling. PhD Thesis. University of Toronto, Toronto, Canada. Pinedo ML (2012). Scheduling: Theory, Algorithms, and Systems, 4th Edition. Springer. Rossi F, van Beek P, and Walsh T (2006). Constraint Programming. Chapter 1. Elsevier. Santibanez P, Aristizabal R, Puterman M, Chow VS, Huang W, Kollmannsberger C, Nordin T, Runzer N, and Tyldesley S (2012). Operations Research Methods Improve Chemotherapy Patient Appointment Scheduling. The Joint Commission Journal on Quality and Patient Safety 38 (12): 541-553. Turkcan A, Zeng B, and Lawley M (2012). Chemotherapy Operations Planning and Scheduling. IIE Transactions on Health care Systems Engineering 2(1): 31-49.