Appointment scheduling in healthcare. Master Thesis

Appointment scheduling in healthcare Alex Kuiper December 9, 0 Master Thesis Supervisors: profdr Michel Mandjes dr Benjamin Kemper IBIS UvA Faculty of Economics and Business Faculty of Science University of Amsterdam

Abstract Purpose: Appointment scheduling has been studied since the mid-0th century Literature on appointment scheduling often models an appointment scheduling system as a queueing system with deterministic arrivals and random service times In order to derive an optimal schedule, it is common to minimize the system s loss in terms of patients waiting times and idle time of the server by using a loss function This approach is translated to a D/G/ queue, which is complex for a broad range of settings Therefore many studies assume service time distributions that lead to tractable solutions, which oversimplifies the problem Also, many studies overcome the complexity through simulation studies, which are often case specific The purpose of this thesis is to offer an approach to deal with arbitrary service times and to give guidance for practitioners in finding optimal schedules Approach: First, we approximate service time distributions by a phase-type fit Second, we compute the waiting and idle times per patient Finally, we run algorithms that minimize, simultaneously or sequentially, the system s loss This approach enables us to find optimal schedules for different loss functions in both the transient and steady-state case Findings: Our approach is an explicit and effective procedure to find optimal schedules for arbitrary service times Optimal schedules are derived for different scenarios; ie for sequential and simultaneous optimization, linear and quadratic loss functions and a broad range of service time distributions Practical implications: The procedure can be used to compute optimal schedules for many practical scheduling issues that can be modeled as a D/G/ queue Value: We present a guideline for optimal schedules that is of value to practitioners in services and healthcare For researchers on appointment scheduling we present a novel approach to the classic problem of scheduling clients on a server Information Title: Appointment scheduling in healthcare Author: Alex Kuiper, akuiper@scienceuvanl, 564769 Supervisors: profdr Michel Mandjes, dr Benjamin Kemper Second readers: drir Koen de Turck, dr Maurice Koster Date: December 9, 0 IBIS UvA Plantage Muidergracht 08 TV Amsterdam http://wwwibisuvanl

Contents Introduction 4 Background of appointment scheduling 6 Literature review 6 Dynamic versus static scheduling 6 The D/G/ model 7 3 The arrival process 7 4 The service time distribution and queue discipline 8 5 Some remarks on scheduling 8 Model description 9 3 Approximation by a phase-type distribution 5 3 Phase-type distribution 5 3 Phase-type approximation 9 33 Phase-type fit 3 34 Recursive procedure for computing sojourn times 4 34 Exponentially distributed service times 4 34 Phase-type distributed service times 6 4 Optimization methods 30 4 Simultaneous optimization 30 4 Sequential optimization 3 4 Quadratic loss 3 4 Absolute loss 33 43 Lag-order method 33 44 Computational results for transient cases 34 5 Limiting distributions 39 5 The D/M/ queue 40 5 Limit solutions in the sequential case 40 5 Limit solutions in the simultaneous case 40 5 The D/E K,K / queue 4 5 The limiting probabilities 43 5 The sojourn time distribution 44 53 The D/H / queue 45 53 The limit probabilities 46 53 The sojourn time distribution 49

CONTENTS 3 54 Computational results in steady-state 5 6 Optimal schedules in healthcare 55 6 Performance under Weibull distributed service times 56 6 Performance under log-normal distributed service times 58 7 Summary and suggestions for future work 59

Chapter Introduction In this thesis we study the classic appointment scheduling problem that practitioners often encounter in processes in services or healthcare In such a setting an appointment refers to an epoch that sets the moment of the client s or patient s arrival in time Next, the client receives a service from the service provider For example, a doctor sees several patients during a clinic session in a hospital A patient arrives exactly on the priorly appointed time epoch Upon arrival, the patient either waits for the previous patient to be served, or is directly seen by the doctor In the latter, the doctor was idle for a certain time period Ideally the schedule is such that the patient has no waiting time and the doctor has no idle time Unfortunately, this case is never realized due to the fact that eg the treatment time for every patient is not constant, but a random variable Therefore, we must find the best appointment schedule such that the expected waiting and idle times are minimized for all patients In this thesis we will study different approaches to achieve this Above we gave an example of an appointment scheduling problem in a healthcare setting, but there are many more, such as scheduling of MRI and CT patients MRI and CT scanners are expensive devices and therefore it is crucial to maximize their utilities, ie minimize idle times So it seems optimal to have a tight schedule for these scanners But in case of complications during scans, too tight a schedule will result in high waiting times for patients Another typical example is scheduling the usage of operating rooms in a hospital or clinic There are only a small number of these special rooms, where various surgeries have to be scheduled on Therefore, the utility of each room should be maximized, ie minimizing the idle time at the expense of the patients (waiting) time But it is known that poor scheduling performance, which results in high waiting times, lead to patients choosing other hospitals or clinics for surgeries In addition, there are numerous examples outside the healthcare setting For instance, ship-to-shore container cranes, where the ships can be seen as clients, the service time is the total time of unloading, and the cranes must be seen as the service provider Too tight a schedule results in waiting ships, which incurs extra costs for the ships On the other hand, unused cranes do not make profit Furthermore, there is a risk that ships choose for competing harbors with lower waiting times These examples show that the cost is twofold: we have both waiting times for clients and idle time for the server Finding an optimal schedule which minimizes both costs is our aim Now we have a feeling in how we can translate some practical problems to an appointment scheduling problem Assume for now that there is a finite number of patients to be sched- 4

5 uled, say N, where the service times B i for i {,, N} are random variables which are independent, and in most cases also identically distributed The waiting and idle times per patient is denoted by W i respectively I i The waiting and idle times are random variables as well, since they depend on the service times of previous scheduled patients Our goal is to minimize the sum of all waiting and idle times over all possible schedules A naive approach would be to schedule all patients equidistantly based on a patient s average service time This schedule, denoted by T, can be written as t = 0 and t i = i j= E[B j] However, a session could take longer than the expected service time When this happens all subsequent patients will have positive waiting times in expectation We will see in Chapter that this schedule is far from optimal, since it leads to infinite waiting times when N tends to infinity An important factor which affects the optimal appointment schedule is the trade-off between W i and I i in the cost function R i If the time of the patient is relatively more expensive, more weight is put on W i and the schedule will become less tight Visa versa, if the doctor s time is considered to be relatively more expensive, more weight is put on I i and the schedule becomes more tight The outline of this thesis is as follows In the upcoming chapter we will discuss the background in the form of a literature review and some preliminaries Because we are interested in a healthcare setting, we consult related literature This gives us a proper framework to derive a mathematical model, with relevant cost functions In the next chapter, Chapter 3, we will introduce phase-type distributions, which can be used to approximate any positive distribution arbitrary accurately We will give a solid theoretical basis to sustain this claim Furthermore, they exhibit a specific property which allows us to use a recursive procedure This procedure will be used to compute optimal schedules for various cost functions for finite amounts of patients, ie the transient case in Chapter 4, first We will use two different optimization approaches: simultaneous and sequential Simultaneous optimization is finding an optimal schedule for all patients jointly, while sequential optimization is a recursive approach in which you optimize patient by patient The latter has the advantage that it reduces the scheduling problem to finding the optimal schedule for one patient each time At the end of this chapter we will compare our findings with relevant literature Secondly, in Chapter 5, we will derive a method to compute the optimal schedule in its steady-state for different coefficients of variation, which is a measure of the variability of the service times Setting the expectation equal to one will allow us to compare the performance of optimal schedules under changes in the coefficient of variation We will compute the optimal schedule in steady-state for both the sequential and simultaneous approach and for different cost functions, which will be used in Chapter 4 In Chapter 6 we check the performance of our approach in simulated examples based on real-life settings We model these examples by data generated from either the Weibull or log-normal distribution The choice of these particular distributions originates from the healthcare setting as explained in Chapter In Chapter 7 we summarize our results and suggest topics for further research Finally I am thankful to my supervisors Benjamin Kemper and Michel Mandjes for their scientific support and guidance during my six-month internship at the Institute for Business and Industrial Statistics of the University of Amsterdam (IBIS UvA) Also, I would like to thank the second readers Maurice Koster and Koen de Turck who made the effort to read and comment on my work

Chapter Background of appointment scheduling In this chapter we perform a background study on appointment scheduling In the upcoming section we review literature on appointment scheduling with a focus on healthcare Further, we derive assumptions for a model So that at the end of this chapter we have a well defined optimization problem in a properly justified model Literature review The research on appointment scheduling dates back to the work of Bailey (95,954) [6, 7], Welch and Bailey (95) [5] and Welch (964) [8] The authors study the phenomenon of scheduled patients who are to be treated in a typical healthcare delivery process This phenomenon of priorly scheduled arrivals that are treated by a service provider is often studied as a queueing system in which jobs arrive following a deterministic arrival process and receive a service which varies in duration After this pioneering work on appointment scheduling the subject has been extensively researched in both services and healthcare The article by Cayirly and Veral (003) [0] gives a good overview on the state of the art in appointment scheduling We use this article to highlight special features for appointment scheduling in healthcare Dynamic versus static scheduling To begin with the objective of outpatient scheduling is to find an appointment system for which we optimize over the system s loss, which is the sum of the expected losses incurred by waiting and idle times Literature on appointment scheduling can be divided into two categories with respect to outpatient scheduling: static and dynamic In the static environment the appointment system is completely determined in advance, in other words off line scheduling This is in contrast to the dynamic case in which changes in the schedule are permitted, so called online scheduling Most literature focuses on the static case; only a few papers such as Fries and Marathe (98) [4] and Liao et al (998b) [8] consider the dynamic case in which the schedule of future arrivals are revised continuously 6

LITERATURE REVIEW 7 The D/G/ model The outpatient scheduling problem can be modeled by a queueing model The so called D/G/ queueing model, by Kendall s three-factor (A/B/C) classification (953) [7] The three parameters are: A, the arrival process, in our case the schedule, is deterministic, denoted by D B, the service time distribution is general, denoted by G This means explicitly that any positive distribution can be taken to model the service times C, the number of servers is set to, since we have a single doctor or practitioner An optimal schedule in this context is the deterministic arrival process of the patients, which minimizes the sum of R i for all i A few studies in healthcare investigated so called multistage models in which patients have to go through several facilities, such as in Rising et al (973) [0], who performed an extensive case study and Swisher et al (00) [], who did a broad simulation study We will consider practitioners and doctors as independent queues, because there is a doctor-patient relationship, often seen in literature: Rising et al (973) and Cox et al (985) [], but also justified by contemporary medical ethics It is a mathematical fact that when there are multiple doctors it is better to have a single flow of patients, who are scheduled to the first available doctor 3 The arrival process The arrival process itself can come in many forms First important factor is the unpunctuality of patients, which is the difference between the time the patient arrives and his appointment time Generally patients arrive more early than late, which is showed in many studies that can be found in Cayirly and Veral (003) On the other hand the doctors unpunctuality can be measured as lateness to the first appointment This kind of lateness is only considered in some studies A second important factor, also addressed in Cayirly and Veral s paper is the presence of no-shows, that is that a patient does not show up at all Empirical studies show that the probability of no-shows ranges from 5% to even 30% Simulation studies show that the no show probability has a greater impact on the appointment schedule than the coefficient of variation and the number of patients per clinic session, see the extensive simulation study by Ho and Lau (99) [9] On the other hand patients can also drop by, not having a priorly scheduled appointment These are called walk-in patients, the presence of these kind of patients is not often incorporated in studies This is of course in line with the static approach of appointment scheduling, since scheduling these walk-in patients on-the-fly will result in a modification to the schedule In case there are walk-ins this should not automatically harm the staticity of the appointment schedule since one can take walk-ins into account in the schedule by seeing walk-in patients only at instances when the doctor is idle Another factor on the arrival process is the presence of companions The D/G/ queueing model has infinite waiting capacity and therefore the presence of companions is not taken into account Moreover, there is no restriction on the number of waiting patients However as remarked in Cayirly and Veral, for hospitals it is important to know how many people are likely to use the waiting room, since hospitals have to facilitate all the waiting patients and their companions

8 CHAPTER BACKGROUND OF APPOINTMENT SCHEDULING 4 The service time distribution and queue discipline So far we discussed the arrival process and the number of service providers (doctors) Remaining is the service time and queue discipline The queue discipline is in all studies on a first-come, first-served (FCFS) basis In case of punctual patients this discipline is the same as serving patients in order of arrival But, if patients are unpunctual doctors can choose to see the next patient when the next patients is already dropped in This reduces his idle time However, assuming that lateness is always less than the scheduled interarrival times, we can assume that the order of arrival will be equal to the scheduled order Now, we discuss the most important factor for this thesis, that is the service time (per patient) The total service time is defined to be the sum of all the time a patient is claiming the doctor s attention, preventing him or her seeing other patients, ie the service times per patient, see Bailey (95) An important quantity in queuing theory is the (squared) coefficient of variation, denoted by (S)CV, of a random variable B i for patient i CV = σ σ, and SCV = µ µ where µ = E[B i ] and σ = Var[B i ] Many analytical studies assume B i to be exponential, obviously because their analytical approach will be intractable otherwise, see Wang [7], Fries and Marathe [4], and Liao et al [8] However, the one-parameter exponential distribution sets the CV =, which is too restrictive and not seen in practice More common in healthcare is data with 035 < CV < 085 Furthermore empirical data collected from clinics and frequency distributions of observed service times display forms that are unimodal and rightskewed, Welch and Bailey (95), Rising et al (973), Cox et al (985), Babes and Sarma (99) [4] and Brahimi and Worthington (99) [8] Examples of such distributions are the log-normal or (highly flexible) Weibull distribution In 95 Bailey already observed that the performance of the system is very sensitive to small changes in appointment intervals Since then many studies have reported that an increase of the variability in service times, ie the SCV s, lead to an increase of both the patients waiting times and the doctor s idle times, and thus incur extra costs Examples of this phenomenon can be found in Rising et al (973), Cox et al (985), Ho and Lau (99), Wang (997), and in Denton and Gupta (003) [] The choice of an optimal appointment schedule depends mainly on the mean and variance, see Ho and Lau (99) and Denton and Gupta (003) Hence it is important to model the observed data well, that is by matching the first two moments, a possible way to do so is by phase-type fit, see Adan and Resing (00) [] Wang showed already in (997) [6] how one can numerically compute an optimal appointment schedule for general phase-type distributions 5 Some remarks on scheduling We finish with some remarks The appointment scheduling problem can also be considered to be discrete so that one can only set schedules at certain times This is the approach followed by Kaandorp and Koole (007) [5] The main advantage of this approach is that there is just a finite number of combinations possible, so that they can use a faster minimization algorithm We consider the continuous time case, which can be seen as the limit case of the discrete problem of Kaandorp and Koole, which is with their methodology impossible to solve due to dimension issues

MODEL DESCRIPTION 9 Based on experience we assume that one patient who has to wait 0 minutes is worse than that 0 clients have to wait only minutes We can incorporate this effect with cost functions, which penalizes the loss incurred by one long waiting times more than the sum of the short waiting times, a common choice for this purpose is a quadratic cost function Another choice is a linear cost function, which is more appropriate in a production environment, when we do not have to incorporate people s experiences We can reason along the same lines for the doctor (or server) We then implement a cost function, either linear or quadratic, on the doctor s idle times Since we deal with randomness, driven by the variable service times, we look at these losses in expectations, which we call risk A more general discussion of choices for cost functions can be found in Cayirly and Veral (003) Herein they define the expected total loss (risk) as a weighted sum of expected waiting times, idle times and overtime We only focus on the linear and quadratic cost functions used in Wang (997) and Kemper et al (0) In the next section we present our model, settings and cost functions in more detail Model description In this section we present our model The optimization problem as stated in the introduction can be written as a minimization of the expected patients waiting times and server s idle times over the arrival epochs: N+ min t,,t N+ (E[I i ] + E[W i ]) () in which W i refers to the patient i s waiting time and I i refers to the server s idle time before it serves the i-th patient This problem is defined in the following setting: There are N + patients to be priorly scheduled in one clinic session So there are no walk-in patients Let B,, B N+ the service times of N + patients The B i s are independent and identically distributed The t i are the scheduled times i {,, N +}, and let x i = t i+ t i, for i {,, N} the interarrival times Scheduled patients are punctual, always show up and are served in order of arrival One wants to minimize a convex cost function R i depending on the expected waiting time of patient i and the expected idle time of the doctor We schedule N + patients, so that we end up with exactly N interarrival times The relation of the interarrival times x i s and the arrival epochs t i s is showed in Figure We consider now the naive schedule of the introduction We propose it as an example, which proves that even a simple, but naive, heuristic can have major impact on (expected) waiting times It shows the relevancy of optimal appointment scheduling

0 CHAPTER BACKGROUND OF APPOINTMENT SCHEDULING Figure : the relation between N + arrival epochs t i and N interarrival times x i Example (A naive appointment schedule) Consider again the appointment schedule T by setting the interarrival times x i equal to the expected service time of patient i i t = 0 and t i = E[B j ] for i =,, N, N + This is a very simple approach, but in fact the service load is equal to It means that per unit time the in- and outflow of patients is equal to, which will lead to infinite waiting times by the following proposition, see Kemper et al (0) Proposition In a D/G/ queue with load starting empty, with the service times B i having variance σ <, the mean waiting time of the N-th patient is given by N E [W N ] σ N π The result also holds in the more general setting of a G/G/ queue, where σ = Var[A i ] + Var[B i ] where A i is the arrival distribution of patient i Proof Let A i be the i-th patient s interarrival time and B i his service time By the Spitzer- Baxter identities [, 3] we have to study with W N waiting time of patient n and [ I k = P k j= E [W N ] = σ N I k () N N k 0 k k= B i A i σ > y By Chebyshev s inequality the integrand is bounded [ ] [ k B P i A i > y min {, y k σ Var k so that I k 0 ( y ) dy = 0 ] dy k B i A i σ dy + dy = y ]},

MODEL DESCRIPTION This gives us a majorant so that we can apply both the Dominated Convergence and the Central Limit Theorem [ ] lim P k B i A i > y = ( Φ(y)) dy = k k σ π Furthermore N [x N I k = ] I Nx /N dx N k 0 Nx /N k= This integrand is again bounded, knowing that I Nx /n <, by x / so 0 x / dx = 4 Using Dominated Convergence Theorem gives us pointwise convergence of lim N N N k= I k = k 0 0 x dx = π π We finish the proof by multiplying the latter expression by σ which gives us the right-hand side of equation () What happens in the last example is that the occupation rate, defined as ρ = E[B i] E[A i ], is equal to The occupation rate ρ is also known as the service load To ensure stability, ie with probability zero we have infinite patients waiting if t, we need that E[B i ] < E[A i ], ie the service load should be less than Let R i = Eg(I i ) + Eh(W i ) the risk per patient, where g, h : R 0 R 0 are cost functions Also, we demand that g, h are convex and satisfy g(0) = h(0) = 0, so that the problem is min R = t,,t N+ R i = N+ min t,,t N+ min E t,,t N+ [ N+ (g(i i ) + h(w i )) ], (3) cf equation () The risk R can also be thought of the system s loss, since it captures all the loss incurred by idle and waiting times Proposition 3 It is always optimal to schedule the first patient at time zero Proof Suppose one has an optimal schedule V where t = a > 0 and the alternative (shifted) schedule W where t = 0 and all t i = t i a then expected idle time of the first patient is larger than 0, because by punctuality E[g(I )] = g(a) + E[g(I )] = g(a) and E[h(W )] = E[h(W )] = 0 Furthermore, for all i {,, N + } E[g(I i )] = E[g(I i )] and E[h(W i )] = E[h(W i )] as patients arrive on their arrival epochs t i or t i = t i a Observe that schedule R V = R W +a, which shows that schedule W is the optimal schedule

CHAPTER BACKGROUND OF APPOINTMENT SCHEDULING By Lindley s recursion formulas, see [9], which are graphically demonstrated in figure Figure, we have I i = max{(t i t i ) W i B i, 0} = max{x i S i, 0}, (4) W i = max{w i + B i (t i t i ), 0} = max{s i x i, 0}, (5) Define now the loss function l(x) Figure : the relations between interarrival, idle, sojourn and waiting times We see that the idle time I i can be written as the interarrival time x i minus the sojourn time S i The waiting time W i can be written as the sojourn time S i minus the interarrival time x i The figure originates from Vink (0) [4] l(x) = g( x) x<0 + h(x) x>0 The function l(x) is also convex which can be deduced easily by the fact that g(x) and h(x) are convex functions satisfying g(0) = h(0) = 0 Proposition 3 and the fact that either idle time is zero and waiting time is positive or visa versa, one can reduce the optimization problem to minimizing over N arrival epochs or interarrival times: R = N+ R i = E [ N+ i= ] [ N ] (g(i i ) + h(w i )) = E l (S i x i )) There are many loss functions possible, but often one considers absolute loss l(x) = x, which corresponds with linear costs or quadratic loss l(x) = x The latter loss function corresponds with quadratic costs of waiting and idle times We can generalize these loss functions to weighted versions, ie let α (0, ): R(α) = = = E N+ i= N+ i= E [ ] α x i S i Si x i <0 + ( α) S i x i Si x i >0 E [ ] α S i x i + ( α) S i x i Si x i >0 [ N+ αi i + ( α)w i ], (6) (7)

MODEL DESCRIPTION 3 and R(α) = = = E N+ i= N+ i= [ ] E α (x i S i ) Si x i <0 + ( α) (S i x i ) Si x i >0 [ ] E α (S i x i ) + ( α) (S i x i ) Si x i >0 [ N+ αi i + ( α)w i ] (8) (9) Furthermore, there are other variables which can incorporate costs, for example the approach by Wang [6] [7] where he minimizes the expected sojourn times per patient N+ E[S i ] = N+ E[W i + B i ] where E[W ] = 0 However, E[B i ] s do not depend on the schedule choice, so that it is equivalent with minimizing waiting time only But minimizing waiting time only will take interarrival times x i so we add the expected system s completion time The expected system s completion time is defined as the sum of arrival time of the last patient t N+ and his expected sojourn time E[S N+ ] We can generalize this also to weighted version by giving weights to the sojourn and completion time Since the system completion time can also be seen as the sum of all idle times plus service times It is equivalent with minimizing idle time only So Wang s choice of cost functions is equivalent with linear costs An equivalent definition is the sum of all the interarrival times plus the expected sojourn time of the last patient We summarize, let α (0, ) N+ R (α) = α E[S i ] + ( α) (t N+ + E[S N+ ]) N+ N+ = α (E[W i ] + E[B i ]) + ( α) (E[I i ] + E[B i ]) N+ = α N+ E[W i ] + ( α) N+ E[I i ] + ( N+ = α E[S i ] + ( α) E[S N+ ] + E[B i ] ) N x i (0)

4 CHAPTER BACKGROUND OF APPOINTMENT SCHEDULING So we distinguished four different costs in terms of time: Completion time: the time when the doctor (server) finishes seeing the last patient Idle time: the time that a server is idle before the next patient comes in Waiting time: the time that a patient has to wait before he or she is seen (served) Sojourn time: the sum of waiting and service time Moreover, we observed that N+ R (α) = R(α) + E[B i ], () which shows the relation between the system s loss of Kemper and Wang Minimizing absolute loss, equation (6), is equivalent with minimizing Wang s R This allows us to compare their methods at the end of Chapter 4 In this chapter we proposed and motivated our model In addition, we showed some preliminary results such as finding an optimal appointment schedule is equivalent with finding optimal interarrival times Moreover, we reduced the problem of minimizing functions of idle and waiting times to minimizing a function of the sojourn time only In the next chapter we focus on a rich class of distributions which lie dense in the class of all distributions and have appropriate properties These are called phase-type distributions We will use these distributions in our minimization problem to compute optimal schedules for transient and the steady-state cases

Chapter 3 Approximation by a phase-type distribution In this chapter we will start with an overview of phase-type distributions The reason why we will focus on phase-type distributions is that they are very flexible in capturing data structures, for example via moment matching [] or via the EM algorithm [3] In the next section we will give a theoretical framework, see [3] and [3], which proves that phase-type distributions can approximate any positive distribution arbitrarily accurately After showing this result we will give explicit formulas of how we can fit phase-type distributions based on its first two moments In the final section we show a recursive procedure of computing the sojourn time distribution of patients in the D/P H/ setting when their service time distribution is phase-type 3 Phase-type distribution The assumption of exponential service times is often used, because of its simplicity memoryless property is what makes it so attractive to make this assumption, that is The P [X > t + s] = P [X > t] P [X > s] The reason why we focus on phase-type distributions is that they can be seen as a generalization of exponential service times They do not satisfy the condition of above, except of course the special case of exponential service times They have another attractive property that is, loosely speaking, that when the process is stopped at an arrival time t i the probability is distributed over the possible number of phases in the system This distribution of probability over the phases can be taken as a new probability vector, which can be used as a start vector in the same phase-type form, but time is then starting at t = 0 instead of t i This property can be exploited to (numerically) compute optimal arrival times First, we start with the precise definition of a phase-type distribution and give some key examples Consider a Markov process J t on a finite state space {0,,, p}, where 0 is absorbing and the other states are transient The infinitesimal generator Q is given as ( ) 0 0 Q = S 0, S 5

6 CHAPTER 3 APPROXIMATION BY A PHASE-TYPE DISTRIBUTION where S is an m m-matrix and S 0 = S, which is the so called exit vector The vector is a column vector of ones of length m, so that each row in Q sums to zero Definition 3 A random variable X distributed as the absorption time inf {t > 0 : J t = 0} with initial distribution (α 0, α) (row vector of length m + ) is said to be phase-type distributed In phase-type representation we say X P H(α, S), since α and S define the characteristics of the phase-type distribution completely The transition matrices P t = exp (Qt) = Q n t n of the Markov process J t can also be written down in block partitions ( ) 0 P t = e St e St, n=0 which gives an expression for the distribution function F (t) = αe St Also, other basic characteristics of phase-type distributions can be derived: The density: f(t) = αe St S 0 = αe St S The Laplace-Stieltjes transform: 0 e st F (ds) = α(si S) S 0 3 The n-th moment: E [X n ] = ( ) n n!αs n Phase-type distributions do not have unique representations, see Example 3 Example 3 (Erlang distribution) This distribution is denoted by E K (µ) and is a special case of the gamma distribution in which the shape parameter is a natural number Its probability density function is given by n! f(t) = µ (µt)k (K )! e µt The interpretation is that a random variable has to go through K exponential phases with same scale parameter So that we can also write this in phase-type representation (α, S), namely α = (, 0,, 0) of dimension K and a matrix S of dimension K K µ µ 0 0 0 µ µ S = 0 0 0 µ µ 0 0 0 µ Its moments are given by E[X n ] = (K + n )! (K )! µ n

3 PHASE-TYPE DISTRIBUTION 7 so that its SCV = K A further generalization is to take a mixture of two Erlang distributions with same scale parameter, denoted by E K,K (µ) This distribution is of special interest to us, since we will use it to approximate distributions with Let E K (µ) with probability p and E K (µ) with probability p, so α = ( p, 0,, 0, p, 0,, 0) (dimension (K + (K )) and µ µ 0 0 0 0 0 0 0 µ µ 0 0 0 0 0 0 0 µ µ 0 0 0 0 S = 0 0 0 µ 0 0 0 0 0 0 0 0 µ µ 0 0, 0 0 0 0 0 µ µ 0 0 0 0 0 0 0 µ µ 0 0 0 0 0 0 0 µ where S is a K + (K ) K + (K )-matrix (Upper left block has dimension K K and lower right block K K ) This is equivalent with the following, more parsimonious, representation, α = (, 0,, 0) and matrix µ µ 0 0 0 µ µ S = 0 0 0 µ ( p)µ 0 0 0 µ with dimension K K Its moments are given by E[EK,K] n (K + n )! + n )! = p + ( p)(k (K )! µ n (K )! µ n [ ] and so its SCV = K (p ), which is in the interval (K+p ) K, K when p varies between [0, ] So we can use a mixture Erlang distributions with K {, 3, } to approximate a distributions with a SCV <, which we will do so in Section 33 In fact by Theorem 35 we know that a mixture of Erlang distributions can approximate any distribution arbitrary accurately Finally this example demonstrated that phase-type representations P H(α, S) are not unique Example 33 (Hyperexponential distribution) The hyperexponential distribution can be seen as a mixture of exponential random variables, with different parameters µ,, µ n > 0 and with α i > 0 and n α i =, such that its density is f(t) = n α i µ i e µ it, t > 0

8 CHAPTER 3 APPROXIMATION BY A PHASE-TYPE DISTRIBUTION This distribution is often denoted as H n (µ,, µ n ; α,, α n ) In phase-type representation the probability vector α = (α,, α n ) and the matrix µ 0 0 S = 0 µ 0 0 0 µ n Consider the special case where n =, such that α = (p, p ) with p i > 0 and p + p = ( ) µ 0 S = 0 µ Then we have F (t) = and its moments are given by p i e µ it E[H n ] = p n! µ n so that the SCV = p µ +p µ (p µ +p µ ), because and f(t) = n! + p µ n, p i µ i e µ it p µ + p µ = p (p µ + p µ ) + p (p µ + p µ ) = p µ + p p (µ + µ ) + p µ p µ + p p µ µ + p µ = (p µ + p µ ) Hence we can use this distribution to approximate distributions with a SCV, which we will show in Section 33 and the theoretical basis for this is given by Theorem 30 Example 34 (Coxian distribution) The Coxian distribution, notation C K, is a wide class in which the mixture Erlang is a special case in which the service times are equal The phase-type representation is given by a vector α = (, 0,, 0) and the matrix µ p µ 0 0 0 µ p µ S = 0 0 0 µk p K µ K 0 0 0 µ K We restrict ourselves to the Coxian- since this distribution is sometimes used in approximations whenever SCV >, since by straightforward computations we have that E [C ] = µ + p µ, E [ C ] = µ + p + p µ µ µ,

3 PHASE-TYPE APPROXIMATION 9 so that we have E [ C] 3 E[C ] This gives SCV = E[C ] E[C ] In general it holds that the SCV of C K is greater or equal to K, since the minimum SCV is obtained when we set p = p = = p K = and µ = µ = = µ K ie an Erlang K distribution for which the SCV = K In general the Coxian distribution is extremely important as any acyclic phase-type distribution has an equivalent Coxian representation, but this lies outside the scope of this thesis 3 Phase-type approximation Before we fit distributions by phase-type distributions we have to prove the validity of these fits In this section we prove that we can approximate any distribution with positive support arbitrarily accurately by a mixture of Erlang distributions, see Tijms (994) [3] Furthermore, we prove that a certain type of distributions, ie with a completely monotone density, can also be approximated arbitrarily accurately by a mixture of exponential distributions, the so called hyperexponential distribution The main idea of this proof can be found in Feller [3], however, here we modified the proof to get the required result Theorem 35 Let F (t) be the cumulative distribution function of a positive random variable with possibly positive mass at t = 0 ie F (0) > 0 For fixed > 0 define the probability distribution function K ( F (t) = p K ( ) t j e t ) + F (0), t 0, j! K= j=0 where p K ( ) = F (K ) F ((K ) ), K =,, Then lim F (x) = F (x) for any continuity point x of F (x), (ie pointwise convergence) Proof Let, x > 0 fixed and U,x be a Poisson distributed random variable with We have ( x j P [U,x = j ] = e x ), j = 0,, j! ( x j ( x j E[U,x ] = e x ) j = x e x ) j! (j )! = x, j=0 j= ( x j Var[U,x ] = E[U,x] x = x e x ) j x (j )! j= ( x j = x e x ) ( (j )! + x x j e x ) (j )! x = x j= j=

0 CHAPTER 3 APPROXIMATION BY A PHASE-TYPE DISTRIBUTION We prove for any continuity point x that lim E [F (U,x)] = F (x) 0 Fix ɛ > 0 and take a continuity point x of F (x) Then there exists a δ > 0 such that F (t) F (x) ɛ for all t with t x δ, so that: E[F (U,x )] F (x) E[ F (U,x ) F (x) ] (Jensen s inequality) = F (k ) F (x) P [U,x = k ] ɛ k=0 k: k x δ P [U,x = k ] + ɛ + P[ U,x x] > δ] = ɛ + P[ U,x E[U,x ] > δ] k: k x >δ P [U,x = k ] So we let 0 ɛ + x δ < ɛ if < ɛδ 4x (Chebyshev s inequality) which gives the desired result F (x) = lim E[F (U,x )] 0 ( x j = lim F (j )e x ) 0 j! j=0 ( x j K = lim e x ) p j ( ) + F (0) 0 j! j=0 j= ( x j = lim p K ( ) e x ) + F (0) 0 j! K=0 j=k K ( = lim p K ( ) 0 t j e t ) j! + F (0), K= So every positive random variable with a distribution function and possibly positive mass at time zero can be approximated by a mixture of Erlang distributed random variables Theorem 35 also holds for approximation by a Coxian distribution, since a mixture of Erlang distributions is a special case of a Coxian distribution We prove the statement in the following corollary Corollary 36 Let F (t) be the probability distribution function of a positive random variable with possibly positive mass at t = 0 ie F (0) > 0, then it can be approximated arbitrarily accurately by a Coxian distribution function for any continuity point x of F (x) j=0

3 PHASE-TYPE APPROXIMATION Proof The statements follows from the fact that mixture Erlang distribution is a special case of a Coxian distribution, which we prove now Let n N and Z = n K= p KE K with p K > 0 and n K= p K = in which E K has an Erlang-K distribution with parameter µ The unique Laplace-Stieltjes transform of the Erlang mixture Z is given by n Z(s) = p K Ẽ K (s), K= where ẼK is the Laplace-Stieltjes transform of E K which is have a Coxian distribution Z defined as Z = Y wp ( q ) Y + Y wp ( q )q Y + Y + Y 3 wp ( q 3 )q q n Y i wp ( q n ) n n Y i wp n q i ( µ µ+s) K On the other hand we q i in which Y i Exp(µ i ) and the probabilities sum up to Hence Z is a mixture of random variables and define K K E K µ i = Y i with Laplace-Stieltjes transform Ẽ K(s) = µ i + s So that Z (s) = n K= ( K Now, we set µ i = µ for all i and find Z (s) = n K= q i ( q K )Ẽ K(s) K q i ( q K ) Ẽ K (s) + } {{ } p K n q j } {{ } p n ) n + q j Ẽ n(s) Ẽ n (s) = n p K Ẽ K (s) = Z(s) K= By uniqueness of the Laplace-Stieltjes transform the statement is proven The hyperexponential distribution can also be used to approximate arbitrarily accurately a specific class of distributions This class should have a probability density function which is completely monotone There are several distributions, which are completely monotone, such as the exponential, hyperexponential, Weibull and Pareto distribution The Weibull distribution is seen in some healthcare settings, see Babes and Sarma (99) [4] Before we prove the statement above in a theorem we present the Extended Continuity Theorem, which relates Laplace-Stieltjes transforms to measures in limits First, we give the definition of a completely monotone function Definition 37 A probability density function f is completely monotone if all derivatives of f exist and satisfy ( ) n f (n) (t) 0 for all t > 0 and n

CHAPTER 3 APPROXIMATION BY A PHASE-TYPE DISTRIBUTION Theorem 38 If f(t) and g(t) are completely monotone functions then the positive linear product of these two is functions is completely monotone Furthermore the product of two completely monotone functions is completely monotone Proof Let a, b > 0 then h(t) = af(t) + bg(t) is completely monotone by linearity For the second statement let h(t) = f(t)g(t), so that by Leibniz rule we have n h(t) t n = n i=0 ( ) n f (i) (t)g (n i) (t), i it follows then by completely monotonicity of functions f and g Theorem 39 (Extended Continuity Theorem) For n =,, let G n be a measure with Laplace-Stieltjes transform F n If F n (x) F (x) for x > x 0, then F is the Laplace-Stieltjes transform of a measure G and G n G Conversely, if G n G and the sequence {F n (x 0 )} is bounded, then F n (x) F (x) for x > x 0 Proof See Feller (967) [3] Theorem 30 Let F (t) be a probability distribution function with a completely monotone probability density function f(t) then there are hyperexponential distribution functions F m, m of the form M m F m (t) = p mn e µmnt, t 0, with µ ni and M m n= p m n =, where p mn > 0 for all i, such that ie uniform convergence n= lim F m(t) = F (t), for all t > 0, m Proof Let F (t) the probability distribution function and consider F (a ay) for fixed a > 0 and variable y (0, ) By Taylor expansion around y = 0, by completely monotonicity of f all the derivatives exist for all y [0, ), we have F (a ay) = n=0 ( a) n F n (a) n! y n = F (a) n=0 a n + ( a) n f n (a) y n+ (n! which holds for y [0, ) We change our variable y to x (0, ) by y = e x/a, so that F a (x) = F (a ae x a ) = F (a) n=0 a ( a)n f n (a) (n + )! n+ a x = F (a) C a(n) e n a x, which is the Laplace transform of an arithmetic measure G(z) giving mass of p n = C a(n) > 0 (by completely monotonicity) to the points z = n a for n =,, In detail, we have that F a (x) = F (a) 0 C a(n) e zx dg a (z) = F (a) n= p a(n) e n a x 0 n=

33 PHASE-TYPE FIT 3 We observe that for any x F a (x) F (x) Applying the Extended Continuity Theorem 39 gives the existence of a measure G(z) G a (z) G(z) with the cumulative distribution function F (x) = 0 e zx dg(z) as it is Laplace-Stieltjes transform So, if F (t) is a cumulative distribution function with a completely monotone probability density function f(t) then there are hyperexponential cumulative distribution functions F m (t) of the form: M m F m (t) = p mn e µmnt, n= which converge to F (t) for all t > 0 if m and M n tend to infinity Observe that F m is indeed a hyperexponential distribution function for m N 33 Phase-type fit In practice it often occurs that the only information of random variables that is available is their mean and standard deviation, based on data only On basis of these two quantities one can fit (approximate) its underlying distribution by a phase-type distribution The only condition is that the random variable for which we approximate its distribution must be positive, sometimes completely monotone The syllabus by Adan and Resing (00) [] suggests specific distributions for this fitting purpose Let X be a positive random variable and (S)CV its (squared) coefficient of variation In case 0 < CV < one fits an Erlang mixture distribution that is with probability p it is an Erlang K and with p an Erlang K, in shorthand notatione K,K (µ; p) The parameters are given by K SCV K, for K =, 3, Secondly we choose p and µ such that p = + SCV ( K SCV ) K( + SCV ) K SCV, µ = K p E[X] then the E K,K distribution matches its expectation E[X] and coefficient of variation CV Because of the fact that we use an Erlang mixture model with the same scale parameter, but different shape parameters, we have that the coefficient of variation is always less than one To get a coefficient of variation greater than we have to vary the scale parameter as well The hyperexponential distribution is the simplest case with different scale parameters So in case CV we choose to fit a hyperexponential distribution with parameters p, p, µ and µ in shorthand notation H (p, p ; µ, µ ) We have four parameters to be estimated, so we set p = ( p ), so that we do not have an atom at zero Furthermore, we use balanced means p = p µ µ

4 CHAPTER 3 APPROXIMATION BY A PHASE-TYPE DISTRIBUTION then the probabilities are given by p = ( + ) SCV SCV + and p = ( ) SCV SCV + with rates µ = p E[X] and µ = p E[X] If SCV = then it reduces p = = p and µ = µ, which is equivalent with the exponential distribution And when SCV then p In case that SCV also a Coxian- distribution can be used with the following parameters µ = E[X], p = SCV, µ = µ p Remark that we use only two moments to fit the data, so that we do not match skewness and kurtosis of the particular distribution On the other hand there is enough freedom to match more moments when we use more general phase-type distributions for fitting However, choosing a parsimonious model is more practical and in most cases sufficient 34 Recursive procedure for computing sojourn times Up till now we have studied phase-type distributions as an approximation tool The reason for this is that in the general D/G/ case we cannot compute the waiting and idle times The solution for this to translate the D/G/ setting to a D/P H/ setting by approximating the general service time distributions by phase-type distributions Wang introduced in 997 [7] a recursive procedure on the calculation of the sojourn time of customers on a stochastic server We can translate this to our scheduling problem in which a doctor is seeing patients Wang s procedure makes use of an attractive property of phasetype distributions mentioned in Section 3 In this section we will explain Wang s approach in detail First, we show the approach for the specific case of exponentially distributed service times Second, we explain it for phase-type distributed service times, in which the phase-type distributions are allowed to differ among patients, which is a further generalization of Wang s result 34 Exponentially distributed service times In this section we present the iterative procedure for exponentially distributed service times, where the service times are not necessarily identical The procedure gives a good idea for the next section where the service times are phase-type distributed From now on, denotes a column vector of ones of appropriate size Suppose we have a service order,, N + We are interested in the sojourn time distribution F Si (t) = P [S i t], t 0 for all (patients) i {,, N + } Let p i,k (t) the probability that patient i sees k patients in front of him after t units of time after his arrival t i, so that the case where k = 0 corresponds

34 RECURSIVE PROCEDURE FOR COMPUTING SOJOURN TIMES 5 to that the patient i is served, so that n P [S i t] = p i,k (t) Define p i = (p i,i, p i,i,, p i,0 ) a row vector of dimension i so that k=0 P [S i t] = p i The first patient, who will be served directly, has infinitesimal generator ( ) 0 0 Q = µ µ and submatrix S = µ Then we have, by definition of phase-type distributions, p = p,0 = e µ t and F S (t) = e µ t, t 0 The second patient, who arrives at time x, will find either no patient in the system with probability p (x ) or the first patient is still in the system with probability p (x ) The first patient follows an exponential distribution with parameter µ Because of the memoryless property of exponential random variables, the waiting time of the second patient is also exponentially distributed with parameter µ So that its sojourn time is governed by the continuous-time Markov chain with infinitesimal generator Let S be the submatrix 0 0 0 Q = 0 µ µ µ 0 µ ( ) µ µ S =, 0 µ then p (t) satisfies the following system of differential equations dp (t) dt = p (t)s with p (0) = (p (t), p (t)) for which the solution is given by p (t) = (p (x ), p (x ))e S t = (e µ x, e µ x )e S t t 0 Since the Markov chain has an acyclic structure the transient solution of p i can be derived by a system of differential equations In general, for the i-th patient ( ) 0 0 Q i =, S 0 i S i

6 CHAPTER 3 APPROXIMATION BY A PHASE-TYPE DISTRIBUTION where S i is the submatrix µ µ 0 0 0 µ µ S i = 0, 0 0 µ n µ n 0 0 0 µ b then p i (t) = (p i (x i ), p i (x i ))e S it for t 0, since it is the solution of dp i (t) dt = p i (t)s i with p i (0) = (p i (x i ), p i (x i )) We point out that the time for the i-th patient starts running when he arrives Then he either waits or is served directly The corresponding interarrival time x i is used in the initial condition for the subsequent patient We summarize the above procedure in a proposition Proposition 3 If the interarrival times are x i for i =,,, N then the sojourn time distribution of the i-th (i =,,, N + ) patient is given by where F i (t) = P [S i (t) t] = p n, p = e S t, p i = ( p i (x i ), p i (x i ) ) e S it for i =, 3,, N + Furthermore, we have E[S ] = µ, E[S i ] = ( p i (x i ), p i (x i ) ) i j=0 µ i j i j=0 µ i j µ i for i =, 3,, N + This proposition is proved by induction The expression for mean sojourn times follows from the properties of phase-type distributions Furthermore, we observe that each iteration the dimension of p i (t) increases by The underlying continuous-time Markov chain is observed at the epochs of arrival times, t i At these points the states of the Markov chain Y n (t) are defined as the number of patients waiting in the system In case there are no patients waiting the arriving patient is in service We remark that only the first i patients affect the sojourn time of patient i 34 Phase-type distributed service times In this section we generalize the recursive procedure from the latter section to phase-type distributed service times We use the article by Wang (997) to describe the procedure

34 RECURSIVE PROCEDURE FOR COMPUTING SOJOURN TIMES 7 He assumes independent and identical phase-type distributed service times, where S of the phase-type representation has an acyclic structure We extend on this by varying the (acyclic) phase-type distributions among patients In detail, let patient i have a phase-type distributed service time distribution, with probability vector α i (dimension m i ) and m i m i -matrix S i Define now the bivariate process {Y i (t), K i (t), t 0} for patient i =,, N +, where Y i (t) N is representing the number of patients in front of the i-th patient and K i (t) N is the particular phase in which the service is in if the server is busy, otherwise K i (t) = 0 Furthermore, the state (0, 0) is the absorbing state and all other states are transient for every patient i Let p (i) j,k (t) the probability that {Y i(t), K i (t), t 0} is in state (j, k) (j patients before him and the server is in phase k) p (i) j,k (t) = P [(Y i(t), K i (t)) = (j, k)] ( p i (t) = p (i) i, (t),, p(i) i,m i (t), p (i) i, (t),, p(i) i,m i (t),, p (i) 0, (t),, p(i) α i = (α i,, α i,,, α i,mi ) 0,m (t) For the first patient, at t = 0, it holds that there is no patient in the system, so that the first patient is phase-type distributed with transition matrix S and α ) p (t) = (p 0,,, p 0,m ) = α e S t F (t) = p (t) Hence P[S i t] = p i (t) Then for the next patient, who is phase-type distributed (α, S ) and arrives at t = x, there are two cases: The process {Y (t), K (t), t 0} will start at state (, k) with probability p 0,k (x ), where k =,,, m The process {Y (t), K (t), t 0} will start at state (0, k) with probability α,k F (x ), where k =,,, m So that the sojourn time of the -nd patient is governed by a continuous-time Markov chain with infinitesimal generator 0 0 0 Q = 0 S S 0 α, S 0 0 S with initial-state distribution (0, p (x ), α F (x )) So that if we let ( S S 0 ) S = α 0 S then the sojourn time of the -nd patient is given by the following system of differential equations dp (t) = p dt (t)s, with initial condition p (0) = (p (x ), α F (x )) The unique solution is p (t) = (p (x ), α F (x ))e S t, t 0 In general, for patient i, who arrives at t i (just after the interarrival time x i ):

8 CHAPTER 3 APPROXIMATION BY A PHASE-TYPE DISTRIBUTION The process {Y i (t), K i (t), t 0} will start at state (i, k) with probability p i i,k (x i ), where k =,,, m The process {Y i (t), K i (t), t 0} will start at state (i, k) with probability p i i 3,k (x i ), where k =,,, m The process {Y i (t), K i (t), t 0} will start at state (, k) with probability p i 0,k (x i ), where k =,,, m i The process {Y i (t), K i (t), t 0} will start at state (0, k) with probability p i 0,k (x i ) = α i,k F i (x i ), where k =,,, m i So that the sojourn time of i-th patient is governed by a continuous-time Markov chain with infinitesimal generator 0 0 0 0 0 0 S S 0 α 0 0 Q i = 0 0 S 0, S 0 i α i 0 0 0 0 S i S 0 i α i S 0 n 0 0 0 S i with initial-state distribution (0, p i (x i ), α i F i (x i )) and submatrix S S 0 α 0 0 0 S 0 S i = S 0 i α i 0 0 0 S i S 0 i α i 0 0 0 S i So that the sojourn time of the i-th patient is given by the following system of differential equations dp i (t) = p dt i (t)s i with initial condition p i (0) = (p i (x i ), α i F i (x i )) The solution is given by p i (t) = (p i (x i ), α i F i (x i ))e S it, for t 0 So for finite number of patients we can compute the individual sojourn time distributions We summarize this recursive procedure and state it as a proposition Proposition 3 If the interarrival times are x i for i =,,, N then the sojourn time distributions F i (t) for (patient) i =,,, N + are given by F i (t) = P[S i t] = p i (t)

34 RECURSIVE PROCEDURE FOR COMPUTING SOJOURN TIMES 9 where p (t) = α e S t, S = S p i (t) = (p i (x i ), α i F i (x i ))e S it for i =, 3,, N + The mean sojourn time is E[S i ] = (p i (x i ), α i F i (x i ))S i This proposition can be proved by induction The expression for the mean sojourn times follow directly from the properties of phase-type distributions, see Section 3 Observe that p i relies completely on p i and its dimension is expanded from i j= m j to i j= m j, the phases of the new patient are added to continuous-time Markov chain Also, the distribution of the i-th patient is a function of x,, x i only, because of the first-come, first-served discipline When all phase-type distributions are exponential distributions we are in the simple case described extensively in the previous subsection In this chapter we introduced phase-type distributions and proved their ability to approximate distributions with a positive support arbitrarily accurately After which we gave explicit and easy-to-use formulas to fit distributions roughly In order to do so, we divided distribution functions into two categories: Distribution functions with a SCV are fitted by a E K,K (µ; p) distribution Distribution functions with a SCV are fitted by a H (µ, µ ; p, p ) distribution Furthermore, we demonstrated a recursive procedure for phase-type distributions to compute the patient s individual sojourn time distributions This procedure will be used in the next chapter for optimization

Chapter 4 Optimization methods In this chapter we obtain optimal schedules in different settings The fact that the distribution functions can be approximated by phase-type distributions gives us the opportunity to use the recursive procedure described in the latter chapter So the first step is to approximate the service times by an appropriate phase-type distribution The second step is to implement the recursive procedure on these phase-type distributed service times to find the sojourn times Third, we optimize over these sojourn times to find optimal schedules Since optimizing simultaneously for all patients is highly complex, we use numerical methods to compute optimal interarrival times Further, we compare the simultaneous approach with the sequential counterpart introduced by Kemper et al (0) This approach arose as a trade-off between computational time and a sufficiently close-to-optimal schedule Vink et al (0) introduced the so called lag-order method, which spans all trade-offs between the sequential and simultaneous approach We will describe the idea behind the lag-order method for the sake of completeness in Section 43 Observe that the optimization problem is over t,, t N+ By the relation between t i s and x i s defined by t i = i j= x j, see Chapter, the problem is equivalent with minimizing over the interarrival times x,, x N only Let us first describe different optimization methods 4 Simultaneous optimization In the classical case one minimizes the system s loss over all possible schedules This means that all patients are scheduled simultaneously, so that we minimize as follows: min R(t,, t N+ ) = min R(x,, x N ) t,,t N+ x,,x N N+ = min E [l(s i x i )] x,,x N = min x,,x N i= N E [l(s i x i )], (4) where l(x) is convex function To optimize simultaneously there is almost no tractable derivation possible Only the exponential case has a tractable solution, see Wang (997) The difficulty is that the x i s are too much interlinked The choice of an optimal x i depends 30

4 SEQUENTIAL OPTIMIZATION 3 implicitly on the previous optimal interarrival times x i,, x This is because the arrivals of preceding patients influence the waiting and idle times of patient i Moreover, the fact that phase-type distributions have no nice representations makes computations intractable Therefore in this case one wants to uses numerical algorithms to find the optimum interarrival times, see the appendix for such an outline of such an algorithm A solution which leads to tractable solution is to consider the problem sequentially as we will do so in the next section 4 Sequential optimization A solution to the complexity of optimizing simultaneously is to consider the optimization problem sequentially This approach is introduced by Kemper et al (0) [6] If one optimizes the schedule sequentially, one minimizes for all i {, N + }, given that you know t i, t i, t min R(t i, t i,, t ) = min E [g(i i ) + h(i i )] t i t i = min E [l(s i x i )] (4) x i In this formula S i depends implicitly on the previous arrival epochs t i,, t or equivalently x i,, x, but these are fixed, since we consider the optimization problem sequentially This reduces the problem drastically, such that there is an optimal schedule under some conditions, see Theorem 4 below, from Kemper (0) [6] Theorem 4 Let l : R R 0 be a convex function with l(0) = 0 Let B,, B N+ be independent non-negative random variables such that [ ( N )] E l B i + y < holds for all y R 0 Let schedule W defined by t = 0, where x j 0 is the value at which i t i = x j, i =,, N +, j= R(l) (x j ) x j = E [ l (S j x j ) ] = 0 or changes sign In case there is no x j satisfying this condition then it is set equal to Then the schedule W sequentially minimizes the loss Proof First we show finiteness: [ ( N )] E [l(s j x j )] E l B i x j <, x j < 0 [ ( N )] E [l(s j x j )] E [l (S j )] + l(x j ) E l B i + l(x j ) <, x j 0

3 CHAPTER 4 OPTIMIZATION METHODS Because of convexity, l (x j ) is monotone and the non-negativity of l(x j ) imply that for all a b b l (x j ) dx j l(b) + l(a) a a By Fubini s Theorem we have b E [ l (S j x j ) ] [ b dx j = E l (S j x j ) ] dx j, combining this with the results above we find b a E [ l (S j x j ) ] dx j E [l(s j b)] + E [l(s j a)] so that E[l(S j x j )] is absolutely continuous with derivative E[l (S j x j )] and therefore convex Hence there exists a minimum, since l(x) R 0 Moreover, E[l (S j x j )] is nonincreasing in x j and non-negative at x j = 0 ie E[l (S j )] 0 always Therefore x j is nonnegative j Since weighted linear and quadratic cost functions satisfy the conditions of Theorem 4, we can derive optimal interarrival times, which we present here as examples 4 Quadratic loss Consider R i the weighted quadratic cost function, cf equation (8) We optimize sequentially, so one has to minimize the following function over x i given x i,, x ] R i (x i,, x ) = E [l(s i x i )] = E [α (S i x i ) {Si x i <0}<0 + ( α) (S i x i ) {Si x i <0}>0 Using Theorem 4 one obtains x i by solving E [ l (S i x i ) ] ] ] = ( α)e [(S i x i ) {Si x i >0} αe [(S i x i ) {Si x i ] <0} = αe [S i x i ] ( α)e [(S i x i ) {Si x i <0} = αe [x i S i ] ( α) Where the latter integral follows from ] E [(S i x i ) {Si x i <0} = = (Fubini) = = = a x i 0 s x i P [S i > t] dt = 0 (s x i ) {s x i <0} df Si (s) x i 0 0 0 x i t+x i dtdf Si (s) df Si (s)dt P[S i > t + x i ] dt P[S i > t] dt

43 LAG-ORDER METHOD 33 Suppose the special case that α = solution reduces to (waiting and idle times are equally weighted) then the x i = ES i Thus the optimal interarrival times are equal to the mean of the (corresponding) sojourn times 4 Absolute loss Consider R i the weighted absolute loss function, cf equation (6) Optimizing sequentially means that we minimize the following function over x i given x i,, x R i (x i,, x ) = E [l(s i x i )] = E [α S i x i Si x i <0 + ( α) S i x i Si x i >0] By Fubini s Theorem the latter can be rewritten E [l(s i x i )] = (Fubini) = = α xi xi 0 s xi t 0 0 xi 0 α dtdf Si (s) + α df Si (s)dt + F Si (t) dt + ( α) s x i x i t x i x i ( α) dtdf Si (s) ( α) df Si (s)dt ( F Si (t)) dt Now, using Theorem 4 one obtains x i by solving E [ l (S i x i ) ] = αf Si (x i ) ( α) ( F Si (x i )) = 0 x i = F S i ( α) So in this example the optimal interarrival times are quantiles of the (corresponding) sojourn time distributions 43 Lag-order method In this section we discuss the lag-order method introduced by Vink et al (0) [5] The key observation is that patient i s waiting time depends on all preceding interarrival times However, we can restrict the influence of preceding interarrival times which are far away from patient i s, since interarrival times further away from x i affect the x i less Hence, we consider for all i =,, N instead of min R i (x i, x i,, x i k ) = min E [l(s i (x i,, x i k ) x i )], (43) x i x i min R i (x i, x i,, x ) = min E [l(s i (x i,, x ) x i )] x i x i This procedure is an optimization method which contains every compromise between optimizing sequentially, k =, to optimizing simultaneously, k = N For the practical consideration of this approach we refer to the paper by Vink et al

34 CHAPTER 4 OPTIMIZATION METHODS 44 Computational results for transient cases In this section we evaluate the simultaneous and sequential optimization for three different SCV values where we set the mean equal to one We choose similar SCV values as in Wang (997) [6] Herein, Wang chooses (as an example) a Coxian-3, exponential and Coxian- distribution to model a SCV = 0786, SCV = and a SCV = 6036 We use our moment matching procedure, see Section 33, to match mean one and the above SCV values We summarize our parameters To model a SCV = 0786 Wang uses a Coxian distribution with three phases, namely, (µ, µ, µ 3 ) = ( 4 3, 8 3, 4) and (p, p ) = (, ) so that α = (, 0, 0) and 4 3 3 0 S = 0 8 4 3 3 0 0 4 We model the same SCV by an E K,K (µ; p) distribution with parameters K =, µ = 6006 and p = 0399744 Ie α = (, 0) and ( ) 6006 0960563 S = 0 6006 SCV = Wang uses an exponential distribution with µ = to match this SCV, by using the phase-type fit we get exactly the same distribution To model a SCV = 6036 Wang uses a Coxian distribution with only two phases, (µ, µ ) = (3, 04333) and p = 0, so that α = (, 0) and ( ) 3 03 S = 0 04333 We model this SCV by a H (µ, µ ; p, p ) distribution with parameters chosen by our matching method In phase-type notation α = (p, p ) = (0740745, 05955) and ( ) 4849 0 S = 0 0585 In the following figures we present our computations of optimal schedules in terms of interarrival times x i for different number of patients N +, where N = 5, 0, 5, 0, 5, different SCV s: SCV = 0786,, and 6036, to match the paper of [6] the chosen phase-type distributions of Wang and our phase-type fit to match the corresponding SCV s, equally weighted linear and quadratic cost functions, simultaneous (Figures 4, 4 and 43) and sequential optimization (Figure 44)

44 COMPUTATIONAL RESULTS FOR TRANSIENT CASES 35 Wang computed in his article optimal schedules for N = 0 by simultaneous optimization over a linear cost function Kemper (0) computed the optimal schedule for exponentially distributed service times, ie SCV = We extend this by also computing optimal schedules for various SCV s We find optimal values in the simultaneous case by the constraint optimizer in MatLab, ie fmincon In case of sequential optimization we use the command fsolve In Figures 4, 4 and 43 optimal schedules for different number of patients N + are plotted We observe the following pattern in the interarrival times: they are low in the beginning of the schedule; then they seem to converge to a (maximum) value attained somewhere in the middle, and then they seem to decrease in exact the opposite way as how they increased These patterns are known as dome shapes, which are also observed in [5] and in [5] In Figure 4(b) we compute the optimal interarrival times for simultaneous optimization with the quadratic costs This can be compared with the paper by Vink [5], where he computed in the exact same setting the optimal schedule for N = 0 If N increases we see that the middle parts of the schedules are converging to a maximum value We will compute these maxima in Chapter 5, since they are the solution of the steadystate In all figures we see that quadratic costs lead to higher interarrival times, ie less tight schedules In all figures we observe that the difference between the optimal schedule with Wang s parameter choice and the phase-type fit of Section 33 is extremely small This gives us an indication that the phase-type fitting procedure grasps the general characteristics well In Figure 44 we compute optimal schedules by sequential optimization We see a huge difference between these schedules and the optimal schedules derived with simultaneous optimization This seems reasonable since the optimization method is completely different The dome shape pattern is replaced by a single increase in interarrival times The interarrival times seem to converge to a maximum as the slope is decreasing Comparing the schedules in Figure 44 for different SCV s, we see that in the linear case, (a) and (c), higher SCV s lead to lower interarrival times in the beginning and greater interarrival times at the end of the schedule, so that the graphs intersect In Figure 44(b) and (d) we see that the graphs with higher SCV s have greater interarrival times always

36 CHAPTER 4 OPTIMIZATION METHODS 8 8 6 6 xi 4 xi 4 5 0 5 0 5 i (a) Linear costs, phase-type fit 5 0 5 0 5 i (b) Quadratic costs, phase-type fit 8 8 6 6 xi 4 xi 4 5 0 5 0 5 i (c) Linear costs, Wang s parameters 5 0 5 0 5 i (d) Quadratic costs, Wang s parameters Figure 4: the optimal schedules in x i s by simultaneous optimization for SCV = 0786 < 8 8 6 6 xi 4 xi 4 5 0 5 0 5 i (a) Linear costs 5 0 5 0 5 i (b) Quadratic costs Figure 4: the optimal schedules in x i s by simultaneous optimization for SCV = The parameter choice for both Wang s method as our phase-type fit is the same

44 COMPUTATIONAL RESULTS FOR TRANSIENT CASES 37 xi 8 6 4 5 0 5 0 5 i (a) Linear costs, phase-type fit xi 8 6 4 5 0 5 0 5 i (b) Quadratic costs, phase-type fit xi 8 6 4 5 0 5 0 5 i (c) Linear costs, Wang s parameters xi 8 6 4 5 0 5 0 5 i (d) Quadratic costs, Wang s parameters Figure 43: the optimal schedules in x i s by simultaneous optimization for SCV = 6036 >

38 CHAPTER 4 OPTIMIZATION METHODS 6 4 8 xi 08 06 SCV = 6036 SCV = SCV = 0786 xi 6 4 SCV = 6036 SCV = SCV = 0786 4 6 8 0 4 i 4 6 8 0 4 i (a) Linear costs, phase-type fit (b) Quadratic costs, phase-type fit 6 4 8 xi 08 06 SCV = 6036 SCV = SCV = 0786 xi 6 4 SCV = 6036 SCV = SCV = 0786 4 6 8 0 4 i 4 6 8 0 4 i (c) Linear costs, Wang s parameters (d) Quadratic costs, Wang s parameters Figure 44: the optimal schedule in x i s by sequential optimization for different SCV s

Chapter 5 Limiting distributions In Chapter 4 we observed some convergence of the optimal interarrival times in sequential and simultaneous optimization seem to converge, see Section 44 In particular, in simultaneous optimization we saw the optimal times in the middle of the schedule converge to the same value when N increased In sequential optimization we noticed the convergence of the (right) tail of the optimal times This indicates that in both cases there is some underlying limit x to which the optimal interarrival times x i converge as N An approach to find these limits is to let the number of patients tend to infinity, but numerical optimization is not doable in this setting, since the computation time will take infinitely long That is why we focus on other ways to derive the limit interarrival times in this chapter We start off with the derivation of interarrival times in case of exponential service times More specifically we look at the steady-state of the D/M/ queue under sequential and simultaneous optimization For this case in particular we can derive the limit interarrival times x analytically For hyperexponential and mixture Erlang service times we are unable to derive analytic results For these phase-type distributed service times we present a method based on the embedded Markov chain In sequential optimization we solve E[l(S x)] x = 0, where we have found explicit formulas for (equally weighted) linear and quadratic cost functions in Chapter 4, that is for linear costs (absolute loss) x = F ( ) S ; for quadratic costs (quadratic loss) x = E[S] For the limit interarrival times in the setting of simultaneous optimization, we argue that these times are equidistant in the limit Furthermore, the sojourn time distribution depends on x, so that for N large R = N+ R i (N + )R i = (N + )E [l(s(x) x)] 39

40 CHAPTER 5 LIMITING DISTRIBUTIONS Hence, minimizing R is equivalent with, see [5] We found that we have to minimize E [l(s(x) x)] x = 0 for linear costs (absolute loss) R i = E[ S(x) x ] or R i = E[S(x)] + x of (0); for quadratic costs (quadratic loss) R i = E[(S(x) x) ] = E[S(x) ] + xe[s(x)] + x By equation () we know that minimizing R i or Ri result in the same optimal schedule for linear costs, which naturally also holds in steady-state 5 The D/M/ queue Suppose the service times are exponentially distributed with parameter µ In the syllabus Queueing Theory by Adan and Resing (00), and Tijms (986) [, ] the limit distribution of the sojourn time is derived for the G/M/ queue P[S i t] = e µ( σ)t, t 0, where σ solves σ = e (µ µσ)x We point out that σ is not equal to the occupation rate ρ, since we do not have a Poisson arrival process 5 Limit solutions in the sequential case When we look at the sequential optimization case, we have derived expression for both the absolute and the quadratic cost functions, which can be used to find the limit interarrival times analytically First, using the fact that in the linear costs case we have x i = F S i ( ) We find ( ) x = F S i = log µ( σ) and F S i (x) = = σ So that for absolute cost we have x = ln() µ 386 µ In case of quadratic cost, we have that x = E[S i ] = µ( σ) and σ = e (µ µσ)x, therefore x = e µ(e ) 580 µ 5 Limit solutions in the simultaneous case For the other case we minimize x i also over the x i -dependent sojourn time distributions To do so we observe that in the transient case the number of interarrival times with the same size increases, which can be seen in the figures in Section 44 So that, when n, we have that x i = x and S i = S for all patients i so for the linear cost case lim N N E [ S i (x i ) x i ] = lim NE[ S(x) x ], N

5 THE D/M/ QUEUE 4 and for the quadratic cost case lim N N E [ (S i (x i ) x i ) ] = lim N NE[(S(x) x) ] We want to minimize both expressions, this is done by taking the derivative and setting it equal to zero ( d d x ) E[ S(x) x ] = (t x)f dx dx S(x) (t) dt + (x t)f S(x) (t) dt = 0, (5) x 0 d dx E[(S(x) x) ] = d ( E[S(x) ] xe[s(x)] + x ) = 0 (5) dx It is known for the G/M/ queue that Using this we find for equation (53) that ( d (t x)f dx S(x) (t) dt + x where we used for σ x that F Si (x)(t) = e µ( σx)t, f Si (x)(t) = µ( σ x )e µ( σx)t x 0 ) (x t)f S(x) (t) dt = d e µ( σx)x µ( σ x )x dx µ(σ x ) = d σ x + log σ x dx µ(σ x ) = σ x + (log σ x )σ x µ(σ x ) σ x = + (log σ x )σ x ( σ x + σ x log σ x ) = 0, σ x = µσ x(σ x ) µσ x x, (53) which we found by implicit differentiation of the unique solution equation: σ x = e µ( σx)x and equation (55) The numerator should equal zero, so + (log σ x )σ x = 0 σ x 0378 (54) Via the unique solution equation we can also express x in terms of σ x : x = log σ x µ(σ x ) (55) Implementing the solution of equation (54) into (55) one finds for the absolute cost the optimal interarrival times in the limit case x 6803 µ

4 CHAPTER 5 LIMITING DISTRIBUTIONS Now we focus on quadratic costs, see equation (5), in order to find its limit interarrival times, we compute E [S(x)] = µ( σ x) so that E [ S (x) ] = µ ( σ x) so that de[s(x)] dx = de[s (x)] dx = σ x µ( σ x), 4σ x µ ( σ x) 3 We fill in equation (5) with the derived results and equations (53) and (55) ( E[S(x) ] xe[s(x)] + x ) 4σ x = µ ( σ x ) 3 σ x x µ( σ x µ( σ x ) + x 4µσ x (σ x ) = µ ( µσ x x)( σ x ) 3 + log σ x + µ(σ x ) σ x log σ x + µ( σ x + σ x log σ x )(σ x ) ( + log σx + σ x ( + log σ x + (log σ) ) ) = µ( σ x + σ x log σ x )(σ x ) = 0, which is equivalent to setting the numerator equal to zero resulting in σ x 05 Implementing the solution for σ x in (55) gives us the limit interarrival time x 8466 µ For phase-type distribution there is no clear derivation to find the limit distribution, since there is no similar limit representation of the sojourn time However, we will focus on two phase-type distributions, proven to be important for fitting in Section 33, for which we derive a method to compute their limit distribution function This enables us to compute limit interarrival times also for cases where SCV 5 The D/E K,K / queue The hyperexponential (H ) and mixture Erlang (E K,K ) distributions are often used in practice to fit unknown distributions by means of their mean and standard deviation In case the data has a (S)CV it is common to fit an E K,K (µ; p) distribution Its density is given by: f B (t) = pµ (µt)k (K )! e µt + ( p)µ (µt)k (K )! e µt (56) In this section we study the D/E K,K / queue In this system patients arrive one by one with interarrival times which are identically and independently distributed with density f A (t) = δ x (t), because arrivals are deterministic The service times are mixture Erlang distributed with density defined in equation (56) For stability we require that for the occupation rate ρ, it holds that ρ = E[A] E[B] = ( p K + ( p) K ) < x µ µ

5 THE D/E K,K / QUEUE 43 So that x is larger than the average service requirement The state of the D/E K,K / queue can be described by the pair (n, t), where n is the total number of phases to be completed and t the elapsed time since the last arrival To solve this problem we need a two dimensional state description, where t is continuous The amount of phases in the system upon arrival is exactly how much work is left before the arriving patient can be served So we look at the system just before arrivals to make computations easier Let N k the total number of phases in the system which are not yet completed just before the k-th arrival The relation between N k+ and N k is then given by { Nk D N k+ =,k+ + K with probability p N k D,k+ + K with probability p where D,k+ and D,k+ are the number of phases completed between the arrival of the k-th and k + -th patient There is a distinction between these two as in the latter case there is one more phase in the system The sequence {N k } k=0 forms a Markov chain, for which we will compute the limit probabilities in the next section These limit probabilities can be used to derive the steady-state sojourn time distribution see Section 5 5 The limiting probabilities In this section we determine the limit distribution The limit probabilities a m solve the equation a m = lim k P [N k = m] a = ap, (57) where a = (a 0, a, a, ) The transition probabilities from one state to another are given by p m,n = P [N k+ = n N k = m], which is a mixture of two Poisson processes: p m,n = pp [Pois(µx) = m n + K ] + ( p)p [Pois(µx) = m n + K] (58) So that we distinguish four cases First the transition probability to a state more than K phases higher than the state you are in is 0 The state m + K from m can only be reached if you are in the Pois(µx) case which has probability ( p) Thirdly, the states between the 0 phase and K phases higher than your starting state can be reached by both Poisson processes However, the latter process added K phases to the system instead of K, so that it has to complete one more phase Finally, the probabilities should add up to one, so that the probability of jumping to zero phases in the system is equal to minus all the probabilities of moving to nonzero phases We summarize For n > m + K we have: p m,n = 0 For n = m + K we have: p m,n+k = ( p)e µx =: β 0 For 0 < n < m + K For n = 0 we have: p m,n = p (µx)m n+k (m n+k )! e µx + ( p) (µx)m n+k m n+k! e µx =: β m n+k we have: p m,0 = m+k n= p m,n

44 CHAPTER 5 LIMITING DISTRIBUTIONS 05 0 0 04 03 0 am 0 am 0 4 0 0 0 5 0 5 0 5 m (a) Lin-lin scale 0 6 0 5 0 5 0 5 m (b) Log-lin scale Figure 5: The a m, probability of finding m phases to be completed, for m [0,,, 5] on two different scales Here, the mixture Erlang distribution is found by a phase-type fit of Section 33 where the SCV = 0786 and E[X] =, the interarrival time x = 3 Now we write the matrix P of transition probabilities as p 0,0 β K β K β β 0 0 p,0 β K+ β K β K β β 0 0 P = p,0 β K+ β K+ β K β K β β 0 0 p 3,0 β K+3 β K+ β K+ β K β K β β 0 Next we use equation (57) to compute the limit probabilities a Since the dimension of a is infinite, we are obliged to choose a certain cutoff point N, we solve(a 0, a,, a N ) = (a 0, a,, a N )P Only in the case of exponential service times one can put in solutions of the form a m = σ m, see Adan and Resing [] Still, cutting off at a certain point seems legible, because by stability the limit probabilities a m exponentially decay to zero for m > M, see Tijms [3] (page 88) and graphically in Figure 5 The kink in the end is because of cutting the probability vector off at N = 5 5 The sojourn time distribution In the last section we calculated the limit probabilities These probabilities give exactly how much work there is left in the system upon arrival, ie the waiting time So one can find the steady-state sojourn time by the convolution formula P [S t] = P [W + B t] = t u=0 P [W t u B = u] P [B = u] du = t u=0 F W (t u)f B (u) dt, where f W (t u) = a 0 + N m= a mµ (µ(t u))m (m )! e µ(t u) and f B (u) is of course the service time defined in equation (56) So that the expressions that are needed for computing the optimal

53 THE D/H / QUEUE 45 interarrival times numerically are given by: P [S t] = a 0 F B (t) + N t (µ(t u))m a m µ e µ(t u) f B (u) du, (m )! m= E[S] = E[W ] + E[B] N = m=0 E[S ] = E[(W + B) ] m= 0 ( a m p m + K + ( p) m + K µ µ = E[W ] + E[W ]E[B] + E[B ] N m(m + ) N = a m µ + + ( p m= K(K ) µ + ( p) ), ( a m p m K µ µ (K + )K µ ) + ( p) m + µ These expressions converge to the true values when N We will see in Section 54 that in our optimizations N = 0 + K is already enough 53 The D/H / queue In this section we study the D/H / queue In this system patients arrive one by one with identically and independently distributed interarrival times with density f A (t) = δ x (t), because arrivals are deterministic The service times are hyperexponentially distributed, which are used when the data has (S)CV It is common to fit an H (p, p ; µ, µ ) distribution in these cases Its density is given by: ) K µ f B (t) = p µ e µ t + p µ e µ t, (59) where p + p = cf Example 33 An hyperexponential variable can be seen as a drawing between the random variables X Exp(µ ) and X Exp(µ ) with probabilities p and p For stability we require that for the occupation rate ρ it holds that ρ = E[A] E[B] = x ( p µ + p µ ) < So that the interarrival time x is larger than the average service requirement The state of the D/H / queue can be described by the triplet (i, m, t), where i denotes the number of patients in the system who are to be served, k the phase (read: type) of the patient in service and t the elapsed time since the last arrival It is more natural to call the specific phase of the patient in service type, because the hyperexponential service times can be seen as a drawing between two types of patients To solve this problem we need a three dimensional state description, where t is continuous It is easier to look at the system just before arrivals x and let N k be the bivariate vector (i, m) just before the k-th arrival First we derive a relation between N k+ and N k N k+ = N k + (, 0) D k+,

46 CHAPTER 5 LIMITING DISTRIBUTIONS where D k+ is the number of phases and patients served between the arrival of the k-th and k + -th patient So N k is a discrete bivariate process, which is the embedded Markov chain We compute for this Markov chain the limit probabilities in the next section Then we use these probabilities to derive the steady-state sojourn time distribution in the final section of this chapter, Section 54 Observe that in this state description we have to consider the phase and number of patients separately, which is in contrast with the approach in Section 5 53 The limit probabilities We determine the limit distribution The limit probabilities a i,m solve the equation a i,m = lim k P [N k = (i, m)] a = ap, (50) where a = (a 0,0, a 0,, a 0,, a,, ) The associated probabilities are p i,m;j,n = P [N k+ = (j, n) N k = (i, m)], where we remark that: p i,m;j,n = 0 for all j > i + independent of the choice of i and j, p i,m;j,n = 0 for j = i + and m n (the patient in service cannot switch phase) So we write the matrix P as: P = = p 0,0;0,0 p 0,0;0, p 0,0;0, 0 p 0,;0,0 p 0,;0, p 0,;0, p 0,;, 0 p 0,;0,0 p 0,;0, p 0,;0, 0 p 0,;, 0 p,;0,0 p,;0, p,;0, p,;, p,;, p,;, 0 p,;0,0 p,;0, p,;0, p,;, p,;, 0 p,;, 0 p,;0,0 p,;0, p,;0, p,;, p,;, p,;, p,;, p,;3, 0 p,;0,0 p,;0, p,;0, p,;, p,;, p,;, p,;, 0 p,;3, 0 p 0,0;0,0 (p 0,0;0,, p 0,0;0, ) 0 P 0,0 β β 0 0 P,0 β β β 0 0 P,0 β 3 β β β 0 0 (5)

53 THE D/H / QUEUE 47 First we look at the probability of serving exactly K patients in x amount of time: ( K P H i < x, } {{ } A K+ H i > x ) = P [A B]), } {{ } B (see Figure 5) = P [B] P [A c ], = P [ K+ i=0 ] [ K ] H i > x P H i > x, where H i are independent random variables with density defined by equation (59) for all i Continuing c A A B B c Figure 5: the relation between the sets A and B in a diagram The whole rectangle must be seen as the complete probability space Ω = A B K+ ( ) [ j ] K+ j K + P [A B] = p j j pk+ j P X i (µ ) + X i (µ ) > x j=0 K ( ) [ j ] K j K p j j pk j P X i (µ ) + X i (µ ) > x, = j=0 K+ j=0 K j=0 ( K + j ) p j pk+ j P [E j (µ ) + E K+ j (µ ) > x] ( ) K p j j pk j P [E j (µ ) + E K j (µ ) > x], (5) where X i and E K are exponentially and Erlang-K distributed with given parameter Upon arrival, however, there is still a patient in service of type m, then K = (i j) patients are (fully) served, for which there are K combinations, the K + -th patient is

48 CHAPTER 5 LIMITING DISTRIBUTIONS still in service and of type n Incorporating this gives four modifications of formula (5): F (K) = p F (K) = p F (K) = p F (K) = p K j=0 K j=0 K j=0 K j=0 ( ) K p j j pk j (G j+,k j (x) G j+,k j (x)), ( ) K p j j pk j (G j+,k j+ (x) G j+,k j (x)), ( ) K p j j pk j (G j+,k j+ (x) G j,k j+ (x)), ( ) K p j j pk j (G j,k j+ (x) G j,k j+ (x)), where G K,K (x) = P [E K (µ ) + E K (µ ) > x] = P [E K (µ ) + E K (µ ) x] = H K,K (t) = x 0 h K,K (t) dt, see Section 53 for an explicit formula for the density function h(t) and cumulative distribution function H(t) Now we are able to compute β K β K = ( F (K ) ) F (K ) F (K ) F (K ) (53) and β 0 β 0 = ( ) e µ x 0 0 e µ x Upon arrival of the first patient the server will either be a type or a type patient, with probabilities p and p, so that the transition vector becomes p 0,0;0,0 p e µ x p e µ x p 0,0;0, = p 0,0;0, p e µ x p e µ x Starting with one patient in service upon arrival we have two vectors p 0,;0,0 p 0,;0, p 0,;0, p 0,;, = p 0,;, n=0 j= p 0,;n,j p µ xe µ x µ µ (e µ x e µ x ) e µ x, 0 p µ p 0,;0,0 p 0,;0, p 0,;0, p 0,;, = p 0,;, n=0 j= p 0,;n,j µ µ (e µ x e µ x ) p µ p µ xe µ x 0 e µ x

53 THE D/H / QUEUE 49 ai 07 06 05 04 03 0 0 ai 0 0 0 0 4 0 6 0 8 0 0 5 0 5 0 5 i (a) Lin-lin scale 0 0 0 5 0 5 0 5 i (b) Log-lin scale Figure 53: The a i = m= a i,m, probability of finding i patients waiting on arriving, for i [0,,, 5] on two different scales Here, the hyperexponential distribution is found by a phase-type fit of Section 33 where the SCV = 6036 and E[X] =, the interarrival time x = 5 In general we have the following two vectors p i,;0,0 i+ j=0 n= p i,;0, p i,;j,n f (i) p i,;0, f (i) p i,;j, p i,;j, p i,;i, p i,;i, p i,,i+, p i,;i+, = f (i j) f (i j), p µ xe µ x µ µ (e µ x e µ x ) e µ x 0 p µ p i,;0,0 p i,;0, p i,;0, p i,;j, p i,;j, p i,;i, p i,;i, p i,;i+, p i,;i+, = i+ j=0 n= p i,;j,n f (i) f (i) f (i j) f (i j) µ µ (e µ x e µ x ) p µ xe µ x 0 e µ x p µ Remark that we have an iterative relation for i which we will use to compute the probabilities (p i,;0,, p i,;0,,, p i,;i+,, p i,;i+, } {{ } p i, ) = (p i,;0,, p i,;0,, p i, ), and p i,;0,0 = (( i ) ) n= j=0 p i,;j,n + p i,;0,n Analogously (p i,;0,, p i,;0,,, p i,;i+,, p i,;i+, } {{ } p i, ) = (p i,;0,, p i,;0,, p i, ), and p i,;0,0 = (( i ) ) n= j=0 p i,;j,n + p i,;0,n 53 The sojourn time distribution We use equation (50), where we cut the vector a off for some number of patients N, since they are exponentially decreasing, see Figure 53 In Section 5 we discuss the validity of

50 CHAPTER 5 LIMITING DISTRIBUTIONS this approach We want to cutoff at a maximum of N patients in the system, therefore we cutoff at patient i = N, since then there is one patient in service and N patients waiting to be served The probabilities in a can then be used to compute the sojourn time distribution P [S t] = P [W + B t] = t u=0 P [W t u B = u] P [B = u] du = t u=0 F W (t u)f B (u) du, where f B (u) is of course the density of the hyperexponential distribution of equation (59) and for the distribution function of the waiting time we find N F W (t) = a 0,0 + N = a 0,0 + i=0 m= i a i,m P H j (t) + X m (µ m ) < t i a i,m i=0 m= j=0 j=0 ( i j ) p j pi j P [E j+ m (µ ) + E i j+m (µ ) < t], where H j H, j with density defined in equation (59) We use this formula to derive relevant expressions for computing the optimal schedules numerically P [S t] = a 0,0 t 0 N f B (u) du + N = a 0,0 F B (t) + N = a 0,0 F B (t) + N = a 0,0 F B (t) + i=0 m= a i,m i=0 m= j=0 t i a i,m P H j (t) + X m (µ m ) < t u f B (u) du i i a i,m i=0 m= j=0 i a i,m i=0 m= j=0 0 ( ) i p j j pi j ( ) i p j j pi j j=0 t 0 t 0 P [E j+ m (µ ) + E i j+m (µ ) < t u] f B (u) du h j+ m,i j+m (t u)f B (u) du ( ) i p j j pi j (p H j+3 m,i j+m (t) + p H j+ m,i j+m (t)), in the last step we used that f B (u) is a mixture of exponential distributions Now we derive the function H K,K (x) and h K,K (x), which are also used to compute the β K in equation (53) Assume without loss of generality that µ > µ (and K, K 0, if K or K is equal

53 THE D/H / QUEUE 5 to zero we have simply F EKi (x)) H K,K (x) = = x t=0 x t=0 F EK (µ )(x t)f EK (µ )(t) dt ( ) K e µ (x t) (µ (x t)) n n! n=0 K = F EK (µ )(x) n=0 K = F EK (µ )(x) = F EK (µ )(x) n=0 e µ x µ n µ K n!(k )! x e µ x (µ x) n µ K n!(k )! µ K K µ (K )! where α = i + K and β = µ µ, and h K,K (x) = x t=0 x n=0 f EK (µ )(x t)f EK (µ )(t) dt µ K t=0 µ (µ t) K e µ t (K )! dt e (µ µ )t (x t) n t K dt n i=0 f Γ(n+,µ )(x) ( n x )( x) i e βt t α dt i t=0 n ( ) n ( x) i Γ(α) i β α F Γ(α,β) (x), = t=0 (K )! (x t)k e µ (x t) µ K (K )! tk e µ t dt µ K µ K K ( K = (K )! (K )! i = = i=0 i=0 ) ( ) i x K i t K +i e µ (x t) e µ t dt µ K µk K ( ) i x K i e µ x (K + i )! (K )!(µ µ ) K i!(k i=0 i)!(µ µ ) i µ K µk K (K )!(µ µ ) K i=0 x t=0 f Γ(α,β) (t) dt ( ) i x K i e µ x (α )! i!(k i)!β i F Γ(K +i,µ µ )(x) dt We have also simple expression for the first and second moments of the sojourn time E[S] = E[W ] + E[B] = N i=0 {a i, (ie[h ] + µ ) + a i, (ie[h ] + µ )} + E[H ], E[S ] = E[W ] + E[W ]E[B] + E[B ] N i ( ) { i (j + )(j + ) = a i, j i=0 +a i, i + j=0 N i=0 j=0 µ (j + )(i j) + + µ µ } (i j)(i j + ) µ } (i j + )(i j + ) µ ( ) { i j(j + ) j(i j + ) j µ + + µ µ ) )} {a i, (ie[h ] + µ + a i, (ie[h ] + µ E[H ] + ( p µ + p ) µ,

5 CHAPTER 5 LIMITING DISTRIBUTIONS [ i ] ( ) where we used E j=0 H j + X m = ie[h ] + µ m = i p µ + p µ + µ m, and i E H j + X m = j=0 = = i j=0 ( ) i p j j pi j E [(E j+ m (µ ) + E i j+m (µ )) ] i ( ) i p j { [ j pi j E E j+ m (µ ) ] + E [E j+ m (µ )] E [E i j+m (µ )] j=0 +E [ Ei j+m (µ ) ]} i j=0 ( i j ) p j pi j { j(j + ) µ j(m j + ) + + µ µ 54 Computational results in steady-state } (m j + )(m j + ) µ In this section we present the main results of this thesis, that is we compute for various SCV s the optimal interarrival times in the limit We derive these optimal times by the methodologies described in the latter two sections We implemented these methodologies in MatLab and used fmincon and fsolve to find the optimal limit interarrival times in the simultaneous and sequential case respectively We present these optimal times in the form of readable figures which can be used as a direct tool for practitioners in choosing an optimal schedule for arbitrary distributions Beside sequential and simultaneous optimization we also distinct linear and quadratic costs, see Figure 54 where we show the dependence of these optimal times on the cutoff point N These figures are composed of the mixture Erlang model of Section 5 and hyperexponential model of Section 53, to model SCV < and SCV respectively We put in the optimal x value found in Section 5 (SCV = ) and used this as a starting value to compute optimal solutions x iteratively for higher SCV s as well for SCV s lower than For this purpose we had to cut the vector a off for a certain N, which was based on limiting behavior of the vector a, derived in Tijms [3] This gave us mathematical ground to choose a cutoff point N after which we neglect the patients in the system Since we chose Erlang mixture distribution for fitting distributions with SCV < and hyperexponential for SCV, we have to investigate the dependence on N separately for these two SCV intervals Moreover, the methodology used in the two intervals differ, so that the cutoff point N in the mixture Erlang case is actual a = (a 0, a,, a N+K ) and in the hyperexponential a = (a 0,0, a 0,, a 0,,, a N,, a N, ) We computed for N = 5, 0, 5, 0 and 5 the optimal interarrival times in four different settings In Figure 54 we show the influence of the cutoff number N on the solutions and see how the optimal schedules converge if N increases Observe that in different settings it suffices to have a lower N than in other settings This is shown in Table 54 where we compare the maximum difference with the findings in the figure by one subsequent N value We conclude that the chosen N values in Figure 54 are reasonable So we know that in all setting the interarrival times are very accurate for N = 5 In Figure 55 we plotted the four different optimal limit interarrival times for SCV (0, 3) with N = 5 We see that simultaneous optimization over quadratic costs increases fast, followed by simultaneous over linear costs Remarkable is that simultaneous optimization over linear

54 COMPUTATIONAL RESULTS IN STEADY-STATE 53 3 5 x 8 6 x 4 N = 0 N = 5 N = 0 N = 5 0 05 5 5 3 SCV (a) Simultaneous optimization with linear costs 5 N = 5 N = 0 N = 5 0 05 5 5 3 SCV (b) Simultaneous optimization with quadratic costs 5 4 3 xn = 5 N = 0 N = 5 N = 0 N = 5 0 05 5 5 3 SCV (c) Sequential optimization with linear costs x 8 6 4 N = 5 N = 0 N = 5 0 05 5 5 3 SCV (d) Sequential optimization with quadratic costs Figure 54: the dependence of the cutoff point N in different settings costs has a similar shape as sequential optimization over quadratic loses The lowest graph is sequential optimization over linear costs Furthermore, we observe a small kink in all graphs at the point SCV = At these points, where SCV =, we have the optimal x as found in Section 5 This kink is the result of the change of phase-type approximation, see Section 33 This figure is of great value to compare the different settings wherein one can optimize That is why it can be used as a benchmark for practitioners to choose an optimal schedule For example, practitioners can use these optimal limit interarrival times in a variant on the simple, but effective, Bailey-Welch rule, see Bailey (95) [6] and Welch and Bailey (95) [5] This rule schedules two appointments at the start of the clinic session, ie at time zero, and individual successive appointments are scheduled equally spaced with length x found in Figure 55 The two appointments at the start of a session compensates for the possible idleness in the beginning of a session So the resulting schedule V for N + Setting x N+5 x N for SCV < x N+5 x N for SCV Sim optim over quad costs x 5 x 0 = 57 0 3 x 0 x 5 = 54 0 3 Sim optim over lin costs x 5 x 0 = 48 0 3 x 5 x 0 = 90 0 3 Seq optim over quad costs x 5 x 0 = 08 0 3 x 0 x 5 = 57 0 3 Seq optim over lin costs x 5 x 0 = 30 0 3 x 30 x 5 = 7 0 3 Table 5: the difference of subsequent optimal interarrival times x N in different settings

54 CHAPTER 5 LIMITING DISTRIBUTIONS 8 6 Simultaneous optimization over quadratic costs Simultaneous optimization over linear costs Sequential optimization over quadratic costs Sequential optimization over linear costs 4 x 8 6 4 0 05 5 5 3 SCV Figure 55: an overview of the optimal interarrival times x for four versions of the optimization problem in the steady-state, where N = 5 patients becomes: t = 0 V = t = 0 t i = (i )x for i = 3, 4, N +

Chapter 6 Optimal schedules in healthcare In this chapter we study the performance of the computed optimal interarrival times in steady state It is typical for analytical studies in optimal appointment scheduling to assume exponentially distributed service times, so that analytical derivations are possible, see Section 5 We generalize this by approaching any service time distribution by a phase-type fit, which is either a mixture Erlang or hyperexponential distribution For these distributions we are able to compute the optimal limit interarrival times, see Section 5 and Section 53 In healthcare the CV varies between 035 and 085 The assumption of exponentially distributed service times is therefore in most cases too restrictive, because it sets the CV equal to one Furthermore the data is often unimodal and right-skewed That is why Weibull and log-normal distributions can be used to describe healthcare data to match these characteristics, see Chapter We will use these distributions to compare the optimal limit interarrival times of the analytical approach with our phase-type-fit approach The phase-type fit for data with a CV < is based on a mixture Erlang distribution, ie a D/E K,K / model In order to do so we perform a Monte Carlo simulation to compute the optimal interarrival times and corresponding system s losses, see equation () for Weibull and log-normal distributed service times We set the mean of the service times equal to one and the standard deviation to 075, so that CV = 075 SCV = 0565 We use the following settings for the simulations: For initialization, we take I + = 00 patients, so that the system is in steady-state Second, we simulate the steady-state for N = 00000 patients Finally, we replicate the simulation study M = 00 times, which will give us a sample of M optimal interarrival times in different settings We use MatLab for the simulation study, wherein the random stream is set by the command: RandStream( mcg6807, Seed,) In the following two sections we show for both Weibull and log-normal distribution how to choose the parameters to match the above conditions for the mean and CV We simulate from these distributions service times and compute the optimal limit interarrival times We compare these values (and system s losses) with our findings in Chapter 5 The results are 55

56 CHAPTER 6 OPTIMAL SCHEDULES IN HEALTHCARE summarized together with the average system s losses, R = M M R i, under various settings in Table 6 In this table W corresponds to Weibull distributed service times and Z to lognormal distributed service times Model D/E K,K / D/M/ Setting x E RW E RZ E x Exp RW Exp RZ Exp Sim & quad 6030 03 3057 8466 406 394 Sim & lin 505 0860 0868 6803 08747 0944 Seq & quad 44 344 6793 580 037 30 Seq & lin 3075 09657 00 3863 08664 0900 Table 6: the optimal interarrival times x E = x N for N = 5 by phase-type approximation in steady-state and the corresponding average systems losses per patient R in different settings for Weibull (W ) and log-normal (Z) distributed service times Similarly for x Exp, which are the optimal interarrival times based on the assumption of exponential service times 30 CV 5 0 5 0 05 00 0 3 Σ, k 4 Figure 6: the possible CV values as functions of k and σ of the Weibull (decreasing line) and log-normal distribution (the dashed, increasing line) The interval enclosed by the blue dashed lines are the CV values, between 035 and 085, seen in typical healthcare data 6 Performance under Weibull distributed service times Here we compute the optimal interarrival times by Monte Carlo simulation for Weibull distributed service times We already presented the performance of the schedules based on our phase-type fit approach and on the assumption of exponentially distributed service times in Table 6 Therefore we had to simulate Weibull distributed service times which matched the mean and CV We show now how we chose the parameters Let W be a (-parameter) Weibull distributed random variable, then its density is given by f W (t) = k ( x ) k e ( λ) x k, t 0, (6) λ λ

6 PERFORMANCE UNDER WEIBULL DISTRIBUTED SERVICE TIMES 57 where k is the shape and λ the scale parameter The mean and variance are given by ( E[W ] = λγ + ), k ( Var[W ] = λ (Γ + ) ( Γ + ) ), k k where Γ(x) is the Euler gamma function, hence SCV (k) = Γ ( + ) ( ) k Γ + k Γ ( + ) (6) k See Figure 6 for a plot of the CV as a function of k To match the mean and CV value we have to solve: λ = Γ ( + ) k Γ ( + ) k Γ ( + ) = 565, k which gives k 3476 and λ 090 For this k the Weibull distribution is indeed rightskewed In fact, the Weibull distribution is for all CV > 03084 right-skewed We simulate data based on these settings and optimize in sample under different settings: either simultaneous or sequential optimization and with either linear or quadratic costs The simulations took less than half a minute for each setting in MatLab The results are presented in Table 6 We see that the Monte Carlo estimates converge fast, ie the standard deviations s are small Comparing the results with Table 6 we see that the difference between the optimal limit interarrival times in case of exponential service times is in the order of 0, while our phasetype fit differs in the order of 0 3 Moreover the average system s loss is reduced significantly in the phase-type fit approach Only in the sequential linear case the system s loss reduction is less It seems that optimizing sequentially is more sensitive to our approach This will become more visible in the setting of the next section Setting x W s(x W ) x W x E x W x Exp RW RW R W E R W R W Exp Sim & quad 5946 00070 00084 059 0307 00005 0099 Sim & lin 5058 00054 00007 0745 0860 0000 00488 Seq & quad 43 0004 0009 0597 395 0005 0078 Seq & lin 338 00033 00063 0075 0954 004 00878 Table 6: the Monte Carlo optimal limit interarrival times x W, its standard deviation s(x W ) and its average system s loss R W in case of Weibull distributed service times under different settings, compared with the values found in Table 6 The sample size M equals 00

58 CHAPTER 6 OPTIMAL SCHEDULES IN HEALTHCARE 6 Performance under log-normal distributed service times In this section we proceed similarly as in Section 6 Let Z be log-normal distributed random variable, then its density is given by f Z (t) = x (ln(x) µ) πσ e σ, t 0, (63) where µ is the location and σ the scale parameter The mean and variance are given by hence σ µ+ E[Z] = e, Var[Z] = e µ+σ e µ+σ, SCV (σ) = e σ, (64) See Figure 6 for a plot of the CV as a function of σ To match the mean and CV value we have to solve for µ and σ µ = σ e σ = 565, which gives σ = ln (565) and µ = ln(565) Using these parameter values we simulate service times and compute optimal schedules under four different settings, cf Table 6 We present our Monte Carlo simulation results in Table 63 We observe again that the optimal limit interarrival times found by the phase-type fit approach are closer and the corresponding system s losses are reduced by an order of 0 Remark that our approach performs better under optimizing simultaneously It can be seen in terms of the optimal interarrival times or Setting x Z s(x Z ) x Z x E x Z x Exp RZ RZ R E Z R Z R Exp Z Sim & quad 666 0047 0063 0804 866 0090 0075 Sim & lin 5085 00077 00033 078 08680 0000 00464 Seq & quad 4398 00064 0056 04 647 00646 0937 Seq & lin 749 00036 0036 04 086 0074 076 Table 63: the Monte Carlo optimal limit interarrival times x Z, its standard deviation s(x Z ) and its average system s loss R Z in case of log-normal distributed service times under different settings, compared with the values found in Table 6 The sample size M equals 00 system s losses that the shape of the log-normal distribution is captured less by a phase-type fit than that of Weibull distribution However, in both cases the reduction in system s loss for each settings is significant So in a healthcare setting we conclude that the phase-type fit approach performs significantly better than optimal schedule based on the assumption of exponentially distributed service times We also observe that the extra loss incurred by the phase-type fit approach is at most 5%, the case of simultaneous optimization with log-normal distributed service times

Chapter 7 Summary and suggestions for future work We studied optimized appointment scheduling in a healthcare setting We derived a mathematical model in which one minimizes the system s loss represented in expected convex functions of waiting and idle times We translated the problem to a queueing model, ie a D/G/ queue, where we assumed that: there are no walk-in patients; the service times are independent identically distributed; scheduled patients are punctual, always show up and are served in order of arrival; one wants to minimize a convex objective function Many analytical studies assume some kind of service time distribution leading to tractable solutions We presented a method to handle general service time distributions The method is to approximate any service time distribution by a phase-type fit that matches the mean and coefficient of variation This allowed us to translate the D/G/ queue to either a D/E K,K / or D/H / queue For these models we showed methods to compute optimal schedules in the transient case and steady state numerically We found optimal schedules based on simultaneous and sequential optimization for linear and quadratic losses We showed that minimizing absolute loss is equivalent to minimizing sojourn time and completion time For these four cases we computed optimal schedules in the transient case, see Chapter 4, and in steady state, see Chapter 5 At the end of this chapter we presented an overview of optimal limit interarrival times for a broad range of coefficients of variation in the form a figure This figure gives practitioners in services and healthcare a guideline to optimize their schedules In Chapter 6 we tested the performance of the schedules in a healthcare setting by Monte Carlo simulation For two typical distributions seen in healthcare data we computed the optimal limit interarrival times and the corresponding losses We concluded that in all four cases the optimal schedules based on the phase-type fit performs significantly better than schedules based on the assumption of exponential service times Moreover, our method performs best in optimizing simultaneously, the extra loss incurred by the approximation was at most 5% A suggestion for future work is to study the problem where we relax some of the model assumptions For example we can incorporate unpunctuality of patients, include no-shows 59

60 CHAPTER 7 SUMMARY AND SUGGESTIONS FOR FUTURE WORK and include walk-in patients This makes the optimization problem considerably harder to solve One can also work on deriving optimal limit interarrival times for other phase-type distributions than the mixture Erlang and hyperexponential distribution We already mentioned in Section 33 that the Coxian distribution is sometimes used to fit data with a CV Indeed, the mixture Erlang and hyperexponential can be seen as specific cases of Coxian distributions Finally, one can approximate general service time distributions better by also matching higher moments In fact we can approximate any distribution arbitrarily accurately with phase-type distributions as we saw in Section 3 We used a phase-type fit based on only the first two moments, which gave us decent results The question is whether incorporating more moments and the resulting efficiency gain will outweigh the increased complexity and computation time

Bibliography [] I Adan and J Resing Queueing theory http://wwwwintuenl/~iadan/queueing pdf, 00 [] S Asmussen Applied probability and queues, volume 5 Springer-Verlag, New York, NY, USA, applications of mathematics edition, 003 Stochastic Modelling and Applied Probability [3] S Asmussen, O Nerman, and M Olsson Fitting phase-type distributions via the em algorithm Scandinavion Journal of Statistics, 3(4):49 44, 996 [4] M Babes and GV Sarma Out-patient queues at the ibn-rochd healt centre The Journal of the Operational Research Society, 4(0):845 855, 99 [5] JD Welch NTJ Bailey Appointment systems in hospital outpatient departments The Lancet, 59(678):05 08, 95 [6] NTJ Bailey A study of queues and appointment systems in hospital out-patient departments, with special reference to waiting-times Journal of the Royal Statistical Society Series B (Methodological), 4():85 99, 95 [7] NTJ Bailey Queueing for medical care Journal of the Royal Statistical Society Series C (Applied Statistics), 3(3):37 45, 954 [8] M Brahimi and DJ Worthington Queueing models for out-patient appointment systems - a case study The Journal of the Operational Research Society, 4(9):733 746, 99 [9] C Ho and H Lau Minimizing total cost in scheduling outpatient appointments Management Science, 38():750 764, 99 [0] T Cayirli and E Veral Outpatient scheduling in health care: a review of literature Production and Operations Management Society, (4):59 549, 003 [] TF Cox, JF Birchall, and H Wong Optimizing the queuing system for an ear, nose and throat outpatien clinic Journal of Applied Statistics, :3 6, 985 [] B Denton and D Gupta A sequential bouding approach for optimal appointment scheduling IIE Transactions, 35:003 06, 003 [3] W Feller An introduction to probability theory and its applications Vol II John Wiley & Sons Ltd, New York, NY, USA, edition, 967 6

6 BIBLIOGRAPHY [4] BE Fries and VP Marathe Determintaion of optimal variable-sized multiple-block appointment systems Operations Research, 9():34 345, 98 [5] GC Kaandorp and G Koole Optimal outpatient appointment scheduling Health Care Management Science, 0(3):7 9, 007 [6] B Kemper, CAJ Klaassen, and M Mandjes Optimized appointment scheduling Unpublished, 0 [7] D G Kendall Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded markov chain Annals Mathematical Statistics, 4(3):338 354, 953 [8] C Liao, CD Pegen, and M Rosenshine Planning timely arrivals to a stochastic production or service system IIE Transactions, 5(5):63 73, 993 [9] DV Lindley The theory of queues with a single server Mathematical Proceedings of the Cambridge Philosophical Society, 48():77 89, 95 [0] EJ Resing, R Baron, and B Averill A system analysis of a university-health-service outpatient clinic Operations Research, (5):030 047, 973 [] JR Swisher, SH Jacobsen, JB Jun, and O Balci Modeling and analyzing a physician clinc environment using discrete-event (visual) simulation Computers & Operations Research, 8():05 5, 00 [] HC Tijms Stochastic models and analysis - a computational approach John Wiley & Sons Ltd, Chichester, UK, 986 [3] HC Tijms Stochastic models an algorithmic approach John Wiley & Sons Ltd, Chichester, UK, 994 [4] W Vink Utility-based appointment scheduling in continuous time: The lag order approximation method Master s thesis, University Utrecht, 0 [5] W Vink, B Kemper, and S Bhulai Utility-based appointment scheduling in continuous time: the lag order approximation method Unpublished, 0 [6] PP Wang Optimally scheduling n customer arrival times for a single-server system Computer Operations Research, 4(8):703 76, 997 [7] PP Wang Sequencing and scheduling n customers for a stochastic server European Journal of Operational Research, 9:79 738, 997 [8] JD Welch Appointment systems in hospital outpatient departments Operational Research Quaterly, 5(3):4 3, 964