Mean-field Dynamics of Load-Balancing Networks with General Service Distributions

Transcription

1 Mean-fied Dynamics of Load-Baancing Networks with Genera Service Distributions Reza Aghajani 1, Xingjie Li 2, and Kavita Ramanan 1 1 Division of Appied Mathematics, Brown University, Providence, RI, USA. 2 Department of Mathematics and Statistics, University of North Caroina Charotte, Charotte, NC, USA. December 2, 215 Abstract We introduce a genera framework for the mean-fied anaysis of arge-scae oad-baancing networks with genera service distributions. Specificay, we consider a parae server network that consists of N queues and operates under the SQ(d) oad baancing poicy, wherein jobs have independent and identica service requirements and each incoming job is routed on arriva to the shortest of d queues that are samped uniformy at random from N queues. We introduce a nove state representation and, for a arge cass of arriva processes, incuding renewa and time-inhomogeneous Poisson arrivas, and mid assumptions on the service distribution, show that the mean-fied imit, as N, of the state can be characterized as the unique soution of a sequence of couped partia integro-differentia equations, which we refer to as the hydrodynamic PDE. We use a numerica scheme to sove the PDE to obtain approximations to the dynamics of arge networks and demonstrate the efficacy of these approximations using Monte Caro simuations. We aso iustrate how the PDE can be used to gain insight into network performance. 1 Introduction Load baancing is an effective method to improve the performance and reiabiity of networks by optimizing resource use. With growth in the use of server farms and computer custer, arge-scae oad baancing networks appear in variety of appications, incuding internet services such as high-traffic web sites, high-bandwidth Fie Transfer Protoco sites, Network News Transfer Protoco (NNTP) servers, Domain Name System (DNS) servers, and databases, as we as coud computing and communications systems. An extensivey studied probem is the design and anaysis of oad-baancing agorithms that aim to improve network performance. This is particuary chaenging for arge-scae networks, where it is not feasibe to impement cassica agorithms ike join-the-shortest-queue, which incur high communication overhead and computationa cost. In this context, randomized agorithms provide an attractive aternative. A popuar agorithm that achieves a better baance between network performance and communication overhead is the so-caed SQ(d) (or supermarket ) agorithm. This agorithm was introduced reza@brown.edu xi47@uncc.edu kavita ramanan@brown.edu 1

2 in the case d = 2 by Vydenskaya et a. in [23] in the simpe setting of a network comprising N homogeneous parae servers, each with its own queue, that process a common stream of jobs that must be routed immediatey on arriva. In the SQ(d) agorithm, upon arriva of a job, d queues are samped independenty and uniformy at random, and the job is routed to the shortest queue amongst those samped. Athough, when d 2 and λ < 1, the stationary distribution of a typica queue is not computabe, the imiting stationary distribution, as N, was expicity computed in [23] and shown to have a doube exponentia tai decay, in contrast to the exponentia decay when d = 1 (which corresponds to random routing). This dramatic improvement in performance gained by adding just one extra random choice is known as the power of two choices, and has ed to substantia interest in this cass of randomized oad baancing schemes. The extension to genera d > 2 was studied by Mitzenmacher [21], and a static bas-and-bins anaog was originay studied in Azar et a. [3]. The anaysis of the SQ(d) mode in the case of exponentia service times is carried out using the so-caed ODE method. This proceeds by first representing the dynamics of the N-server network by a Markov process S (N) = (S (N) ; 1), where S (N) (t) represents the number of queues that have or more jobs at time t, and then showing that, as N, the sequence of suitaby scaed Markov processes converges weaky (on finite time intervas) to the unique soution of a countabe system of couped [, 1]-vaued ordinary differentia equations (ODEs). This imit resut is obtained by a simpe appication of Kurtz s theorem (see Theorem in [11]), generaized to countabe state spaces. Insight into the equiibrium behavior is obtained by first showing that, under the stabiity condition λ < 1, each Markov process S (N) is ergodic, then characterizing the imit of the sequence of scaed stationary distributions as the unique equiibrium point of this system of ODEs [23], and finay expicity identifying this equiibrium point. However, in most rea-word appications, service times are typicay not exponentiay distributed. For exampe, statistica anayses suggest that service times foow a Log-Norma distribution in (mediumscae) ca centers [7], a Gamma distribution in Automatic Teer Machines [16], and studies of content downoad in Amazon S3 suggest that downoad times foow a shifted exponentia distribution [8, 17]. Moreover, Phase-type distributions are used to approximate more compicated service time distributions [1, 22]. In the case of genera service times, in order to describe the evoution of the system it is not sufficient to keep track of the fraction of queues with jobs at any time. For each job in service, one has aso to keep track of its age (the amount of time the job has spent in service) or its residua service time. In the system with N servers, this requires keeping track of N additiona random variabes, and thus the dimension of the Markovian state representation grows with N, which is not convenient for obtaining a imit theorem. 1.1 Prior Work The supermarket mode and its various modifications have been extensivey studied for the case of exponentia service distributions. The path-space evoution of the supermarket mode was studied by Graham [13], the maximum equiibrium queue ength was anayzed in [18], and strong approximations were obtained in [19]. The basic mode has aso been generaized to incorporate features of reevance in appications such as oad migration, oad steaing and threshods (see, e.g., [12], [2] and [15]). Athough resuts on the exponentia service time mode were obtained amost two decades ago, to the best of our knowedge, there appears to be no prior work that characterizes the transient behavior of the supermarket mode with genera service time distributions, and unti recenty, there was amost no work on the equiibrium distribution under the sub-criticaity condition λ < 1. Recent progress on the equiibrium behavior was made in a nice series of papers by Bramson et a. [4, 5, 6], using the so-caed cavity method. In particuar, in [6], in was shown that for the sub-cass of power aw distributions with exponent β with β > 1, the imiting stationary distribution has a douby exponentia tai if β > 2

3 d/(d 1), an exponentia tai if β = d/(d 1) and a power aw tai if β < d/(d 1). However, it shoud be noted that the tais of the imiting stationary distribution provide ony imited information about the stationary distribution of the N-server system because imits do not interchange, that is, the tai of the imiting stationary distribution is not the imit of the tais of the N-server stationary distributions. Moreover, the resuts in [6] assume that the cumuative arrivas are Poisson and the service distribution has a decreasing hazard rate function. According to the authors [6, Page 3], extending their resuts beyond these assumptions using their framework appears to be a difficut probem. 1.2 Our Contributions In this paper, we introduce a new framework for the study of oad baancing networks with genera service time distributions. The work combines stochastic modeing (choosing a suitabe state representation), with techniques from probabiity (convergence resuts), PDEs (anaysis of the imit), numerica anaysis (stabe schemes for soving the PDE) and simuations (vaidation) to provide engineering insights into the performance of the network under the SQ(2) oad baancing agorithm in the presence of genera service times. Our specific contributions incude the foowing: 1. Deveopment of the PDE method for anayzing oad-baancing networks with genera service distributions, which can be viewed as a generaization of the we known ODE method [23, 21] used to study arge-scae networks with exponentia service distributions. Our method appies to the arge cass of service distributions that have a density and finite mean, incuding Pareto, Log-Norma, Gamma and Phase type distributions. In particuar, it does not require the decreasing hazard rate function assumption imposed in [6]. 2. The PDE can be used to approximate not ony the queue ength distribution but aso other quaity of service (QoS) parameters such as the virtua waiting time. 3. In contrast with previous work, our framework aso enabes the study of transient behavior of oad-baancing networks with genera service distributions, in time-homogeneous networks as we as networks with time-inhomogeneous (Poisson) arrivas, both of which seem reevant for rea-word appications. 4. The stabe numerica scheme that we use to approximate the PDE provides a computationay efficient aternative to Monte Caro simuations that coud be usefu for studying the performance and design of arge networks, as iustrated in Sections 6 and Athough, it is known that networks with heavy-taied service distributions have worse steadystate performances [6], using our method, we identified the somewhat surprising phenomenon that they can ead to better performance with respect to some transient QoS parameters, as detaied in Section 7. The rest of the paper is organized as foows. The structure and representation of the oad-baancing network are described in Section 2. The hydrodynamic PDE is presented in Section 3. Section 4 contains the main resuts of the paper, and their proofs are given in Section 5. In Section 6, we present the numerica scheme to sove the PDE and vaidate the main resuts using Monte Caro simuations. Engineering insights gained by the PDE are iustrated in Section 7. Finay, concusions and future works are discussed in Section 8. 3

4 E(t) 2 A Load Baancing Network 2.1 Mode Description Figure 1: The oad-baancing network. Consider a network of N homogeneous parae servers, each with its own infinite capacity queue, that processes a common stream of arriving jobs (see Figure 1) that are routed immediatey on arriva according to a oad-baancing agorithm. Each server foows a first-in-first-out (FIFO) service poicy, and is non-iding in the sense that it cannot be ide if there is a job waiting in the queue. Hence, if a job is routed to an ide server, it immediatey starts receiving service, and otherwise, it is paced at the end of the queue. We index servers by i = 1,..., N. Let E (N) be the cumuative arriva process, that is, for every t, E (N) (t) is the number of job arrivas during the interva [, t]. Jobs are indexed by j Z, and each job j has a service time requirement of v j. The sequence of service times {v j } j Z is assumed to be i.i.d., independent of the arriva process, and distributed according to a genera distribution function G. Let G represent the compementary service distribution, i.e., G(x) = 1 G(x). For each server i and at each time t, the number of jobs in server i (incuding the one receiving service and the ones waiting in the queue) is caed the queue ength of server i at time t, and is denoted by X (N),i (t). The tota number of jobs in system is denoted by X (N) (t), whence X (N) (t). = N i=1 X(N),i (t). We study the performance of the network described above under a randomized oad baancing agorithm that we refer to as the SQ(d) agorithm. Based on this agorithm, upon arriva of a job, d queues are samped independenty and uniformy at random, and the job is then routed to the queue with minimum ength (with ties broken uniformy at random). In this paper, we present our resuts for the case d = 2; however, the extension for d 2 is amost immediate. 2.2 Assumptions The foowing assumptions are imposed on the cumuative arriva process and service time distribution. Assumption I. (Arriva Process) For every N N, E (N) is a time-inhomogeneous Poisson process with rate λ (N) ( ). = a (N) λ( ), where λ( ) is a non-negative, ocay square integrabe function (that is, T λ(t)dt < for every T < ) and {a(n) } N N is a sequence of positive rea numbers satisfying a (N) /N 1 as N. Remark 2.1. Assumption I is satisfied by most interesting cases incuding the heavy traffic imit λ N ( ) = Nλ( ) β N. However, as shown in forthcoming work, the resuts of this paper hod for more genera sequences of arriva rates. 4

5 Reca that when a distribution G has a density g, the hazard rate function h associated with that distribution is defined by h(x) =. g(x), x [, L), (1) 1 G(x) where L. = sup{x [, ) : G(x) < 1}. Assumption II. (Service Distribution) The distribution function G has a finite mean equa to 1, and density g. Aso, the hazard rate h is uniformy bounded by a constant H, that is, sup h(x) H. x [,L) Remark 2.2. Assumption II is satisfied by a wide cass of service time distributions, incuding any Pareto distribution with finite mean, Phase-Type distributions, Log-Norma distributions and Gamma distributions. Athough the boundedness assumption on h can be reaxed, it makes the representation of the resuts in Sections 4 consideraby easier. Note that since for every a, b [, L), b a h(x)dx = n(g(a)) n(g(b)), h is aways ocay integrabe on [, L), but not integrabe. Therefore, Assumption II impies that L =, that is, the distribution G is supported on the whoe [, ). 2.3 An Important State Descriptor For every 1 and time t, et S (N) (t) denote the subset of a servers with queue ength at east at time t, that is, S (N) (t) =. { } i = 1,..., N : X (N),i (t), (2) and denote by S (N) the number of such servers, that is, S (N) (t) =. { } # i = 1,..., N : X (N),i (t) = #S (N) (t). (3) Note that S (N) 1 (t) is the set of servers with at east one job, to which we refer as busy servers. As each server uses a FIFO poicy and is non-iding, there is a unique job being served at each busy server. Hence, at each time t, we can assign to every busy server i an age, denoted by a (N),i (t), which is defined to be the amount of time that the current job has been receiving service. As discussed in the introduction, if the service times are exponentiay distributed, the process {(S (N) (t); 1); t } is a convenient state descriptor for the N-server network. But, since other service distributions do not have the memoryess property (exponentia is unique), the dynamics of the network depends on the ages of a jobs in service. Hence, in the case of genera serviced distributions, the foowing state descriptor turns out to be convenient. For every N N, 1 and t, define the functions Z (N) (t, ) as Z (N) (t, r). = i S (N) G(a (N),i (t) + r), r, (4) G(a (N),i (t)) where note that the summation is over servers at queues with ength at east. For an intuitive understanding of this quantity, note that for any N N, the conditiona probabiity that a job which is being 5

6 served at time t in a busy server i wi sti be in system at time t + r for r, is G(a (N),i (t) + r)/g(a (N),i (t)). Therefore, Z (N) (t, r) is the expected number of jobs that are being served at time t in a server with queue ength at east and that wi sti be in the system at time t + r. Note in particuar that Z (N) (t, ) is equa to the cardinaity of S (N) (t), that is 3 A PDE Approximation Z (N) (t, ) = S (N) (t). (5) In this section, we introduce a set of partia integro-differentia equations that we ca the hydrodynamic PDE. As shown in Section 4, these couped equations uniquey characterize the imit, as N, of the scaed state descriptors Z (N) defined as Z (N) (t, r) =. 1 N Z(N) (t, r). The equations are described in Section 3.1. Then, in Section 3.2, we show how these set of PDEs reduce to the set of ODEs obtained in [23] when the service distribution is exponentia. Finay, we provide an intuitive interpretation of the hydrodynamic PDE in Section Description of the Hydrodynamic PDE Define C 1 1 [, ) to be the space of continuousy differentiabe functions f with bounded derivative, such that f is bounded by 1. Aso, define X to be the set of functions ϕ on [, ) [, ) such that for every t, ϕ(t, ) C 1 1 [, ) with rϕ(t, ) denoting the derivative with respect to the second variabe, and for every r, the mapping t ϕ(t, r) is measurabe. Now we define the PDE. Definition 3.1. For every function λ and set of functions Z C 1 1 [, ), 1, a set of functions (Z ; 1) with Z X for a 1 is said to sove the hydrodynamic PDE associated to (λ, Z ; 1) if for every t, r, Z 1 satisfies Z 1 (t, r) = Z1 (t + r) G(t + r u) r Z 2 (u, )du + λ(u)g(t + r u)(1 Z 1 (u, ) 2 )du, (6) with the boundary condition Z 1 (t, ) = Z1 () + ( r Z 1 (u, ) r Z 2 (u, ) ) du + λ(u) ( 1 Z 1 (u, ) 2) du, (7) and for 2, Z satisfies Z (t, r) = Z (t + r) G(t + r u) r Z +1 (u, )du (8) + λ(u)(z 1 (u, ) + Z (u, ))(Z 1 (u, t + r u) Z (u, t + r u))du, with the boundary conditions Z (t, ) = Z () + ( r Z (u, ) r Z +1 (u, ) ) du + λ(u) ( Z 1 (u, ) 2 Z (u, ) 2) du. (9) Note that by equations (6) and (8), Z is the initia condition for Z, that is Z (, r) = Z (r), for a 1 and r. Remark 3.2. Athough the equations (6)-(9) are partia integro-differentia equations and not partia differentia equations in the cassic sense, we sti refer to them as hydrodynamic PDE for conciseness. 6

7 3.2 Reduction to ODE in the Exponentia Case To better iustrate the hydrodynamic PDE, we show that when the service distribution is exponentia and the arriva rate λ( ) λ is constant, it reduces to the set of ordinary differentia equations (ODEs) obtained in [23, 21]. Substituting the compimentary CDF G(x) = e x for the exponentia distribution in definition (4) of Z (N), we have Z (N) e (t, r) = a(n),i (t) r i S (N) e a(n),i (t) = e r S (N) (t). As made rigorous in Theorem 4.2, this suggests the imit satisfies Z (t, r) = e r S (t), with S equa to the imit of S (N) /N. Substituting G(r) = e r, Z and its derivative r Z (t, r) = e r S (t) in (8), for every 2 we obtain S (t) = e t S () + e (t u) S +1 (u)du + λ e (t u) (S 1 (u) 2 S (u) 2 )du. (1) Taking derivatives with respect to t of both sides of the equation above yieds d dt S (t) = e t S () + S +1 (t) + λ e (t u) ( S 1 (t) 2 S (u) 2) du. e (t u) S +1 (u)du λ ( S 1 (t) 2 S (t) 2) Using (1) to eiminate the third and fifth terms on the right-hand side of the ast equation, we obtain d dt S (t) = (S (t) S +1 (t)) + λ(s 1 (t) 2 S (t) 2 ). (11) Exacty anaogous cacuations can be shown to see that equations (6) d dt S 1(t) = (S 1 (t) S 2 (t)) + λ(1 S 1 (t) 2 ). (12) Simiary, substituting G and Z in (8), we obtain and, again by taking derivative, we have Moreover, substituting Z (t, ) = S (t) and r Z (t, r) = S (t) in (7) and (9), we again obtain (12) and (11), respectivey. Note that the ODEs (12)-(11) coincide with the equations that were previousy obtained, e.g., (1) in [21]. 3.3 Interpretation of the Hydrodynamic PDE Existence and uniqueness of soutions to the PDE are estabished in Theorems 4.2 and 4.1. Here, we first provide a heuristic expanation of each term in the hydrodynamic PDE (6)-(9). Interpreting Z as the imit of the sequence {Z (N) /N} defined in (4), note that Z (t, r) represents the (imit of the) fraction of jobs that were in service at time t at a queue of ength at east and that were sti in service at time t + r. There are three sources that contribute to Z (t, r), corresponding to the three terms on the right-hand side of (8). The first is the expected fraction of jobs that were aready in service at time at a queue of ength at east and that wi sti be in service at time t + r, which is given by the first term on the right-hand side of (8), namey Z (, t + r) = Z (t + r). 7

8 The second source accounts for any job that entered service at some time u [, t] due to competion of the job ahead of it in the queue, and that woud sti not have competed service at time t + r (equivaenty, jobs that have a service time greater than t + r u). For such a job to be in a queue of ength or more at time t, it must have been a queue of ength + 1 just prior to the departure of the job ahead of it. Now, to estimate the expected rate of entry into service of such jobs, consider a server that is busy serving a job j with age a(u) at time u. Given this information, the probabiity that the job wi depart within the next ɛ units of time is equa to P { v j < a(u) + ɛ v j > a(u) } = G(a(u) + ɛ) G(a(u)) 1 G(a(u)) h(a(u))ɛ. Therefore, the expected departure rate of jobs from servers with queues of ength + 1 or greater (conditiona on their ages) is roughy h(a (N),i (u)), i S (N) +1 (u) which can be rewritten as i S (N) +1 (u) g(a (N),i (u)) 1 G(a (N),i (u)) = rz (N) +1 (u, ). Thus, r Z (u, ) is (the imit of) the expected departure rate at time u from servers with a queue of ength at east + 1, which is aso the expected entry rate into service into a queue of ength. Mutipying this by G(t + r u), the fraction of such jobs that wi sti be in service at time t + r, and integrating over a possibe vaues of u [, t], we obtain the second term on the right-hand side of (8). The ast contribution is due to the routing of new jobs at some time u [, t] to a queue of ength 1 such that the job in service at that queue at time u is sti in service at time t + r. Note that such a queue has ength or more at time t and the job in service at that queue in time u is in service both at times t and t + r, and hence contributes to Z (t, t + r). To compute the number of such jobs, first note that the tota arriva rate of jobs at time u is λ(u), and the expected fraction of such jobs that get routed to a queue of ength 1 under the SQ(2) routing agorithm is computed in Section 5.3 and is equa to S (N) 1(u) 2 S (N) (u) 2, which by (5) is aso equa to Z (N) 1 (u, )2 Z (N) (u, ) 2 (here, we use the convention S (N) prior to time u is Z (N) 1 t + r is Z (N) 1 1.) Now, the tota number of jobs in service at a queue of exacty ength 1 just (u, ) Z(N) (u, ) and the number of these that wi sti be in service at time (u, t + r u) Z(N) (u, t + r u), which impies the fraction of such jobs sti in service at time t + r is given by the ratio. Mutipying the ratio by the previousy computed arriva rate of jobs into queues of ength 1 at time u and integrating over u [, t], we obtain the third term. The boundary conditions (7) and (9) are basicay mass baance equations. Reca that Z (N) (t, ) is the number of jobs receiving service at a queue of ength at east at time t, and simiary, Z (N) (, ) is the number of jobs that were receiving service at time at such a queue. Note that S (N) = Z (N) (, ) decreases ony due to departures from servers with queue ength equa to and increases ony due to arrivas routed to servers with queue ength equa to 1. As discussed above, r Z (N) +1 (u, r) is the departures rate at time u of jobs from a queue of ength + 1 or more at the time of departure, and hence the second term on the right-hand side of (9) represents the imit of the cumuative number of such departures in the interva [, t] from queues of ength exacty. Finay, the third term on the righthand side of (9) represents the tota number of arrivas to servers at queues of ength exacty 1 in the interva [, t], whose form can be deduced from the routing probabiities computed above. 8

9 4 Main Resuts In this section we state the main resuts of the paper; the proofs are deferred to Section 5. First in Section 4.1, we show that the PDE (6)-(9) have at most one soution in a suitabe space. Then in Section 4.2, we show that under suitabe assumptions on the sequence of initia distributions, the scaed sequence } N N has a imit which satisfies the PDE (and hence, is the unique soution). Using this convergence resut, we show in Theorem 4.3 and Theorem 4.4 that the queue ength distribution of a typica queue and the mean virtua waiting time in the N-server network converges to certain functionas of the soution to the hydrodynamic PDE. Moreover, we estabish a propagation of chaos resut, showing that, at any finite time, the queue ength distributions of any finite set of queues are asymptoticay independent. {Z (N) 4.1 Uniqueness of the Soution to the PDE We now state our first resut. We denote by L 1 oc (, ) the space of ocay integrabe functions (that is, functions f on [, ) that satisfy T f (t)dt < for every T < ). Theorem 4.1. Suppose Assumptions I-II hod. Then, for every non-negative function λ L 1 oc (, ) and Z C1 1 [, ), 1, then the hydrodynamic PDE (6)-(9) has at most one soution. 4.2 Convergence of the State Descriptor Z (N) Theorem 4.2 beow shows that the soution to the hydrodynamic PDE characterizes the imit of Z (N) as N gets arge. To state the theorem, we need to impose an additiona assumption on the initia conditions. Assumption III. (Initia Condition) a. Amost surey, for every 1 and every function f C b [, ), the imit im N 1 N exist. Moreover, im sup N E[X (N) ()] <. f (a (N),i ()). i S (N) () b. For every N, the initia conditions of the N-server network are exchangeabe, in the sense that for every permutation σ on the set of server indices {1,..., N}, the distribution of the vector ( ) X (N),i (), 1(X (N),i () > )a (N),i (); i = 1,..., N does not depend on the choice of σ. Note that in particuar, substituting f = G( + r)/g( ) in Assumption III.a shows that for every r, the foowing imit exists: Z (r) =. im Z (N) (, r). (13) N This theorem is estabished in a companion paper [1]. 9

10 Theorem 4.2. Suppose λ and G satisfy Assumptions I and II, et Assumption III.a hod and et Z, 1, be as in (13). Then amost surey, for 1 and t, r, the imit Z (t, r). = im N Z (N) (t, r), (14) exists and is the unique soution of the hydrodynamic PDE (6)-(9) associated to (λ, Z, 1). if Assumption III.b aso hods and then for every t we have Typica Queue Length Distribution sup P{X (N),i () } <, (15) N [ sup E Z (N) ] (t, ) <. (16) N As stated beow, Theorem 4.2 aso aows us to characterize the distribution of a typica queue in the network and estabish an asymptotic independence resut. Theorem 4.3. Suppose λ, G, Z are as in Theorem 4.2, and et (Z ; 1) be the unique soution to the hydrodynamic PDE associated to (λ, Z ; 1). Then, for every 1 and t, { } im P X (N),1 (t) = Z (t, ). (17) N and the queue ength of different servers are asymptoticay independent, that is, for every t, K 1 and 1,..., K 1, { } im P X (N),1 (t) 1,..., X (N),K K (t) K = Z k (t, ). (18) N k=1 4.4 Convergence of Virtua Waiting Times The virtua waiting time at any time t is the time that a virtua customer that hypotheticay arrives at t has to wait in order to receive service. As another appication of Theorem 4.2, our next resut shows that for arge N, the hydrodynamic PDE aso provides an approximation to the mean virtua waiting time in an N-server network. Theorem 4.4. Suppose λ, G, Z are as in Theorem 4.2, and et (Z ; 1) be the unique soution to the hydrodynamic PDE associated to (λ, Z ; 1). If, in addition, the initia queue engths satisfy (15) and a (N),i () < T for some T < and every i = 1,..., N, then [ ] im E W (N) (t) [Z (t, r) Z +1 (t, r)] dr. (19) N = Z (t, ) 2 + [Z (t, ) + Z +1 (t, )] 2 1 Remark 4.5. The uniform boundedness assumption on the initia ages is reasonabe for transient anaysis since it wi be satisfied by any network that started empty a finite time interva ago. However, the assumption can be reaxed, as iustrated in Figure 3(b) in Section We impose it ony to simpify the proofs. Remark 4.6. We can show that in fact, as N, the sequence of virtua waiting time distributions (and not just their means) converge to a imit distribution whose characteristic function can be expressed as a functiona of the soution to the hydrodynamic PDE. We do not present the detais of the proof, but it is simiar to that of Theorem

11 5 Proofs of Main Theorems Now we prove the main Theorems of the paper stated in Section 4. The uniqueness resut of Theorem 4.1 is first proved in Section 5.1. The proof of Theorem 4.2 is rather technica, and requires estabishing a imit theorem for a sequence of interactive measure-vaued processes describing the N-server network, which is carried out in a companion paper [1]. Theorem 4.3 is proved in Section 5.2. Then in section 5.3, we carry out the cacuation to compute the so-caed routing probabiities corresponding to the SQ(d) agorithm, which is required for the proof of Theorem 4.4. We have singed out this cacuation because it is the ony part of the proof which depends on the particuar oad-baancing agorithm invoked in the network. The proof of Theorem 4.4 is then given in Section Proof of the Uniqueness Theorem Throughout this section, we adopt the foowing notation. For every function f in [, ), that is bounded on finite intervas and every T, we denote. f T = sup f (t). t T Aso, for functions f 1, f 2 on [, ), f 1 f 2 (t). = f 1(s) f 2 (t s)ds is the (one-sided) convoution of f 1 and f 2. Proof of Theorem 4.1. Fix a non-negative function λ L 1 oc [, ) and Z = (Z ; 1), and et Z = (Z ; 1) and Z = ( Z ; 1) both be soutions to the hydrodynamic PDE (6)-(9) associated to (λ, Z ). Defining the functions D (t) =. r Z (s, )ds, t, and using integration by parts, we can rewrite G(t + r s) r Z +1 (s, )ds = G(r)D +1 (t) + D +1 (s)g(t + r s)ds. (2) Substituting (2) into (6) and (8), and the definition of D into (7) and (9), we see that for every 1, Z satisfies for t, r, Z (t, r) = Z (r + t) + G(r)D +1(t) + R (t, r) D +1 (s)g(t + r s)ds with boundary condition Z (t, ) = Z () D (t) + D +1 (t) + Λ (t), where t R (t) = λ(s)g(t + r s)( 1 Z1 2(s, )) ds if = 1, λ(s)( Z 1 (s, ) Z (s, ) )( Z 1 (s, t + r s) Z (s, t + r s) ) ds if 2, and Λ (t) =. t λ(s)( 1 Z1 2(s, )) ds if = 1, λ(s)( Z 1 2 (s, ) Z2 (s, )) ds if 2. (21) (22) 11

12 Simiary, Z satisfies anaogous equations. Defining H. = H H for H = Z, D, R, Λ and 1, for a 1 and t, r we have with boundary condition Z (t, r) = G(r) D +1 (t) + R (t, r) D +1 (s)g(t + r s)ds, (23) Now for every t and 1, define Z (t, ) = D (t) + D +1 (t) + Λ (t). (24) V (t) =. sup Z (t, r). (25) r Note that V is measurabe because it is the supremum over measurabe functions, and is bounded by 2 because Z (t, ) and Z (t, ) are both in C 1 1 [, ). By definition (21) of R 1, we have, R 1 (t, r) = λ(s)g(t + r s) ( Z 1 (s, ) + Z 1 (s, ) ) Z 1 (s, )ds, and hence, R 1 (t, r) 2 λ(s)v 1 (s)ds, t, r. (26) Simiary for 2, by definition (21) of R, R (t, r) = + and hence, for a t, r with λ(s) ( Z 1 (s, ) + Z (s, ) )( Z 1 (s, t + r s) Z (s, t + r s) ) ds λ(s) ( Z 1 (s, ) + Z (s, ) )( Z 1 (s, t + r s) Z 1 (s, t + r s) ) ds, R (t, r) 4 λ(s) ( V 1 (s) + V (s) ) ds. (27) Furthermore, to bound D for 2, we substitute D +1 from (24) in (23) and set r = to concude that D satisfies the renewa equation Fix 2 and note that by definition (22) of Λ, Λ (t) = and hence, D (t) = g D (t) + F (t), t, F (t). = Λ (t) g Λ (t) + g Z (t, ) R (t, ). λ(s) ( Z 1 (s, ) + Z 1 (s, ) ) Z 1 (s, )ds λ(s) ( Z (s, ) + Z (s, ) ) Z (s, )ds, Λ (t) 4 λ(s) ( V 1 (s) + V (s) ) ds, t. (28) Reca that V (t) 2 for a t and 1, and due to the oca integrabiity of λ, (28) impies that Λ (t) is uniformy bounded on any finite interva t [, T]. Hence, g Λ (t) Λ T g(s)ds <, t [, T]. 12

13 Simiary, the bound (27) shows that R (t) is aso bounded for finite t. Therefore, F is bounded on finite intervas, and hence by the renewa theorem (see Theorem V.2.4 in [2]) D (t) = F (t s)du G (s), where U G is the renewa measure corresponding to the service distribution G. Since G has a density g, U G satisfies x U G (x) = 1 + u G (y)dy, x, and the density u G is bounded on finite intervas and satisfies the equation u G = g u G + g due to Proposition V.2.7 in [2]. Therefore, D (t) can be written as D (t) = F (t) + u G F (t) = Λ (t) + u G Z (t, ) R (t, ) u G R (t, ). (29) For every fixed T and a t T, by the definition of the convoution operator and (25), u G Z (t, ) Aso, using the bound (27) we have u G R (t, ) 4 =4 u G (s) Z (t s, ) ds u G T V (s)ds. (3) 4U G (T) s u G (t s) λ(v) ( V 1 (v) + V (v) ) dv ds λ(v) ( V 1 (v) + V (v) ) u G (t s)ds dv v λ(s) ( V 1 (s) + V (s) ) ds. (31) Bounding the terms on the right-hand side of (29) using inequaities (27), (28), (3) and (31), for every 2 and t T, we have D t C T (1 + λ(s))(v 1 (s) + V (s))ds, (32) with C T. = 8 + 4U G (T) + u G T. Next, substituting the bound (32), but with repaced by + 1, (26) and (27) into (23), we obtain that for a t T and r, and for 2, Z 1 (t, r) 4(C T G(r) + 1) (1 + λ(s))(v 1 (s) + V 2 (s))ds, (33) Z (t, r) 4(C T G(r) + 1) (1 + λ(s))(v 1 + V (s) + V +1 (s))ds. (34) Taking supremum over r on both sides of (33) and (34), we have and for 2, V 1 (t) 8C T (1 + λ(s))(v 1 (s) + V 2 (s))ds, (35) V (t) 8C T (1 + λ(s))(v 1 (s) + V (s) + V +1 (s))ds. (36) 13

14 Now define V(t) =. 2 V (s). (37) 1 Note that V is measurabe, and V(t) = if and ony if V (t) = for a 1. Considering the weighted sums of both sides of (35) and (36) over 2, we obtain V(t) 26C T (1 + λ(s))v(s)ds, t T. (38) An appication of Gronwa s inequaity shows that V, and hence, Z = Z for a. This competes the proof. 5.2 Proof of Theorem 4.3 Proof of Theorem 4.3. First note that since the routing agorithm is symmetric with respect to queue indices, the queue engths and age distributions, which are initiay exchangeabe by III.b, remain exchangeabe at a finite times t. In particuar, the distribution of the vector (X (N),σ(i) (t); i = 1,..., N) is the same for every permutation σ on {1, 2,..., N}. Since Z (N) S (N) (t), we have [ [ E Z (N) ] N (t, ) = 1 ( ) ] N E 1 X (N),i (t) = 1 N i=1 N i=1 (t, ) = S (N) (t) and by definition (3) of { } { } P X (N),i (t) = P X (N),1 (t), (39) where the ast equaity is due to exchangeabiity. By Theorem 4.2, Z (N) (t, ) converges to Z (t, ) as N, amost surey, and since Z (N) (t, ) = S (N) (t) is bounded by 1, the convergence aso hods in expectation by the bounded convergence theorem. Therefore, (17) foows on taking the imit as N of both sides of (39). Simiary, for m = 1,..., n, since Z (N) m (t, ) = S (N) m (t) by (5) and S (N) (t) is defined by (3), we have [ ] [ n N E Z (N) m (t, ) = 1 N m=1 n E i 1 =1... N i n =1 ) ( ) 1 (X ] (N),i 1 (t) X (N),i n (t) n = 1 N N } N n... P {X (N),i 1 (t) 1, X (N),i n (t) n i 1 =1 i n =1 } = P {X (N),1 (t) 1,..., X (N),n (t) n, (4) where the ast equaity is again due to exchangeabiity. By another use of Theorem 4.2, n m=1 Z(N) m (t, ), converges to n m=1 Z m (t, ) as N, amost surey, and in expectation, using the bounded convergence theorem. Taking the imit as N of both sides of (4), we obtain (18). 5.3 Routing Probabiities For every server index i and queue ength, define the routing probabiity p(i, ; t) to be the conditiona probabiity, given the state of the network at time t, that the oad-baancing agorithm woud route a hypothetica job arriving at time t to server i, when its queue ength is. Defining the vector X (N) (t) of a queue engths and ages, X (N) (t) =. ( ) X (N),i (t), a (N),i (t); i = 1,..., N. (41) 14

15 and denoting by κ(t) the (random) index to which the virtua job arriving at time t is routed, the routing probabiity p(i, ; t) is defined by p(i,, t) =. ( ) { } 1 X (N),i (t) = P κ(t) = i X (N) (t). (42) Now we compute the routing probabiities p(i, ; t) for 1 and under the SQ(2) agorithm described in Section 2.1. If κ 1 and κ 2 are indices of queues chosen independenty and uniformy at random, then κ(t) is the index associated with the shorter queue (with ties being broken uniformy at random). The job is routed to a server of queue ength exacty if and ony if both κ 1 and κ 2 have queue engths at east, and at east one of them has queue ength. Given the vector of a queue engths in X (N) (t), this happens with probabiity (S (N) (t)) 2 (S (N) +1(t)) 2. Since a servers with queue ength equa to are equay ikey to be chosen and there are S (N) (t) S (N) (t) of them, for 1 we have +1 ( ) 1 p(i, ; t) = 1 X (N),i (t) = N ( S (N) (t) + S (N) +1(t) Remark 5.1. Note that the form of the routing probabiity for the SQ(d) agorithm is ceary the same as in the exponentia case. To appy our framework to other oad baancing agorithms, one woud have to repace the expression in (43) with the routing probabiities associated with that agorithm. 5.4 Proof of Theorem 4.4 Proof of Theorem 4.4. Suppose that the virtua job arriving at time t is routed to queue i, that is κ(t) = i. If the server i is ide (i.e., X (N),i (t) = ), the virtua waiting time is zero; otherwise if X (N),i (t) = for some 1, the virtua waiting time W (N) (t) is the sum of service times v j of jobs waiting in queue i pus the residua time b (N),i (t) of the job in service at server i at time t, that is, 1 (κ(t) = i) W (N) (t) = 1 v j + b (N),i (t). j=1 Summing over a possibe queue indices i and queue engths, we have where and W (N) 1 (t). = W (N) 2 (t). = ) (43) W (N) (t) = W (N) 1 (t) + W (N) 2 (t), (44) N i=1 N i=1 1 1 ( 1 κ(t) = i, X (N),i (t) = ) 1 j=1 v j, (45) ( ) 1 κ(t) = i, X (N),i (t) = b (N),i (t). (46) To compute the expectation of W (N) 1 (t), note that by Assumption II, the service times v j of jobs that are sti waiting in queues satisfy E[v j ] = 1 and are independent of a queue engths and ages at time t, as we as κ(t). Therefore, taking the conditiona expectation given X (N) (t) of both sides of (45) and using the definition (42) of p(i,, t), we have [ ] E W (N) 1 (t) X (N) (t) = N i=1 1 = ( 1) 1 { P κ(t) = i, X (N),i (t) = X (N) (t) N i=1 ( ) 1 X (N),i (t) = p(i, ; t). } 1 j=1 E [ v j ] 15

16 Taking expectations of both sides of the ast equation, substituting p(i, ; t) from (43), recaing that the number of servers with queue ength equa to is S (N) Z (N) (t, ) from (5), we see that E[W (N) 1 (t)] = 1 N E [ ( 1) 1 ( = 1 N 1( [( 1)E S (N) S (N) [ ( = ( 1)E Z (N) 1 = E 2 (t) S (N) +1 (t), and using the equaity S(N) (t) = (t) + S (N) ) N ( ) ] +1(t) 1 X (N),i (t) = i=1 (t) + S +1(t) (N) ) ( ) 2 ( (t, ) Z (N) S (N) +1 (t, ) ) 2 ] (t) S (N) +1 (t) )] [ ( Z (N) ) ] 2 (t, ). (47) To compute the expectation of W (N) 2 (t), note that according to the SQ(2) agorithm, the random queue indices κ 1 and κ 2 are independent of a other random variabes, and therefore, given X (N) (t), κ(t) is independent of a residua service times. Therefore, taking conditiona expectations of both sides of (46) and invoking the definition (42) of p(i, ; t) again, we have [ ] E W (N) 2 (t) X (N) (t) = N 1 i=1 ( ) [ ] 1 X (N),i (t) = p(i, ; t)e b (N),i (t) X (N) (t). (48) The residua service time b (N),i (t) of the job that begins being processed by server i at time t satisfies b (N),i (t) = v J(i;t) a (N),i (t), where J(i; t) is the index of the job begin processes in server i at time t. It foows from the i.i.d assumption on the service times and their independence from the arriva process (Assumption II) that the residua service time b (N),i (t) depends on the state variabe X (N) (t) ony through the age a (N),i (t). A compete rigorous justification of this intuitive assertion is rather ong and technica, and hence wi be presented esewhere. Hence using equation (3.2) of Section V.3 in [2] in the second equaity beow, for every r, we have and therefore, { } { } P b (N),i (t) > r X (N) (t) = P b (N),i (t) > r a (N),i (t) = G(a(N),i (t) + r), G(a (N),i (t)) [ ] E b (N),i (t) X (N) G(a (t) = (N),i (t) + r) dr. (49) G(a (N),i (t)) Now, substituting equations (43) and (49) in (48), taking expectations of both sides, and using definition (4) of Z (N) and equation (5), we can see that [ ] E[W (N) 1 ( 2 (t)] = N E S (N) (t) + S +1(t) (N) ) N ( ) G(a 1 X (N),i (t) = (N),i (t) + r) dx 1 i=1 G(a (N),i (t)) [ ( = E Z (N) (t, ) + Z (N) ) ( +1 (t, ) Z (N) (t, r) Z (N) ) ] +1 (t, r) dr. (5) 1 16

17 Finay, taking the imit as N in (5), we obtain [ ] im E W (N) N 1 (t) = 2 [ im E N Z (N) (t, ) 2] = (Z (t, )) 2. (51) 2 The exchange of imit and summation in the first equaity is justified by the bound (16), Z (N) (t, ) 1 and the dominated convergence theorem, whie the second equaity foows from Theorem 4.2 and the bounded convergence theorem. Moreover, by the uniform boundedness assumption on the ages imposed in Theorem (4.4), a (N),i (t) < T + t. Therefore, since G(r)dr = 1 because the service time has mean 1, and by definition (4) of Z (N), Z (N) (t, r)dr 1 N i S (N) (t) G(r) G(T + t) dr 1 G(T + t) S(N) Therefore, taking the imit as N of both sides of (5), we have [ ] im E W (N) N 2 (t) = im E N 1 = E 1 [ ( Z (N) (t, ) + Z (N) ) ( +1 (t, ) [ (Z (t, ) + Z +1 (t, )) (t) G(r)dr Z (N) 1 G(T + t). (52) (t, r) Z (N) ) ] +1 (t, r) dr ] (Z (t, r) Z +1 (t, r)) dr. (53) The exchange of imit and summation in the first equaity is justified by (16), Z (N) (t, r) Z (N) (t, ) 1 and the dominated convergence theorem, whie the second equaity hods due to Theorem 4.2, (52) and the bounded convergence theorem. The resut foows from (51) and (53). In this section we vaidate the resuts in Theorem 4.3 and Theorem 4.4 using Monte Caro (MC) simuations. First in Section 5.5, we present a stabe scheme for numericay approximating the soution to the hydrodynamic PDE. Then in Section 5.6, we compare the resuts obtained from Monte Caro simuation and numerica soutions to PDE for a variety of different service distributions and in each case, we observe a cose match. 5.5 A Numerica Approximation Scheme Note that the hydrodynamic PDE is comprised of a countabe number of integro-differentia equations, corresponding to each Z. Therefore, in order to numericay sove the hydrodynamic PDE (6)-(9), we truncate the state space and ony sove for Z (t, r), = 1,..., L, t T, r R, for suitabe L N and R, T <. Next, we discretize r and t on uniform meshes: Ẑ (t m, r n ). = Z (mδ, nδ) with m T /δ and n R /δ. Then we sove the equations (6)-(9) numericay by the finite difference method and the expicit forward Euer scheme[14]. That is, for fixed t m > and r n >, we appy the foowing approximations m +δ Ẑ 1 (t m + δ, r n ) = Ẑ 1 (t m, r n + δ) t m G(t m + δ + r n u) r Ẑ 2 (u, )du +δ + λ(u)g(t m + δ + r n u) ( 1 Ẑ 1 (u, ) 2) du t Ẑ 1 (t m, r n + δ) G(r n ) ( Ẑ 2 (t m, δ) Ẑ 2 (t m, ) ) + G(r n ) ( 1 Ẑ 1 (t m, ) 2) +δ λ(u)du. t 17

18 and for 2, m +δ Ẑ (t m + δ, r n ) = Ẑ (t m, r n + δ) t m G(t m + δ + r n u) r Ẑ +1 (u, )du m +δ + λ(u) ( Ẑ 1 (u, ) + Ẑ (u, ) ) ( Ẑ 1 (u, t m + δ + r n u) Ẑ (u, t m + δ + r n u) ) du t m Ẑ (t m, r n + δ) G(r n ) ( Ẑ +1 (t m, δ) Ẑ +1 (t m, ) ) + ( Ẑ 1 (t m, ) + Ẑ (t m, ) ) ( Ẑ 1 (t m, r n ) Ẑ (t m, r n ) ) m +δ λ(u)du. The numerica scheme above has been stabiized by appying the upwind scheme [9] to the derivatives with respect to r, otherwise, the numerics wi bow up in a short time interva. Using these approximations, we can update Ẑ (t m + δ, ) from Ẑ (t m, ). 5.6 Vaidation of Resuts We now compare the empirica queue ength distribution and mean virtua waiting time obtained from the Monte Caro simuation of an N-server network, with the corresponding imit quantities (17) and (19), as predicted by the numerica approximation of the PDE. We consider a sequence of networks indexed by the number of servers N, with a Poisson arriva process with rate λ =.5, and the foowing initia conditions. Each server has initiay one job with initia age equa to zero, that is X (N),i () = 1 and a (N),i () = for a i = 1,..., N. Note that this sequence of initia conditions satisfies the conditions of Theorem 4.2, and converges to the initia condition (Z ( ); 1) for the hydrodynamic PDE, where for r, Z1 (r) = G(r), Z (r) =, 2. t m Queue Length Distribution 6 Simuation Resuts First we compute the probabiity that a typica (fixed) queue has ength at east at time t, for t [, 1] in a network of N = 1 servers, using Monte Caro simuation with 1 reaizations. Then we compare this probabiity with the quantity obtained by numericay soving the PDE using the method described in Section 5.5, with L = 6, R = 2 and δ =.1. We make this comparison for a variety of (unit mean) service time distribution, incuding Pareto (with shape parameter β = 2.25), Log-Norma (with shape parameter σ =.33), Gamma (with shape parameter k = 2) and Hyper-Exponentia (with parameters λ 1 =.5 and λ 2 = 2), which is a specia case of a Phase-type distribution. The resuts are iustrated in Figure 2 for = 1 and = 2, and a cose match is observed in a cases. In our setting, the run time for the Monte Caro method is approximatey 2 3 hours, which is orders of magnitude onger than the time taken to numericay approximate the PDE, which is around 7 9 seconds Waiting Time Next, we compare the mean virtua waiting time in the same setting described above, but with the arriva rate now set to λ =.7. We measure the mean virtua waiting from a Monte Caro simuation as foows: at any time t, we determine which server a virtua arriving job woud have been routed to 18

19 (a) Pareto (b) Log-Norma (c) Gamma (d) Hyper-Exponentia Figure 2: Comparison of the estimate for P(X (N),1 (t) ) obtained from MC simuation (in red) versus the numerica approximation of Z (t, ) (in bue) during t [, 1] for = 1 (top row) and = 2 (bottom row). by choosing two servers uniformy at random, and picking the one with shorter queue ength, and then observe the waiting time in that server. The average is taken over 2 reaizations. We compare this to the foowing approximation of the imit provided by Theorem 4.4: [ ] E W (N) (t) L Ẑ (t, ) 2 + =2 L 1 =1 [Ẑ (t, ) + Ẑ +1 (t, ) ] R /δ [Ẑ (t, r j ) Ẑ +1 (t, r j ) ] δ. (54) j= where {Ẑ ; 1} is the numerica soution of the PDE described in Section 5.5. The resut of the comparison for the Pareto distribution (with mean set to 1 and shape parameter β = 3) and is potted in Figure 3(a). We observe good agreement between the two curves. Furthermore, reca that the actua waiting time of a job is the difference between the arriva and service entry times of that job. We aso pot the average of the average of the actua waiting time. At each time t, the atter quantity is defined to be the sum of the waiting times of a jobs arrived in that time sot in the mesh, divided by the number of jobs that arrived in that time sot. We observe that the mean virtua waiting time is a good approximation to the average actua waiting time as we More Genera Initia Conditions Finay, to iustrate that the assumption a (N),i () T on initia ages is not necessary, we vaidate our resut for another sequence of networks with the foowing initia condition: there are initiay two jobs in each queue, and the initia ages are a independent and distributed according to G(x)dx (which is the stationary age distribution in a renewa process with inter-arriva distribution G). The corresponding initia condition for the PDE are given by Z (r) = r G(x)dx, = 1, 2, Z (r) =, 3. 19

20 Figure 3: a. Mean virtua sojourn times from MC (red) and PDE (bue) as we as the averaged actua sojourn times from MC (green) for the Pareto distribution with β = 3. b. Mean virtua waiting times from MC (red) and PDE (bue) for Gamma service distribution and unbounded initia ages. We vaidate the approximation given in (54) using Monte Caro simuations for a network with N = 1 servers and a Gamma service time distribution (with shape parameter k = 2). The resut is depicted in Figure 3(b). 7 Engineering Insights In practice, transient Quaity of Service (QoS) parameters are of particuar interest in many appications. However, to the best of our knowedge, prior to this work, oad-baancing networks with genera service distributions have ony been studied in steady state and under constant Poisson arrivas [6]. We iustrate how our PDE approximation can be used to shed insight into transient phenomena of practica reevance. In Section 7.1, we study the time it takes for a congested network to get rid of a backog of jobs, and in Section 7.2 we anayze the performance of a oad baancing network with time-varying, periodic, Poisson arrivas, consisting of intermittent high and ow periods. 7.1 Initia Backog Given a network that is congested with a backog of jobs, the system administrator woud ike to know how ong it woud take for the system to get rid of the backog and for the QoS to return coser to the norma operating point. In this section, we consider a network with a arge initia backog, and hence a arge initia virtua waiting time, and study the time it takes for the system to unoad to the extent that the mean virtua waiting time reaches haf of its initia vaue, which we refer to as reaxation time. We investigate the effect of service distribution statistics on the reaxation time. In each case, the goa is to iustrate how the PDE approximation may be used to hep uncover interesting (and possiby unexpected) network phenomena. In the presence of genera service distributions, in order to capture congestion in the initia distribution of the network, one has to specify not ony the distribution of the number of jobs in the different queues, but aso the distribution of ages of jobs being served at these queues. There are a number of different configurations that coud refect a congested initia state. In order to generate initia conditions that may naturay occur in practice, we consider an initia state that corresponds to the distribution resuting from a network that has been experiencing a higher than norma arriva rate for a period of time. 2

21 Figure 4: a. Mean virtua waiting time for network with initia backog and Pareto service distribution with β = 1.25 (red) and β = 2.5 (bue), using the PDE method. b. Reaxation time vs. Median for Pareto, Log-Norma and Weibu service time distributions. Specificay, we first consider a network that starts empty and runs with nomina arriva rate λ =.6 for T = 1 units of time, and then experienced an arriva rate that is about 8 times the nomina arriva rate, namey λ = 5, for a period of T b = 2 units prior to zero. The distribution of this network at time T + T b then represents a congested state for the network and is used as the initia condition for our PDE approximations. We then assume that at time, the mean arriva rate reverts back to its nomina vaue λ =.6 and we study the reaxation time, namey the time it takes for the mean virtua waiting time to reach haf its initia vaue. In Figure 4(a), we pot the evoution in time of the mean virtua waiting time for (nomina) arriva rate λ =.6, both when the service distribution is Pareto with unit mean and parameter β = 1.25 (heavytaied) and Pareto with parameter β = 2.5 (ight-taied). The curves are obtained using the numerica scheme to sove the PDE with cutoff vaues L = 12 and R = 2 and step size δ =.1. An interesting observation is that the heavy-taied case ceary has a smaer reaxation time than the ight-taied case. This phenomenon may seem particuary surprising in ight of the steady-state resut of Bramson et. a in [6], which shows that for Pareto service time distributions with unit mean, the tai of the imit steady state queue-ength distribution has a doube exponentia decay when β > 2 (ight-taied), in contrast to the case when β < 2 (heavy-taied), when it has ony a power aw decay. Athough the tais of the imit steady-state distribution do not represent the imit of the tais of the steady-state distribution in the N-server system (since the N and tai decay imits are typicay not interchangeabe), the resuts in [6] suggest that from the point of view of equiibrium queue ength, the ight-taied Pareto distribution shows better performance. The transient phenomenon observed above aso persists when the initia condition is instead chosen to be a arge number of jobs uniformy distributed amongst different queues, a starting with zero ages, which may roughy correspond to congestion caused due to a sudden arge spurt of job arrivas into the network. A possibe heuristic expanation for the contrasting behavior observed in the transient case is that when a Pareto service distribution is fixed to have mean equa to one, the median of the distribution decreases with a decrease in β. As a resut, when the heavier the tai of the service distribution, the greater the fraction of initiay backogged (and newy arriving) jobs with smaer service requirements, and the smaer the fraction of jobs with very ong service times. This heps servers in a arge number of queues to get rid of their backog and reduce their queue engths faster, whereas the jobs with ong service times ead to arge queue engths in ony a sma fraction of servers. The atter jobs do not 21