Optimal Hiring of Cloud Servers A. Stephen McGough, Isi Mitrani EPEW 2014, Florence
Scenario How many cloud instances should be hired? Requests Host hiring servers The number of active servers is controlled by the host.
Dynamic optimization problems: In a system whose state is a random process, decide at various moments in time how many servers to employ in order to minimize long-term performance and operating costs. time
Case 1: Batch Arrivals Decision instants are when jobs arrive In batches Arrival rate λ Service rate µ Batch size distribution b i How many servers should be hired at each arrival instant? variable time
Case 2: Dynamically Controlled M/M/n/J Queue n servers currently active Service rate µ Arrival rate λ J maximum jobs Queue fixed time How many servers should be hired at each hiring instant?
General framework (Semi-Markov Decision Process) state i action a i state j action a j time Identify best action a i to take for state i Characteristics: Average interval to next decision instant: τ i (a) One step cost, i.e. average cost incurred until next decision instant c i (a) Transition probability to next state p i,j (a) Policy set A = a i, i=1,, (an action for each state)
Policy Set Stationary policy A Actions depend only on state not on prior history Average cost incurred during interval (0,t): Z A (t) Long-term average cost per unit time: g(a) = lim t 1 t E[Z A(t)]. g(a) does not depend on initial state
Determining Cost For a given policy set A The average cost g(a) can be computed by introducing auxiliary variables v j One for each state And solving the set of simultaneous liner equations: v j = c j (A) τ j (A)g(A)+ J p j,k (A)v k ; j =1, 2,...,J, k=1 Make unique solution by setting v k = 0 for some state k
Determining A* Find an optimal policy using a policy improvement algorithm: 1. Choose an initial stationary policy A 2. Compute v i and g(a) by solving the set of simultaneous liner equations 3. For each i find the action a* which minimizes the right hand side of: J v j = c j (A) τ j (A)g(A)+ p j,k (A)v k ; j =1, 2,...,J, k=1 4. If A* = A we re finished Else let A = A* and repeat from 2 The algorithm is guaranteed to terminate in a finite number of iterations
Heuristics and Policies Greedy Heuristic: For every state j choose the action which minimizes the cost in the current interval The one-step-cost c j (n) λb Fixed policy fixed number of nservers = 0.7µ To cope with most extreme events aim for average server occupancy of 70% λb Case 1: n =. Case 2: n = 0.7µ λ 0.7µ..
Results
2. If j>n,thenn jobs are being served and j n are waiting. The next event to occur is either a service completion, with probability nµ/(λ + nµ), or an arrival of a new batch, with probability λ/(λ+nµ). The average interval until that event is 1/(λ Case + nµ), and 1:Batch there are j jobs Arrivals present during it. If the next event is a service completion, then the decision period continues with j 1 jobs present; otherwise it terminates and there is no further contribution to T j (n). This provides a recurrence relation, Decision instants: batch arrivals System state: number j of jobs present j T j (n) = Action taken: n servers hired Average length of decision interval: 1/λ Transition probabilities: p j,k (t) Equation (4), together with the recurrences (5), allow the holding times T j (n) to be computed Closed easily form all expressions j and n. Theaveragecost,c j (n), incurred during a decision period is the sum of the holding cost and the server cost: One-step cost of decision n: λ + nµ + nµ λ + nµ T j 1(n) ; j = n +1,n+2,...,J. (5) c j (n) =c 1 T j (n)+c 2 n 1 λ. (6) Recurrence relation for T j holding time Before addressing the transition probabilities p j,k (n), consider the probability, q j,k (n), that there will be k jobs present just before the next decision epoch, given that there are j jobs now and n servers are available. That is the probability that
Case 2:Dynamically Controlled M/M/n/J Queue Decision instants: discrete System state: number of jobs present j Action taken: n servers hired Average length of decision interval: τ Transition probabilities: p j,k (t) Numerical solution for transient transition probabilities One-step cost of decision n: c j (n) = [ c 1 j + L j 2 Optimal Hiring of Cloud Servers 9 ] + c 2 n τ. (17) Using these L expressions, j average number the optimal of jobs policy in the cansystem be computed during as described in section 2. interval τ N.B. One might wish to relax the assumptions that jobs arrive in a Pois-
Case 1:Batch Arrivals 500 450 400 350 Optimal policy Greedy heuristic Fixed policy + g 300 250 200 150 100 + + + + 50 4 6 8 10 12 14 16 18 20 Fig. 3. Batch arrivals: varying unit holding cost c 1
Case 2:Dynamically Controlled M/M/n/J Queue g 36 Optimal policy + 34 Greedy heuristic Fixed policy 32 + 30 28 + 26 24 + 22 + 20 18 + 16 10 11 12 13 14 15 16 17 18 ρ Fig. 4. Fixed hiring periods: varying offered load
Questions stephen.mcgough@durham.ac.uk Isi.mitrani@newcastle.ac.uk