Optimal Hiring of Cloud Servers A. Stephen McGough, Isi Mitrani. EPEW 2014, Florence

Transcription

1 Optimal Hiring of Cloud Servers A. Stephen McGough, Isi Mitrani EPEW 2014, Florence

2 Scenario How many cloud instances should be hired? Requests Host hiring servers The number of active servers is controlled by the host.

3 Dynamic optimization problems: In a system whose state is a random process, decide at various moments in time how many servers to employ in order to minimize long-term performance and operating costs. time

4 Case 1: Batch Arrivals Decision instants are when jobs arrive In batches Arrival rate λ Service rate µ Batch size distribution b i How many servers should be hired at each arrival instant? variable time

5 Case 2: Dynamically Controlled M/M/n/J Queue n servers currently active Service rate µ Arrival rate λ J maximum jobs Queue fixed time How many servers should be hired at each hiring instant?

6 General framework (Semi-Markov Decision Process) state i action a i state j action a j time Identify best action a i to take for state i Characteristics: Average interval to next decision instant: τ i (a) One step cost, i.e. average cost incurred until next decision instant c i (a) Transition probability to next state p i,j (a) Policy set A = a i, i=1,, (an action for each state)

7 Policy Set Stationary policy A Actions depend only on state not on prior history Average cost incurred during interval (0,t): Z A (t) Long-term average cost per unit time: g(a) = lim t 1 t E[Z A(t)]. g(a) does not depend on initial state

8 Determining Cost For a given policy set A The average cost g(a) can be computed by introducing auxiliary variables v j One for each state And solving the set of simultaneous liner equations: v j = c j (A) τ j (A)g(A)+ J p j,k (A)v k ; j =1, 2,...,J, k=1 Make unique solution by setting v k = 0 for some state k

9 Determining A* Find an optimal policy using a policy improvement algorithm: 1. Choose an initial stationary policy A 2. Compute v i and g(a) by solving the set of simultaneous liner equations 3. For each i find the action a* which minimizes the right hand side of: J v j = c j (A) τ j (A)g(A)+ p j,k (A)v k ; j =1, 2,...,J, k=1 4. If A* = A we re finished Else let A = A* and repeat from 2 The algorithm is guaranteed to terminate in a finite number of iterations

10 Heuristics and Policies Greedy Heuristic: For every state j choose the action which minimizes the cost in the current interval The one-step-cost c j (n) λb Fixed policy fixed number of nservers = 0.7µ To cope with most extreme events aim for average server occupancy of 70% λb Case 1: n =. Case 2: n = 0.7µ λ 0.7µ..

11 Results

12 2. If j>n,thenn jobs are being served and j n are waiting. The next event to occur is either a service completion, with probability nµ/(λ + nµ), or an arrival of a new batch, with probability λ/(λ+nµ). The average interval until that event is 1/(λ Case + nµ), and 1:Batch there are j jobs Arrivals present during it. If the next event is a service completion, then the decision period continues with j 1 jobs present; otherwise it terminates and there is no further contribution to T j (n). This provides a recurrence relation, Decision instants: batch arrivals System state: number j of jobs present j T j (n) = Action taken: n servers hired Average length of decision interval: 1/λ Transition probabilities: p j,k (t) Equation (4), together with the recurrences (5), allow the holding times T j (n) to be computed Closed easily form all expressions j and n. Theaveragecost,c j (n), incurred during a decision period is the sum of the holding cost and the server cost: One-step cost of decision n: λ + nµ + nµ λ + nµ T j 1(n) ; j = n +1,n+2,...,J. (5) c j (n) =c 1 T j (n)+c 2 n 1 λ. (6) Recurrence relation for T j holding time Before addressing the transition probabilities p j,k (n), consider the probability, q j,k (n), that there will be k jobs present just before the next decision epoch, given that there are j jobs now and n servers are available. That is the probability that

13 Case 2:Dynamically Controlled M/M/n/J Queue Decision instants: discrete System state: number of jobs present j Action taken: n servers hired Average length of decision interval: τ Transition probabilities: p j,k (t) Numerical solution for transient transition probabilities One-step cost of decision n: c j (n) = [ c 1 j + L j 2 Optimal Hiring of Cloud Servers 9 ] + c 2 n τ. (17) Using these L expressions, j average number the optimal of jobs policy in the cansystem be computed during as described in section 2. interval τ N.B. One might wish to relax the assumptions that jobs arrive in a Pois-

14 Case 1:Batch Arrivals Optimal policy Greedy heuristic Fixed policy + g Fig. 3. Batch arrivals: varying unit holding cost c 1

15 Case 2:Dynamically Controlled M/M/n/J Queue g 36 Optimal policy + 34 Greedy heuristic Fixed policy ρ Fig. 4. Fixed hiring periods: varying offered load

16 Questions