Head-of-line processor sharing: optimal control

Transcription

1 N O Head-of-line processor sharing: optimal control? Sandra van Wijk A.C.C.v.Wijk@tue.nl Joint work with Çagdas Büyükkaramikli 2 E H A C A N F D H A C O 2 E F F A N F D F F N + F F Advisors: Ivo Adan and Geert-Jan van Houtum SMMSO Conference 2011 Kuşadası, Turkey May 28 June 2, 2011

2 Introduction 2/23 Workload in M/M/1 queue Application: e.g. production system

3 Introduction 3/23 Workload in M/M/1 queue Extra revenue for serving opportunity customers

4 Introduction 4/23 Workload in M/M/1 queue However, waiting times (and hence costs) of regular customers increase

5 Introduction 5/23 Workload in M/M/1 queue Regular and opportunity customers Head-of-line processor sharing

6 O N Model 6/23 2 E H A C A N F 2 E F F A N F Research Question: What is optimal control policy?

7 O N Model 7/23 2 E H A C A N F 2 E F F A N F D H A C O D F F N + F F? Research Question: What is optimal control policy?

8 Analysis 8/23 Markov Decision Problem Value function: V n+1 (x, y) 1 = h opp (x) + h reg (y) + λ opp + λ reg + µ + α ( λ reg V n (x, y + 1) { } min{vn (x + 1, y), V + λ n (x, y) + C opp } if x = 0 opp V n (x, y) + C opp if x = 1 { }) + min µ [0, µ] c (µ) + µv n ((x 1) +, y) + ( µ µ)v n (x, (y 1) + ), V 0 0. Structural properties of value function imply optimal control policy structure.

9 Analysis 9/23 Value function Structural property: Multimodularity (MM, cf. Hajek, 1985) Multimodularity (for 2 dimensions): Supermodularity: f (x, y) + f (x + 1, y + 1) f (x + 1, y) + f (x, y + 1), Superconvexity(1,2): f (x + 2, y) + f (x, y + 1) f (x + 1, y) + f (x + 1, y + 1), Superconvexity(2,1): f (x, y + 2) + f (x + 1, y) f (x, y + 1) + f (x + 1, y + 1), which implies: Convexity(1): f (x, y) + f (x + 2, y) 2 f (x + 1, y), Convexity(2): f (x, y) + f (x, y + 2) 2 f (x, y + 1).

10 H H H H H H Analysis 10/23 Value function Structural property: Multimodularity (MM, cf. Hajek, 1985) Multimodularity (for 2 dimensions): H 5 K F A K = H E J O 5 K F A H? L A N E J O 5 K F A H? L A N E J O H H H H + L A N E J O + L A N E J O

11 H H H H H H H H H H Analysis 11/23 Value function Structural property: Multimodularity (MM, cf. Hajek, 1985) Multimodularity (for 2 dimensions): H 5 K F A K = H E J O 5 K F A H? L A N E J O 5 K F A H? L A N E J O H H H H + L A N E J O + L A N E J O

12 Analysis 12/23 Value function Prove that V n is MM by induction on n: V 0 0 is MM, Assume V n is MM, show that V n+1 is MM. Recall: V n+1 (x, y) = h opp (x) + h reg (y) + ( λ reg V n (x, y + 1) 1 λ opp + λ reg + µ + α { } min{vn (x + 1, y), V + λ n (x, y) + C opp } if x = 0 opp V n (x, y) + C opp if x = 1 { }) + min c (µ) + µv n ((x 1) +, y) + ( µ µ)v n (x, (y 1) + ), µ [0, µ]

13 Analysis 13/23 Value function Rewrite value function using event operators (cf. Koole, 2006) V n+1 (x, y) = T costs ( T unif ( TC A(1) V n (x, y), T A(2) V n (x, y), T CT D(1) V n (x, y) )) Event operators: Arrivals of opportunity customers (decision!) Arrivals of regular customers T C A(1) f (x, y) = min{v n (x + 1, y), V n (x, y) + C opp } T A(2) f (x, y) = V n (x, y + 1)

14 Analysis 14/23 Value function Rewrite value function using event operators (cf. Koole, 2006) V n+1 (x, y) = T costs ( T unif ( TC A(1) V n (x, y), T A(2) V n (x, y), T CT D(1) V n (x, y) )) Event operators (ctd.): Service completions (decision!) { } T CT D(1) f (x, y) = c(µ) + µv n ((x 1) +, y) + (1 µ)v n (x, (y 1) + ) min µ [0,1]

15 Analysis 15/23 Value function Rewrite value function using event operators (cf. Koole, 2006) V n+1 (x, y) = T costs ( T unif ( TC A(1) V n (x, y), T A(2) V n (x, y), T CT D(1) V n (x, y) )) Event operators (ctd.): Costs Uniformization T costs f (x, y) = h opp (x) + h reg (y) + f (x, y) T unif ( f 1, f 2, f 3 )(x, y) = λ opp f 1 (x, y) + λ reg f 2 (x, y) + µ f 3 (x, y) λ opp + λ reg + µ + α

16 Analysis 16/23 Value function Use known results for operators (Koole, 2006): V n is MM T C A(1) V n is MM V n is MM T A(2) V n is MM V n is MM T unif V n is MM V n is MM T costs V n is MM And T CT D(1)?

17 Analysis 17/23 Value function Compare T CT D(1) to departure operator in tandem queue: T CT D(1) f (x, y) = T CT D(1) f (x, y) = min µ [0,1] min µ [0,1] { } c(µ) + µv n ((x 1) +, y) + (1 µ)v n (x, (y 1) + ) { } c(µ) + µv n ((x 1) +, y + 1) + (1 µ)v n (x, y) Known result: V n is MM T CT D(1) V n is MM Transformation: y y 1 check x = 0, y = 0 and x > 0, y = 0 Then: V n is MM T CT D(1) V n is MM

18 Analysis 18/23 Main result Value function is MM. Implies optimal policy structure. Optimal policy structure for a head-of-line processor sharing model, with adjustable weights and two types of customers: Threshold T for admitting opportunity customer: accept if y T, reject it otherwise. The optimal server speed dedicated to the opportunity customer is a monotone deceasing function in x.

19 Examples 19/23 Example 1: c(µ) 0 λ reg = 3, λ opp = 1, C opp = 8, µ = 10, h opp (x) = x and h reg (y) = 0.05 y 2 if y < 20; h reg (y) = 100 y otherwise. Optimal policy accept/reject opportunity customer: x\y Threshold policy with T = 6. The optimal fraction µ [0, 1] of server speed for opp. customer: x\y So, opportunity customers either gets full attention of the server, or no attention at all, with threshold y = 10.

20 Examples 20/23 Example 1: c(µ) 0 If c(µ) 0 (or: constant) also threshold policy for optimal server speed for opp. customer. Because T CT D(1) minimizes a linear function: T CT D(1) f (x, y) { = min c(µ) + µv n ((x 1) +, y) µ [0,1] } + (1 µ)v n (x, (y 1) + )

21 Examples 21/23 Example 2: c(µ) = 0 Same parameters, now: c(µ) = 0 if 0 µ < 0.25; 0.5 if 0.25 µ < 0.50; 1 if 0.50 µ < 0.75; 1.5 if 0.75 µ 1. Threshold T = 5 for accepting opportunity customers. The optimal fraction µ [0, 1] of the service speed for the opp. customer: x\y Indeed monotone decreasing.

22 Extensions 22/23 Model extensions Accept or reject regular customers Queueing of opportunity customers

23 Further research & Conclusion 23/23 Further research Steady state probability distribution Total service rate increases or decreases when the server divides its attention to two customers Multiple types of opportunity customers Conclusion Optimal policy structure for a head-of-line processor sharing model, with adjustable weights and two types of customers.