Stochastic Models for Inventory Management at Service Facilities

Size: px

Start display at page:

Download "Stochastic Models for Inventory Management at Service Facilities"

Paulina Gallagher
7 years ago
Views:

1 Stochastic Models for Inventory Management at Service Facilities O. Berman, E. Kim Presented by F. Zoghalchi University of Toronto Rotman School of Management Dec, 2012

2 Agenda 1 Problem description Deterministic Model Probabilistic Model 2 Markov Decision Process 3 Optimal Policy Finite queue capacity model 4 Heuristic Policy 5 Numerical Results 6 Summary

3 Problem description Markov Decision Process Optimal Policy Heuristic Deterministic Policy Model Numerical Probabilistic Results Summary Model Problem Description a facility provides service to costumers using items of inventory. queues might form, we may run out of inventory there is a cost associated with waiting costumers (in queue), holding items (in inventory), placing order OBJECTIVE: minimize the total cost in long-run

4 Problem description Markov Decision Process Optimal Policy Heuristic Deterministic Policy Model Numerical Probabilistic Results Summary Model Deterministic Model Recall the deterministic model discussed in class: costumers arrive with constant non-random rate λ service takes place with rate µ > λ there are again waiting, holding and order costs. QUESTION: when should we place an order, so as to minimize total cost?

5 Problem description Markov Decision Process Optimal Policy Heuristic Deterministic Policy Model Numerical Probabilistic Results Summary Model T µ : inventory level is positive, there is a queue: inventory depletes at rate µ, queue depletes at rate λ µ T λ : no queue, inventory depletes at rate λ T 0 : inventory is empty, queue forms at rate λ

6 Problem description Markov Decision Process Optimal Policy Heuristic Deterministic Policy Model Numerical Probabilistic Results Summary Model average inventory: I = λ Q average queue length: C = λ Q ( Q+I0 2 T µ + I 0 2 T λ ) C max 2 (T µ + T 0 ) one can write times above, in terms of Q and I 0

7 Problem description Markov Decision Process Optimal Policy Heuristic Deterministic Policy Model Numerical Probabilistic Results Summary Model Total Cost TC = αi + βc + γ λ Q By typical optimization methods (taking partial derivatives): 2γλ Q α + β = α ρα + β We ll use this solution as a heuristic in the present problem

8 Problem description Markov Decision Process Optimal Policy Heuristic Deterministic Policy Model Numerical Probabilistic Results Summary Model Probabilistic Model the model is as follows: We denote by x = (x 1, x 2 ) the number of customers in the system (x 1 ) and inventory level (x 2 ) At a decision epoch, there are two admissible actions: do not order and order Given the order quantity Q, what is the inventory policy that minimizes total cost? We need to solve the underlying Markov Decision Process.

9 MDP A MDP is a 4-tuple (X, A, P. (.,.), R. (.,.)) where X is the set of states A is the set of actions P a (x, x ) = Pr(X t+1 = x X t = x, a t = a) R a (x, x ) is the reward for transition from state s to s.

10 Problem is to find a policy π(x) : X A so as to maximize a discounted reward function usually such as: β n R π(xn)(x n, X n+1 ) i=1 And answer is: π(x) = arg max P a (x, x [ ) Ra (x, x ) + βv (x ) ] a x V (x) = P π(x) (x, x ) [ R π(x) (x, x ) + βv (x ) ] x

11 Let r t (x 1, x 2 )(resp. s t (x 1, x 2 )) denote the expected discounted cost in state x given that the do not order (resp. order) policy is taken at time t. Dynamic Programming Recursive equations where (λ + µ = 1) V t (x 1, x 2 ) = min(r t (x 1, x 2 ), s t (x 1, x 2 )) r t (x 1, x 2 ) = 2 c i x i + β(λv t 1 (x 1 + 1, x 2 ) + µv t 1 (D(x 1, x 2 ))) i=1 s t (x 1, x 2 ) = 2 c i x i +K+β(λV t 1 (x 1 +1, x 2 +Q)+µV t 1 (D(x 1, x 2 +Q))) i=1 D(x 1, x 2 ) = (x 1 1, x 2 1) if x 1,2 > 0

12 Problem description Markov Decision Process Optimal Policy Heuristic Finite queue Policy capacity Numerical model Results Summary Threshold based policy Theorem 1 for t = 1,..., T, V t (x 1, x 2 ) = r t (x 1, x 2 ), x 1 0, x 2 > 0 V t (0, 0) = r t (0, 0) Proof is based on induction. Interpretation: If inventory (x 2 ) > 0, do not order. If the system is empty, do not order.

13 Problem description Markov Decision Process Optimal Policy Heuristic Finite queue Policy capacity Numerical model Results Summary Theorem 2 When inventory level is zero, the optimal policy is a threshold-based policy, i.e., for t = 1,..., T there exists a threshold value Θ t, Θ t = min{x 1 {1, 2,..., } : s t (x 1, 0) < r t (x 1, 0)} such that it is optimal to order iff x 1 Θ t

14 Problem description Markov Decision Process Optimal Policy Heuristic Finite queue Policy capacity Numerical model Results Summary Finite queue capacity model r t (x 1, x 2 ) = 2 c i x i + +λs1(x 1 = M) i=1 + β(λv t 1 (Ax 1, x 2 ) + µv t 1 (D(x 1, x 2 ))) 2 s t (x 1, x 2 ) = c i x i + K + λs1(x 1 = M) i=1 + β(λv t 1 (Ax 1, x 2 + Q) + µv t 1 (D(x 1, x 2 + Q))) where Ax 1 = x if x 1 < M and x 1 otherwise. Assumption P: S > c 1 /(1 β) = c 1 (1 + β + β ), recall c 1 is the costumer waiting cost and β discount factor. Theorem Under assumption P, the optimal policy is again a threshold-based policy

15 Heuristic Policy There is no closed form for the optimal order quantity Q, but numerical investigations shows cost is a convex function of Q.. Following the deterministic model (Berman, et al.) we approximate the optimal order quantity by 2Kλ Q h c1 + c 2 = c 2 ρc 2 + c 1 Given the order quantity Q h, what is the threshold value Θ?

16 determining Θ: server delays x 1 customers until there are Θ customers and then puts an order the ordering cost will be shared by Θ/(1 ρ) customers that server will serve on average before queue is exhausted. delaying costumers causes extra cost Therefore, subject to Θ = min Θ Θ 1 ρ Q { f (Θ) = K Θ/(1 ρ) + c } 1(Θ x 1 ) λ

17 { Θ = min f (Θ) = Θ K Θ/(1 ρ) + c } 1(Θ x 1 ) λ since f (Θ) is a convex function of Θ, by taking the derivative we have: Θ = λk(1 ρ)/c 1

18 Table 1 shows 8 input data sets and table 2 is the corresponding threshold value and total cost. it shows that in heavy traffic mode, the heuristics work pretty well. as ρ 1, Θ 1 Example M Q K c 1 c 2 λ µ ρ Example Optimal Θ Huristic Θ h %Suboptimal

19 Summary this paper uses MDP to solve stochastic inventory control problem. optimum policy is to be patient, and wait for a threshold finite queue capacity, under some logical assumption, behaves similar to the infinite capacity model. effective heuristic policies, based on the deterministic model were proposed

Optimal Control of a Production-Inventory System with both Backorders and Lost Sales

Optimal Control of a Production-Inventory System with both Backorders and Lost Sales Saif Benjaafar, 1 Mohsen ElHafsi, 2 Tingliang Huang 3 1 Industrial and Systems Engineering, University of Minnesota,