Fluid Models for Production-Inventory Systems

Transcription

1 Fluid Models for Production-Inventory Systems by Keqi Yan A dissertation submitted to the faculty of the University of North Carolina at Chapel Hill in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Statistics and Operations Research. Chapel Hill 2006 Approved by Advisor: Vidyadhar G. Kulkarni Reader: Amarjit Budhiraja Reader: Tugrul Sanli Reader: Jayashankar M. Swaminathan Reader: Paul H. Zipkin

2 c 2006 Keqi Yan ALL RIGHTS RESERVED ii

3 ABSTRACT Keqi Yan: Fluid Models for Production-Inventory Systems (Under the direction of Professor Vidyadhar G. Kulkarni) We consider a single stage production-inventory system whose production and demand rates are modulated by a finite state Markov chain called the environment. Supplementary orders can be placed from external suppliers when needed. We model this system by a fluid-flow system and derive the limiting distribution of the bivariate process (fluid level, environment state). We present a stochastic decomposition property for this fluid model and hence prove that the classical deterministic Economic- Order-Quantity (EOQ) policy is still optimal in this stochastic environment under certain assumptions. We extend the results to more general models: 1. When backlogging is allowed, we investigate the optimal reorder-point / orderquantity (r, q) policy. We prove that for a given order quantity q, the optimal reorder point r (q) can be explicitly given by the well-known newsboy solution. We also show that in a special case the optimality of the deterministic EOQ policy with backlogging holds. 2. When the order quantity can be environment-dependent, we derive the limiting distribution and then calculate the optimal order quantity for each ordering state. 3. When there are positive leadtimes, we consider three cases: orders are processed iii

4 sequentially, in parallel, or the leadtimes have different distributions depending on the number of outstanding orders. Assuming there exists an upper limit N for the number of outstanding orders, this model generalizes the emergency-supply model, selective lost-sales model, and can also be an approximation of infinite-supplier model when N is large enough or the probability that there are N outstanding orders is small. We derive the limiting distribution and the optimal (r, q) policy. We prove that for a given q, the optimality of the newsboy solution for r still holds. We also illustrate numerically how to calculate the optimal N which balances the backlogging cost and emergency-supply / lost-sale cost. iv

5 ACKNOWLEDGEMENTS I have waited for a long time for this chance to express my deepest gratitude to my advisor, Professor Vidyadhar G. Kulkarni. He always earns the respect and admiration of all his students, and for me, he is the hero in my four years journey in the pursuit of the Ph.D. at Chapel Hill. His impact on my attitude in research and philosophy about the world will benefit the reminder of my life. I would also like to thank Professor Paul Zipkin for introducing me to the field of inventory management. During the process of finishing this dissertation, he gave me priceless suggestions and always kindly encouraged me like a father figure. Professor Jayashankar M. Swaminathan s teaching opened a wide door to the supply chain area. With Dr. Tugrul Sanli, I worked on a demonstration of the software Inventory Replenishment Planning, which was my first exposure to the concept of inventory control and spurred my interest in this area. I am also very thankful to Professor Amarjit Budhiraja for his inspiring questions and suggestions. Special thanks to my supervisors and friends at SAS Institute from where I have received tuition support for three years, and have gained valuable experience in implementing operations research methodologies in software and solving real-world problems. I really enjoyed and have learned so much from these past years internship there. It was an inseparable part of my student life in North Carolina. Of course I would also like to thank my parents Yan Yuchi, He Daying and my sister Yan Shuli. However, I know that words are redundant here, in comparison to their unconditional love and support, which are far beyond any language in the world. v

6 CONTENTS LIST OF FIGURES xi LIST OF SYMBOLS xii 1 Introduction Stochastic Fluid Models Stochastic Inventory Control Problems Basic EOQ Model Backlogging EOQ Model Environment-Dependent Order Quantities Stochastic Leadtimes Fluid Model Introduction The Standard Fluid Model The Fluid Model with Jumps vi

7 2.4 Differential Equations for the Limiting Distribution Solution to the Differential Equations Case of Distinct Eigenvalues Case of Repeated Eigenvalues A Special Case: A = I Stochastic Decomposition Property Laplace Stieltjes Transform Method Sample Path Method Uniform Limiting Distribution Examples A Two-State Example A Machine Shop Example A Basic Production-Inventory Model Introduction The Model Optimal Order Quantity Stochastic EOQ Theorem Minimum Cost Rate vii

8 3.4 Inventory Model with Backlogging Cost Rate Calculation of the (r, q) Policy Optimal (r, q) Policy Newsboy Solution for the Optimal r for a Given q Stochastic EOQ Policy with Backlogging A Numerical Example Environment-Dependent Order Quantities Introduction Piecewise Function Method Sample Path Decomposition Method Laplace-Stieltjes Transform Method The Cost Model A Numerical Example Stochastic Leadtimes Introduction Serial Processing System Parallel Processing System General Order Processing viii

9 5.5 Selective Lost Sale Model The Cost Model Cost Rate Calculation Newsboy Solution for the Optimal Reorder Point A Numerical Example Limiting Distribution Optimal Ordering Policy Optimal Production Rate Sensitivity Analysis Conclusions and Future Research Conclusions Future Research Model with Semi-Markov Process as Background process Environment-Dependent Order Quantities and Reorder Points Numerically Stable Methods for the Stochastic Leadtime Model in Serial Processing System Environment-Dependent Ordering Policies with Stochastic leadtimes Bibliography 117 ix

10 LIST OF FIGURES 2.1 A sample path of the (buffer level, environment state) process Decomposition of the X(t) process Correspondence of the processes X(t), Z(t), X 1 (t), Y 0 (t) and Z 0 (t) Correspondence of the processes X(t), Z(t), X 1 (t), Y 1 (t) and Z 1 (t) Limiting distribution when r > d Limiting distribution when r < d The steady-state ccdf The steady-state pdf The optimal order quantity vs. production rate The minimum total cost and the optimal production rate The inventory level process when allowing backlogging The optimal order quantity vs. production rate The optimal reorder point vs. production rate The optimal order-up-to level q + r vs. production rate The minimum cost vs. production rate A sample path of X(t) and Z(t) with environment-dependent order quantities x

11 4.2 Piecewise function method Sample path decomposition method The optimal order quantities vs. production rate Sample paths of P (t) and X(t) with stochastic leadtimes The steady-state ccdf of the P (t) process The steady-state pdf of the P (t) process The steady-state ccdf of X(t) and P (t) The steady-state pdf of X(t) and P (t) The optimal order quantity vs. production rate (varying n) The optimal reorder point vs. production rate (varying n) The minimal cost vs. production rate (varying n) The optimal order quantity vs. production rate (varying ν) The optimal reorder point vs. production rate (varying ν) The minimum cost vs. production rate (varying ν) The optimal order quantity vs. production rate (varying N) The optimal reorder point vs. production rate (varying N) The probability that there are N outstanding orders (varying N) The minimum cost vs. production rate (varying N) xi

12 LIST OF SYMBOLS a j Equation (2.17) and (4.27), Theorem 4.2, Theorem 4.6. a A row vector of a j s in Equation (4.27). a kj Defined in Equation (5.15) and (5.22). A Transition probability matrix [α ij ], Section 2.3. A (i) Defined in Theorem 4.3. Ā Defined in Equation (5.26). b Backorder penalty-cost rate, Section c b c h c o c p Steady-state backlogging cost rate. Steady-state holding cost rate. Steady-state ordering cost rate. Steady-state production cost rate. c j Coefficients in Equation (2.16). c (i) k Defined in Theorem 4.2. d Demand rate in Section d i Demand rate when the environment process is in state i, Section D(s) Defined in Theorem 4.6. e e = [1,..., 1] t. E(x) Expectation of the random variable x. Ẽ ii (s) Defined in Equation (4.31). f(j, x) Defined in Equation (2.33). F j (x) Limiting cdf of the inventory level process at state j, Equation (2.21). F j (s) LST of F j (x), Equation (2.22). F (s) A row vector of LST of F i (x) s, Equation (2.23). xii

13 g(i, x) Defined in Equation (2.36). G(x) G (x) G (x) = G(x) = [G 1 (x),..., G n (x)]. [ dg1 (x) dx,..., dgn(x) dx ]. G j (t, x) Defined in Equation (2.11). G j (x) Defined in Equation (2.12). G (i) j (x) Defined in Section 4.3. G (i) (x) Defined in Section 4.3. Ḡ (i) j (x) Defined in Section 5.1. Ḡ (i) (x) Defined in Section 5.1. Ḡ(x) Defined in Section 5.1. h Inventory holding-cost rate, Section 3.1. H j (x) Limiting Cumulative distribution function of the fluid level in state j, in standard fluid model without jumps, Section 2.2. H(x) [H 1 (x),..., H n (x)], Section 2.2. H (x) [ ] dh1 (x),..., dhn(x), Section 2.2. dx dx I (i) Defined in Equation (4.2). k Fixed set-up cost to place an order, Section 3.1. k 1 k 2 Fixed set-up cost to place an order from a regular supplier, Section 5.6. Fixed set-up cost to place an order from an emergency supplier, Section 5.6. L Defined in Equation (4.30). m + m 0 m m Number of background states with positive input rate. Number of background states with zero input rate. Number of background states with negative input rate. Number of background states with nonzero input rate. xiii

14 M (j) ii Defined in Equation (4.39). M (j) n A diagonal matrix diag(m (j) ii ). Number of the states of the environment processes. N Upper limit of the number of outstanding orders, Section 5.1. O(t) Number of outstanding orders at time t, Section 5.1. p (i) Defined in Section 4.3. p ik Defined in Section 4.3. p (k) (x) Defined in Section 4.3. p ik (x) Defined in Section 4.3. P, P = [p ik ], Section 4.3. P (t) Inventory position at time t. P (t) Defined in Equation (5.1). p 1 Purchasing cost rate, Section 3.3. Purchasing cost rate from a regular supplier, Section p 2 Production cost rate, Section 3.3. Purchasing cost rate from the emergency supplier, Section p 3 Production cost rate, Section q Order quantity. q i Order quantity when the order is place in state i. q ij Transition rate of the environment process from state i to state j. ˆq ij Defined in Equation (2.30). Q Q = [q ij ]. Q Defined in Equation (5.4), (5.12) and (5.19). r Reorder point. R Defined in Equation (5.25). xiv

15 R i Net input rate when the environment process is in state i. R i = r i d i. S i i-th order epoch, Section t Time variable. T First passage time T = inf{t 0 : X(t) = 0}, Equation (4.10). T 1 Defined in Section T 2 Defined in Section T 2n+1 Defined in Section T 2n+2 Defined in Section T j (x) Steady-state complementary cdf of the inventory level in the backlogging model, Equation (3.8). T (x) Vector of T j (x) s, Equation (3.8). u Production rate of one machine, Section x Inventory level variable. X(t) Inventory level at time t. X 1 (t) Defined in Section X 2 (t) Defined in Section Y Defined in Section Y 0 (t) Defined in Section Y 1,n Defined in Section Z(t) State of the environment process at time t. Z 0 (t) Defined in Section xv

16 α ij Defined in Section 2.3. β Defined in Theorem 2.3. β (i) β (i) = G (0)RI (i), Theorem 4.1. δ A small positive number, Section 2.4. δ ij δ ij = 1 if i = j, and 0 otherwise. Net demand rate in steady state, Section 3.2. η kj Defined in Equation (2.31). θ Defined in Section λ Repair rate of failed machine, Section 2.7. λ i i-th generalized eigenvalue, Equation (2.5). λ (k) i Defined in Section 5.3, and Equation (5.20). µ Failure rate of one machine, Section 2.7. ν Leadtime distribution parameter, Section 5.1. π Defined in Equation (2.1). ˆπ i Defined in Equation (2.32). π(j, x) Defined in Equation (2.35). τ(j, x) Expected sojourn time of the SMP in state (j, x), Section τ j τ j = τ(j, x), x, Section φ i The row vector (eigenvector) corresponding to λ i such that φ i (λ i R Q) = 0. φ 0 A constant row vector in the expression of G(x), Theorem 2.4. φ (i) 0 Defined in Theorem 4.2. φ (k) i Defined in Section 5.3, and Equation (5.20). Φ Defined in Equation (4.4). Φ (k) i Defined in Equation (5.14) and (5.21). xvi

17 ψ j (λ j R Q)ψ j = 0, Section 4.4. Ψ Ψ = R 1 Φ 1, Section 4.4. Ω State space of the environment process. Ω + Ω + = {i Ω : R i 0}. Ω Ω = {i Ω : R i < 0}. xvii

18 Chapter 1 Introduction In this thesis we study a type of production-inventory models that can be seen as a stochastic fluid-flow system. We consider a single product, single location problem. The system has production, demand, and external supply. As the environment evolves over time, the production and demand rates are piecewise constant functions determined by the exogenous environment process. When the production rate exceeds the demand rate, the inventory increases, and when the demand rate exceeds the production rate, it decreases. When needed, replenishment orders can be placed from external suppliers. The inventory under continuous review thus can be viewed as a fluid process that fluctuates according to the evolution of the underlying background process. We assume the external environment undergoes recurring changes in a stochastic fashion, and may be modeled as Markovian. For example, production rates and demand rates change due to weather, economy, competition, seasonal promotion, customer status, and forecasting, etc. Some other example are as described in Mitra (1988) where the author studied a producer and consumer problem in a machine shop where the production rate changes according to the number of working machines. There are costs to hold products in inventory, to backlog unsatisfied orders,

19 to purchase and to produce. There is also a fixed set-up cost every time an order is placed with an external supplier. Our objective is to find an optimal ordering and production policies that minimizes the long-run average cost. 1.1 Stochastic Fluid Models First in Chapter 2 we study a fluid model to establish the fundamental theory for the production-inventory system. We view the inventory level under continuous review as a fluid level process. When the buffer is empty the fluid level jumps to a predetermined level q instantaneously, and at the same time the environment state jumps to another state with a given probability (it may stay unchanged). Between two consecutive jumps the background process is a continuous time Markov chain. At the jump epoch the environment process jumps according to a transition matrix A. We first derive the stability condition for this system and then derive a set of first order non-homogeneous linear differential equations to describe the limiting behavior of the bivariate (buffer level, environment state) process. We also determine the boundary conditions and give explicit solutions to the differential equations. Particularly for a special case A = I, we use two parallel methods to obtain an interesting stochastic decomposition property: in steady state, the buffer content in the fluid model with jumps is the sum of two independent random variables, one of which has a uniform distribution over [0, q], and the other is the steady-state buffer level in the standard fluid model without jumps. We also consider a more specific case where the fluid input rate is always negative over all the environment states. In this case the fluid level has uniform distribution in steady state, and is independent of the environment state. Most of the classical research on stochastic fluid systems allows only continuous 2

20 change in the buffer content. Under that assumption, the limiting joint distribution of the (buffer level, environment state) process is computed as a solution of a set of ordinary differential equations in terms of the eigenvalues and eigenvectors of the underlying system. There are also studies about fluid models where instantaneous jumps occur when the environment state changes and the size of the jump depends on the state of the environment (see Kulkarni, Tzenova and Adan (2005), Miyazawa and Takada (2002) and Sengupta (1989)). Another related model is the so called clearing system (see El-Taha (2002), Serfozo and Stidham (1978) and Whitt (1981)). A clearing system can be regarded as the reverse of our model. In a clearing system the fluid process jumps to zero when it reaches a certain positive level. However, there is no explicit environment process in these models. The paper that comes closest to our analysis is Berman, Stadje and Perry (2006) where the authors consider a two state CTMC as the environment process. The methodology of their analysis is different with ours. 1.2 Stochastic Inventory Control Problems Beginning from Chapter 3, we study the production-inventory problem using the theories developed in Chapter Basic EOQ Model In Chapter 3 we start from the basic model where there is no backlogging and zero leadtime, i.e., when the inventory on hand is zero, a supplementary order is placed and arrives instantaneously, and the order size q is independent of the environment state when the order is placed. 3

21 In a deterministic setting with constant demand rate, the classical Economic- Order-Quantity (EOQ) model describes the trade-off between the constant set-up cost and the variable holding cost. The earliest work on this is Harris (1913). For a modern review of the determinist models, see Zipkin (2000). In this thesis, we establish the stochastic EOQ theorem that shows in a CTMC environment the standard deterministic EOQ formula remains optimal if we replace deterministic demand rate by the expected net demand rate in steady state. In addition to the ordering policy, we also consider the optimal production policy: choose the optimal production capacity that achieves the best combination of outsourcing and inhouse-production. We show this mainly with numerical results. Interestingly, the optimal policy does not suggest always depending on inhouse-production, even if the production cost is less than the outsourcing price Backlogging EOQ Model Later in Chapter 3, from Section 3.4 we extend the basic model to allow backlogging: an external order is not placed until the inventory level reaches the preset reorder point r. We derive the optimal ordering-production policy, which achieves the tradeoff point of production cost, fixed ordering cost, holding cost and backlogging cost. Particularly, when production is always less than the demand rate (for example, the system does not make its own product), we prove that the optimality of the deterministic EOQ formula with backlogging still holds in this stochastic environment if one replaces the deterministic demand rate by the expected net demand rate in steady state. In the literature, dynamic control of inventory systems have been classified as periodic review models and continuous review models. The continuous review model 4

22 can be further classified according to the demand and production are discrete or continuous. (1) Discrete demands and production: Poisson demands in the continuous review models are studied in depth, see Scarf (1958), Karlin and Scarf (1958), Galliher et al. (1959), and Morse (1958). These early papers are reviewed in Scarf (1963). Poisson demand is generalized by Finch (1961), Rubalskiy (1972a,b) and Sivazlian (1974) to unit demands arriving at epochs following a renewal process. Song and Zipkin (1993) considered the case of Markov modulated Poisson demands. (2) Continuous fluid models: Berman and Perry (2004) studied a fluid model where the production and demand rates depend on the inventory level. Browne and Zipkin (1991) studied a model with continuous demand driven by a Markov process, which can be regarded as a special case of the model in this thesis. To our knowledge, although there are papers studying similar problems, none includes the result about the explicit Stochastic EOQ theorem presented in this thesis Environment-Dependent Order Quantities In Chapter 4 we relax the assumption that the order quantity is predetermined, i.e., the order quantity is allowed to depend upon the environmental state when the order is placed. For example, if we can observe the environmental state and can base our inventory replenishment decisions on that information, the policy that allows the order quantities to depend on the state of the environment is certainly no worse than the simple reorder-point/order-quantity policy. We use three methods to derive the limiting distributions of the inventory level: 5

23 (1) Consider appropriate non-overlapping intervals of the inventory level and within each interval derive the differential equations following the methodology of Chapter 2. Then the final limiting distribution is a piecewise function consisting of the functions derived in all these intervals. (2) Decompose the sample path of the inventory level into different cycles and reduce this problem to the basic fluid model of Chapter 2 in individual cycles. Then the overall limiting distribution is a weighted average of the limiting distribution functions in all cycles. (3) Consider individual intervals of the inventory level as in (1), but use Laplace- Stieltjes transforms instead of solving differential equations to obtain the limiting distribution function. Then based on the limiting distribution of the inventory level, we derive the longrun average cost and hence determine the optimal environment-dependent order quantities. In the literature Berman, Stadje and Perry (2006) studied a similar model with a two-state random environment. They consider order quantities that depend on the state of the environmental state and derive the optimal order quantities to maximize the system revenue. However, their calculation of optimal order quantities is based on the explicit results of the steady-state distribution of this two-state system. When the background has more than two states, their method becomes impractical. The general fluid EOQ models with multiple order quantities studied in this thesis seem to be new. 6

24 1.2.4 Stochastic Leadtimes In Chapter 5 we extend the model further to allow stochastic leadtimes. Three order processing fashions are considered: (1) Orders are processed sequentially, and hence orders never cross in time. Interarrival times between orders are i.i.d. exponential. (2) Orders are processed in parallel fashion, and leadtimes are i.i.d. exponential random variables. So orders can cross in time. (3) Inter-arrival times of the outstanding orders have exponential distributions whose parameters depends on the number of outstanding orders. This generalizes the previous two cases. We assume there exists an upper limit N of the number of outstanding orders. When there are N outstanding orders and the inventory position decreases to the reorder point again, we either obtain an emergency order instantaneously with higher ordering costs, or lose sales with the penalty costs. If the limiting probability that the number of outstanding orders is N is very small, this provides a good approximation to the models with no upper limit on the number of outstanding orders We derive the optimal ordering-production policy which minimizes the sum of the production cost, fixed ordering cost, holding cost, backlogging cost, and emergency ordering cost (or lost-sale penalty). Minimum cost and limiting distribution under the optimal policy are also calculated. In the literature, there is a sizeable body of work on inventory systems with stochastic leadtimes and Markov modulated demands. However, most of this literature is concerned with Markov modulated Poisson process models of demands. An 7

25 extensive review of this literature is given in Zipkin (2000). As far as we know, there is very little work on Markov modulated fluid models in the context of the productioninventory systems. One relevant work is that of Browne and Zipkin (1991), where the authors assume continuous stochastic demand, but no production. 8

26 Chapter 2 Fluid Model 2.1 Introduction In this chapter we study a stochastic fluid-flow system consisting of a single infinite capacity buffer. The buffer content increases or decreases according to a fluid-flow rate modulated by an environment which is a stochastic process with finite state space. Whenever the buffer is empty, it is refilled to a predetermined level instantaneously, and at the same time the environment state jumps to another state with a given probability (it may stay unchanged). Figure 2.1 illustrates a sample path of the (buffer level, environment state) process. Our primary motivation for considering this model is to provide fundamental theories to study a production-inventory system modulated by a Markovian environment. For example, the fluid process can be viewed as the inventory level under continuous review. The environment process represents the background state, for example, production or sales seasons. A jump in the fluid level represents an external order placement or order arrival, and the transition of the background state at the jump point can be a result of repairs of production facility, etc.

27 Figure 2.1: A sample path of the (buffer level, environment state) process. The outline of this chapter is as follows. In section 2.2, we present some preliminary results about the standard fluid model without jumps. In section 2.3 we describe the model with jumps in detail and derive the stability condition. In section 2.4 we derive a system of first order non-homogeneous linear differential equations for the limiting distribution of the bivariate (buffer level, environment state) process. We also determine the boundary conditions needed to solve those differential equations. In section 2.5, we derive explicit solutions to the differential equations. An interesting stochastic decomposition property is given in Section 2.6 about a special case where the background state does not change at jump epochs: in steady state, the buffer content in the fluid model with jumps is the sum of two independent random variables, one of which has a uniform distribution over [0, q], and the other is the steady-state buffer level in the standard fluid model without jumps. We also consider a more specific case where the fluid input rate is always negative over all the environment states. In this case the fluid level has uniform distribution in steady state, and is independent of the environment state. In section 2.7, we illustrate our methodology 10

28 with an analytic example as well as a numerical one. 2.2 The Standard Fluid Model In this section, we present some preliminary results about the standard fluid model with infinite capacity buffer. See the survey paper Kulkarni (1997) for an extensive overview of the research in this area. Let X(t) be the fluid level in the buffer at time t. The rate of change of the fluid level is modulated by a continuous time Markov chain {Z(t), t 0} on a finite state space Ω = {1, 2,..., n} with generator matrix Q = [q ij ]. As long as Z(t) is in state i, the fluid level process {X(t), t 0} changes at rate R i. Note that R i may be either negative or positive. Let π = [π 1, π 2,..., π n ] be the limiting distribution of the {Z(t), t 0} process, i.e., π is the unique solution to πq = 0, n π i = 1. i=1 The system is stable if and only if the expected input rate is negative, i.e., (2.1) n π i R i < 0. i=1 Let R = diag(r 1,..., R n ) be the diagonal n n matrix with the input rate R i as the ith entry on the diagonal. Let e = [1,..., 1] t be an n 1 column vector of ones. Then the stability condition can be written in matrix form as follows πre < 0. (2.2) 11

29 When the stability condition (2.2) holds, the following limits exist: H j (x) = lim t P {X(t) x, Z(t) = j}, x 0, j Ω. Let H(x) = [H 1 (x),..., H n (x)], and [ H dh1 (x) (x) = dx,..., dh ] n(x). dx The next theorem gives the differential equations satisfied by H(x). Theorem 2.1. Assume the stability condition (2.2) holds. The vector H(x) satisfies H (x)r = H(x)Q, x 0. (2.3) The boundary conditions are given by H j (0) = 0, j : R j > 0, (2.4a) H( )e = 1. (2.4b) Let (λ, φ) be a generalized (eigenvalue, eigenvector) pair that solves φq = λφr. (2.5) 12

30 Let Ω + = {i Ω : R i > 0}, (2.6) Ω 0 = {i Ω : R i = 0}, (2.7) Ω = {i Ω : R i < 0}, (2.8) and m + = Ω +, m 0 = Ω 0 and m = Ω. It is known that the number of eigenvalues that satisfy Equation (2.5) is m = m + + m (counting multiplicities). When the stability condition holds, one eigenvalue is 0, m + have negative real part, and m 1 have positive real part. We index the eigenvalues so that λ 1,..., λ m+ have negative real parts, λ m+ +1 = 0, and λ m+ +2,..., λ m have positive real parts. It is easy to see that φ m+ +1 = π is a valid eigenvector corresponding to the eigenvalue 0. When the eigenvalues are all distinct, the solution to the differential equations in Theorem 2.1 is given by m + H(x) = a i e λix φ i + π, if x > 0, i=1 where the coefficients a 1,..., a m+ are given by the unique solution to the following system of m + linear equations: m + a i φ ij + π j = 0, j Ω +, i=1 where φ ij is the j-th element in φ i. Let H j (s) = 0 e sx dh j (x) 13

31 be the Laplace-Stieltjes transform (LST) of H j (x) and H(s) = [ H1 (s), H 2 (s),..., H n (s)]. Taking transforms of (2.3), and noticing that H(x) has a jump at 0 of size H(0), and a density H (x) for x > 0, we get H(s) = sh(0)r(sr Q) 1. (2.9) It follows that there is a unique vector H(0) satisfying conditions (2.4a) and (2.4b) that makes H(s) a valid LST of a vector of random variables. We shall use this fact in deriving results in section The Fluid Model with Jumps Now we describe a fluid-flow model with infinite capacity buffer that we analyze in this chapter. As before, let X(t) be the fluid level in the buffer at time t. The rate of change of the fluid level is modulated by a stochastic process {Z(t), t 0} on a finite state space Ω = {1, 2,..., n}. As long as Z(t) is in state i, the fluid level process {X(t), t 0} changes at rate R i. When X(t) reaches zero it jumps to a predetermined level q instantaneously. Let S 0 = 0 and S k be the kth jump time. We assume that over (S k, S k+1 ) the process {Z(t), t (S k, S k+1 )} behaves as an irreducible CTMC on Ω with generator matrix Q = [q ij ]. Furthermore, when the {X(t), t 0} process jumps at time S k, the {Z(t), t 0} process changes instantaneously with probability α ij defined as follows: α ij = P {Z(S k +) = j Z(S k ) = i}, i, j Ω. 14

32 Let A = [α ij ]. It is clear that {(X(t), Z(t)), t 0} is a bivariate Markov process. Next we derive the condition when this process is stable, i.e., it has a limiting distribution. Let π be as in Equation (2.1). Note that π is not the limiting distribution of Z unless A = I. Theorem 2.2. The process {(X(t), Z(t)), t 0} is stable if and only if n π i R i < 0. (2.10) i=1 Proof. Let S k be the k-th jump epoch in the {X(t), t 0} process, with S 0 = 0. Let Z k = Z(S k +). It is easy to see that {Z k, k 0} is a DTMC on state space Ω. Since Q is assumed to be irreducible it can be seen that {Z k, k 0} has a single closed communication class Ω Ω that is positive recurrent. Without loss of generality, suppose (X(0), Z(0)) = (q, i) for some i Ω. Let N = min{k 0 : Z k = i}. It is clear that X(S N ) = q, Z(S N ) = i and that {(X(t), Z(t)), t 0} is a regenerative process that regenerates at time S N. Thus from the theory of the regenerative process (see Heyman (1982)), the limiting distribution of the {(X(t), Z(t)), t 0} process exists if E(S N ) <. Since Z(0) = i Ω, it follows that N is the number of steps needed by the {Z k, k 0} process to go from state i to state i. Since Ω is finite, E(N) <. Now from Kulkarni (2002), it follows that E(S 1 X(0) = x, Z(0) = j) < n if and only if π i R i < 0. Now, i=1 E(S N ) E(N) max j Ω {E(S 1 X(0) = q, Z(0) = j)} <. 15

33 This proves the theorem. 2.4 Differential Equations for the Limiting Distribution Let G j (t, x) = P {X(t) > x, Z(t) = j}, x 0, t 0, j Ω. (2.11) Assume the stability condition (2.10) holds so that the following limits exist: G j (x) = lim t P {X(t) > x, Z(t) = j}, x 0, j Ω. (2.12) In this section we show how to compute G(x) = [G 1 (x),..., G n (x)]. (2.13) We use the notation [ G dg1 (x) (x) = dx,..., dg ] n(x). dx The next theorem gives the differential equations satisfied by G(x). Theorem 2.3. Assume the stability condition (2.10) holds. The limiting distribution G(x) is continuous on [0, ) and is a piecewise differentiable function on (0, q) and (q, ). It satisfies G (x)r = G(x)Q + β, 0 < x < q, (2.14a) G (x)r = G(x)Q, x > q, (2.14b) 16

34 where the row vector β is given by β = G (0)RA. The boundary conditions are given by G( ) = 0, (2.15a) G j (q + ) = G j (q ), j / Ω 0, (2.15b) G j(0) = 0, j Ω +, (2.15c) G(0)e = 1. (2.15d) Proof. The differential equations follow from the standard derivation of Chapman Kolmogorov equations for Markov processes. We assume at time 0, (X(0), Z(0)) is in steady-state, i.e., for all state j Ω, P {X(0) > x, Z(0) = j} = G j (x). First consider the x < q case. We consider a time interval [0, δ] where δ > 0. During [0, δ], the {Z(t), t 0} process behaves like a usual CTMC with generator matrix Q if the {X(t), t 0} process does not hit zero. Otherwise the {Z(t), t 0} process changes state according to the matrix A and the {X(t), t 0} process jumps to q. Thus we have G j (x) = P {X(δ) > x, Z(δ) = j} = n P {X(δ) > x, Z(δ) = j X(0) > x R i δ, Z(0) = i} P {X(0) > x R i δ, Z(0) = i} i=1 + P {X(δ) > x, Z(δ) = j X(0) R i δ, Z(0) = i} P {X(0) R i δ, Z(0) = i} i Ω 17

35 When X(0) > x R i δ, we get P {X(δ) > x, Z(δ) = j X(0) > x R i δ, Z(0) = i} = P (Z(δ) = j Z(0) = i) = δ ij + q ij δ + o(δ), where δ ij = 1 if i = j, and 0 otherwise. Also note that when X(0) R i δ, P {X(δ) > x, Z(δ) = j X(0) R i δ, Z(0) = i} = α ij + O(δ), where O(δ) is a function of δ that goes to 0 as δ goes to 0. Using the fact that O(δ)(G i (0) G i ( R i δ)) = o(δ), we get the following: G j (x) = n (δ ij + q ij δ) G i (x R i δ) + α ij (G i (0) G i ( R i δ)) + o(δ) i=1 i Ω = n G j (x R j δ) + q ij δg i (x R i δ) + α ij (G i (0) G i ( R i δ)) + o(δ). i=1 i Ω Rearrange and divide both sides by δ to get G j (x) G j (x R j δ) δ = n q ij G i (x R i δ) + G i (0) G i ( R i δ) α ij + o(δ). δ i Ω i=1 Letting δ 0, we get G j(x)r j = n G i (x) q ij + i=1 i Ω α ij R i G i(0). This shows that G(x) is differentiable over (0, q). Later we shall show that G j(0) = 0 if j Ω +, (boundary condition (2.15c)). Hence we get Equation (2.14a), with β j = n G i(0)r i α ij. i=1 Now consider the x > q case. When x > q we do not need to consider the jumps 18

36 in the {X(t), t 0} process from 0 to q. Thus G j (x) = P {X(δ) > x, Z(δ) = j} = n P {X(δ) > x, Z(δ) = j X(0) > x R i δ, Z(0) = i} P {X(0) > x R i δ, Z(0) = i}. i=1 Following the same steps as in x < q case, we get Equation (2.14b). This also proves that G(x) is differentiable for x (q, ). As for boundary conditions, Equation (2.15a) follows because G(x) is the complementary distribution function of the fluid level in steady state. The boundary condition (2.15b) for all states j / Ω 0 is obvious from the fact that there is no probability mass at (q, j), i.e., G j (x) is continuous at x = q, if j / Ω 0. Equation (2.15c) holds because 1/(G j(0)r j ) can be seen to be the expected time between two consecutive visits by the {(X(t), Z(t)), t 0} process to the state (0, j). If j Ω +, this mean time is infinity. Hence G j(0) = 0 when j Ω +. From the definition of G j (x) we have Therefore, G j (0) = lim t P {X(t) [0, ), Z(t) = j}. n G j (0) = 1, which is Equation (2.15d). j=1 2.5 Solution to the Differential Equations In this section, we give the solution to the differential equations (2.14a) and (2.14b). We shall treat the cases with distinct eigenvalues and repeated eigenvalues separately. 19

37 2.5.1 Case of Distinct Eigenvalues Assume that all eigenvalues are distinct and hence the eigenvectors φ i s are linearly independent. The next theorem gives the main result. Theorem 2.4. The solution to the differential equations in Theorem 2.3 is given by m G(x) = c i e λix φ i + c 0 xπ + φ 0, 0 x q, (2.16) i=1 m + G(x) = a i e λix φ i, x > q, (2.17) i=1 where the coefficients a 1, a 2,..., a m+, c 1, c 2,..., c m, c 0 and the vector φ 0 are given by the unique solution to the following system of linear equations: m c i φ i QA + c 0 πr(a I) + φ 0 Q = 0, (2.18a) i=1 m + a i e λiq φ ij i=1 i=1 m c i e λiq φ ij c 0 qπ j φ 0j = 0, j / Ω 0, (2.18b) m c i λ i φ ij + c 0 π j = 0, j Ω +, (2.18c) i=1 ( m ) c i φ i + φ 0 e = 1, (2.18d) i=1 φ 0 e = 1, (2.18e) where φ ij is the j-th element in φ i, i = 0, 1,..., m. Proof. In section 2.2 we see that the homogenous equations (2.14b) have solutions of form G(x) = m c i e λix φ i. i=1 20

38 It can be shown that the nonhomogeneous equations (2.14a) have solutions of form G(x) = m c i e λix φ i + c 0 xπ + φ 0 i=1 if and only if c 0 xπ + φ 0 is a particular solution to (2.14a). Using G(x) = c 0 xπ + φ 0 in (2.14a), we get c 0 πr = c 0 xπq + φ 0 Q + β = φ 0 Q + β. The last equation holds because πq = 0. Substituting ( m ) β = G (0)RA = c i λ i φ i + c 0 π RA i=1 and noting that λ i φ i R = φ i Q, we obtain c 0 πr = φ 0 Q + m c i φ i QA + c 0 πra, i=1 which can be rearranged to get Equation (2.18a). When x > q, G(x) has a solution m + of the form G(x) = a i e λix φ i (Note that boundary condition 2.15a implies that the i=1 coefficient a i has to be zero when Re(λ i ) 0). Because there is no probability mass in (q, j) for j / Ω 0, the boundary condition in Equation (2.15b) reduces to m + a i e λiq φ ij = i=1 m c i e λiq φ ij + c 0 qπ j + φ 0j, j / Ω 0, (2.19) i=1 Rearranging (2.19) we get (2.18b). Equation (2.18c) and (2.18d) follow directly from boundary conditions (2.15c) and (2.15d). The total number of unknown coefficients is m + + m + n + 1. Notice that the number of independent equations in (2.18a) is n 1, since the rank of the matrix Q is n 1; the number of independent equations is m in (2.18b), and m + in (2.18c). 21

39 Including Equation (2.18d) we have m + + m + n independent equations satisfied by m + + m + n + 1 coefficients. Since any particular solution will work, we use Equation (2.18e) to determine a unique particular solution. Thus we have as many equations as unknowns Case of Repeated Eigenvalues When there are repeated eigenvalues we solve this problem using generalized eigenvectors. Let (λ 1, φ (1) 1 ), (λ 2, φ (2) 1 ),, (λ K, φ (K) 1 ) be K solutions to Equation (2.5), and λ 1, λ 2,...λ K are K distinct eigenvalues. Assume λ 1,...λ K+ have negative real part, λ K+ +1 = 0, and λ K+ +2,...λ K have positive real part. Let n i be the multiplicity of the K + K eigenvalue λ i. Clearly n i 1 and n i = m +, n i = m 1. i=1 i=k + +2 The general solution to the homogeneous equations G (x)r = G(x)Q is given by G(x) = K i=1 n i e λ ix c (i) j j=1 j k=1 x j k (j k)! φ(i) k, (2.20) where c (i) j s are constant coefficients, and φ(i) s are generalized eigenvectors satisfying φ (i) k Q = λ iφ (i) k R + φ(i) k 1 R, k = 2,..., n i. Theorem 2.5. The solution to the differential equations in Theorem 2.3 is given by G(x) = K i=1 K + G(x) = i=1 n i e λ ix c (i) j j=1 n i e λ ix j=1 a (i) j j k=1 k=1 k x j k (j k)! φ(i) k + c 0xπ + φ 0, if 0 x q, j x j k (j k)! φ(i) k, if x > q, 22

40 where the coefficients a (i) j s, c(i) j to the following system of linear equations: K + e λ iq i=1 n i j=1 a (i) j ( K i=1 j k=1 n i λ i j=1 s, c 0 and the vector φ 0 are given by the unique solution c (i) j φ(i) j + q j k (j k)! φ(i) kl K i=1 n i j=2 c (i) j φ(i) j 1 n i e λ iq c (i) j j=1 ( K i=1 n i λ i j=1 ) j k=1 RA + c 0 πr(a I) + φ 0 Q = 0, q j k (j k)! φ(i) kl c 0 qπ l φ 0l = 0, l / Ω 0, ) c (i) j φ(i) jl + n i i=1 j=2 c (i) j φ(i) j 1,l j=1 + c 0 π l = 0, l Ω +, ( K ) n i c (i) j φ(i) j + φ 0 e = 1, φ 0 e = 1, where φ (i) kl is the l-th element in φ (i) k. Proof. Follow the same lines as in the proof of Theorem 2.4 with only changes in the general solution to the homogeneous equations. Remark. When dealing with large matrices, the generalized eigenvectors are often numerically difficult to compute. There are alternative methods that are numerically better behaved to evaluate the general solution to the homogeneous equations, e.g., Putzer (1966). 2.6 A Special Case: A = I In this section we consider a special case where the background process state does not change when the fluid level jumps to q, i.e., the case A = I. 23

41 2.6.1 Stochastic Decomposition Property When A = I, as mentioned before, there exists an interesting stochastic decomposition property of the limiting distribution of the {(X(t), Z(t)), t 0} process, which says in steady state the buffer content in the fluid model with jumps is the sum of two independent random variables: a U(0, q) random variable and the buffer content in a fluid model with no jumps. Next we shall prove this decomposition property from two different aspects Laplace Stieltjes Transform Method Let F j (x) = lim t P {X(t) x, Z(t) = j}, x 0, j Ω, (2.21) and F j (s) = 0 e sx df j (x) (2.22) be the Laplace Stieltjes transform (LST) of F j (x), and F (s) = [ F1 (s), F 2 (s),..., F n (s)]. (2.23) The next theorem gives the stochastic decomposition property of the limiting distribution of the {(X(t), Z(t)), t 0} process. Theorem 2.6. Suppose A = I and the stability condition (2.10) holds. Then F (s) = 1 e sq sq H(s), (2.24) where H(s) is the LST of the limiting distribution function of the standard fluid model without jumps, given by Equation (2.9). 24

42 Proof. Since A = I and the stability condition (2.10) holds, we have G(0) = π and hence G(x) = π F (x). Clearly, F (0) = 0. Thus F (x)r = F (x)q + F (0)R, 0 x q i, F (x)r = F (x)q, x > q i. Thus we have q 0 e sx F (x)dx R = q 0 e sx (F (x)q + F (0)R)dx and e sx F (x)dx R = q q e sx F (x)qdx. Thus F (s)r = 0 e sx F (x)qdx + = 1 s F (s)q + 1 s q 0 e sx F (0)Rdx ( 1 e sq ) F (0)R. (2.25) Rearranging Equation (2.25), we get F (s) = 1 e sq sqf (0)R(sR Q) 1. (2.26) sq Recalling from section 2.2, for the standard fluid model without jumps, we have H(s) = sh(0)r(sr Q) 1. (2.27) We have seen that there is a unique vector H(0) satisfying Equations (2.4a) and (2.4b) that makes H(s) in Equation (2.27) a valid LST of a random vector. From 25

43 equation (2.26) it is clear that sqf (0)R(sR q) 1 must be a valid LST of a random vector since (1 e sq )/(sq) is the LST of U(0, 1) random variable. Since the boundary conditions of Equation (2.15c) implies that F j(0) = 0 if j Ω +, which are the same conditions satisfied by H(0) (see Equation (2.4a)), we must have qf (0)R = ch(0)r, for some constant c. The condition F ( )e = 1 implies that c = 1. This proves our result. Remark. Theorem 2.6 indicates that in steady state the buffer content in the fluid model with jumps is the sum of two independent random variables: a U(0, q) random variable and the buffer content in a fluid model with no jumps. Interestingly, similar property has been observed in queuing models with server vacations. See Fuhrmann (1984), Fuhrmann (1985) and Shanthikumar (1986) Sample Path Method We begin by decomposing the {X(t), t 0} process into two components. Let S 0 = 0, X(0) = q and S i be the i-th order point (i 1). Define X 1 (t) = min S n u t {X(u)}, S n t < S n+1 and X 2 (t) = X(t) X 1 (t). Figure 2.2 illustrates the sample paths of the original {X(t), t 0} process and the two resulting processes {X 1 (t), t 0} and {X 2 (t), t 0}. The following two theorems together state the stochastic decomposition properties represented by these 26

44 Figure 2.2: Decomposition of the X(t) process. component processes {X 1 (t), t 0} and {X 2 (t), t 0}. Theorem 2.7. The process {X 2 (t), t 0} is independent of q. Proof. Assume that Z(0) Ω and define T 1 = min{t 0 : Z(t) Ω + Ω 0 }. Regardless of the value of q, X(t) always decreases over (0, T 1 ), except for possible jumps of size q when it hits zero. Thus X 2 (t) is zero over (0, T 1 ). T 1 is independent of q and hence {X 2 (t), t [0, T 1 )} is independent of q. 27

45 Now define T 2 = min{t > T 1 : X(t) = X(T 1 )}. Note that T 2 is also independent of q, X 2 (T 1 ) = X 2 (T 2 ) = 0 and X 2 (t) > 0 for t (T 1, T 2 ). The sample path of {X(t), t (T 1, T 2 )} is independent of q, since X(t) never reaches 0 for any t (T 1, T 2 ). Thus the sample path of {X 2 (t), t (T 1, T 2 )} is independent of q. Define T 2n+1 = min{t T 2n : Z(t) Ω + Ω 0 }, and T 2n+2 = min{t T 2n+1 : X(t) = X(T 2n+1 )}. Since {X 2 (t), t 0} goes through these two cycles alternately over (T 2n, T 2n+1 ) and (T 2n+1, T 2n+2 ) independently, it is clear that {X 2 (t), t 0} is independent of q. Theorem 2.8. The limiting distribution of the process {X 1 (t), t 0} is uniform over (0, q). Proof. First note that the sample paths of {X 1 (t), t 0} have right derivative everywhere. Define I(t) = 0 if the right derivative of X 1 (t) is strictly negative at t, and I(t) = 1 if the right derivative of X 1 (t) is zero at t. Now lim P (X 1(t) x) t = lim t P (X 1 (t) x I(t) = 0)P (I(t) = 0) + lim t P (X 1 (t) x I(t) = 1)P (I(t) = 1). (2.28) Next we will show that lim P (X 1(t) x I(t) = ζ) = x/q, ζ {0, 1}. (2.29) t 28

46 First we construct two new processes {Y 0 (t), t 0} and {Z 0 (t), t 0} by eliminating the segments of the sample paths of {X 1 (t), t 0} and {Z(t), t 0} over the time intervals (T 2n+1, T 2n+2 ] for all n 0. The sample paths of the {Y 0 (t), t 0} and {Z 0 (t), t 0} processes corresponding to the sample paths of {X 1 (t), t 0} and {Z(t), t 0} are shown in Figure 2.3. From Figure 2.3 we can see that {Y 0 (t), t 0} can be thought of as a fluid model modulated by the stochastic process {Z 0 (t), t 0} with state space Ω. It can be seen that {Z 0 (t), t 0} is a CTMC with generator matrix ˆQ = [ˆq ij ], (i, j Ω ) given by ˆq ij = q ij + q ik η kj, i, j Ω, (2.30) k Ω + Ω 0 where η kj = P (Z(T 2n+2 ) = j Z(T 2n+1 ) = k), k Ω + Ω 0, j Ω. (2.31) Thus the {(Y 0 (t), Z 0 (t)), t 0} process satisfies the hypothesis of Theorem 2.9. Hence it follows that lim P (Y 0(t) x, Z 0 (t) = i) = x t q ˆπ i, (2.32) where ˆπ i is the steady-state probability of the CTMC with generator matrix ˆQ in state i. However, our construction of the Y 0 process implies that lim P (Y 0(t) x, Z 0 (t) = i) = lim P (X 1(t) x I(t) = 0). t t This proves Equation (2.29) for ζ = 0. Now for ζ = 1, we define Y 1,n = X 1 (T + 2n+1) and Z 1,n = Z(T + 2n+1) for n 0. Now construct a semi-markov process (SMP) {(Z 1 (t), Y 1 (t)), t 0} with embedded DTMC {(Z 1,n, Y 1,n ), n 0}, so that the n-th sojourn time of this SMP is given by 29

47 Figure 2.3: Correspondence of the processes X(t), Z(t), X 1 (t), Y 0 (t) and Z 0 (t). 30

48 T 2n+2 T 2n+1. Clearly the sample path of {Y 1 (t), t 0} is identical to the one obtained by eliminating the segments of the sample path of {X 1 (t), t 0} over the intervals (T 2n, T 2n+1 ] for all n 0. Figure 2.4 illustrates the sample paths of the {Y 1 (t), t 0} and {Z 1 (t), t 0} processes corresponding to the sample paths of {X 1 (t), t 0} and {Z(t), t 0} processes. Define f(j, x)dx = lim t P {Z(t) = j, x Y 1 (t) x + dx}. (2.33) According to the theory of SMP (see Kulkarni (1995)), where π(j, x)τ(j, x)dx f(j, x)dx = q, (2.34) π(k, y)τ(k, y)dy y=0 k Ω + Ω 0 π(j, x)dx = lim n P {Z 1,n = j, x Y 1,n x + dx}, (2.35) and τ(j, x) is the expected sojourn time of the SMP in state (j, x). Clearly τ(j, x) is independent of x, hence we denote τ(j, x) as τ j for all x. Let g(i, x)dx = lim t P {Z(t) = i, x Y 0 (t) x + dx}, (i Ω ). (2.36) From Equation (2.32), we see that g(i, x) = ˆπ i q, (i Ω ). (2.37) Hence using Equation (2.37), π(j, x) = g(i, x)q ij = 1 ˆπ i q ij. (2.38) q i Ω i Ω 31

49 Figure 2.4: Correspondence of the processes X(t), Z(t), X 1 (t), Y 1 (t) and Z 1 (t). 32

50 Substituting Equation (2.38) into (2.34), we have f(j, x) = 1 q i Ω q 1 y=0 q k Ω + Ω 0 = 1 q i Ω k Ω + Ω 0 ˆπ i q ij τ j ˆπ i q ik τ k dy i Ω ˆπ i q ij τ j. ˆπ i q ik τ k i Ω Thus the limiting probability density function of {Y 1 (t), t 0} process is given by f(x) = f(j, x) j Ω + Ω 0 = 1 ˆπˆq ij τ j q j Ω + Ω 0 i Ω ˆπˆq ik τ k i Ω k Ω + Ω 0 = 1 q. (2.39) Equation (2.39) indicates the limiting distribution of {Y 1 (t), t 0} is uniform over [0, q]. This proves Equation (2.29) for ζ = 1. Hence from (2.28) lim P (X 1(t) x) = x t q. This proves theorem 2.8. Remark. Using these two theorems we see that the limiting distribution of X(t) is a sum of two independent random variables: X 1 (t) is uniform and X 2 (t) is independent of q. 33

51 2.6.2 Uniform Limiting Distribution Now we study a more specific case where R i < 0 for all i Ω, and the background process state does not change when the buffer content jumps, i.e., A = I. Without loss of generality assume that X(0) = q. Then it is clear that X(t) [0, q], t 0. The next theorem gives the steady state distribution of X(t). Theorem 2.9. When R < 0 and A = I, G(x) = (1 1 x)π, x [0, q]. (2.40) q Proof. In this special case, the differential equations are reduced to G (x)r = G(x)Q + β, (2.41) where β = G (0)R, (2.42) with boundary conditions: G(q) = 0, G(0)e = 1. (2.43a) (2.43b) It is easy to see that (2.40) is the solution to the differential equation system (2.41) with boundary conditions (2.43a) and (2.43b). Remark. Theorem 2.9 implies that in steady state, the buffer content is uniformly distributed on [0, q], and is independent of the state of the environment. This is consistent with Theorem 2.6 since in this case the buffer content in the fluid model without jumps is zero with probability one in steady state. Similar results have been 34

52 observed by Browne and Zipkin (1991). 2.7 Examples A Two-State Example Consider a machine shop with only one machine. Whenever the machine is up, it produces items continuously at rate r, and it fails after an exp(µ) amount of time. If it is down, there is no production, and it takes exp(λ) amount of time to fix it. The machine is as good as new after repairs complete. The demand occurs at a constant rate d independent of the state of the machine. Whenever the inventory reaches zero, an external supply of amount q is ordered and arrives instantaneously. This produces a special case of the model in Section 2.3 with the following parameters: Q = λ λ µ µ, R = d 0 0 r d. The stability condition Equation (2.10) reduces to λ(r d) µd < 0. We consider two cases. Case 1: r > d. In this case, when the machine is up the production rate is greater than the demand rate. Thus the inventory hits zero only when the machine is down. We give explicit expressions of the limiting distribution in two sub-cases. (1) A = I. This implies the machine state does not change when placing an order 35

53 from the external supplier. The solution given in Theorem 2.4 reduces to G down (x) = (r d)π 2 qdθ (e θx e θ(x q) ) x > q, (r d)π 2 e θx π 1 qdθ q x + π 2 rµ (d r ) + π q (λ+µ) dθ 1 0 x q, G up (x) = π 2 qθ (eθx e θ(x q) ) x > q, π 2 qθ eθx π 2 q x π 2 rλ (d r + ) + π q (λ+µ) dθ 2 0 x q, where θ = λ(d r)+dµ d(d r), π 1 = µ λ+µ, and π 2 = (2) A = λ. λ+µ. This implies that we replace the machine instantaneously if it is down when we place an order from the external supplier. In this case the solution is given by G down (x) = G up (x) = λ(r d)e θx +dµe θ(x q) when x > q, dθ(r+q(λ+µ)) ( ) λ(r d) e θx + µx rπ 2µ π θd θd 1d + π 1 when 0 < x < q, 1 r+q(λ+µ) (r d)λe θx dµe θ(x q) (r d)θ(r+q(λ+µ)) when x > q, ( λ θ eθx + λx rπ 2λ θd 1d ) + π 2 when 0 < x < q. 1 r+q(λ+µ) Case 2: r < d. In this case the inventory can hit zero when the machine is either up or down. (1) A = I. This is a 2-state special case of section The solution on [0, q] is 36

54 given by G down (x) = G up (x) = ( 1 1 ) µ q x λ + µ, ( 1 1 ) λ q x λ + µ. (2) A = The solution on [0, q] is given by G down (x) = (d r)dµeθx + µ( rλ + d(λ + µ))e θq x + µe θq (dqµ (d r)(d qλ)), drµ e θq (qrλ(λ + µ) d(rµ + q(λ + µ) 2 )) G up (x) = d2 µe θx + λ( rλ + d(λ + µ))e θq x + e θq (q(d r)λ 2 + dµ(d + qλ)). drµ e θq (qrλ(λ + µ) d(rµ + q(λ + µ) 2 )) Now we use λ = 1, µ = 2, d = 1, q = 1. We display the steady-state complementary cumulative distribution function G up (x) + G down (x) and the density function f(x) = G up(x) G down (x) in Figure 2.5 and Figure 2.6 for the two cases when r = 0.5 < d and r = 2.5 > d. Note that the density functions in Figure 2.5 are discontinuous at x = q in this case. However the complementary cumulative distribution functions are always continuous. Figure 2.5: Limiting distribution when r > d. 37

55 Figure 2.6: Limiting distribution when r < d A Machine Shop Example Now we consider a machine shop that has n independent and identical machines, each behaving as described in section Each machine has its own repair person. Let Z(t) be the number of working machines at time t. Thus the environment process {Z(t), t 0} has n + 1 states, i.e., Ω = {0, 1,..., n}. Suppose the demand rate is directly proportional to the number of machines. To be specific, we have demand rate d i = n and production rate r i = i u for all i Ω, where u is the production rate of each working machine. The background state does not change when placing an order. We plot the steady-state complementary cumulative distribution functions (ccdf) and the probability density functions (pdf) when n = 3, d = 3, u = 2.5, λ = 1, µ = 2 and q = 3. 38

56 Figure 2.7: The steady-state ccdf. Figure 2.8: The steady-state pdf. 39

57 Chapter 3 A Basic Production-Inventory Model 3.1 Introduction Beginning from this chapter we shall study a type of production-inventory models that can be seen as the fluid model in Chapter 2. We consider a single product problem. The production and demand rates are piecewise constant functions determined by an underlying exogenous CTMC. When the production rate exceeds the demand rate, the inventory increases, and when the demand rate exceeds the production rate, it decreases. Thus the inventory under continuous review is a fluid process that fluctuates according to the evolution of the underlying background process. This characterizes the situations in which the external environment undergoes recurring changes in a stochastic fashion, and can be regarded as Markovian. We follow the classical reorder-point/order-quantity policy ((r,q) policy): when the inventory level (stock on hand minus back orders) decreases to the reorder point

58 r, a replenishment order of size q is placed from an external supplier. There are costs to hold products in inventory, to purchase and to produce. There is also a fixed set-up cost every time an order is placed with an external supplier. Our objective is to find the optimal (r, q) pair that minimizes the long-run average cost. In the literature, in a deterministic setting with constant demand rate, the classical Economic-Order-Quantity (EOQ) model describes the trade-off between the constant set-up cost and the variable holding cost (see Zipkin (2000)). In such a model the demand occurs continuously at a constant rate d and there is a holding cost h per item per unit time. When the inventory reaches zero, an order of size q is placed and it arrives immediately. It costs k to place an order. The optimal value of q (EOQ) that minimizes the holding plus ordering cost per unit of time is given by q = 2kd h. (3.1) We establish the stochastic EOQ theorem that shows in a CTMC environment the standard deterministic EOQ formula remains optimal if we replace deterministic demand rate by the expected net demand rate in steady state. In this chapter we first study a basic EOQ type model: no backlogging, and leadtime is zero. Section 3.2 models this inventory problem as a fluid-flow system. In Section 3.3 we prove the optimality of what we called stochastic version of the EOQ formula, which is derived by replacing the deterministic demand rate in the classical EOQ formula with the expected net demand rate in steady state. In Section we calculate the minimum cost under that optimal ordering policy, and obtain the optimal production rate which is the best combination between outsourcing and production. Section 3.4 extends the basic model to allowing backlogging. We derive the optimal ( reorder-point/order-quantity ) policy to achieve the trade-off point of fixed ordering cost, holding cost and backlogging cost under this stochastic circumstance. 41

59 For a given order quantity we show that the optimal reorder-point is given by the well-known newsboy solution. Particularly, in the special case where production is always less than the demand rate, we prove that the optimality of the deterministic EOQ formula with backlogging still holds in this stochastic environment. 3.2 The Model We study a production-inventory system in which the inventory level process {X(t), t 0} is modulated by a background process {Z(t), t 0}. We assume that {Z(t), t 0} is an irreducible CTMC on state space Ω = {1, 2,..., n} with rate matrix Q = [q ij ]. When Z(t) is in state i, the production occurs continuously at a constant rate r i, and demand occurs at rate d i. As long as Z(t) = i, {X(t), t 0} changes at rate R i = r i d i. When the inventory level reaches the order point r, an order is placed for the fixed amount q, the order quantity. Currently we assume leadtimes are zero, and no backlogging is allowed. This implies that it is optimal to set r = 0. We also assume all orders are of the same size q regardless of the state of the CTMC when the order is placed. This is an appropriate model when we can base our inventory replenishment decisions only on the inventory level and not on the state of the CTMC. This may be because knowledge of the state of the background CTMC is unavailable, or to simplify the ordering policies. (We will consider state dependent order sizes in Chapter 4.) When the inventory level reaches 0, it jumps instantaneously to q, but there is no change in the state of the CTMC at this jump epoch. Thus the {(X(t), Z(t))t 0} process can be seen to be a special case of the one studied in Chapter 2 with A = I. The stability condition is given by Equation (2.10). 42

60 Since A = I we know that π is the limiting distribution of Z(t). Thus n = π i R i (3.2) i=1 is the net expected demand rate. The condition of stability can be written as > 0. This makes intuitive sense, since if 0, there would be no reason for placing orders from an external supplier. 3.3 Optimal Order Quantity Next we consider the costs of operating the system. The total cost consists of three parts: holding cost, ordering cost, and production cost. We use the following notation: h: cost to hold one item in inventory for one unit of time; k: fixed set-up cost whenever an order is placed; p 1 : cost to purchase one item from the external supplier; p 2 : cost to produce one item. Let c h (q), c o (q) and c p (q) be the steady-state holding, ordering and production cost rates respectively as functions of the order quantity q. The total cost rate c(q) is given by c(q) = c h (q) + c o (q) + c p (q). (3.3) Next we shall calculate the optimal order quantity that minimizes the total cost rate. 43

61 3.3.1 Stochastic EOQ Theorem Theorem 3.1. Suppose > 0. Then the optimal order quantity q that minimizes the total cost rate c(q) is given by q = 2k h. (3.4) Proof. First we calculate c h (q). Recall in Section we have shown that when A = I, the limiting distribution of X(t) is a sum of two independent random variables: one is uniform (X 1 ) and the other is the steady-state buffer content in a standard fluid model without jumps (X 2 ). Thus E(X 1 ) = q and E(X 2 2) is independent of q. Thus c h (q) = he(x) = h(e(x 1 ) + E(X 2 )) = hq 2 + he(x 2). Next we calculate c o (q). From the results on renewal reward processes we get c o (q) = k + p 1 q E(S i S i 1 ), where S i is the ith order point (i 1). In steady state, the average net demand during a cycle time (S i, S i 1 ) has to be equal to the amount of the external supply. Hence we have E(S i S i 1 ) = q. 44

62 Thus c o (q) = (k + p 1q) q = k q + p 1. (3.5) Finally c p (q) = p 2 n π i r i, (3.6) i=1 and it is independent of q. Now, from Equation (3.3), the total cost rate is given by c(q) = c h (q) + c o (q) + c p (q) = hq 2 + k q + C, where C = he(x 2 )+p 1 +c p (q) is independent of q. Clearly, c(q) is a convex function of q, and it is minimized at q = 2k h. Remark. The optimal order quantity q of Equation (3.4) is the classical EOQ formula with the deterministic demand rate replaced by the steady-state expected net demand rate. A Numerical Example. Consider a machine shop as described in section We investigate the effect of the production rate increases on the optimal order quantity q. Consider a system with λ = 1, µ = 2, h = 5, k = 0.5, p 1 = 8 and p 2 = 5. Let u vary in (0, 3). We plot the optimal values of q s in Figure 3.1 for 1 n 5. Note 45

63 Figure 3.1: The optimal order quantity vs. production rate. that for a fixed n, the q decreases with u. This makes intuitive sense because as the production increases the net demand rate decreases. Note that the q reaches zero when u increases to 3. This is because the system is unstable for u 3 and hence we do not need to order from the external supplier. It should be noted that there are numerical difficulties when the parameter values make the rate R i in some states close to zero. We have simply avoided such values and used interpolation to produce the above graphs. This example is used repeatedly in this thesis, and this comment applies to all numerical experiments relating to it Minimum Cost Rate Given the optimal order quantity q, we are able to calculate the corresponding minimum cost rate: c(q ) = c h (q ) + c o (q ) + c p (q ). (3.7) 46

64 First we calculate holding cost rate: c h (q ) =he(x) =h x=0 G(x)dx e. Substituting the expression for G(x) given by Equation (2.16) and (2.17), we have ( ( q m ) c h (q ) = h c i e λix φ i + c 0 xπ + φ 0 dx + ( m+ x=0 i=1 x=q i=1 ) ) a i e λix φ i dx e. Recall that we have assumed that the eigenvalues λ 1,..., λ m+ have negative real parts, λ m+ +1 = 0, and λ m+ +2,..., λ m have positive real parts. Thus c h (q ) =h i m + +1 c i ( e λ i q 1 ) φ i + c 0q 2 m λ i 2 π + + q (φ 0 + c m+ +1φ m+ +1) i=1 a i e λix φ i e λ i Ordering cost rate and production cost rate can be obtained directly from Equation (3.5) and (3.6). Thus c o (q ) = k q c p (q ) = p 2 + p 1 n π i r i, A Numerical Example. Consider the machine shop example of Section Given the optimal order quantity q, we calculate the corresponding minimum cost rate. i=1 47

65 Figure 3.2: The minimum total cost and the optimal production rate. From Figure 3.2 we can see that under the optimal ordering policy, the corresponding minimum cost rate is also a convex function of the production rate u. Thus there exists one optimal production rate u that achieves the trade-off point between outsourcing and producing. Interestingly, even if the production cost rate p 2 is less than the outsourcing cost rate p 1, the optimal policy does not suggest us depending on inhouseproduction too much, in which circumstance the increase in holding cost due to the high production rate annihilates the advantage of the cost difference between the purchasing inhouse-production. Also note that although q 0 as u 3, the expected total cost does not approach zero. This is a consequence of the stochastic model, as opposed to the deterministic one. 48

66 3.4 Inventory Model with Backlogging In the previous sections we considered a model where we place an order as soon as the inventory on hand is zero. Many businesses find it practical to operate with planned backlogging. In this section we consider the same system as in the sections above, but allow backlogging, and assume that unsatisfied demands are fully backlogged. Let X(t) be the net inventory level at time t (i.e., the inventory on hand at time t - backorders at time t). We always use any inventory on hand to fill demands; backorders accumulate only when we run of stock entirely. Thus if X(t) is positive, it represents the amount of inventory on hand. If it is negative, it represents the negative of the amount of backorders at time t. Now besides all the costs occurring in the previous setting, there is also backlogging cost. We consider a policy under which we place an order of size q whenever the inventory level decreases to the reorder point r. We assume zero leadtimes, so the orders arrive instantiates. Clearly an optimal policy should have r 0 to reduce unnecessary holding cost. However, unlike in the deterministic set up, we may not have q > r. The net inventory level is always in (r, ). Figure 3.3 illustrates a typical sample path of the {X(t), t 0} process. Figure 3.3: The inventory level process when allowing backlogging. 49

67 Note that under this policy, the stability condition is the same as in (2.10). Let T j (x) = lim t P (X(t) > x, Z(t) = j). (3.8) The next theorem shows how to compute T (x) = [T 1 (x), T 2 (x),..., T n (x)]. Theorem 3.2. Let G(x)(x 0) be as in Theorem 2.3. Then T (x) = G(x r), x r. (3.9) Proof. Follows from the fact that the sample path of the inventory level process with backorder is identical to that without the backorder shifted down by r Cost Rate Calculation of the (r, q) Policy Now suppose it costs b to backlog one unit of demand for one unit of time. Let c b (r, q), c h (r, q), c o (r, q) and c p (r, q) be the steady state backlogging, holding, ordering and producing cost rates respectively as functions of the order quantity q and reorder point r. The total cost rate c(r, q) is thus given by c(r, q) = c h (r, q) + c b (r, q) + c o (r, q) + c p (r, q). (3.10) The next theorem gives the long-run average costs. 50

68 Theorem 3.3. Let G(x)(x 0) be as in Theorem 2.3. Then c h (r, q) = h G(y)dy e, (3.11) y= r c b (r, q) = br b r y=0 G(y)dy e, (3.12) c o (r, q) = k q + p 1, (3.13) c p (r, q) = p 2 n π i r i,. (3.14) i=1 Proof. First, for the holding cost rate, we get c h (r, q) = he(x + ) = h = h = h 0 t 0 0 lim P {X(t) > x}dx T (x)dx e G(x r)dx e. Letting y = x r, we get c h (r, q) = h G(y)dy e. y= r The backlogging cost rate is given by c b (r, q) = be(x ) = b = b 0 t 0 r = br b lim P {X(t) x}dx (T (r) T (x))dx e 0 r G(x r)dx e. 51

69 Letting y = x r, we get r c b (r, q) = br b G(y)dy e. y=0 Ordering cost rate and production cost rate are both independent of r, and hence can be obtained directly from Equations (3.5) and (3.6) Optimal (r, q) Policy Theorem 3.3 gives the long-run average costs in terms of the distribution function G( ). Thus we can find the optimal (r, q ) pair that minimizes the total cost c(r, q) Newsboy Solution for the Optimal r for a Given q The next theorem shows that for a given q, the corresponding optimal r is given by the well-known newsboy solution. Theorem 3.4. For a given q, the optimal r (q) is given by the unique solution to G( r) e = b h + b. (3.15) Proof. Since c o (r, q) and c p (r, q) are independent of r, we have r c(r, q) = r c h(r, q) + r c b(r, q). 52

70 Substituting c h (r, q) and c b (r, q) by their expressions in Equations (3.11) and (3.12), and noticing that G(x) is independent of r, we get c(r, q) = hg( r) e b + bg( r) e. r We also have 2 r 2 c(r, q) = hg ( r) e bg ( r) e 0. Hence c(r, q) is a convex function of r for a given q. Thus it is minimized when c(r, q) = 0. This yields Equation (3.15). r Unfortunately, proving joint convexity of c(r, q) is hard. Numerically, one can minimize the function of single variable c(r (q), q) to obtain the optimal order quantity q. Then the optimal reorder point is given by r = r (q ). There is one special case when q and r can be obtained analytically. We describe it below Stochastic EOQ Policy with Backlogging We consider a special case when R i < 0 for all i Ω. It follows from the results in Section and Theorem 3.2 that the limiting distribution of the {X(t), t 0} process is uniformly distributed on (r, q + r), and is independent of Z. Thus the cost rate is given by q+r x 0 c(r, q) = h 0 q dx b x r q dx + k q + p 1 + c p (r, q) = h q (q + r)2 + br2 q + k q + p 1 + c p (r, q) 53

71 It can be shown that c(r, q) is jointly convex function of (r, q), and is minimized at 2k(b + h) q = (3.16) ( hb ) h r = q. (3.17) b + h Equations (3.16) and (3.17) are identical to the well-known optimal (r, q) policy (see Zipkin (2000)) with deterministic demand rate. Here we have shown that it remains optimal in this stochastic settings if we replace the deterministic demand rate with the steady-state expected net demand rate A Numerical Example Consider the machine shop example of Section Suppose the backlogging cost rate is b = 8. We plot the optimal q s and r s when the number of machines is 1, 2,...,5 in Figures 3.4 and 3.5. Figure 3.4: The optimal order quantity vs. production rate. 54

72 Figure 3.5: The optimal reorder point vs. production rate. From Figures 3.4 and 3.5 we can see a few interesting points: (1) When production rate u < 1, this example falls into the special case R < 0. Thus (r, q ) is given by the deterministic policy Equations (3.16) and (3.17) with = n(3 u)/3. (2) As u increases to 3, the reorder point r goes to. This is contrary to the deterministic results. Actually, when u 3, more and more background states have positive net input rates, and for these states, the input rates are also larger and larger; in addition, for those states with negative input rates, the net output rates are smaller and smaller. As a result, even though the expected input rate is still negative, once the background changes to a state with positive input rate, the inventory level increases quickly, and the probability that the background changes to a state with negative input rate decreases. When such a state eventually is reached, the inventory level decreases, but slowly. Thus the optimal reorder point decreases in an effort to keep the inventory level close to 55

73 zero to decrease the holding cost. (3) The order quantity q is increasing when u 3. This is also inconsistent with the deterministic model. In this example, as long as u < 3, there are still states with negative input rates. As u 3, the r is further and further below zero. Thus the increasing q is a compensation to bring the inventory level back to zero at the ordering state (which must have a negative input rate) to reduce the backlogging cost. (4) As functions of u, q and r have different characteristics on q + r < 0 and q + r > 0. Figure 3.6 shows the graph of q + r as a function of u. Figure 3.6: The optimal order-up-to level q + r vs. production rate. Figure 3.6 shows that q + r decreases from a positive quantity to a negative quantity as u 3. From Equations (3.11) and (3.12), we can see that the holding cost rate and backlogging cost rate both have different expressions when r < q and r > q. Hence q and r are also piecewise corresponding to the piecewise cost functions. 56

74 Minimum Cost Rate. Consider the machine shop example of Section Given the optimal ordering policy (r, q ), we calculate the corresponding minimum cost rate. Figure 3.7: The minimum cost vs. production rate. Similar to the example without backlogging, in Figure 3.7, we can obtain the optimal production cost rate as a function of u. As u 3, the system becomes more and more unstable, and the total cost goes to infinity since the holding cost and backlogging cost both go to infinity. It is clear from Figure 3.7 that for each n, there is an optimal production rate u that minimizes the total cost rate. This optimal production rate increases with n. Comparing with Figure 3.2 we can see that the minimum cost is a little lower with option of planned backlogging. The difference is slight because we choose a relative small fixed ordering cost rate k=

75 Chapter 4 Environment-Dependent Order Quantities 4.1 Introduction In this chapter we continue with the production-inventory model of Chapter 3. As before, we assume leadtime is zero, and there is no backlogging. Thus it is optimal to place an order when the inventory level reaches zero. In the previous chapter we had assumed that the size of the order was predetermined and independent of the state of the environment when the order is placed. In this chapter we relax that assumption, that is, we assume that the order size is allowed to depend upon the environmental state when the order is placed. To be precise, let X(t) be the inventory level at time t, and Z(t) be the environment state at time t. We place an order of size q i if the inventory hit zero at state i, (i Ω ), where Ω is as defined in 2.8. We assume that if an order is placed when the environment is in state i, the order size is q i, i Ω. Figure 4.1 illustrates a typical sample path of the

76 Figure 4.1: A sample path of X(t) and Z(t) with environment-dependent order quantities. inventory level. It is easy to see that the stability condition for the bivariate Markov process (X(t), Z(t)) remains the same as given in Theorem We assume that the system is stable and let G(x) be as defined in In the literature, to the best of our knowledge, most research in Markovian environment is restricted to simple (r, q) policies: when the inventory level hits the reorder point r, an order is placed and all orders are of the same size q regardless of the state of the background process. (See Browne and Zipkin (1991), and Song and Zipkin (1996)). This single-order-size policy is appropriate under some circumstance as mentioned in Chapter 3. However, if we can observe the environmental state and can base our inventory replenishment decisions on that information, this simple (r, q) policy is not optimal. Clearly, the policy that allows the order sizes to depend on the state of the background process at the reordering epoch is no worse than the simple (r, q) policy. For example, in the fashion industry, it makes sense to change order sizes according to the seasons. As 59

77 another example, if a workshop places an order when inventory is zero, it would be better to decide how much to order based on the number of working machines at the time of ordering. This chapter is organized as follows. In Section 4.2, we derive a system of differential equations for the joint distribution function of the inventory level and environmental state in steady state over non-overlapping intervals of the inventory level. Then the final limiting distribution is a piecewise function consisting of the functions derived in all the intervals. In Section 4.3, we decompose the sample path of the inventory level into different type of cycles and then calculate the overall limiting distribution as a weighted average of the limiting distribution functions in all cycles. In Section 4.4 we still consider the non-overlapping intervals of the inventory level, but instead of solving differential equations we use Laplace-Stieltjes transforms to obtain the limiting distribution function. In our computational experiments, this method is most efficient among the three. Then in Section 4.5, given the limiting distribution of the inventory level, we derive the long-run average cost and hence determine the optimal order quantities {q i : i Ω }. At the end, we numerically investigate the machine shop example and give the optimal order sizes for all the background states. 4.2 Piecewise Function Method As mentioned before, we place an order of size q i if the inventory hit zero at state i, (i Ω ). We index the states such that Ω = {1, 2,, m }, and 0 < q 1 q m 1 q m, and define q 0 = 0. We consider the interval (q i, q i+1 ), as indicated in Figure

78 Figure 4.2: Piecewise function method. We can analyze the limiting distribution function vector G(x) over each interval (q i, q i+1 ) by using the methods developed in Chapter 2. The following theorem gives the main result. Theorem 4.1. Assume the stability condition (2.10) holds. The distribution G(x) satisfies the following differential equations G (x)r = G(x)Q + β (i), q i 1 < x < q i, i = 1, 2,...m, (4.1a) G (x)r = G(x)Q, x > q m, (4.1b) where β (i) = G (0)RI (i), and i 1 i 1 I (i) i 1 0 = i 1. n 1 C A. (4.2) 61

79 is a modified identity matrix that has 1 as its j-th diagonal entry if j i, and zeros as all other entries. The boundary conditions are given by G( ) = 0, (4.3a) G(q + i ) = G(q i ), i Ω, (4.3b) G j(0) = 0, j Ω +, (4.3c) G(0) e = 1. (4.3d) Proof. First consider the case q i 1 < x < q i. If the inventory level reaches 0 in state j Ω the inventory level immediately jumps to a level greater than or equal to q i only if j i. Now G j(0) = 0 if j Ω + (boundary condition (4.3c)). Hence in the vector β (i), the only nonzero entries correspond to the states k i. Now consider the x > q m case. For any state j, the inventory level can not jump above x due to an order. Thus the analysis in this case is the same as that of the x > q case in Theorem 2.3 in Chapter The boundary conditions follow along the same lines as in the proof of Theorem The next theorem gives the solutions to the equations in Theorem 4.1. Theorem 4.2. The solution to the differential equations in Theorem 4.1 is given by G(x) = m k=1 m + c (i) k eλ kx φ k + c (i) 0 xπ + φ (i) 0, q i 1 < x < q i, i = 1, 2,...m (4.4a) G(x) = a k e λkx φ k, x > q m, (4.4b) k=1 where the coefficients a 1, a 2,..., a m+, c (i) 1, c (i) 2,..., c (i) m, c (i) 0 and the vectors φ (i) 0, (i = 62

80 1, 2,...m ) are given by the unique solution to the following system of linear equations: ( m k=1 m + a k e λ kq m φkj k=1 m k=1 (c (i) k c (1) k eλ kx φ k + c (1) 0 π m k=1 ) RI (i) c (i) 0 πr + φ (i) 0 Q = 0, i Ω (4.5a) c (m ) k e λ kq m φkj c (m ) 0 q m π j φ (m ) 0j = 0, j / Ω 0, (4.5b) c(i 1) k )e λ kq i φ kj + (c (i) 0 c (i 1) 0 )q i π j + (φ (i) 0j φ(i 1) 0j ) = 0, i Ω, j / Ω 0, m k=0 ( m k=1 (4.5c) c (1) k λ kφ kj + c (1) 0 π j = 0, j Ω +, (4.5d) c (1) k φ k + φ (1) 0 ) e = 1, (4.5e) φ (i) 0 e = 1, i Ω. (4.5f) Proof. First, we know that when x > q m, G(x) satisfies a homogeneous differential equation (4.1a), and on the other intervals (q i 1, q i ), i = 1, 2,..., m, G(x) satisfies a nonhomogeneous differential equation (4.1b). Thus, from the same argument as in Theorem 2.4, we see that the solution is a piecewise function as given in Equation (4.4a) and (4.4a). Next we consider the boundary conditions. Equation (4.5a) follows from the fact that the non-exponential parts c (i) 0 πx + φ (i) 0 are particular solutions to Equation (4.1b). Notice that when i = 1, the number of independent equations in (4.5a) is n 1, otherwise, it is n. Equation (4.5b) and (4.5c) are the consequence of the continuity at q i s for the states with non-zero input rates. The total number of independent equations in (4.5b) and (4.5c) is m m. Equation (4.5d) and (4.5e) follow from the boundary condition (4.3c) and (4.3d). Together they yield m equations. We include Equation (4.5f) to uniquely specify the particular solutions to 63

81 use. This yields m + + (m + n + 1) m independent equations. Thus altogether we have m + + (m + n + 1) m equations to obtain the m + + (m + n + 1) m unknowns in Equations (4.4a) and (4.4b). This uniquely determines the expression for G(x). This method gives the limiting distribution function as a piecewise differentiable function on m + 1 intervals. However, to get the G(x) function, we need to solve a system of equations in m + +(m+n+1)m unknown variables. In our computational experiments, although this method is straightforward, we often have had numerical troubles when dealing with large number of states or large values of q i s. 4.3 Sample Path Decomposition Method In this section, we analyze the multiple order size problem using the concept cycle type. Recall that we used S k to denote the kth order epoch. Now define the interval [S k, S k+1 ) as the kth cycle. We say that the kth cycle is of type i if Z(S k +) = i. Next define I(t) to be the type of the cycle in progress at time t. Figure 4.3 shows a typical sample path of the {X(t), t 0} and the {I(t), t 0} processes. Note that I(t) Ω for all t 0. We have Now define G j (x) = lim t P {X(t) > x, Z(t) = j} = lim P {X(t) > x, Z(t) = j I(t) = i}p (I(t) = i) t i Ω = P {X(t) > x, Z(t) = j I(t) = i} lim P {I(t) = i}. t t i Ω lim G (i) j (x) = lim P {X(t) > x, Z(t) = j I(t) = i}, i Ω, t 64

82 Figure 4.3: Sample path decomposition method. and p (i) = lim t P {I(t) = i}, i Ω, Hence G j (x) = i Ω G (i) j (x)p(i). Using the vector notation G (i) (x) = [ ] G (i) 1 (x), G (i) 2 (x),..., G (i) m (x), we have G(x) = i Ω G (i) (x)p (i). Next we shall show how to compute G (i) (x) and p (i) respectively. To compute G (i) (x), we consider the sample path of {X(t), t 0} restricted to 65

83 the time intervals where I(t) = i. This is identical to the sample path of a fluid model with jumps as studied in Chapter 2, where the environment process jumps to state i whenever an order is placed regardless of the current background state. This insight yields the next theorem for the results for G (i) (x). Theorem 4.3. The limiting distribution G (i) (x) is a piecewise differentiable function on (0, q i ) and (q i, ) satisfying G (i) (x)r = G (i) (x)q + β (i), 0 x q i, (4.6a) G (i) (x)r = G (i) (x)q, x > q i, (4.6b) where the row vector β (i) is given by β (i) = G (i) (0)RA (i). The square matrix A (i) = i , n in which the ones appear in the ith column. The boundary conditions are given by G (i) ( ) = 0, (4.7a) G (i) j (q+ i ) = G(i) j (q i ), j : j / Ω 0 (4.7b) G (i) j (0) = 0, j : j Ω +, (4.7c) G (i) (0)e = 1. (4.7d) Proof. Consider the sample path of {X(t), t 0} restricted to the time intervals where I(t) = i. In Figure 4.3 the shadowed area shows such a restricted sample path. 66

84 Notice that the sample path of the i-th type cycle can be thought of as the sample path of a special case of the fluid process of Chapter 2 where the background state jumps to state i every time the fluid process jumps. Thus the A matrix in Chapter 2 is given by A (i) as in the theorem. The rest of the results follows from Theorem 2.3. To compute the probability p (i), notice that {I(t), t 0} is a semi-markov process with embedded Markov renewal sequence {(Z(S k ), S k ), k 0}. According to the theory of semi-markov process (e.g., Theorem 9.27 in Kulkarni (1995)), p (i) = ˆπ iτ i, (4.8) ˆπ j τ j j where ˆπ i is the limiting distribution of the embedded DTMC {Z(S k ), k 0} satisfying π j = i Ω π i p ij, j Ω π j = 1, (4.9) and τ i is the expected sojourn time in ith type cycle. Next we will show how to get p ij and τ i in Lemma 4.1 and Lemma 4.2. We assume at time 0, (X(0), Z(0)) is in steady-state. Define the first passage time in the {X(t), t 0} process as follows T = inf{t 0 : X(t) = 0}. (4.10) 67

85 Let P = [p ik ] where p ik = P {I k+1 = k I k = i} (4.11) = P {Z(T ) = k X(0) = q i, Z(0) = i}, i, k Ω. (4.12) The next lemma gives the differential equations and boundary conditions satisfied by the vector p (k) (x) = [p 1k (x), p 2k (x),..., p nk (x)], where p ik (x) = P {Z(T ) = k X(0) = x, Z(0) = i}, i, k Ω, (4.13) hence we have p ik = p ik (q i ). We use generalized eigenvalues and eigenvectors (λ j, φ j ) from Equation (2.5). Lemma 4.1. The vector p (k) (x) satisfies p (k) (x)r + p (k) (x)q = 0. (4.14) The solution to the above differential equation is given by m + p (k) (x) = a j e λjx φ j (4.15) j=1 where the coefficients a 1, a 2,..., a m+ is determined by the following boundary conditions p (k) ( ) = 0, (4.16) p ii (0) = 1, (4.17) p ik (0) = 0, if i k, i, k Ω. (4.18) Proof. The differential Equation (4.14) follows from the standard derivation of 68

86 Chapman Kolmogorov equations for Markov processes. Define p ik (t, x) = P {Z(T ) = k X(t) = x, Z(t) = i}. (4.19) We consider a time interval [0, δ] where δ > 0. p ik (x) = j Ω (P {Z(T ) = k X(0) = x, Z(0) = i, X(δ) = x + R i δ, Z(δ) = j} P {X(δ) = x + R i δ, Z(δ) = j X(0) = x, Z(0) = i}) Notice that P {X(δ) = x + R i δ, Z(δ) = j X(0) = x, Z(0) = i} is the transition probability from state i to state j during δ amount of time. Thus P {X(δ) = x + R i δ, Z(δ) = j X(0) = x, Z(0) = i} = P (Z(δ) = j Z(0) = i) = δ ij + q ij δ + o(δ), where δ ij = 1 if i = j, and 0 otherwise. Thus p ik (x) = j Ω(δ ij + q ij δ) p jk (x + R i δ) + o(δ) = p ik (x + R i δ) + j Ω p jk (x + R i δ)q ij δ + o(δ). Rearrange and divide both sides by δ to get p ik (x) p ik (x + R i δ) δ = j Ω p jk (x + R i δ)q ij + o(δ). Letting δ 0, we get p ik(x)r i = j Ω p jk (x) q ij. (4.20) Writing Equation (4.20) in matrix form we get Equation (4.14). 69

87 We need to solve m linear systems in m + unknowns each to get the matrix P. Then we need to solve one linear system of m unknowns to obtain ˆπ in Equation (4.9). Next we calculate the expected sojourn time τ i. Notice that it is actually the mean first passage time from state (q i, i). Define τ i (x) = E(T X(0) = x, Z(0) = i), i Ω. (4.21) Then we get τ i = τ i (q i ). Kulkarni and Tzenova (2002) have developed methods to compute τ i (x). We give their main result below. Lemma 4.2. The vector τ(x) = (τ 1 (x), τ 2 (x),..., τ m (x)) satisfies τ (x)r + τ(x)q + e = 0. (4.22) Its solution is given by m + τ(x) = c k e λkx φ k + c 0 xe + φ 0, (4.23) k=1 where the coefficients c 1, c 2,..., c m+, c 0 and the vector φ 0 are determined by Qφ 0 = ( 1 R + I)e, (4.24) m + c k φ ki + φ 0i = 0, i Ω. (4.25) k=1 Proof. See the proof of Theorem 3.1 and 3.2 in Kulkarni and Tzenova (2002). (4.23). We need to solve one linear system of m + +n+1 unknowns to obtain τ i in Equation 70

88 Now we summarize the results in the following theorem. Theorem 4.4. Assume the stability condition (2.10) holds. The limiting distribution of the inventory level is a weighted average of those over the different cycles, given by G(x) = G (i) (x)p (i), (4.26) i Ω where G (i) (x) is given by Theorem 4.3, and p (i) is given by Equation (4.8). This method involves solving smaller linear systems of equations than the previous method. In our experience, this method is numerically more stable. 4.4 Laplace-Stieltjes Transform Method In this section we develop a third method which is computationally superior to the previous two methods. First we compute LST of the cumulative distribution function in steady state of the environment-dependent order size problem. Let F j (x) and F (s) be as defined in Equation (2.21) and (2.23). Theorem 4.5. Let F j (x) be as defined in Equation (2.21), and F (x) = [F 1 (x), F 2 (x),..., F n (x)]. Then its LST F (s) is given by F (s) = a D(s)(sR Q) 1, (4.27) where a = [a 1, a 2,..., a n ] is a row vector with its i-th entry a i = F i (0)R i, if i Ω, 71

89 and 0 otherwise, and D(s) = diag(1 e sq i ) (using q i = 0 if i / Ω ). get Proof. Using the fact that G(x) = π F (x) in Equation (4.1a) and (4.1b), we F (x)r = F (x)q + F (0)RI (i), q i 1 < x < q i, i = 1, 2,...m, (4.28a) F (x)r = F (x)q, x > q m, (4.28b) where I (i) is defined in Equation (4.2). Thus we have e sx F (x)r dx = m e sx F (x)qdx i=1 qi q i 1 e sx F (0)RI (i) dx. Thus F (s)r = 1 s F (s)q + 1 m (e sq i 1 e sq i )F (0)RI (i) s i=1 m F (s)r = 1 s F (s)q + 1 (1 e sq i )(F (0)RI (i) F (0)RI (i+1) ) s i=1 = 1 s F (s)q + 1 s F (0)R D(s), (4.29) where D(s) = diag(1 e sq i ). Rearranging Equation (4.29) we get Equation (4.27). Remark. Define diagonal matrix L with q i, if i Ω, L ii = 0 otherwise. (4.30) 72

90 Define another diagonal matrix Ẽ(s) with 1 e sq i sq Ẽ ii (s) = i, if i Ω, 0 otherwise. (4.31) One can rewrite Equation (4.27) as F (s) = alẽ(s)s(sr Q) 1. (4.32) This shows that the LST F (s) is a product of tow matrices. The first matrix contains the LSTs of uniform (0, q i ) random variables, and the second matrix appears in the LST of the limiting distribution of the standard fluid model without jumps (see Equation 2.9). This is the generalization of the decomposition result of Section Next we shall show how to invert the LST to get the cumulative distribution function F (x). We need the following notation. Let (λ, φ) be as defined in Equation (2.5). As before, we assume λ 1,..., λ m+ have negative real parts, λ m+ +1 = 0, and λ m+ +2,..., λ m have positive real parts. We call φ a left-eigenvector. Let Φ = φ 1 φ 2. φ m. Let ψ j be the j-th generalized right-eigenvector that solves (λ j R Q)ψ j = 0. Let Ψ be the square matrix whose j-th column is ψ j. When all eigenvalues are distinct, and R is invertible, we can choose Ψ = R 1 Φ 1. 73

91 The next theorem shows how to compute F (x) in terms of the eigenvalues and eigenvectors. Theorem 4.6. Assume the eigenvalues are distinct, and R is invertible. Then n F (x) = a D (j) (x)ψ j φ j, (4.33) j=1 where D (j) (x) is a diagonal matrix such that when λ j 0 D (j) ii (x) = e λ j x 1 λ j, when 0 x q i, e λ j x λ j (1 e λ jq i ) when x > q i, (4.34) and when λ j = 0 The coefficient vector a is uniquely determined by D (j) ii (x) = min{x, q i}. (4.35) am (j) ψ j = 0, if λ j > 0, (4.36) alψ m+ +1φ m+ +1 e = 1, (4.37) a i = 0, if i / Ω. (4.38) where M (j) is a diagonal matrix with M (j) ii = 1 e λ jq i, (4.39) and L is defined in Equation (4.30). Proof. One can show that when the eigenvalues are distinct, and R is invertible, (sr Q) 1 = n j=1 ψ j φ j 1 s λ j. 74

92 Thus from Equation (4.27) we have F (s) = a n j=1 D (j) (s)ψ j φ j, (4.40) where D (j) (s) is a diagonal matrix. Its i-th entry on the diagonal is given by D (j) ii = 1 e sq i s λ j. (4.41) We can see that the i-th entry on the diagonal of D (j) (s) is an LST of the piecewise function given by Equation (4.34) and (4.35). Thus we have Equation (4.33). Now we consider how to determine a. Since F ( ) = π, all the coefficients of e λ jx have to be zero when λ j > 0. Thus we have am (j) ψ j φ j λ j = 0 when λ j 0. (4.42) Since λ j 0 and φ j 0, the only possibility for Equation (4.42) to be valid is am (j) ψ j = 0, which is Equation (4.36). Equation (4.37) is a consequence of i Ω F ( ) = 1. From Equation (4.33) we have m + F ( ) = a D (j) ( )ψ j φ j + alψ m+ +1φ m+ +1. (4.43) j=1 When λ j < 0, e λ jx 0 as x, hence Equation (4.43) reduces to (4.37). Thus we need to solve only one linear system in n unknowns to completely determine G(x). Of course we need the additional computation of the right eigenvectors 75

93 Ψ to implement this method. 4.5 The Cost Model Once we have the limiting distribution G(x), we are able to calculate the longrun average cost to operate this system, and hence determine the optimal scenario (q 1, q 2,..., q m ) that minimizes the cost rate in steady state. As in Chapter 3, we consider the following costs: h: cost to hold one item in inventory for one unit of time; k i : fixed ordering cost whenever an order is placed at state i; p 1 : cost to purchase one item from the external supplier; p 2 : cost to produce one item. The following theorem gives the long-run average cost. We assume that the production rate is r i and the demand rate is d i. Thus R i = r i d i. Theorem 4.7. Assume that the system is stable with limiting distribution G(x). Let c h, c o and c p be the steady-state holding, ordering and production cost rates respectively. The total cost rate c is given by c = c h + c o + c p, (4.44) where c h = he(x) = h c o = 0 G(x)dx, (4.45) i Ω (k i + p 1 q i )G i(0)r i, (4.46) 76

94 and c p = p 2 n π i r i. (4.47) i=1 Proof. Equation (4.45) follows from the standard calculation of holding cost. In steady state, in one unit of time the inventory level hits zero at state i Ω for G i(0)r i times. Each time it costs k i to place an order and p 1 q i to purchase q i units of product. Thus the ordering cost rate is given as Equation (4.46). As to the production cost, as long as the environment is in state i, the production rate is r i, thus the long-run average production cost is given by Equation (4.47). Remark. In fact, the price p 1, and production cost rate p 2 have no effect on deriving the optimal scenario (q 1, q 2,..., q m ). We assumed our orders that do not affect the external environment (i.e., the production and demand rate), and thus the part of the demand that can not be satisfied by the production has to be supplied with external orders. Hence the production cost and purchasing cost are independent of the ordering policies. 4.6 A Numerical Example Consider a machine shop with three machines as described in section When the inventory decreases to zero and there are i working machines (i = 0, 1, 2, 3), we place an order of size q i. We investigate the effect of the production rate increases on the optimal order quantities q i s. Consider λ = 1, µ = 2, h = 5, k = 0.5, p 1 = 8, p 2 = 5, and d = 3. Let u vary in (0, 3). We plot the optimal values of q i s in Figure

95 Figure 4.4: The optimal order quantities vs. production rate. Note that when the production rate is zero, the optimal order sizes are the same for all states. This makes intuitive sense because when u = 0, R i = 3 for all i = 0, 1, 2, 3, and hence the states have no effect on the inventory level. In this case, all the qi s are given by the basic EOQ formula given in the previous chapter, and matches the example in Figure 3.1 when n = 3 and u = 0. Note that the q3 jumps down to zero when u increases to 1. This is because when there are 3 working machines and u 1, the net input rate 3u d is positive, and hence we would never place an order in that state. Also note that q3 jumps down to zero instead of approaching zero gradually. This is because just before u increases to 1, the inventory level can reach zero at state 3, and hence we still need to place an order of size q 3. However, q3 has to be large enough, since there are other states with negative net input rate and the average net input rate over all the states are still negative. Similar observation can be made for q2 when u increases to 2, but q2 jumps to zero from a smaller level, this can be explained as that when the expected 78

96 net demand rate is smaller, the less we need to order from the external supplier. Finally, all q i s are zero when u reaches 3. This is because then the expected net input rate is nonnegative and the system is not stable, and hence we never need to place an order. 79

97 Chapter 5 Stochastic Leadtimes 5.1 Introduction In this chapter we extend the inventory model of Chapter 3 to a more general case where the order leadtimes can be positive. To recapitulate, we study a productioninventory system in which the inventory level process {X(t), t 0} is modulated by a background process {Z(t), t 0}. We assume that {Z(t), t 0} is an irreducible CTMC on state space Ω = {1, 2,..., n} with rate matrix Q = [q ij ]. When Z(t) is in state i, the production occurs continuously at a constant rate r i, and demand occurs at rate d i. As long as Z(t) = i, {X(t), t 0} changes at rate R i = r i d i. Since there are leadtimes we need to allow backlogging. We assume that if X(t) > 0 it denotes the actual inventory on hand at time t, and X(t) < 0, X(t) denotes the backorders at time t. We begin by defining inventory position as the inventory on hand minus backorders plus the amount that has been ordered but has not arrived (called outstanding orders). Let P (t) be the inventory position at time t. When the inventory position decreases

98 to a pre-specified reorder point r, we place an order of size q, which arrives after a random period of time. Note that the order size is independent of the environmental state at the time of order placement. We may place a new order before a previous order arrives. We use O(t) to denote the number of outstanding orders at time t. Let X(t) be the inventory level at time t. If X(t) > 0 it denotes the actual inventory on hand at time t, and X(t) < 0, X(t) denotes the backorders at time t. Thus the inventory position at time t is given by P (t) = X(t) + q O(t) Figure 5.1 illustrates a typical sample path of the {P (t), t 0} and {X(t), t 0} processes. As before, let Z(t) be the environment state at time t, and define Figure 5.1: Sample paths of P (t) and X(t) with stochastic leadtimes. P (t) = P (t) r. (5.1) From Figure 5.1, it is clear that the { P (t), t 0} process is the same as the basic inventory model studied in Chapter 3 modulated by the environment process {Z(t), t 0}. Thus we can compute the limiting distribution of the {( P (t), Z(t)), t 0} process 81

99 by following the methodology developed in Chapter 3. However, our main interest is in computing the limiting distribution of the inventory level process {X(t), t 0}. Since X(t) = P (t) + r q O(t), we need the limiting joint distribution of ( P (t), O(t)) to compute the limiting distribution of X(t). In order to compute this we consider a triviate process {( P (t), Z(t), O(t)), t 0}. One can think of this as the stochastic fluid process { P (t), t 0} modulated by the extended environment process {(Z(t), O(t)), t 0}. We consider three cases: (1) Orders are processed in sequential fashion, and hence orders never cross in time. We assume that if there is at least one outstanding order, the next order will be received after an exp(ν) amount of time. We shall call this the serial processing system. (2) Orders are processes in parallel fashion. The leadtimes for individual orders are i.i.d. exp(ν). Thus if there are i outstanding orders at time t, the next order placed is delivered at rate iν. We shall call this the parallel processing system. (3) A general case: when there are i outstanding orders, the next order is delivered at rate ν i. In all cases we assume that at most N outstanding orders are allowed. This makes sense because usually a business does not like too many outstanding orders pending with the supplier due to the penalty cost to backlog its own demands when 82

100 the inventory level is much lower than zero. Thus we assume that when there are N outstanding orders and the inventory position decreases to the reorder point r again, we place the new order from an emergency supplier and that emergency order arrives immediately. This can also be thought of as a type of selective lost-sale models when the demands are canceled when the number of outstanding orders reaches N. If the limiting probability that the number of outstanding orders is N is very small, this will provide a good approximation to the serial case or parallel case models with no upper limit on the number of outstanding orders. With this assumption it is clear that the state space of the bivariate process {(Z(t), O(t)), t 0} is {(j, i) : j Ω, 0 i N}. It is easy to see that in this case this trivariate process {( P (t), Z(t), O(t)), t 0} is stable if π i R i < 0, (5.2) i Ω where π i = lim t P {Z(t) = i}. We shall assume that this stability condition is satisfied from now on. Let and Ḡ (i) j (x) = lim P { P (t) > x, Z(t) = j, O(t) = i}, x 0, j Ω, 0 i N, t Ḡ (i) (x) = [Ḡ(i) ] 1 (x), Ḡ(i) 2 (x),..., Ḡ(i) n (x), 0 i N, Ḡ(x) = [ Ḡ (0) (x), Ḡ(1) (x),..., Ḡ(N) (x) ]. 83

101 5.2 Serial Processing System In this section we consider serial processing case as described before. The next theorem gives the differential equations satisfied by Ḡ(x). We use the following notation: R = R R... R, (5.3) Q = Q νi Q νi νi Q νi, (5.4) and Ā = 0 I 0 I I I, (5.5) where R and Q are the matrices defined in Chapter 2. Thus R, Q, and Ā are all n(n + 1) n(n + 1) matrices. Theorem 5.1. Assume the stability condition (5.2) holds. The limiting distribution Ḡ(x) is continuous on [0, ) and is a piecewise differentiable function on (0, q) and 84

102 (q, ). It satisfies Ḡ (x) R = Ḡ(x) Q, x > q, (5.6) Ḡ (x) R = Ḡ(x) Q + Ḡ (0) RĀ, 0 < x < q. (5.7) The boundary conditions are given by Ḡ( ) = 0, (5.8a) Ḡ (i) j (q+ ) = Ḡ(i) j (q ), j / Ω 0, 0 i N (5.8b) Ḡ (i) j (0) = 0, j Ω +, 0 i N (5.8c) Ḡ(0)e = 1. (5.8d) Proof. First consider the case 0 < x < q. Define Ḡ (i) j (t, x) = P { P (t) > x, Z(t) = j, O(t) = i}, x 0, t 0, j Ω, 0 i N. Assume at time 0, the system is in steady state. We consider a time interval [0, δ] where δ > 0. When there are no outstanding orders, i.e., i = 0, at time δ, the system can reach ( P (δ) > x, Z(δ) = j, O(δ) = 0) from ( P (0) > x, Z(0) = k, O(0) = 0) due to the transition of the background state, or from ( P (0) > x, Z(0) = j, O(0) = 1) due to the arrival of the outstanding order. Thus following from the standard derivation of Chapman-Kolmogorov equations for Markov processes, one can get Ḡ (0) (x)r = Ḡ(0) (x)q + νḡ(1) (x). (5.9) When 1 i N 1, the system can reach ( P (δ) > x, Z(δ) = j, O(δ) = i) from ( P (0) > x, Z(0) = k, O(0) = i) due to the transition of the background state, or from ( P (0) > x, Z(0) = j, O(0) = i + 1) due to the arrival of the outstanding order, or 85

103 from ( P (0) > x, Z(0) = j, O(0) = i 1) due to the placement of a new order. Thus following the same lines as in the proof of Theorem 2.3, one can get Ḡ (i) (x)r = Ḡ(i) (x)(q νi) + νḡ(i+1) (x) + Ḡ(i 1) (0)R, 1 i N 1. (5.10) When i = N, the system can reach ( P (δ) > x, Z(δ) = j, O(δ) = N) from ( P (0) > x, Z(0) = k, O(0) = N), or from ( P (0) > x, Z(0) = j, O(0) = N 1) due to the placement of a regular order, or from ( P (0) > x, Z(0) = j, O(0) = N) due to the placement of an emergency order since at time 0 the number of outstanding orders already at the maximum allowed number N. Thus we can get Ḡ (N) (x)r = Ḡ(N) (x)(q νi) + νḡ(n) (x) + (Ḡ(N 1) (0) + Ḡ(N) (0))R. (5.11) Using the notation Ḡ(x), R, Q, and Ā, Equation (5.9), (5.10), and (5.11) can be written as Equation (5.7). Rest of the equation follows similarly. Now for the x > q case, since the placement of a new order can not bring inventory position to above x, there is no corresponding term in Equation (5.6). The boundary conditions are the same as in Theorem 2.3. To solve this special system of differential equations using spectral methods given in Chapter 2, repeated eigenvalues are inevitable because of the repeating blocks in the R, Q matrices. One method using Jordan Canonical form has been given in Section

104 5.3 Parallel Processing System In this section, we consider the parallel processing case as described in Section 5.1. We can see that Ḡ(x) satisfies the same form of differential equations with a different Q matrix as given below. Q = Q νi Q νi 2νI Q 2νI NνI Q NνI. (5.12) We use the R and Ā as given in Equation (5.25) and (5.26). The above matrix is the result of the fact that orders arrive at rate νo(t) at time t, and when there are N outstanding orders, any extra orders arrive instantaneously. Using the spectral methods we can derive the limiting distribution function in terms of eigenvalues and eigenvectors related with R and Q. However, with the special Q given in (5.12), we are able to calculate the eigenvalues and eigenvectors in an easier way. Define λ (k) i to be a scalar and φ (k) i to be a 1 n vector that satisfies φ (k) i (Q kνi) = λ (k) i φ (k) i R, k = 0, 1, 2,...N, i = 1, 2,..., n. (5.13) The next theorem shows how to construct the (eigenvalue, eigenvector) pairs of the n(n + 1) n(n + 1) matrices R and Q ( ) using, φ (k). λ (k) i i 87

105 Theorem 5.2. Define a 1 n(n + 1) vector [ Φ (k) i = a k0 φ (k) i a k1 φ (k) i a k2 φ (k) i ] a kn φ (k) i, k = 0, 1, 2,...N, i = 1, 2,..., n, (5.14) where Then the pair ( λ (k) i ( 1) j( k j), if j k, a kj = 0, otherwise. ), Φ (k) i satisfies (5.15) Φ (k) i Q = λ (k) i Φ (k) i R, k = 0, 1, 2,...N, i = 1, 2,..., n. (5.16) ( Proof. To verify that ), Φ (k) i is a valid (eigenvalue, eigenvector) pair, substitute Φ (k) i have λ (k) i of Equation (5.14) in Equation (5.16). Using a kj = 0 when j > k, we φ (k) i φ (k) i φ (k) i (a k0 Q + a k1 νi) = λ (k) a k0 φ (k) i R, (5.17a) i (a k1 (Q νi) + a k2 2νI) = λ (k) i a k1 φ (k) i R, (5.17b). (a k,k 1 (Q (k 1)νI) + a kk kνi) = λ (k) i a k,k 1 φ (k) i R, (5.17c) φ (k) i a kk (Q kνi) = λ (k) i a kk φ (k) i R. (5.17d) 88

106 Since a kj 0 when 0 j k, Equations (5.17a) to (5.17d) can be simplified as ( φ (k) i Q + ( φ (k) i Q + a k1 ( 2ak2 a k1 1 ) νi a ) k0 ) νi ( ( ) φ (k) kakk i Q + (k 1) νi a k,k 1 φ (k) i. ) = λ (k) i φ (k) i R, (5.18a) = λ (k) i φ (k) i R, (5.18b) = λ (k) i φ (k) i R, (5.18c) (Q kνi) = λ (k) i φ (k) i R. (5.18d) Further more, Equation (5.15) implies that a k1 = 2a k2 1 =... = ka kk (k 1) = k. a k0 a k1 a k,k 1 Thus Equations (5.18a) to (5.18c) reduce to Equation (5.18d), which is valid according to the definition of φ (k) i and λ (k) i. This proves the theorem. Remark. The above theorem enables us to obtain the n(n + 1) eigenvalues and n(n + 1) independent eigenvectors of the large matrices ( Q, R) in terms of those for the smaller matrices (Q, R). This results in a substantial savings in computation. 5.4 General Order Processing In this section we extend the model to a more general case in which when there are k outstanding orders, the next order arrives after exp(ν k ) amount of time. Thus the 89

107 Q matrix changes to Q = Q ν 1 I Q ν 1 I ν 2 I Q ν 2 I ν N I Q ν N I. (5.19) When ν i = ν for all i, this model reduces to the serial processing case of Section 5.2; when ν i = iν for all i, it reduces to the parallel processing case of Section 5.3. Define λ (k) i to be a scalar and φ (k) i to be a 1 n vector that satisfies φ (k) i (Q ν k I) = λ (k) i φ (k) i R, k = 0, 1, 2,...N, i = 1, 2,..., n. (5.20) The next theorem shows how to construct the (eigenvalue, eigenvector) pairs of the n(n + 1) n(n + 1) matrices R and Q ( ) using, φ (k). Theorem 5.3. Assume all ν i s are distinct. Define a 1 n(n + 1) vector λ (k) i i [ Φ (k) i = a k0 φ (k) i a k1 φ (k) i a k2 φ (k) i ] a kn φ (k) i, k = 0, 1, 2,...N, i = 1, 2,..., n, (5.21) where j 1 ν k ( ) ν j 1 ν k νl, if j k, a kj = l=1 0, otherwise. (5.22) Then Φ (k) i and λ (k) i satisfies Φ (k) i Q = λ (k) i Φ (k) i R, k = 0, 1, 2,...N, i = 1, 2,..., n. (5.23) 90

108 ( Proof. To verify that λ (k) i ), Φ (k) i is a valid (eigenvalue, eigenvector) pair, substitute Φ (k) i of Equation (5.21) in Equation (5.23). Following the proof of Theorem 5.2, we can show that Equation (5.23) is valid if and only if ( φ (k) i Q + ( φ (k) i Q + a k1ν 1 a k0 ( ak2 ν 2 a k1 ν 1 ) ( ( ) φ (k) akk ν k i Q + ν k 1 I a k,k 1 φ (k) i ) I = λ (k) i φ (k) i R (5.24a) ) I = λ (k) i φ (k) i R (5.24b). ) = λ (k) i φ (k) i R (5.24c) (Q ν k I) = λ (k) i φ (k) i R. (5.24d) Since Equation (5.22) implies that a k1 ν 1 = a k2ν 2 ν 1 =... = a kkν k ν k 1 = ν k, a k0 a k1 a k,k 1 Equations (5.24a) to (5.24c) reduce to Equation (5.24d), which is valid according to the definition of φ (k) i and λ (k) i. This proves the theorem. Note that Theorem 5.3 does not apply to the serial processing case of Section Selective Lost Sale Model In this section we consider the selective lost sale problem as described before. Assume all the demands are canceled when the number of outstanding orders reaches N. We have the following Theorem. Theorem 5.4. Suppose the demand rate d j = 0 for all the environment states j when O(t) > N. The limiting distribution function is described by Theorem 5.1 with the 91

109 following R, Ā matrices: R = R R... R U, (5.25) where U = diag([u 1, u 2,..., u n ]), where u j is the production rate in state j. Ā = 0 I 0 I I 0, (5.26) and Q is given in Equation (5.4), (5.12) or (5.19) according to different order processing models. Proof. When there are N outstanding orders, all the demands are lost. Thus for any background state j, R j = u j. Hence R j 0 for all j when O(t) = N. According to boundary condition (5.8c), we have Ḡ(N) j (0) = 0. Thus the N-th diagonal block in Ā can be arbitrary. For the computational convenience, we let it be 0. The rest of the proof follows the same lines as in the proof of Theorem

110 5.6 The Cost Model Once we have the limiting distribution Ḡ(x), we are able to calculate the long-run average cost to operate this system, and hence determine the optimal order quantity q and the optimal reorder point r that minimize the cost rate in steady state Cost Rate Calculation We consider the following costs: h: cost to hold one item in inventory for one unit of time; b: cost to backlog one unit of demand for one unit of time; k 1 : fixed set-up cost to place a regular order when there are less than N outstanding orders; k 2 : fixed set-up cost to place an emergency order when there are N outstanding orders, or 0 in the selective lost-sale model; p 1 : cost to purchase one item from a regular supplier; p 2 : cost to purchase one item from the emergency supplier, or the lost-sale penalty to cancel one unit of demand; p 3 : cost to produce one item. Let c h (r, q), c b (r, q), c o (r, q) and c p (r, q) be the steady state holding, backlogging, ordering and producing cost rates respectively as functions of the order quantity q. The total cost rate c(r, q) is thus given by c(r, q) = c b (r, q) + c h (r, q) + c o (r, q) + c p (r, q). (5.27) 93

111 The following theorem gives the long-run average costs. Theorem 5.5. Assume that the system is stable with limiting distribution Ḡ(x) = [ Ḡ (0) (x), Ḡ(1) (x),..., Ḡ(N) (x) ]. The cost functions are given by c h (r, q) = h N [ r q j=0 y=jq r Ḡ(j) (y)dy e if r < 0, ( h (r jq)ḡ(j) (0) + ) y=0 Ḡ(j) (y)dy j=0 ] if 0 r Nq, + N j= r q y=jq r Ḡ(j) (y)dy e h N ( (r jq)ḡ(j) (0) + ) y=0 Ḡ(j) (y)dy e if r > Nq. j=0 (5.28) c b (r, q) = b N ( (jq r)ḡ(j) (0) ) jq r Ḡ (j) (y)dy e if r < 0, y=0 j=0 N ( b (jq r)ḡ(j) (0) ) jq r Ḡ (j) (y)dy e if 0 r Nq, y=0 j= r q 0 if r > Nq. (5.29) N 1 c o (r, q) = (k 1 + p 1 q)ḡ(j) (0)R e + (k 2 + p 2 q)ḡ(n) (0)R e (5.30) j=0 c p (r, q) = p 3 n π i r i. (5.31) i=1 94

112 Proof. The holding cost is given by Thus if 0 r Nq, c h (r, q) = he(x + ) = h = h = h c h (r, q) =h 0 t 0 N j=0 lim P {X(t) > x}dx lim P { P (t) + r qo(t) > x}dx t 0 r q ( r jq j=0 + h N j= r q x=0 x=0 lim P { P (t) > x + jq r, O(t) = j}dx. t Ḡ (j) (0)dx + x=r jq Ḡ (j) (x + jq r)dx e. ) Ḡ (j) (x + jq r)dx e Letting y = x + jq r, we get c h (r, q) = h +h r q j=0 ( (r jq)ḡ(j) (0) + ) y=0 Ḡ(j) (y)dy e N j= r q y=jq r Ḡ(j) (y)dy e. (5.32) Similarly we can show that if r < 0, and if r > Nq, c h (r, q) = h c h (r, q) = h N j=0 N j=0 y=jq r Ḡ (j) (y)dy e, (5.33) ( ) (r jq)ḡ(j) (0) + Ḡ (j) (y)dy e. (5.34) y=0 Combining Equation (5.32), (5.33), and (5.34) together, we can get Equation (5.28). 95

113 The backlogging cost is given by c b (r, q) = be(x ) = b = b = b Thus if 0 r Nq, 0 t 0 0 N j=0 lim P {X(t) x}dx lim P { P (t) + r qo(t) x}dx t lim P { P (t) x + jq r, O(t) = j}dx. t r q 0 c b (r, q) =b =b j=0 + b x= 0 N j= r q x= 0 N j= r q x=r jq lim P { P (t) x + jq r, O(t) = j}dx t lim P { P (t) x + jq r, O(t) = j}dx t lim P { P (t) x + jq r, O(t) = j}dx t Letting y = x + jq r, we get c b (r, q) =b =b =b N jq r j= r q y=0 jq r N j= r q N j= r q y=0 lim P { P (t) y, O(t) = j}dy t (Ḡ(j) (0) Ḡ(j) (y))dy e ( jq r ) (jq r)ḡ(j) (0) Ḡ (j) (y)dy e. (5.35) y=0 Similarly we can show that if r < 0, c b (r, q) = b N j=0 ( jq r ) (jq r)ḡ(j) (0) Ḡ (j) (y)dy e, (5.36) y=0 96

114 and if r > Nq, c b (r, q) = 0. (5.37) Combining Equation (5.35), (5.36), and (5.37) together, we can get Equation (5.29). Equation (5.30) follows a similar argument in the proof of Theorem 4.7. The production cost c p (r, q) is the same as given in Theorem Newsboy Solution for the Optimal Reorder Point Let X be the inventory level in steady state. Define Y = X r and G Y (y) = P {Y > y}. The next theorem shows that in the setting with stochastic leadtimes, the optimality of the newsboy solution given by Theorem 3.4 still holds. Theorem 5.6. For a given q, c(r, q) is a convex function of r, and it is minimized at ( ) b r (q) = G 1 Y. (5.38) h + b Proof. As before, c(r, q) = c b (r, q) + c h (r, q) + c o (r, q) + c p (r, q) = h = h = h P {X > x}dx + b 0 P {Y > x r}dx + b r Nq 0 P {X x}dx + c o (r, q) + c p (r, q) r Nq P {Y x r}dx + c o (r, q) + c p (r, q) 0 G Y (x r)dx + b (1 G Y (x r))dx + c o (r, q) + c p (r, q). r Nq 97

115 Letting y = x r, we have c(r, q) = h r r G Y (y)dy + b (1 G Y (y))dx + c o (r, q) + c p (r, q). Nq Since Y = X r, we get G Y (y) = N Ḡ(y + jq) e. j=0 Since this is independent of r, and c o (r, q) and c p (r, q) are independent of r, we have r c(r, q) = hg Y ( r) b + bg Y ( r). and 2 r 2 c(r, q) = hg Y ( r) bg Y ( r) 0. Hence c(r, q) is a convex function of r for a given q. Thus it is minimized when c(r, q) = 0. This yields Equation (5.38). r The problem of minimizing the function c(r, q) reduces to that of minimizing the function of a single variable c(r (q), q) to obtain the optimal order quantity q. Then the optimal reorder point is given by r = r (q ). This needs to be done numerically. 98

116 5.7 A Numerical Example Limiting Distribution Consider a single machine as described in section The machines switches between up and down with rates λ = 1, µ = 2. When the machine is up, the production rate is 2.5, and when the machine is down, it is zero. The demand rate is always 1. We set reorder point r = 0, and whenever the inventory position decreases to 0 we place an external order of size q = 5. Assume orders are processed simultaneously and the processing time is exp(0.1). We allow at most 2 outstanding orders. In Figure 5.2 and 5.3 we plot the steady-state complementary cumulative distribution functions and the probability density functions of P (t) ( equal to P (t) in this case) when the number of outstanding orders are 0, 1 and 2 respectively, and also when the machine is either down or up. Figure 5.2: The steady-state ccdf of the P (t) process. 99

117 Figure 5.3: The steady-state pdf of the P (t) process. In this example lim t P {O(t) = i} is , and respectively for i = 0, 1 and 2. The probability that we reach the upper limit on the number of outstanding orders is , which indicates that the probability we need to turn to the emergency supplier is very small. Note that when there is no outstanding order the density function is also continuous at q. This is different from Figure 2.8 in Chapter 2, in which the order placement causes a jump in the probability density at q. Here, when there is no outstanding order, the order placement causes a jump in Ḡ(1) (q); when there is 1 outstanding order, the order placement causes a jump in Ḡ(2) (q); when there are 2 outstanding orders, the placement of the emergency order also causes a jump in Ḡ(2) (q). Thus nothing can cause a jump in Ḡ(0) (q) and hence the probability density function Ḡ(0) (x) is continuous over (0, ). 100

118 Using X(t) = P (t) + r q O(t), we can get the limiting distribution of the actual inventory level process. Figure 5.4 and Figure 5.5 show the steady-state complementary cumulative functions and probability density functions of X(t) and P (t). Figure 5.4: The steady-state ccdf of X(t) and P (t). Figure 5.5: The steady-state pdf of X(t) and P (t). 101

119 5.7.2 Optimal Ordering Policy Next we investigate the optimal (r, q) policy. As before, we use the n machine example of Section Let the production rate vary in (0, 3). Demand rate equals the number of machines. Let the cost parameters be h = 5, b = 8, k 1 = 0.5, p 1 = 8, k 2 = 2, p 2 = 16, and p 3 = 5. We plot the optimal values of q s and r s in Figure 5.6 and 5.7 for n = 1, 2,.., 5. Figure 5.6: The optimal order quantity vs. production rate (varying n). 102

120 Figure 5.7: The optimal reorder point vs. production rate (varying n). Note that in Figure 5.7, different from Section where r is always negative, here r can be positive. This is because when there is leadtime we need to have sense of anticipation and place orders a head of time. As before, when production rate increases to 3, r decreases to. As for the optimal order quantity q, in contrast to the behavior of q in Section , here the q is a monotonic decreasing function of u. In Section when r is very low, the order quantity q starts increasing to bring the inventory level back to zero at the ordering state (which must have a negative input rate) to reduce the backlogging cost. Here, because now there is a positive leadtime between placing an order and actually receiving that order, and during the leadtime the background might change to another state. As u increases to 3, the probability that the background changes to a state with positive input rate increases. Hence, even when r still goes to as in Section , the increasing of q can not bring the inventory level at the ordering state back closer to zero. Thus when u goes to 3, q keeps decreasing 103

121 as a result of that our overall system becomes more and more independent of the external supplies Optimal Production Rate We plot the minimal costs corresponding to the optimal ordering policy. Figure 5.8: The minimal cost vs. production rate (varying n). As in Section 3.3.2, Figure 5.8 indicates that there is an optimal production capacity for each machine for a given n Sensitivity Analysis Next we investigate the effect of parameter changes on the optimal solution. Consider a machine shop with a single machine. We use the following parameters: N = 2, h = 5, b = 8, k 1 = 0.5, p 1 = 8, k 2 = 2, p 2 = 14 and p 3 = 5. Figure show the 104

122 optimal order quantities and reorder points and the corresponding minimum costs when the leadtime distribution has parameter ν = 0.1, 0.5 and 1 respectively. Figure 5.9: The optimal order quantity vs. production rate (varying ν). Figure 5.10: The optimal reorder point vs. production rate (varying ν). 105

123 Figure 5.11: The minimum cost vs. production rate (varying ν). (1) Figure 5.11 shows that when ν is larger, i.e., the external orders come quicker, the minimal cost is lower. This is to be expected since shorter leadtimes imply less uncertainty about the inventory position. (2) From Figure 5.10 we see that the larger ν is, the lower the optimal reorder point r is. This is reasonable since if the leadtimes are short, there is no need to place orders from external supplier when the inventory position is still high. (3) In Figure 5.10 when production rate goes to 3 and the system approaches selfsufficiency, the reorder point r decreases from positive to negative infinity, and the difference between the three curves also decreases. Recall that in Section when there is no leadtime, r is always negative, as an effect of the fixed set-up cost; in Section 5.7.3, r can be positive or negative, as an effect of the positive leadtime. (4) Figure 5.9, when ν = 0.1, q is a decreasing function of production rate, which 106

124 is similar to what we have observed in Section 5.7.3; when ν = 0.5, q decreases much slower; when ν = 1, the q curve is similar to the one in Section where we assumed zero leadtimes. Thus, the simplification about zero leadtime is a reasonable approximation when the leadtime is small enough to be neglected. Next, we fix ν = 0.1 and plot the parameters of the optimal policies and corresponding minimum costs when the number of maximum allowed outstanding orders N is 1, 2, 3,4 and 5 respectively. Figure 5.12: The optimal order quantity vs. production rate (varying N). 107

125 Figure 5.13: The optimal reorder point vs. production rate (varying N). Figure 5.14: The probability that there are N outstanding orders (varying N). 108

126 Figure 5.15: The minimum cost vs. production rate (varying N). From Figure 5.15 we can see that when production rate is small, the more outstanding orders are allowed, the less the cost is. This is because we do not want to apply the emergency supplies too often since they cost more than the regular supplies. However, for larger production rate the situation reverses: the more flexibility we have, the higher is the cost. This is because when the reorder point decreases very negative, compared to the backlogging cost, the difference between the emergency supplies and regular supplies becomes less important. Thus in this situation we would rather take advantage of the emergency supplies to increase the inventory level immediately. Figure 5.14 confirms this: under the optimal policies, the probability that we turn to the emergency supplier is actually large. Thus, for a given production-inventory system, we can draw graphs similar to Figure 5.15 which illustrates the minimal cost functions for different N values, and their minimal function values at their optimal production rates u i. Then we can find the optimal N whose minimal function value is the smallest (In Figure 5.15 from the locally enlarged graph, we can see N = 3. ). Also we obtain the corresponding production rates u. Then we go back to Figure 5.12 and 5.13 and use these optimal 109

127 N and u to determine the optimal order quantity q and reorder point r. Going further back we can also calculate the steady-state distribution of the inventory level and inventory position under this optimal operation policy (N, u, q, r ). This problem is hence solved completely. 110

128 Chapter 6 Conclusions and Future Research 6.1 Conclusions In this thesis we have studied the optimization problem of a production-inventory system by modeling it as a stochastic fluid-flow system. We have considered a single product whose production and demand rates are piecewise constant functions determined by the environment. The inventory under continuous review thus can be viewed as a fluid process that fluctuates according to the evolution of the underlying environment process. In Chapter 2 we first studied the fluid model to establish the fundamental theory for the production- inventory system. The fluid level increases or decreases according to a rate modulated by a background stochastic process with finite state space. When the fluid level hits zero, it instantaneously jumps to a pre-determined positive level. Between two consecutive jumps the background process is a continuous time Markov chain. At the jump epoch the environment process jumps according to a transition matrix A. We have developed methods of computing the limiting behavior of this

129 system. For the special case A = I, we have derived an interesting decomposition property of the steady-state buffer content as the sum of two independent random variables: a U(0, q) random variable, and a steady-state buffer content in a standard fluid model with no jumps. We have also shown a specific case where the input rates are negative for all the background states, the limiting distribution of the fluid level is uniform, and independent of the background process. Beginning from Chapter 3, we study the production-inventory system using the theories developed in Chapter 2. We started from the basic EOQ model: the inventory level increases or decreases according to the changes of the production and demand rates; when the inventory on hand is zero, a supplementary order is placed and arrives instantaneously, and this order always has a fixed positive quantity q. We have proved that the optimal q which achieves the trade-off between fixed ordering cost and holding cost is given by what we called stochastic version of the EOQ formula, which is derived by replacing the deterministic demand rate in the classical EOQ formula with the expected net demand rate in steady state. We have also showed with numerical examples how to determine the optimal production rate u to achieve the best combination of outsourcing and inhouse-production. Then we have extended the basic model to allow backlogging: an external order is not placed until the inventory level reaches the reorder point r. We have derived methods to get the optimal (r, q, u ) policy, which achieves the trade-off point of production cost, fixed ordering cost, holding cost and backlogging cost. We have shown that under the optimal policy, r is given by the well-known newsboy solution. Particularly, in the special case where production is always less than the demand rate, we have proved that the optimality of the deterministic EOQ formula with backlogging still holds in this stochastic environment if one replaces the deterministic demand rate by the expected net demand rate in steady state. 112

130 In Chapter 4, we extended the production-inventory model to allow the order size to depend upon the environmental state when the order is placed. Clearly, if we can observe the environmental state and can base our inventory replenishment decisions on that information, this environment-dependent ordering policy is no worse than the simple (r, q) policy. In Chapter 4, we used three different methods to study the limiting behavior of the (inventory level, background environment) process, and derive the optimal ordering policy (q 1, q 2,..., q m ). In Chapter 5, we extended the model to allow stochastic leadtimes. We have considered three cases: orders are processed sequentially, in parallel, or the inter-arrival times of the outstanding orders have exponential distributions whose parameters depend on the number of outstanding orders. We assumed there exists an upper limit N of the number of outstanding orders. Thus this model can serve as the selective lost-sales model, or an emergency supplier model, or a reasonable approximation of the infinite-server parallel processing model in the literatures. We have derived the optimal policy (N, r, q, u ) which achieves the trade-off point of production cost, fixed ordering cost, holding cost, backlogging cost, and emergency ordering cost (or lost-sale penalty). Minimum cost and limiting distribution under the optimal policy are also calculated. Thus, we have developed a whole set of methodologies to solve increasingly complex stochastic production- inventory problems. On our way towards the ending point, as mentioned above, we have discovered quite a few interesting properties. All the results are also verified with numerical calculations. If we look retrospectively, every step can reduce to the previous settings as a special case until we eventually go back to the well-known deterministic inventory models. As stated in the individual chapters, these types of production-inventory systems are very common in reality. However, there are only few papers in literature which 113

131 have studied some special cases. The methodologies of fluid models developed in this thesis provide a systematic method to analyze to these kind of problems, and facilitate future extensions. 6.2 Future Research In this section we discuss a few possible extensions to the research presented in this thesis Model with Semi-Markov Process as Background process It seems feasible to relax the assumption of background CTMC to a semi-markov process (SMP). The aim is to prove that the results of the basic model remain valid in this more general situation. Clearly the results remain valid if the sojourn time in the SMP are phase-type distributions. The idea is to construct an appropriate larger CTMC. Since Phase-type distributions are dense in the set of all continuous distributions on [0, ), we believe one can show that the results hold for a semi- Markov background process with continuous sojourn times. We further believe that the results hold for more general semi-markov processes as long as the sample paths of the {X(t), t 0} process are not periodic with probability one. Rigorous proof of this remains to be shown. 114

132 6.2.2 Environment-Dependent Order Quantities and Reorder Points In Chapter 4 we developed methods to model the problem allowing environmentdependent order quantities. A further extension would be also allowing backorders. Thus we can consider policies that order q i where the inventory level falls to r i in state i. The problem there is to find optimal parameters q i s and r i s to minimize the holding, ordering, and backlogging costs. We expect the same methods can be used Numerically Stable Methods for the Stochastic Leadtime Model in Serial Processing System Repeated eigenvalues are unavoidable in Section 5.2 where we calculate the limiting distributions for the serial processing case in the model with stochastic leadtimes. We use Jordan Canonical form and the method in Putzer (1966) to in our numerical experiments. However, both methods experiences numerical difficulties in dealing with large-scale problems. Using the general model in Section 5.4 seems to be a good approximation when we let the parameters ν i be very close to each other. We believe further research is needed to develop a numerically stable method to analyze this model Environment-Dependent Ordering Policies with Stochastic leadtimes In Chapter 5 we considered the case of stochastic leadtimes under the restriction that the order quantity and the reorder point were independent of the state of the 115

133 environment. It will be interesting to allow environment-dependent order quantities and reorder points when the leadtimes are not negligible. We expect our current methods are already sufficient in solving this more complicated problem. However, the state space of the background process will become large and hence we need to develop numerically stable methods to carry out the computation. 116

134 Bibliography [1] Berman, O. and Perry, D. (2004). An EOQ model with state dependent demand rate. Eur. J. Operat. Res., In Press. [2] Berman, O., Stadje, W. and Perry, D. (2006). A Fluid EOQ Model with a Two-State Random Environment. Probab. Engrg. Inform. Sci. 20, [3] Browne, S. and Zipkin, P. (1991). Inventory Models with Continuous Stochastic Demands. Ann. Appl. Prob. 1, [4] El-Taha, M. (2002). A Sample-Path Condition for the Asymptotic Uniform Distribution of Clearing Processes. Optimization 51, [5] Fuhrmann, S. (1984) A note on the M/G/1 queue with server vacations, Operat. Res. 32, [6] Fuhrmann, S. and Cooper, R. (1985) Stochastic decomposition in the M/G/1 queue with generalized vacations, Operat. Res. 33, [7] Hadley, G and Whitin, T. (1963) Analysis of inventory systems. Prentice- Hall, N.J. [8] Heyman, P. and Sobel, M. (1982) Stochastic Models in Operations Research. Vol. I. McGraw-Hill, New York. [9] Kella, O. and Whitt, W. (1992). A storage model with a two state random environment. Operat. Res. 40, [10] Kulkarni, V. (1995). Modeling and Analysis of Stochastic Systems. Chapman and Hall, London. [11] Kulkarni, V. (1997). Fluid models for single buffer systems, in: J.H. Dshalalow (Ed.), Frontiers in Queueing, Models and Applications in Science and Engineering. Ed. J.H. Dshalalow, CRC Press, Boca Raton, FL, [12] Kulkarni, V. and Tzenova, E. (2002). Mean first passage times in fluid queues. Operat. Res. Lett. 30, [13] Kulkarni, V., Tzenova, E. and Adan, I. (2005). Fluid Models with Jumps, Stochastic Models. 21, [14] Mitra, D. (1988). Stochastic theory of a fluid model of producers and consumers coupled by a buffer. Adv. Appl. Prob. 20, [15] Miyazawa, M. and Takada, H. (2002) A matrix exponential form for hitting probabilities and its application to a Markov-modulated fluid queue with downward jumps. J. Appl. Prob. 39,

135 [16] Neuts, M. F. (1981). Matrix Geometric Solutions in Stochastic Models. Johns Hopkins University Press, Baltimore, MD. [17] Putzer, E. J. (1966). Avoiding the Jordan canonical form in the discussion of linear systems with constant coefficients. Amer. Math. Monthly 73, 2-7. [18] Serfozo, R. and Stidham, S. (1978). Semi-stationary clearing processes. Stoch. Proc. Appl. 6, [19] Sengupta, B. (1989) Markov processes whose steady state distribution is matrixexponential with an application to the GI/PH/1 queue. Adv. in Appl. Prob 21, [20] Serfozo, R. and Stidham, S. (1978). Semi-stationary clearing processes. Stoch. Proc. Appl. 6, [21] Shanthikumar, J. (1986) On stochastic decomposition in M/G/1 type queues with generalized server vacations, Operat. Res. 36, [22] Song, J. and Zipkin, P. (1996). Inventory Control with Information About Supply Conditions. Mgmt. Sci. 42, [23] Song, J. and Zipkin, P. (1993). Inventory control in a fluctuating demand environment.operat. Res. 41, [24] Whitt, W. (1981). The stationary distribution of a stochastic clearing process. Operat. Res. 29, [25] Zipkin, P. (1986). Stochastic leadtimes in continuous time inventory models. Naval Res. Logist. Quart. 33, [26] Zipkin, P. (2000). Foundations of Inventory Management. McGraw-Hill, Boston. 118