Inventory Control under Substitutable Demand: A Stochastic Game Application

Transcription

1 Inventory Control under Sustitutale Demand: A Stochastic Game Application Zeynep Müge Avṣar Department of Operations Planning and Control Faculty of Technology Management Eindhoven University of Technology P.O. Box 513, 56 MB Eindhoven, The Netherlands Melike Baykal-Gürsoy Industrial Engineering Department Rutgers, The State University of New Jersey Piscataway, NJ , USA June 3, 15 Astract Sustitutale product inventory prolem is analyzed using the concepts of stochastic game theory. It is assumed that there are two sustitutale products that are sold y different retailers and the demand for each product is random. Game theoretic nature of this prolem is the result of sustitution etween products. Since retailers compete for the sustitutale demand, ordering decision of each retailer depends on the ordering decision of the other retailer. Under the discounted payoff criterion, this prolem is formulated as a two-person nonzero-sum stochastic game. In the case of linear ordering cost, it is shown that there exists a Nash equilirium characterized y a pair of stationary ase stock strategies for the infinite horizon prolem and this is the unique Nash equilirium within the class of stationary ase stock strategies. 1 Introduction This study focuses on investigating the equilirium strategies for sustitutale product inventory control systems within the class of stationary ase stock strategies. When different products are sold y different retailers, sustitution etween these products causes the retailers to decide on their order quantities in a competitive environment, and thus, the game theoretic nature of the prolem arises. In this article, a nonzero-sum discounted stochastic game formulation is given for the two-product prolem. The retailers oserve their inventory levels periodically and take actions according to their ordering policies. It is assumed that oth retailers ehave rationally, i.e., each retailer tries to optimize his own payoff. The set-up cost of each retailer is assumed to e zero. Sustitutale product inventory prolem was first studied y McGillivray and Silver [7] in the Economic Order Quantity(EOQ) context. Later, Parlar and Goyal [1] and Khouja and Mehrez and Rainowitz [5] gave single-period formulations for an inventory system with two sustitutale products independently of each other. In [8], Parlar proposed a Markov Decision Process model to find the optimal ordering policies for perishale and sustitutale products from the point of view of one retailer. Parlar s study in [9] is a game theoretic analysis of the inventory control under sustitutale demand. He modeled the two-product singleperiod prolem as a two-person nonzero-sum game and showed that there exists a unique Nash equilirium. 1

2 As an extension of the model in [9], Wang and Parlar [15] studied the three-product single-period prolem. In this study, the work in [9] is extended -to- the infinite horizon and lost sale case. The solution of the corresponding nonzero-sum stochastic game is considered over the class of stationary ase stock strategies ecause this makes oth implementation of the strategies and analysis of the prolem easier. It is shown that under the discounted payoff criterion retailers possess a stationary ase stock Nash strategy pair which is the unique Nash equilirium within the class of stationary ase stock strategies. Stationary ase stock strategies are expressed y constant order-upto-levels. If the inventory at the eginning of a period is elow -the- order-upto-level, then an order is given to ring inventory amount to that level; otherwise, no action is taken. Also, it is oserved that cooperation of retailers leads to a etter total payoff than the sum of the individual payoff amounts of the retailers in the non-cooperative case. There are two other models that are related to the sustitutale product inventory model. In [14], Veinott considered a single retailer inventory prolem under acklogging, udget and/or capacity limitations. He gave conditions to ensure that the ase stock ordering policy is optimal for the expected discounted cost criterion. Later, Ignall and Veinott [4] considered the same model and otained new conditions under which a myopic ordering policy (a policy of minimizing the expected cost in one period only) is optimal for a sequence of periods. An important one of these conditions is the so-called sustitute property. This property holds when the myopic policy is such that increasing the initial inventory of any product does not increase the stock on hand after ordering of any other product. This property arises in the models that include some kind of product sustitution such as sustituting storage space for one product for that of another, or the demand for a product at one location for that at another location in a multi location inventory model. But, the sustitution etween products in the sense of this paper destroys the optimality of the myopic policy. Since there is only one retailer, there is no competition in this model. In [6], Kirman and Soel considered a dynamic oligopoly model with inventories. In oligopolies, a small numer of firms produce homogeneous or comparale goods competitively. Firms compete y increasing the demand for their product via advertisement, pricing or y keeping inventories. They considered the case of acklogging and discounting, and analyzed this model using the stochastic game approach. For the infinite horizon case, they gave conditions under which the game has a Nash equilirium and each firm has a ase stock type myopic policy. One condition requires that the demand function is smooth. This condition eliminates the cases such as all customers always choose to uy from the firm with the lowest price. Although there is competition in this model, the sustitution etween products is not considered. Organization of this article is as follows: The prolem and the notation are introduced, and the model is developed in section. Then, in section 3, analyses are presented for the use of stationary ase stock strategies and cooperation of the retailers is discussed. Model of the Sustitutale Product Inventory Prolem In the analysis of the sustitutale product inventory prolem over infinite horizon, concepts of nonzero-sum stochastic games are used. Two retailers of different products who compete for the sustitutale demand of these products are the players of the game. Demand distriutions of the products and the sustitution rates are known y oth players. For the nonzero-sum stochastic game formulation, Nash equilirium is considered. Unilateral deviations of either of the players from his Nash strategy do not improve his expected payoff. Retailers oserve their inventory levels at the eginning of each period and make their ordering decisions accordingly. A period is named (indexed) y the numer of periods from the eginning of that period until the end of the planning horizon, i.e., period n means that there are n decision epochs to go until the end of the planning horizon. Let X n and Y n e the independently and identically distriuted(i.i.d) random variales denoting the demand -for- product 1 and, respectively, in period n. Product i is sold for q i

3 per unit, i = 1,. Order-ing- cost is a linear function of the order quantity Q in for product i, i = 1,, in period n. c i, that satisfies < c i < q i, is the ordering- cost per unit of product i, i = 1,. Orders are delivered instantaneously. l i is the unit lost sale cost and h i is the inventory holding cost per unit of product i per period. Sustitution rates are given as the proailities that a customer switches from one type of product to the other when the product demanded is sold out. a () is the proaility that a customer of product 1 () switches to product (1) given that product 1 () is sold out. Let I n and J n e the inventory levels of retailers I and II, respectively, at the eginning of period n. At each epoch n, (I n, J n ) denotes the state of the stochastic process and (Q 1n, Q n ) denotes the action pair taken y the retailers. Then, z 1n = I n + Q 1n and z n = J n + Q n are the inventory levels just after the orders are replenished. The ackward dynamic equations associated with the state variales are given as: I n 1 = [z 1n x n [y n z n ] + ] + and J n 1 = [z n y n a[x n z 1n ] + ] + where [a] + = max, a}. Note that if retailer II cannot satisfy demand Y n fully, then the remaining demand (Y n z n ) + is satisfied y Retailer I and vice versa. Let P(I,J)(z 1 1 I)(z J) e the one-period expected payoff for the first player in state (I, J) when the order quantities are Q 1 and Q, i.e., when the order-upto-levels are z 1 = I + Q 1 and z = J + Q. By suppressing the suscript n, the one-period expected payoff has the following form: p 1 (I,J)(z 1 I)(z J) = l 1E[(X z 1 ) + ]+c 1 (z 1 I)+h 1 E[z 1 X (Y z ) + ] q 1 E[min X, z 1 +min (Y z ) +, (z 1 X) + ]. The second retailer s expected payoff is defined similarly. For the sake of simplicity of the analysis, assume that the nonnegative random demand variales X and Y have continuous density functions f and g, respectively, with finite expectations. Let f() =, g() =. The corresponding cumulative and complementary cumulative functions will e denoted y F, G and F, Ḡ, respectively. One-period expected payoff p 1 (I,J)(z 1 I)(z J) has the following form: p 1 (I,J)(z 1 I)(z J) = l 1 E(X) (q 1 + l 1 ) +h 1 (z 1 x) f(x)dx + c 1 (z 1 I) (q 1 + h 1 ) (q 1 + h 1 ) z z1 z+ z 1 x z + z 1 x xf(x)dx (q 1 + l 1 )z 1 f(x)dx z 1 (y z ) g(y)f(x)dydx (z 1 x) g(y)f(x)dydx. Parlar [9] investigated this prolem for the single-period case. He analyzed the -reward- function c 1 and showed that it is concave in z 1. He also proved that there exists a unique Nash equilirium specified with order-upto-levels, say S 1 and S for retailers I and II, respectively. For the discrete demand case, it is possile to numerically solve the single-period prolem although the size of the state space may make it impractical. Under the long-run average payoff criterion, the nonlinear programming formulation developed y Filar et.al. [] can e used to compute Nash strategies. If the discounted payoff criterion is considered, then NLP due to Raghavan and Filar [11] is availale. 3 Stationary Base Stock Nash Strategies The purpose of this section is to investigate Nash equilirium of the infinite horizon sustitutale product inventory prolem within the class of stationary ase stock strategies. To this end, first the finite horizon prolem is analyzed from the viewpoint of retailer I y assigning a stationary ase stock strategy to retailer II. The results otained are then extended for the infinite horizon prolem and it is oserved that when retailer II uses a stationary ase stock strategy the optimal strategy of retailer I is also a stationary ase 3

4 stock strategy. Finally, existence and uniqueness of a Nash solution within the class of stationary ase stock strategies are proved. In the multi-period model analyzed in this article, each retailer tries to minimize his expected payoff. Define L(z 1, z ) as one-period expected payoff except the ordering cost P 1 (z 1 I) as follows: L 1 (z 1, z ) = p 1 (I,J)(z 1 I)(z J) c 1(z 1 I) = l 1 (z 1 x)f(x)dx (q 1 + l 1 )z 1 + l 1 E(X) +(q 1 + h 1 ) z+ z 1 x (z 1 x [y z ] + )g(y)f(x)dydx. Let C 1n (I, J) represent the minimum expected discounted payoff of retailer I for the remaining n periods until the end of the planning horizon given that the eginning inventory I n (J n ) of retailer I (II) is I (J) and the inventory level of product 1 () is z 1 (z ) just after the replenishment. For player II, C n (I, J) is defined similarly. The discount factor is assumed stationary and will e denoted y γ, < γ < 1. C 1n (I, J) satisfies the following functional equation: C 1n (I, J) = min z1 I c 1 (z 1 I) + L 1 (z 1, z ) + γ C 1(n 1) ([z 1 x [y z ] + ] +, [z y a[x z 1 ] + ] + ) g(y)f(x)dydx, where the first two terms at the right hand side correspond to the one-period expected payoff. Here, optimal action of retailer I (the minimizing value of z 1 in the aove equation) is determined for a given order-uptolevel z of retailer II. Note that the function that is minimized has a constant part, i.e., c 1 I, and a variale part, say D 1n (z 1, z ). Thus, C 1n (I, J) = min D 1n (z 1, z ) c 1 I, z 1 I and the minimization is performed only on D 1n (z 1, z ). Consider the finite horizon prolem for retailer I when a stationary ase stock strategy is employed y retailer II. Relative to the initial inventories of product less than or equal to z, retailer II starts every period with z units of product. As in Scarf[1], the results presented in this section are otained y an inductive analysis of D 1n (z 1, z ): D 1n (z 1, z ) = c 1 z 1 + L 1 (z 1, z ) + γ C 1(n 1) ([z 1 x [y z ] + ] +, [z y a[x z 1 ] + ] + ) g(y)f(x)dydx, for every n. Considering the value of I n 1 for different pairs of demand values, D 1n (z 1, z ) may e further decomposed as follows: D 1n (z 1, z ) = c 1 z 1 + L 1 (z 1, z ) + γg(z ) z+ z 1 x +γ z ( +γc 1(n 1) (, z ) C 1(n 1) (z 1 x, z ) f(x)dx C 1(n 1) ((z 1 x (y z ), z ) g(y)f(x)dydx z + z 1 x g(y)f(x)dydx + F (z 1 ) If D 1n (z 1, z ) is convex in z 1 for a given order-upto-level z of retailer II, then optimal strategy of retailer I is a ase stock strategy. Order-upto-level of this strategy is the minimizing point of D 1n (z 1, z ), which will e denoted y S 1n. Note that S 1n is a function of z. Lemma shows that for a given z in [, ), D 1n (z 1, z ) is convex in z 1 and the minimizing point S 1n is greater than zero. Note that S 1n > ecause lim z1 z 1 D 1n (z 1, z ) < for every n. 4 ).

5 Lemma 1 If retailer II uses a stationary ase stock strategy with order-upto-level z, then for n = 1,,... (i) D 1n (z 1, z ) is convex in z 1, (ii) lim z1 z 1 D 1n (z 1, z ) <. Proof: The proof is given y induction on the numer of periods remaining. For period n = 1, D 11 (z 1, z ) = c 1 z 1 + L 1 (z 1, z ) holds y taking C 1 =. The partial derivative of D 11 (z 1, z ) is given using the Leinitz s rule of differentiation: Then, z 1 D 11 (z 1, z ) = c 1 (q 1 + l 1 ) z 1 lim z 1 (q 1 + h 1 ) z 1 f(x)dx + h 1 z + z 1 x z 1 D 11 (z 1, z ) = c 1 (q 1 + l 1 ) f(x)dx g(y)f(x)dydx. = (q 1 + l 1 c 1 ) <, f(x)dx since q 1 > c 1, and the proof of (ii) is complete. The second partial derivative of D 11 (z 1, z ) is ( D 11 (z 1, z ) = l 1 f(z 1 ) + (q 1 + h 1 ) g(z + z 1 x ) f(x) ) dx + f(z 1 )G(z ), which proves (i). Assume that the lemma is true for periods, 3,..., n. By the induction assumption, the optimal strategy of retailer I in period n is to order upto S 1n if the inventory level is elow S 1n and not to order if it is aove S 1n. Hence, C 1n (I, z ) = c 1 (S 1n I) + C 1n (S 1n, z ) = c 1 I + D 1n (S 1n, z ) if I < S 1n, c 1 I + D 1n (I, z ) if I S 1n. To show that the lemma is true for period (n + 1), D 1(n+1) (z 1, z ) is rewritten elow using the value of C 1n in (1) y comparing S 1n and the inventory level at the eginning of period n, I n. Note that I n may take values elow or aove S 1n. This is ecause z 1 may e greater than or equal to S 1n and the demand in period (n + 1) determines I n. For z 1 > S 1n, the following inequalities are used in writing D 1(n+1) (z 1, z ): I n = (z 1 x) (y z ) < S 1n I n = (z 1 x) (y z ) S 1n I n = z 1 x < S 1n if z 1 S 1n < x z 1, y < z, I n = z 1 x S 1n if x z 1 S 1n, y < z, if z 1 S 1n < x < z 1, z y z + z1 x or x z 1 S 1n, z + z1 x S1n y z + z1 x, if x z 1 S 1n, z y z + z1 x S1n. Now, using (1) for each of the ranges of x and y aove, D 1(n+1) (z 1, z ) is written as follows: D 1(n+1) (z 1, z ) = c 1 z 1 + L 1 (z 1, z ) + γd 1n (S 1n, z ) γc 1 +γ [z1 S 1n] + z+ z 1 x ( z1 x [y z ] +) g(y)f(x)dydx z+ z 1 x S 1n (D 1n (z 1 x [y z ] +, z ) D 1n (S 1n, z )) g(y)f(x)dydx. (1) 5

6 When the fourth term aove is comined with L 1, D 1(n+1) (z 1, z ) takes the following form: D 1(n+1) (z 1, z ) = c 1 z 1 + l 1 (z 1 x)f(x)dx (q 1 + l 1 )z 1 + l 1 E(X) + γd 1n (S 1n, z ) +(q 1 + h 1 γc 1 ) [z1 S 1n] + +γ z+ z 1 x z+ z 1 x S 1n ( z1 x [y z ] +) g(y)f(x)dydx (D 1n (z 1 x [y z ] +, z ) In order to prove (ii), take the first partial derivative with respect to z 1. z 1 D 1(n+1) (z 1, z ) = A 1 (z 1, z ) [z1 S 1n] + +γ z+ z 1 x S 1n D 1n (S 1n, z )) g(y)f(x)dydx. () z 1 D 1n (z 1 x [y z ] +, z ) g(y)f(x)dydx, (3) where A 1 (z 1, z ) is the derivative of the first six terms of D 1(n+1) (z 1, z ) in (), i.e., A 1 (z 1, z ) = c 1 (q 1 + l 1 ) f(x)dx + (h 1 γc 1 ) z 1 (q 1 + h 1 γc 1 ) z + z 1 x g(y)f(x)dydx. f(x)dx Since S 1n > y the induction assumption, the second term of z 1 D 1(n+1) (z 1, z ) vanishes at z 1 =. D 1(n+1) (z 1, z ) is decreasing at small z 1 ecause the first partial derivative takes a negative value when z 1 goes to zero as shown elow: lim z 1 z 1 D 1(n+1) (z 1, z ) = c 1 (q 1 + l 1 ) f(x)dx = (q 1 + l 1 c 1 ) <. To show (i), the second partial derivative with respect to z 1 is analyzed. At any z 1 < S 1n, z1 D 1(n+1) (z 1, z ) = l 1 f(z 1 ) + (q 1 + h 1 γc 1 )f(z 1 )G(z ) and it is nonnegative since q 1 > c 1. At z 1 S 1n, z1 +(q 1 + h 1 γc 1 ) D 1(n+1) (z 1, z ) = l 1 f(z 1 ) + (q 1 + h 1 γc 1 )f(z 1 )G(z ) + (q 1 + h 1 γc 1 ) + γ( G(z )f(z 1 S 1n ) + S 1n + γ z+ z 1 x S 1n g(z + z 1 x S 1n z1 g(z + z 1 x ) f(x) dx g(z + z 1 x S 1n ) f(x) dx, ) f(x) dx ) D 1n (z 1, z ) z1=s z 1n 1 D 1n (z 1 x [y z ] +, z ) g(y)f(x)dydx, where the last term is nonnegative since, y the induction assumption, D 1n (z 1 +h, z ) is convex in z 1 for any h. The first three terms are also nonnegative. z 1 D 1n (z 1, z ) z1=s 1n is either zero with a finite S 1n value or 6

7 negative with infinite S 1n. If S 1n is finite, then the fourth term is zero. Otherwise, the only case that needs to e analyzed is the first case where z 1 < S 1n. Remark 1: (i) The function, defined y the first six terms of D 1(n+1) (z 1, z ) in (), is convex in z 1. The derivative of this function is exactly the same as D 11 (z 1, z )/z 1 except that every h 1 is replaced with (h 1 γc 1 ). The second partial derivative of this function is ( A 1 (z 1, z ) z1 = l 1 f(z 1 ) + (q 1 + h 1 γc 1 ) g(z + z 1 x ) f(x) ) dx + f(z 1 )G(z ), z 1 which is nonnegative at z 1 for any given z. (ii)d 1(n+1) (z 1, z ) and z 1 D 1(n+1) (z 1, z ) are continuous at z 1 = S 1n. Next, it is shown that for any given nonnegative z, D 1n (z 1, z ) attains its minimum at a finite S 1n. If D 1n (z 1, z ) has multiple minima, then S 1n is the smallest z 1 value at which the minimum is attained. It is proved that the sequence S 1n } n=1 is monotonic nondecreasing and converges to a finite limit as N. Lemma If retailer II uses a stationary ase stock strategy with order-upto-level z, then (i) for every n = 1,,..., D 1n (z 1, z ) is minimized at a finite z 1 value, (ii) S 1(n+1) S 1n for n = 1,,..., (iii) S 1n } n=1 is convergent. Proof: (i) The proof is given y induction. In the analysis of the curve z 1 D 11 (z 1, z ) = in the (z 1, z ) plane, implicit differentiation gives the derivative of z with respect to z 1, which will e denoted y dz11. The superscript 11 is used ecause it is otained from the first retailer s cost function in period n = 1. is derived from Since l 1 f(z 1 ) + (q 1 + h 1 )f(z 1 )G(z ) + (q 1 + h 1 ) ( dz ) z dz 11 = 1 l 1f(z 1 ) + (q 1 + h 1 )f(z 1 )G(z ) (q 1 + h 1 ) z 1 g(z + z1 x )f(x)dx, g(z + z 1 x )f(x)dx =. is negative, the curve z 1 D 11 (z 1, z ) = is strictly decreasing in the (z 1, z ) plane. Thus, given any z a lower ound for the optimal z 1 value is otained y letting z go to infinity in z 1 D 11 (z 1, z ) =. Denote this lower ound y z 11. It satisfies z 11 f(x)dx = q1+l1 c1 q 1+l 1+h 1. Similarly, for any given z let z 11 denote the highest value of the optimal z 1. z 11 is the solution of z 1 D 11 (z 1, ) = where z is equal to zero, i.e., Since q1+l1 c1 q 1+l 1+h 1 z11 (q 1 + h 1 ) G( z 11 x )f(x)dx + l 1 F ( z 11 ) = q 1 + l 1 c 1. < 1, it is oserved that z <. Also, since dz < at any (z 1, z ), z 11 <. Hence, given any z [, ) the convex cost function D 11 (z 1, z ) is minimized at a finite z 1 value, say S 11, etween z 11 and z 11. 7

8 Assuming that (i) is true for period n, the curve z 1 D 1(n+1) (z 1, z ) = is analyzed in the (z 1, z ) plane in order to show that (i) also holds for period (n + 1). The last term of D 1(n+1) (z 1, z ) in () vanishes for z 1 < S 1n and the first partial derivative of D 1(n+1) (z 1, z ) ecomes A 1 (z 1, z ). For that reason, efore proceeding with the second step of induction, ehavior of A 1 (z 1, z ) is investigated. Implicit differentiation of A 1 (z 1, z ) = gives dz 1 = 1 l 1f(z 1 ) + (q 1 + h 1 γc 1 )f(z 1 )G(z ) (q 1 + h 1 γc 1 ) z 1 g(z + z1 x )f(x)dx which has the same form as the derivative otained for n = 1 except that coefficient (q 1 +h 1 γc 1 ) is included instead of (q 1 +h 1 ). Since q 1 > c 1 and < γ < 1, the curve A 1 (z 1, z ) = is strictly decreasing in the (z 1, z ) plane. Using the same reasoning for n = 1, it is easy to see that, given any z [, ), A 1 (z 1, z ) vanishes at a finite value of z 1. The lower ound z 1 for the optimal value of z 1 satisfies z 1 f(x)dx = q1+l1 c1 q 1+l 1+h 1 γc 1 and the upper ound z 1 satisfies z1 (q 1 + h 1 γc 1 ) G( z 1 x z1 )f(x)dx + l 1 f(x)dx = q 1 + l 1 c 1. Note that z 11 < z 1. For a given z, let z 1 z e the value of z 1 at which A 1 (z 1, z ) is equal to zero. From the oservations aove, z 1 z <. Also, since A 1 (z 1, z ) = z1 D 11 (z 1, z ) γc 1 z 1 z+ z 1 x g(y)f(x)dydx. and z 1 D 11 (z 1, z ) is zero at z 1 = S 11 and S 11 >, A 1 (S 11, z ) = γc 1 S11 z+ S 11 x g(y)f(x)dydx <. The relation aove shows that S 11 < z 1 z ecause D 11 (z 1, z ) is a convex function of z 1. By induction, assume that given any z [, ), D 1n (z 1, z ) attains its minimum at a finite value S 1n less than or equal to z 1 z, i.e., z 1 D 1n (z 1, z ) = at z 1 = S 1n such that S 1n z 1 z. Note that, the integration term in (3) vanishes at z 1 = S 1n. Over the range of x and y values given y the doule integration term in (3), (z 1 x [y z ] + ) S 1n holds if z 1 > S 1n. Since D 1n (z 1, z ) is a convex function of z 1 and its minimum is achieved at S 1n, z 1 D 1n (w, z ) is nonnegative at w = z 1 x [y z ] + such that w S 1n. Hence, the integration term in (3) is nonnegative for z 1 > S 1n. On the other hand, since A 1 (z 1, z ) is the first partial derivative of a convex function as pointed out y remark 1 and A 1 (z 1 z, z ) =, A 1 (z 1, z ) for z 1 z 1 z. Then, it is oserved that z 1 D 1(n+1) (z 1, z ) for z 1 z 1 z. This shows that D 1(n+1) (z 1, z ) is nondecreasing in z 1 for z 1 z 1 z. By the convexity of D 1(n+1) (z 1, z ), the minimizing point of D 1(n+1) (z 1, z ), i.e., S 1(n+1), is less than or equal to z 1 z. (ii) Since D 1(n+1) (z 1, z ) is a convex function and its minimum is achieved at z 1 = S 1(n+1), it is sufficient to show that z 1 D 1(n+1) (z 1, z ) < for z 1 < S 1n and every n 1,,...}. When z 1 S 1n, the first partial derivative z 1 D 1(n+1) (z 1, z ) is equal to A 1 (z 1, z ). Since A 1 (z 1, z ) is the derivative of a convex function and S 1n z 1 z from (i), z 1 D 1(n+1) (z 1, z ) < for z 1 < S 1n. Then, y the use of convexity of D 1(n+1) (z 1, z ) in z 1, S 1(n+1) S 1n holds. (iii) Convergence of S 1n } n=1 results from the oservation that S 1n} n=1 is a monotonic nondecreasing sequence in compact space [, z 1 z ]. 8

9 Over a finite horizon, say N-period horizon, order-upto-levels S 1n for n = 1,..., N, form optimal nonstationary ase stock strategy of retailer I given the second retailer s stationary ase stock strategy with order-upto-level z. In order to determine optimal strategy of the first retailer over infinite horizon, the limiting ehavior of the payoff function C 1n (I, z ) is analyzed and the corresponding functional equation is given in lemma 3. Lemma 3 Given the second retailer s stationary ase stock strategy with order-upto-level z, C 1n (I, z ) converges uniformly for all I in a finite interval. The limit function C 1 (I, z ) is a continuous function of I and it is the unique ounded solution to C 1 (I, z ) = min z1 I c1 (z 1 I) + L 1 (z 1, z ) + γ C 1 ([z 1 x [y z ] + ] +, z ) g(y)f(x)dydx }. Proof: In lemma (i), it is shown that, for any z [, ), an upper ound for S 1n, n = 1,,..., is z 1 z which is given y the solution of A 1 (z 1 z, z ) =. Also, z 1 < is an upper ound for z 1 z. Since the expected values E(X) and E(Y ) are also assumed to e finite, C 1n (I, z ) is ounded for all I in [, z 1 ]. In order to estalish the convergence of C 1n (I, z ), the notation and the method introduced y Bellman, Glickserg and Gross [1] and later used y Iglehart [3] is considered. Let T 1 e the operator defined as follows: T 1 (z 1, I, C 1 z ) = c 1 (z 1 I) + L 1 (z 1, z ) + γ C 1 ([z 1 x [y z ] + ] +, z ) g(y)f(x)dydx. By assuming C 1 (I, z ) = for every I the optimality equation can e written as C 1(n+1) (I, z ) = min z1 I T 1 (z 1, I, C 1n z )} for every n. Let z1n I denote the optimal z 1 value given the initial inventory is I. Note that z1n I S1n if I < S = 1n, I if I S 1n. By the optimality of z I 1(n+1) and zi 1n in periods (n + 1) and n, respectively, C 1(n+1) (I, z ) = T 1 (z I 1(n+1), I, C 1n z ) T 1 (z I 1n, I, C 1n z ), C 1n (I, z ) = T 1 (z I 1n, I, C 1(n 1) z ) T 1 (z I 1(n+1), I, C 1(n 1) z ), hold. These relations give the following upper ound for the difference etween C 1(n+1) (I, z ) and C 1n (I, z ): C 1(n+1) (I, z ) C 1n (I, z ) max k=n,n+1 T1 (z I 1k, I, C 1n z ) T 1 (z I 1k, I, C 1(n 1) z ) } max k=n,n+1 γ Then, the aove inequality is rewritten as C 1n ([z I 1k x [y z ] + ] +, z ) C 1(n 1) ([z I 1k x [y z ] + ] +, z ) g(y)f(x)dydx max C1(n+1) (I, z ) C 1n (I, z ) } γ max C1n (I, z ) C 1(n 1) (I, z ) }. I z 1 I z 1 Using this relation successively, one otains max C1(n+1) (I, z ) C 1n (I, z ) } γ n max C 11 (I, z ) }, n = 1,,.... I z 1 I z 1 Since C 11 (I, z ) is ounded for all I in I z 1, the series n= C 1(n+1)(I, z ) C 1n (I, z ) converges for I z 1, which means that the series n= (C 1(n+1)(I, z ) C 1n (I, z )) converges asolutely. This }. 9

10 implies that lim n (C 1(n+1) (I, z ) C 1n (I, z )) =. As a result, C 1n (I, z ) converges for every I in the finite interval [, z 1 ]. Note that the convergence is uniform. From (1), one can easily oserve that C 1n (I, z ) is a continuous function. This leads to the continuity of the limit function C 1 (I, z ). In order to show that C 1 (I, z ) satisfies the functional equation given in the lemma, consider lim min n z 1 I min z 1 I T1 (z 1, I, C 1(n 1) z ) } = lim n C 1n(I, z ). Note that z 1 z 1. For any I, T 1 (z 1, I, C 1(n 1) z ) is a continuous function of z 1. Hence, the limit and the minimization operations can e interchanged as follows: } lim T 1(z 1, I, C 1(n 1) z ) = C 1 (I, z ). n For I eing restricted to the interval [, z 1 ], y the ounded convergence theorem, the limit operation and the doule integral in T 1 (z 1, I, C 1(n 1) z ) can e interchanged and the aove relation can e written as } C 1 (I, z ) = min T 1 (z 1, I, lim C 1(n 1) z ) z 1 I n = min z 1 I T 1(z 1, I, C 1 z )}. Since C 1n (I, z ) is a contraction, y the fixed point theorem for the contraction mappings C 1 (I, z ) is the unique ounded solution. The next step is to determine the optimal strategy of retailer I for infinite horizon prolem. For this purpose, ehavior of lim n D 1n (z 1, z ) is investigated and it is oserved that this limiting function is minimized at lim n S 1n. Lemma 4 Over infinite horizon, if retailer II uses a stationary ase stock strategy with order-upto-level z, then the first retailer s optimal strategy is also a stationary ase stock strategy with order-upto-level z 1 z. Proof: The proof is ased on the analysis of lim n D 1n (z 1, z ), i.e., ( c 1 z 1 + L 1 (z 1, z ) + γ lim n ) C 1(n 1) ([z 1 x [y z ] + ] +, z ) g(y)f(x)dydx. Denote lim n D 1n (z 1, z ) y D 1 (z 1, z ). Using the ounded convergence theorem on the right hand side of the aove equation, one otains D 1 (z 1, z ) = c 1 z 1 + L 1 (z 1, z ) + γ C 1 ([z 1 x [y z ] + ] +, z ) g(y)f(x)dydx. For the infinite horizon prolem, D 1 (z 1, z ) determines the optimal strategy of the first retailer as a response to his opponent s stationary ase-stock strategy with order-upto-level z. D 1 (z 1, z ) is convex ecause it is the limit of a sequence of convex functions. From lemma, S 1n } n=1 is a monotonic nondecreasing sequence. In order to show that lim n S 1n is z 1 z and it is the minimizing point of D 1 (z 1, z ), one needs to prove that z 1 z is the least upper ound for the range of sequence S 1n } n=1. Consider the first partial derivative of D 1(n+1) (z 1, z ) in (3). At z 1 = S 1n, the derivative is equal to A 1 (S 1n, z ), which is nonpositive since S 1n z 1 z. For every z 1 < z 1 z, A 1 (z 1, z ) is negative. This shows that a point in [, z 1 z ) can not e the least upper ound for the range of S 1n } n=1. Thus, since the range of S 1n } n=1 is (, z 1 z ], z 1 z is the least upper ound. 1

11 In order to show that z 1 z is the solution for the infinite horizon prolem, consider the first partial derivative of the limiting cost function D 1(n+1) (z 1, z ). From (3), z 1 D 1 (z 1, z ) = A 1 (z 1, z ) + γ [z 1 z 1 z ] + z+ z 1 x z 1 z z 1 D 1 (z 1 x [y z ] +, z ) g(y)f(x)dydx. For z 1 < z 1 z, A 1 (z 1, z ) takes negative values. At z 1 = z 1 z, the first derivative of D 1 (z 1, z ) with respect to z 1 is A 1 (z 1 z, z ) which is zero. Then, since D 1 (z 1, z ) is convex, z 1 z is the minimizing point. Lemma 4 implies that if one retailer restricts himself to stationary ase stock strategies and if this is declared y that retailer, his opponent can also restrict himself to the stationary ase stock strategies for optimizing his payoff. The results given aove are all otained under the assumption that retailer II uses a stationary ase stock strategy. If the first retailer s strategy is given as a stationary ase stock strategy, then the same results follow for retailer II. Based on these oservations, in the remaining of this section it is shown that there exists a Nash equilirium which is unique within the class of stationary ase stock strategies. Below, Nash equilirium of stationary ase stock strategies is defined for the two-person nonzero-sum stochastic game formulation of the infinite horizon sustitutale product inventory control prolem. The payoff functions of this game are D 1 and D, the latter of which is given y the limit of D n as n tends to infinity. Definition 1 (S 1, S ) is called a Nash equilirium relative to initial inventory levels [, S 1] [, S ] if D 1 (S 1, S ) D 1 (z 1, S ), for all z 1, and D (S 1, S ) D (S 1, z ), for all z. Nash condition implies that, if a retailer takes his Nash strategy his opponent can not improve his payoff y taking any strategy other than his Nash strategy. Before proceeding with the main theorem for the existence and uniqueness of a Nash equilirium of the sustitutale product inventory prolem within the class of stationary ase stock strategies, it should e pointed out that, from now on, the case with infinite order quantities will not e considered. Remark : If one of the retailers gives an order of infinite units, then he can satisfy every customer for his product, i.e., no one switches to the other product. In such a case, the other retailer would not have any hope of having sustitutale demand and so he decides to satisfy only the demand for his product. In other words, for this retailer the prolem reduces to a single player prolem. The former retailer goes into ankruptcy ecause the expected value of demand is finite for each product. Hence, if a retailer orders infinitely many units, then his cost ecomes infinite (and this is the worst he could do). This remark leads to another way of oserving the validity of lemma (i) when a retailer orders infinite units. Then, his cost ecomes infinite regardless of his opponent s order-upto-level, i.e., lim z1 D 1n (z 1, z ) = for all z and lim z D n (z 1, z ) = for all z 1. As shown efore, for any order-upto-level z [, ) of the second retailer, retailer I chooses his own order-upto-level in the finite interval [, z 1 ]. Such ounds are otained also for the second retailer. For a given z 1 [, ), let z A (z 1, z ) = c (q + l ) g(y)dy + (h γc ) g(y)dy z (q + h γc ) 11 z z 1+ z y a f(x)g(y)dxdy.

12 Then, the implicit differentiation of A (z 1, z ) = gives l g(z ) dz + (q + h )g(z )F (z 1 ) dz + (q + h ) From this relation, one otains dz = ( a dz ) z f(z 1 + z y a )g(y)dy =. (q + h γc ) z f(z 1 + z y ( a )g(y)dy (q + h γc ) g(z )F (z 1 ) + ). z f(z 1 + z y a ) g(y) a dy + l g(z ) By symmetry, A (z 1, z ) = is also a strictly decreasing curve in the (z 1, z ) plane. This can e seen y oserving the validity of the discussion in the proof of lemma (i) when z 1 is fixed in D (n+1) (z 1, z ). The lower ound z for z z1 is given y the solution of lim z1 A (z 1, z ) =. Then, z g(y)dy = q +l c q +l +h γc < 1 and so z is finite. Similarly, the upper ound z is otained when z 1 = in A (z 1, z ) =. Also, z 1 is finite ecause dz < and z 1 is finite. Nash strategies of the retailers within the class of stationary ase stock strategies are characterized in theorem 1. Theorem 1 The infinite horizon sustitutale product inventory control prolem has a Nash equilirium characterized y stationary order-upto-levels, say S 1 and S, relative to the initial inventory levels I S 1 and J S of retailers I and II, respectively. This is the unique Nash equilirium within the class of stationary ase stock strategies. Proof: Suppose that (S1, S) is a solution of A 1 (z 1, z ) = and A (z 1, z ) = for (z 1, z ). Namely, S1 = z 1 S and S = z S 1. From lemma 4, given S as the order-upto-level of the second retailer s stationary ase stock strategy, D 1 (z 1, S) is a convex function that is minimized at z 1 S. Recall that z 1 S is the solution of A 1 (z 1, S) =. Hence, one condition of Nash equilirium, namely D 1 (S1, S) D 1 (z 1, S), for all z 1, is satisfied at S1 = z 1 S. Similarly, given S1 as the first retailer s order-upto-level, D (S1, z ) is convex and its minimizing point z S 1 is otained y solving A (S1, z ) =. Thus, the other Nash condition also holds, i.e., D (S1, S) D (S1, z ), for all z, S = z S 1. The next step is to address the question on the existence of a pair (S1, S) that satisfies oth A 1 (S1, S) = and A (S1, S) =. Recall from the proof of lemma (i) that the curve A 1 (z 1, z ) = is strictly decreasing, and for every z in [, ), z 1 z takes values etween z 1 and z 1 <. Similarly, given any z 1 [, ), the analysis of A (z 1, z ) = gives the lower and upper ounds z and z <, respectively, for z z1. For the existence and uniqueness of the Nash solution one needs to show that there exists only one point, (S1, S), at which oth A 1 (S1, S) = and A (S1, S) = hold. This is true only if the curve A 1 (z 1, z ) = is decreasing faster than A (z 1, z ) = in the (z 1, z ) plane. Compare dz1 and dz using the method in [9]. Let K = (q 1 + h 1 γc 1 )f(z 1 )G(z ) >, L = l 1 f(z 1 ) >, Z = l g(z ) >, then, dz1 and dz are written as dz1 M = (q 1 + h 1 γc 1 ) z 1 g(z + z1 x )f(x)dx >, R = (q + h γc ) z f(z 1 + z x a )g(y)dy >, T = (q + h γc )F (z 1 )g(z ) >, = 1 (K+L) M, dz = R T + R a +Z. The difference of the derivatives is dz dz1 = M(T + Z) + (K + L)(T + R a + Z) + ( 1 a )RM M(T + R a + Z), which is positive since every term oth in the numerator and the denominator are positive. Hence, there exists a unique intersection of the curves A 1 (z 1, z ) = and A (z 1, z ) = in (z 1, z ) plane. 1

13 Remark 3: Nash equilirium identified aove for the infinite horizon prolem is myopic ecause it is the Nash solution of the static (one-period) game with the following payoff functions of the retailers for every (z 1, z ) pair: c 1 z 1 + L 1 (z 1, z ) γc 1 c z + L (z 1, z ) γc z z+ z 1 x + z y a ( z1 x [y z ] +) g(y)f(x)dydx, (4) ( z y a[x z 1 ] +) f(x)g(y)dxdy. (5) The model developed in this section satisfies the conditions presented y Soel in [13] to guarantee the existence of myopic equilirium strategies in stochastic games with finite state and action spaces. Below, an explanation is given for the satisfaction of each condition: (i) The instantaneous payoff function is the summation of two terms, one term is a function of the actions taken and the other is a function of the current state as shown elow: r 1 (I,J)(z 1 I)(z J) = c 1 z 1 + L 1 (z 1, z )} c 1 I, r (I,J)(z 1 I)(z J) = c z + L (z 1, z )} c J; (ii) The transition proailities do not depend on the current state ut on the actions taken as shown in (??); (iii) From theorem 1, the static Nash non-cooperative game in (4), (5) has an equilirium; (iv) Under the equilirium strategies of the static game, all transitions occur etween the states in [, S 1] [, S ]. In other words, equilirium strategies of the static game are feasile for the states in [, S 1] [, S ]. An option for the retailers is to cooperate for the est total payoff for oth products and then share this amount. The analysis of this case is simply an extension of the discussion aout cooperation in [9] to the multi-period model as shown y the following remark: Remark 4: Cooperation always gives etter total payoff than the summation of the payoff amounts of the two retailers in the strict non-cooperative case. Below, four different cases are considered to compute total payoff when the retailers cooperate. Lost sale cost is not incurred if demand of one product is satisfied y the other product. Case 1: x n z 1n, y n z n payoff c = q 1 x n + q y n c 1 Q 1n c Q n h 1 I n 1 h J n 1, I n 1 = z 1n x n, J n 1 = z n y n. Case : x n z 1n, y n > z n payoff c = q 1 min z 1n, x n + (y n z n )} + q z n c 1 Q 1n c Q n h 1 I n 1 l ((1 )(y n z n )) l max, (y n z n ) (z 1n x n )}, I n 1 = max, (z 1n x n ) (y n z n )}, J n 1 =. 13

14 In this case, (1 )(y n z n ) is the demand for product lost ecause customers do not accept sustitution and max, (y n z n ) (z 1n x n )} is the sustitutale demand for product which is lost when there is not enough stock of product 1. Case 3: x n > z 1n, y n z n payoff c = q 1 z 1n + q min z n, y n + a(x n z 1n )} c 1 Q 1n c Q n h J n 1 l 1 ((1 a)(x n z 1n )) l 1 max, a(x n z 1n ) (z n y n )}, I n 1 =, J n 1 = max, (z n y n ) a(x n z 1n )}. Here, (1 a)(x n z 1n ) denotes the amount that can not e sustituted y product and a(x n z 1n ) (z n y n ) is the sustitutale amount which is lost if it is greater than zero. Case 4: x n > z 1n, y n > z n payoff c = q 1 z 1n + q z n c 1 Q 1n c Q n, I n 1 =, J n 1 =. Now, comparison of the non-cooperative case and cooperation of the retailers immediately leads to the oservation that payoff c payoff 1 + payoff for every possile x n, y n, z 1n, z n values. Note that payoff c = payoff 1 + payoff in cases 1 and 4. This means that one-period expected total payoff is greater than the summation of the expected payoffs of the retailers in the non-cooperative case, i.e., r c (I,J)Q 1Q r 1 (I,J)Q 1Q + r (I,J)Q 1Q for every (I, J), (Q 1, Q ), where r c is defined as the one-period expected payoff when there is cooperation. Also, in each of the four cases aove inventory levels at the eginning of the coming period are the same as in the non-cooperative case. Then, for any given initial inventory levels and any given strategy pair, the expected total discounted payoff in the case of cooperation is etter than the summation of the individual discounted payoff amounts of the retailers incurred in the non-cooperative case. In order to find an optimal joint strategy of the retailers when they cooperate, one needs to proceed with a single-retailer multi-product model. Finite-horizon dynamic programming formulation would then have the following form: C cn (I, J) = min (z1,z ) (I,J) r(i,j)(z c 1 I,z J) + γ C c(n 1) ([z 1 x [y z ] + ] +, [z y a[x z 1 ] + ] + ) g(y)f(x)dydx, where C cn (I, J) is defined as the minimum expected discounted total payoff for the remaining n periods until the end of the horizon. In order to investigate the structure of optimal ordering strategies for this formulation or when n goes to infinity, the analysis should e performed within the context of single-retailer multi-product dynamic inventory control. 4 Conclusion In this study, infinite horizon sustitutale product inventory prolem is formulated as a two-person nonzerosum discounted stochastic game and Nash ordering strategies of the retailers are investigated within the class of stationary ase stock strategies. It is assumed that the set-up costs are zero. The analysis is ased on minimizing the discounted payoff function of one retailer given that the other retailer is using a stationary 14

15 ase stock strategy. It is shown that his optimal strategy is also a stationary ase stock strategy. The existence of a unique Nash equilirium is proved within the class of stationary ase stock strategies. Also, cooperation of the retailers is oserved as a solution dominating the non-cooperative solution alternatives in the sense of giving etter expected total discounted payoff than the summation of the payoffs of the retailers in the non-cooperative case. The infinite horizon model presented in this article is an extension of the single-period prolem considered in [9]. Parlar conjectured the existence of (s, S)-type Nash strategies for multi-period prolem in the same article. The work here in this article proves the validity of this conjecture when there is no set-up cost. The analysis also shows that the stationary ase stock Nash strategies of the retailers are myopic in accordance with the results otained in [6] for a class of dynamic oligopolies and the generalization of these results in [13]. Relaxation of the constraints under which the sustitutale product inventory prolem is analyzed in this article underlines future research directions as itemized elow: The ordering cost is a linear function of the quantity ordered. Analyzing the prolem when the set-up costs are nonzero and investigating the validity of Parlar s conjecture on the existence of (s, S)-type Nash strategies remain as further research sujects. The Nash equilirium identified in this article can e attained if oth retailers restrict themselves to stationary ase stock strategies. Analysis of the sustitutale product inventory prolem over a larger strategy space would address the question on the existence of other Nash strategies of different types. The discount factor, demand distriutions and the sustitution proailities are considered stationary. However, there may e cases where those are non-stationary, e.g., the sustitution proailities might change over time as a function of the actions taken y the retailers. Consideration of the prolem under such non-stationary conditions would also lead to the investigation of the prolem over larger strategy spaces. A natural extension would e the analysis of the prolem under the average expected payoff criterion. In proceeding along any further research direction, cooperation of the retailers would turn out as an implementale option to e studied as compared to the non-cooperative case. References [1] R. Bellman, I. Glickserg, and O. Gross. On the Optimal Inventory Equation. Management Science, :83 14, [] J.A. Filar, T.A. Schultz, F. Thuijsman, and D.J. Vrieze. Nonlinear Programming and Stationary Equiliria in Stochastic Games. Mathematical Programming, 5:7 38, [3] D.L. Iglehart. The Optimality of (s,s) Policies in the Infinite Horizon Dynamic Inventory Prolem. Management Science, 9:59 67, [4] E. Ignall and A.F. Veinott. Optimality of Myopic Inventory Policies for Several Sustitute Products. Management Science, 15:84 34, [5] M. Khouja, A. Mehrez, and G. Rainowitz. A Two-Item Newsoy Prolem with Sustitutaility. International Journal of Production Economics, 44:67 75, [6] A.P. Kirman and M.J. Soel. Dynamic Oligopoly with Inventories. Econometrica, 4:79 87,

16 [7] A.R. McGillivray and E.A. Silver. Some Concepts for Inventory Control under Sustitutale Demand. Infor, 16:47 63, [8] M. Parlar. Optimal Ordering Policies for A Perishale and Sustitutale Product: A Markov Decision Model. Infor, 3:18 195, [9] M. Parlar. Game Theoretic Analysis of the Sustitutale Product Inventory Prolem with Random Demands. Naval Research Logistics, 35:397 49, [1] M. Parlar and S.K. Goyal. Optimal Ordering Decisions for Two Sustitutale Products with Stochastic Demands. Opsearch, 1:1 15, [11] T.E.S. Raghavan and J.A. Filar. Algorithms for Stochastic Games-A Survey. ZOR-Methods and Models of Operations Research, 35:437 47, [1] H. Scarf. The Optimality of (s,s) policies in the Dynamic Inventory Prolem. Mathematical Methods in the Social Sciences, pages 196, 196. [13] M.J. Soel. Myopic Solutions of Markov Decision Processes and Stochastic Games. Operations Research, 9:995 19, [14] A.F. Veinott. Optimality Policy for Multi-Product, Dynamic, Nonstationary Inventory Prolem. Management Science, 1:6, [15] Q. Wang and M. Parlar. A Three-Person Game Theory Model arising in Stochastic Inventory Control Theory. European Journal of Operations Research, 76:83 97,