Awell-known result in the Bayesian inventory management literature is: If lost sales are not observed, the

Transcription

1 MANUFACTURING & SERVICE OPERATIONS MANAGEMENT Vol. 10, No. 2, Sprng 2008, pp ssn essn nforms do /msom INFORMS Dynamc Inventory Management wth Learnng About the Demand Dstrbuton and Substtuton Probablty L Chen TrueDemand, Inc., Los Gatos, Calforna 95032, l@tdemand.com Erca L. Plambeck Graduate School of Busness, Stanford Unversty, Stanford, Calforna 94305, elp@stanford.edu Awell-known result n the Bayesan nventory management lterature s: If lost sales are not observed, the Bayesan optmal nventory level s larger than the myopc nventory level one should stock more to learn about the demand dstrbuton. Ths result has been proven n other studes under the assumpton that nventory s pershable, so the myopc nventory level s equal to the Bayesan optmal nventory level wth observed lost sales. We break that equvalence by consderng nonpershable nventory. We prove that wth nonpershable nventory, the famous stock more result s often reversed to stock less, n that the Bayesan optmal nventory level wth unobserved lost sales s lower than the myopc nventory level. We also prove that makng lost sales unobservable ncreases the Bayesan optmal nventory level; n ths specfc sense, the famous stock more result of other studes generalzes to the case of nonpershable nventory. When the product s out of stock, a customer may accept a substtute or choose not to purchase. We ncorporate learnng about the probablty of substtuton. Ths reduces the Bayesan optmal nventory level n the case that lost sales are observed. Reducng the nventory level has two benefcal effects: to observe and learn more about customer substtuton behavor and for a nonpershable product to reduce the probablty of overstockng n subsequent perods. Fnally, for a capactated producton-nventory system under contnuous revew, we derve maxmum lkelhood estmators MLEs of the demand rate and probablty that customers wll wat for the product. Acceptng a rancheck for delvery at some later tme s a common type of substtuton. We nvestgate how the choce of base-stock level and producton rate affect the convergence rate of these MLEs. The results renforce those for the Bayesan, uncapactated, perodc revew system. Key words: Bayesan nventory management; unknown demand dstrbuton; unobserved lost sales; substtuton probablty; Bayesan dynamc programmng; optmal nventory control; maxmum lkelhood estmator; make-to-stock queue Hstory: Receved: May 13, 2005; accepted: January 3, Publshed onlne n Artcles n Advance December 17, Introducton Imagne a retaler purchasng nventory for an nnovatve product. If the retaler runs out of stock, customers may accept a substtute or choose not to purchase. The optmal nventory level for the product depends on both the demand dstrbuton and substtuton probablty. Furthermore, the choce of nventory level affects the sales data and customer behavor observed by the retaler, and thus shapes the retaler s opportunty for learnng about the demand dstrbuton and substtuton probablty. Ths paper provdes nsghts and gudelnes for dynamcally managng the nventory of an nnovatve product whle learnng about the demand dstrbuton and substtuton probablty. The early lterature on Bayesan nventory management for a sngle tem wth unknown demand dstrbuton assumes that the tem s nonpershable and customer demand s observed perfectly Scarf 1959, Azoury 1985, Lovejoy These papers focus on methods to reduce the computatonal complexty of the statstcal nventory control problem by explorng the conjugate pror dstrbuton structure and the state-space reducton technque. A more recent stream 236

2 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 237 of research on ths topc has put more emphass on the dervaton of qualtatve nsghts about how the presence of Bayesan learnng affects the optmal nventory decson. Assumng the tem s pershable, Harpaz et al recognze that when lost sales are not observed, one should ntally ncrease the nventory level to learn more about the demand dstrbuton. Further assumng that demand has an exponental dstrbuton wth gamma pror on the mean, Larvere and Porteus 1999 derve a closedform expresson for the Bayesan optmal nventory level and confrm that t exceeds the Bayesan myopc nventory level n every perod. Dng et al and Lu et al. 2005, 2006 extend ths stock more result to pershable nventory systems wth a general contnuous demand dstrbuton. Bensoussan et al study a smlar problem wth Markovan demand. Followng ths lterature, we say that the Bayesan myopc nventory polcy maxmzes the currentperod expected proft, pretendng, n the case of nonpershable nventory, that the unsold unt can be returned at the dscounted orgnal cost. The expectaton s taken wth respect to updated pror dstrbuton that ncorporates all past demand observatons. Usng ths polcy, the myopc decson maker fals to account for the potental beneft of gatherng nformaton now to mprove future-perod profts and also fals to account for nventory nteracton between perods. However, the myopc decson maker does update her choce of nventory level n every perod based on past demand observatons. We wll subsequently use the terms myopc nventory level and Bayesan myopc nventory level nterchangeably. The myopc nventory polcy has two propertes that are mportant for our analyss. Frst, n the case of pershable nventory, assumng that the decson maker observes lost sales, the myopc nventory polcy s dentcal to the Bayesan optmal nventory polcy Larvere and Porteus 1999, Dng et al Second, n managng nonpershable nventory wth an nfnte horzon, the optmal nventory polcy wthout Bayesan updatng s a statonary polcy wth nventory level equal to the ntal myopc nventory level Heyman and Sobel 1984, p. 66, Proposton 3-1. Therefore, assumng the plannng horzon s suffcently long for the nonpershable nventory case, all statements below regardng the myopc nventory level n the ntal perod are also true when one substtutes the optmal nventory level wthout Bayesan learnng. Specfcally, when the myopc nventory level s lower hgher than the Bayesan optmal nventory level, one should stock more stock less to account for the opportunty to learn about the demand dstrbuton and substtuton probablty. We extend the Bayesan nventory management lterature n three drectons. The frst s to work wth general dscrete demand dstrbutons. Most of the lterature on Bayesan nventory management assumes a contnuous demand dstrbuton. We fnd the assumpton of dscrete demand smplfes many of our proofs. Second, we relax the pershablty assumpton, whch destroys the equvalence between the myopc nventory level and the Bayesan optmal nventory level wth observed lost sales. We show that the stock more result n Harpaz et al. 1982, Larvere and Porteus 1999, and Dng et al s often reversed to stock less when the Bayesan optmal nventory level wth unobserved lost sales s compared wth the myopc nventory level. That s, wth unobserved lost sales and nonpershable nventory, one should often stock less than the myopc nventory level to reduce the rsk of overstockng n the subsequent perods. However, when the comparson s made between the Bayesan optmal nventory level wth unobserved lost sales and the Bayesan optmal nventory level wth observed lost sales, we show that the stock more result n Harpaz et al. 1982, Larvere and Porteus 1999, and Dng et al generalzes to the case of nonpershable nventory. That s, makng lost sales unobservable ncreases the Bayesan optmal nventory level, regardless of whether nventory s pershable or nonpershable. The latter nsght s useful for computng effectve nventory levels n practce. Exact computaton of the Bayesan optmal nventory level wth unobserved lost sales s tractable only wthn a lmted newsvendor dstrbuton famly Larvere and Porteus In contrast, wth observed lost sales, the Bayesan optmal nventory level s relatvely easy to compute usng the results n Azoury 1985 and Lovejoy 1990, whch apply for a broad class of demand dstrbutons. Therefore, our result that observng lost sales reduces the Bayesan optmal nventory level provdes

3 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 238 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS a useful lower bound for heurstc management of nonpershable nventory wth unobserved lost sales. The thrd drecton n whch we extend the Bayesan nventory management lterature s to consder the effect of learnng about the probablty that a customer wll accept a substtute product. Intutvely, one learns more about the substtuton probablty by lowerng the nventory level and thus forcng more customers to consder the substtute product. We show that ths ntutve nsght s vald when lost sales are observed, but s not generally true when lost sales are unobserved. In 2, we follow Nahmas and Smth 1994 and model customers decsons to accept a substtute when the product s out of stock by a seres of ndependent and dentcally dstrbuted..d. Bernoull trals. Increasng the substtuton probablty reduces the expected underage cost and thus reduces the optmal nventory level. Assumng that the substtuton probablty and unknown parameter of the demand dstrbuton are subject to a jont pror belef, we make comparsons among the Bayesan optmal nventory level wth observed lost sales and wth unobserved lost sales and the myopc nventory level. The results are summarzed n Table 1. For analytc tractablty, we assume that the substtute product s always avalable and the substtuton decson s observed. Many researchers have addressed jont nventory management for multple products under customer substtuton wthout the added complexty of Bayesan learnng. Ths work can be categorzed nto papers that assume centralzed control Parlar and Goyal 1984, Ernst and Kouvels Table 1 Comparsons Between Bayesan Optmal Inventory Levels wth Unobserved and Observed Lost Sales and the Myopc Inventory Level Pershable nventory Nonpershable nventory Estmate demand y O = y M, y U y O, y O y M, y U y O, parameter only y U y M. y U y M. Estmate demand y O y M, y U y O, y O y M, y U y O, parameter and y U y M. y U y M. substtuton probablty p Notes. y O denotes the Bayesan optmal nventory level wth observed lost sales, y U the Bayesan optmal nventory level wth unobserved lost sales, and y M the myopc nventory level. ndcates that the relatonshp could be ether greater than or less than dependng on the cost structure and relatve uncertantes. 1999, Noonan 1995, Agrawal and Smth 1998, Smth and Agrawal 2000, Rajaram and Tang 2001, Mahajan and van Ryzn 2001a and papers wth nventory competton between multple retalers Parlar 1988, Wang and Parlar 1994, Lppman and McCardle 1997, Mahajan and van Ryzn 2001b, Netessne and Rud Mahajan and van Ryzn 1999 provde a more detaled descrpton of the lterature on nventory management under customer substtuton. For complex systems wth multple products or capacty constrants, the Bayesan approach becomes ntractable and maxmum lkelhood estmators MLEs are commonly employed. For example, for a multproduct nventory system wth a Posson demand process and perodc observaton of nventory levels, Anupnd et al apply the expectatonmaxmzaton EM algorthm to compute MLEs for the demand rate for each product and substtuton probabltes. For a sngle tem wth capactated producton and two dstrbuton channels, Armony and Plambeck 2005 nvestgate the systematc errors n the MLEs for demand rate, and the renegng rate and the consequent errors n capacty nvestment caused by unsuspected duplcate orderng. In 3, we consder a capactated producton-nventory system M/M/1 make-to-stock queue wth balkng under contnuous revew. We compute MLEs for the demand rate and probablty p that when the product s out of stock, a customer wll choose to wat for the product rather than balk. Acceptng a rancheck for delvery of the product at some later tme s a common type of substtuton. For any fnte tme horzon, when lost sales customers that balk are not observed, the accuracy of the MLE for s monotoncally ncreasng n the base-stock nventory level and producton rate, but the accuracy of p s not. Wth observed lost sales, the accuracy of the MLE for does not depend on the base-stock level, and the accuracy of the MLE for p s monotoncally decreasng n the base-stock level and producton rate. It follows that wth observed lost sales, one should ntally reduce the nventory level or producton rate to mprove the accuracy of the MLE for p, whch, n turn, wll mprove future decson makng. These results for the capactated producton-nventory system renforce the results for Bayesan nventory management summarzed n Table 1.

4 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS Bayesan Inventory Management Ths secton consders perodc revew Bayesan nventory management for a product wth unknown demand dstrbuton and unknown stockout-based substtuton probablty. Wthout mposng any dstrbutonal assumptons, we compare the Bayesan optmal nventory level wth observed lost sales, the Bayesan optmal nventory level wth unobserved lost sales, and the myopc nventory level. We frst study the relatvely smple case wth learnng only about the demand dstrbuton, then study the case wth learnng about the demand dstrbuton and substtuton probablty Model The product wll be stocked and sold for N perods. At the begnnng of each perod = 1N, the nventory manager selects an nventory level for the product. The nventory level s acheved mmedately after the decson we are assumng a neglgble delvery lead tme from the suppler. For each unt of the product, the producton cost s c and the sellng prce s r, wth r>c>0. At the end of each perod, a unt holdng cost h s charged to any leftover stock. As a base case, we assume that the nventory s nonpershable and can be used to satsfy demand n subsequent perods. At the end of the sellng season,.e., n perod N + 1, we assume that any unsold nventory wll be salvaged at a unt value of c s, wth c s c. If the demand durng a perod exceeds the nventory level, the manager s charged a stockout penalty q per unt of shortage. In addton, each customer who arrves when the product s out of stock s offered a substtute product. If a customer accepts the substtute, then the manufacturer receves a contrbuton margn of m per unt from sellng the substtute. To avod trvalty, we assume that r c m, whch guarantees that sellng the product from nventory makes economc sense. Acceptng a ran check for delvery at the end of the perod s a common type of substtuton. In ths case, the margn m s the product sellng prce r less the producton cost c and any expedtng/handlng cost. The objectve of the nventory manager s to maxmze total dscounted expected proft. We denote the dscount factor by 0 <1. The demand n each perod s ndependently and dentcally generated by a general nonnegatve dscrete demand dstrbuton. The probablty mass functon of the demand dstrbuton s denoted by f, = where s an unknown parameter or vector of unknown parameters wth. In a perod when stockout occurs, each unt of excess demand generates an ndependent Bernoull tral: Each customer s wllng to accept the substtute wth probablty p and becomes a lost sale wth probablty 1 p. Let z denote the startng nventory, y the chosen nventory level, and the random total quantty sold, ncludng sales of the substtute product caused by stockout x wll be used to denote the realzaton of total quantty sold. For the moment, suppose that the manager observes all customer demand ncludng any lost sales. In other words, for each perod, the manager observes not only the total sales x, but also the demand realzaton. Gven values of and p, and an nventory level of y, the lkelhood of observng demand realzaton and sales x n a perod s f f = x<y y p x y 1 p x f f y x p = x y 1 f x y 0 otherwse. Analogously, f lost sales are not observed.e., the manager only observes sales of the regular product and, n the event of stockout, the number of customers that accept the substtute, the lkelhood of observng total sold quantty x can be wrtten as f y x p fx = y p x y 1 p x f x y =x When p = 0, 2 becomes fx f y x = f =y f x<y f x = y 0 otherwse, f x<y f x y 2

5 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 240 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS whch s the dscrete demand verson of the lostsales case wthout substtuton studed n Harpaz et al. 1982, Larvere and Porteus 1999, and Dng et al On the other hand, f p = 1, we have f y x p = fx for all x 0, whch means that the quantty sold n a perod s an exact observaton of the demand. Lkelhood functons 1 and 2 wll be used to update the posteror dstrbuton n the observed and unobserved lost-sales cases, respectvely. The expected proft for a sngle perod as a functon of the demand parameter and substtuton probablty p s by straghtforward algebra remny + me y + cy z hey + qe y + = cz h + cy + r + h r + h + q mp yf where s the mean demand gven densty f. Let v z p denote the maxmum total dscounted expected proft over perods N wth z unts of on-hand nventory at the begnnng of perod as a functon of the demand parameter and substtuton probablty p. We assume that the on-hand nventory s zero at the begnnng of Perod 1. Usng a well-known smplfyng technque see Heyman and Sobel 1984, p. 79, we can wrte the optmalty equatons as v 1 0 p = max Ry p + v 2 y + pf y 0 =0 v z p = max Ry p + v +1 y + pf y z where =0 =y for = 2N 1 v N z p = max y z R N y p 3 Ry p = h + cy + r + h r + h + q mp yf =y R N y p = h N + cy + r + h N r + h N + q mp yf wth h = h c and h N = h c s. Now assume that, startng n Perod 1, and p are not known but are subject to certan ndependent pror dstrbutons,.e., the jont pror dstrbuton 1 p = 1 1 p, wth 1 and 1 p beng the margnal densty for and p, respectvely. Gven a jont pror dstrbuton p for perod, the posteror dstrbuton calculated from the demand nformaton observed n perod s used as the pror dstrbuton +1 p for the subsequent perod + 1. Hence, gven a specfc Bayesan pror updatng scheme, the optmalty equatons for the Bayesan nventory management can be expressed as v = max E y 0 1 p Ry p+ v z = max E y z p + =0 Ry p =0 =y v 2 y + 2 f v +1 y + +1 f = 2N 1 v N z N = max y z E N pr N y p 4 From the optmalty equatons 4, t s also useful to denote the objectve functon for each perod as for = 1N 1, G y = E p Ry p + v +1 y + +1 f and =0 G N y N = E N pr N y p For each perod, the maxmum s attaned at some fnte nteger values of y because the expected proft goes to as y goes to. We denote the Bayesan optmal nventory level wth unobserved lost sales as y U, the Bayesan optmal nventory level wth observed lost sales as y O, and the myopc nventory

6 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 241 level as y M. We desgnate the same superscrpts U, O, and M to the value functons v and objectve functons G for these three cases, respectvely. Below are detaled descrptons of the Bayesan updatng schemes wth observed and unobserved lost sales Observed Lost-Sales Case. Let us denote the margnal dstrbuton of the jont pror dstrbuton p for perod by and p for and p, respectvely. Gven observatons of x and nventory level y for perod, the posteror dstrbuton densty +1 p xy can be updated usng the lkelhood functon 1 accordng to Bayes rule: +1 pxy f p f =x y f d = y x y y y x y p x y 1 p x f p p x y 1 p x f p d dp f >x=y x y p x y 1 p x f1p >0 p p x y 1 p x f p d dp y x y f x>y Because 1 p = 1 1p, substtutng ths relaton nto 5, we have 2 p xy p f = x y = p yx y 1 p f x y where 2 1 = f 1 f 1 d and 2 p yx y 1 p y x y p x y 1 p x 1 p 1 y f x =y p 0 x y x y 1 p x 1 p dp = y x y p x y 1 p x 1p >0 1 p 1 y f x>y 0 + p x y x y 1 p x 1 p dp 5 Hence, the updated pror dstrbutons of and p for Perod 2 are also ndependent. By nducton, the updated pror dstrbutons of and p of all subsequent perods are ndependent. The Bayesan updatng of and p can thus be performed separately based on ther respectve margnal dstrbutons. Ths clean updatng process s a result of the observablty of lost-sales nformaton: Full nformaton s avalable for each parameter-updatng scheme and thus prevents them from tanglng up. For ease of notaton, n the rest of the paper ether +1 p yx y or +1 p wll be shorthand for +1 p yx y p whenever the meanng s clear from the context Unobserved Lost-Sales Case. Now let us consder the unobserved lost-sales case. For perod, let p be the jont pror dstrbuton densty of and p and y be the nventory level selected. If total sales x s observed n perod, the posteror dstrbuton densty of and p s gven by the followng equaton, accordng to Bayes rule and lkelhood functon 2: +1 p xy p f y x p p 1 0 f y x f x y p p d dp = f y x p1p > 0 p 1 f y 0 + x f x>y p p d dp For ease of notaton, n the rest of the paper we wrte +1 p xy p wth the shorthand expresson +1 p xy or +1 p x n the case of x<y because the posteror s not affected by y whenever the meanng s clear from the context. As we can see from lkelhood functon 2 and updatng scheme 6, unlke the observed lost-sales case, the posterors for and p are mxed up once x y s observed Updatng Demand Parameter Only In ths secton, we consder the case that only demand parameter s updated and the substtuton probablty p s assumed known. When lost sales are not observed, the general consensus n the lterature s that one should stock more than the myopc nventory level to ncrease the probablty of observng an 6

7 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 242 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS exact demand realzaton Harpaz et al. 1982, Larvere and Porteus 1999, Dng et al However, ths nsght s derved based on the assumpton that nventory s pershable cannot be carred over to the next perod. We relax the pershablty assumpton n ths secton. We frst show that ths stock more result remans vald when the Bayesan optmal nventory level wth unobserved lost sales s compared to that wth observed lost sales. However, when the Bayesan optmal nventory level wth unobserved lost sales s compared to the myopc nventory level, we show examples n whch the opposte result stock less holds. In these examples, one should reduce the nventory level to mtgate the rsk of overstockng n the subsequent perods. Defne gx = gx + 1 gx for any functon gx. It s easy to verfy that for the case wth observed lost sales, we have, for = 1N 1, G O y = E Ry p + y v O +1 y x +1 xf x p 7 x=0 It s straghtforward to verfy through backward nducton that G O y are concave n y see Scarf 1959 for the contnuous demand dstrbuton case. For the case wth unobserved lost sales, we have, for = 1N 1, G U y = E Ry p + y x=0 v U +1 y x +1 xf x p + v U +1 0 y+1 +1 xy + 1f x p x=y v U x yf y x p 8 x=y The concavty of G U y s dffcult to establsh for general dstrbutons. However, we can establsh the followng result: Proposton 1. For = 2N, y 0, and any, the followng holds: E v U 0 y+1 xy + 1f x p x=y E v U 0 x yf y x p 9 x=y n other words, the expected future value functon s ncreasng n the current-perod nventory level y expectaton taken when demand exceeds y. The proof of Proposton 1 s n the appendx. Proposton 1 shows that f p s known, then, condtonal on the event that demand exceeds the currentperod nventory level y, stockng an addtonal unt above y wll always ncrease the dscounted expected proft n future perods because the manager wll observe one more unt of customer demand. If the product were pershable, the second terms of 7 and 8 would be zero. Hence, by Proposton 1, the stock more result would follow. The case wth nonpershable nventory s complcated by these addtonal two terms. Nevertheless, we have the followng unequvocal result. Theorem 1. Suppose that the substtuton probablty p s known and lost sales are not observed. Regardless of whether the nventory s pershable or nonpershable, gven the same pror, the Bayesan optmal nventory level wth unobserved lost sales s greater than the Bayesan optmal nventory level wth observed lost sales,.e., y U y O. Proof. We frst consder the case wth nonpershable nventory. To prove y U y O, t suffces to show that G U y GO y, for all y 1. Wth Proposton 1, t thus suffces to show that v+1 U y x +1 x vo +1 y x +1 x. We show t by backward nducton. It s easy to verfy that gven the same pror dstrbuton N, these above clams hold. Now, assume that these are true for case + 1;.e., gven any pror +1, we have yu +1 yo +1 and G U +1 y GO +1 y for all y 1. A key observaton s that gven the same pror at the begnnng of perod, the updated posteror +1 x after observng sales x wth x<y for both the observed and unobserved lost-sales cases remans

8 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 243 dentcal. Hence, we can successfully apply the nducton assumpton of perod + 1. Specfcally, we consder three cases: Case 1. y+1 U yo +1 >z, we have vu +1 z +1 = v+1 O z +1 = 0. Case 2. y+1 U z>yo +1, we have vu +1 z +1 = 0 v+1 O z +1. Case 3. z>y+1 U yo +1. Notce that snce yo +1 z, by concavty of G O +1, we have GO +1 z 0 for all z z. Hence, the optmal decson n the observed lost-sales case s to place zero order. As a result, v O +1 z +1 = GO +1 z For the unobserved lost-sales case, however, G U +1 s not necessarly concave. Therefore, there exst two scenaros when startng nventory n perod + 1sz: a do not order, or b place an order to an nventory level greater than z. In scenaro a, we have v U +1 z +1 = vu +1 z+1 +1 vu +1 z +1 G U +1 z+1 +1 GU +1 z +1 = G U +1 z +1 where the nequalty follows from the fact that v+1 U z GU +1 z and v+1 U z +1 = GU +1 z +1. By the nducton assumpton: G U +1 y GO +1y for all y 1, and the observaton that v+1 O z = GO +1z for z>y+1 O, we conclude that vu +1 z +1 v+1 O z +1. In scenaro b, we have v+1 U z +1 = 0, whch, agan, means v+1 U z +1 vo +1 z +1 because the latter s nonpostve as shown n 10. Therefore, we have shown that v+1 U z +1 v+1 O z +1 for all z 0. Wth Proposton 1, we conclude G U y GO y, for all y 1, and by concavty of G O y, we obtan yu y O. Ths completes the nducton proof. For the case wth pershable nventory, we frst replace h and h N n 3 wth h. We know that G O y s concave n y, and G O y = E Ry p By Proposton 1 and the pershable nventory assumpton, we mmedately have G U y G O y, for all y 1, and hence, yu y O. Theorem 1 generalzes the observaton n Harpaz et al. 1982, Larvere and Porteus 1999, and Dng et al that makng lost sales unobservable ncreases the Bayesan optmal nventory level to allow for nonpershable nventory and partal lost sales 0 p 1. In general, lack of lost-sales nformaton nduces a Bayesan nventory manager to ncrease the nventory level to observe more exact demand. Our dervaton s based on a general dscrete demand dstrbuton. An analogous result holds for general contnuous demand dstrbutons, but the analogous proof requres more notaton and s more complex. Theorem 1 provdes a relatvely easy-to-compute lower bound on the Bayesan optmal nventory level for systems wth unobserved lost sales and nonpershable nventory settng whch s only tractable wthn a lmted newsvendor dstrbuton famly; see Larvere and Porteus Ideally, one can leverage the Bayesan optmal nventory level wth observed lost sales whch s tractable for much broader dstrbuton famles; see Azoury 1985 and Lovejoy 1990 to obtan approxmate solutons to the much harder unobserved lost-sales case. Desgnng effcent computatonal methods to determne the Bayesan optmal nventory level s beyond the scope of ths paper, but wll be a natural extenson followng the lne of ths research work. Now let us compare the Bayesan optmal nventory level wth the myopc nventory level. By the defnton of myopc nventory level gven n 1, we mmedately have, for = 1N 1, G M y = E Ry p 11 When the nventory s pershable, we know that the Bayesan optmal nventory level wth observed lost sales s equvalent to the myopc nventory level. Hence, by Theorem 1, the Bayesan optmal nventory level wth unobserved lost sales s greater than the myopc nventory level. When the nventory s nonpershable, comparng 7 wth 11 and observng that v+1 O z +1 x 0 for all z 0, we fnd that the Bayesan optmal nventory level wth observed lost sales s less than the myopc nventory level,.e., y O y M for all. If we compare 8 wth 11, on the one hand, as shown n Proposton 1, the nventory manager

9 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 244 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS wants to stock more than the myopc nventory level to observe exact demand f she knows the demand s gong to be hgh. On the other hand, n case the realzed demand s low, the nventory manager may be left wth excess nventory after updatng wth the low-demand observaton. In the latter case, the manager would prefer to stock less than the myopc nventory level because y x=0 vu +1y x +1 xf x p, the second term of 8 s negatve due to the fact that v+1 U z +1 x 0 for all z 0. There s a trade-off between stockng more to observe more demand nformaton and stockng less to avod excess nventory when the product s nonpershable and the Bayesan optmal nventory level wth unobserved lost sales s compared wth the myopc nventory level. The next result provdes gudance about the condtons under whch one should stock more than the myopc nventory level to account for the opportunty to learn about the demand dstrbuton. Proposton 2. Suppose that the substtuton probablty p s known and lost sales are not observed. If there exsts a perod j j <N n whch for a gven pror the Bayesan optmal nventory level wth unobserved lost sales s greater than the myopc nventory level,.e., y U j y M j, then for any perod <j, gven the same startng pror, y U y M. Proof. Based on the myopc polcy defnton, t s straghtforward that gven the same pror, y M = yj M for any <j. Thus, for any y yj 1 M = ym j yj U, from 8 we have G U j 1 y 0, whch means the optmal nventory yj 1 U n perod j 1 must be greater than yj 1 M. By nducton, ths result holds for any perod <j. Proposton 2 tells us that ncreasng the plannng horzon tends to make the ntal Bayesan optmal nventory level wth unobserved lost sales rse above the myopc nventory level. However, wth shorter horzons such as N = 2 perods, we commonly observe the stock less result Numercal Examples. Suppose that we have N = 2 perods to sell a nonpershable product. For smplcty, we assume that p = 0,.e., all unmet demand s lost, and that the demand n each perod follows an ndependent geometrc dstrbuton wth an unknown parameter, subject to a beta pror wth parameters and. It s straghtforward to show that the predctve demand dstrbuton for the frst perod s f 1 x = ++x/++x +1, where s the gamma functon. If >1, whch we assume henceforth, then the mean of the predctve demand dstrbuton equals / 1 see Johnson et al. 1992, p Furthermore, f the demand realzed n the frst perod s below the nventory level,.e., x<y, the posteror for unknown parameter s beta dstrbuton wth parameters = + 1 and = +x; otherwse, f demand s censored at y, the posteror s gamma wth parameters = and = + y. To llustrate the effect of the pershable nventory assumpton on Bayesan nventory decsons, we assume that nventory s nonpershable but can be salvaged at the end of frst perod for cs 1 c1 s c per unt and at the end of second perod for c s = c per unt. For cs 1 = c, ths s equvalent to the pershable nventory case. When cs 1 c s h, one would never salvage nventory at the end of Perod 1, so ths s equvalent to the nonpershable nventory case wth no salvagng opton. Gven the salvagng opton at the end of the frst perod, the optmal nventory polcy for Perod 2 whch s also the last perod s an L U-type,.e., the manager would order up to L f z 2 <L z 2 s the on-hand nventory at the begnnng of Perod 2; order nothng f L z 2 U ; and salvage nventory down to U f z 2 >U. The optmal Perod 1 nventory level for ether the Bayesan or myopc case cannot be expressed n closed form and s not necessarly equal to the orderup-to level L of Perod 2, but t can be easly determned numercally n ths two-perod example. In partcular, we assume that sellng prce r = 10, purchase cost c = c s = 2, holdng cost h = 1, shortage cost q = 8, and dscount factor = 1. The salvage value cs 1 s vared between one and two, wth cs 1 = 1 equvalent to the nonpershable case and cs 1 = 2 equvalent to the pershable case. Fgure 1 shows the Bayesan optmal nventory level wth observed and unobserved lost sales and myopc nventory level for Perod 1 wth a beta pror = 2, = 4. The coeffcent of varaton of ths beta pror s / = 05345, mplyng a relatvely hgh degree of uncertanty about the parameter. The resultng predctve demand mean for

10 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 245 Fgure 1 Inventory level Impact of Frst-Perod Salvage Value on the Optmal Inventory-Level Decsons Optmal nventory level for perod 1 α = 2, β = 4 y M myopc 9 y U unobserved lost sales y O observed lost sales Frst-perod salvage value c1 s Perod 1 s / 1 = 4. Fgure 1 llustrates the analytc result n Theorem 1: The Bayesan optmal nventory level wth unobserved lost sales s weakly greater than that wth observed lost-sales case. Fgure 1 also shows that the Bayesan optmal nventory level wth unobserved lost sales y U s less than the myopc level y M n most of the parameter regon. Specfcally, ths stock less result holds for all salvage values cs and s reversed to stock more only when cs 1 > 19 whch essentally corresponds to the case of pershable nventory. Ths llustrates that the penalty of overstockng even wth the frst-perod salvage opton tends to domnate the nformatonal beneft of observng exact demand by ncreasng nventory. Ths stock less result also holds for the exponental demand dstrbuton. Followng Larvere and Porteus 1999, we focus on the exponental dstrbuton to obtan a closed-form expresson for the Bayesan optmal nventory level wth unobserved lost sales. Proposton 3 Chen Suppose that the demand dstrbuton s exponental and the product s nonpershable wth the close-out salvage value equal to the orgnal cost c s = c. For a two-perod problem wth censored demand observaton, the Bayesan optmal nventory level wth unobserved lost sales s less than the myopc nventory level,.e., y1 U <ym 1. Derved n a two-perod settng, ths stock less result s the opposte of the stock more obtaned n Larvere and Porteus 1999 for the same model but wth pershable nventory. The proof s n the frst author s unpublshed PhD thess Chen In concluson, the stock less result s remarkably robust n our examples. Often, the Bayesan nventory manager wll have less nventory than the myopc level because the potental cost of overstockng n subsequent perods because demand s lower than ntally antcpated outweghs the potental nformatonal gans from ncreasng nventory to observe a more exact realzaton of demand Updatng Both Demand Parameter and Substtuton Probablty p In ths secton, we study the case n whch both demand parameter and substtuton probablty p are updated accordng to the Bayes rule. To dfferentate ths case from the cases studed n the prevous secton, we use the superscrpt and U 2 to ndcate the cases of updatng both and p wth observed and unobserved lost sales, respectvely. Because the myopc nventory level n ths case s equvalent to the prevous secton f the known p value s replaced by the pror mean of p n 11, we keep usng the superscrpt M to ndcate the myopc case n ths secton. We frst assume that the manager can observe any lost sales. Ths assumpton s vald, for nstance, when customer orders arrve through a call center or over the Internet and detaled order data s recorded. We have, for = 1N 1, G =E p y Ry p + y =0 v +1 y +1 pf + EK y 1 v =y+1 +1p y 1K E K y v p yk f 12 where E K n denotes expectaton taken over random varable K wth bnomal dstrbuton n p.

11 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 246 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS Proposton 4. Suppose that lost sales are observed. The ongong expected value functon for the next perod ncreases wth the number of substtuton trals observed n the current perod. For any perod 2N, number of trals n 0 and pror 1 p: E 1 pe K n+1 v z p n + 1K 1 E 1 pe K n v z p n K 1 Ths result s derved based on the fact that when lost sales are observed, updatng and p can be separated. The ntuton behnd t s that the greater the number of substtuton trals, the greater the expected nformatonal benefts from Bayesan updatng. The proof together wth two auxlary lemmas, s gven n the appendx. Applyng Proposton 4 to 12, we mmedately see that the last term s negatve, whch means one s nduced to stock less nventory to ncrease the number of customer substtuton trals. Because both G M y and G O y are concave n y, the optmal nventory levels y M and y O can be unquely determned by the frst-order dfference condtons. In contrast, G y s not necessarly concave n y. Nevertheless, we have the followng unequvocal result. Theorem 2. Suppose that lost sales are observed. Regardless of whether the product s pershable or nonpershable, for any 1N, gven the same pror, the Bayesan optmal nventory level wth updatng of both demand parameter and substtuton probablty p s less than the Bayesan optmal nventory level wth updatng only of demand parameter,.e., y y O. Proof. We frst prove for the case of nonpershable nventory. We need to show G y G O y, for all y 1. Wth Proposton 4, t suffces to show that v+1 z +1 p vo +1 z +1 p for all z 0. We show ths by backward nducton. It s easy to verfy that, gven the same pror dstrbuton N p, the result holds. Now assume that ths s true for case + 1,.e., gven any pror +1 p, we have G +1 y +1 G O +1 y +1, for all y 1, and y+1 yo +1. Consder three cases: Case 1. z<y+1 yo +1, we have v +1 z +1 p = vo +1 z +1 p = 0. Case 2. y+1 z<yo +1, we have v +1 z +1 p 0 = vo +1 z +1 p. Case 3. y+1 O z, we frst show that one would not order n the full updatng case. Notce that because y+1 O z, by concavty of GO +1, we have G O +1 z 0 for all z z. Hence, by nducton assumpton, G +1 z G O +1 z 0 for all z z. Therefore, the optmal nventory levels for both and O are z n ths case. As a result, for the pror +1 p, we have v +1 z +1 p = G +1 z +1 p GO +1 z +1 p = v+1 O z +1 p. Therefore, we have shown that v+1 z +1 p vo +1 z +1 p for all z 0. Wth Proposton 4, we have G y G O y, for all y 1, and by concavty of G O y, we have y y O. For the pershable nventory case, we only need to replace h and h N n 3 wth h the unt dsposal/ salvage cost of the pershable product. Because no nventory s carred over to subsequent perods, the nteracton between perods s based purely on nformaton updatng. We have and G G O y = G M y = E p Ry p + y = E pry p =y+1 EK y 1 v p y 1K E K y v p yk f where h and h N are replaced by h the unt salvage cost of the pershable product. By arguments analogous to the proof of Proposton 4, we can show that for the pershable product case, E 1 pe K n+1 v 0 p n + 1K 1 E 1 pe K n v 0 p n K 1 for = 2N n 0 and any p. Hence, we mmedately have G y G O y = G M y By concavty of G O y, we have y y O.

12 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 247 Theorem 2 confrms the ntuton that to learn about the substtuton probablty, a forward-lookng manager should reduce the nventory level to observe more trals of customer substtuton behavor. From the prevous secton, we know that y O y M. Hence, by Theorem 2, we mmedately have y y M ;.e., the Bayesan optmal nventory level wth observed lost sales updatng both parameters and p s less than the myopc nventory level. Theorem 2 s derved based on a general dscrete demand dstrbuton. Assumng dscrete rather than contnuous demand, as n most of the Bayesan nventory lterature, we can explctly capture the customer substtuton behavors as ndependent Bernoull trals and thus ncorporate learnng about the probablty that customers wll accept a substtute. In summary, we have observed two benefcal effects from nventory reducton. Frst, reducng the nventory level ncreases the number of observatons of customer substtuton behavor and thus mproves estmaton of p hence, y y O. Second, reducng the nventory level reduces the rsk of havng excess nventory n the next perod n the case of observng poor sales n the current perod hence, y O y M. Eppen and Iyer 1997 study Bayesan nventory management of a fashon product nonpershable durng the sellng season wth observed lost sales and allow for salvagng n any perod. For a class of demand dstrbutons that nclude normal, Posson, and negatve bnomal, they prove that the optmal polcy s of the form order-up-to L and salvage-downto U where L U When we allow for salvagng n every perod n our model wth general dscrete demand and unknown substtuton probablty, the optmal polcy s not necessarly of the L U type because the expected total reward functon for ether orderng or salvagng s not necessarly concave n the nventory level y. The optmal nventory level y for perod after orderng or salvagng s a functon, possbly complex, of the nventory remanng at the end of perod 1. Nevertheless, Theorem 2 holds when we allow for salvagng n each perod see the onlne appendx 1 The proof follows the same basc lne of 1 An onlne appendx to ths paper s avalable on the Manufacturng & Servce Operatons Management webste nforms.org/ecompanon.html. argument as the one above but s, of course, more complex. Now let us consder the case when lost sales are not observed. The frst-order dfference of the objectve functon for perod s gven by G U 2 y = E p Ry p + y x=0 v U 2 +1 y x +1 p xf x p pxy+1f y+1 x p v U 2 x=y v U pxyf y x p x=y 13 The second term n 13 s negatve because v+1 U 2 z +1 p x 0 for all z 0, but the sgn of the last term s no longer defnte because Proposton 1 no longer holds when we update and p. As a result, makng lost sales unobservable may ether ncrease or decrease the Bayesan optmal nventory level ths s proven by the numercal examples mmedately below. Qualtatvely, from the observed lostsales case, we have shown that one should stock less nventory to better estmate p. However, n 2.2, we have shown that one would stock more to better estmate when lost sales are not observed. Havng a better estmate of s also helpful n nferrng p from observng the number of customers that chose the substtute but not lost sales. The followng specally constructed numercal examples and analytc result for the case of exponental demand shed some lght on the condtons under whch one should stock more versus less Specal Examples. Ths subsecton provdes examples n whch demand parameter s known, but one may ether stock more or stock less than the myopc nventory level to estmate the substtuton probablty p. Consder a two-perod problem n whch cost parameters are set as r = 10, c = 0, m = 10, q = 0, and h = 3. The unknown substtuton probablty p takes value n 02 09, wth a pror belef of Prp = 02 = w and

13 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 248 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS Fgure 2 Inventory level Impact of Pror Belef Parameter on the Optmal Inventory-Level Decsons Optmal nventory level for perod Pror belef parameter w Prp = 09 = 1 w. The known dscrete demand dstrbuton s gven by the followng. Demand Probablty f Fgure 2 shows the Bayesan optmal nventory level wth observed and unobserved lost sales as well as the myopc nventory level n the frst perod where the pror belef parameter w vares from 0 to 0.4. From ths example, we see that the Bayesan optmal nventory level wth unobserved lost sales s greater than or equal to the observed lost-sales case n all w values. However, the Bayesan nventory wth unobserved lost sales can be ether greater than when w = 014 or less than when w = 030 the myopc nventory level. One would stock more wth unobserved lost sales n ths example because of the need for reducng ambguty about the number of customers who arrve after stockout. In partcular, n the event that the number sold equals the nventory level, one cannot tell whether the realzed demand was perfectly matched to the nventory level or whether t exceeded the nventory level but no customers were wllng to substtute. In the former case, one dd not learn anythng about the customer substtuton probablty, and thus the posteror on p remans the same as the pror, whle n the latter case, the posteror s shfted downward because one learns that customers are not lkely to substtute. Wthout observng lost sales, one cannot dscern whch of these cases occurred, and thus t may be proftable to ncrease the nventory level to separate them especally when the probablty s hgh for the crtcal pont where demand realzaton equals the myopc nventory level. From the above example, we make the followng remark: Remark 1. Suppose that the demand dstrbuton s known but the substtuton probablty s not known. If lost sales are not observed, n general, the Bayesan optmal nventory level may be ether greater than or less than the myopc nventory level. For the case when the demand parameter and substtuton probablty p are unknown, analyss of the unobserved lost-sales case s generally ntractable. However, the two-perod model wth exponental demand dstrbuton ntroduced n Proposton 3 of s tractable. For ths specal case, we can dentfy the condtons under whch the Bayesan optmal nventory level s greater than the myopc nventory level. Proposton 5 Chen Suppose that the demand dstrbuton s exponental and the product s pershable. For a two-perod problem wth censored demand observaton and unknown p 0 1, dependng on the pror uncertanty and cost parameters, ether of the followng two clams hold exclusvely: a For any pror probablty w 0 1 for p, the Bayesan optmal nventory level wth updatng of both demand parameter and substtuton probablty p s greater than the myopc nventory level,.e., y1 U 2 >y1 M. b There exsts a pror probablty w 0 1 for p, subject to y1 U 2 >y1 M when 0 w<w, and y1 U 2 <y1 M when w <w<1. Furthermore, w s decreasng n m the contrbuton margn of the substtute product. The proof s n the frst author s unpublshed PhD thess Chen Proposton 5 says that two scenaros can happen n ths specal settng: a The Bayesan nventory manager wll stock more than the myopc level to learn about demand, regardless of her belefs about the substtuton probablty the stock more effect domnates the stock less effect; or b the Bayesan nventory manager may stock more or less, dependng on the pror belef

14 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 249 of the substtuton probablty. If p = 0 s lkely, she stocks more because demand s more lkely to be censored, and vce versa. Furthermore, as the contrbuton margn on the substtute ncreases, the Bayesan nventory manager wll tend to stock less than the myopc nventory level because learnng the substtuton probablty wll ncrease expected proft n the followng perod. In summary, when lost sales are not observed, the sgn and magntude of the devaton of the Bayesan optmal nventory level from the myopc nventory level depend on the value of actve learnng.e., the expected gan n future proft obtaned by devatng from the myopc nventory level. Ths, n turn, depends on the pror uncertanty and cost structure of the underlyng model. For nstance, when the product s pershable, f the stockout penalty s hgh, the stock more effect tends to domnate the stock less effect because wth hgh servce level the chance of stockout s low; hence, the beneft of estmatng the substtuton probablty becomes neglgble. 3. Maxmum Lkelhood Estmaton for a Capactated System In ths secton, we consder a system wth constraned capacty, operated accordng to a base-stock polcy. Customers arrve accordng to a Posson process wth rate. Let Zt denote the nventory level n the system at tme t. IfZt s postve, the arrvng customer purchases one unt of the product. Otherwse, wth probablty p the customer wll jon the queue to purchase the product at a later tme, and wth probablty 1 p he or she wll balk or leave the system wthout makng a purchase. The nventory level s revewed n contnuous tme, and when t falls below the base-stock level B, producton starts. The producton lead tmes are..d. exponental wth rate. Therefore, the system s essentally an M/M/1 maketo-stock queue wth state-dependent arrval rates: It s f Zt > 0 and p otherwse. The process Zt s a contnuous-tme Markov chan wth a state space of B. We denote the process n the steady state by Z and defne = /. We begn by showng how operatng costs and the optmal base-stock level vary wth the probablty p that customers wll wat. The proofs of these results are elementary; hence, they are omtted. Proposton 6. The steady-state dstrbuton of Zt s gven by PZ = z 1 1 p 1 p 1 p B+1 B z f 0 z B = 1 1 p 1 p 1 p B+1 B z p z f z 0 The steady-state probablty that the product s out of stock s PZ 0 = 1 B 1 p 1 p B+1 Furthermore, PZ 0 s decreasng n the base-stock level B and ncreasng n the load factor. Let h be the holdng cost rate and r the foregone revenue on each of the lost sales. The system s longrun average cost rate s gven by CBp = hez + + r1 ppz 0 = h 1 pb+1 + B B p 1 p B+1 1 p1 B + r 1 p 1 p B+1 The followng proposton gves the optmal basestock level B p as a functon of the watng probablty p: Proposton 7. The optmal base-stock level B p = arg mn B 012 CB p s characterzed by 1 gb r /h and 2 gb + 1 r /h where gb = 1 p B 1/1 p1 B. Furthermore, B p s decreasng n p, and CB p p s decreasng n p. Bayesan optmzaton s ntractable n ths contnuous-revew queueng model. Instead, we derve MLEs and show how ther rate of convergence vares wth the base-stock level. Qualtatvely, by modfyng the base-stock level to ncrease the rate of convergence of the MLEs, the system manager rapdly learns about the demand rate and the probablty that a customer s wllng to wat.

15 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 250 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 3.1. Unobserved Lost-Sales Case Suppose that when the product s out of stock, the manager observes customers that jon the queue, but does not observe customers that balk. Let 1 t be the amount of tme the product s out of stock durng 0t, and N 1 t the number of customers that jon the queue and wat whle the product s out of stock durng 0t. Fnally, let denote weak convergence. Proofs for Propostons 8 and 9 are gven n the appendx. Proposton 8. If s known, then the MLE for p s gven by ˆpt = N 1 t/ 1 t. Furthermore, t 1/2 ˆpt p p/pz 0 N0 1 as t, where N0 1 s the standard normal dstrbuton. Proposton 8 establshes that the accuracy of the MLE ˆpt s ncreasng n PZ 0. Because PZ 0 s decreasng n B and ncreasng n, as shown n Proposton 6, we conclude that the accuracy of ˆpt s decreasng n B and ncreasng n. In other words, lowerng the ntal base-stock level B or reducng the producton rate wll lead to more accurate estmaton of p. Hence, the manager should set the base-stock level below B p durng an expermentng perod to ncrease the MLE accuracy. When s unknown, one can derve smlar MLEs and the weak convergence results. Let N 2 t denote the number of arrvals that occur whle the product s n stock durng 0t, and let 2 t denote the amount of tme that the product s n stock durng 0t. Proposton 9. The MLEs for and p are gven by ˆt = N 2 t/ 2 t and ˆpt = N 1 t 2 t/n 2 t 1 t respectvely. Furthermore, as t, t 1/2 ˆt N0 1 PZ>0 and t 1/2 p ˆpt p PZ 0 + p 2 PZ>0 N01 Proposton 9 establshes that ncreasng the basestock level B or the producton rate wll ncrease the accuracy of ˆt, whch s analogous to the stock more result n Theorem 1. However, the accuracy of ˆpt s not monotonc n the base-stock level B or the producton rate. For small B or, ncreasng B or ncreases the accuracy of ˆpt by mprovng the estmate of. As B or becomes very large, ˆpt becomes less accurate because lttle watng behavor s observed. Ths resembles the nsghts we obtaned n Observed Lost-Sales Case Suppose that all customer arrvals are observed. Then, the MLEs for and p become ˆt = At/t and ˆpt = N 1 t/a 1 t, where At s the number of customers that arrve durng 0t, and A 1 t s the number of customers that arrve whle the product s out of stock and must decde whether to wat or balk. Clearly, the choce of base-stock level does not affect the maxmum lkelhood estmaton of. Furthermore, reducng the base-stock level B mproves the accuracy of ˆpt by ncreasng the number of observatons of the decson to wat or balk. Ths renforces the stock less result obtaned n Theorem 2 of Concludng Remarks A well-known result n Harpaz et al. 1982, Larvere and Porteus 1999, and Dng et al s that for a pershable product wth unobserved lost sales, one should stock more when learnng about the demand dstrbuton. In contrast, we show that the Bayesan optmal nventory level may be lower than the myopc nventory level.e., n some cases one should stock less to learn about the substtuton probablty, or for a nonpershable product. We also prove that makng lost sales unobservable ncreases the Bayesan optmal nventory level; n ths specfc sense, the famous stock more result of Harpaz et al. 1982, Larvere and Porteus 1999, and Dng et al generalzes to the case of nonpershable nventory. Our results are for a sngle product, but serve as a buldng block toward analyss of systems wth multple, nteractng products. In general, customers may substtute among multple alternatves when ther top choce s not avalable, and the nventory manager must estmate the probablty of substtuton among multple products. Retalers wll soon be able to use RFID to contnuously observe when a product s out of stock. If the consequent decsons by customers to backorder or substtute an alternatve product can be captured, both Theorem 2 and Proposton 8 suggest that wth ths vsblty retalers should lower

16 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 251 nventory levels to learn more about whether customers wll wat or substtute an alternatve product. Insofar as the decson of customers to make substtutons can be observed, our general Bayesan analyss for the sngle-tem problem can be extended to problems nvolvng dynamc estmaton of multple substtuton probabltes n consumer choce models such as the multnomal logt model. Effectve heurstcs are needed n practce. For example, Kok and Fsher 2007 propose a heurstc for estmatng mult-tem demand and substtuton and optmzng the product assortment; n a supermarket applcaton, ths heurstc acheves a 50% proft mprovement. Our results are most nsghtful for nventory and capacty management for an nnovatve product lke the Toyota Prus hybrd. Crtcally, operatons managers must estmate customers wllngness to wat as well as the demand dstrbuton. Toyota has profted from mantanng low capacty and nventory levels and learnng that customers wll wat n excess of sx to eght months for the Prus. Interestngly, Toyota has allocated car nventory wthn the Unted States such that customers must wat longer on the coasts than n other parts of the country. Ths lkely reflects regonal dfferences n customers wllngness to wat Garennes 2004, Nauman Acknowledgments The authors thank an anonymous senor edtor and two referees as well as Mor Armony, Peter Glynn, Robert Phllps, and Evan Porteus for helpful nsghts and suggestons. Ths research was supported by Nth Orbt and by the Natonal Scence Foundaton under Grant DMI Appendx f y Lemma A1. For any gven, p 0 1, and y 0, y+1 x p and f x p satsfy the followng relatonshp: f y x p f y+1 x p for x<y = f y+1 y+1 x p + 1 p f x + 1 p for x = y p f y+1 y+1 x p + 1 p f x + 1 p for x>y Proof. By defnton 2 and some elementary algebrac manpulatons. Proof of Proposton 1. We prove the case of p>0; the case of p = 0 can be easly verfed followng the same steps. Because p s known, we wrte f y x as shorthand for f y x p. The sequence of predctve sales denstes f y x = 1 2N satsfes x= f y f y x 1 d f y x x 1y 1 1 d =1 =2N By backward nducton, usng Lemma A1, we frst show that the followng holds: for = 2N, z 0, and any, fx = y, E v U z xy f y x E v U z xy + 1 f y+1 x + 1 p v U z x + 1y+ 1 f y+1 x and f x>y, E v U z xy f y x E p v U z xy + 1 f y+1 x + 1 p v U z x + 1y+ 1 f y+1 x For case = N, we verfy 15 as follows. E v U N z xy f y x = max R N y f y y x z d = max R N y p f y+1 y x + 1 p z f y+1 x + 1 d p max y z + 1 p max y z = E p v U R N y f y+1 x d R N y f y+1 x + 1 d z xy + 1 f y+1 x + 1 p v U z x + 1y+ 1 f y+1 x where 16 follows from denttes from Lemma A1. Now, assume the result holds for case + 1. We check for case : E v U z xy f y x =max Ry + v U y +1 z y x + +1 x y xy x =0 f y x f y x d

17 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 252 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS =max Ry f y y x z d + v U +1 y x + +1 x y xy x =0 f y x f y x d =max Ry f y y x z d+ x =0 E y x v U +1 y x + +1 xy f y x max Ry f y y x z d+ =max y z x =0 E y x p v U +1 y x + +1 xy+1 x +1 p v U +1 y x + +1 x+1y+1 f y+1 x+1 Ry p f y+1 + x =0 p f y+1 y+1 x +1 p f x+1 d v U +1 y x + +1 x y xy+1 f y x f y+1 x d + 1 p v+1 U y x + x =0 +1 x y x+1y+1 f y x f y+1 x+1 d p max Ry y z + x =0 v U +1 y x + +1 x y xy+1 f y x f y+1 x d +1 p max Ry y z + v U +1 y x + +1 x y x+1y+1 x =0 f y x f y+1 x+1 d =E p v U z xy+1 f y+1 x +1 p v U z x+1y+1 y+1 f 17 Note that nequalty 17 follows from nducton assumpton. Hence, 15 holds. Smlarly, we can show that 14 also holds. E Apply 14 and 15 to the rght-hand sde of 9, we have v U 0 x yf y x x=y E v U 0 y+1 yy + 1f y + 1 p v U 0 y+1 y + 1y+ 1f y p v U 0 y+1 xy + 1f x + 1 p x=y+1 v U 0 y+1 x + 1y+ 1 f x + 1 = E v U 0 y+1 xy + 1f x x=y Hence, nequalty 9 holds. Lemma A2. For = 2N, n 0, and any p the followng holds: E pe K n+1 v z p n + 1K = E pp E pe K n v z p n K + E p1 p E pe K n v z p n K where p = p p/ p p dp, and p = 1 p p/ 1 p p dp. Proof. The rght-hand sde of the equaton can be expressed as n p p dp v z pk+1 1 p n k p p k=0 k+1 1 p n k pdp n k p k+1 1 p n k p dp + 1 p p dp p p dp n v z p k 1 p n k+1 p pk 1 p k=0 n k+1 p dp n k p k 1 p n k+1 p dp 1 p p dp n = v z p k+1 1 p n k p k=0 p k+1 1 p n k p dp n p k+1 1 p n k p dp k n + v z p k 1 p n k+1 p pk 1 p k=0 n k+1 p dp n p k 1 p n k+1 p dp k n+1 = v z p k 1 p n k+1 p pk 1 p n k+1 p dp k=1

18 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 253 n p k 1 p n k+1 p dp k 1 n + v z p k 1 p n k+1 p pk 1 p k=0 n k+1 p dp n p k 1 p n k+1 p dp k v z p k 1 p n k+1 p pk 1 p k=0 n k+1 p dp n + 1 p k 1 p n k+1 p dp k n+1 = = E pe K n+1 v z p n + 1K where the dentty n+1 k = n k 1 + n k s used n the secondto-last equalty. Lemma A3. For = 2N, n 0, and any p, f E pe K 1 v then z p 1K v z E pe K n+1 v z p n + 1K E pe K n v z p n K Proof. By nducton, the case of n = 0 holds by the lemma condton. Now, assume t s true for case n 1. By Lemma A2, we have E pe K n+1 v z p n + 1K = E pp E pe K n v z p n K + E p1 p E pe K n v z p n K E pp E pe K n 1 v z p n 1K +E p1 p E pe Kn 1 v z pn 1K = E pe K n v z p n K The nequalty follows from the nducton assumpton and the last equalty follows from Lemma A2. Proof of Proposton 4. The sequence of predctve demand denstes f = 1 2N satsfes f 1 d = 1 f = f 1 1 d = 2N Now prove by backward nducton. It s straghtforward to verfy that E N 1 pe K 1 v N z N N p 1K N 1 v N z N N 1 Hence, by Lemma A3, we have E N 1 pe K n+1 v N z N N p n + 1K N 1 E N 1 pe K n v N z N N p n K N 1 Now, assume t s true for the case + 1. By Lemma A2, we have E 1 pe K1 v z p1k 1 = p 1 pdp maxe y z p11 1 y Ry p+ v +1 y +1 p f =0 + E K y 1 v =y+1 +1 p y 1K f + 1 p 1 pdp maxe y z p10 1 y Ry p+ v +1 y +1 p f + =y+1 =0 E K y 1 v p y 1K f max E y z 1 p Ry p y + E 1 pe K1 v +1 y +1 p f + =0 p 1 pdp E EK y 1 p11 1 v +1 z p y 1K + 1 p 1 pdp E p10 1 E K y 1 v +1 z p y 1K f =y+1 y =max E Ry p + y z 1p E 1 pe K1 =0 v +1 y +1 p f + E 1 p =y+1 E K y v +1 z p yk 1 f

19 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty 254 Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS max E y z 1 p Ry p y + E 1 p v +1 y +1 1 p f + =0 =y+1 E 1 pe K y 1 v z y =max E y z 1 p Ry p p y 1K 1 f f+ v +1 =0 =y+1 y +1 1 p E K y 1 v +1 z p y 1K 1 f =v z 1 p The last nequalty follows from the nducton assumpton and the equalty rght before that s by Lemma A2. Applyng Lemma A3 agan, we conclude that for n 0, E 1 pe K n+1 v z p n + 1K 1 E 1 pe K n v z p n K 1 Proof of Proposton 8. ˆpt = N 1 t/ 1 t can be obtaned by solvng the frst-order condton of the lkelhood functon gven that s known over perod 0t. We focus on showng the weak convergence result of ˆpt. Frst, note that t 1/2 N 1 t/ 1 t p = t 1/2 N 1 t p 1 t/ 1 t/t. Because lm t 1 t/t = PZ 0 almost surely a.s., t suffces to show that t 1/2 N 1 t p 1 t ppz 0 N0 1, ast. Defne Mt= N 1 t p 1 t = N 1 t t p1zs sds, 0 whch s a martngale. Scale Mt as M t = 1/2 N 1 t/ t/ p1zs sds. Now we show that the quadratc 0 varaton of M t, denoted by M M t, converges to ppz 0 as 0. Note that M M t = lm m j=1 = lm m M jt/m M j 1t/m 2 m j=1 m N jt/m N j 1t/m Utlze the fact that m 2 lm N jt/m N j 1t/m m j=1 = lm m j=1 jt/m p 1Zs 0ds 2 j 1t/m m N jt/m N j 1t/m = Nt/ Also, lm m j=1 m 2p lm jt/m j 1t/m m j=1 1Zs 0ds N jt/m N j 1t/m m 2p t/m N jt/m N j 1t/m = lm m 2ptN t//m = 0 Smlarly, lm m m j=1 p jt/m 1Zs j 1t/m 0ds2 = 0. Hence, we have M M t = N t/. Now, let M n t = n N t/ n t/ n p1zs 0ds, wth 0 n 0. By Theorem 2.18 of Hall and Heyde 1980, t s easy to verfy that lm n n M n t = 0. Because n can be any real postve decreasng sequence to zero, we have lm 0 M t = 0 a.s. Hence, lm 0 Nt/ = lm 0 p = pt lm t/ 0 t/ 0 0 = ptpz 0 1Zs 0ds 1Zs 0 ds/t/ The last equalty follows from the property of contnuoustme Markov chan. Also, check condton that for T>0, lm E 0 sup t T = lm 0 1/2 E M t M t sup t T Nt/ Nt / lm 1/2 = 0 0 Thus, by the martngale central lmt theorem Theorem of Ether and Kurtz 1986, we conclude that M t ppz 0 Bt, as 0 where Bt s the standard Brownan moton process. Now, f we let t = 1, we have 1/2 N 1 1/ 1/ p1zs sds ppz 0 0 N0 1, as 0, whch completes our proof for the convergence result. Proof of Proposton 9. ˆt = N 2 t/ 2 t, and ˆpt = N 1 t 2 t/n 2 t 1 t can be obtaned by solvng the frstorder condtons of the lkelhood functon. By smlar proof steps of Proposton 8, t s straghtforward to show t 1/2 ˆt /PZ>0N 0 1, ast. Now we focus on showng the convergence result for ˆpt. Frst, note that t 1/2 N 1 t 2 t/n 2 t 1 t p = t 1/2 2 t/ 1 t N 1 t p 1 t + pn 2 t 2 t/n 2 t/t. Note that lm t N 2 t/t = PZ >0 a.s. Let = PZ >0/ PZ 0. Defne Mt = N 1 t p 1 t + pn 2 t 2 t. Hence, t suffces to show that t 1/2 Mt ppz>0 + p 2 PZ>0N 0 1 as t.

20 Chen and Plambeck: Dynamc Inventory Management: Demand Dstrbuton and Substtuton Probablty Manufacturng & Servce Operatons Management 102, pp , 2008 INFORMS 255 Notce that Mt s a martngale. Scale Mt as t/ M t = 1/2 N 1 t/ p 1Zs 0ds + p N 2 t/ 0 t/ 0 1Zs > 0ds Checkng the quadratc varaton of M t, we have M t M t = lm m m j=1 = 2 N 1 t/ + p 2 N 2 t/ N 1 jt/m N 1 j 1t/m jt/m p 1Zs 0ds j 1t/m + p N 2 jt/m N 2 j 1t/m jt/m 2 1Zs 0ds j 1t/m Now, by the Law of Large Numbers of the martngale, we have lm 2 N 1 t/ + p 2 N 2 t/ 0 t/ t/ = lm 2 p 1Zs 0ds+ p = ppz>0 + p 2 PZ>0t 1Zs > 0ds Also, verfy that for T > 0, lm 0 Esup t T M t M t = 0. Thus, by the martngale central lmt theorem Theorem of Ether and Kurtz 1986, we have M t ppz>0 + p 2 PZ>0 Bt as 0. Lettng t = 1, we mmedately obtan the convergence result for ˆpt. References Agrawal, N., S. Smth Optmal retal assortments for substtutable tems purchased n set. Naval Res. Logst. Quart Anupnd, R., M. Dada, S. Gupta Estmaton of consumer demand wth stockout based substtuton: An applcaton to vendng machne products. Marketng Sc Armony, M., E. Plambeck The mpact of duplcate orders on demand estmaton and capacty nvestment. Management Sc Azoury, K. S Bayes soluton to dynamc nventory models under unknown demand dstrbuton. Management Sc Bensoussan, A., M. Cakanyldrm, S. P. Seth A mult-perod newsvendor problem wth partally observed demand. Math. Oper. Res. Forthcomng. Chen, L Optmal nformaton acquston, nventory control, and forecast sharng n operatons management. Unpublshed PhD thess, Stanford Unversty, Stanford, CA. Dng,., M. L. Puterman, A. Bs The censored newsvendor and the optmal acquston of nformaton. Oper. Res Eppen, G. D., A. V. Iyer Improved fashon buyng wth Bayesan updates. Oper. Res Ernst, R., P. Kouvels The effects of sellng package goods on nventory decsons. Management Sc Ether, S. N., T. G. Kurtz Markov Processes: Characterzaton and Convergence. John Wley & Sons, New York. Garennes, C Area car buyers lne up to purchase hybrds. News-Gazette June 2, Champagn-Urbana, IL. Hall, P., C. C. Heyde Martngale Lmt Theory and Its Applcaton. Academc Press, New York. Harpaz, G., W. Y. Lee, R. L. Wnkler Learnng, expermentaton, and the optmal output decsons of a compettve frm. Management Sc Heyman, D. P., M. J. Sobel Stochastc optmzaton. Stochastc Models n Operatons Research, Vol. II. McGraw-Hll Book Company, New York. Johnson, N. L., S. Kotz, A. W. Kemp Unvarate Dscrete Dstrbutons, 2nd ed. John Wley & Sons, New York. Kok, G. A., M. L. Fsher Demand estmaton and assortment optmzaton under substtuton: Methodology and applcaton. Oper. Res Larvere, M. A., E. L. Porteus Stalkng nformaton: Bayesan nventory management wth unobserved lost sales. Management Sc Lppman, S. A., K. F. McCardle The compettve newsboy. Oper. Res Lovejoy, W. S Myopc polces for some nventory models wth uncertan demand dstrbutons. Management Sc Lu,., J. S. Song, K. Zhu On The censored newsvendor and the optmal acquston of nformaton. Oper. Res Lu,., J. S. Song, K. Zhu Analyss of pershable nventory systems wth censored data. Oper. Res. Forthcomng. Mahajan, S., G. van Ryzn Retal nventores and consumer choce. S. Tayur, R. Ganeshan, M. Magazne, eds. Quanttatve Models for Supply Chan Management. Kluwer Academc Publshers, Norwell, MA, Mahajan, S., G. van Ryzn. 2001a. Stockng retal assortments under dynamc consumer substtuton. Oper. Res Mahajan, S., G. van Ryzn. 2001b. Inventory competton under dynamc consumer choce. Oper. Res Nahmas, S., S. A. Smth Optmzng nventory levels n a two-echelon retaler system wth partal lost sales. Management Sc Nauman, M Buyers lne up and wat for the Toyota gaselectrc hybrd. Mercury News August 26. San Jose, CA, 1c 2c. Netessne, S., N. Rud Centralzed and compettve nventory models wth demand substtuton. Oper. Res Noonan, P. S When customers choose: A mult-product, mult-locaton newsboy model wth substtuton. Workng paper, Emory Unversty, Atlanta, GA.