This paper can be downloaded without charge from the Social Sciences Research Network Electronic Paper Collection:

Transcription

1 Workng Paper Coordnatng Prcng and Inventory Replenshment wth Nonparametrc Demand Learnng Boxao Chen Department of Industral and Operatons Engneerng Unversty of Mchgan Xul Chao Department of Industral and Operatons Engneerng Unversty of Mchgan Hyun-Soo Ahn Stephen M. Ross School of Busness Unversty of Mchgan Ross School of Busness Workng Paper Seres Workng Paper No. 194 June 015 Ths paper can be downloaded wthout charge from the Socal Scences Research Network Electronc Paper Collecton: UNIVERSITY OF MICHIGAN

2 Coordnatng Prcng and Inventory Replenshment wth Nonparametrc Demand Learnng Boxao Chen 1, Xul Chao and Hyun-Soo Ahn 3 Abstract We consder a frm e.g., retaler) sellng a sngle nonpershable product over a fnte-perod plannng horzon. Demand n each perod s stochastc and prce-dependent, and unsatsfed demands are backlogged. At the begnnng of each perod, the frm determnes ts sellng prce and nventory replenshment quantty, but t knows nether the form of demand dependency on sellng prce nor the dstrbuton of demand uncertanty a pror, hence t has to make prcng and orderng decsons based on hstorcal demand data. We propose a nonparametrc data-drven polcy that learns about the demand on the fly and, concurrently, apples learned nformaton to determne replenshment and prcng decsons. The polcy ntegrates learnng and acton n a sense that the frm actvely experments on prcng and nventory levels to collect demand nformaton wth the least possble proft loss. Besdes convergence of optmal polces, we show that the regret, defned as the average proft loss compared wth that of the optmal soluton when the frm has complete nformaton about the underlyng demand, vanshes at the fastest possble rate as the plannng horzon ncreases. Keywords: dynamc prcng, nventory control, demand learnng, nonparametrc estmaton, nonpershable products, asymptotc optmalty. 1 Department of Industral and Operatons Engneerng, Unversty of Mchgan, Ann Arbor, MI Emal: boxchen@umch.edu Department of Industral and Operatons Engneerng, Unversty of Mchgan, Ann Arbor, MI Emal: xchao@umch.edu 3 Department of Technology and Operatons, Ross School of Busness, Unversty of Mchgan, Ann Arbor, MI Emal: hsahn@umch.edu 1 Electronc copy avalable at:

3 1 Introducton Balancng supply and demand s a challenge for all frms, and falure to do so can drectly affect the bottom-lne of a company. From the supply sde, frms can use operatonal levers such as producton and nventory decsons to adjust nventory level n pace of uncertan demand. From the demand sde, frms can deploy marketng levers such as prcng and promotonal decsons to shape the demand to better allocate the lmted or excess) nventory n the most proftable way. Wth the ncreasng avalablty of demand data and new technologes, e.g., electronc data nterchange, pont of sale devces, clck stream data etc., deployng both operatonal and marketng levers smultaneously s now possble. Indeed, both academcs and practtoners have recognzed that substantal benefts can be obtaned from coordnatng operatonal and prcng decsons. As a result, the research lterature on jont prcng and nventory decsons has rapdly grown n recent years, see, e.g., the survey papers by Petruzz and Dada 1999), Elmaghraby and Kesknocak 003), Yano and Glbert 003), and Chen and Smch-Lev 01). Despte the volumnous lterature, the majorty of the papers on jont optmzaton of prcng and nventory control have assumed that the frm knows how the market responds to ts sellng prces and the exact dstrbuton of uncertanty n customer demand for any gven prce. Ths s not true n many applcatons, partcularly wth demand of new products. In such settngs, the frm needs to learn about demand nformaton durng the dynamc decson makng process and smultaneously tres to maxmze ts proft. In ths paper, we consder a frm sellng a nonpershable product over a fnte-perod plannng horzon n a make-to-stock settng that allows backlogs. In each perod, the frm sets ts prce and nventory level n antcpaton of prce-senstve and uncertan demand. If the frm had complete nformaton about the underlyng demand dstrbuton, ths problem has been studed by, e.g., Federgruen and Hechng 1999), among others. The pont of departure ths paper takes s that the frm possesses lmted or even no pror knowledge about customer demand such as ts dependency on sellng prce or the dstrbuton of uncertanty n demand fluctuaton. We develop a nonparametrc data-drven algorthm that learns the demand-prce relatonshp and the random error dstrbuton on the fly. We also establsh the convergence rate of the regret, defned as the average proft loss per perod of tme compared wth that of the optmal soluton had the frm known the random demand nformaton, and that s fastest possble for any learnng algorthm. Ths work s the frst to present a nonparametrc data-drven algorthm for the classc jont prcng and nventory control problem that not only shows the convergence of the proposed polces but also the convergence rate for regret. Electronc copy avalable at:

4 1.1 Related lterature Almost all early papers n jont prcng and nventory control, e.g., Whtn 1955), Federgruen and Hechng 1999), and Chen and Smch-Lev 004), among others, assume that a frm has complete knowledge about the dstrbuton of underlyng stochastc demand for any gven sellng prce. The complete nformaton assumpton provdes analytc tractablty necessary for characterzng the optmal polcy. The extenson to the parametrc case the frm knows the class of dstrbuton but not the parameters) has been studed by, for example, Subrahmanyan and Shoemaker 1996), Petruzz and Dada 00), and Zhang and Chen 006). Chung et al. 011) also consder the problem of dynamc prcng and nventory plannng wth demand learnng, and they develop learnng algorthms usng Bayesan method and Markov chan Monte Carlo MCMC) algorthms, and numercally evaluate the mportance of dynamc prcng. An alternatve to the parametrc approach s to model the frm s problem n a nonparametrc settng. Under ths framework, the frm does not make specfc assumptons about underlyng demand. Instead, the frm makes decsons solely based on the collected demand data, see Burnetas and Smth 000). Our work falls nto ths category. To our best knowledge, Burnetas and Smth 000) s the only paper that consders the jont prcng and nventory control problem n a nonparametrc settng. The authors consder a maketo-stock system for a pershable product wth lost sales and lnear costs, and propose an adaptve polcy to maxmze average proft. They assume that the prce s chosen from a fnte set and formulate the prcng problem as a mult-armed bandt problem, and show that the average proft under ther approxmaton polcy converges n probablty. No convergence rate or performance bound s obtaned for ther algorthm. Other approaches n the lterature on developng nonparametrc data-drven algorthms nclude onlne convex optmzaton Agarwal et al. 011, Znkevch 003, Hazan et al. 006), contnuumarmed bandt problems Auer et al. 007, Klenberg 005, Cope 009), and stochastc approxmaton Kefer and Wolfowtz 195, La and Robbns 1981, and Robbns and Monro 1951). In fact, Burnetas and Smth 000) s an example of mplementng such algorthms to the jont prcng and nventory control problem. However, these methodologes requre that the proposed soluton be reachable n each and every perod, whch s not the case wth our problem. Ths s because, n a demand learnng algorthm of jont prcng/nventory control problem, n each perod the algorthm utlzes the past demand data to prescrbe a prcng decson and an order up-to level. However, f the startng nventory level of the perod s already hgher than the prescrbed order up-to level, then the prescrbed nventory level for the perod cannot be reached. Actually, that s precsely the reason that Burnetas and Smth 000) focused on the case of pershable product hence the 3

5 frm has no carry-over nventory and the nventory decson obtaned by Burnetas and Smth 000) based on mult-armed bandt process can be mplemented n each perod). Agarwal et al. 011), Auer et al. 007), and Klenberg 005) propose learnng algorthms and obtan regrets that are not as good as ours n ths paper. Znkevch 003) and Hazan et al. 007) present machne learnng algorthms n whch the the exact gradent of the unknown objectve functon at the current decson can be computed, and ther results have been appled to dynamc nventory control n Huh and Rusmevchentong 009). However, n the jont prcng and nventory control problem wth unknown demand response, the gradent of the unknown objectve functon cannot be obtaned thus the method cannot be appled. 1. Postonng of ths paper The closest related research works to ours are Besbes and Zeev 015), Lev et al. 007) and Lev et al. 010), offerng nonparametrc approaches to pure prcng problem wth no nventory) and pure nventory control problem wth no prcng), respectvely. Besbes and Zeev 015) consder a dynamc prcng problem n whch a frm chooses ts sellng prce to maxmze expected revenue. The frm does not know the determnstc demand curve.e., how the average demand changes n prce) and learns t through nosy demand realzatons, and the authors establsh the suffcency of lnear approxmatons n maxmzng revenue. They assume that the frm has nfnte supply of nventory, or, alternatvely, the seller has no nventory constrant. In ths case, snce the expected revenue n each perod depends only on ts mean demand, the dstrbuton of random error s mmateral n ther learnng algorthm and analyss. On the other hand, n the dynamc newsvendor problem consdered n Lev et al. 007, 010), the essence for effectve nventory management s to strke a balance between overage cost and underage cost, for whch the dstrbuton of uncertan demand plays a key role. Lev et al. 007) and Lev et al. 010) apply Sample Average Approxmaton SAA) to estmate the demand dstrbuton and average cost functon, and they analyze the relatonshp between sample szes and accuracy of estmatons and nventory decsons. Our problem has both dynamc prcng and nventory control, and the frm knows nether the relatonshp between demand and sellng prce nor the dstrbuton of demand uncertanty. In Besbes and Zeev 015), the authors only need to estmate the average demand curve n order to maxmze revenue, and demand dstrbuton nformaton s rrelevant. In a remark, Besbes and Zeev 015) state that ther method of learnng the demand curve can be appled to maxmzng more general forms of objectve functons beyond the expected revenue whch, however, does not apply to our settng. Ths s because, n the general form presented n Besbes and Zeev 015), 4

6 the objectve functon stll has to be a known functon n terms of prce and the demand curve for a gven prce and a gven demand curve. Thus the frm must know the exact expresson of the objectve functon when the estmate of a demand curve s gven. In our problem, even wth a gven prce and nventory level and a gven demand curve, the objectve functon cannot be wrtten as a known determnstc functon. Indeed, ths functon contans the expected nventory holdng and backorder costs that depend on the dstrbuton of demand fluctuaton, whch s also unknown to the frm. In fact, the latter s a major techncal challenge encountered n ths paper because, as we wll explan below, the estmaton of the demand uncertanty, therefore also of the expected holdng/shortage cost, cannot be decoupled wth the estmaton of the average demand curve, whch s gathered through prce expermentaton. Standard SAA method s mplemented to the newsvendor problem by Lev et al. 007) and Lev et al. 010) whch, however, cannot be appled to our settng for determnng nventory decsons. In Lev et al. 007) and Lev et al. 010), dynamc nventory control s studed n whch prcng s not a decson and t s assumed mplctly) to be gven. The only nformaton the frm s uncertan about s the dstrbuton of random fluctuaton. Therefore, the frm can observe true realzatons of demand fluctuaton whch are used to buld an emprcal dstrbuton. In our model, however, the frm knows nether how average demand responds to the sellng prce demand curve) nor the dstrbuton of fluctuatng demand, but both of them affect demand realzatons. For any estmaton of average demand curve, the error of ths estmate wll affect the estmaton of dstrbuton of random demand fluctuaton. Hence, through the realzaton of random demand we are unable to obtan a true realzaton of random demand error wthout knowng the exact average demand functon. As a result, the standard SAA analyss s not applcable n our settng because unbased samples of the random error cannot be obtaned. Because the frm does not know the exact demand curve a pror, ts estmate of error dstrbuton usng demand data s nevtably based, and as a result, the data-drven optmzaton problem constructed to compute the prcng and orderng strateges s also based. Because of ths bas, t s no longer true that the soluton of the data-drven problem usng SAA must converge to the true optmal soluton. Fortunately, we are able to show that as the learnng algorthm proceeds, the bases wll be gradually dmnshng and that allows us to prove that our learnng algorthm stll converges to the true optmal soluton. Ths s done by establshng several mportant propertes of the newsvendor problem that bound the errors of based samples. One man contrbuton of ths paper s to explctly prove that the soluton obtaned from a based data-drven optmzaton problem stll converges to the true optmal soluton. Fnally, we hghlght on the result of the convergence rate of regret. Besbes and Zeev 015) obtan a convergence rate of T 1/ log T ) for ther dynamc prcng problem, where T s the length of 5

7 the plannng horzon. For the pure dynamc nventory control problem, Huh and Rusmevchentong 009) present a machne learnng algorthm wth convergence rate T 1/. For the jont prcng and nventory problem, we show that the regret of our learnng algorthm converges to zero at rate T 1/, whch s also the theoretcal lower bound. Thus, ths paper strengthens and extends the exstng work by achevng the tghtest convergence rate for the problem wth jont prcng and nventory control. One mportant mplcaton of our fndng s that the lnear demand approxmaton scheme of Besbes and Zeev 015) actually acheves the best possble convergence rate of regret, whch further mproves the result of Besbes and Zeev 015). That s, nothng s lost n the learnng algorthm n approxmatng the demand curve by a lnear model. 1.3 Organzaton The rest of ths paper s organzed as follows. Secton formulates the problem and descrbes the data-drven learnng algorthm for prcng and nventory control decsons. The followng two sectons Sectons 3 and 4) present our major theoretcal results together wth a numercal study, and the man steps of the techncal proofs, respectvely. The paper concludes wth a few remarks n Secton 5. Fnally, the detals of the mathematcal proofs are gven n the Appendx. Formulaton and Learnng Algorthm We consder an nventory system n whch a frm e.g., a retaler) sells a nonpershable product over a plannng horzon of T perods. At the begnnng of each perod t, the frm makes a replenshment decson, denoted by the order-up-to level, y t, and a prcng decson, denoted by p t, where y t Y = [y l, y h ] and p t P = [p l, p h ] for some known lower and upper bounds of nventory level and sellng prce, respectvely. We assume p h > p l snce otherwse, the problem s the pure nventory control problem and learnng algorthms have been developed n Huh and Rusmevchentong 009), Lev et al. 007), and Lev et al. 010). Durng perod t and when the sellng prce s set to p t, a random demand, denoted by D t p t ), s realzed and fulflled from on-hand nventory. Any leftover nventory s carred over to the next perod, and n case the demand exceeds y t, the unsatsfed demand s backlogged. The replenshment leadtme s zero,.e., an order placed at the begnnng of a perod can be used to satsfy demand n the same perod. Let h and b be the unt holdng and backlog costs per perod, and the unt purchasng cost s assumed, wthout loss of generalty, to be zero. The model as descrbed above s the well-known jont nventory and prcng decson problem studed n Federgruen and Hechng 1999), n whch t s assumed that the frm has complete 6

8 nformaton about the dstrbuton of Dt p t ). In ths paper we consder a settng where the frm does not have pror knowledge about the demand dstrbuton. In general, the demand n perod t s a functon of sellng prce p t n that perod and some random varable ɛ t, and t s stochastcally decreasng n p t. The most popular demand models n the lterature are the addtve demand model D t p t ) = λp t ) + ɛ t and multplcatve demand model D t p t ) = λp t ) ɛ t, where λ ) s a strctly decreasng determnstc functon and ɛ t, t = 1,,..., T, are ndependent and dentcally dstrbuted random varables. In ths paper, we shall study both addtve and the multplcatve demand models. However, the frm knows nether the functon λp t ) nor the dstrbuton functon of random varable ɛ t. The frm has to learn from hstorcal demand data, that are the realzatons of market responses to offered prces, and use that nformaton as a bass for decson makng. multplcatve demand. Suppose ɛ t has fnte support [l, u], wth l 0 for the case of To defne the frm s problem, we let x t denote the nventory level at the begnnng of perod t before replenshment decson. We assume that the system s ntally empty,.e., x 1 = 0. The system dynamcs are x t+1 = y t D t p t ) for all t = 1,..., T. An admssble polcy s represented by a sequence of prces and order-up-to levels, p t, y t ), t 1}, where p t, y t ) depends only on realzed demand and decsons made pror to perod t, and y t x t,.e., p t, y t ) s adapted to the fltraton generated by p s, y s ), D s p s ); s = 1,..., t 1}. The frm s objectve s to fnd an admssble polcy to maxmze ts total proft. If both the functon of λ ) and the dstrbuton of ɛ t are known a pror to the frm complete nformaton scenaro), then the optmzaton problem the frm wshes to solve s max p t, y t) P Y y t x t T t=1 p t E[ D t p t )] he[y t D t p t )] + be[ D t p t ) y t ] +), 1) where E stands for mathematcal expectaton wth respect to random demand D t p t ), and x + = maxx, 0} for any real number x. However, snce n our settng the frm does not know the demand dstrbuton, the frm s unable to evaluate the objectve functon of ths optmzaton problem. We develop a data-drven learnng algorthm to compute the nventory control and prcng polcy. It wll be shown n Secton 3 that the average proft of the algorthm converges to that of the case when complete demand dstrbuton nformaton s known a pror, and that the prcng and nventory control parameters also converge to that of the optmal control polcy for the case wth complete nformaton as the plannng horzon becomes long. To save space we shall only present the algorthm and analytcal results for the multplcatve demand model. The results and analyses for the addtve demand case are analogous, and we only hghlght the man dfferences at the end of ths secton. 7

9 Remark 1. For ease of exposton, n ths paper we assume the support of uncertanty ɛ t s bounded. Ths can be relaxed, and all the results hold as long as we assume the moment generatng functons of the relevant random varables are fnte n a small neghborhood of 0, or lght taled. Case of complete nformaton about demand. In the case of complete nformaton n whch the frm knows λ ) and the dstrbuton of ɛ t, t follows from 1) that, f p, y ) s the optmal soluton of each ndvdual term max pe[ D t p)] he[y D t p)] + be[ D t p) y] +}. ) p P,y Y and that ths soluton s reachable n every perod,.e., x t y for all t, then p, y ) s the optmal polcy for each perod. We refer to p and y as the optmal prce and optmal order up-to level or optmal base-stock level), respectvely. It s clear that the reachablty condton s satsfed f the system s ntally empty, whch we assume. We fnd t convenent to analyze ) usng a slghtly dfferent but equvalent form. logarthm on both sdes of D t p t ) = λp t ) ɛ t, we obtan Takng log D t p t ) = log λp t ) + log ɛ t, t = 1,..., T. Denote D t p t ) = log D t p t ), λp t ) = log λp t ) and ɛ t = log ɛ t. Then, the logarthm of demand can be wrtten as D t p t ) = λp t ) + ɛ t, t = 1,..., T. 3) We shall refer to λ ) as the demand-prce functon or demand-prce curve) and ɛ t as random error or random shock). Clearly, λ ) s also strctly decreasng n p P. Hence, n the case of complete nformaton, the frm knows the functon λ ) and the dstrbuton of ɛ t, and when the frm does not know functon λ ) and the dstrbuton of ɛ t, whch s our case, the frm wll need to learn about them. Wthout loss of generalty, we assume E[ɛ t ] = E[log ɛ t ] = 0. If ths s not the case,.e., E[log ɛ t ] = a 0, then E[loge α ɛ t )] = 0, thus f we let ˆλ ) = e a λ ) and ˆɛt = e a ɛ t, then D t p t ) = ˆλp t )ˆɛ t, and ˆλ ) and ˆɛ t satsfy the desred propertes. For convenence, let ɛ be a random varable dstrbuted as ɛ 1. In terms of λ ) and ɛ, we defne Gp, y) = pe λp) E [ e ɛ] he [ y e λp) e ɛ] + [ + be e λp) e ɛ y ] } +. Then problem ) can be re-wrtten as Problem CI: max p P,y Y = max p P Gp, y) 4) he [ y e λp) e ɛ] + [ + be e λp) e ɛ y ] } } +. pe λp) E [ e ɛ] mn y Y 8

10 The nner optmzaton problem mnmzaton) determnes the optmal order-up-to level that mnmzes the expected nventory and backlog cost for gven prce p, and we denote t by y e λp)). The outer optmzaton solves for the optmal prce p. Let the optmal soluton for 4) be denoted by p and y, then they satsfy y = ye λp ) ). The analyss above stpulates that the frm knows the demand-prce curve λp) and the dstrbuton of ɛ, thus we refer to t as problem CI complete nformaton). Learnng algorthm. In the absence of the pror knowledge about the demand process, the frm needs to collect the demand nformaton necessary to estmate λp) and the emprcal dstrbuton of random error ɛ, thus prce and nventory decsons not only affect the proft but also the demand nformaton realzed. The major dffculty les n that, the estmatons of demand-prce curve λp) and the dstrbuton of random error cannot be decoupled. Ths s because, the frm only observes realzed demands, hence wth any estmaton of demand-prce curve, the estmaton error transfers to the estmaton of the random error dstrbuton. Indeed, we are not even able to obtan unbased samples of the random error ɛ t. In our algorthm below we approxmate λp) by an affne functon, and construct an emprcal but based) error dstrbuton usng the collected data. We dvde the plannng horzon nto stages whose lengths are exponentally ncreasng n the stage ndex). At the start of each stage, the frm sets two pars of prces and order-up-to levels based on ts current lnear estmaton of demand-prce curve and based) emprcal dstrbuton of random error, and the collected demand data from ths stage are used to update the lnear estmaton of demand-prce curve and the based emprcal dstrbuton of random error. These are then utlzed to fnd the prcng and nventory decson for the next stage. The algorthm requres some nput parameters v, ρ and I 0, wth v > 1, I 0 > 0, and 0 < ρ 3/4 p h p l )I 1/4 0. To ntate the algorthm, t sets ˆp 1, ŷ 11, ŷ 1 }, where ˆp 1 P, ŷ 11 Y, ŷ 1 Y are the startng prcng and order-up-to levels. For 1, let 1 I = I 0 v, δ = ρi 1 ) 1 4, and t = I k wth t 1 = 0, 5) where I 0 v s the largest nteger less than or equal to I 0 v. The followng s the detaled procedure of the algorthm. Recall that x t s the startng nventory level at the begnnng of perod t, p t s the sellng prce set for perod t, and y t x t ) s the order-upto nventory level for perod t, t = 1,..., T. The number of learnng stages s n = log v 1 v I 0 v ), T +1 where x denotes the smallest nteger greater than or equal to x. k=1 Data-Drven Algorthm DDA) 9

11 Step 0. Intalzaton. Choose v > 1, ρ > 0 and I 0 > 0, and ˆp 1, ŷ 11, ŷ 1. Compute I 1 = I 0 v, δ 1 = ρi 0 ) 1 4, and ˆp 1 + δ 1. Step 1. Settng prces and order-up-to levels for stage. For = 1,..., n, set prces p t, t = t + 1,..., t + I, to p t = ˆp, t = t + 1,..., t + I, p t = ˆp + δ, t = t + I + 1,..., t + I ; and for t = t + 1,..., t + I, rase the nventory levels to y t = max ŷ 1, x t }, t = t + 1,..., t + I, y t = max ŷ, x t }, t = t + I + 1,..., t + I. Step. Estmatng the demand-prce functon and random errors usng data from stage. Let D t = log D t p t ) be the logarthm of demand realzatons for t = t + 1,..., t + I, and compute t ˆα +1, ˆβ +I ) } +1 ) = argmn D t α βp t ), 6) α,β η t = D t ˆα +1 ˆβ +1 p t ), for t = t + 1,..., t + I. 7) Step 3. Defnng and maxmzng the proxy proft functon, denoted by G DD +1 p, y). Defne G DD +1p, y) = peˆα +1 ˆβ +1 p 1 I t +I 1 e ηt I Then the data-drven optmzaton s defned by t +I ) h y eˆα +1 ˆβ + +1 p e ηt ) )} +b eˆα +1 ˆβ + +1 p e ηt y. Problem DD: max p,y) P Y GDD +1p, y) = max peˆα +1 ˆβ +1 p 1 t +I e ηt 8) p P I t 1 +I ) mn h y eˆα +1 ˆβ + ) )}} +1 p e ηt + b eˆα +1 ˆβ + +1 p e ηt y. y Y I Solve problem DD and set the frst par of prce and nventory level to ˆp +1, ŷ +1,1 ) = arg 10 max p,y) P Y GDD +1p, y),

12 and set the second prce to ˆp +1 + δ +1 and the second order-up-to level to ŷ +1, = arg max y Y GDD +1ˆp +1 + δ +1, y). In case ˆp +1 + δ +1 P, set the second prce to ˆp +1 δ +1. Remark. When ˆβ +1 > 0, the objectve functon n 8) after mnmzng over y Y s unmodal n p. To see why ths s true, let d = eˆα +1 ˆβ +1 p and thus p = ˆα +1 log d ˆβ +1 wth d D = [d l, d h ], where d l = eˆα +1 ˆβ +1 p h max d D d ˆα +1 log d ˆβ +1 and d h = eˆα +1 ˆβ +1 p l. Then the optmzaton problem 8) s equvalent to 1 I t +I t=t 1 +1 e ηt ) mn y Y 1 I t +I hy de η t ) + + bde ηt y) +)}}. The objectve functon of ths optmzaton problem s jontly concave n y, d) hence t s concave n d after mnmzng over y Y. Thus, t follows from p = ˆα +1 log d s strctly decreasng n d ˆβ +1 that the objectve functon n 8) after mnmzaton over y) s unmodal n p P. Remark 3. In Step 3 of DDA, the second prce s set to ˆp +1 δ +1 when ˆp +1 + δ +1 > p h. In ths case our condton ρ 3/4 p h p l )I 1/4 0 ensures that ˆp +1 δ +1 p l, thus ˆp +1 δ +1 P. Ths s because, when ˆp +1 > p h δ +1, we have ˆp +1 δ +1 > p h δ +1 p h δ 1 = p h ρi 0 ) 1/4 p l, where the last nequalty follows from the condton on ρ. Dscusson of algorthm and ts connectons wth the lterature. above, teraton focuses on stage that conssts of I perods. In our algorthm In Step 1, the algorthm sets the orderng quantty and sellng prce for each perod n stage, and they are derved from the prevous teraton. In Step, the algorthm uses the realzed demand data and least-squares method to update the lnear approxmaton, ˆα +1 ˆβ +1 p, of λp) and computes a based sample η t of random error ɛ t, for t = t + 1,..., t + I. Note that η t s not a sample of the random error ɛ t. Ths s because ɛ t = D t p t ) λp t ) and the logarthm of) observed demand s D t p t ). However as we do not know λp), t s approxmated by ˆα +1 ˆβ +1 p, therefore η t = D t p t ) ˆα +1 ˆβ +1 p t ) D t p t ) λp t ) = ɛ t. For the same reason, the constructed objectve functon for holdng and shortage costs s not a sample average of the newsvendor problem. In the tradtonal SAA, mathematcal expectatons are replaced by sample means, see e.g., Kleywegt et al. 001). Lev et al. 007) and Lev et al. 010)) apply SAA method n dynamc 11

13 newsvendor problems. The argument above shows that the tradtonal analyses that show SAA leads to the optmal soluton s not applcable to our settng. Indeed, n our nner layer optmzaton, we face a newsvendor problem for whch the frm needs to balance holdng and shortage cost, and the knowledge about demand dstrbuton s crtcal. However, the lack of samples of random error ɛ t makes the nner loop optmzaton problem sgnfcantly dfferent from Lev et al. 007) and Lev et al. 010)), whch consder pure nventory control problems and samples of random errors are avalable for applcatons of SAA result and analyss. Because of ths, t s not guaranteed that the SAA method wll lead to a true optmal soluton. The DDA algorthm ntegrates a process of earnng explotaton) and learnng exploraton) n each stage. The earnng phase conssts of the frst I perods startng at t + 1, durng whch the algorthm mplements the optmal strategy for the proxy problem G DD p, y). In the next I perods of learnng phase that starts from t + I + 1, the algorthm uses a dfferent prce ˆp + δ and ts correspondng order-up-to level. The purpose of ths phase s to allow the frm to obtan demand data to estmate the rate of change of the demand wth respect to the sellng prce. Note that, even though the frm devates from the optmal strategy of the proxy problem n the second phase, the polces, ˆp + δ, ŷ, ) and ˆp, ŷ,1 ), wll be very close to each other as δ dmnshes to zero. We wll show that they both converge to the true optmal soluton and the loss of proft from ths devaton converges to zero. The prcng part of our algorthm s smlar to the pure prcng problem consdered by Besbes and Zeev 015) as we also use lnear approxmaton to estmate the demand-prce functon then maxmze the resultng proxy proft functon. Although our algorthm s heavly nfluenced by ther work, there s a key dfference. Besbes and Zeev 015) consder a revenue management problem and they only need to estmate the determnstc demand-prce functon, and the dstrbuton of random errors s mmateral n ther analyss. In our model, however, due to the holdng and backloggng costs, the dstrbuton of the random error s crtcal and that has to be learned durng the decson process, but t cannot be separated from the estmaton of demand-prce curve, as dscussed above. Therefore, due to the lack of unbased samples of random error and that the learnng of demandprce curve and the random error dstrbuton cannot be decoupld, we are not able to prove that the DDA algorthm converges to the true optmal soluton by usng the approaches developed n Besbes and Zeev 015) for the prcng problem and n Lev et al. 007) for the newsvendor problem. To overcome ths dffculty, we construct several ntermedate brdgng problems between the data-drven problem and the complete nformaton problem, and perform a seres of convergence analyses to establsh the man results. 1

14 Performance Metrcs. To measure the performance of a polcy, we use two metrcs proposed n Besbes and Zeev 015): consstency and regret. An admssble polcy π = p t, y t ), t 1) s sad to be consstent f p t, y t ) p, y ) n probablty as t. The average per-perod) regret of a polcy π, denoted by Rπ, T ), s defned as the average proft loss per perod, gven by [ Rπ, T ) = Gp, y ) 1 T ] T E Gp t, y t ). 9) Obvously, the faster the regret converges to 0 as T, the better the polcy. In the next secton, we wll show that the DDA polcy s consstent, and we wll also characterze the rate at whch the regret converges to zero. t=1 3 Man Results In ths secton, we analyze the performance of the DDA polcy proposed n the prevous secton. We wll show that under a farly general assumpton on the underlyng demand process, whch covers a number of well-known demand models ncludng logt and exponental demand functons, the regret of DDA polcy converges to 0 at rate OT 1/ ). We also present a numercal study to llustrate ts effectveness. Recall that the demand n perod t s D t p t ) = λp t ) ɛ t. As λp) s strctly decreasng, t has strctly decreasng nverse functon. Let λ 1 d) be the nverse functon of λp), whch s defned on d [d l, d h ] = [ λp h ), λp l ) ]. We make the followng assumpton. Assumpton 1. The functon λp) satsfes the followng condtons: ) The revenue functon d λ 1 d) s concave n d [ d l, d h]. ) 0 < λ p) λp) λ p)) < for p [ p l, p h]. The frst condton s a standard assumpton n the lterature on jont optmzaton of prcng and nventory control see e.g., Federgruen and Hechng 1999, and Chen and Smch-Lev 004), and t guarantees that the objectve functon n problem CI after mnmzng over y s unmodal n p. The second assumpton mposes some shape restrcton on the underlyng demand functon, and smlar assumpton has been made n Besbes and Zeev 015). Techncally, ths condton assures that the prces converge to a fxed pont through a contracton mappng. Some examples that satsfy both condtons of Assumpton 1 are gven below. Example 1. The followng functons satsfy Assumpton 1. 13

15 ) Exponental models: λp) = e k mp, m > 0. ) Logt models: λp) = a ek mp 1+e k mp for a > 0, m > 0, and k mp < 0 for p P. ) Iso-elastc constant elastcty) models: λp) = kp m for k > 0 and m > 1. We now present the man results of ths paper. Recall that p and y are the optmal prcng and nventory decsons for the case wth complete nformaton. Theorem 1 Polcy Convergence) Under Assumpton 1, the DDA polcy s consstent,.e., p t, y t ) p, y ) n probablty as t. Theorem 1 states that both prcng and orderng decsons from the DDA algorthm converge to the true optmal soluton p, y ) n probablty. Note that the convergence of nventory decson y t y s stronger than the convergence of order up-to levels ŷ,1 y and ŷ, y. Ths s because, the order up-to levels may or may not be achevable for each perod, thus the resultng nventory levels may overshoot the targetng order up-to levels. Theorem 1 shows that, despte these overshoots, the realzed nventory levels converge to the true optmal soluton n probablty. Convergence of nventory and prcng decsons alone does not guarantee the performance of DDA polcy s close to optmal. Our next result shows that DDA s asymptotcally optmal n terms of maxmzng the expected proft. Theorem Regret Convergence Rate) Under Assumpton 1, the DDA polcy s asymptotcally optmal. More specfcally, there exsts some constant K > 0 such that [ RDDA, T ) = Gp, y ) 1 T ] T E Gp t, y t ) KT 1. 10) t=1 Theorem shows that as the length of plannng horzon, T, grows, the regret of DDA polcy vanshes at the rate of O T 1/), hence DDA polcy s asymptotcally optmal as T goes to nfnty. Thus, even though the frm does not have pror knowledge about the demand process, the performance of the data-drven algorthm approaches the theoretcal maxmum as the plannng horzon becomes long. In Keskn and Zeev 014), the authors consder a parametrc data-drven prcng problem wth no nventory decson) where the demand error term s addtve and the average demand functon s lnear, and they prove that no learnng algorthm can acheve a convergence rate better than OT 1/ ). Our problem nvolves both prcng and nventory decsons, and the frm does not have pror knowledge about the parametrc form of the underlyng demand-prce functon 14

16 or the dstrbuton of the random error, and our algorthm acheves O T 1/), whch s the theoretcal lower bound. One nterestng mplcaton of ths fndng s that, lnear model n demand learnng acheves the best regret rate one can hope for, thus our result offers further evdence for the suffcency of Besbes and Zeev s lnear model. A numercal Study. We perform a numercal study on the performance of the DDA algorthm, and present our numercal results on the regret. We consder two demand curve envronments for λp): 1) exponental e k mp : k [k, k], m [m, m], where [k, k] = [0.1, 1.7], [m, m] = [0.3, ], ) logt e k mp 1+e k mp : k [k, k], m [m, m], where [k, k] = [ 0.3, 1], [m, m] = [,.5]. And we consder fve error dstrbutons for ɛ t : ) truncated normal on [0.5, 1.5] wth mean 1 and varance 0.1, ) truncated normal on [0.5, 1.5] wth mean 1 and varance 0.5, ) truncated normal on [0.5, 1.5] wth mean 1 and varance 0.35, v) truncated normal on [0.5, 1.5] wth mean 1 and varance 0.5, v) unform on [0.5, 1.5]. Here truncated normal on [a, b] wth mean µ and varance σ s defned as random varable X condtonng on X [a, b], where X s normally dstrbuted wth mean µ and varance σ. Followng Besbes and Zeev 015), for each combnaton of the above demand curve-error dstrbuton specfcatons, we randomly draw 500 nstances from the parameters k and m accordng to a unform dstrbuton on [k, k] and [m, m]. For each draw, we compute the percentage of proft loss per perod defned by Rπ, T ) Gp, y ) 100%. Then we compute the average proft loss per perod over the 500 draws and report them n Table 1. In all the experments, we set p l = 0.51, p h = 4, y l = 0, y h = 3, b = 1, h = 0.1, I 0 = 1, and ntal prce ˆp 1 = 1, ntal nventory order up-to level ŷ 11 = 1, ŷ 1 = 0.3. We test two values of ρ, ρ = 0.5 and ρ = 0.75, and two values of v, namely, v = 1.3 and v =. Table 1 summarzes the results when the underlyng demand curve s exponental, and Table dsplays the results when the underlyng demand curve s logt. Combnng both tables, one sees that when T = 100 perods, on average the proft loss from the DDA algorthm falls between 11% 15

17 Table 1: Exponental Demand Tme Perods T = 100 T = 500 T = 1000 T = 5000 T = ρ v = 1.3 v = v = 1.3 v = v = 1.3 v = v = 1.3 v = v = 1.3 v = Normal σ = Normal σ = Normal σ = Normal σ = Unform Maxmum Average and 14% compared to the optmal proft under complete nformaton, n whch DDA starts wth no pror knowledge about the underlyng demand. When T = 500, the proft loss s further reduced to between 5% and 8%. The performance gets better and better when T becomes larger. Also, t s seen from the table that the overall performance of algorthm s better when the varance of the demand s smaller, whch s ntutve. As mentoned earler, Theorems 1 and contnue to hold for the addtve demand model D t p t ) = λp t ) + ɛ t wth mnor modfcatons. Specfcally, we need to modfy Assumpton 1 to Assumpton 1A below. Assumpton 1A. The demand-prce functon λp) satsfy the followng condtons: ) p λp) s unmodal n p on p P. ) 1 < λ p) λp) < 1, for all p P. λ p)) Note that these are exactly the same assumptons made n Besbes and Zeev 015) for the revenue management problem, and examples that satsfy Assumpton 1A nclude a) lnear wth λp) = k mp, m > 0, b) exponental wth λp) = e k mp, m > 0, and c) logt wth λp) = e k mp 1+e k mp, m > 0, e k mp < 3 for all p P. The learnng algorthm for the addtve demand model s smlar to that of the multplcatve 16

18 Table : Logt Demand Tme Perods T = 100 T = 500 T = 1000 T = 5000 T = ρ v = 1.3 v = v = 1.3 v = v = 1.3 v = v = 1.3 v = v = 1.3 v = Normal σ = Normal σ = Normal σ = Normal σ = Unform Maxmum Average demand case, except that there s no need to transform t usng the logarthm of the determnstc porton of demand and the logarthm of random demand error. Instead, the algorthm drectly estmates λp) usng affne functon and computes the based samples of the random demand error n each teraton. 4 Sketches of the Proof In ths secton, we present the man deas and steps n provng the man results of ths paper. In the frst subsecton, we elaborate on the techncal ssues encountered n the proofs. The key deas of the proofs are dscussed n Subsecton 4., and the major steps for the proofs of Theorems 1 and are gven n Subsectons 4.3 and 4.4, respectvely. 4.1 Techncal ssues encountered To prove Theorem 1, we wll need to show E [ ˆp +1 p ) ] 0, E [ ˆp +1 + δ +1 p ) ] 0, as ; 11) E[y ŷ +1,1 ) ] 0, E[y ŷ +1, ) ] 0, as, 1) 17

19 where p s the optmal soluton of max p P where Jλp)) s defned as Qp, λp)) = max pe λp) E [ e ɛ] } Jλp)), p P Jλp)) = mn he [ y e λp) e ɛ] + [ + be e λp) e ɛ y ] } +. y Y However, both Q, ) and J ) are unknown to the frm because all the expectatons cannot be computed. To estmate J ), n 8) of the learnng algorthm we use the data-drven based estmaton of J+1 DD ˆα +1 ˆβ 1 +1 p) = mn y Y I and ˆp +1 s the optmal soluton of t +I max p P QDD +1p, ˆα +1 ˆβ +1 p) = max peˆα +1 ˆβ +1 p 1 p P I ) h y eˆα +1 ˆβ + ) )} +1 p e ηt + b eˆα +1 ˆβ + +1 p e ηt y, t +I } e ηt J+1 DD ˆα +1 ˆβ +1 p), n whch Q DD +1, ) s random and s constructed based on based random samples η t. To prove the convergence of the data-drven solutons to the true optmal soluton, we face two major challenges. The frst one s the comparson between J+1 DDˆα +1 ˆβ +1 p) and Jλp)) as functons of p. In J+1 DD, the true demand-prce functon s replaced by a lnear estmaton and, due to lack of knowledge about dstrbuton of random error, the expectaton s replaced by an arthmetc average from based samples η t not true samples of random error ɛ t. To put t dfferently, the objectve functon for J+1 DD s not a sample average approxmaton, but a based-sample average approxmaton. The second challenge les n the comparson of Q DD +1 p, ˆα +1 ˆβ +1 p) and Qp, λp)). Snce Q DD +1 s a functon of J DD +1 that s mnmum of a based-sample average approxmaton, the errors n replacng ɛ t by η t carry over to Q DD +1, makng t dffcult to compare ˆp +1, ŷ +1,1 ) and ˆp +1 + δ +1, ŷ +1, ) wth p, y ). To overcome the frst dffculty, we establsh several mportant propertes of the newsvendor problem and bound the errors of based samples Lemmas A, A3, A4, A8 n the Appendx). For the second, we dentfy hgh probablty events n whch unform convergence of the data-drven objectve functons can be obtaned Lemmas A1, A5, A6, and A7 n the Appendx). We note that n the revenue management problem settng, Besbes and Zeev 015) also prove the convergence result 11). In Besbes and Zeev 015), p s the optmal soluton of max p P Qp, λp)), and ˆp +1 s the optmal soluton of max p P Qp, ˆα +1 ˆβ +1 p), where Q, ) s a known and determnstc functon Qp, λp)) = pλp). As Besbes and Zeev 015) pont out, 18

20 ther analyss extends to more general functon Qp, λp)) n whch Q, ) s a known determnstc functon. Ths, however, s not true n our settng as Q, ) s not known, and as a matter of fact, one cannot even fnd an unbased sample average to estmate Q, ). Therefore, the challenges dscussed above were not present n Besbes and Zeev 015). 4. Man deas of the proof To compare the polcy and the resultant proft of DDA algorthm wth that of the optmal soluton, we frst note that these two problems dffer along several dmensons. For example, n DDA we approxmate λp) by an affne functon and estmate the parameters of the affne functon n each teraton, and we approxmate the expected revenue and the expected holdng and shortage costs usng based sample averages. These dfferences make the drect comparson of the two problems dffcult. Therefore, we ntroduce several ntermedate brdgng problems, and n each step we compare two adjacent problems that dffer just n one dmenson. For convenence, we follow Besbes and Zeev 015) to ntroduce notaton We proceed to prove 11) as follows: ᾰz) = λz) λ z)z, βz) = λ z), z P. 13) E [ p ˆp +1 ) ] E[ p p ᾰˆp ), βˆp ) ) } } Comparson of problems CI and B1 Lemma A1 + p ᾰˆp ), βˆp ) ) p +1 ᾰˆp ), βˆp )) + } } Comparson of problems B1 and B Lemma A5 p +1 ᾰˆp ), βˆp ) ) ˆp +1 } } Comparson of problems B and DD Lemma A6 and Lemma A7 where the two new prces p, ) and p +1, ) are the optmal solutons of two brdgng problems. Specfcally, we let p α, β ) denote the optmal soluton for the frst brdgng problem B1 defned by Brdgng Problem B1: max p P pe α βp E [ e ɛ] mn y Y 14) ) ], he [y e α βp e ɛ] + [ } + + be e α βp e ɛ y] }, 15) whle p +1 α, β) denotes the optmal soluton for the second brdgng problem B defned by Brdgng Problem B: max pe α βp 1 p P I t +I e ɛt ) mn y Y 1 I t +I 19 h y e α βp e ɛt) + + b e α βp e ɛt y ) + )}}. 16)

21 Moreover, for gven p P, we let ye α βp ) denote the optmal order-up-to level for problem B1, and ỹ +1 e α βp ) denote the optmal order-up-to level for problem B. By Lemma A n the Appendx, the objectve functons for problems B1 and B are unmodal n p after mnmzng over y Y when β > 0. Comparng 15) wth 4), t s seen that problem B1 smplfes problem CI by replacng the demand-prce functon λp) by a lnear functon α βp, whle problem B s obtaned from problem B1 after replacng the mathematcal expectatons n problem B1 by ther sample averages,.e., problem B s the SAA of problem B1. Comparng 16) wth 8), t s noted that problems B and DD dffer n the coeffcents of the lnear functon as well as the arthmetc averages. More specfcally, n B the real random error samples ɛ t, t = t + 1,..., t + I, are used, whle n problem DD, based error samples η t are used n place of ɛ t, t = t + 1,..., t + I. Furthermore, note that the optmal prces for problems CI and B1, p and p ᾰˆp ), βˆp ) ), are determnstc, but the optmal solutons of problems B and DD, p +1 ᾰˆp ), βˆp ) ) and ˆp +1, are random. Specfcally, p +1 ᾰˆp ), βˆp ) ) s random because ɛ t s random, whle ˆp +1 s random due to demand uncertanty from perods 1 to t +1. Hence, to show the rght hand sde of 14) converges to 0, we wll frst develop an upper bound for p p ᾰˆp ), βˆp ) ) by comparng problems CI and B1, and the result s presented n Lemma A1. Snce p +1 ᾰˆp ), βˆp ) s random, we compare the two problems B1 and B and show the probablty that p ᾰˆp ), βˆp ) ) p +1 ᾰˆp ), βˆp ) ) exceeds some small number dmnshes to 0 n Lemma A5. Smlarly, n Lemma A6 and Lemma A7 we compare problems B and DD and show the probablty that p +1 ᾰˆp ), βˆp ) ) ˆp +1 exceeds some small number also dmnshes to 0. Fnally, we combne these several results to complete the proof of 11). The dea for provng 1) s smlar, and that also reles heavly on the two brdgng problems Lemmas A6, A7, and A8). The detaled proofs for Theorem 1 and Theorem are gven n Subsectons 4.3 and 4.4. In the subsequent analyss, we assume that the space for feasble prce, P, and the space for order-up-to level, Y, are large enough so that the optmal solutons p and optmal ye λp) ) over R + for gven p P for problem CI fall nto P and Y, respectvely; and for gven q P, the optmal solutons p ᾰq), βq) ) and y eᾰq) βq)p ) for gven p P over R + for problem B1 fall nto P and Y, respectvely. Note that both problem CI and problem B1 depend only on prmtve data and do not depend on random samples, hence these are mld assumptons. We remark that our results and analyses contnue to hold even f these assumptons are not satsfed as long as we modfy Assumpton 1) to p ᾰz), βz) ) / z < 1 for z P. Ths condton reduces to Assumpton 1) f the optmal solutons for problem CI and problem B1 satsfy the feasblty condtons descrbed above. We end ths subsecton by lstng some regularty condtons needed to prove the man theoretcal 0

22 results. Regularty Condtons: ) ye λq) ) and y eᾰq) βq)p ) are Lpschtz contnuous on q for gven p P,.e., there exsts some constant K 1 > 0 such that for any q 1, q P, ye λq1) ) ye λq) ) K 1 q 1 q, 17) y eᾰq 1) βq 1 )p ) y eᾰq ) βq )p ) K1 q 1 q. 18) ) Gp, ȳe λp) )) has bounded second order dervatve wth respect to p P. ) E[D t p)] > 0 for any prce p P. v) λp) s twce dfferentable wth bounded frst and second order dervatves on p P. v) The probablty densty functon f ) of ɛ t satsfes mnfx), x [l, u]} > 0. It can be seen that all the functons n Example 1 satsfy the regularty condtons above wth approprate choces of p l and p h. 4.3 Proof of Theorem 1 The proofs for the convergence results are techncal and rely on several lemmas that are provded n the Appendx. In ths subsecton, we outlne the man steps n establshng the frst man result, Theorem 1. Convergence of prcng decsons. To prove the convergence of prcng decsons, we contnue the development n 14) as follows: E [ p ˆp +1 ) ] [ p E p ᾰˆp ), βˆp ) ) p + ᾰˆp ), βˆp ) ) p +1 ᾰˆp ), βˆp ) ) + p+1 ᾰˆp ), βˆp ) ) ) ] ˆp +1 [ E γ p ˆp + p ᾰˆp ), βˆp ) ) p +1 ᾰˆp ), βˆp ) ) + p+1 ᾰˆp ), βˆp ) ) ) ] ˆp +1 ) 1 + γ E [ p ˆp ) ] [ p +K E ᾰˆp ), βˆp ) ) p +1 ᾰˆp ), βˆp ) ) + p+1 ᾰˆp ), βˆp ) ) ) ] ˆp +1 ) 1 + γ E [ p ˆp ) ] +K 3 E [ p ᾰˆp ), βˆp ) ) p +1 ᾰˆp ), βˆp ) ) ] + K 3 E [ p+1 ᾰˆp ), βˆp ) ) ˆp +1 ], 19) 1

23 where the frst nequalty follows from the expanson n 14), the second nequalty follows from Lemma A1, and the thrd nequalty s justfed by γ < 1 n Lemma A1 and some constant K, and the last nequalty holds for some approprately chosen K 3 because of the nequalty a + b) a + b ) for any real numbers a and b. To bound E [ p ᾰˆp ), βˆp ) ) p +1 ᾰˆp ), βˆp ) ) ] n 19), by Lemma A5 one has, for some constant K 4, E [ p ᾰˆp ), βˆp ) ) p +1 ᾰˆp ), βˆp ) ) ] K e 4I ξ dξ = 5π 1 K4. 0) 4I 1 And to bound E [ p+1 ᾰˆp ), βˆp ) ) ˆp +1 ] n 19), by Lemma A6 and Lemma A7, when s large enough greater than or equal to defned n the proof of Lemma A7), for some postve constants K 5, K 6, and K 7 one has [ E p+1 ᾰˆp ), βˆp ) ) ] ˆp +1 [ E ᾰˆp ) ˆα +1 + βˆp ) ˆβ +1 + ᾰˆp + δ ) ˆα +1 + βˆp + δ ) ˆβ ) ] +1 K I p h p l) E [K 6 ᾰˆp ) ˆα +1 + βˆp ) ˆβ +1 + ᾰˆp + δ ) ˆα +1 + βˆp + δ ) ˆβ +1 )] + 8 I p h p l) K 7 I 1. 1) Substtutng 0) and 1) nto 19), one has E [ p ˆp +1 ) ] ) 1 + γ E [ p ˆp ) ] + K 8 I 1. Lettng 1+γ = θ, we further obtan E [ ˆp +1 p ) ] 1 θ ˆp 1 p ) + K 8 θ j I 1 j K 1 9v 1 ) θ j v 1 ) j. ) We choose v > 1 that satsfes θv 1 < 1, then there exsts a postve constant K 10 such that 1 j=0 θj v 1 ) j K 10, therefore, for some constants K 11 and K 1, j=0 E [ ˆp +1 p ) ] K 11 v 1 ) K 1 I 1. 3) Moreover, we have, for some postve constant K 13, E [ ˆp +1 + δ +1 p ) ] E [ ˆp +1 p ) ] + δ +1 K 13 I 1 0, as. 4) j=0

24 Ths completes the proof of 11). Because mean-square convergence mples convergence n probablty, ths shows that the prcng decsons from DDA converge to p n probablty. Convergence of nventory decsons. To prove y t converges to y n probablty, we frst prove the convergence of order up-to levels 1). For some constant K 14, we have [ E y ] ŷ +1,1 y E[ e λp ) ) ye λˆp +1) ) + y e λˆp +1) ) y eᾰˆp +1) βˆp +1 )ˆp +1 ) + y ) eᾰˆp +1 βˆp +1 )ˆp +1 ) y eᾰˆp ) βˆp ) )ˆp +1 + y eᾰˆp ) βˆp ) )ˆp +1 ỹ+1 e ᾰˆp ) βˆp ) )ˆp +1 ỹ+1 + e ᾰˆp ) βˆp ) ) )ˆp +1 ] ŷ+1,1 y K 14 E[ e λp ) ) ye λˆp +1) ) + y e λˆp +1) ) y eᾰˆp +1) βˆp +1 )ˆp +1 ) 5) } } } } Dfference between p and ˆp +1 Zero + y eᾰˆp +1) βˆp ) +1 )ˆp +1 y e ᾰˆp ) βˆp ) )ˆp +1 } } Dfference between ˆp +1 and ˆp + y eᾰˆp ) βˆp ) )ˆp +1 ỹ+1 e ᾰˆp ) βˆp ) )ˆp +1 + ỹ +1 e ᾰˆp ) βˆp ) )ˆp +1 )] ŷ+1,1. } } } } Comparson of problems B1 and B Lemma A8 Comparson of problems B and DD Lemma A6 and Lemma A7 We want to upper bound each term on the rght hand sde of 5). Frst, t follows from 17) that, for some constant K 15 t holds E[ y e λp ) ) y e λˆp +1) ) ] K 15 E [ p ˆp +1 ]. By defnton of ᾰp) and βp) n 13) one has ᾰˆp +1 ) βˆp +1 )ˆp +1 = λˆp +1 ), thus the second term on the rght hand sde of 5) vanshes. For the thrd term, we apply the Lpschtz condton on y eᾰq) βq)p ) n 18) to obtan, for some constants K 16 and K 17, [ y E e ᾰˆp +1) βˆp ) +1)ˆp +1 y eᾰˆp ) βˆp )ˆp ) +1 ] K [ 16E ˆp +1 ˆp ] By Lemma A8, we have, for some constants K 18 and K 19, E [ y e ᾰˆp ) βˆp )ˆp +1 ) ỹ +1 e ᾰˆp ) βˆp )ˆp +1 ) ] K 18 K 17 E [ p ˆp + p ˆp +1 )]. + 0 e 4I ξ dξ K 19 I, 6) 3