Heavy Traffic Analysis of a Simple Closed Loop Supply Chain

Transcription

1 Heavy Traffic Aalysis of a Simple Closed Loop Supply Chai Arka Ghosh, Sarah M. Rya, Lizhi Wag, ad Aada Weerasighe April 8, 2 Abstract We cosider a closed loop supply chai where ew products are produced to order ad retured products are refurbished for resellig. The solutio to a price-settig problem eforces the heavy traffic coditio, uder which we address the productio rate cotrol problem for two types of cost fuctios. We solve a drift-cotrol problem for a approximate system drive by a correlated two-dimesioal Browia motio. The solutios to this system are the used to obtai asymptotically optimal cotrol policies. We also coduct a umerical study to explore the effects of differet parameters o the optimal productio rates ad the resultig costs. MSC2 Subject Classificatio: Primary: 6K25, 6K2, 93E2; Secodary: 6K, 6K3. OR/MS subject classificatio: Primary: Queues - Diffusio Models, Queues - Limit Theorems, Queues - Networks, Queues - Optimizatio; Secodary: Ivetory/Productio - Policies - Marketig/pricig. Ruig Title: Heavy Traffic Aalysis of Closed Loop Supply Chai Departmet of Statistics, Iowa State Uiversity, Ames, IA 5 Departmet of Idustrial ad Maufacturig Systems Egieerig, Iowa State Uiversity, Ames, IA 5 Correspodig author. [email protected], Phoe: (55) Departmet of Idustrial ad Maufacturig Systems Egieerig, Iowa State Uiversity, Ames, IA 5 Departmet of Mathematics, Iowa State Uiversity, Ames, IA 5

2 Itroductio We cosider a queueig etwork model of a sigle firm that ca cotrol its productio rate of ew products but ot their price i a competitive market. It produces ew products to order. It allows customers to retur some products after sale ad refurbishes the returs for resale at a price that it chooses to balace the demads for ew ad refurbished products. The refurbished products are held i ivetory. We assume that a customer is willig to wait while a ew product is produced to her specificatios, but potetial buyers of refurbished products are impatiet. While stylized, the model captures essetial elemets of a firm like Dell, Ic., which assembles ew products to order, offers a geerous retur policy, ad sells its stock of refurbished products i a olie store. The relevat costs are associated with keepig customers waitig for ew products, maitaiig capacity to maufacture at a give rate, ad losig potetial sales of refurbished products. We derive a asymptotically optimal policy, which cosists of the productio rate for ew products ad the relative price of refurbished products, for this closed loop supply chai i heavy traffic. Closed loop supply chais that ecompass productio, distributio, product returs, reprocessig ad resale have gaied icreasig attetio recetly for both evirometal ad ecoomic reasos. Reprocessig typically retais some of the value added by the origial maufacturig process while prevetig potetially harmful disposal ad coservig both material ad eergy. To the origial producer or a third party, reprocessig ad resellig products ca yield profits by reducig the cost of providig a fuctioal product ad expadig the market. The status of havig bee sold ad retured may reduce the attractiveess of reprocessed products, yet a discouted price ca create a lower-ed market segmet of cosumers who are ot willig to pay the full price for a ew product but will accept a reprocessed oe for a reduced price. This price should be low eough to make reprocessed products attractive compared with ew oes ad prevet their ivetory from accumulatig. O the other had, too low a price for refurbished products could caibalize the demad ad profits eared by ew products. Optimal pricig strategies for remaufactured goods have bee aalyzed i differet cotexts [8,,, 26]. Collectig or receivig ad the refurbishig ad resellig products itroduces ucertaities i additio to those already preset i maufacturig ad sellig ew products. The availability of previously distributed products for refurbishmet is subject to purchasers decisios o whether ad whe to retur them. Variabil- 2

3 ity of the demad ad product flows ca create cogestio or shortages that reduce the efficiecy ad ecoomic viability of the closed loop supply chai. Queueig models have bee employed i a umber of studies to aalyze the effectiveess of closed loop supply chai maagemet policies uder steady state coditios [4, 22, 3, 32], which imply o-egligible idle times i the service facilities. However, may maagers recogize that idleess may reduce profit ad prefer to utilize expesive processig resources as fully as possible by settig prices to icrease demad. Such high utilizatio correspods to heavy traffic i the queueig model. I recet years, several authors (e.g., [, 2, 3, 6, 7, 2, 27, 29]) have employed heavy traffic approximatios of various physical queueig etworks ad used techiques from stochastic cotrol theory to obtai good queueig cotrol policies. We examie the two decisio variables, price ad productio rate, sequetially. First, we formulate a price-settig problem (also kow as the static plaig problem) to maximize profit i the fluid-scaled system. The optimal solutio aturally imposes the heavy traffic coditios (see [29] for a similar aalysis). The heavy traffic coditios require that the arrival rates ad the service rates of the queueig system are balaced i some sese. We show that a solutio to a profit maximizatio problem for the fluid scaled queueig system (the so-called static plaig problem) aturally imposes the heavy traffic assumptio o our model. Ituitively, this ca be iterpreted as follows: if the maufacturer decides to maximize profit based o the average behavior of the system, the optimal prices will eforce that the arrivals (fuctios of the price-variable) match the services ad, hece, satisfy the heavy traffic coditios. Secod, uder the heavy traffic coditios, we solve the problem of fidig a optimal productio rate to miimize a appropriate cost fuctio. Such heavy traffic aalyses ofte follow a sequece of steps outlied by Harriso [7] (see also [5]), which ivolves solvig a diffusio cotrol problem (called the Browia Cotrol Problem or the BCP) that approximates the queueig cotrol problem ad the iterpretig its solutio to obtai meaigful cotrol policies for the origial queueig cotrol problem. I this paper, we cosider two commo forms of cost fuctioals: log-ru average (ergodic) cost ad the ifiite horizo discouted cost. For each of these cost fuctios, we carry out the aalysis followig Harriso s scheme: we first formulate ad solve the BCP; the we propose a cadidate for optimal cotrol policy for the queueig model by iterpretig the solutio of the BCP; ad fially, we prove the asymptotic optimality of the proposed policy usig weak covergece methods. We also discuss 3

4 some comparative statics ad carry out a umerical study to explore the effect of system parameters o the optimal productio rates ad resultig costs. The mai cotributios of this paper are the followig: This is the first paper to our kowledge that successfully applies heavy traffic machiery to optimize performace of a closed loop supply chai. A atural price settig problem is show to eforce heavy traffic coditios i such a supply chai. This paper also provides complete heavy traffic aalysis to obtai optimal productio rates uder the two most commo cost fuctios i the cotrol literature. Despite the existece of a large literature for heavy traffic aalysis of queueig etworks, most articles with such provably optimal solutios focus o oe-dimesioal problems. There are very few such complete aalyses for two-dimesioal models prior to this oe (see [3, 7]). This article provides oe such aalysis for a two-dimesioal model where the associated diffusio model is drive by a two-dimesioal correlated Browia motios. Havig solved the diffusio cotrol problem, we establish the mai asymptotic optimality results usig properties of a appropriate Skorohod map (regulator map) ad weak covergece techiques. The rest of the paper is orgaized as follows: I Sectio 2, we describe the model ad the cotrol problems i detail. Next, we discuss the static plaig problem ad the heavy traffic coditios for our queueig etwork. Our mai theorems (Theorems 2.7 ad 2.8) describig asymptotically optimal policies are also stated i this sectio. I Sectio 3, we address the two BCPs for two differet choices of the cost fuctio. Sectio 4 cotais weak covergece aalysis to prove the mai results. Sectio 5 cotais some comparative statics ad umerical aalysis of the two cost problems. Fially i Sectio 6, we summarize the paper, provide a compariso of our results with the steady-state aalysis of similar models uder the average cost fuctioal ad coclude with possible extesios to this work. A Appedix cotais proofs of some of the more stadard results that are used i our aalysis. 2 Problem Descriptio We study a simple model of a closed loop supply chai i which a producer maufactures ew products to order. Some ew products are retured by the customers after evaluatio. We assume that ay ew product may be retured after sale with probability β (, ). These retured 4

5 products ca o loger be sold as ew. Istead, they are ispected, refurbished ad placed ito ivetory to be resold (see Figure ). As i Vorasaya ad Rya [33], we assume the producer is a price-taker i the market for ew products, whose exogeously-determied price is p N, ormalized so that < p N <. It sets the price for refurbished products, p R, such that p R < p N. Cosumer (ormalized) valuatio of ew products, deoted as p, is uiformly distributed o (, ). A cosumer who is willig to pay a price p for a ew product is willig to pay at most δp for a refurbished product, where < δ <. Give the prices, the cosumer chooses betwee ew ad refurbished products to maximize his/her surplus: max{p p N, δp p R, }. If p R δp N the δp p R < p p N for ay p < ; therefore, we assume p R < δp N to guaratee some demad for refurbished products. Likewise, we assume p R p N ( δ) because otherwise, p p N < δp p R for ay p < ad o demad would exist for ew products. A strategic decisio variable for the producer is ρ p R p N, (2.) such that ρ ( δ p N, δ). I terms of this price ratio, the ormalized demad rate for ew products represets the proportio of a fixed umber of customers per uit time who will buy the ew product, i.e., those for whom p > p N ad p p N > δp ρp N, ad is give by: λ N (ρ) = p N( ρ). (2.2) δ The correspodig demad rate for refurbished products is λ R (ρ) = p ( N ρ ), (2.3) δ δ which represets the proportio of customers for whom δp > p R ad p p N < δp ρp N. I our model, the demads for ew ad refurbished products follow Poisso processes with the rates λ N (ρ) ad λ R (ρ) for a chose value of ρ. These ad other parameters are costat over a implicit study horizo represeted by the model, which is reasoable for a product category such as busiess laptop, but ot iteded for specific models withi that category. We assume that the time required to produce a ew product is expoetially distributed with rate µ > ad that 5

6 the maufacturig server is ot allowed to idle uless the queue of ew product orders is empty. Whe a demad for a refurbished item arrives, if such a product is available i ivetory the the demad is satisfied; otherwise, the customer is lost. Let X (t) deote the legth of the ew product customer queue ad X 2 (t) deote the umber of refurbished products i ivetory at time t. The, give X i () = x i, i =, 2, we model X ad X 2 as: ( t ) X (t) = x + N (λ N (ρ)t) N 2 µ {X (s)>}ds, (2.4) ( t )] ( t ) X 2 (t) = x 2 + Φ [N 2 µ {X (s)>}ds N 3 λ R (ρ) {X2 (s)>}ds, (2.5) where N i ( ), i =, 2, 3, are idepedet uit Poisso processes. For ay oegative iteger m, Φ(m) = m k= φ k, where {φ k } is a sequece of i.i.d Beroulli(β) radom variables. Here ad for the rest of the paper, A, will deote the idicator fuctio of a Borel set A (i.e. A (x) = if x A ad A (x) =, if x / A). I the above display, Φ(m) represets the (radom) umber of products that are retured by customers out of the first m purchased products. See Chapter 6 of [24] to see a more geeral costructio of jump-markov process, with state space Z, as a Figure : Closed loop supply chai etwork liear combiatio of time chaged versios of uit Poisso process as i (2.4) ad (2.5). We assume that all retured products are refurbished ad that retur, if it occurs, ad refurbishmet 6

7 are both istataeous. This assumptio approximates the situatio where returs are maily due to buyer remorse or umet expectatios rather tha ay real defect, so that refurbishmet amouts to ispectio or testig ad repackagig. I Sectio 6 we describe extesios to icorporate expoetially-distributed delays i retur ad/or refurbishmet as well as disposal of some fractio of returs. Defie processes L ad L 2 as follows: t L (t) = µ t {X (s)=}ds, L 2 (t) = λ R (ρ) {X2 (s)=}ds. (2.6) The parameter µ represets the average umber of ew products that ca be maufactured per uit time. The process L ( ) is defied as µ multiplied by the time that the maufacturig server has idled so far. I that sese, L (t) represets the average umber of ew-product customers that could have bee served i the iterval [, t] durig the server s idle time whe o ew-product customers are waitig. Usig a similar iterpretatio, L 2 (t) captures the average umber of lost sales of refurbished products. We assume that the cost for storig refurbished products is maily fixed with respect to quatity, ad therefore little affected by policies that ifluece X 2 (t). Ideally, we prefer policies which produce fewer lost sales of refurbished goods. This preferece is reflected i the defiitios of the cost fuctioals i (2.2)-(2.3) below (see Sectio 6 for possible extesios of the model). Our goal i this paper is to optimize () the price of refurbished products relative to ew products ad (2) the productio rate of ew products. We carry out this optimizatio i two steps. First, we solve a static plaig problem i terms of the fluid-scaled processes ad the log term average demad rates, which are assumed to satisfy (2.2) ad (2.3). The profit is maximized by settig the price ratio ad log term average productio rate so the system is i heavy traffic. Secod, we carry out a heavy traffic aalysis of the system, ad fid a asymptotically optimal service rate uder optimal prices. As it is commoly doe for such aalysis, we will cosider a sequece of etworks (idexed by a parameter ), each havig the same structure, but the parameters of the -th etwork deped o the idex, ad we will require that as, the system achieves heavy traffic (see Assumptio 2. ad 2.4 below). A physical etwork that is close to beig i heavy-traffic ca be thought of as oe elemet of this sequece with a large value of. Hece, from ow o, we will cosider a 7

8 sequece of etworks idexed by ad all the processes ad parameters deped o (deoted by a superscript, e.g., λ N (ρ), X (t), etc.). We assume that ρ does ot deped o (i.e., ρ ρ), ad its optimal value will be determied by the limit behavior of the system (i the static plaig problem). Note that, sice this queueig model is a Jackso etwork, the queue legths of each etwork i the sequece ca be aalyzed exactly i steady-state. I Sectio 5, we illustrate how the result of such a prelimit aalysis coicides with the asymptotic aalysis for a special case of the log-ru average cost fuctio. However, the discouted cost depeds o the trasiet behavior of the queue legths, ad we lack ay exact characterizatio of the arrival process to the secod queue durig the trasiet phase. A policy cosists of the price ratio ρ (, ) as defied i (2.) ad the maufacturig rate sequece {µ }. We assume the followig basic covergece properties for the parameters of this model: Assumptio 2. There exist θ i IR, i =, 2, 3, { λ N (ρ) >, ρ [, ]}, { λ R (ρ) >, ρ [, ]}, µ >, ad x, x 2 such that (i) (ii) (λ N (ρ) λ N (ρ)) θ 2, (λ R (ρ) λ R (ρ)) θ 3 for all ρ [, ], (2.7) (µ µ) θ, (2.8) (iii) βθ 2 = θ 3 ad ˆx i = x i / x i as, i =, 2. (2.9) Remark 2.2 The assumptios i (2.7) state that there are log-ru average rates (for arrivals) to which the parameters of the -th system coverge. They also specify that this covergece takes place at the rate of θ i, where θ i, i = 2, 3, are the covergece rates, which is the atural rate of covergece for heavy traffic assumptio of diffusio scaled systems. It has bee show (for cotrol problems ivolvig liear holdig costs) that for ay admissible policy {µ } that produces fiite asymptotic costs (see (2.2) ad (2.3) below), (2.8) holds (see [34]). So i our aalysis, we restrict cosideratio to admissible cotrols that satisfy (2.8). The first part of (2.9) is a techical assumptio that reduces the problem dimesio: because of this, the limitig diffusio cotrol problem is effectively oe-dimesioal. To be more specific, the terms u ad ũ i (4.53)- (4.54) coverge to the same costat u (usig (4.5)) which is the drift parameter goverig both the 8

9 processes i the limitig model (3.9). Existece of such asymptotic limits is a stadard assumptio i heavy traffic aalysis. We will carry out the asymptotic aalysis of the diffusio scaled queueig model. Therefore, we eed to defie the diffusio scale before itroducig the cost fuctioal. The aalysis also ivolves the so-called fluid-scaled processes. For ay process ψ ( ) described here, ψ ( ) ad ˆψ ( ) will deote the fluid- ad diffusio-scaled processes respectively, give by: ψ (t) = ψ (t), ˆψ (t) = ψ (t), for all t, =, 2,.... (2.) I this paper, we aalyze two types of cost fuctioals: the log-ru average cost (also kow as the ergodic cost ) ad the ifiite horizo discouted cost, each of which ivolves the followig compoets: a cotrol cost for the service rate, a backorder cost for ew products, ad a liear cost per lost customer of refurbished products. Here we assume that the ivetory of the refurbished products does ot icur ay variable cost for the maufacturer; hece, the cost fuctioals iclude o holdig cost for the refurbished products. These compoets of costs are give i terms of fuctios c( ), h( ) ad a costat pealty rate k, which satisfy the followig assumptios. Assumptio 2.3 The fuctios c( ) ad h( ) are oegative, cotiuous, odecreasig ad covex o [, ) ad k is a positive costat. Also, c(x) =, for x ad there exist K ad N > such that h( ) satisfies h(x) K( + x N ) for all x. Let u (ρ) = (µ λ R(ρ)/β), ρ [, ]. (2.) Uder the heavy traffic coditios described i Sectio 2., the quatity (βµ λ R (ρ)) ca be thought of as the et ivetory growth rate for the refurbished products i the -th system. This quatity teds to zero at the rate as approaches ifiity, as show i (4.5)) i Sectio 4. Here, u defied i (2.) captures this rate. As argued i Remark 3., the limit of u is assumed to be o-egative. Hece, for simplicity, we oly allow o-egative values for u (ρ) for all 9

10 ad ρ [, ]. The log-ru average cost is give by : Î (x, x 2, ρ, {µ }). = lim if lim sup = lim if T lim sup T T [( ) ] c (u (ρ)) + h( ˆX (s)) ds + k dˆl 2 (s) T E [c (u (ρ)) + T T E h( ˆX (s))ds + k ] T E ˆL 2 (T ).(2.2) For a fixed discout rate α >, the ifiite horizo discouted cost is give by: Ĵ (x, x 2, ρ, {µ }). = lim if E = lim if c (u (ρ)) α ( ) ) ] 2 e [(c αs (u (ρ)) + ˆX (s) ds + k dˆl 2 (s) + E [ ( ] 2 e αs ˆX (s)) ds + k dˆl 2 (s). (2.3) I this paper we solve the ifiite horizo discouted cost problem with backorder cost h(x) = x 2. The term c(u (ρ)) i (2.2) ad (2.3) represets the cost of choosig productio rate µ relative to the arrival rate for refurbished products (suitably scaled), while the term h( ˆX (s)) is the cost per backorder (of ew items) per uit time. The ifiitesimal quatity, k dˆl 2 (s), is the pealty for lost sales of refurbished items. Here x ad x 2 are the (asymptotic) iitial legths of the backorders of ew items ad ivetory of refurbished items as defied i (2.9). Note that the choice of u uiquely determies the service rate sequece µ. Sice c(x) = for all x, for each the cotrol cost ca be thought of as c(µ ) =. c(u (ρ)) = c( (µ λ R (ρ)/β)), which is a icreasig fuctio of µ, for each fixed λ R (ρ). Prior to aalysis, for both cotrol problems with costs (2.2) ad (2.3), it is ot clear which amog the three compoets of the cost is domiat. Eve if the cost fuctios ivolve oly diffusio-scaled processes, to be able to carry out the aalysis oe eeds to have the fluid system stable. Hece, we defie the static plaig problem below, ad deduce the coditios for heavy traffic. The process of solvig the static plaig problem also solves the problem of price settig (i.e., choosig ρ). 2. Static Plaig Problem Static plaig problems are formulated by costructig a system where the fluid-scaled processes are replaced by their log-ru averages (or fluid limits) ad solvig a suitable optimizatio problem ivolvig those averages (see [9, 25, 28, 29]). I the fluid limit, we formulate a determiistic

11 problem to choose ρ ad µ that maximize the profit rate subject to stability coditios o both queues. The profit cosists of reveue from the sale of both ew ad refurbished returs less the cost per uit time associated with producig ew products at rate µ. Let γ( ) be a odecreasig fuctio. The profit maximizatio problem is: max ρ, µ p N ( β) λ N (ρ) + p R β λ N (ρ) γ( µ) s.t. λn (ρ) µ β λ N (ρ) λ R (ρ). The first term i the objective fuctio is reveue per uit time from the sale of ew products (less a refud for retured products), whose sales are limited by demad. The secod term is reveue per uit time from the sale of refurbished products, whose sales are limited by supply. The first costrait esures that supplies of ew products are sufficiet to meet the demad o average. The secod costrait restricts the supply of refurbished products to ot exceed their demad; otherwise, ivetories of refurbished products would accumulate without boud. The objective is separable ito its reveue ad cost compoets, where reveue depeds oly o ρ ad cost depeds oly o µ. Clearly, the optimal limitig productio rate equals its lower boud: µ = λ N (ρ ), where ρ is the optimal price ratio. The reveue rate is proportioal to: ( β + βρ) λ N (ρ) = ( β + βρ)( p N δ + p N δ ρ), which is a covex quadratic fuctio of ρ that is miimized by ρ = p N( 2β) β( δ) 2βp N. The largest feasible value for ρ is foud uiquely by solvig the secod costrait as a equality: ρ = δ [p N( + β) β( δ)]. p N ( + δβ) It is easy to verify that ρ ρ for ay p N > if β < 2 (i fact, ρ < i this case). Therefore, reveue is a icreasig fuctio of feasible ρ, so that ρ = ρ. The uique solutio to the static

12 plaig problem is give by (ρ, µ ) such that λ N (ρ ) = β λ R (ρ ) ad µ = λ N (ρ ). This relatio costitutes the heavy traffic coditio, ad for the rest of the paper, we will assume that all admissible policies satisfy this coditio, i additio to Assumptio 2.. See Figure 2. Figure 2: Price determiatio from static plaig problem. Assumptio 2.4 (Heavy Traffic) Ay admissible policy satisfies µ = λ N (ρ ) = β λ R (ρ ). There are differet equivalet methods of arrivig at the above described heavy traffic assumptio eeded for aalyzig diffusio-scaled systems (see [5, 7]), ad it ca be verified that those methods also yield the same heavy traffic coditio as we have here. For example, oe covetioal way for defiig heavy traffic (see [8]) is to require that the followig holds: There exists a uique optimal solutio ( r, x ) satisfyig r = ad A x = to the followig liear program, miimize r subject to R x = α, A x r ad x. Here the decisio variables x represet average rates at which activities are udertake ad the objective is a vector of upper bouds o the utilizatio rates for processig resources; the costats R, A ad α are related to the parameters of the etwork: The average rates of arrival to the two servers from outside the system are give by α (ote that i our formulatio of the ivetory process 2

13 dyamics, λ R serves as the service rate ad ot a exteral arrival rate), while the iput-output matrix R ad capacity-cosumptio matrix A are defied as α = λ N (ρ), R = µ β µ λ R (ρ), A =. See [8] for more details o this formulatio of heavy traffic. It is easy to verify that the (ρ, µ ) satisfyig the coditios i Assumptio 2.4 also satisfies the heavy traffic coditio as defied i [8]. Remark 2.5 The heavy traffic coditios alog with (2.2) ad (2.3) applied to the fluid limit imply that µ = λ N (ρ ) = p N + βδ = β λ R (ρ ). The assumptio δ p N < ρ = ρ < δ holds true for ay p N <. Defiitio 2.6 (Two queueig cotrol problems) Uder Assumptios 2. ad 2.4, the price variable is set as ρ, which determies the demad rates of ew ad refurbished products, as well as the log-ru average service rate µ. A sequece of service rates {µ } is said to be admissible if it satisfies Assumptios 2. ad 2.4 with ρ ad µ. The first queueig cotrol problem is to fid a asymptotically optimal service rate sequece {µ } that miimizes Î(x, x 2, {µ }). = Î(x, x 2, ρ, {µ }) (2.4) over all admissible cotrols {µ }. The secod queueig cotrol problem is to fid a asymptotically optimal service rate sequece {µ } that miimizes Ĵ(x, x 2, {µ }). = Ĵ(x, x 2, ρ, {µ }) (2.5) over all admissible cotrols {µ }. 3

14 The followig are the two mai theorems of this article, which show the existece of optimal cotrols for two queueig cotrol problems described i Defiitio 2.6. Theorem 2.7 There exists a u a such that µ, a = β λ R(ρ ) + u a, =, 2,... (2.6) is a asymptotically optimal sequece of service rates for the first queueig cotrol problem defied i Defiitio 2.6. Furthermore, this u a satisfies the followig: u a = argmi u e y[ ( ) σ 2 ] c(u) + h 2u y dy, where σ 2 = λ N + µ. Theorem 2.8 There exists a u d such that µ, d = β λ R(ρ ) + u d, =, 2,... (2.7) is a asymptotically optimal sequece of service rates for the secod queueig cotrol problem defied i Defiitio 2.6. The existece of u a is proved i Theorem 3.4, ad that of u d is established i Theorem 3.4. Note that, i each of the cotrol problems, the choices i (2.6) ad (2.7) are ot uique. For example, for the first problem, µ, a = λ N(ρ ) + u a, =, 2,... (2.8) is aother such choice. Note that these two choices are asymptotically equivalet, i the sese that the behavior of the diffusio-scaled system uder these two choices are the same (as a cosequece of the fact that u ad ũ defied i (4.5) are asymptotically equivalet). 4

15 3 Browia Cotrol Problems The Browia cotrol problem (BCP) for a queueig etwork is formulated by replacig the liear combiatio of cetered processes i the scaled queue equatios (see the martigale terms Ŵ i i =, 2 defied i Sectio 4) by suitable Browia motios, ad costructig a diffusio cotrol problem ([5, 6] etc.). The solutios to such cotrol problems ofte cotai useful isights about the queueig cotrol problems, ad are commoly used i such aalysis. I the ext sectio, we will establish that the sequece ( ˆX, ˆX 2 ) coverges weakly to a twodimesioal process (X, X 2 ), which is a reflectig diffusio with state space i the first quadrat of R 2. Furthermore, (X, X 2 ) satisfies the followig stochastic differetial equatios: for X (t) = x ut + σ W (t) + L (t) X 2 (t) = x 2 + βut + σ 2 W 2 (t) βl (t) + L 2 (t), (3.9) where W ( ) ad W 2 ( ) are two stadard Browia motio processes ad they are correlated. Their depedece is described by E[W (t)w 2 (t)] = rt, where r = µβ σ σ 2 ad the costats σ ad σ 2 are give by σ = λn + µ ad σ 2 = λr + µβ( β). The local-time processes L ad L 2 are o-decreasig ad satisfy L () = L 2 () =. Furthermore, t [X (s)>]dl (s) = ad t [X2 (s)>]dl 2 (s) =, for all t >. The local time processes L ad L 2 keep the state processes, respectively X ad X 2, o-egative. X (t) represets the limitig queue legth for the ew product at time t ad X 2 (t) represets the limitig ivetory of the refurbished products at time t >. Remark 3. The costat u i (3.9) is the cotrol parameter which captures the rate at which the maufacturig rate (of ew products) deviates uder diffusio scalig from the oes specified by heavy traffic. The o-egativity of u guaratees the fiiteess of the cost fuctioal i (3.2), sice otherwise X i (3.9) will be trasiet leadig to ifiite holdig costs. Hece, we focus oly o cotrols u throughout the article. 5

16 The stochastic system described i (3.9) is kow as a Browia cotrol system. I the followig two subsectios, we will cosider this system uder the two types of cost structures, ad solve the correspodig cotrol problems i each case. 3. BCP with log-ru average cost First we describe the log-term average cost structure associated with the first cotrol problem. For such a Browia cotrol system described by (3.9), we cosider the log term expected average cost fuctio give by I(x, x 2, u) = lim sup T [ T ] T E (c(u) + h(x (s)))ds + k L 2 (T ). (3.2) Recall that c(u) represets the cotrol cost, h(x (t)) represets the holdig cost for queue legth X (t) ad the costat k > represets the pealty per lost customer for refurbished products. Sice c(u) is time idepedet, the cost fuctio i (3.2) ca be writte as I(x, x 2, u) = c(u) + lim sup T [ T ] T E h(x (s))ds + k dl 2 (t). (3.2) I the followig discussio, we ited to obtai a optimal cotrol u that miimizes I(x, x 2, u) over all costat cotrols u. We ca represet the value fuctio of this stochastic cotrol problem by V (x, x 2 ) = if I(x, x 2, u). u [ ] Next, usig the equatio for X ( ), we ited to compute lim T T E T h(x (s))ds we itroduce the costat γ(u) for each cotrol u by explicitly. First γ(u) = 2u σ 2 h(x)e 2ux σ 2 dx. (3.22) Sice h( ) has polyomially-bouded growth (see Assumptio 2.3), this costat γ(u) is fiite for each u. The followig propositio coects a part of the cost i (3.2) with γ(u). The proof of this result ivolves applicatio of the Itô s formula, ad is somewhat stadard. Hece, we state this as a propositio here without proof, ad describe a proof i the Appedix. 6

17 Propositio 3.2 Let (X, X 2 ) satisfy (3.9). The lim T T E T h(x (s))ds exists ad equals γ(u). I the ext lemma, we examie how E[L 2 (T )] grows as a fuctio of T. Sice u > i (3.9), X ( ) is a reflectig diffusio process with costat egative drift u. It has a statioary distributio with expoetially decayig tail ad lim T T (L (T ) ut ) = a.s. (see [5], page 29). Therefore, the magitude of the term (ut L (t)) must be at most of the order sup s t W (s). Hece usig the equatio for X 2 ( ) i (3.9), we expect that the process X 2 ( ) does ot reach the origi that ofte. Cosequetly, L 2 (t) icreases at a much slower rate tha L (t), as t teds to ifiity. Ituitively, i the absece of a variable ivetory cost for refurbished products, the cost fuctio emphasizes customer service with the result that lost sales of refurbished products are egligible i the limit. I the followig lemma, we verify this ituitio by provig that lim T L 2 (T ) T as well as i L. = a.s. Lemma 3.3 Let L 2 be the local time process of X 2 i (3.9). The L 2 (T ) E[L 2 (T )] lim = a.s. ad lim =. T T T T Proof. L has the represetatio (see [5]) { } L (t) = max, if (x + σ W (s) us). (3.23) s t Cosider the Browia motio W ( ) ad the maximum process M defied by M (t) = sup W (s). s t The (3.23) implies that L (t) ut σ M (t), for all t. Similarly L 2 has the represetatio { } L 2 (t) = max, if (x 2 + σ 2 W 2 (s) + βus βl (s)). (3.24) s t Agai we itroduce M 2 (t) = sup W 2 (s). Notice that s t if [x 2 + σ 2 W 2 (s) + βus βl (s)] = sup [β(l (s) us) σ 2 W 2 (s) x 2 ]. s t s t 7

18 Usig the estimate L (t) ut σ M (t) for all t, we obtai β(l (s) us) σ 2 W 2 (s) x 2 βσ M (s) + σ 2 M 2 (s) βσ M (t) + σ 2 M 2 (t), for all s t. Also, βσ M (t) + σ 2 M 2 (t) for all t. Therefore, it follows that { } max, sup s t [β(l (s) us) σ 2 W 2 (s) x 2 ] βσ M (t) + σ 2 M 2 (t), for all t. Hece by (3.24), we obtai that L 2 (t) σ 2 M 2 (t)+βσ M (t), for all t >. By the properties of the maximum process of Browia motio (see pages 95 ad 2 of [2]), we kow that lim M lim 2 (T ) T T that T M (T ) T = = a.s., ad E[M i (t)] C T for i =, 2, where C is a costat. Therefore, it follows L 2 (T ) E[L 2 (T )] lim = a.s. ad lim =. T T T T Usig Propositio 3.2 ad Lemma 3.3, we ca provide the followig explicit represetatio of the cost fuctio I( ) i (3.2): I(x, x 2, u) = c(u) + γ(u) = c(u) + 2u σ 2 h(x)e 2ux σ 2 dx. (3.25) This expressio ca be simplified to obtai I(x, x 2, u) = e y[ ( ) σ 2 ] c(u) + h 2u y dy. (3.26) The above computatios establish the followig theorem. Theorem 3.4 The value fuctio V (x, x 2 ) of the log-ru average cost problem described i (3.2)-(3.2) is idepedet of (x, x 2 ) ad it has the followig represetatio V V (x, x 2 ) = if u e y[ ( ) σ 2 ] c(u) + h 2u y dy. 8

19 Furthermore, for F (u) = e y[ ( ) σ 2 ] c(u) + h 2u y dy, (3.27) a optimal cotrol u a > is give by F (u a) = mi u F (u). To compute u a we differetiate the above fuctio of u to obtai F (u) = c (u) σ2 2u 2 ( ) σ e y yh 2 2u y dy. To fid a cadidate for u a, we let F (u) =, which yields the followig ecessary coditio: 2(u a) 2 c (u a) = σ 2 ( ) σ e y yh 2 2u y dy. a I the case where both c( ) ad h( ) are covex twice differetiable icreasig fuctios, the above coditio is also sufficiet, because F (u) = c (u) + σ4 2u 4 ( ) σ e y y 2 h 2 2u y dy + σ2 u 3 ( ) σ e y y h 2 2u y dy >. For example, whe c(x) = x m ad h(x) = x q, where m ad q, we obtai u a = ( ( ) q σ 2 q ) m q! m+q. (3.28) BCP with ifiite horizo discouted cost I the previous sectio, we have oticed that the expected cost k E[L 2 (T )] which represets the pealty icurred from lost customers for refurbished products durig the time iterval [, T ] grows E[L at a rate much slower tha T as T teds to ifiity. I fact, lim 2 (T )] T T is equal to zero. For this reaso, the optimal cotrol policy developed i the previous sectio is ot iflueced by this cost compoet. To capture the effect of the pealty icurred from lost customers for refurbished products, we also cosider a ifiite horizo discouted cost structure for the same model i (3.9). I this case, the cost fuctioal as well as the optimal policy are affected by the cost compoet correspodig to the lost customers for refurbished products as well as by the iitial data x ad 9

20 x 2 of (3.9). I our aalysis of this cost structure, we use h(x) = x 2 to perform explicit computatios. A mai difficulty i our aalysis is to obtai a explicit formula for E[L 2 (T )] i this two-dimesioal model described i (3.9). For this reaso, we are able to establish a otrivial optimal cotrol u d > for the discouted cost oly whe the iitial data (x, x 2 ) belog to a certai regio i R 2. Here we aalyze the ifiite horizo discouted cost structure give by J(x, x 2, u) = E e αt [(c(u) + X (t) 2 )dt + k dl 2 (t)], (3.29) where α > ad k > are positive costats. We ca rewrite this cost fuctioal i the form J(x, x 2, u) = c(u) α + Φ(x, u) + Ψ(x, x 2, u), (3.3) where ad Φ(x, u) = E Ψ(x, x 2, u) = k E e αt X (t) 2 dt, (3.3) e αt dl 2 (t). (3.32) The value fuctio for this cotrol problem is give by Q(x, x 2 ) = if u J(x, x 2, u). (3.33) I the followig lemma, for a give cotrol u, we compute Φ(x, u) described i (3.3). The proof of the lemma is give i the Appedix. Lemma 3.5 Let Φ(x, u) be defied by (3.3). The Φ(x, u) = α ( ) where λ (u) = σ (u 2 + 2ασ 2 2 ) u. [ ( x u ) ( )] 2 σ 2 + α α + u2 α 2 2u α 2 λ (u) e λ (u)x, (3.34) For our two-dimesioal model described i (3.9), ext cosider the fuctioal Ψ give i 2

21 (3.32). Here we are uable to compute Ψ(x, x 2, u) explicitly. Therefore, we obtai a upper boud for the quatity Ψ(x, x 2, u) Ψ(x, x 2, ) i the ext lemma. Here, Ψ(x, x 2, ) represets the cost defied by (3.32) i the case of zero cotrol. To idetify the depedece of the processes o the cotrol u, we rewrite our model equatio (3.9) i the followig form: X u (t) = x ut + σ W (t) + L u (t) (3.35) X u 2 (t) = x 2 + βut + σ 2 W 2 (t) βl u (t) + L u 2(t), where L u ad Lu 2 are local time processes of Xu ad Xu 2 respectively. Next we itroduce the process X u by X u (t) = x 2 + βut + σ 2 W 2 (t) + L u (t), (3.36) where L u (t) is the local time process of Xu at the origi ad hece L u (t) for all t. Notice that (3.36) ca be rewritte as ( X u (t) = x 2 + βut + σ 2 W 2 (t) βl u (t) + βl u (t) + L ) u (t). (3.37) We compare (3.37) with the secod equatio i (3.35). Recall that the local time process L u 2 is the miimal cotiuous o-decreasig process which keeps the sum (x 2 +βut+σ 2 W 2 (t) βl u (t)+lu 2 (t)) o-egative. But i (3.37), Xu (t) for all t ad therefore we obtai the iequality L u 2(t) βl u (t) + L u (t), for all t. (3.38) This estimate of L u 2 (t) will be useful i the ext propositio. Propositio 3.6 Let the iitial value (x, x 2 ) be fixed. The the cost fuctioal J(x, x 2, u) defied i (3.29) is cotiuous i the cotrol variable u. Proof. For J(x, x 2, u), we cosider the represetatio (3.3). The fuctio c(u) is cotiuous i u ad by lemma 3.5, Φ(x, u) is also cotiuous i u. Therefore, it remais to show that Ψ(x, x 2, u) is cotiuous i the variable u. 2

22 For ay u, by (3.32) ad Fubii s theorem, we obtai [ Ψ(x, x 2, u) = k E t= [ = k E ( s=t αe αs L u 2(s)ds s= ) ] αe αs ds dl u 2(t) ]. Therefore Ψ(x, x 2, u) = α k E e αt L u 2(t)dt = α k e αt E[L u 2(t)]dt. (3.39) t= t= Next we fix u ad let {u } coverge to u. We assume that u K for some fixed costat K >. It suffices to show that local time process L u (t) has the represetatio lim Ψ(x, x 2, u ) = Ψ(x, x 2, u). For each u, the u u { } L u (t) = max, sup (us σ W (s) x ), (3.4) s t ad therefore, for each T > it is evidet that L u (t) coverges to Lu (t) uiformly o [, T ]. Next, L u 2 (t) has the represetatio { } L u 2(t) = max, sup (βl u (s) βus σ 2 W 2 (s) x 2 ). (3.4) s t Sice u u ad L u (t) ad Lu 2 (t) coverge uiformly to Lu (t) ad Lu 2 (t), respectively, for all t T, from the above represetatio (3.4)-(3.4). For each u, L u (t) x + Kt + σ sup s t W (s) ad L u (t) x 2 + β Kt + β L u (t) + σ 2 sup W 2 (s). Now let M(t) be the s t process defied by M(t) = β x + x β Kt + β σ sup s t W (s) + σ 2 sup W 2 (s). s t Usig Doob s iequality we obtai E[M(t) 2 ] C o ( + t 2 ), where C o > is a geeric costat. Hece E[M(t)] C o ( + t) ad E e αt M(t)dt <. Sice L u 2 (t) M(t) ad Lu 2 (t) coverges to L u 2 (t) a.s. as u teds to u, we ca apply the Domiated Covergece Theorem to coclude that lim E e αt L u (t)dt = E e αt L u (t)dt. 22

23 Hece, lim Ψ(x, x 2, u ) = Ψ(x, x 2, u) ad this completes the proof. Remark 3.7 Sice lim c(u) = + ad J(x, x 2, u) > c(u) u α, the above propositio guaratees the existece of a o-egative optimal cotrol u d. Next, we obtai some sufficiet coditios that guaratee a strictly positive optimal cotrol u, which leads to a o-trivial asymptotically optimal sequece of cotrols for the origial sequece of cotrolled queueig systems. Lemma 3.8 Let Ψ(x, x 2, u) ad Ψ(x, x 2, ) be as described i (3.32). The Ψ(x, x 2, u) Ψ(x, x 2, ) α β k e αt (E[L u (t)] E[L (t)])dt+k α e αt E[ L u (t)]dt+kx 2, (3.42) where L u, L ad L u are the local time processes described i (3.35). Proof. Usig (3.39), we obtai Ψ(x, x 2, u) Ψ(x, x 2, ) = kα e αt (E[L u 2(t)] E[L 2(t)])dt. (3.43) t= Next we estimate (E[L u 2 (t)] E[L 2 (t)]) for each t. By (3.38), we have E[Lu 2 (t)] βe[lu (t)] + E[ L u (t)]. O the other had, usig the secod equatio of (3.35), we have E[L 2 (t)] βe[l (t)] + x 2 = E[X 2 (t)], for all t. Therefore, E[L 2 (t)] βe[l (t)] x 2 for all t. Cosequetly, E[L u 2(t)] E[L 2(t)] β [ E[L u (t)] E[L (t)] ] + E[ L u (t)] + x 2 for all t. Thus, from this estimate i (3.43) we have (3.42). I the ext lemma, we compute the itegrals i (3.42). 23

24 Lemma 3.9 (i) For each u, α ( ) where λ (u) = σ u 2 + 2ασ 2 2 u. (ii) For each u, α ( ) where λ 2 (u) = σ β 2 u 2 + 2ασ βu. e αt E[L u (t)]dt = ) ( u 2α 2 + 2ασ 2 + u e λ (u)x, (3.44) e αt E[ L u (t)]dt = ) ( β 2α 2 u 2 + 2ασ 22 βu e λ 2(u)x 2, (3.45) Proof. First otice that α e αt E[L u (t)]dt = E e αt dl u (t). Let Q(x) = e λ(u)x, where ( ) λ (u) = σ u 2 + 2ασ 2 2 u. The Q satisfies σ 2 2 Q (x) uq (x) αq(x) =, for x > ad Q () = λ (u). Next, we cosider the first equatio of (3.35) ad apply Itô s lemma to Q(X u (t))e αt to obtai T E[Q(X u (T ))]e αt = Q(x ) λ (u)e e αt dl u (t). We let T ted to + ad obtai Q(x ) = λ (u)e e αt dl u (t). Hece (3.44) follows. The proof of (3.45) follows essetially alog the same steps by usig Q(x) = e λ 2(u)x where λ 2 (u) is give i (3.45) ad the process X i (3.36). The followig propositio follows from Lemmas 3.8 ad

25 Propositio 3. Ψ(x, x 2, u) Ψ(x, x 2, ) kβ ) [( u 2α 2 + 2ασ 2 + u + k 2α ] e λ (u)x 2ασ 2 2α e x σ [( β 2 u 2 + 2ασ 22 βu ) e λ 2(u)x 2 ] + kx 2. (3.46) The proof of this propositio is straightforward ad therefore omitted. Remark 3. The estimates we have obtaied i the proof of the above propositio also yield the followig upper boud of the cost fuctioal J(x, x 2, u) defied i (3.3): J(x, x 2, u) < c(u) α [ β + Φ(x, u) + k λ (u) e λ (u)x + ] λ 2 (u) e λ 2(u)x 2 + kx 2. (3.47) I the ext propositio, we obtai a sufficiet coditio which guaratees J(x, x 2, u) < J(x, x 2, ) where the cost fuctioal J is defied i (3.3). Propositio 3.2 Let (x, x 2 ) be the iitial data i (3.9). If there is a cotrol u that satisfies [ ( α kβ 2u ) ] [ α 2 λ (u) e λ (u)x + αk λ 2 (u) e λ 2(u)x 2 αkβσ + αkx 2 < e 2α x σ + 2u 2α α ( x u ) ] c(u), α where λ 2 (u) ad λ (u) are described i (3.45) ad (3.34), respectively, the J(x, x 2, u) < J(x, x 2, ). Proof. Usig (3.3) we observe that J(x, x 2, u) < J(x, x 2, ) if ad oly if Ψ(x, x 2, u) Ψ(x, x 2, ) < [Φ(x, ) Φ(x, u)] c(u) α. (3.48) Next we ca use the estimate (3.46) i Propositio 3.. Therefore, the iequality [ kβ ( λ (u) e λ (u)x ) ] 2ασ 2 2α 2α e x σ + k λ 2 (u) e λ 2(u)x 2 + kx 2 < [Φ(x, ) Φ(x, u)] c(u) α implies the iequality i (3.48). Usig (3.34) ad followig a straightforward computatio, we 25

26 obtai ( kβ 2u ) α 2 λ (u) e λ (u)x + k λ 2 (u) e λ 2(u)x 2 + kx 2 < 2u (x α 2 u ) + kβσ α e 2α σ 2α x c(u) α. This iequality is same as (3.47) ad hece the result follows. Remark 3.3. If x 2 ad u remai fixed ad x teds to ifiity the the right had side of the iequality i (3.47) teds to ifiity while the left had side of (3.47) teds to αk λ 2 (u) e λ 2(u)x 2 + αkx 2. Therefore, for fixed x 2 ad u, large values of x satisfy (3.47). 2. If 2u > α 2 kβ ad if there is a poit ( x, x 2 ) that satisfies 2u α ( x u ) c(u ) αkx 2 α > k ( ) β 2 u 2 + 2ασ2 2 βu, the the above iequality holds for all x x. It is a straightforward matter to check that the assumptio of Propositio 3.2 is true. Hece J(x, x 2, u) < J(x, x 2, ) for all x x. Next, we itroduce the regio A = {(x, x 2 ) : There exists u > where (x, x 2, u) satisfies (3.47) }. (3.49) This set A is o-empty as explaied i the above remark. Theorem 3.4 Let the iitial data (x, x 2 ) of (3.9) belog to the set A i (3.49). The there is a optimal cotrol u d > such that Q(x, x 2 ) = J(x, x 2, u ), where Q is the value fuctio defied i (3.33). Proof. We obtai J(x, x 2, u) < J(x, x 2, ) for some u > by usig Propositio 3.2. O the other had by (3.3), J(x, x 2, u) > c(u) α lim u + c(u) = +, there exists u 2 for all u. Sice c( ) is strictly icreasig ad > such that c(u) α > J(x, x 2, ) for all u > u 2. Cosequetly, if u J(x, x 2, u) = if <u<u2 J(x, x 2, u). By Propositio 3.6, J(x, x 2, ) is cotiuous i u variable. Therefore, there exists a u d such that < u d < u 2 which satisfies J(x, x 2, u d ) = 26

27 if u J(x, x 2, u). 4 Asymptotic Optimality Our objective i this sectio is to use the optimal cotrols derived for the BCPs i the previous sectio for the costructio of asymptotically optimal cotrols for the queueig cotrol problem (as described i Defiitio 2.6). This costructio is used to prove the mai theorems of the article, Theorems 2.7 ad 2.8. Throughout this sectio, ρ ad µ (see Assumptio 2.4) are fixed ad for simplicity of otatio, we will deote µ by µ ad omit ρ from all otatios, i.e. deote λ N (ρ ), λ N (ρ ), λ R (ρ ), λ R (ρ ) by λ N, λ N, λ R, λ R respectively. I this sectio, we will use the stadard otatio D([, ), IR) for the set of all right cotiuous fuctios from [, ) to IR with left limits. All the processes are defied o D([, ), IR) uless specified otherwise. e D([, ), IR) will deote the idetity fuctio, i.e. e(t) = t for all t. The covergece i distributio of a sequece of processes Φ ( ) to Φ( ) will be deoted as Φ Φ or by Φ ( ) Φ( ). Whe sup Φ (s) Φ(s) as, for all t, we will write that s t Φ Φ uiformly o compact sets, or uiformly o compacts. 4. Scaled processes ad a Skorohod map We have defied two types of scaligs for the various processes i (2.) above. Here we obtai coveiet represetatios for the rescaled processes that are relevat for our aalysis. Recall the defiitio of u i (2.). We defie aother similar quatity ũ below. Note that by Assumptio 2., these two quatities are asymptotically equivalet: There exists u such that u = ) (µ λ R β u, ũ = (µ λ N ) u, as. (4.5) 27

28 Next, we itroduce the martigales which are related to the Poisso processes i the heavy-traffic aalysis of the queueig cotrol problem. ˆM (t) = [N (λ Nt) λ Nt], ˆM 2 (t) = ( t ) [N 2 µ { ˆX (s)>} ds ˆM 3 (t) = [ Φ (N 2 ˆM 4 (t) = ( [N 3 ( t Ŵ (t) = ˆM (t) ˆM 2 (t), t t µ { ˆX (s)>} ds )) β λ R { ˆX 2 (s)>} ds ) t µ { ˆX (s)>} ds ], t µ { ˆX (s)>} ds ], λ R { ˆX 2 (s)>} ds ], (4.5) Ŵ 2 (t) = ˆM 3 (t) ˆM 4 (t), (4.52) for all t, =, 2,.... From the defiitio of the processes i (2.4)-(2.6) ad the diffusio scaled processes i (2.) (with a simple chage of variable formula t g(s)ds = t g(s)ds) we ca write the scaled state processes as ˆX (t) = ˆx + N (λ Nt) ( N 2 t µ { ˆX (s)>} ds ) = ˆx ũ t + Ŵ (t) + ˆL (t), (4.53) ˆX 2 (t) = ˆx 2 + ( t )) Φ (N 2 µ { ˆX (s)>} ds ( t ) N 3 λ R { ˆX 2 (s)>} ds where = ˆx 2 + βu t + Ŵ 2 (t) β ˆL (t) + ˆL 2 (t), (4.54) ˆL (t) = µ for all =, 2,..., t. t { ˆX (s)=} ds, ˆL 2 (t) = λ t R { ˆX 2 (s)=} ds, (4.55) The proof of asymptotic optimality uses the followig maps ad their properties. Lemma 4. (A two-dimesioal Skorohod map) Let u, u 2, β ad let u = (u, u 2 ). For each x, x 2 ad ad w = (w, w 2 ) D([, ), IR) D([, ), IR) with w i (), i =, 2, there exist uique q i, l i D([, ), IR), i =, 2, satisfyig the followig properties: (i) q (t) = x u t + w (t) + l (t), t, 28

29 (ii) q 2 (t) = x 2 + βu 2 t + w 2 (t) βl (t) + l 2 (t), t, (iii) l i ( ) is odecreasig, l i () = ad q i (t)dl i (t) = for i =, 2. Defie the followig maps Γ u i, ˆΓ u i, i =, 2 as follows: for a give w as above, let Γu i (w) = q i, ˆΓ u i (w) = l i, i =, 2. We will deote the map (Γ u i ( ), ˆΓ u i ( ) : i =, 2) as the Skorohod map relevat for this problem. Proof of the existece of the above map is straightforward. For x D([, ), IR) with x(), defie the followig maps: φ(x)(t) = x(t) + ψ(x)(t), where ψ(x)(t) = if mi{x(s), }, for t. (4.56) s t The above maps are called oe-dimesioal Skorohod maps i [, ) (see [23, 3]). Usig the above maps it is easy to verify that if w i D([, ), IR), i =, 2, the followig represetatios hold for the Skorohod maps defied i Lemma 4.. Γ u (w) = φ(x ue + w ), ˆΓu (w) = ψ(x ue + w ), Γ u 2 (w) = φ ( x 2 + βue + w 2 βˆγ u (w) ), ˆΓu 2 (w) = ψ (x 2 + βue + w 2 βˆγ u (w) ). (4.57) It is well kow that the maps φ ad ψ are both Lipschitz cotiuous maps i the uiform topology (see [23] for example). More precisely, for x, x. D([, ), IR) ad x T = sup x(s), we have s T φ(x) φ(x ) T C x x T, ψ(x) ψ(x ) T C x x T, (4.58) for some C >. Usig the represetatios i (4.57) above, oe ca verify that the Skorohod maps defied i Lemma 4. are cotiuous fuctios i the metric of uiform covergece o compact 29

30 sets i the followig sese: For all T > implies lim u i u i =, lim w i w i T =, i =, 2 lim Γu i (w, w 2 ) Γu i (w u, w 2 ) T =, lim ˆΓ i (w, w 2 ) ˆΓ u i (w, w 2 ) T =, for i =, 2. (4.59) This cotiuity property will be crucial for establishig some of the covergece results i the proofs below. 4.2 Weak covergece aalysis ad proof of Theorems 2.7 ad 2.8 Note that for ay admissible policy {µ }, there exists u, such that ũ ad u both coverge to u as teds to ifiity. To simplify otatio, we will use the followig abbreviatio for this sectio: For ū = (ũ, u ) ad ū = (u, u), Γ. i = Γū i, ˆΓ.. i = ˆΓū i, Γ i = Γ ū i, ˆΓ. i = ˆΓū i, for i =, 2. We start with the followig lemma, which describes equivalet represetatios of the cost fuctioals i the queueig etwork cotrol problems as well as the Browia cotrol problems i Sectio 3. Lemma 4.2 The log-ru average cost fuctioals for the queueig etwork ad the BCP i (2.2) ad (3.2), respectively, have the followig represetatio: Î(x, x 2, {µ }) = lim if where ˆγ({µ }) = [ c (u ) + ˆγ({µ }) + k lim sup T ( ) λ ( ) ( N i λ µ h N µ i= I(x, x 2, u) = c(u) + γ(u), where γ(u) = 2u σ 2 ) (ˆL ] T E 2 (T ), ) i, (4.6) h(x)e 2ux σ 2 dx. (4.6) The ifiite horizo discouted cost fuctioals for the queueig etwork ad the BCP i (2.3) ad (3.29), respectively, have the followig represetatio: Ĵ(x, x 2, ρ, {µ }) = lim if J(x, x 2, u) = c(u) α c (u ) α + E + E αe αt [ t 3 [ t ( ) 2 αe αt ˆX (s) ds + k ˆL 2 (t)] dt, (4.62) ] (X (s)) 2 ds + k L 2 (t) dt. (4.63)

31 Proof. To verify (4.6), first ote that for each fixed, ˆX { } state-space L. = j : j =,, 2,..., ad jump rates give by is a jump-markov process with λ N if j = i +, i L Q (i, j) = µ, if j = i, i L \ {}, otherwise. Straightforward calculatios (solvig the balace equatios) yields that the ivariat distributio for ˆX is that of a radom variable X, where X follows a Geometric distributio with parameter a = (λ N /µ ). Therefore, it follows that [ T ] lim T T E h( ˆX (s))ds = E [h(x )]. Note that by the assumptios o h( ), ad distributio of X, the expectatio o the right side is fiite. This proves that the represetatio i (4.6) is accurate. The represetatio i (4.6) follows from Propositio 3.2 ad Lemma 3.3. The proof of the represetatios of the discouted cost fuctioals i (4.62)-(4.63) are stadard ad similar to that of Lemma 4.4 of [2]. Propositio 4.3 The processes ˆX ad ˆX 2 i (4.53)-(4.54) satisfy ( ˆX, ˆX 2, ˆL, ˆL 2 ) = ( Γ (Ŵ ), Γ 2 (Ŵ ), ˆΓ (Ŵ ), ˆΓ ) 2 (Ŵ ). (4.64) For the processes Ŵ ad Ŵ 2 defied i (4.52), the followig covergece holds: Ŵ (Ŵ, Ŵ 2 ) (W, W 2 ) W, (4.65) where W is a two-dimesioal Browia motio as described i (3.9). Also, the followig holds. ( ˆX, ˆX 2, ˆL, ˆL 2 ) (X, X 2, L, L 2 ) ad, where (X, X 2, L, L 2 ) =. ( Γ (W ), Γ 2 (W ), ˆΓ (W ), ˆΓ ) 2 (W ), (4.66) 3

32 ad (W, W 2, X, X 2, L, L 2 ) satisfies all the coditios o the processes ivolved i defiig the BCPs i (3.9). The proof of the above propositio is somewhat stadard i the heavy traffic literature. We skip the proof here, ad preset oe i the Appedix. The followig basic lemma will be used i our proof of the mai result. A proof of this lemma ca be foud i the Appedix as well. Lemma 4.4 Let {a } be a sequece such that a a as ad h( ) be the cost fuctio used i our aalysis. The the followig holds a k= ( ) ( k h a ) k a h(x)e ax dx, as. (4.67) We will use the followig momet estimates i our aalysis to establish the covergece of the cost fuctioals. Propositio 4.5 The followig estimates hold: There exist costats C, C 2 > such that for all ad t [ E sup s t E ˆX (s) 4 ] [ ) ] 2 (ˆL 2 (t) C ( + t 2 + t 4) ad (4.68) C 2 ( + t + u ũ 2 t 2), (4.69) where u ad u are as described i (4.5). Proof. From the represetatio of ˆX i (4.66) ad (4.57)-(4.58), we have [ ] E sup ˆX (s) 4 s t [ C 4 E sup s t x ũ s + Ŵ (s) 4 ] [ C E (x ) 4 + (ũ ) 4 t 4 + E ( )] sup Ŵ (s) 4, (4.7) s t for some costat C >, idepedet of ad t. Sice {x } ad {ũ } are both coverget sequeces, by Doob s iequality for the martigale Ŵ ( ), ad usig the fact that E(Ŵ (t))4 C t 2 (for some C > ) we have the proof of the first estimate (4.68) of the propositio. 32

33 For the secod estimate, ote that from (4.53), we have [ ] E sup u s ˆL (s) 2 s t C C [ ( u ũ 2 t 2 + E sup s t [ u ũ 2 t 2 + (x ) 2 + E ũ s ˆL (s) 2 )] ( ) sup Ŵ (s) 2 + E s t ( )] sup ˆX (s) 2 s t C ( u ũ 2 t t + t 2), (4.7) where the last estimate follows usig (4.68) ad argumets similar to those used i obtaiig (4.7). Here, C > represets a geeric costat idepedet of ad t ad the value of this costat varies from lie to lie of (4.7). Now, from the represetatio of ˆL i (4.53), we have [ ) ] 2 E (ˆL 2 (t) C 2 [ ( ) ( )] (x 2 ) 2 + E sup Ŵ2 (s) 2 + βe sup u s ˆL (s) 2. s t s t The secod estimate ow follows from (4.7), the Doob s iequality for the martigale Ŵ 2 ( ), ad the fact that E(Ŵ 2 (t))2 C t, for some C >. Now, usig the results above, we prove the mai theorems of the paper, viz. Theorems 2.7 ad 2.8, regardig optimal cotrols for the queueig etwork cotrol problem. Proof of Theorem 2.7: First, we prove the asymptotic aalysis of our proposed policy for the ergodic cost problem. Sice for ay admissible policy {µ }, the correspodig {u } coverges to some u, we have from the cotiuity of c( ) that c(u ) c(u), as. (4.72) Also, ote that for such policies, a. λ = ( N µ ) = (ũ /µ ) coverges to a =. ū µ = 2u, sice σ 2 = λ N + µ = 2 µ by Assumptio 2.4. Therefore, by Lemma 4.4, we have σ 2 ˆγ ({µ }) γ(u), as, (4.73) 33

34 where ˆγ ad γ are as described i (4.6) ad (4.6). By Propositio 4.5, we have that lim if [ ] lim sup T T E ˆL 2 (T ) lim if [ ] lim sup C2 ( + T + u T T ũ 2 T 2 ) =. (4.74) Hece by Lemma 4.2, we have that for ay admissible cotrol policy {µ }, with u covergig to some u, Î(x, x 2, {µ }) = I(x, x 2, u). (4.75) Note that from the costructio of our proposed policy {µ, a }, we have that the correspodig {u, } coverges to u a, where u a is as i Theorem 3.4. Hece we have that from Theorem 3.4 ad (4.75) that for ay admissible policy {µ } Î(x, x 2, {µ }) = I(x, x 2, u) I(x, x 2, u a) = Î(x, x 2, {µ, a }). (4.76) This proves the asymptotic optimality of our proposed policy for the queueig etwork problem with log-ru average cost. Proof of Theorem 2.8: Now we prove the optimality result for the ifiite horizo discouted cost problem. Note that for ay admissible cotrol policy {µ } with the correspodig {u } covergig to some u, we have from Propositio 4.3 that ( ˆX (s)) 2 ds (X (s)) 2 ds, as. (4.77) This follows from the fact that ˆX X ad the fuctio x(s)2 ds is a cotiuous map o D([, ), IR) with respect to the uiform metric o compacts, ad hece cotiuous mappig theorem applies. Combiig the last part of Propositio 4.3 ad (4.77), we have for all fixed t [ t ( ) 2 ˆX (s) ds + k ˆL 2 (t)] [ t ] (X (s)) 2 ds + k L 2 (t), as. (4.78) 34

35 Observe that from Propositio 4.5, we have for each fixed t, [ t E ( ) ] 2 2 ˆX (s) ds + k ˆL 2 (t) C 3 [ + t 2 + t 4 ], for all, (4.79) where C 3 > is a costat idepedet of. From (4.78) ad (4.79) we get that for each fixed t, [ t E ( ) 2 ˆX (s) ds + k ˆL 2 (t)] [ t ] E (X (s)) 2 ds + k L 2 (t), as. (4.8) Now from (4.79), it is easy to verify that [ t αe αt E ( ) ] 2 2 ˆX (s) ds + k ˆL 2 (t) dt C 4 for all, (4.8) where C 4 > is a costat idepedet of ad t. This boud together with the covergece i (4.8) implies that as, [ t αe αt E ( ) 2 ˆX (s) ds + k ˆL 2 (t)] dt [ t ] αe αt E (X (s)) 2 ds + k L 2 (t) dt.(4.82) Hece by Lemma 4.2, we have that for ay admissible cotrol policy {µ }, with u covergig to some u, Ĵ(x, x 2, {µ }) = J(x, x 2, u). (4.83) Usig the same argumets as i obtaiig (4.75), we get Ĵ(x, x 2, {µ }) = J(x, x 2, u) J(x, x 2, u d ) = Ĵ(x, x 2, {µ, d }), (4.84) where u d is the optimal drift for the BCP as give i Theorem

36 5 Comparative Statics ad Numerical Aalysis I this sectio, we examie the sesitivity of the asymptotically optimal policy to chages i the problem parameters. I our stylized model, the cosumer willigess-to-pay for refurbished items, δ, ad the retur fractio, β, are treated as exogeous parameters, but i reality they may be iflueced by the producer. For istace, marketig efforts or warraties could icrease δ while geerous retur policies would icrease β. O the other had, costs of backorders for ew products ad lost sales of refurbished products are difficult to estimate. We ivestigate the impact of β ad δ o the optimal productio rate. We focus o the case where the backorder ad service costs are polyomials ad vary the costat cost, k, per lost sale of refurbished product. From the expressios give i Remark 2.5, it is easy to verify that uder the heavy traffic coditios, λ N (ρ ) δ <, λ R (ρ ) δ <, λ N (ρ ) β <, ad λ R (ρ ) β >. Whe miimizig average cost, Theorem 3.4 implies that oly the costs associated with the forward portio of the closed loop supply chai (i.e., those that deal with ew products) ifluece the optimal policy. The usual tradeoff betwee backorder ad productio costs exists, ad the decrease with both δ ad β i the rate of orders for ew products suggests that the optimal cotrol u a should decrease with respect to both parameters. Ideed, for the case where c(x) = x m ad h(x) = x q, by substitutig σ = 2 λ N (ρ ) i (3.28), we fid u a β = qδ (m + q)( + βδ) [ qq! m ( ) q ] pn m+q < + βδ ad u a δ = β u a δ β <. I the ifiite horizo discouted cost case, the situatio is more complicated because the lost sales cost persists i the diffusio limit ad the optimal cotrol also depeds o the iitial legth of the queue for ew products, x. (Here we assume x 2 = because if time zero represets the start of productio, there would be o iitial ivetory of refurbished products.) The effect of δ o the optimal cotrol is predictable because the demads for ew ad refurbished products both decrease 36

37 with δ uder the heavy traffic coditios; therefore, we expect (ad have verified umerically) that for give x ad k, u d arg mi u d J(x,, u d ) decreases with δ. However, because the relatioship of u d with β is ot so clear either mathematically or ituitively, we resort to umerical aalysis. Aalytical results from the Browia cotrol problem guaratee cotiuity of the cost fuctio with respect to u ad existece of optimal u d, ad also provide a exact closed form expressio for oe compoet of the cost. Simulatio is required to approximate u d for ay particular set of parameters. Let c(u) = u 2, h(x) = x 2 ad cosider p N =.9, α =., ad δ =.65 (Hauser ad Lud [2] report that remaufactured products are typically sold at 45% to 65% of the price of comparable ew products). To umerically optimize the ifiite horizo expected discouted cost, we geerate two idepedet stadard Browia motio processes B (t) ad B 2 (t) for t =,.,.2,..., 5. Let W (t) = B (t) ad W 2 (t) = rb (t) + r 2 B 2 (t), where r = µβ ( λn + µ)( λ R + µβ( β)). Computig X, L, L 2 from (W, W 2 ) usig (4.66) of Propositio 4.3, we approximate J(x,, u d ) from the sample mea of realizatios, ad obtai the approximate optimal u = u d. The sample size is sufficietly large to make the stadard error of the estimate of the cost less tha or equal to % of its value. Figure 3 shows a plot of the (estimated) J(5,, u d ) for β =., δ =.65, k = 5 as a fuctio of u d. While we have bee able to prove oly that u d > exists, the discouted cost fuctio appears to be covex. Figure 4 plots the estimated u d agaist β for various k give x = ad x = 5; the optimal u a for the ergodic cost F (u a ) is also show for referece. Observe that the optimal cotrol icreases with k, as expected, because icreasig lost sales costs prompt faster productio for a fixed retur fractio to icrease the supply of retured products to refurbish. The optimal cotrol u d does ot show mootoicity with respect to β for either x = or x = 5, which highlights the complexity of maagig the closed loop supply chai uder ucertaity. We also compared results from the average ad discouted cost cases umerically as α for the same parameter values alog with x = 5, x =, β =. ad k =. Table suggests that as the discout parameter α decreases to zero, the optimal value fuctios are asymptotically related by αj(5,, u d (α)) F (u a) ad also u d (α) u a holds. These results suggest that a Abelia limit relatioship for the value fuctios of the ifiite horizo discouted cost problem ad log-ru average cost problem could be exteded to this settig. For oe-dimesioal diffusio models, such Abelia limit theorems were established i [35]. 37

38 Figure 3: Discouted cost fuctio J(x = 5, x 2 =, u d ) for β =., δ =.65, k = 5. Table : Covergece of αj(5,, u d (α)) ad u d (α) to F (u a) =.2656 ad u a = α αj(5,, u d (α)) u d (α) Discussio I this paper, we have examied a simple model of a closed loop supply chai ad performed what to our kowledge is the first heavy traffic aalysis of such a system. By solvig a static plaig problem to maximize profit, we derived the price ratio betwee ew ad remaufactured products that would achieve heavy traffic. The we established the existece of asymptotically optimal cotrols for both average ad discouted costs i the diffusio limit, cosiderig costs of backorders for ew products, lost sales of refurbished products, ad maufacturig ew products. From these cotrols, for each cost fuctioal we derived a asymptotically optimal sequece of service rates for the queuig cotrol problem as the system approaches heavy traffic. A importat isight resultig from the mathematical aalysis is that the cotrol that miimizes log-ru average cost per uit time is ot iflueced by the cost compoet from refurbished product lost sales. We showed aalytically that the limitig average cost optimal productio rate decreases with both the 38

39 Figure 4: Cotrol that miimizes ifiite horizo discouted cost for x = ad x = 5 respectively. product retur rate ad the relative amout cosumers are willig to pay for refurbished products. By umerical aalysis we foud that the willigess-to-pay parameter has a similar effect o the limitig discouted cost optimal productio rate but the effect of the retur rate varies with iitial coditios ad the magitude of the refurbished product lost sales cost. Pre-Limit Aalysis: I the average-cost case, the classical steady-state aalysis of idetical systems ca be obtaied i special cases. Here we compare that approach with our aalysis for a sequece of systems i steady-state ad approachig the heavy traffic limit. The model satisfies the assumptios of a Jackso ope queueig etwork. Therefore, i the sequece of etworks approachig heavy traffic, if for each we have λ N (ρ ) < µ ad βλ N (ρ ) < λ R (ρ ), the a steady state exists ad arrivals of refurbished products to the secod queue follow a Poisso process with rate βλ N (ρ ). For example, suppose λ R (ρ) = λ R (ρ) + θ 2 / ad λ N = λ N (ρ) + βθ 2 / /f(), where θ 2 > ad < f() ω( ()), i.e., lim /f(). The Assumptio 2. is satisfied ad λ N (ρ ) < λ R (ρ )/β, so that a steady state exists for each at the optimal price provided that µ > λ N (ρ ). For the remaider of this sectio, we assume ρ = ρ ad suppress it i the otatio. Arrivals of refurbished products to the secod queue follow a Poisso process with rate βλ N. 39

40 The log ru average cost i the th system that correspods to equatio (2.2) is: I(x, x 2, u ) = c(u ) + lim sup E T T T ( X h (t) ( ) X = c(u ) + Eh ( ) + k lim sup T ) dt + lim sup T ) E (ˆL 2 (T ). T kλ T R E T {X 2 (s)=}ds Note that, uder Assumptios 2. ad 2.4, the third term vaishes as, as show i (4.74). The secod term ca be evaluated as: j= h(j/ ( λ ) j ( ) ) N µ λ N µ. I particular, if h(x) = x 2, this backorder cost equals: λ N (µ + λ N ) λ (µ λ = N ( N )2 u + λ R ( u + λ R β β ) + λ N λ ) 2 2( λ N ) 2 u 2. N This expressio agrees with the result of Theorem 3.4. Note that for geeral fuctios h( ), it is ot feasible to evaluate the expected backorder cost i closed form. The scalig of the queue legth by is a remider that as the sequece of etworks approaches heavy traffic, the average legth of the queue of waitig customers will icrease. However, if the steady-state backorder cost ca be approximated, this expressio suggests that i practice, the system could be desiged by choosig to achieve a tolerable magitude for E[h(X ( ))] ad the usig the result of Theorem 2.7 to set fid a ear-optimal value for µ, that achieves a appropriate tradeoff betwee service cost ad backorder cost. For the ifiite-horizo discouted cost, i priciple the backorder cost portio could be evaluated for the th system i steady state usig kow results o the trasiet behavior of the M/M/ queue (see [3], p.98). However, the lack of a exact represetatio for the arrival process to the secod queue i the trasiet phase precludes evaluatio of the lost sales portio of the discouted cost fuctio. The product form of the steady-state distributio of queue legths is based o the departure process from the first queue beig a Poisso process, but this holds oly i steady-state. 4

41 Extesios ad Future Directios: We assume that all retured products are selected for refurbishmet. This restrictio ca be removed i a straightforward maer, if we assume that for every purchased product, there is a probability β (, ) of beig retured by the customer, ad out of those retured, each has a probability β 2 (, ) of beig selected for refurbishmet. This case is actually covered by our model with β = β β 2. We also assumed refurbishmet is istataeous ad product returs, if they occur, do so immediately after purchase. To exted our model to icorporate refurbishmet lead times, oe ca replace the N 2 term i (2.5) by N 4 ( t ( u )) ) γφ (N 2 µ {X (s)>}ds du, where N 4 ( ) is aother uit Poisso process, idepedet of all the other variables ad processes, ad γ >. I this represetatio, γ is the delay rate for refurbishmet uder the assumptio that refurbishmet takes a radom amout of time followig a expoetial distributio with rate γ. This is appropriate if most retured products require very little effort to refurbish but a few have serious defects requirig legthy repairs. Similarly, a expoetially-distributed delay betwee purchase ad retur of a ew product ca be icluded by estig aother uit Poisso process with its correspodig delay rate. The aalysis of such a model is similar to the oe cosidered here, but for simplicity we did ot cosider such geeralizatios i this paper. Our model does ot iclude a variable storage cost for refurbished products. Icludig such a cost would require additioal restrictios to cotrol the legth of the secod queue. This would lead to a truly two-dimesioal cotrol problem ad require a much more difficult aalysis. We proved existece of a asymptotically optimal cotrol for ifiite horizo discouted cost oly for the specific backorder cost fuctio h(x) = x 2 but arbitrary covex fuctios should be cosidered. Fially, aalytical exploratios of the comparative statics for the discouted cost case as well as Abelia limits, available oly umerically so far, could be attempted. Ackowledgmets: Arka Ghosh is supported by the NSF grat DMS Aada Weerasighe is supported by the ARO grats W9NF532 ad W9NF7424. We are grateful for their support. 4

42 Refereces [] B. Ata. Dyamic cotrol of a multiclass queue with thi arrival streams. Operatios Research, 54(5): , 26. [2] B. Ata ad S. Kumar. Heavy traffic aalysis of ope processig etworks with complete resource poolig: asymptotic optimality of discrete review policies. Aals of Applied Probability., 5(A):33 39, 25. [3] S. L. Bell ad R. J. Williams. Dyamic schedulig of a system with two parallel servers i heavy traffic with resource poolig: Asymptotic optimality of a threshold policy. The Aals of Applied Probability, :68 649, 2. [4] P. Billigsley. Covergece of probability measures. Wiley Series i Probability ad Statistics: Probability ad Statistics. Joh Wiley & Sos Ic., New York, secod editio, 999. A Wiley- Itersciece Publicatio. [5] M. Bramso ad R. J. Williams. O dyamic schedulig of stochastic etworks i heavy traffic ad some ew results for the workload process. I Proceedigs of the 39th IEEE Coferece o Decisio ad Cotrol. IEEE, New York, 2. [6] M. Bramso ad R. J. Williams. Two workload properties for Browia etworks. Queueig Systems, 45:9 22, 23. [7] A. Budhiraja ad A. P. Ghosh. A large deviatios approach to asymptotically optimal cotrol of crisscross etwork i heavy traffic. The Aals of Applied Probability, 5(3): , 25. [8] L.G. Debo, L.B. Toktay, ad L.N. Va Wassehove. Market segmetatio ad product techology selectio for remaufacturable products. Maagemet Sciece, 5(8):93 25, 25. [9] L.G. Debo, L.B. Toktay, ad L.N. Va Wassehove. Joit life-cycle dyamics of ew ad remaufactured products. Productio ad Operatios Maagemet, 5(4):498 53, 26. [] M.E. Ferguso ad L.B. Toktay. The effect of competitio o recovery strategies. Productio ad Operatios Maagemet, 5(3):35 368, 26. [] G. Ferrer ad J.M. Swamiatha. Maagig ew ad remaufactured products. Maagemet Sciece, 52():5 26, 26. [2] A. P. Ghosh ad A. Weerasighe. Optimal buffer size ad dyamic rate cotrol for a queueig etwork with impatiet customers i heavy traffic. Submitted, iastate.edu/~apghosh/reegig_queue.pdf [3] D. Gross ad C.M. Harris. Fudametals of Queueig Theory, 3rd ed. Joh Wiley ad Sos, New York, 998. [4] V.D.R. Guide, Jr., G.C. Souza, L.N. Va Wassehove, J.D. Blackbur. Time value of commercial product returs. Maagemet Sciece, 52(8):2 24, 26. [5] J.M. Harriso. Stochastic Cotrol Theory ad Applicatios. Joh Wiley ad Sos, New York,

43 [6] J.M. Harriso. Browia models of queueig etworks with heterogeeous customer populatio. Stochastic Differetial Systems, Stochastic Cotrol Theory ad Their Applicatios. Spriger, New York, pages 47 86, 988. [7] J.M. Harriso. A broader view of Browia etworks. Aals of Applied Probability, 3(3): 9 5, 23. [8] J.M. Harriso. Browia models of ope processig etworks: Caoical represetatio of workload. Aals of Applied Probability, :75 3, 2. [9] J.M. Harriso ad R.J. Williams. Browia models of ope queueig etworks with homogeeous customer populatios. Statistics, 22:77 5, 987. [2] W. Hauser ad R.T. Lud. The remaufacturig idustry: Aatomy of a giat. Techical report, Bosto Uiversity, 23. [2] I. Karatzas ad S.E. Shreve. Browia Motio ad Stochastic Calculus. Spriger-Verlag, New York, Secod Editio, 99. [22] M.E. Ketzeberg, G.C. Souza, ad V.D.R. Guide. Mixed assembly ad disassembly operatios for remaufacturig. Productio ad Operatios Maagemet, 2(3):32 335, 23. [23] L. Kruk, J. Lehoczky, K. Ramaa, ad S. Shreve. A explicit formula for double reflected processes i [, a]. Aals of Probability, 35:74 768, 27. [24] T. G. Kurtz. Approximatio of populatio processes. CBMS-NSF Regioal Coferece Series i Applied Mathematics, 36, 98. [25] C. Maglaras ad A. Zeevi. Pricig ad capacity sizig for systems with shared resources: approximate solutios ad scalig relatios. Maagemet Sciece, 49:8 38, 23. [26] P. Majumder ad H. Groeevelt. Competitio i remaufacturig. Productio ad Operatios Maagemet, (2):25 4, 2. [27] S.P. Mey. Sequecig ad routig i multiclass queueig etworks. part II: Workload relaxatios. SIAM J. Cotrol ad Optimizatio, 42():78-27, 23. [28] E.L. Plambeck. Optimal leadtime differetiatio via diffusio approximatios. Operatios Research, 52:23-228, 24. [29] E.L. Plambeck ad A.R. Ward. Optimal Cotrol of a high-volume assemble-to-order system. Mathematics of Operatios Research, 3: , 26. [3] G.C. Souza, M.E. Ketzeberg, V.D.R. Guide, Jr. Capacitated remaufacturig with service level costraits. Productio ad Operatios Maagemet, : , 22. [3] A. V. Skorokhod. Stochastic equatios for diffusios i a bouded regio. Theory of Probability ad its Applicatios, 6: , 96. [32] L.B. Toktay, L. M. Wei, S. Zeios. Ivetory maagemet of remaufacturable products. Maagemet Sciece, 46():42-426, 2. [33] J. Vorasaya, S.M. Rya. Optimal price ad quatity of refurbished products. Productio ad Operatios Maagemet, ,

44 [34] A. Ward ad S. Kumar. Asymptotically optimal admissio cotrol of a queue with impatiet customer. Mathematics of Operatios Research, Vol 33 ():67 22, 28. [35] A. Weerasighe. A Abelia limit approach to a sigular ergodic cotrol problem. SIAM J. Cotrol Optim. 44 (25),

45 7 Appedix Proof of Propositio 3.2: Cosider the fuctio Q defied o [, ) by Q(x) = 2 σ 2 x e 2u σ 2 r r (γ(u) h(y))e 2u σ 2 y dydr. (7.85) The Q satisfies σ 2 2 Q uq + h(x) = γ(u) for x > ad Q () =. (7.86) Usig (3.22) ad (7.85), we also obtai Q (x) = 2 σ 2 e = 2 σ 2 e 2u σ 2 x x 2u σ 2 x x (γ(u) h(y))e 2u σ 2 y dy (h(y) γ(u))e 2u σ 2 y dy. (7.87) Therefore, Q (x) > for all x ad Q() =, ad cosequetly Q(x) > for all x >. Next we cosider the process X (t) i (3.9), ad itroduce the stoppig time τ N for each N as follows: if{t : X (t) N}, τ N = +, if the above set is o-empty, otherwise. Let a b. = mi{a, b}. We apply Itô s lemma to Q(X (t τ N )) ad use (7.86) to obtai t τn E[Q(X (t τ N ))] + E h(x (s))ds = Q(x) + γ(u)e[t τ N ]. (7.88) Next we ited to estimate E[Q (t τ N )]. Usig (7.87), we have ( ) 2 < Q (x) e σ 2 2u σ 2 x x h(y)e 2u σ 2 y dy. (7.89) But h( ) has polyomially-bouded growth ad, therefore, for ay < ɛ < u, we have < h(y) < σ 2 K ɛ e ɛy for all y >. Here, K ɛ > is a positive costat which may deped o ɛ >. Combiig 45

46 this with (7.89), we obtai < Q (x) ( 2/σ) ( ) 2 K ɛ e ɛx /( 2u ɛ), for all x. Upo itegratio, σ 2 we have < Q(x) ( 2/σ) ( ) 2 K ɛ e ɛx /ɛ( 2u ɛ) for all x. Therefore, σ 2 E[Q(X (t τ N ))] K ɛ E[e ɛx (t τ N ) ], (7.9) where K ɛ is a positive costat. We choose δ = 2ɛ ad apply Itô s lemma to obtai, E[e δx (T τ N ) ] = t τn ( ) e δx σ 2 + δe 2 δ u e δx(s) ds + δe[l (t τ N )] e δx + δe[l (t τ N )]. By (3.9), E[L (t τ N )] = ue[t τ N ] + E[X (t τ N )] x. Hece, E[e δx (T τ N ) ] e δx + u δ T + δ E[X (t τ N )]. Notice that E[ X (T τ N ) ] + E[X (T τ N ) 2 ] ad agai with the use of Itô s lemma, we obtai t τn E[ X (T τ N ) 2 ] = x 2 + σe[t 2 τ N ] 2uE X (s)ds x 2 + σt. 2 Cosequetly, E[e δx (T τ N ) ] e δx + δx 2 + (uδ + σ 2 δ)t + δ. (7.9) Sice δ = 2ɛ, we have E[e ɛx (T τ N ) ] E[e δx (T τ N ) ] e δx + δx 2 + δ(u + σ 2 )T + δ Usig (7.9), (7.9) ad the above estimate, we obtai E[Q(X (t τ N ))] K ɛ e 2ɛx + 2ɛx 2 + 2ɛ(u + σ 2 )T + 2ɛ 46

47 We use this estimate together with (7.88) to obtai T τn E h(x (s))ds γ(u)e[t τ N ] Q(x) + E[Q(X (t τ N ))] Q(x) + K ɛ e 2ɛx + 2ɛx 2 + 2ɛ(u + σ 2 )T + 2ɛ. Next we let τ N icrease to + ad divide it by T to obtai lim T T E T h(x (s))ds = γ(u). Proof of Lemma 3.5: Fix u, Notice that Φ(, u) give i (3.34) satisfies the differetial equatio σ 2 2 Y uy αy + x 2 = for x > ad Y () =. (7.92) Sice u is fixed, we relabel Φ(x, u) by Φ(x). We apply Itô s lemma to Φ(X (t))e αt ad usig (7.92) we obtai T τn E[Φ(X (T τ N ))e α(t τ N ) ] = Φ(x ) E e αt X (t) 2 dt, (7.93) where (τ N ) is a sequece of stoppig times defied by if{t : X (t) N}, τ N = +, if the above set is o-empty, otherwise. Usig (3.34), we observe that Φ(x) H( + x 2 ), for all x, where C > is a geeric costat. Therefore, [ ] E Φ(X (T τ N )) e α(t τ N ) H( + X (T τ N ) 2 )e α(t τ N ). (7.94) To prove the assertio of the lemma, we ited to show that lim lim E [ Φ(X (T τ N )) e α(t τ N ) ] = T N ad use it together with (7.93). For this, first we apply Itô s lemma to X (t) 2 e ɛt for ay fixed ɛ > ad obtai the upper boud E [ X (T τ N ) 2 e ɛ(t τ N ) ] x 2 + σ2 ɛ N ted to ifiity ad usig Fatou s lemma, we have E [ X (t) 2 e ɛt] x 2 + σ2 ɛ for ay t >. By lettig for ay t >. Usig 47

48 this estimate together with Itô s lemma for X 4 (t)e ɛt, we obtai [ ] T τn E X(T 4 τ N )e ɛ(t τ N ) x 4 + 6σE 2 x 4 + 6σ 2 x 4 + 6σ 2 T ( x 2 + σ2 ɛ X 2 (s)e ɛs ds E[X(s)e 2 ɛs ]ds ) t. Next, we use Hölder s iequality ad obtai [ ] E X(T 2 τ N )e α(t τ N ) [ E [X ]] (T 4 τ N )e α(t τ /2 [ N ) E (e )] α(t τ /2 N ) [ ( ) ] /2 x 4 + 6σ 2 x 2 + σ2 [ ( )] T E e α(t τ /2 N ). α By lettig N ted to ifiity, τ N ad thus [ ] [ ( ) ] /2 lim E X(T 2 τ N )e ɛ(t τ N ) x 4 + 6σ 2 x 2 + σ2 T e α 2 T. N α Combiig this with (7.93) ad (7.94) we obtai Φ(x ) E T [ e αt X(t)dt 2 c e αt + [ ( ) ] /2 ] x 4 + 6σ 2 x 2 + σ2 T e α 2 T α By lettig T ted to ifiity, right had side teds to zero ad the assertio of the lemma follows. Proof of Propositio 4.3: From the represetatios of ˆX, ˆX 2 i (4.53)-(4.54) we get (4.64), usig the properties of the Skorohod map defied i Lemma 4.. To verify (4.65), first ote that from the fuctioal cetral limit theorem for Poisso processes ad Assumptio 2., we have ( ˆM, ˆM 2, ˆM 4 ) (M, M 2, M 4 ) as, (7.95) where M, M 2, M 4 are three idepedet Browia motio startig from, with drift ad diffusio 48

49 parameters λ N, µ, λ R respectively. Also ote that, if we defie ζ (t). = Φ ([t]) β[t], for t,, the it follows that ζ ( ) Z( ), as, (7.96) where Z is a driftless Browia motio, startig from zero with diffusio parameter β( β). From (7.95) it follows that ( ) M2, M 2 ( ) (, ), as. (7.97) Also ote that from the (4.64) ad properties of Skorohod maps i (4.57), (4.56), Assumptio 2.4, (4.52) ad (7.97) that ( [ x ˆL ( ) = ψ (µ λ N)e( ) + M ( ) ]) M 2 ( ), as. (7.98) Hece, usig Assumptio 2., we have θ (t) = N 2 ( t ) µ { ˆX (s)>} ds = ˆM 2 (t) + µ t ˆL (t) µe( ), as,(7.99) usig (7.97) ad (7.98). Therefore, by a radom time chage theorem (see Sec. 4 of [4]) we obtai ˆM 3 ( ) = ζ (θ ( )) Z( µ ). = M 3 ( ), as, (7.) where M 3 is a driftless Browia motio startig from with diffusio parameter µβ( β). It is easy to verify that M 3 is idepedet of M ad M 4 ad sice ˆM 2, ˆM 3 = β ˆM 2, ˆM 2, we have from (7.95) that M 2, M 3 (t) = β µt. Hece, defiig W = M M 2, W 2. = M3 M 4, we see that W = (W, W 2 ) satisfies the descriptio i (3.9). Also, by the covergece results i (7.95) ad 49

50 (7.) as well as the fact that the limits are cotiuous, we obtai (Ŵ, Ŵ 2 ) (W, W 2 ), as. This completes the proof of (4.65). From (4.65) ad usig Skorohod embeddig theorem, we have that Ŵ W almost surely uiformly o compacts, ad hece by (4.64) ad (4.59), the claim i (4.66) follows. The last statemet of the propositio follows trivially from the properties of the Skorohod map defied i Lemma 4.. Proof of Lemma 4.4 Cosider a sequece of radom variables {Y }, where for each, X follows a Geometric distributio with parameter a, ad let X be a Expoetial radom variable with parameter a. Sice a a as, it is easy to verify that ( a ) e a, as. Straightforward calculatios usig the above yields X X. Therefore for all M >, usig the cotiuity of c( ), we have [ ( ) X E h ] { } X M E [ h(x) {X M} ] as. Hece, for all M >, a k M ( ) ( k h a ) k M a h(x)e ax dx, as. (7.) Note that there exists M > such that h(x)e ax decreases as a fuctio of x for all x M. Fix ay ɛ >. There exists M M such that a M h(x)e ax dx < ɛ 3. (7.2) 5

51 Sice M M, we have that the itegrad below is decreasig ad so from (7.2) k M+ ( ) ( k h a ) k M h(x)e ax dx. = b M. (7.3) Fix (M) such that a a ɛ/(3b M ), for all (M). From (7.2) ad(7.3) we have for all (M), a k M+ ( ) ( k h a ) k a a + a k M+ k M+ h a a b M + a ( ) ( k h a ) k ( k ) ( a ) k M h(x)e ax dx 2ɛ 3.(7.4) Hece, from (7.2) ad (7.4) we have that for all ɛ >, there exists large M >, such that a k M+ ( ) ( k h a ) k a M h(x)e ax dx < ɛ, (7.5) for (M). From (7.) ad (7.5), the proof of the lemma is complete. 5