Heavy Traffic Analysis of a Simple Closed Loop Supply Chain

Transcription

1 Heavy Traffic Aalysis of a Simple Closed Loop Supply Chai Arka Ghosh, Sarah M. Rya, Lizhi Wag, ad Aada Weerasighe April 8, 2 Abstract We cosider a closed loop supply chai where ew products are produced to order ad retured products are refurbished for resellig. The solutio to a price-settig problem eforces the heavy traffic coditio, uder which we address the productio rate cotrol problem for two types of cost fuctios. We solve a drift-cotrol problem for a approximate system drive by a correlated two-dimesioal Browia motio. The solutios to this system are the used to obtai asymptotically optimal cotrol policies. We also coduct a umerical study to explore the effects of differet parameters o the optimal productio rates ad the resultig costs. MSC2 Subject Classificatio: Primary: 6K25, 6K2, 93E2; Secodary: 6K, 6K3. OR/MS subject classificatio: Primary: Queues - Diffusio Models, Queues - Limit Theorems, Queues - Networks, Queues - Optimizatio; Secodary: Ivetory/Productio - Policies - Marketig/pricig. Ruig Title: Heavy Traffic Aalysis of Closed Loop Supply Chai Departmet of Statistics, Iowa State Uiversity, Ames, IA 5 Departmet of Idustrial ad Maufacturig Systems Egieerig, Iowa State Uiversity, Ames, IA 5 Correspodig author. smrya@iastate.edu, Phoe: (55) Departmet of Idustrial ad Maufacturig Systems Egieerig, Iowa State Uiversity, Ames, IA 5 Departmet of Mathematics, Iowa State Uiversity, Ames, IA 5

2 Itroductio We cosider a queueig etwork model of a sigle firm that ca cotrol its productio rate of ew products but ot their price i a competitive market. It produces ew products to order. It allows customers to retur some products after sale ad refurbishes the returs for resale at a price that it chooses to balace the demads for ew ad refurbished products. The refurbished products are held i ivetory. We assume that a customer is willig to wait while a ew product is produced to her specificatios, but potetial buyers of refurbished products are impatiet. While stylized, the model captures essetial elemets of a firm like Dell, Ic., which assembles ew products to order, offers a geerous retur policy, ad sells its stock of refurbished products i a olie store. The relevat costs are associated with keepig customers waitig for ew products, maitaiig capacity to maufacture at a give rate, ad losig potetial sales of refurbished products. We derive a asymptotically optimal policy, which cosists of the productio rate for ew products ad the relative price of refurbished products, for this closed loop supply chai i heavy traffic. Closed loop supply chais that ecompass productio, distributio, product returs, reprocessig ad resale have gaied icreasig attetio recetly for both evirometal ad ecoomic reasos. Reprocessig typically retais some of the value added by the origial maufacturig process while prevetig potetially harmful disposal ad coservig both material ad eergy. To the origial producer or a third party, reprocessig ad resellig products ca yield profits by reducig the cost of providig a fuctioal product ad expadig the market. The status of havig bee sold ad retured may reduce the attractiveess of reprocessed products, yet a discouted price ca create a lower-ed market segmet of cosumers who are ot willig to pay the full price for a ew product but will accept a reprocessed oe for a reduced price. This price should be low eough to make reprocessed products attractive compared with ew oes ad prevet their ivetory from accumulatig. O the other had, too low a price for refurbished products could caibalize the demad ad profits eared by ew products. Optimal pricig strategies for remaufactured goods have bee aalyzed i differet cotexts [8,,, 26]. Collectig or receivig ad the refurbishig ad resellig products itroduces ucertaities i additio to those already preset i maufacturig ad sellig ew products. The availability of previously distributed products for refurbishmet is subject to purchasers decisios o whether ad whe to retur them. Variabil- 2

3 ity of the demad ad product flows ca create cogestio or shortages that reduce the efficiecy ad ecoomic viability of the closed loop supply chai. Queueig models have bee employed i a umber of studies to aalyze the effectiveess of closed loop supply chai maagemet policies uder steady state coditios [4, 22, 3, 32], which imply o-egligible idle times i the service facilities. However, may maagers recogize that idleess may reduce profit ad prefer to utilize expesive processig resources as fully as possible by settig prices to icrease demad. Such high utilizatio correspods to heavy traffic i the queueig model. I recet years, several authors (e.g., [, 2, 3, 6, 7, 2, 27, 29]) have employed heavy traffic approximatios of various physical queueig etworks ad used techiques from stochastic cotrol theory to obtai good queueig cotrol policies. We examie the two decisio variables, price ad productio rate, sequetially. First, we formulate a price-settig problem (also kow as the static plaig problem) to maximize profit i the fluid-scaled system. The optimal solutio aturally imposes the heavy traffic coditios (see [29] for a similar aalysis). The heavy traffic coditios require that the arrival rates ad the service rates of the queueig system are balaced i some sese. We show that a solutio to a profit maximizatio problem for the fluid scaled queueig system (the so-called static plaig problem) aturally imposes the heavy traffic assumptio o our model. Ituitively, this ca be iterpreted as follows: if the maufacturer decides to maximize profit based o the average behavior of the system, the optimal prices will eforce that the arrivals (fuctios of the price-variable) match the services ad, hece, satisfy the heavy traffic coditios. Secod, uder the heavy traffic coditios, we solve the problem of fidig a optimal productio rate to miimize a appropriate cost fuctio. Such heavy traffic aalyses ofte follow a sequece of steps outlied by Harriso [7] (see also [5]), which ivolves solvig a diffusio cotrol problem (called the Browia Cotrol Problem or the BCP) that approximates the queueig cotrol problem ad the iterpretig its solutio to obtai meaigful cotrol policies for the origial queueig cotrol problem. I this paper, we cosider two commo forms of cost fuctioals: log-ru average (ergodic) cost ad the ifiite horizo discouted cost. For each of these cost fuctios, we carry out the aalysis followig Harriso s scheme: we first formulate ad solve the BCP; the we propose a cadidate for optimal cotrol policy for the queueig model by iterpretig the solutio of the BCP; ad fially, we prove the asymptotic optimality of the proposed policy usig weak covergece methods. We also discuss 3

4 some comparative statics ad carry out a umerical study to explore the effect of system parameters o the optimal productio rates ad resultig costs. The mai cotributios of this paper are the followig: This is the first paper to our kowledge that successfully applies heavy traffic machiery to optimize performace of a closed loop supply chai. A atural price settig problem is show to eforce heavy traffic coditios i such a supply chai. This paper also provides complete heavy traffic aalysis to obtai optimal productio rates uder the two most commo cost fuctios i the cotrol literature. Despite the existece of a large literature for heavy traffic aalysis of queueig etworks, most articles with such provably optimal solutios focus o oe-dimesioal problems. There are very few such complete aalyses for two-dimesioal models prior to this oe (see [3, 7]). This article provides oe such aalysis for a two-dimesioal model where the associated diffusio model is drive by a two-dimesioal correlated Browia motios. Havig solved the diffusio cotrol problem, we establish the mai asymptotic optimality results usig properties of a appropriate Skorohod map (regulator map) ad weak covergece techiques. The rest of the paper is orgaized as follows: I Sectio 2, we describe the model ad the cotrol problems i detail. Next, we discuss the static plaig problem ad the heavy traffic coditios for our queueig etwork. Our mai theorems (Theorems 2.7 ad 2.8) describig asymptotically optimal policies are also stated i this sectio. I Sectio 3, we address the two BCPs for two differet choices of the cost fuctio. Sectio 4 cotais weak covergece aalysis to prove the mai results. Sectio 5 cotais some comparative statics ad umerical aalysis of the two cost problems. Fially i Sectio 6, we summarize the paper, provide a compariso of our results with the steady-state aalysis of similar models uder the average cost fuctioal ad coclude with possible extesios to this work. A Appedix cotais proofs of some of the more stadard results that are used i our aalysis. 2 Problem Descriptio We study a simple model of a closed loop supply chai i which a producer maufactures ew products to order. Some ew products are retured by the customers after evaluatio. We assume that ay ew product may be retured after sale with probability β (, ). These retured 4

5 products ca o loger be sold as ew. Istead, they are ispected, refurbished ad placed ito ivetory to be resold (see Figure ). As i Vorasaya ad Rya [33], we assume the producer is a price-taker i the market for ew products, whose exogeously-determied price is p N, ormalized so that < p N <. It sets the price for refurbished products, p R, such that p R < p N. Cosumer (ormalized) valuatio of ew products, deoted as p, is uiformly distributed o (, ). A cosumer who is willig to pay a price p for a ew product is willig to pay at most δp for a refurbished product, where < δ <. Give the prices, the cosumer chooses betwee ew ad refurbished products to maximize his/her surplus: max{p p N, δp p R, }. If p R δp N the δp p R < p p N for ay p < ; therefore, we assume p R < δp N to guaratee some demad for refurbished products. Likewise, we assume p R p N ( δ) because otherwise, p p N < δp p R for ay p < ad o demad would exist for ew products. A strategic decisio variable for the producer is ρ p R p N, (2.) such that ρ ( δ p N, δ). I terms of this price ratio, the ormalized demad rate for ew products represets the proportio of a fixed umber of customers per uit time who will buy the ew product, i.e., those for whom p > p N ad p p N > δp ρp N, ad is give by: λ N (ρ) = p N( ρ). (2.2) δ The correspodig demad rate for refurbished products is λ R (ρ) = p ( N ρ ), (2.3) δ δ which represets the proportio of customers for whom δp > p R ad p p N < δp ρp N. I our model, the demads for ew ad refurbished products follow Poisso processes with the rates λ N (ρ) ad λ R (ρ) for a chose value of ρ. These ad other parameters are costat over a implicit study horizo represeted by the model, which is reasoable for a product category such as busiess laptop, but ot iteded for specific models withi that category. We assume that the time required to produce a ew product is expoetially distributed with rate µ > ad that 5

6 the maufacturig server is ot allowed to idle uless the queue of ew product orders is empty. Whe a demad for a refurbished item arrives, if such a product is available i ivetory the the demad is satisfied; otherwise, the customer is lost. Let X (t) deote the legth of the ew product customer queue ad X 2 (t) deote the umber of refurbished products i ivetory at time t. The, give X i () = x i, i =, 2, we model X ad X 2 as: ( t ) X (t) = x + N (λ N (ρ)t) N 2 µ {X (s)>}ds, (2.4) ( t )] ( t ) X 2 (t) = x 2 + Φ [N 2 µ {X (s)>}ds N 3 λ R (ρ) {X2 (s)>}ds, (2.5) where N i ( ), i =, 2, 3, are idepedet uit Poisso processes. For ay oegative iteger m, Φ(m) = m k= φ k, where {φ k } is a sequece of i.i.d Beroulli(β) radom variables. Here ad for the rest of the paper, A, will deote the idicator fuctio of a Borel set A (i.e. A (x) = if x A ad A (x) =, if x / A). I the above display, Φ(m) represets the (radom) umber of products that are retured by customers out of the first m purchased products. See Chapter 6 of [24] to see a more geeral costructio of jump-markov process, with state space Z, as a Figure : Closed loop supply chai etwork liear combiatio of time chaged versios of uit Poisso process as i (2.4) ad (2.5). We assume that all retured products are refurbished ad that retur, if it occurs, ad refurbishmet 6

7 are both istataeous. This assumptio approximates the situatio where returs are maily due to buyer remorse or umet expectatios rather tha ay real defect, so that refurbishmet amouts to ispectio or testig ad repackagig. I Sectio 6 we describe extesios to icorporate expoetially-distributed delays i retur ad/or refurbishmet as well as disposal of some fractio of returs. Defie processes L ad L 2 as follows: t L (t) = µ t {X (s)=}ds, L 2 (t) = λ R (ρ) {X2 (s)=}ds. (2.6) The parameter µ represets the average umber of ew products that ca be maufactured per uit time. The process L ( ) is defied as µ multiplied by the time that the maufacturig server has idled so far. I that sese, L (t) represets the average umber of ew-product customers that could have bee served i the iterval [, t] durig the server s idle time whe o ew-product customers are waitig. Usig a similar iterpretatio, L 2 (t) captures the average umber of lost sales of refurbished products. We assume that the cost for storig refurbished products is maily fixed with respect to quatity, ad therefore little affected by policies that ifluece X 2 (t). Ideally, we prefer policies which produce fewer lost sales of refurbished goods. This preferece is reflected i the defiitios of the cost fuctioals i (2.2)-(2.3) below (see Sectio 6 for possible extesios of the model). Our goal i this paper is to optimize () the price of refurbished products relative to ew products ad (2) the productio rate of ew products. We carry out this optimizatio i two steps. First, we solve a static plaig problem i terms of the fluid-scaled processes ad the log term average demad rates, which are assumed to satisfy (2.2) ad (2.3). The profit is maximized by settig the price ratio ad log term average productio rate so the system is i heavy traffic. Secod, we carry out a heavy traffic aalysis of the system, ad fid a asymptotically optimal service rate uder optimal prices. As it is commoly doe for such aalysis, we will cosider a sequece of etworks (idexed by a parameter ), each havig the same structure, but the parameters of the -th etwork deped o the idex, ad we will require that as, the system achieves heavy traffic (see Assumptio 2. ad 2.4 below). A physical etwork that is close to beig i heavy-traffic ca be thought of as oe elemet of this sequece with a large value of. Hece, from ow o, we will cosider a 7

8 sequece of etworks idexed by ad all the processes ad parameters deped o (deoted by a superscript, e.g., λ N (ρ), X (t), etc.). We assume that ρ does ot deped o (i.e., ρ ρ), ad its optimal value will be determied by the limit behavior of the system (i the static plaig problem). Note that, sice this queueig model is a Jackso etwork, the queue legths of each etwork i the sequece ca be aalyzed exactly i steady-state. I Sectio 5, we illustrate how the result of such a prelimit aalysis coicides with the asymptotic aalysis for a special case of the log-ru average cost fuctio. However, the discouted cost depeds o the trasiet behavior of the queue legths, ad we lack ay exact characterizatio of the arrival process to the secod queue durig the trasiet phase. A policy cosists of the price ratio ρ (, ) as defied i (2.) ad the maufacturig rate sequece {µ }. We assume the followig basic covergece properties for the parameters of this model: Assumptio 2. There exist θ i IR, i =, 2, 3, { λ N (ρ) >, ρ [, ]}, { λ R (ρ) >, ρ [, ]}, µ >, ad x, x 2 such that (i) (ii) (λ N (ρ) λ N (ρ)) θ 2, (λ R (ρ) λ R (ρ)) θ 3 for all ρ [, ], (2.7) (µ µ) θ, (2.8) (iii) βθ 2 = θ 3 ad ˆx i = x i / x i as, i =, 2. (2.9) Remark 2.2 The assumptios i (2.7) state that there are log-ru average rates (for arrivals) to which the parameters of the -th system coverge. They also specify that this covergece takes place at the rate of θ i, where θ i, i = 2, 3, are the covergece rates, which is the atural rate of covergece for heavy traffic assumptio of diffusio scaled systems. It has bee show (for cotrol problems ivolvig liear holdig costs) that for ay admissible policy {µ } that produces fiite asymptotic costs (see (2.2) ad (2.3) below), (2.8) holds (see [34]). So i our aalysis, we restrict cosideratio to admissible cotrols that satisfy (2.8). The first part of (2.9) is a techical assumptio that reduces the problem dimesio: because of this, the limitig diffusio cotrol problem is effectively oe-dimesioal. To be more specific, the terms u ad ũ i (4.53)- (4.54) coverge to the same costat u (usig (4.5)) which is the drift parameter goverig both the 8

9 processes i the limitig model (3.9). Existece of such asymptotic limits is a stadard assumptio i heavy traffic aalysis. We will carry out the asymptotic aalysis of the diffusio scaled queueig model. Therefore, we eed to defie the diffusio scale before itroducig the cost fuctioal. The aalysis also ivolves the so-called fluid-scaled processes. For ay process ψ ( ) described here, ψ ( ) ad ˆψ ( ) will deote the fluid- ad diffusio-scaled processes respectively, give by: ψ (t) = ψ (t), ˆψ (t) = ψ (t), for all t, =, 2,.... (2.) I this paper, we aalyze two types of cost fuctioals: the log-ru average cost (also kow as the ergodic cost ) ad the ifiite horizo discouted cost, each of which ivolves the followig compoets: a cotrol cost for the service rate, a backorder cost for ew products, ad a liear cost per lost customer of refurbished products. Here we assume that the ivetory of the refurbished products does ot icur ay variable cost for the maufacturer; hece, the cost fuctioals iclude o holdig cost for the refurbished products. These compoets of costs are give i terms of fuctios c( ), h( ) ad a costat pealty rate k, which satisfy the followig assumptios. Assumptio 2.3 The fuctios c( ) ad h( ) are oegative, cotiuous, odecreasig ad covex o [, ) ad k is a positive costat. Also, c(x) =, for x ad there exist K ad N > such that h( ) satisfies h(x) K( + x N ) for all x. Let u (ρ) = (µ λ R(ρ)/β), ρ [, ]. (2.) Uder the heavy traffic coditios described i Sectio 2., the quatity (βµ λ R (ρ)) ca be thought of as the et ivetory growth rate for the refurbished products i the -th system. This quatity teds to zero at the rate as approaches ifiity, as show i (4.5)) i Sectio 4. Here, u defied i (2.) captures this rate. As argued i Remark 3., the limit of u is assumed to be o-egative. Hece, for simplicity, we oly allow o-egative values for u (ρ) for all 9

10 ad ρ [, ]. The log-ru average cost is give by : Î (x, x 2, ρ, {µ }). = lim if lim sup = lim if T lim sup T T [( ) ] c (u (ρ)) + h( ˆX (s)) ds + k dˆl 2 (s) T E [c (u (ρ)) + T T E h( ˆX (s))ds + k ] T E ˆL 2 (T ).(2.2) For a fixed discout rate α >, the ifiite horizo discouted cost is give by: Ĵ (x, x 2, ρ, {µ }). = lim if E = lim if c (u (ρ)) α ( ) ) ] 2 e [(c αs (u (ρ)) + ˆX (s) ds + k dˆl 2 (s) + E [ ( ] 2 e αs ˆX (s)) ds + k dˆl 2 (s). (2.3) I this paper we solve the ifiite horizo discouted cost problem with backorder cost h(x) = x 2. The term c(u (ρ)) i (2.2) ad (2.3) represets the cost of choosig productio rate µ relative to the arrival rate for refurbished products (suitably scaled), while the term h( ˆX (s)) is the cost per backorder (of ew items) per uit time. The ifiitesimal quatity, k dˆl 2 (s), is the pealty for lost sales of refurbished items. Here x ad x 2 are the (asymptotic) iitial legths of the backorders of ew items ad ivetory of refurbished items as defied i (2.9). Note that the choice of u uiquely determies the service rate sequece µ. Sice c(x) = for all x, for each the cotrol cost ca be thought of as c(µ ) =. c(u (ρ)) = c( (µ λ R (ρ)/β)), which is a icreasig fuctio of µ, for each fixed λ R (ρ). Prior to aalysis, for both cotrol problems with costs (2.2) ad (2.3), it is ot clear which amog the three compoets of the cost is domiat. Eve if the cost fuctios ivolve oly diffusio-scaled processes, to be able to carry out the aalysis oe eeds to have the fluid system stable. Hece, we defie the static plaig problem below, ad deduce the coditios for heavy traffic. The process of solvig the static plaig problem also solves the problem of price settig (i.e., choosig ρ). 2. Static Plaig Problem Static plaig problems are formulated by costructig a system where the fluid-scaled processes are replaced by their log-ru averages (or fluid limits) ad solvig a suitable optimizatio problem ivolvig those averages (see [9, 25, 28, 29]). I the fluid limit, we formulate a determiistic

11 problem to choose ρ ad µ that maximize the profit rate subject to stability coditios o both queues. The profit cosists of reveue from the sale of both ew ad refurbished returs less the cost per uit time associated with producig ew products at rate µ. Let γ( ) be a odecreasig fuctio. The profit maximizatio problem is: max ρ, µ p N ( β) λ N (ρ) + p R β λ N (ρ) γ( µ) s.t. λn (ρ) µ β λ N (ρ) λ R (ρ). The first term i the objective fuctio is reveue per uit time from the sale of ew products (less a refud for retured products), whose sales are limited by demad. The secod term is reveue per uit time from the sale of refurbished products, whose sales are limited by supply. The first costrait esures that supplies of ew products are sufficiet to meet the demad o average. The secod costrait restricts the supply of refurbished products to ot exceed their demad; otherwise, ivetories of refurbished products would accumulate without boud. The objective is separable ito its reveue ad cost compoets, where reveue depeds oly o ρ ad cost depeds oly o µ. Clearly, the optimal limitig productio rate equals its lower boud: µ = λ N (ρ ), where ρ is the optimal price ratio. The reveue rate is proportioal to: ( β + βρ) λ N (ρ) = ( β + βρ)( p N δ + p N δ ρ), which is a covex quadratic fuctio of ρ that is miimized by ρ = p N( 2β) β( δ) 2βp N. The largest feasible value for ρ is foud uiquely by solvig the secod costrait as a equality: ρ = δ [p N( + β) β( δ)]. p N ( + δβ) It is easy to verify that ρ ρ for ay p N > if β < 2 (i fact, ρ < i this case). Therefore, reveue is a icreasig fuctio of feasible ρ, so that ρ = ρ. The uique solutio to the static

12 plaig problem is give by (ρ, µ ) such that λ N (ρ ) = β λ R (ρ ) ad µ = λ N (ρ ). This relatio costitutes the heavy traffic coditio, ad for the rest of the paper, we will assume that all admissible policies satisfy this coditio, i additio to Assumptio 2.. See Figure 2. Figure 2: Price determiatio from static plaig problem. Assumptio 2.4 (Heavy Traffic) Ay admissible policy satisfies µ = λ N (ρ ) = β λ R (ρ ). There are differet equivalet methods of arrivig at the above described heavy traffic assumptio eeded for aalyzig diffusio-scaled systems (see [5, 7]), ad it ca be verified that those methods also yield the same heavy traffic coditio as we have here. For example, oe covetioal way for defiig heavy traffic (see [8]) is to require that the followig holds: There exists a uique optimal solutio ( r, x ) satisfyig r = ad A x = to the followig liear program, miimize r subject to R x = α, A x r ad x. Here the decisio variables x represet average rates at which activities are udertake ad the objective is a vector of upper bouds o the utilizatio rates for processig resources; the costats R, A ad α are related to the parameters of the etwork: The average rates of arrival to the two servers from outside the system are give by α (ote that i our formulatio of the ivetory process 2

13 dyamics, λ R serves as the service rate ad ot a exteral arrival rate), while the iput-output matrix R ad capacity-cosumptio matrix A are defied as α = λ N (ρ), R = µ β µ λ R (ρ), A =. See [8] for more details o this formulatio of heavy traffic. It is easy to verify that the (ρ, µ ) satisfyig the coditios i Assumptio 2.4 also satisfies the heavy traffic coditio as defied i [8]. Remark 2.5 The heavy traffic coditios alog with (2.2) ad (2.3) applied to the fluid limit imply that µ = λ N (ρ ) = p N + βδ = β λ R (ρ ). The assumptio δ p N < ρ = ρ < δ holds true for ay p N <. Defiitio 2.6 (Two queueig cotrol problems) Uder Assumptios 2. ad 2.4, the price variable is set as ρ, which determies the demad rates of ew ad refurbished products, as well as the log-ru average service rate µ. A sequece of service rates {µ } is said to be admissible if it satisfies Assumptios 2. ad 2.4 with ρ ad µ. The first queueig cotrol problem is to fid a asymptotically optimal service rate sequece {µ } that miimizes Î(x, x 2, {µ }). = Î(x, x 2, ρ, {µ }) (2.4) over all admissible cotrols {µ }. The secod queueig cotrol problem is to fid a asymptotically optimal service rate sequece {µ } that miimizes Ĵ(x, x 2, {µ }). = Ĵ(x, x 2, ρ, {µ }) (2.5) over all admissible cotrols {µ }. 3

14 The followig are the two mai theorems of this article, which show the existece of optimal cotrols for two queueig cotrol problems described i Defiitio 2.6. Theorem 2.7 There exists a u a such that µ, a = β λ R(ρ ) + u a, =, 2,... (2.6) is a asymptotically optimal sequece of service rates for the first queueig cotrol problem defied i Defiitio 2.6. Furthermore, this u a satisfies the followig: u a = argmi u e y[ ( ) σ 2 ] c(u) + h 2u y dy, where σ 2 = λ N + µ. Theorem 2.8 There exists a u d such that µ, d = β λ R(ρ ) + u d, =, 2,... (2.7) is a asymptotically optimal sequece of service rates for the secod queueig cotrol problem defied i Defiitio 2.6. The existece of u a is proved i Theorem 3.4, ad that of u d is established i Theorem 3.4. Note that, i each of the cotrol problems, the choices i (2.6) ad (2.7) are ot uique. For example, for the first problem, µ, a = λ N(ρ ) + u a, =, 2,... (2.8) is aother such choice. Note that these two choices are asymptotically equivalet, i the sese that the behavior of the diffusio-scaled system uder these two choices are the same (as a cosequece of the fact that u ad ũ defied i (4.5) are asymptotically equivalet). 4

15 3 Browia Cotrol Problems The Browia cotrol problem (BCP) for a queueig etwork is formulated by replacig the liear combiatio of cetered processes i the scaled queue equatios (see the martigale terms Ŵ i i =, 2 defied i Sectio 4) by suitable Browia motios, ad costructig a diffusio cotrol problem ([5, 6] etc.). The solutios to such cotrol problems ofte cotai useful isights about the queueig cotrol problems, ad are commoly used i such aalysis. I the ext sectio, we will establish that the sequece ( ˆX, ˆX 2 ) coverges weakly to a twodimesioal process (X, X 2 ), which is a reflectig diffusio with state space i the first quadrat of R 2. Furthermore, (X, X 2 ) satisfies the followig stochastic differetial equatios: for X (t) = x ut + σ W (t) + L (t) X 2 (t) = x 2 + βut + σ 2 W 2 (t) βl (t) + L 2 (t), (3.9) where W ( ) ad W 2 ( ) are two stadard Browia motio processes ad they are correlated. Their depedece is described by E[W (t)w 2 (t)] = rt, where r = µβ σ σ 2 ad the costats σ ad σ 2 are give by σ = λn + µ ad σ 2 = λr + µβ( β). The local-time processes L ad L 2 are o-decreasig ad satisfy L () = L 2 () =. Furthermore, t [X (s)>]dl (s) = ad t [X2 (s)>]dl 2 (s) =, for all t >. The local time processes L ad L 2 keep the state processes, respectively X ad X 2, o-egative. X (t) represets the limitig queue legth for the ew product at time t ad X 2 (t) represets the limitig ivetory of the refurbished products at time t >. Remark 3. The costat u i (3.9) is the cotrol parameter which captures the rate at which the maufacturig rate (of ew products) deviates uder diffusio scalig from the oes specified by heavy traffic. The o-egativity of u guaratees the fiiteess of the cost fuctioal i (3.2), sice otherwise X i (3.9) will be trasiet leadig to ifiite holdig costs. Hece, we focus oly o cotrols u throughout the article. 5

16 The stochastic system described i (3.9) is kow as a Browia cotrol system. I the followig two subsectios, we will cosider this system uder the two types of cost structures, ad solve the correspodig cotrol problems i each case. 3. BCP with log-ru average cost First we describe the log-term average cost structure associated with the first cotrol problem. For such a Browia cotrol system described by (3.9), we cosider the log term expected average cost fuctio give by I(x, x 2, u) = lim sup T [ T ] T E (c(u) + h(x (s)))ds + k L 2 (T ). (3.2) Recall that c(u) represets the cotrol cost, h(x (t)) represets the holdig cost for queue legth X (t) ad the costat k > represets the pealty per lost customer for refurbished products. Sice c(u) is time idepedet, the cost fuctio i (3.2) ca be writte as I(x, x 2, u) = c(u) + lim sup T [ T ] T E h(x (s))ds + k dl 2 (t). (3.2) I the followig discussio, we ited to obtai a optimal cotrol u that miimizes I(x, x 2, u) over all costat cotrols u. We ca represet the value fuctio of this stochastic cotrol problem by V (x, x 2 ) = if I(x, x 2, u). u [ ] Next, usig the equatio for X ( ), we ited to compute lim T T E T h(x (s))ds we itroduce the costat γ(u) for each cotrol u by explicitly. First γ(u) = 2u σ 2 h(x)e 2ux σ 2 dx. (3.22) Sice h( ) has polyomially-bouded growth (see Assumptio 2.3), this costat γ(u) is fiite for each u. The followig propositio coects a part of the cost i (3.2) with γ(u). The proof of this result ivolves applicatio of the Itô s formula, ad is somewhat stadard. Hece, we state this as a propositio here without proof, ad describe a proof i the Appedix. 6

17 Propositio 3.2 Let (X, X 2 ) satisfy (3.9). The lim T T E T h(x (s))ds exists ad equals γ(u). I the ext lemma, we examie how E[L 2 (T )] grows as a fuctio of T. Sice u > i (3.9), X ( ) is a reflectig diffusio process with costat egative drift u. It has a statioary distributio with expoetially decayig tail ad lim T T (L (T ) ut ) = a.s. (see [5], page 29). Therefore, the magitude of the term (ut L (t)) must be at most of the order sup s t W (s). Hece usig the equatio for X 2 ( ) i (3.9), we expect that the process X 2 ( ) does ot reach the origi that ofte. Cosequetly, L 2 (t) icreases at a much slower rate tha L (t), as t teds to ifiity. Ituitively, i the absece of a variable ivetory cost for refurbished products, the cost fuctio emphasizes customer service with the result that lost sales of refurbished products are egligible i the limit. I the followig lemma, we verify this ituitio by provig that lim T L 2 (T ) T as well as i L. = a.s. Lemma 3.3 Let L 2 be the local time process of X 2 i (3.9). The L 2 (T ) E[L 2 (T )] lim = a.s. ad lim =. T T T T Proof. L has the represetatio (see [5]) { } L (t) = max, if (x + σ W (s) us). (3.23) s t Cosider the Browia motio W ( ) ad the maximum process M defied by M (t) = sup W (s). s t The (3.23) implies that L (t) ut σ M (t), for all t. Similarly L 2 has the represetatio { } L 2 (t) = max, if (x 2 + σ 2 W 2 (s) + βus βl (s)). (3.24) s t Agai we itroduce M 2 (t) = sup W 2 (s). Notice that s t if [x 2 + σ 2 W 2 (s) + βus βl (s)] = sup [β(l (s) us) σ 2 W 2 (s) x 2 ]. s t s t 7

18 Usig the estimate L (t) ut σ M (t) for all t, we obtai β(l (s) us) σ 2 W 2 (s) x 2 βσ M (s) + σ 2 M 2 (s) βσ M (t) + σ 2 M 2 (t), for all s t. Also, βσ M (t) + σ 2 M 2 (t) for all t. Therefore, it follows that { } max, sup s t [β(l (s) us) σ 2 W 2 (s) x 2 ] βσ M (t) + σ 2 M 2 (t), for all t. Hece by (3.24), we obtai that L 2 (t) σ 2 M 2 (t)+βσ M (t), for all t >. By the properties of the maximum process of Browia motio (see pages 95 ad 2 of [2]), we kow that lim M lim 2 (T ) T T that T M (T ) T = = a.s., ad E[M i (t)] C T for i =, 2, where C is a costat. Therefore, it follows L 2 (T ) E[L 2 (T )] lim = a.s. ad lim =. T T T T Usig Propositio 3.2 ad Lemma 3.3, we ca provide the followig explicit represetatio of the cost fuctio I( ) i (3.2): I(x, x 2, u) = c(u) + γ(u) = c(u) + 2u σ 2 h(x)e 2ux σ 2 dx. (3.25) This expressio ca be simplified to obtai I(x, x 2, u) = e y[ ( ) σ 2 ] c(u) + h 2u y dy. (3.26) The above computatios establish the followig theorem. Theorem 3.4 The value fuctio V (x, x 2 ) of the log-ru average cost problem described i (3.2)-(3.2) is idepedet of (x, x 2 ) ad it has the followig represetatio V V (x, x 2 ) = if u e y[ ( ) σ 2 ] c(u) + h 2u y dy. 8

19 Furthermore, for F (u) = e y[ ( ) σ 2 ] c(u) + h 2u y dy, (3.27) a optimal cotrol u a > is give by F (u a) = mi u F (u). To compute u a we differetiate the above fuctio of u to obtai F (u) = c (u) σ2 2u 2 ( ) σ e y yh 2 2u y dy. To fid a cadidate for u a, we let F (u) =, which yields the followig ecessary coditio: 2(u a) 2 c (u a) = σ 2 ( ) σ e y yh 2 2u y dy. a I the case where both c( ) ad h( ) are covex twice differetiable icreasig fuctios, the above coditio is also sufficiet, because F (u) = c (u) + σ4 2u 4 ( ) σ e y y 2 h 2 2u y dy + σ2 u 3 ( ) σ e y y h 2 2u y dy >. For example, whe c(x) = x m ad h(x) = x q, where m ad q, we obtai u a = ( ( ) q σ 2 q ) m q! m+q. (3.28) BCP with ifiite horizo discouted cost I the previous sectio, we have oticed that the expected cost k E[L 2 (T )] which represets the pealty icurred from lost customers for refurbished products durig the time iterval [, T ] grows E[L at a rate much slower tha T as T teds to ifiity. I fact, lim 2 (T )] T T is equal to zero. For this reaso, the optimal cotrol policy developed i the previous sectio is ot iflueced by this cost compoet. To capture the effect of the pealty icurred from lost customers for refurbished products, we also cosider a ifiite horizo discouted cost structure for the same model i (3.9). I this case, the cost fuctioal as well as the optimal policy are affected by the cost compoet correspodig to the lost customers for refurbished products as well as by the iitial data x ad 9

20 x 2 of (3.9). I our aalysis of this cost structure, we use h(x) = x 2 to perform explicit computatios. A mai difficulty i our aalysis is to obtai a explicit formula for E[L 2 (T )] i this two-dimesioal model described i (3.9). For this reaso, we are able to establish a otrivial optimal cotrol u d > for the discouted cost oly whe the iitial data (x, x 2 ) belog to a certai regio i R 2. Here we aalyze the ifiite horizo discouted cost structure give by J(x, x 2, u) = E e αt [(c(u) + X (t) 2 )dt + k dl 2 (t)], (3.29) where α > ad k > are positive costats. We ca rewrite this cost fuctioal i the form J(x, x 2, u) = c(u) α + Φ(x, u) + Ψ(x, x 2, u), (3.3) where ad Φ(x, u) = E Ψ(x, x 2, u) = k E e αt X (t) 2 dt, (3.3) e αt dl 2 (t). (3.32) The value fuctio for this cotrol problem is give by Q(x, x 2 ) = if u J(x, x 2, u). (3.33) I the followig lemma, for a give cotrol u, we compute Φ(x, u) described i (3.3). The proof of the lemma is give i the Appedix. Lemma 3.5 Let Φ(x, u) be defied by (3.3). The Φ(x, u) = α ( ) where λ (u) = σ (u 2 + 2ασ 2 2 ) u. [ ( x u ) ( )] 2 σ 2 + α α + u2 α 2 2u α 2 λ (u) e λ (u)x, (3.34) For our two-dimesioal model described i (3.9), ext cosider the fuctioal Ψ give i 2

21 (3.32). Here we are uable to compute Ψ(x, x 2, u) explicitly. Therefore, we obtai a upper boud for the quatity Ψ(x, x 2, u) Ψ(x, x 2, ) i the ext lemma. Here, Ψ(x, x 2, ) represets the cost defied by (3.32) i the case of zero cotrol. To idetify the depedece of the processes o the cotrol u, we rewrite our model equatio (3.9) i the followig form: X u (t) = x ut + σ W (t) + L u (t) (3.35) X u 2 (t) = x 2 + βut + σ 2 W 2 (t) βl u (t) + L u 2(t), where L u ad Lu 2 are local time processes of Xu ad Xu 2 respectively. Next we itroduce the process X u by X u (t) = x 2 + βut + σ 2 W 2 (t) + L u (t), (3.36) where L u (t) is the local time process of Xu at the origi ad hece L u (t) for all t. Notice that (3.36) ca be rewritte as ( X u (t) = x 2 + βut + σ 2 W 2 (t) βl u (t) + βl u (t) + L ) u (t). (3.37) We compare (3.37) with the secod equatio i (3.35). Recall that the local time process L u 2 is the miimal cotiuous o-decreasig process which keeps the sum (x 2 +βut+σ 2 W 2 (t) βl u (t)+lu 2 (t)) o-egative. But i (3.37), Xu (t) for all t ad therefore we obtai the iequality L u 2(t) βl u (t) + L u (t), for all t. (3.38) This estimate of L u 2 (t) will be useful i the ext propositio. Propositio 3.6 Let the iitial value (x, x 2 ) be fixed. The the cost fuctioal J(x, x 2, u) defied i (3.29) is cotiuous i the cotrol variable u. Proof. For J(x, x 2, u), we cosider the represetatio (3.3). The fuctio c(u) is cotiuous i u ad by lemma 3.5, Φ(x, u) is also cotiuous i u. Therefore, it remais to show that Ψ(x, x 2, u) is cotiuous i the variable u. 2

22 For ay u, by (3.32) ad Fubii s theorem, we obtai [ Ψ(x, x 2, u) = k E t= [ = k E ( s=t αe αs L u 2(s)ds s= ) ] αe αs ds dl u 2(t) ]. Therefore Ψ(x, x 2, u) = α k E e αt L u 2(t)dt = α k e αt E[L u 2(t)]dt. (3.39) t= t= Next we fix u ad let {u } coverge to u. We assume that u K for some fixed costat K >. It suffices to show that local time process L u (t) has the represetatio lim Ψ(x, x 2, u ) = Ψ(x, x 2, u). For each u, the u u { } L u (t) = max, sup (us σ W (s) x ), (3.4) s t ad therefore, for each T > it is evidet that L u (t) coverges to Lu (t) uiformly o [, T ]. Next, L u 2 (t) has the represetatio { } L u 2(t) = max, sup (βl u (s) βus σ 2 W 2 (s) x 2 ). (3.4) s t Sice u u ad L u (t) ad Lu 2 (t) coverge uiformly to Lu (t) ad Lu 2 (t), respectively, for all t T, from the above represetatio (3.4)-(3.4). For each u, L u (t) x + Kt + σ sup s t W (s) ad L u (t) x 2 + β Kt + β L u (t) + σ 2 sup W 2 (s). Now let M(t) be the s t process defied by M(t) = β x + x β Kt + β σ sup s t W (s) + σ 2 sup W 2 (s). s t Usig Doob s iequality we obtai E[M(t) 2 ] C o ( + t 2 ), where C o > is a geeric costat. Hece E[M(t)] C o ( + t) ad E e αt M(t)dt <. Sice L u 2 (t) M(t) ad Lu 2 (t) coverges to L u 2 (t) a.s. as u teds to u, we ca apply the Domiated Covergece Theorem to coclude that lim E e αt L u (t)dt = E e αt L u (t)dt. 22

23 Hece, lim Ψ(x, x 2, u ) = Ψ(x, x 2, u) ad this completes the proof. Remark 3.7 Sice lim c(u) = + ad J(x, x 2, u) > c(u) u α, the above propositio guaratees the existece of a o-egative optimal cotrol u d. Next, we obtai some sufficiet coditios that guaratee a strictly positive optimal cotrol u, which leads to a o-trivial asymptotically optimal sequece of cotrols for the origial sequece of cotrolled queueig systems. Lemma 3.8 Let Ψ(x, x 2, u) ad Ψ(x, x 2, ) be as described i (3.32). The Ψ(x, x 2, u) Ψ(x, x 2, ) α β k e αt (E[L u (t)] E[L (t)])dt+k α e αt E[ L u (t)]dt+kx 2, (3.42) where L u, L ad L u are the local time processes described i (3.35). Proof. Usig (3.39), we obtai Ψ(x, x 2, u) Ψ(x, x 2, ) = kα e αt (E[L u 2(t)] E[L 2(t)])dt. (3.43) t= Next we estimate (E[L u 2 (t)] E[L 2 (t)]) for each t. By (3.38), we have E[Lu 2 (t)] βe[lu (t)] + E[ L u (t)]. O the other had, usig the secod equatio of (3.35), we have E[L 2 (t)] βe[l (t)] + x 2 = E[X 2 (t)], for all t. Therefore, E[L 2 (t)] βe[l (t)] x 2 for all t. Cosequetly, E[L u 2(t)] E[L 2(t)] β [ E[L u (t)] E[L (t)] ] + E[ L u (t)] + x 2 for all t. Thus, from this estimate i (3.43) we have (3.42). I the ext lemma, we compute the itegrals i (3.42). 23

24 Lemma 3.9 (i) For each u, α ( ) where λ (u) = σ u 2 + 2ασ 2 2 u. (ii) For each u, α ( ) where λ 2 (u) = σ β 2 u 2 + 2ασ βu. e αt E[L u (t)]dt = ) ( u 2α 2 + 2ασ 2 + u e λ (u)x, (3.44) e αt E[ L u (t)]dt = ) ( β 2α 2 u 2 + 2ασ 22 βu e λ 2(u)x 2, (3.45) Proof. First otice that α e αt E[L u (t)]dt = E e αt dl u (t). Let Q(x) = e λ(u)x, where ( ) λ (u) = σ u 2 + 2ασ 2 2 u. The Q satisfies σ 2 2 Q (x) uq (x) αq(x) =, for x > ad Q () = λ (u). Next, we cosider the first equatio of (3.35) ad apply Itô s lemma to Q(X u (t))e αt to obtai T E[Q(X u (T ))]e αt = Q(x ) λ (u)e e αt dl u (t). We let T ted to + ad obtai Q(x ) = λ (u)e e αt dl u (t). Hece (3.44) follows. The proof of (3.45) follows essetially alog the same steps by usig Q(x) = e λ 2(u)x where λ 2 (u) is give i (3.45) ad the process X i (3.36). The followig propositio follows from Lemmas 3.8 ad

25 Propositio 3. Ψ(x, x 2, u) Ψ(x, x 2, ) kβ ) [( u 2α 2 + 2ασ 2 + u + k 2α ] e λ (u)x 2ασ 2 2α e x σ [( β 2 u 2 + 2ασ 22 βu ) e λ 2(u)x 2 ] + kx 2. (3.46) The proof of this propositio is straightforward ad therefore omitted. Remark 3. The estimates we have obtaied i the proof of the above propositio also yield the followig upper boud of the cost fuctioal J(x, x 2, u) defied i (3.3): J(x, x 2, u) < c(u) α [ β + Φ(x, u) + k λ (u) e λ (u)x + ] λ 2 (u) e λ 2(u)x 2 + kx 2. (3.47) I the ext propositio, we obtai a sufficiet coditio which guaratees J(x, x 2, u) < J(x, x 2, ) where the cost fuctioal J is defied i (3.3). Propositio 3.2 Let (x, x 2 ) be the iitial data i (3.9). If there is a cotrol u that satisfies [ ( α kβ 2u ) ] [ α 2 λ (u) e λ (u)x + αk λ 2 (u) e λ 2(u)x 2 αkβσ + αkx 2 < e 2α x σ + 2u 2α α ( x u ) ] c(u), α where λ 2 (u) ad λ (u) are described i (3.45) ad (3.34), respectively, the J(x, x 2, u) < J(x, x 2, ). Proof. Usig (3.3) we observe that J(x, x 2, u) < J(x, x 2, ) if ad oly if Ψ(x, x 2, u) Ψ(x, x 2, ) < [Φ(x, ) Φ(x, u)] c(u) α. (3.48) Next we ca use the estimate (3.46) i Propositio 3.. Therefore, the iequality [ kβ ( λ (u) e λ (u)x ) ] 2ασ 2 2α 2α e x σ + k λ 2 (u) e λ 2(u)x 2 + kx 2 < [Φ(x, ) Φ(x, u)] c(u) α implies the iequality i (3.48). Usig (3.34) ad followig a straightforward computatio, we 25

26 obtai ( kβ 2u ) α 2 λ (u) e λ (u)x + k λ 2 (u) e λ 2(u)x 2 + kx 2 < 2u (x α 2 u ) + kβσ α e 2α σ 2α x c(u) α. This iequality is same as (3.47) ad hece the result follows. Remark 3.3. If x 2 ad u remai fixed ad x teds to ifiity the the right had side of the iequality i (3.47) teds to ifiity while the left had side of (3.47) teds to αk λ 2 (u) e λ 2(u)x 2 + αkx 2. Therefore, for fixed x 2 ad u, large values of x satisfy (3.47). 2. If 2u > α 2 kβ ad if there is a poit ( x, x 2 ) that satisfies 2u α ( x u ) c(u ) αkx 2 α > k ( ) β 2 u 2 + 2ασ2 2 βu, the the above iequality holds for all x x. It is a straightforward matter to check that the assumptio of Propositio 3.2 is true. Hece J(x, x 2, u) < J(x, x 2, ) for all x x. Next, we itroduce the regio A = {(x, x 2 ) : There exists u > where (x, x 2, u) satisfies (3.47) }. (3.49) This set A is o-empty as explaied i the above remark. Theorem 3.4 Let the iitial data (x, x 2 ) of (3.9) belog to the set A i (3.49). The there is a optimal cotrol u d > such that Q(x, x 2 ) = J(x, x 2, u ), where Q is the value fuctio defied i (3.33). Proof. We obtai J(x, x 2, u) < J(x, x 2, ) for some u > by usig Propositio 3.2. O the other had by (3.3), J(x, x 2, u) > c(u) α lim u + c(u) = +, there exists u 2 for all u. Sice c( ) is strictly icreasig ad > such that c(u) α > J(x, x 2, ) for all u > u 2. Cosequetly, if u J(x, x 2, u) = if <u<u2 J(x, x 2, u). By Propositio 3.6, J(x, x 2, ) is cotiuous i u variable. Therefore, there exists a u d such that < u d < u 2 which satisfies J(x, x 2, u d ) = 26

27 if u J(x, x 2, u). 4 Asymptotic Optimality Our objective i this sectio is to use the optimal cotrols derived for the BCPs i the previous sectio for the costructio of asymptotically optimal cotrols for the queueig cotrol problem (as described i Defiitio 2.6). This costructio is used to prove the mai theorems of the article, Theorems 2.7 ad 2.8. Throughout this sectio, ρ ad µ (see Assumptio 2.4) are fixed ad for simplicity of otatio, we will deote µ by µ ad omit ρ from all otatios, i.e. deote λ N (ρ ), λ N (ρ ), λ R (ρ ), λ R (ρ ) by λ N, λ N, λ R, λ R respectively. I this sectio, we will use the stadard otatio D([, ), IR) for the set of all right cotiuous fuctios from [, ) to IR with left limits. All the processes are defied o D([, ), IR) uless specified otherwise. e D([, ), IR) will deote the idetity fuctio, i.e. e(t) = t for all t. The covergece i distributio of a sequece of processes Φ ( ) to Φ( ) will be deoted as Φ Φ or by Φ ( ) Φ( ). Whe sup Φ (s) Φ(s) as, for all t, we will write that s t Φ Φ uiformly o compact sets, or uiformly o compacts. 4. Scaled processes ad a Skorohod map We have defied two types of scaligs for the various processes i (2.) above. Here we obtai coveiet represetatios for the rescaled processes that are relevat for our aalysis. Recall the defiitio of u i (2.). We defie aother similar quatity ũ below. Note that by Assumptio 2., these two quatities are asymptotically equivalet: There exists u such that u = ) (µ λ R β u, ũ = (µ λ N ) u, as. (4.5) 27

28 Next, we itroduce the martigales which are related to the Poisso processes i the heavy-traffic aalysis of the queueig cotrol problem. ˆM (t) = [N (λ Nt) λ Nt], ˆM 2 (t) = ( t ) [N 2 µ { ˆX (s)>} ds ˆM 3 (t) = [ Φ (N 2 ˆM 4 (t) = ( [N 3 ( t Ŵ (t) = ˆM (t) ˆM 2 (t), t t µ { ˆX (s)>} ds )) β λ R { ˆX 2 (s)>} ds ) t µ { ˆX (s)>} ds ], t µ { ˆX (s)>} ds ], λ R { ˆX 2 (s)>} ds ], (4.5) Ŵ 2 (t) = ˆM 3 (t) ˆM 4 (t), (4.52) for all t, =, 2,.... From the defiitio of the processes i (2.4)-(2.6) ad the diffusio scaled processes i (2.) (with a simple chage of variable formula t g(s)ds = t g(s)ds) we ca write the scaled state processes as ˆX (t) = ˆx + N (λ Nt) ( N 2 t µ { ˆX (s)>} ds ) = ˆx ũ t + Ŵ (t) + ˆL (t), (4.53) ˆX 2 (t) = ˆx 2 + ( t )) Φ (N 2 µ { ˆX (s)>} ds ( t ) N 3 λ R { ˆX 2 (s)>} ds where = ˆx 2 + βu t + Ŵ 2 (t) β ˆL (t) + ˆL 2 (t), (4.54) ˆL (t) = µ for all =, 2,..., t. t { ˆX (s)=} ds, ˆL 2 (t) = λ t R { ˆX 2 (s)=} ds, (4.55) The proof of asymptotic optimality uses the followig maps ad their properties. Lemma 4. (A two-dimesioal Skorohod map) Let u, u 2, β ad let u = (u, u 2 ). For each x, x 2 ad ad w = (w, w 2 ) D([, ), IR) D([, ), IR) with w i (), i =, 2, there exist uique q i, l i D([, ), IR), i =, 2, satisfyig the followig properties: (i) q (t) = x u t + w (t) + l (t), t, 28

29 (ii) q 2 (t) = x 2 + βu 2 t + w 2 (t) βl (t) + l 2 (t), t, (iii) l i ( ) is odecreasig, l i () = ad q i (t)dl i (t) = for i =, 2. Defie the followig maps Γ u i, ˆΓ u i, i =, 2 as follows: for a give w as above, let Γu i (w) = q i, ˆΓ u i (w) = l i, i =, 2. We will deote the map (Γ u i ( ), ˆΓ u i ( ) : i =, 2) as the Skorohod map relevat for this problem. Proof of the existece of the above map is straightforward. For x D([, ), IR) with x(), defie the followig maps: φ(x)(t) = x(t) + ψ(x)(t), where ψ(x)(t) = if mi{x(s), }, for t. (4.56) s t The above maps are called oe-dimesioal Skorohod maps i [, ) (see [23, 3]). Usig the above maps it is easy to verify that if w i D([, ), IR), i =, 2, the followig represetatios hold for the Skorohod maps defied i Lemma 4.. Γ u (w) = φ(x ue + w ), ˆΓu (w) = ψ(x ue + w ), Γ u 2 (w) = φ ( x 2 + βue + w 2 βˆγ u (w) ), ˆΓu 2 (w) = ψ (x 2 + βue + w 2 βˆγ u (w) ). (4.57) It is well kow that the maps φ ad ψ are both Lipschitz cotiuous maps i the uiform topology (see [23] for example). More precisely, for x, x. D([, ), IR) ad x T = sup x(s), we have s T φ(x) φ(x ) T C x x T, ψ(x) ψ(x ) T C x x T, (4.58) for some C >. Usig the represetatios i (4.57) above, oe ca verify that the Skorohod maps defied i Lemma 4. are cotiuous fuctios i the metric of uiform covergece o compact 29

30 sets i the followig sese: For all T > implies lim u i u i =, lim w i w i T =, i =, 2 lim Γu i (w, w 2 ) Γu i (w u, w 2 ) T =, lim ˆΓ i (w, w 2 ) ˆΓ u i (w, w 2 ) T =, for i =, 2. (4.59) This cotiuity property will be crucial for establishig some of the covergece results i the proofs below. 4.2 Weak covergece aalysis ad proof of Theorems 2.7 ad 2.8 Note that for ay admissible policy {µ }, there exists u, such that ũ ad u both coverge to u as teds to ifiity. To simplify otatio, we will use the followig abbreviatio for this sectio: For ū = (ũ, u ) ad ū = (u, u), Γ. i = Γū i, ˆΓ.. i = ˆΓū i, Γ i = Γ ū i, ˆΓ. i = ˆΓū i, for i =, 2. We start with the followig lemma, which describes equivalet represetatios of the cost fuctioals i the queueig etwork cotrol problems as well as the Browia cotrol problems i Sectio 3. Lemma 4.2 The log-ru average cost fuctioals for the queueig etwork ad the BCP i (2.2) ad (3.2), respectively, have the followig represetatio: Î(x, x 2, {µ }) = lim if where ˆγ({µ }) = [ c (u ) + ˆγ({µ }) + k lim sup T ( ) λ ( ) ( N i λ µ h N µ i= I(x, x 2, u) = c(u) + γ(u), where γ(u) = 2u σ 2 ) (ˆL ] T E 2 (T ), ) i, (4.6) h(x)e 2ux σ 2 dx. (4.6) The ifiite horizo discouted cost fuctioals for the queueig etwork ad the BCP i (2.3) ad (3.29), respectively, have the followig represetatio: Ĵ(x, x 2, ρ, {µ }) = lim if J(x, x 2, u) = c(u) α c (u ) α + E + E αe αt [ t 3 [ t ( ) 2 αe αt ˆX (s) ds + k ˆL 2 (t)] dt, (4.62) ] (X (s)) 2 ds + k L 2 (t) dt. (4.63)

31 Proof. To verify (4.6), first ote that for each fixed, ˆX { } state-space L. = j : j =,, 2,..., ad jump rates give by is a jump-markov process with λ N if j = i +, i L Q (i, j) = µ, if j = i, i L \ {}, otherwise. Straightforward calculatios (solvig the balace equatios) yields that the ivariat distributio for ˆX is that of a radom variable X, where X follows a Geometric distributio with parameter a = (λ N /µ ). Therefore, it follows that [ T ] lim T T E h( ˆX (s))ds = E [h(x )]. Note that by the assumptios o h( ), ad distributio of X, the expectatio o the right side is fiite. This proves that the represetatio i (4.6) is accurate. The represetatio i (4.6) follows from Propositio 3.2 ad Lemma 3.3. The proof of the represetatios of the discouted cost fuctioals i (4.62)-(4.63) are stadard ad similar to that of Lemma 4.4 of [2]. Propositio 4.3 The processes ˆX ad ˆX 2 i (4.53)-(4.54) satisfy ( ˆX, ˆX 2, ˆL, ˆL 2 ) = ( Γ (Ŵ ), Γ 2 (Ŵ ), ˆΓ (Ŵ ), ˆΓ ) 2 (Ŵ ). (4.64) For the processes Ŵ ad Ŵ 2 defied i (4.52), the followig covergece holds: Ŵ (Ŵ, Ŵ 2 ) (W, W 2 ) W, (4.65) where W is a two-dimesioal Browia motio as described i (3.9). Also, the followig holds. ( ˆX, ˆX 2, ˆL, ˆL 2 ) (X, X 2, L, L 2 ) ad, where (X, X 2, L, L 2 ) =. ( Γ (W ), Γ 2 (W ), ˆΓ (W ), ˆΓ ) 2 (W ), (4.66) 3

32 ad (W, W 2, X, X 2, L, L 2 ) satisfies all the coditios o the processes ivolved i defiig the BCPs i (3.9). The proof of the above propositio is somewhat stadard i the heavy traffic literature. We skip the proof here, ad preset oe i the Appedix. The followig basic lemma will be used i our proof of the mai result. A proof of this lemma ca be foud i the Appedix as well. Lemma 4.4 Let {a } be a sequece such that a a as ad h( ) be the cost fuctio used i our aalysis. The the followig holds a k= ( ) ( k h a ) k a h(x)e ax dx, as. (4.67) We will use the followig momet estimates i our aalysis to establish the covergece of the cost fuctioals. Propositio 4.5 The followig estimates hold: There exist costats C, C 2 > such that for all ad t [ E sup s t E ˆX (s) 4 ] [ ) ] 2 (ˆL 2 (t) C ( + t 2 + t 4) ad (4.68) C 2 ( + t + u ũ 2 t 2), (4.69) where u ad u are as described i (4.5). Proof. From the represetatio of ˆX i (4.66) ad (4.57)-(4.58), we have [ ] E sup ˆX (s) 4 s t [ C 4 E sup s t x ũ s + Ŵ (s) 4 ] [ C E (x ) 4 + (ũ ) 4 t 4 + E ( )] sup Ŵ (s) 4, (4.7) s t for some costat C >, idepedet of ad t. Sice {x } ad {ũ } are both coverget sequeces, by Doob s iequality for the martigale Ŵ ( ), ad usig the fact that E(Ŵ (t))4 C t 2 (for some C > ) we have the proof of the first estimate (4.68) of the propositio. 32