Revenue Management for a Multiclass Single-Server Queue via a Fluid Model Analysis

Transcription

1 OPERATIONS RESEARCH Vol. 54, No. 5, September October 6, pp ssn 3-364X essn nforms do.87/opre INFORMS Revenue Management for a Multclass Sngle-Server Queue va a Flud Model Analyss Constantnos Maglaras Columba Busness School, Columba Unversty, 49 Urs Hall, 3 Broadway, New York, New York 7, c.maglaras@gsb.columba.edu Motvated by the recent adopton of tactcal prcng strateges n manufacturng settngs, ths paper studes a problem of dynamc prcng for a multproduct make-to-order system. Specfcally, for a multclass M n /M/ queue wth controllable arrval rates, general demand curves, and lnear holdng costs, we study the problem of maxmzng the expected revenues mnus holdng costs by selectng a par of dynamc prcng and sequencng polces. Usng a determnstc and contnuous (flud model) relaxaton of ths problem, whch can be justfed asymptotcally as the capacty and the potental demand grow large, we show the followng: () greedy sequencng (.e., the c -rule) s optmal, () the optmal prcng and sequencng decsons decouple n fnte tme, after whch () the system evoluton and thus the optmal prces depend only on the total workload. Buldng on () (), we propose a one-dmensonal workload relaxaton to the flud prcng problem that s smpler to analyze, and leads to ntutve and mplementable prcng heurstcs. Numercal results llustrate the near-optmal performance of the flud heurstcs and the benefts from dynamc prcng. Subject classfcatons: revenue management; yeld management; dynamc prcng; queung; sequencng; flud models. Area of revew: Manufacturng, Servce, and Supply Chan Operatons. Hstory: Receved November 3; revsons receved December 4, August 5; accepted September 5.. Introducton The last decade has been marked by a growng nterest n the adopton of dynamc prcng strateges n such dverse areas as the arlne, hotel, and retal ndustres. In most of these cases, the frm controls a fxed capacty of resources (e.g., the number of seats n a flght) that have to be sold up to a deadlne (e.g., the flght departure tme). By dynamc prcng, we refer to the tactcal optmzaton of the prce of a product or servce (e.g., of an arlne tcket) as a functon of the remanng capacty and tme-to-go to maxmze the expected revenues extracted from these fxed resources. Ths practce, often referred to as revenue management, s supported by sophstcated nformaton systems processng large amounts of demand data, and reles on an mplct assumpton that the frm can apply such prce changes n a relatvely effcent manner. More recently, manufacturng frms have also started evaluatng the use of such tactcal economc optmzaton tools, wth one notable example comng from the automotve ndustry n the context of ts effort to market and produce custom cars n a make-to-order fashon. Broadly speakng, automoble manufacturers try to dynamcally adjust the prce, target lead tme, rebate, etc., for a new order as a functon of the exstng outstandng orders, and smultaneously select the approprate producton schedule to optmze ther proftablty. Jont use of economc and operatonal controls allows the manufacturer to be more responsve to changes n the market condtons and fluctuatons n the operatng envronment due to randomness of the demand and producton functons. Operatonally, ths rases several nterestng questons. For example, n what ways are the economc and operatonal decsons coupled? What are the benefts of dynamc prcng n a producton settng? And what are practcal and effcent prcng and sequencng heurstcs for such problems? Wth ths motvaton, ths paper studes the problem of jontly optmzng over the dynamc prcng and sequencng polces for the multproduct, sngle-server queung system. Broadly speakng, ths problem les n the nterface of stochastc network theory and revenue management, and the approach taken n ths paper combnes modellng and analyss technques from these two areas. Its results llustrate the potental benefts of jontly optmzng prcng and producton decsons, and offer some nsght on how to practcally ntegrate these two functons that tend to operate separately n many organzatons. We consder a make-to-order frm that produces multple products, that s modelled as a sngle-server multclass M n /M/ queue. The frm s assumed to operate n a market wth mperfect competton, and has power to nfluence the demand for the varous products by varyng ts prce menu. Assumng a general demand curve, lnear holdng costs ncurred by the frm, and convex capacty costs, we study the problem of fndng the optmal state-dependent prcng and sequencng strategy as well as a statc vector of producton rates to optmze the system s long-run expected proft rate. A natural startng pont would be to formulate an approprate control problem for a multclass 94

2 Operatons Research 54(5), pp , 6 INFORMS 95 queue wthn the framework of Markov decson processes (MDPs). Whle MDPs provde detaled descrptons of the system dynamcs and the optmal control problem, they are wth the excepton of very restrcted examples not amenable to exact analyss. Ths paper studes an approxmate formulaton of the proft maxmzaton problem of nterest, posed n the context of the assocated determnstc and contnuous flud model approxmaton to the underlyng stochastc producton system. Ths can be rgorously justfed through a strong-law-of-large-numbers type of scalng n settngs where the producton rate and potental demand grow proportonally large. Such models have been used successfully n the lterature both n revenue management settngs that lack producton dynamcs, and n producton systems that do not nclude the tactcal prcng decsons. The man fndngs of ths paper are the followng. () Capacty choce. We show that the long-run average proft maxmzaton problem for the flud model reduces to a statc problem of choosng a vector of target demand rates and the servce rate vector that maxmze profts n the absence of holdng costs (Theorem ). The optmal servce rate vector makes the capacty constrant bndng (Corollary ). Ths problem determnes the optmal capacty, but s too coarse to specfy good prcng and sequencng polces. () Structural analyss of the jont prcng and sequencng problem. Fxng the capacty at the value prescrbed above, we then focus on the nfnte horzon total proft crteron to determne the optmal sequencng and prcng polces. We show that reasonable polces eventually dran the queues (Proposton ), characterze the propertes of the assocated value functon (Proposton ), and construct an optmalty verfcaton result for ths problem and through the assocated Bellman equaton (Proposton 3). We fnally show that sequencng decsons are made accordng to the c -rule (Proposton 4) and that the demand rates are nonncreasng functons of the queue length (Proposton 5). (3) State-space collapse and workload relaxaton. A consequence of the flud model optmal sequencng and demand controls s that after a fnte tme the queue length s always n a (effcent) confguraton where all of the workload s held at the cheapest product class (Theorem ). Ths state-space collapse result smplfes the soluton of the control problem from then onwards. It also suggests formulatng a workload relaxaton for our optmal prcng problem, whch focuses on the evoluton of the workload process, and uses an aggregated demand model and an approprate holdng cost rate. Ths one-dmensonal formulaton s smpler to analyze than the multproduct one (Theorem 3), leads to ntutve and mplementable heurstcs, and s often solvable n closed form ( 4.3 studes the lnear demand case). (4) Manageral nsghts. The key nsghts gleaned from our analyss are the followng: (a) the polcy that maxmzes the tme-average profts n the determnstc flud model nvests n scarce capacty and operates the system at almost full utlzaton, ncreasng prces and reducng demands f backlogs grow large; (b) orders are sequenced accordng to the greedy (c ) rule to mnmze nstantaneous holdng costs rrespectve of the prcng decsons; (c) the sequencng and prcng decsons are decoupled after an ntal transent perod whose length s characterzed, n the sense that thereafter prcng decsons are made as a functon of the aggregate system workload, whch does not depend on the sequencng rule. The last two nsghts are characterstcs of the optmal polces n the flud model formulaton of the jont prcng and sequencng problem. Together they suggest a heurstc that sequences jobs accordng to the c -rule and prces accordng to the soluton of a flud control problem formulated n terms of the aggregate system workload. The latter s smpler to solve and leads to ntutve polcy recommendatons. It also mnmzes the amount of nformaton sharng between the producton and prcng functons of an organzaton, whch s appealng from the vewpont of operatonalzng these jont decsons. The numercal results of 5. llustrate the effectveness of these heurstcs when compared to the soluton to the orgnal MDP formulaton. The remander of ths paper s structured as follows: Ths secton concludes wth a bref lterature revew. Secton descrbes the model, 3 studes the assocated flud control problem, and 4 studes ts workload relaxaton. Secton 5 summarzes the key nsghts of our analyss and reports some numercal results. Lterature Revew. Our work s related to two bodes of lterature focusng on stochastc processng network theory and revenue management, respectvely. Standard textbooks on queung networks provde some background on sngle-server queues, and standard dynamc programmng textbooks, such as Bertsekas (995), provde the necessary background for the soluton of the underlyng MDP problem formulatons. An early paper from Low (974) studed a sngle-product, multserver problem, and showed the monotoncty of the optmal prce polcy and proposed an teratve algorthm for computng t. Ths paper also consdered lnear holdng costs as a surrogate for watng-tme penaltes ncurred by the frm. The analyss n ths paper uses flud model approxmatons for queues wth statedependent parameters that were developed by Mandelbaum and Pats (995) for the case of exponental servce tmes and Posson arrval streams. In part, ths paper extends ther results by addng a control dmenson to some of ther models. The use of flud models for dynamc prcng n manufacturng systems has been dscussed n Kleywegt (), whle the lterature on flud models for purposes of sequencng and routng control s large; see, e.g., Chen and Yao (993), Avram et al. (995), Maglaras (), and the references theren. Workload formulatons arse n stochastc network control problems that are consdered n

3 96 Operatons Research 54(5), pp , 6 INFORMS the context of ther approxmate Brownan model formulatons. Ths dea and ts consequences n polcy desgn have been poneered by the work of Harrson (988, ) and Harrson and Van Meghem (996). Workload flud models were frst ntroduced n Harrson (995), whle the use of workload relaxatons of flud model control problems nvolvng sequencng, routng, and admsson control were proposed n Meyn (). The work by Gallego and van Ryzn (994, 997), the revew papers by McGll and van Ryzn (999), Btran and Caldentey (3), and Elmaghraby and Kesknocak (3), and the book by Tallur and van Ryzn (4) provde background on prcng and revenue management. Bller et al. () dscuss the dynamc prcng problem n the context of the automotve ndustry, and provde a determnstc, fnte horzon analyss of the sngle-product case. Some of the results n Maglaras and Messner (6) that studed multproduct revenue management problems are used n 4. There are many papers that are tangentally related to ours nsofar as they too combne some form of prcng wth the analyss of producton or nventory systems. Mendelson and Whang (99) and a stream of related papers have focused on statc prcng and sequencng control for socal welfare optmzaton n a multproduct M/M/ queue facng a market of heterogeneous prce- and delay-senstve users. Afeche (4) looks at a smplfed form of ths model under a revenue-maxmzng objectve, and provdes a thorough revew of statc prcng papers n queues, mostly under an atomstc customer demand model (cf. comment at the end of ). Chen and Frank () look at dynamc prcng for a sngle-product M/M/ queue usng dynamc programmng arguments, and Kalsh (983), Kachan and Peraks (), and Kleywegt () are examples of papers that study dynamc prcng ssues usng some form of flud or dffuson model. Examples of papers that nclude nventory control wth some element of prcng control decsons are Federgruen and Hechng (999) and Chen and Smch-Lev (4a, b).. Model Formulaton Consder a sngle-server producton faclty (the frm) that offers multple products, ndexed by = I, to a market of prce-senstve users. It operates n a market wth mperfect competton, and has power to nfluence ts vector of demand rates by varyng ts prce menu p. The demand process s assumed to be an I-dmensonal nonhomogeneous Posson process wth rate vector p determned through a demand functon that maps the prce vector nto a vector of nstantaneous demand rates, L, where I s the set of feasble prce vectors, and L = x x= p p I + s the set of achevable demand rate vectors. Note that only depends on the tme t through the prce posted at that nstance. We assume that L s a convex set, the demand functon s contnuously dfferentable and bounded, and (a) for each product, p s strctly decreasng n p, (b) for each p = p p p + p I, there exsts a null prce p p such that lm p p p p p =, and (c) the revenue rate p p = p p s bounded for all p and has a fnte maxmzer. (For any two n-vectors, x y wll denote ther nner product.) Under these assumptons, there exsts an nverse demand functon p, p L, that maps an achevable vector of demand rates nto a correspondng vector of prces p. Although, n general, ths nverse mappng need not be unque, t turns out that t s for common examples of demand relatons; see Tallur and van Ryzn (4, 7.3.). Followng a standard practce from revenue management, we may then vew the demand rate vector as the frm s control, and once ths s determned derve the correspondng prces usng the nverse demand functon. In ths case, the expected revenue rate wll be denoted by r, where r = p We wll assume that r s contnuous, bounded, and strctly concave, and denote ts maxmzer by = arg max r L. Infnte capacty buffers are assocated wth each product, and Q t wll denote the number of product jobs n the system (.e., n queue or n servce) at tme t. Ther servce tmes are..d. exponentally dstrbuted wth mean m (or rate = /m ). The load or traffc ntensty of the system when the demand vector s s defned as = m For future use, we defne the aggregate revenue functon as the maxmum achevable revenue rate when all products jontly consume capacty at rate, { R = max r L } / = () and denote by r the correspondng maxmzer. We wll assume that r s nondecreasng n for all products. Ths appears to be a mld assumpton that s satsfed by many commonly used demand models such as the lnear, exponental, pareto, and multnomal logt, to lst but a few examples. From the propertes of r t follows that R s concave, bounded, and has a fnte maxmzer that we denote by = arg max R = m L = m. When, capacty s scarce n the sense that the revenuemaxmzng demand rates would make the system unstable, whereas f <, the capacty s ample. Example. The lnear demand model s gven by p = b p j b j p j or n vector form p = Bp, where s the market potental for product and b, b j are the prce and cross-prce senstvty parameters. The nverse demand and revenue functons are p = B and r = B, respectvely. To ensure that these expressons are well defned and satsfy our assumptons, we wll requre that b >, and ether b > j b j or b > j b j for all. Both condtons relate to the margnal

4 Operatons Research 54(5), pp , 6 INFORMS 97 effect of prce changes to ndvdual and total demand, and guarantee that B exsts and has egenvalues wth postve real parts (Horn and Johnson 994, Theorem 6..). Fnally, the aggregate revenue functon defned through () s R = + + for r r wth = r r r r I, and the constants and r depend on the model parameters, B,, and are such that R s contnuous, almost everywhere dfferentable, and ncreasng for all. (The calculaton of these constants s gven n the appendx.) The frm has dscreton wth respect to the sequencng of jobs at the server, and prcng decsons for each product. Wthn each product, orders are processed n frstn-frst-out (FIFO), the server can only work on one job at any gven tme, and preemptve-resume type of servce s allowed. Under these assumptons, a sequencng polcy takes the form of the I-dmensonal cumulatve allocaton process T t t wth T =, where T t denotes the cumulatve tme that the server has allocated to class jobs up to tme t. In addton, T t s contnuous and nondecreasng, and satsfes the capacty constrant T t T s t s for s t< () Let p t be the vector of prces posted at tme t and t be the correspondng vector of demand rates. As mentoned above, we wll treat the demand rate vector as the control, and nfer the correspondng prce vector va the nverse demand functon. The demand polcy s the I-dmensonal process t t. Both T and are restrcted to be nonantcpatng controls;.e., decsons at tme t can only depend on nformaton that s avalable up to that tme. Fnally, the frm ncurs two types of cost. The frst s a lnear congeston cost gven by c Q t, where c > for all products. It ether captures the weghted delay costs ncurred by all outstandng orders or f approprate some noton of cost assocated wth work-n-progress nventory nvolved n producton. The second s the cost of operatng a faclty wth processng capabltes equal to, whch s h per unt tme, where h I + + s a convex, strctly ncreasng n each of ts arguments, dfferentable functon. The processng capablty vector s statc,.e., selected at tme t = and fxed thereafter. The frm s problem s to select the (statc) capacty vector, the sequencng polcy T, and the demand rates to maxmze the long-run average proft rate, vz [ ] maxmze lm t t Ɛ r s c Q s ds h (3) Remark. Let A t be the number of product orders that have arrved up to tme t. The frm s problem s to choose, p, T to maxmze lm t /t Ɛ p s da s c Q s ds h. Usng a standard result for ntensty control problems (see Brémaud 98, II.), ths can be rewrtten as (3). For stable, statonary Markov polces,.e., demand rates that can be expressed n the form = Q, (3) reduces to the steady-state expected proft crteron. We wll not rgorously justfy these ponts because our subsequent analyss wll not address (3) drectly but nstead rely on the use of determnstc and contnuous flud model approxmatons. Dscusson of Modellng Assumptons. In terms of probablstc assumptons, the one regardng the Posson nature of the demand processes s mportant to be able to justfy the determnstc flud models used n ths paper as rgorous lmts under dynamc prcng polces; ths uses the results from Mandelbaum and Pats (995). The assumpton on exponental servce tmes could be extended to allow for general dstrbutons at lttle addtonal cost, by easly adjustng the asymptotc results n Mandelbaum and Pats (995). (The exponental servce tme assumpton n Mandelbaum and Pats was mposed because the servce rate was allowed to be state dependent; wth constant servce rates as n ths paper one can get the strong-law type of lmt for the servce processes usng renewal theory.) Because we make use of the exponental assumpton n the numercal experments, where we compare the performance of our derved heurstcs aganst the soluton of the dynamc program assocated wth (3), we wll proceed under that smplfyng assumpton. In any case, allowng for general servce tme dstrbutons would have no effect on the flud model analyss that dsregards the second moment nformaton. As n most papers on prcng n queues and revenue management, our model assumes that self-nterested customers decde whether to place an order based solely on the prce vector at the tme of ther arrval;.e., they are strategc n makng purchase selectons by explctly or mplctly optmzng some form of a personal utlty functon, but they are not strategc n selectng the tmng of ther arrval n response to the frm s prcng strategy. Ths allows one to address the frm s prcng problem as an optmal ntensty control problem not nvolvng a game-theoretc analyss; see Larvere and Van Meghem (4) for a dscusson of ths pont and a justfcaton of the Posson arrval process assumpton as the soluton of such a game-theoretc analyss for a related model. Also, n our model the demand relatonshp captures the aggregate behavor of all potental customers. Ths s n contrast to more detaled models, such as the one used by Mendelson (985), Mendelson and Whang (99), and Van Meghem (), that obtan the demand rates through a customer-by-customer analyss based on more prmtve model elements such as personal utlty functons and prce and delay senstvty parameters. Fnally, because flud models best capture the transent behavor of the underlyng system, t seems more natural to consder an undscounted crteron n the flud model formulaton, whch motvated the objectve gven above. An alternate formulaton could consder an nfnte horzon dscounted proft crteron. Numercal tests showed that the

5 98 Operatons Research 54(5), pp , 6 INFORMS flud heurstcs obtaned for the undscounted crteron n 3 4 performed well even when compared to the stochastc dynamc programmng soluton for a dscounted objectve provded that the dscount factors were moderate,.e., where the tme scale for dscountng s long compared to job servce and nterarrval tmes. 3. Analyss of the Assocated Flud Model Control Problem The control problem posed above could be addressed usng the theory of MDPs (f one restrcts attenton to exponentally dstrbuted servce tmes), but ths s both analytcally and computatonally hard due to the multdmensonal nature of the state space. The approach taken n ths paper reles on the assocated flud control problem that wll be developed below and studed over the next two sectons. 3.. Formulaton of the Assocated Flud Control Problem The flud model s derved by replacng the dscrete and stochastc demand and producton processes by contnuous flows wth the correspondng determnstc rates. It s rgorously derved as a lmt under a strong-law-of-largenumbers type of scalng when we let the producton rate and demand grow proportonally large. Ths amounts to embeddng our problem n a sequence of systems wth model prmtves that scale accordng to n = n c n = c and h n n = nh (4) These scalng relatons mply that t s reasonable to also grow the capacty proportonally to n accordng to n = n, and would result n a problem where the varous revenue and cost contrbutons are all of the same order of magntude that s tself proportonal to n. Scalng the ntal condton accordng to Q n = nz, and studyng the lmt of q n t = Q n t /n as the scalng parameter n grows large, we get that q n t converges to a contnuous flud lmt process denoted by q t. In the sequel, let M = dag I, and u t denote the fracton of the server capacty dedcated to processng an order of product at tme t; n the flud model the server s allowed to splt ts effort across dfferent product classes. The flud model equatons (see Mandelbaum and Pats 995) are q t = z + q t s ds M u t u s ds q = z (5) u t t L (6) Two other quanttes of nterest are the traffc ntensty at tme t defned as t = m t, and the server workload w t = m q t, whch s the amount of tme needed for the server to clear the current backlog dsregardng any future arrvals. For future reference, we note that w t = w + s ds t + I t (7) where w = m z and I t = t u s ds s the cumulatve server dleness up to tme t. In the sprt of (3), t s natural to consder maxmzng the tme-average proft crteron [ ] lm r s c q s ds h (8) t t In consderng ths problem, we wll also restrct attenton to controls, u that are rght-contnuous functons wth left lmts (RCLL) and are nonantcpatng,.e., decsons at tme t can only use nformaton that has been made avalable up to that tme, whch we summarze below: t u t are nonantcpatng and RCLL for t (9) The next theorem shows that ths crteron s useful n selectng the optmal capacty vector, but, as ts proof hghlghts, s too coarse to dentfy good prcng and sequencng polces. The latter s addressed n a more refned formulaton gven n (3). Theorem. Consder the problem of maxmzng (8) over, and u subject to the flud model equatons (5) (6) and (9). Then, the optmal tme-average proft rate s defned by { =max r h L } / > () whle the optmal capacty vector s the assocated optmzer, whch s unque. Proof. Step. We demonstrate a feasble control for ths problem wth fnte average proft rate. Pck any vector and consder the polcy: t = and u t s any nondlng rule (.e., u t = ) for t w ; and, { t = ˆ = arg max r L } / and u t = M t for all t>w. From (5), t follows that under that polcy q t = for all t w, and that lm t /t r s c q s ds h = r ˆ h, whch s fnte. Step. We show that any optmal control must satsfy the stablty condton lm sup t t s ds () Suppose for some convergng subsequence, lm sup /t s ds > t Then, from (7) we get that w t and q t as t. Because r s bounded, t follows that lm sup t /t r s c q s ds =, whch s suboptmal. By contradcton, we establsh ().

6 Operatons Research 54(5), pp , 6 INFORMS 99 Step 3. Assume > (because otherwse t = for all t ) such that s well defned. Recall the defnton of R n () and note that t r s c q s ds h t r s ds h R s ds h t ( ) R s ds h t The frst nequalty follows the fact that q t, the second from the defnton of R, the thrd from Jensen s nequalty, and the last one by notng that () can be rewrtten as max R h L / =, where the constrant s needed due to (), and / s convex n for. The control specfed n Step for = acheves the upper bound. Fnally, the concavty of r h mples the unqueness of, whch completes the proof. Hereafter we wll fx the capacty vector to defned va (), but to smplfy notaton we wll denote t by and use ˆ n place of ˆ. We also relabel the products such that c c c I I () The structure of () and the assumptons on r and h mply the followng: Corollary. The capacty s scarce n the sense that () m ˆ = and () ˆ. It s worth remarkng that ths flud model analyss wll nvest n zero processng capacty f the margnal revenue rate at = s smaller than the margnal cost of capacty at =. Ths was also observed n the numercal results of 5. Whle ths crteron s useful n optmzng over the optmal level of producton capacty, t s too coarse to use n the flud model to construct good prcng and sequencng polces for the underlyng multproduct revenue management problem. Indeed, any control u that drans the queue and then uses ˆ u thereafter s optmal for the tme-average crteron because the transent phase untl the queue s empted s washed out of the objectve. Instead, we wll consder the total proft crteron r t c q t dt (3) where r = r r ˆ. (The capacty has been optmzed va the tme-average problem, and ts assocated cost need not be ncluded n ths formulaton. Note that ncludng t n the total proft crteron could result n a dfferent optmum for, but ths would no longer be optmal for the hgher-order crteron gven n (8).) Total proft crtera are common n flud model control problems n the lterature because they emphasze the transent system behavor, whch s well captured through the flud equatons. The remander of ths secton wll study the problem of maxmzng (3) over u subject to the flud model equatons (5) (6) and (9). 3.. Characterzaton of the Flud Optmal Controls I. Prelmnary Structural Results. The frst result establshes that good control polces must be stable n that they eventually empty the queue-length vector, and that there exsts an optmal soluton. Restrctng attenton to such polces, we can then characterze the optmal one as the soluton to the assocated Bellman equaton. Proposton. Consder the problem of maxmzng (3) over and u subject to (5) (6) and (9). Then: () t suffces to restrct attenton to controls under whch q t as t, and () there exsts an optmal par u and an assocated trajectory q n the sense that r t c q t dt r t c q t dt where u s any other feasble control and q s the assocated trajectory. The proof s relegated to the appendx. To facltate analyss and the numercal computatons that wll follow, we wll restrct attenton to queue lengths that le n a large but bounded doman. In detal, assumng that each queue s no greater than a large constant N, we wll express the set of possble queue-length vectors by = q m q N w, where N w = m N. That s, we express the doman as a functon of the aggregate workload m q as opposed to the ndvdual queue lengths, because as t wll turn out the workload s nonncreasng, whereas the queue lengths are not. For concreteness, we wll enforce ths condton by mposng the constrant that t ˆ f m q t = N w ; we wll show at the end of ths subsecton that ths condton s never nvoked, and therefore does not change the structure of the optmal polcy; see the remark followng Proposton 5. Next, we proceed wth an nformal dervaton of the Bellman equaton assocated wth (3) that characterzes the optmal polcy. Let V q denote the optmal proft extracted under (3) startng from q. The exstence of ths functon follows from Proposton. The next proposton summarzes the man structural propertes of the value functon that are used n subsequent analyss. The proof s gven n the appendx.

7 9 Operatons Research 54(5), pp , 6 INFORMS Proposton. Let V q denote the optmal proft extracted under (3) startng from q. Then: () for all q, <V q, and V = ; () V s Lpschtz contnuous, and therefore t s almost everywhere (a.e.) dfferentable; () V s concave; and (v) V z V for all z (componentwse). Usng the defnton of V and ts a.e. dfferentablty, a standard dynamc programmng argument gves that V q t = max L u q r c q t t + V q t whch leads to the Bellman equaton c q = max L u q + V q t Mu t + o t r + V q Mu q and V = (4) where q = u u, u, j j u j j s.t. q j =. (The last set of constrants ensures that q t.) Let, u denote the maxmzers n (4). Proposton 3. Consder the problem of maxmzng (3) subject to (5) (6) and (9). Let q t denote the queuelength trajectory under u defned through (4). Then, u s optmal n the sense that for any q = z and any feasble u under whch q t as t, ( V z = r t c q t ) dt ( ) r t c q t dt Proof. From (5), we have that dq t /dt = t Mu t a.e., and the defnton of the Bellman equaton gves that r t c q t + V q t t Mu t t (5) Integratng (5) over t, we get that ( ) r s c q s ds dv q s = V z V q t (6) Lettng t and usng the facts that q t, V =, and V s contnuous, we conclude that for all z, V z r t c q t dt. To complete the proof we need to show that V z s ndeed the total proft under u. Because (6) holds wth equalty under u,t suffces to show that q t ast. We argue by contradcton. Suppose that lm sup t q t = q. Then, ths mples that lm sup t V q t = V q <, and therefore that r t c q t dt >V z >. Onthe other hand, from Proposton, f q t ast, then r t c q t dt =, whch leads to a contradcton. Hence, q t and consequently V q t as t, whch completes the proof. II. Characterzaton of Optmal Sequencng and Prcng Polces. Usng the characterzaton of the optmal controls n terms of the Bellman equaton gven n (4), we wll next show the followng propertes for the optmal polces: (a) sequencng decsons are made accordng to the c -rule, and (b) the demand vector s bounded above by ˆ. Proposton 4. Fx any demand rate trajectory t for t. Then, the c -rule defned by { u t = arg max c u u u } j t j u j j s.t. q j t = (7) s pathwse optmal n that c q t c q t for all t, where q t and q t denote the queue-length trajectores under u t and any other feasble allocaton u t, respectvely. Sketch of Proof. The proof follows along the lnes of Avram et al. (995, 4.) usng the assocated Hamltonan formulaton; the extenson to an nfnte horzon s done along the lnes of Seerstad and Sydsaeter (987, ). See also Chen and Yao (993, 3). A consequence of Proposton 4 s that V s nonncreasng n. Ths s establshed as follows. For z = e w, where e s the th unt vector and w> s small, let be the optmal demand vector trajectory startng from that ntal condton and q t be the correspondng queue-length trajectory under and the c -rule. Then, for j<, V z j r t c q j t dt r t c q t dt = V z where the frst nequalty follows because need not be optmal startng from z j, and the second can be establshed usng the propertes of the c -rule (e.g., usng an nducton argument on the number of classes). For small w, we also have that V z = V + V w + O w, whch together wth the above nequalty can be used to obtan the desred result. The next result establshes that the optmal demand rate s upper bounded by = ˆ. The proof s gven n the appendx. Proposton 5. The optmal demand rates q defned va (4) satsfy q ˆ for all q. A consequence of ths last result s that the aggregate load nto the system at any tme t s t = m t. Usng (7), ths mples that the workload n the system s nonncreasng n tme, and, n turn, that f q, then q t for all t. As a result, the soluton of the flud optmal control problem enforces the exogenous constrant

8 Operatons Research 54(5), pp , 6 INFORMS 9 mposed on the behavor of the system when t reaches the boundary of, and moreover the optmal control does not depend on the sze of the set, as measured by the parameter N w. Both of the propertes derved n the last two propostons are exploted n the next secton to characterze the evoluton of an optmally controlled system, and to propose an approprately constructed one-dmensonal relaxaton to the flud control problem. They are also helpful n numercally computng the optmal demand rates q. III. Comments on Computng the Optmal Prcng Polcy. Despte the smple structure of ths flud control problem, t s stll not solvable n closed form, and one has to resort to numercal optmzaton technques to compute the optmal prcng strategy. The smplest way to do so would be to dscretze over tme and solve a tractable concave maxmzaton problem. Specfcally, the objectve comprses of the concave revenue term mnus the lnear holdng cost for each tme perod. The flud model dynamcs are captured through a set of lnear equalty constrants of the form q t + = q t + t Mu t, where t s now a dscrete ndex. Addtonal constrants for the allocaton rule are that u t and u t, for the demand rates that t L, and for the state that q t. The complexty of ths problem grows wth the number of products and the number of tme perods. Instances of modest sze wth tens of products and a few hundred tme perods that result n a few thousand varables and constrants can be computatonally tractable for commonly used demand models such as the lnear, exponental, and soelastc ones. 4. State-Space Collapse and a One- Dmensonal Workload Relaxaton of the Flud Model Proft Maxmzaton Problem The structural propertes of the flud optmal sequencng and prcng polces have one mportant mplcaton about the optmal system behavor that we show n Theorem : The optmal queue-length trajectory couples n fnte tme wth a trajectory that holds all of ts workload n the cheapest product class, and subsequently evolves as the optmal soluton to an approprately defned sngleproduct problem. Motvated by ths state-space collapse property, we subsequently propose and analyze a relaxaton to the flud prcng problem that s based on ts onedmensonal workload rather than the I-dmensonal queuelength vector. 4.. Flud-Scale State-Space Collapse We frst ntroduce some useful notaton. For any workload poston w, we defne w = arg mn c q q m q = w = w (8) to be the queue-length vector that holds workload w and has mnmum cost. For the lnear holdng cost structure of our model, ths corresponds to keepng all the workload nto the cheapest and lowest-prorty class, whch by our labellng conventon s Class ;.e., c w = c w. Theorem. Let q t denote the optmal trajectory for the problem of maxmzng (3) subject to (5) (6). Then, for any z, q t = m q t for all t T z = > m z /ˆ. Proof. From Propostons 4 and 5, we know that q t s the trajectory under the c -rule wth t ˆ for all t. Consder any z wth ntal workload w = m z, and note that the lowest prorty class s not served untl all hgh-prorty classes have been draned,.e., u t = for all t, where = nf t q t = >. Now, the hgh-prorty class queue lengths q t for = I are upper bounded by the queue length of an I class queue wth ntal condton q = z z 3 z I and t = ˆ. The latter system has traffc ntensty ˆ /, and drans ts ntal workload n T z = > m z /ˆ tme under any nondlng sequencng rule (.e., u t = f q t ). Moreover, q t = for all t T z. It follows that T z and q t = for all > and t T z. Ths completes the proof. The dynamcs of the optmally controlled multclass flud queue for t are gven by ẇ t = + t and q t = w t (9) wth t = m t. That s, the queue length evolves on the hyperplane defned through q = m q, or n other words, the state space has effectvely collapsed to that of the one-dmensonal workload process. Gven Theorem and the defnton of R, t s easy to see that for all t, the proft maxmzaton problem posed through (3) reduces to that of maxmzng R t c w t dt subject to (9) and where R = R R s analogous to the defnton of r. The state of ths problem s the one-dmensonal workload w t, and the control s the onedmensonal aggregate capacty consumpton rate t. The aggregate control s mapped to product level demand rates (and thus prces) usng the mappng r defned through (), whch selects the demand rate vector that maxmzes the nstantaneous revenue rate subject to the constrant m = t. The workload s mapped to a queue-length vector through the mappng w. Optmzng the total system profts for t s a one-dmensonal control problem, whch s much smpler analytcally and computatonally than the multdmensonal (queue-length) formulaton that one started wth.

9 9 Operatons Research 54(5), pp , 6 INFORMS 4.. The Workload Relaxaton of the Flud Prcng Problem Workload formulatons of network control problems have been used extensvely n the context of heavy-traffc theory, where they emerge naturally as equvalent formulatons to the approxmatng Brownan control problem formulatons; see Harrson (988, ) and Harrson and Van Meghem (996). Roughly speakng, Brownan approxmatons are derved n a way that nvolves a compresson of tme that makes the ntal transent perod of length T z neglgble, and where the queue length can be shown to always evolve on that mnmum cost manfold, q = m q. Motvated by these results, Meyn () recently proposed to use such workload relaxatons to flud model network control problems nvolvng sequencng and routng control, mplctly dsregardng ths short ntal transent perod where the queue length s not on that effcent confguraton. In ths paper, we extend Meyn s dea to ncorporate the dynamc prcng element, and propose a workload relaxaton to our proft maxmzaton problem. The sequencng decsons are greedy (as shown n Proposton 4), and are suppressed n ths model posed drectly n terms of the aggregate workload process for the approprate cost parameter. The latter s partally justfed by Theorem. Specfcally, we propose the followng relaxaton to the flud control problem posed through (3) and (5) (6): Choose the load t for t to maxmze R t w t dt () subject to (7), whch descrbes the workload dynamcs, the control constrant t max where () max = max = m L and = mn c. Ths s equvalent to the queue-length formulaton of (3) and (5) (6) f the queue starts at a mnmum cost confguraton z = m z, because n that case q = m q. On the contrary, f z m z, () s only an approxmaton to (3) for some ntal transent perod of length no more than T z, where the queue length reaches a mnmum cost state. Maxmzng () subject to (7) and () s a sngle-product problem for the revenue functon R and the holdng cost rate. It s much easer to analyze, often n closed form, and the numercal results of the next secton wll llustrate that t generates heurstcs wth very good performance. Moreover, ts soluton translates nto easly mplementable polces for the underlyng multproduct proft maxmzaton problem by: (a) orders are sequenced accordng to the c -rule, and (b) products are prced to nduce the demand rates t = r t. The remander of ths secton studes the sngle-product workload relaxaton. We use the notaton V for the assocated value functon and w for ts optmal control. Our frst result specalzes the fndngs of Propostons and 5 to the sngle-product case. Theorem 3. Consder the problem of maxmzng () subject to (7) and (), and let V w denote the assocated value functon startng from an ntal workload poston w. Then: () V s concave and nonncreasng n w, and () w s nonncreasng n w. Proof. The concavty follows from Proposton. Consder two ntal condtons w >w, and denote by w t, w t the assocated optmal trajectores. Let = nf t : w t = w, whch s well defned and fnte because from Proposton, w t ast. Then, V w = R t w t dt + V w Because w >w, t follows that s ds <, whch mples from the concavty of R that ( ) R s ds R s ds and therefore that V w V w. Ths proves property (). To prove part (), we note that from Proposton we get that V s Lpschtz contnuous, and therefore a.e. dfferentable. Ths mples that part () can be restated as V w and s nonncreasng n w. Adaptng (4), we get that the Bellman equaton for the workload control problem s w = max y R + V w y () where the maxmzaton s over max and y ; y t s nterpreted as the total effort exerted by the server on all product classes,.e., y t = u t. As n Proposton, t s easy to show that = = m ˆ, and that y w = for all w. Usng the propertes of R and the nonncreasng nature of V w, we readly conclude that w s nonncreasng n w and complete the proof. An nterestng ssue that arses s to compare the tme t takes untl state-space collapse s acheved, wth the tme t takes for the entre system to dran, whch tself follows from Proposton. We have been unable to get a crsp characterzaton or bound for the relaton between the two, however, a back-of-the-envelope analyss shows that the dfference can be sgnfcant. Specfcally, the rate at whch the system s drftng towards the state-space collapse poston s t ˆ /, where t = t r t / s the aggregate load due to products = I. Smlarly, the rate at whch the system s draned s t. The tme taken to reach each of these two states s nversely proportonal to these loads. Suppose, for example, that the average values along the optmal workload trajectory for t and t are 95 and 75, respectvely (.e., Product demand consumed % of the system s processng capacty). Then, the tme requred to dran the system would be fve tmes longer than the tme needed to acheve state-space collapse. Ths smple example llustrates that the relatve dfference between these two

10 Operatons Research 54(5), pp , 6 INFORMS 93 quanttes s lkely to be sgnfcant when t s optmal to prce n a way that Product consumes a sgnfcant porton of the system s processng capacty, and where the holdng cost parameters are such that the average load over the optmal trajectory s hgh. The monotoncty of w mples that the product-level demands have a nested structure. Corollary. Let I w = r w > be the set of products offered when the workload s w. Then, I w I w + x for all w x. Proof. Ths s a drect consequence of the fact that w s nonncreasng n w and (by assumpton) r s nondecreasng n. The last result has mportant practcal mplcatons n terms of the optmal set of product offerngs at ncreasng levels of congeston as measured by the aggregate system workload. Specfcally, accordng to the soluton to the workload flud model prcng problem, the frm wll offer products n a nested structure, by tactcally removng the ones that become not proftable as congeston ncreases and the target aggregate capacty consumpton w decreases; once a product s removed, t s never rentroduced at hgher levels of congeston. Fnally, R w s nterpreted as the margnal or opportunty cost of addtonal work arrvng when the workload s equal to w. Often the smple structure of the workload formulaton can be exploted to solve ths prcng problem n closed form. Even f that s not possble dependng on the form of the demand model, ths one-dmensonal problem s much smpler to tackle numercally usng the dscretzaton approach mentoned earler n comparson to the multproduct formulaton Closed-Form Soluton of Workload Formulaton for the Lnear Demand Model The lnear demand model s often used n practce due to ts smple and ntutve structure, ts tractablty when embedded n mathematcal optmzaton formulatons of revenue management problems, and the fact that ts parametrc form s sutable for statstcal estmaton. Ths subsecton provdes a closed-form soluton to the workload relaxaton for the case of the lnear demand model. Recall from the descrpton n that the lnear demand model s of the form p = Bp, and ts revenue functon s r = B. Its assocated aggregate revenue functon R s defned through () and s shown n the appendx that t can be expressed as R = + + for r r for = r r r r I, and constants and r that depend on the model parameters, B,, and are such that R s contnuous, almost everywhere dfferentable, and ncreasng for all. The value of r s that of the smallest aggregate capacty consumpton rate above whch t s optmal to start offerng the most proftable products. Recall that = mn c. Startng wth the Bellman equaton n () and expandng R nto R R, we get that for all w, w = arg max R + V w max (3) and R + w = R w + V w w (4) We wll use the frst expresson to express V w n terms of w, and the second one to pontwse solve for w for all w. Specfcally, the frst-order optmalty condton for w s that R = V w whch gves that at the optmum and for some I, V w = w (5) Usng (4) and (5), we get that the optmal drft w satsfes a quadratc equaton of the form + + R w = the soluton of whch s that w w = + where = R + / The value of the ndex above s mplctly determned such that the soluton w r r. It s straghtforward but tedous to show that there s a unque value of for whch ths expresson s consstent;.e., f one computes w through the above expresson, takng the value of as gven, then w s ndeed n the nterval r r. Ths s captured n the followng defnton. Frst, for = I, defne w = + + / + and w = nf w w =. Second, set [ ] + w w = + whenever w w w (6) where ths last expresson has ncorporated the mplct constrant that. Note that the w s are such that w s contnuous and decreasng n w, = w I w I w, and that w / + = r. The optmalty of ths control s establshed usng the verfcaton result of Proposton 3 for the functon { w V w w w w = R w w w

11 94 Operatons Research 54(5), pp , 6 INFORMS Fnally, w s dsaggregated nto product demands va the mappng r as follows: r w = m b for î w r w = otherwse and where î w = max w r defnes the set of products the frm wll offer, and ( = j î w )( m j j w j î w m j b jj) a dervaton of these expressons s gven at the end of the appendx. Fnally, prces are nferred through the nverse demand functon. 5. Dscusson and Numercal Results The frst part of ths secton offers a short summary of the key nsghts gleaned from our analyss regardng the structure of the optmal prcng and sequencng polces. The second part reports on a set of numercal results that compare the prcng and sequencng heurstcs extracted from the flud model analyss to the soluton of the underlyng dynamc program, as well as the best statc prcng polcy. 5.. Man Insghts The analyss of the two precedng sectons leads to several nsghts brefly dscussed below. () Invest n scarce capacty. If one assumes that the demand model s known and t s statonary (as consdered n ths paper), then t s optmal n the flud model to nvest n producton capacty that s scarce. That s, the demand vector that would maxmze the nstantaneous revenues n the absence of the capacty constrant would make the system unstable. Ths s ntutve because the revenue functon s concave and the capacty cost s convex, makng margnal revenue contrbuton close to the revenuemaxmzng demand vector small compared to the margnal cost of the extra capacty needed to cope wth that demand. () Hghresource utlzaton and congeston prcng. Operatonally, ths choce of capacty vector nduces the frm to operate ts processng resources at close to full utlzaton, moderatng excess backlogs through a (dynamc) ncrease n one or more prces. Ths behavor was establshed under the optmal flud control polcy, but was numercally observed to also hold under the optmal control of the MDP formulaton assocated wth the orgnal problem of. (3) Nested prcng polcy. As the system backlog grows large, under the flud optmal prcng polcy the frm ncreases prces n a way that effectvely removes products from the market n a nested fashon, accordng to ther margnal revenue contrbuton. (4) Sequencng and prcng decsons decouple. These two elements of control are essentally decoupled n the flud control formulaton nsofar as sequencng s done accordng to the greedy c -rule ndependent of the prcng decsons, and prcng eventually depends only on the system workload rather than the ndvdual queue lengths, whch are themselves nsenstve to the choce of the sequencng rule. (5) Prcng as functon of workload. A reasonable relaxaton to the orgnal problem s to sequence jobs usng the greedy polcy and prce accordng to the soluton of a flud control problem formulated n terms of the aggregate system workload, whch s smpler to solve and leads to ntutvely appealng polces. It s also practcal to mplement because t only requres modest nformaton sharng between the prcng and producton functons of an organzaton. The numercal results that follow llustrate the effectveness of ths heurstc when compared to the soluton to the orgnal MDP formulaton, as well as to the optmal statc prcng polces. (6) Workload relaxatons of revenue management problems. A generalzaton of the last few remarks suggests the use of workload relaxatons for revenue management of make-to-order systems. Whle workload flud models are essentally heurstcally derved because flud-scale statespace collapse results smlar to that of Theorem offer only partal support for ther valdty, they seem to capture some of the essental elements of the underlyng prcng and operatonal control problems whle mantanng a far amount of tractablty. (Further justfcaton can be obtaned through a dffuson analyss, by adoptng the arguments of Harrson and Van Meghem 996 to systems wth dynamc prcng (drft) control capablty.) Fnally, workload flud model relaxatons lead to practcally mplementable solutons wth modest coordnaton requrements between the prcng and producton functons based on the aggregated workload nformaton. 5.. Implementable Heurstcs and Numercal Results We conclude wth a numercal study of the performance of the capacty choce, prcng, and sequencng polces that are extracted va the flud model analyss. Conceptually, ths can be separated nto two ssues: (a) How good are the heurstcs that are derved from analyss of flud model proft maxmzaton formulatons, and (b) what s the mpact of the workload relaxaton on the performance of these heurstcs. Ths subsecton s splt nto two parts, focusng on sngleproduct and two-product problems, respectvely, that effectvely address these ssues n sequence. Implementaton of the Flud Heurstcs. The precedng analyss has culmnated n two heurstcs that are based on the soluton of the (multdmensonal) flud control proft maxmzaton problem of 3 and ts workload relaxaton of 4, respectvely. Ths secton studes the performance of the

12 Operatons Research 54(5), pp , 6 INFORMS 95 prcng rule extracted from the workload relaxaton, whch s summarzed through the aggregate control w and s mapped to a vector of demand rates through r w. Instead of selectng the prce vector that maps nto ths vector of demands, we propose to ntroduce a small tunable parameter and prce to nduce the demand vector w = r w + (7).e., we propose to use the state-dependent control as extracted from the workload flud model formulaton, but perturb ts aggregate decson w by a constant shft that tres to correct for some of the dealzatons of the determnstc flud model. Ths leaves the shape and structure of the proposed control unchanged, as computed through the flud model analyss. Its use s farly economcal n the sense that t uses a sngle parameter to perturb the aggregated control decson, and use the mappng from to s to dsaggregate ts effect to the multple products n a way that explots the structure of the proposed polcy. Fnally, the magntude of ths adjustment s small n comparson to the dynamc component w, and one would expect that n the asymptotc regme sketched n 3. t s of order o.e., n asn n the context of (4), makng the resultng shfts to the demand rates to be of order o n, whle the demand rates themselves are of order n. Smlar adjustments to statc flud prces have appeared n other papers, and were ether heurstcally proposed as n Gallego and van Ryzn (994) for the pershable, sngle-product revenue maxmzaton problem, or analytcally derved as n Maglaras and Zeev (3) n the context of optmal statc prcng for a sngle-product stochastc servce system. Ths tunable parameter s selected ether numercally or through smulaton. For example, for the sngle-product problem wth Markovan assumptons, t s straghtforward to compute the steady-state proft rate under the statedependent demand polcy Q defned through (7) and numercally optmze over. A smlar calculaton s possble but tedous for the multproduct case, where a smulaton-based optmzaton approach may be more sutable. The latter can also be used f the servce tmes follow general dstrbutons. Sngle-Product Problems. Ths frst part studes the capacty nvestment decson under dfferent polces, as well as the relatve performance of the flud prcng heurstcs aganst the soluton of the MDP formulaton assocated wth the problem of maxmzng (3) descrbed n. FM() refers to a drect mplementaton of the flud control law f Q,.e., wth =, whle FM( ) s the flud polcy wth the optmally tuned parameter. These three dynamc prcng polces wll also be compared to Statc, the optmal statc prcng polcy wthout admsson control, and Statc + Adm., whch refers to the optmal statc prce wth admsson control. The two statc-prce systems behave lke M/M/ and M/M//K queues, respectvely. () Performance wthoptmzed capacty. Table compares the capacty choces and proft rates under these fve canddate polces n a sngle-product model wth a lnear demand functon and lnear cost of capacty. 3 We make a few observatons: (a) The flud prcng heurstcs outperform the statc prcng polces by.5% 5%, whle admsson control adds up to % to the proft rate acheved under a statc prcng polcy. Such performance gans have sgnfcant mpact to the frm s proftablty. (b) The capacty under the FM() polcy s computed through (), and seems to systematcally underestmate the ones under the MDP and FM( ) polces that were both computed through a numercal search. (c) The suboptmalty gaps ncreased as the holdng cost parameter c grew larger, and the optmal capacty level decreased as a functon of ts cost parameter h. Note that for large h, t may become unproftable to operate the frm,.e., the optmal capacty nvestment s =. For example, accordng to (), the frm would not nvest n processng capacty f r h. (d) The expected traffc ntensty n all these test cases ranged from.7 to.95 (see also Table ). () Performance comparson wthcommon capacty. Table compares the proft rate under these polces operatng under a common value of processng capacty that was computed based on our flud model analyss usng (). Ths solates the performance effect of the prcng decson under each canddate polcy. Our results show that the relatve advantage of the flud heurstcs over the statc prcng polces ncreases when consdered under a common capacty choce. In addton, as the prce senstvty parameter gets large (n the lower half of ths table), the performance gan due to dynamc prcng ncreases and the gaps between the flud and statc heurstcs wden, and also Table. Proft rate and capacty nvestment under a lnear demand model: p = 4 p. MDP FM FM( ) Statc Statc + Adm. c h Ɛ Gap (%) Gap (%) Gap (%) Gap (%)

13 96 Operatons Research 54(5), pp , 6 INFORMS Table. Performance comparson wth common capacty selected usng (). MDP FM() FM( ) Statc Statc + Adm. b c h Ɛ Gap (%) Gap (%) Gap (%) Gap (%) the gap between the MDP polcy and all other heurstcs grows larger. Ths s due to the fact that as the prce senstvty parameter ncreases, the revenue rates decrease substantally (c.f., r = /b), ndrectly makng the holdng cost term more sgnfcant. Indeed, close nspecton of the optmal polcy shows that n such cases the frm operates the system wth very few jobs n the queue, where the nature of the statc heurstcs and the dealzatons of the flud approxmatons become more pronounced. In general, the accuracy of the flud heurstcs mproved as the sze of the system as measured by the potental demand and processng capacty grew larger, whch s consstent wth the scalng gven n (4). Intutvely, ths says that prcng heurstcs extracted from flud approxmatons are expected to perform well n settngs where the actual processng tme of each order s much smaller than the actual tme t takes for ths order to go through the system. (3) General demand model. Table 3 reports on a small set of results for a model wth an exponental demand model. We note that n ths case the flud polcy was computed numercally by solvng the Bellman equaton (4) at each queue-length poston Q. For the exponental demand model, the flud prcng polces outperformed the statc ones by about % 3%, whle smlar optmalty gaps were observed under the soelastc demand model. To recaptulate, the man nsghts extracted from our numercal results of the sngle-product model are the followng: () The dynamc heurstcs FM() and FM( ) outperform the statc prcng polces by.5% 5%, whch s sgnfcant. Admsson control adds about % of profts to statc prcng. () The effect of dynamc prcng and the overall performance gaps ncreases as functons of the holdng cost and prce senstvty parameters. In both of these cases, the statc polces prce conservatvely to control congeston costs when queues buld up. (3) The flud model analyss sets capacty accordng to (), whch tends to underestmate the optmal choces under both MDP and FM( ), albet by relatvely small margns. Its effect on the maxmum achevable proft rate under the MDP polcy ranged from.%.4% n the experments that we ran. In settngs where capacty s dffcult to change whle demand models and compettve effects may vary substantally over tme, t may be practcal to adopt the flud model soluton as a way to set capacty, and use prcng to fne-tune the frm s performance. Under all canddate polces t s optmal to nvest n scarce capacty unless the frm s operatng n an envronment wth very small capacty costs and very large congeston (holdng) costs. Multproduct Problems. The last results look at a two-product system under a lnear demand model. They focus on the performance of the polcy extracted va the workload relaxaton of 4, whch sequences jobs accordng to the c -rule and prces as a functon of the system workload. The examples tested below have two nonsubsttutable products that follow a lnear demand relatonshp, that s, b j = for all j. The flud polcy that we tested was the one specfed through (7) extracted from the workload relaxaton wth an optmzed parameter. For the lnear demand model, the expresson for f Q becomes [ ] + w f Q = + + where w = m Q, = R + /,,, were as defned n the appendx n (3) (34), = mn c c, and R = max r m =. Table 3. Sngle-product, exponental demand p = e b p wth common capacty selected usng (). MDP FM() FM( ) Statc Statc + Adm. b c h Ɛ Gap (%) Gap (%) Gap (%) Gap (%)

14 Operatons Research 54(5), pp , 6 INFORMS 97 Table 4. The two-product model wth lnear demand. MDP FM-work Statc + Adm. b b c, c Ɛ Ɛ Ɛ Gap (%) Ɛ Ɛ Gap (%) Ɛ Ɛ , Notes. Flud polcy was computed usng the workload relaxaton wth an optmzed parameter (cf. (7)). In all experments, = = 4 and h = h =. The test cases reported n Table 4 correspond to relatvely small problem nstances that kept the numercal soluton of the two-dmensonal dynamc program assocated wth the stochastc formulaton of tractable. The selected parameters tested a range of scenaros for the relatve revenue and holdng cost contrbuton of each product. The prmary observaton from the results of Table 4 s that the polcy extracted va the workload relaxaton had smlar performance gaps to those observed for the sngle-product models for a wde range of parameters, whle the optmalty gaps of the statc prcng polcy wth admsson control degraded n ths multproduct settng. Ths suggests that the restrcton of the workload relaxaton that forces prces to be functons of the workload had lttle mpact on the performance of the flud heurstcs, and plausbly that the prcng decsons under the MDP polcy may also mostly depend on the total workload and not the ndvdual queue lengths. Ths s explored graphcally n Fgure. Specfcally, the plots n Fgure explore the form of the MDP demand polcy as a functon of the system workload, and to a large extent llustrate that the MDP controls ndeed resemble those extracted va the flud workload Fgure. MDP controls as functons of the system workload for a two-product model wth a lnear demand functon wth B =, = = 4, c =, c = 4, and h = h =. 4 3 Case : Λ = Λ = λ (w) * λ (w) * 5 5 Workload 6 Case : Λ =, Λ = Workload λ (w) * 4 λ (w) * 5 5 Workload 8 Case 3: Λ = 6, Λ = Workload λ (w) * Workload λ (w) * 5 5 Workload

15 98 Operatons Research 54(5), pp , 6 INFORMS Fgure. Steady-state probablty dstrbuton under MDP controls for a two-product model wth a lnear demand functon wth B =, = = 4, c =, c = 4, and h = h =. Case : Λ = Λ = 8 Case : Λ =, Λ = Probablty.4. Probablty Q Q Q Q relaxaton of the prevous secton. The top two panels focus on a problem wth a symmetrc demand model for two products, whle the lower panels are for problems where Product contrbutes hgher revenues than Product. In all cases, Product ncurred hgher holdng costs. Fgure plots the MDP controls as functons of the total workload. The multple values for for each w reflect the fact that the MDP soluton depends on the two-dmensonal queuelength vector Q Q, and thus the controls at dfferent queue-length confguratons that hold the same workload may dffer. The relatvely narrow spread of values at each w lends credblty to the proposed approxmaton of the MDP controls wth state-dependent functons of the workload. Moreover, plots of the two-dmensonal steadystate dstrbutons shown n Fgure reveal that most of the probablty s concentrated n states where (a) the system workload s held n the cheaper queue (here Product ) or, stated dfferently, where the queue s n an effcent confguraton; and (b) the aggregate workload s modest (w 4 and w 4, respectvely), allowng the system manager to effectvely modulate the traffc ntensty nto the system by mostly adjustng the demand of Product, whch contrbutes less revenue. (We plotted the dstrbutons for the frst two parameter sets n Fgure. The thrd set wth = 6 and = 8 s smlar, although n that case the system operates almost lke a sngle-product system.) These observatons, of course, depend on the magntude of the holdng cost parameters and the varablty of the servce tme dstrbutons. We tested the former and observed smlar behavor for a wde range of holdng cost vectors, but dd not check for the dependence of these results on the servce tme dstrbuton, as ths would render the underlyng problem of ntractable. Partal evdence n support of usng a prcng polcy that depends on the workload rather than the queue-length vector even n the presence of general servce tmes can be obtaned usng an analyss of a second-order refnement of our formulaton based on a dffuson control problem; see, e.g., Çelk and Maglaras (5). The above results demonstrate that the prcng and sequencng polces extracted from the flud model workload formulaton perform well n a varety of parameter settngs. An nterestng drecton for future work would be to extend ths analyss to multproduct stochastc processng networks. Appendx. Proofs Proof of Proposton. Part (): We start by notng that the control specfed n Step of the proof of Theorem can be employed here as well to establsh that there exsts a par of feasble control polces and u that result n a fnte objectve functon. (Detals are omtted.) We wll frst show that s ds, and then deduce the desred result. Step. (a) Suppose that t dt =+. From (7), t follows that w t and q t as t. Because r = r r ˆ, r t c q t dt c q t dt =. Ths s suboptmal, and by contradcton, t dt<. (b) Smlarly, f lm t s ds = N for some N>, then there exsts T> such that for t T, N/ t s ds 3N/, or equvalently, + N/ t /t s ds +3N/ t. Usng the defnton of R and Jensen s nequalty, we get that for any t T, r s ds R s ds ( ) tr s ds t ( t R + R 3N ) t + o /t where R = dr /d = >. Lettng t, we get that r s ds 3N/ R. In addton, from (7),

16 Operatons Research 54(5), pp , 6 INFORMS 99 we get that w t N/ for all t T, whch, n turn, mples that c q t c w t c N/ (follows from the product labellng of ()). Ths mples that r t c q t dt 3N R T c q t dt= Argung by contradcton, we conclude that s ds, and as a sde result that r t dt. Step. Suppose that lm sup q t > > for some t product. Then, lm sup t c q s ds =+. Because r t dt, t follows that lm nf r s t c q s ds =, whch s agan suboptmal. We conclude that t suffces to restrct attenton to controls under whch q t ast. Part () of the proof follows from Seerstad and Sydsaeter (987, Theorem 6.). Proof of Proposton. Part (): From Part (b) of Proposton, we have that startng from any z and under any canddate optmal control, r t dt. Because c q t dt, ths mples that r t c q t dt and V z. For z =, the statc control ˆ M ˆ s feasble, t keeps q t = for all t, and r t c q t dt =. Part (): Consder two ntal condtons z, z wth z z, and let = V z V z. We wsh to bound V z V z =. Suppose that > and consder any feasble trajectory from z to z, and let be the frst tme that ths trajectory reaches z. Then, usng the optmalty of V z, we have that V z whch mples that r s c q s ds + V z r s c q s ds c q s ds K for N = max c N w. To get a bound on, we wll construct a control that takes the state from z to z as follows: (a) the system manager wll produce z z + orders of class = I; and (b) prce n a way such that t = ˆ for all t z z + /ˆ. Step (a) wll be completed n no longer than k z k z k + / k tme unts, whle Step (b) wll be completed n no longer than max z z + /ˆ. Combnng the two gves the bound N z z where N = max /ˆ and for any vector x, x = x. A smlar bound can be constructed n the case where < to show that < N N z z for all z z. That s, V s Lpschtz contnuous wth constant N N. Part (): Suppose that u are the optmal controls startng from ntal condtons z for =, and q t are the correspondng queue-length trajectores. Let z = z + z and defne t = t + t and u t = u t + u t. Then, t s easy to check that because L s convex, t L, and that u t and u t. Moreover, under, u, q t = z + s ds M u s ds = q t + q t t Hence, the polcy u s feasble. The concavty of V s establshed by notng that ( V z r t + t c q t + q t ) dt + r t c q t dt = V z + V z r t c q t dt where the frst nequalty follows from V s optmalty and the second from the concavty of r. Part (v): Pck any > and let = nf z z, V z k V k + for some k, and denote by z a lmtng vector along any subsequence that acheves the nfmum. If z =, the property we wsh to prove holds automatcally. (a) Suppose that z k >. Pck > small and note that the concavty of V mples that V z V z e k + V z e k k V z e k V z V z k and where e k s the kth unt vector. Addng these expressons and dvdng by gves that V z k V z e k k. From the defnton of z, t follows that V z k V k + and V z e k k < V k +, whch leads to a contradcton. Therefore, z k =. (b) Suppose that z k = and z j for some j k. A frst-order Taylor expanson gves that V z + e k = V z + V z k + o where we say that f x s o x f lm x f x /x =. From the concavty of V, we also get that V z + e k V z e j + V z e j j + V z e j k Combnng the last two expressons, we get that [ V z e j + V z e j j V z ] + [ ] V z e j k V z k + o where the frst term s o. Dvde by and let to get that V z k V k +, whch agan leads to a contradcton. Lettng gves that V z V for all z, whch completes the proof of part (v).

17 93 Operatons Research 54(5), pp , 6 INFORMS Proof of Proposton 5. The proof s by nducton on k q = max q >. Ifk q =, q = and = ˆ (from Propostons and ). Assume that the property holds f k q = and consder the case k q = +. In the remander of ths proof, u q wll denote the soluton to (7) as a functon of the varable and the queue-length vector q. The proof of the nducton step uses the followng result. Lemma. For any y I, let q y = arg max r + y Mu q L, and recall that V q V for all q. Then, q V q q V. From Lemma, t suffces to show that q y ˆ ; where we use the shorthand notaton y = V. Usng the defnton of the c -rule, we frst obtan an expresson for the allocaton control u q : j j u j q = ( >j ( >j ) + j > k q ) + j = k q j <k q (8) whch depends on q through k q = +, the ndex of the hghest-prorty nonempty class. Lettng f + = r + y Mu q and + = arg max f + L, the nducton step reduces to showng that + ˆ. Usng (8), we get that f + = f + g +, where f = r + y j + y ( l> l / l ) + [ ( y j j ) + ] l / l j l>j ( g + = y ) + ( l / l y + + ) + l / l l> l>+ [ ( + y + + ) + l / l + ] l>+ These expressons wll allow us to make use of the nducton hypothess,.e., that = arg max f L ˆ. Note that f + s the sum of concave functons, and thus t s concave tself. Smple algebrac manpulatons gve that g + = f l + l / l, and g + = y y + + l + l / l, otherwse. (The last asserton used the fact that V s decreasng n ; cf. the comment after Proposton 4.) It follows that frst, max f + L { = max f + L l + } l / l and second, from the propertes of and the functonal form of g +, that f + / j for all j. Ths establshes the nducton hypothess + ˆ, and completes the proof. Sketch of Proof of Lemma. Smlarly to the proof of Proposton 5, ths lemma can be proved by nducton on k q = max q >. Expresson (8) stll gves u q, whch only depends on q through the ndex k q. Ths mples that q y s also a functon of the queue length through k q. Denotng q y by y, when k q =, the nducton step that one would wsh to show s that y y x for any x I +. The arguments used above can be adapted n ths settng to establsh the desred result. Detals are omtted. Dervaton of Aggregate Revenue Functon Assocated wth the Lnear Demand Model. The lnear demand model s gven by p = b p j b j p j where s the market potental for product and b, b j are the prce and cross-prce senstvty parameters. Ths s expressed n vector form as p = Bp for the obvous choce of, B. Under the assumptons lsted n, the revenue functon r = B, whch s a concave quadratc. The aggregate revenue functon s defned through () (reproduced here): R = max r m = whch can be wrtten as a concave, pecewse quadratc functon n of the form R = + + for r r The dervaton gven below demonstrates how to compute the constants r gven the model parameters, B,. Ths s done for the specal case where there are no product substtuton and/or complementarty effects;.e., b j = for all j, and B = dag b b II. For convenence, we assume that products are labelled such that /m b /m b I /m I b II. Then, r = and the remanng constants,,, r are defned recursvely as follows. Frst, { r = mn = m l l = } (9) m b m b { r = mn = m l + m l l = l = } 3 (3) m b m b m 3 b 33 and so on. Second, the product-level demand rates that correspond to some are gven by r = m b for î r = otherwse and

18 Operatons Research 54(5), pp , 6 INFORMS 93 where ( = j î )( m j j j î m j b jj ) for î = max r (3) the last expressons used our labellng conventon. Note that s decreasng and î s ncreasng n, respectvely. Intutvely, the frm starts by offerng the most proftable product when s very small, and then sequentally ntroduces more products as the target consumpton rate ncreases. Fnally, R = j î r j j r j /b jj, whch gves that ( î = m j jj) b (3) j î ( î = j î ( î = 4b j î jj m j j )( j j î ) ( j î m j b jj) (33) ) m j ( j m j jj) b (34) j î For example, when <r, we have that î =,.e., only Product s offered, and therefore we should get that R = /m /m /b, whch agrees wth the constants = / m b, = / m b, and = gven by the above expressons. A smlar argument can be appled when the cross-prce senstvty parameters are nonzero. Endnotes. These capture varable producton and work-n-process nventory costs.. A detaled dervaton of these equatons under the assumptons that n /n and the state-dependent demand rate satsfes n n /n as n can be found n Mandelbaum and Pats (995), whch focused on performance analyss of queues wth state-dependent parameters n the absence of any economc consderatons. 3. The model parameters are not selected to match any partcular busness applcaton, but rather to be representatve of the many test cases that we tred n terms of ther suboptmalty gaps, traffc ntenstes, and relatve dfference of ther optmal capactes. Acknowledgments The author s grateful to Jan Van Meghem, the anonymous assocate edtor, and three referees for ther helpful comments durng the revew of ths paper. References Afeche, P. 4. Incentve-compatble revenue management n queueng systems: Optmal strategc dleness and other delayng tactcs. Workng paper, Kellogg School of Management, Northwestern Unversty, Evanston, IL. Avram, F., D. Bertsmas, M. Rcard Flud models of sequencng problems n open queueng networks; an optmal control approach. F. Kelly, R. Wllams, eds. Stochastc Networks, Vol. 7. Proceedngs of the IMA. Sprnger-Verlag, New York, Bertsekas, D. P Dynamc Programmng and Optmal Control, Vols. and. Athena Scentfc, Belmont, MA. Bller, S., L. M. A. Chan, D. Smch-Lev, J. Swann.. Dynamc prcng and the drect-to-consumer model n the automotve ndustry. E-Commerce J. (Specal edton on dynamc prcng) 5() Btran, G., R. Caldentey. 3. An overvew of prcng models for revenue management. Manufacturng Servce Oper. Management 5(3) 3 9. Brémaud, P. 98. Pont Processes and Queues: Martngale Dynamcs. Sprnger-Verlag, New York and Hedelberg, Berln, Germany. Çelk, S., C. Maglaras. 5. Dynamc prcng and leadtme quotaton for a mult-class make-to-order queue. Workng paper, Columba Unversty Busness School, New York. Chen, H., M. Z. Frank.. State dependent prcng wth a queue. IIE Trans. 3. Chen, H., D. Yao Dynamc schedulng of a multclass flud network. Oper. Res. 4(6) 4 5. Chen, X., D. Smch-Lev. 4a. Coordnatng nventory control and prcng strateges wth random demand and fxed orderng cost: The fnte horzon case. Oper. Res. 5(6) Chen, X., D. Smch-Lev. 4b. Coordnatng nventory control and prcng strateges wth random demand and fxed orderng cost: The nfnte horzon case. Math. Oper. Res. 9(3) Elmaghraby, W., P. Kesknocak. 3. Dynamc prcng: Research overvew, current practces and future drectons. Management Sc. 49() Federgruen, A., A. Hechng Combned prcng and nventory control under uncertanty. Oper. Res. 47(3) Gallego, G., G. van Ryzn Optmal dynamc prcng of nventores wth stochastc demand over fnte horzons. Management Sc. 4(8) 999. Gallego, G., G. van Ryzn A multproduct dynamc prcng problem and ts applcatons to network yeld management. Oper. Res. 45() 4 4. Harrson, J. M Brownan models of queueng networks wth heterogeneous customer populatons. W. Flemng, P. L. Lons, eds. Stochastc Dfferental Systems, Stochastc Control Theory and Applcatons, Vol.. Proceedngs of the IMA, Sprnger-Verlag, New York, Harrson, J. M Balanced flud models of multclass queueng networks: A heavy traffc conjecture. F. Kelly, R. Wllams, eds. Stochastc Networks, Vol. 7. Proceedngs of the IMA, Sprnger-Verlag, New York,. Harrson, J. M.. Brownan models of open processng networks: Canoncal representaton of workload. Ann. Appl. Probab Harrson, J. M., J. A. Van Meghem Dynamc control of Brownan networks: State space collapse and equvalent workload formulatons. Ann. Appl. Probab Horn, R. A., C. R. Johnson Matrx Analyss. Cambrdge Unversty Press, Cambrdge, UK. Kachan, S., G. Peraks.. A flud dynamcs model of dynamc prcng and nventory control for make-to-stock manufacturng systems. Workng paper, Sloan School of Management, Massachusetts Insttute of Technology, Cambrdge, MA. Kalsh, S Monopoly prcng and dynamc demand and producton cost. Marketng Sc Kleywegt, A. J.. An optmal control problem of dynamc prcng. Workng paper, Georga Insttute of Technology, Atlanta, GA. Larvere, M., J. A. Van Meghem. 4. Strategcally seekng servce: How competton can generate Posson arrvals. Manufacturng Servce Oper. Management 6() 3 4. Low, D. W Optmal dynamc prcng polces for an m/m/s queue. Oper. Res

19 93 Operatons Research 54(5), pp , 6 INFORMS Maglaras, C.. Dscrete-revew polces for schedulng stochastc networks: Trajectory trackng and flud-scale asymptotc optmalty. Ann. Appl. Probab. (3) Maglaras, C., J. Messner. 6. Dynamc prcng strateges for multproduct revenue management problems. Manufacturng Servce Oper. Management 8() Maglaras, C., A. Zeev. 3. Prcng and capacty szng for systems wth shared resources: Approxmate solutons and scalng relatons. Management Sc. 49(8) Mandelbaum, A., G. Pats State-dependent queues: Approxmatons and applcatons. F. Kelly, R. Wllams, eds. Stochastc Networks, Vol. 7. Proceedngs of the IMA, Sprnger Verlag, New York, McGll, J., G. van Ryzn Revenue management: Research overvew and prospects. Transportaton Sc. 33() Mendelson, H Prcng computer servces: Queueng effects. Comm. ACM 8(3) 3 3. Mendelson, H., S. Whang. 99. Optmal ncentve-compatble prorty prcng for the M/M/ queue. Oper. Res. 38(5) Meyn, S. P.. Sequencng and routng n multclass queung networks: Part I: Feedback regulaton. SIAM J. Control Optm. 4(3) Seerstad, A., K. Sydsaeter Optmal Control Theory wth Economc Applcatons. Elsever, Amsterdam, The Netherlands. Tallur, K., G. van Ryzn. 4. The Theory and Practce of Revenue Management. Kluwer Academc Publshers, Dordrecht, The Netherlands. Van Meghem, J.. Prce and servce dscrmnaton n queueng systems: Incentve compatblty of Gc polcy. Management Sc. 46(9)