Sensitivity Analysis of a Dynamic Fleet Management Model Using Approximate Dynamic Programming

Transcription

1 Sensiiviy Analysis of a Dynamic Flee Managemen Model Using Appoximae Dynamic Pogamming HUSEYIN TOPALOGLU School of Opeaions Reseach and Indusial Engineeing, Conell Univesiy, Ihaca, New Yok 14853, USA, opaloglu@oie.conell.edu WARREN B. POWELL Depamen of Opeaions Reseach and Financial Engineeing, Pinceon Univesiy, Pinceon, New Jesey 08544, USA, powell@pinceon.edu We pesen acable algoihms o assess he sensiiviy of a sochasic dynamic flee managemen model o flee size and load availabiliy. In paicula, we show how o compue he change in he objecive funcion value in esponse o an addiional vehicle o an addiional load inoduced ino he sysem. The novel aspec of ou appoach is ha i does no equie muliple simulaions wih diffeen values of he model paamees, and in his espec, i diffes fom ial-and-eo-based wha-if analyses. Numeical expeimens show ha he poposed mehods ae accuae and compuaionally aacive. Received Febuay 2004; acceped Febuay Subjec classificaions: Tanspoaion: models: newok. Dynamic pogamming/opimal conol: applicaions. Aea of eview: Opimizaion. 1

2 Given a vehicle flee and a sochasic pocess chaaceizing he load aivals in a anspoaion newok, he pimay objecive of flee managemen models is o make he vehicle eposiioning and vehicle-o-load assignmen decisions so ha some pefomance measue pofi, cos, deadhead miles, numbe of seved loads, ec. is opimized. Howeve, besides making hese vehicle eposiioning and assignmen decisions, an impoan quesion ha is commonly ovelooked by many flee managemen models is how he pefomance measues would change in esponse o a change in ceain model paamees. Fo example, feigh caies ae ineesed in how much hei pofis would incease if hey inoduced an addiional vehicle ino he sysem o if hey seved an addiional load on a ceain affic lane. Raiload companies wan o esimae he minimum numbe of ailcas ha is necessay o cove he andom shippe demands. The Ailif Mobiliy Command is ineesed in he impac of limied aibase capaciies on he delayed shipmens. Answeing such quesions, in one way o anohe, equies sensiiviy analysis of he undelying flee managemen model esponsible fo making he vehicle allocaion decisions. In his pape, we develop efficien sensiiviy analysis mehods fo a sochasic flee managemen model peviously developed in Godfey & Powell 2002a and Godfey & Powell 2002b. model fomulaes he poblem as a dynamic pogam, decomposing i ino ime-saged subpoblems, and eplaces he value funcions wih specially-sucued appoximaions ha ae obained hough an ieaive impovemen scheme. Two aspecs of his model ae cucial o ou wok. 1 Due o he special sucue of he value funcion appoximaions, he subpoblem ha needs o be solved fo each ime peiod is a min-cos newok flow poblem. This enables us o use he well-known elaionships beween he sensiiviy analyses of min-cos newok flow poblems and min-cos flow augmening ees. In paicula, we can use he fac ha he change in he opimal soluion of a min-cos newok flow poblem in esponse o a uni change in he supply of a node o a uni change in he uppe bound of an ac is chaaceized by a min-cos flow augmening pah. 2 Leing T be he se of ime peiods in he planning hoizon, jus as he value funcions {V : T } descibe an opimal vehicle allocaion policy hough he so-called opimaliy equaion see Pueman 1994, a se of value funcion appoximaions {Ṽ : T } descibe a possibly subopimal vehicle allocaion policy. Thus, given a ajecoy of load ealizaions {d : T }, one can hink of he ajecoy of vehicle allocaion decisions {x This : T } induced by policy unde load ealizaion ajecoy d = {d : T }. In his pape, we exploi he afoemenioned sensiiviy elaionships o compue he change in he decision ajecoy {x : T } in esponse o a change in a poblem paamee and o assess how changes in he cuen decisions affec he fuue ime peiods. paicula, we develop mehods o compue how much he pofis would change if an addiional vehicle o an addiional load wee inoduced ino he sysem. In 2

3 Ou wok is moivaed by he fac ha he feigh anspoaion indusy is ineesed in whaif scenaios, and his convenionally efes o changing ceain paamees of he model and eunning i. Howeve, hee ae obvious advanages associaed wih being able o exac sensiiviy infomaion fom a single model un. Fo example, hee can be many paamees whose impac on he model pefomance is of inees, and making one o moe un fo each paamee may be impossible. Fuhemoe, he decision-make may simply no have an idea abou he ciical paamees, and i is impoan o poin ou whee he bigges bang fo he buck lies. Also, sensiiviy infomaion is useful in deemining he opimal flee size and mix, o making picing decisions. A well-known class of models, fom which one can quickly obain sensiiviy infomaion, fomulaes he poblem ove a sae-ime newok whee he nodes epesen he supply of vehicles in diffeen saes a diffeen ime peiods and he acs epesen he vehicle movemens. Examples of such models fom hee diffeen indusies ae Danzig & Fulkeson 1954, Hane, Banha, Johnson, Masen, Nemhause & Sigismondi 1995 and Holmbeg, Jobon & Lundgen 1998, and we efe he eade o Dejax & Cainic 1987 and Powell, Jaille & Odoni 1995 fo deailed suveys. Fo hese ypes of models, sensiiviy infomaion is eadily obained by using he dual soluion. Howeve, hese models ae inheenly deeminisic and can incopoae he andom fuue load aivals only hough he expeced values. The model ha we analyze in his pape falls ino he caegoy of sochasic models, which decompose he poblem wih espec o ime peiods and assess he impac of he cuen decisions on he fuue hough value funcions. Howeve, since pacical flee managemen models involve lage numbes of decision vaiables and possible load ealizaions, sandad sochasic opimizaion mehods ae no feasible fo compuing he value funcions. Theefoe, mos of he sochasic flee managemen models evolve aound he idea of appoximaing he value funcion in a acable manne. Fo sochasic flee managemen models ohe han he one in Godfey & Powell 2002a and Godfey & Powell 2002b, we efe he eade o Fanzeskakis & Powell 1990, Cainic, Gendeau & Dejax 1993, Cavalho & Powell 2000 and Topaloglu & Powell Given a se of value funcion appoximaions, hese models behave jus like simulaion models, geneaing diffeen ajecoies of vehicle allocaion decisions fo diffeen ajecoies of load ealizaions. Theefoe, a key quesion fo hei sensiiviy analysis is o be able o assess how he decision ajecoies change when ceain model paamees ae peubed. Ou appoach has similaiies wih infiniesimal peubaion analysis, which efes o compuing he gadien of a pefomance measue in a discee-even dynamic sysem wih espec o an inpu paamee see Glasseman 1991 and Ho & Cao Howeve, fo he sysems we conside, he ansiions beween he saes ae moe complex han hose ha ae convenionally consideed by discee-even dynamic 3

4 sysems, since hese ansiions ae govened by he soluions of min-cos newok flow poblems. In his pape, we make he following eseach conibuions. We develop efficien sensiiviy analysis mehods fo a sochasic flee managemen model peviously developed in Godfey & Powell 2002a and Godfey & Powell 2002b. In paicula, we show how o compue he change in he objecive value of his model in esponse o an addiional vehicle o an addiional load inoduced ino he sysem. An accuae, bu edious, sensiiviy analysis mehod is o physically change a paamee of inees and eun he model. We compae ou mehods wih his bue foce appoach and show ha hey ae quie accuae even when heoeical analysis equies ceain appoximaions. The es of he pape is oganized as follows. In Secion 1, we biefly descibe he flee managemen model used houghou he pape. Undesanding his model is impoan because we analyze he sensiiviy of he objecive value unde he policy pescibed by his paicula model. In Secion 2, we conside poblems wih a single vehicle ype and show how o compue he change in he objecive value in esponse o an addiional vehicle o an addiional load inoduced ino he sysem. Secion 3 exends hese esuls o poblems wih muliple vehicle ypes. The compuaional expeimens pesened in Secion 4 show he accuacy of he poposed sensiiviy analysis mehods. 1 Poblem Fomulaion We have a flee of vehicles o seve he loads of diffeen ypes ha aive ove ime. A evey ime peiod, a ceain numbe of loads ene he sysem, and we have o decide which loads o cove and o which locaions we should eposiion he empy vehicles. We ae ineesed in maximizing he oal expeced pofi ove a finie hoizon, bu we fomulae ou model o minimize cos fo compaibiliy wih he min-cos newok flow lieaue. We assume ha advance infomaion abou he fuue loads is no available and he loads ha canno be coveed in a given ime peiod ae seved by an emegency subconaco. These enable us o assume ha he uncoveed loads immediaely leave he sysem. Fo noaional beviy, we assume ha i akes one ime peiod o move beween any pai of locaions. I is saighfowad o exend ou analysis o he case whee hee ae muli-peiod avel imes by using he appoach descibed in Topaloglu & Powell We define he following. T = Se of ime peiods in he planning hoizon, T = {1,..., T }. I = Se of locaions in he anspoaion newok. K = Se of available vehicle ypes. L = Se of movemen modes, L = {0,..., L}. Movemen modes epesen diffeen ways in which a vehicle can move fom one locaion o anohe. Movemen mode 0 always 4

5 x k ijl c k ijl coesponds o empy eposiioning, wheeas ohe modes coespond o caying diffeen ypes of loads. = Numbe of vehicles of ype k dispached fom locaion i o j a ime peiod using movemen mode l. = Cos of dispaching one vehicle of ype k fom locaion i o j a ime peiod using movemen mode l. D ijl = Random vaiable epesening he numbe of loads ha need o be caied fom locaion i o j a ime peiod and coespond o movemen mode l. As will be clea sholy, he andom vaiable D ijl seves as an uppe bound on he decision vaiables {x k ijl : k K}. Since he movemen mode 0 coesponds o empy eposiioning and he empy eposiioning movemens ae no bounded, we assume ha D ij0 = fo all i, j I, T. In pacice, he movemen modes in L \ {0} may coespond o diffeen ypes of loads o diffeen shippes, and we usually have c k ijl < 0 when l L \ {0}. The vehicle ype may eflec he size of he vehicle, he skill level of he dive of he vehicle, he abiliy of he vehicle o saisfy ceain safey o saniay equiemens, o a combinaion of hese facos, which ulimaely deemine whehe i is feasible o cove a ceain ype of load wih a ceain ype of vehicle and wha pofi is obained by doing so. If i is infeasible o cove a load of ype l wih a vehicle of ype k, hen we capue his by leing c k ijl = fo all i, j I, T. Thoughou he pape, we use d ijl o denoe a paicula ealizaion of D ijl. By suppessing some of he indices in he vaiables above, we denoe a veco composed of he componens anging ove he suppessed indices. Fo example, we have d = {d ijl : i, j I, l L}, d = {d ijl : i, j I, l L, T }. To capue he sae of he sysem a ime peiod, we define k i = Numbe of vehicles of ype k ha ae available a locaion i a ime peiod. The veco = {i k : i I, k K} compleely defines he sae of he vehicles a ime peiod. Given his sae veco and he ealizaion of he loads a ime peiod, he se of feasible decision vecos and he se of sae vecos geneaed by hese decisions a he nex ime peiod ae given by Y, d = { x, +1 : x k ijl = k i fo all i I, k K 1 j I l L x k ijl k j,+1 = 0 fo all j I, k K 2 i I l L x k ijl d ijl fo all i, j I, l L 3 k K x k ijl Z + } fo all i, j I, l L, k K. 4 5

6 We ae ineesed in Makovian deeminisic policies ha minimize he oal expeced cos ove he planning hoizon. A Makovian deeminisic policy can be chaaceized by a sequence of decision funcions {X, : T } such ha X, maps he sae veco and he ealizaion of he loads d a ime peiod o a decision veco x. One can also define he sae ansiion funcions {R+1, : T } of policy such ha R +1, maps he sae veco and he ealizaion of he loads a ime peiod o a sae veco fo he nex ime peiod. We noe ha given X,, R+1, can easily be defined by noing he sae ansiion consains in 2. Then, fo a given sae veco and ealizaion of fuue loads {d,..., d T } a ime peiod, he cumulaive cos funcion fo policy can be wien ecusively as F, d, d +1,..., d T = c X, d + F+1 R +1, d, d +1, d +2,..., d T, 5 wih he bounday condiion F T +1 = 0. By epeaed applicaion of 5, we obain F 1 1, d 1,..., d T = c1 X 1 1, d 1 + c 2 X 2 R 2 1, d 1, d c T X T R T R T 1..., d T 2, d T 1, d T, 6 which is he oal cos incued ove he whole planning hoizon when we use policy, he iniial sae veco is 1 and he ealizaion of he loads is {d 1,..., d T }. Assuming ha, given, D is independen of {D : = 1,..., 1}, i can be shown ha he opimal policy is Makovian deeminisic, saisfying = agmin Π E { F1 } 1, D 1,..., D T 1, whee Π is he se of Makovian deeminisic policies. This opimal policy can be found by compuing he value funcions hough he so-called opimaliy equaion see Pueman 1994 { V = E } min c x + V x, +1 Y,D In his case, he decision and ansiion funcions fo he opimal policy become X, d, R+1, d = agmin c x + V x, +1 Y,d Thoughou he pape, o keep he pesenaion simple, we assume ha he cos veco c is peubed by small andom amouns so ha poblem 8 has a single opimal soluion. Unde his assumpion, he decision and sae ansiion funcions ae popely defined, and ou poofs become easie. Compuing he value funcions {V : T } hough 7 is inacable fo almos all poblem insances of pacical significance, since i equies enumeaing ove all possible values of and aking an expecaion ove he muli-dimensional andom vaiable D fo all T. In his pape, we follow a class of subopimal policies poposed in Godfey & Powell 2002a, which ae obained by eplacing {V : T } in 8 wih sepaable appoximaions {Ṽ : T } of he fom Ṽ = i I k K Ṽ k i k i 9 6

7 whee each Ṽ i k is a one-dimensional piecewise-linea convex funcion wih poins of nondiffeeniabiliy being a subse of posiive ineges. In his case, fo a policy chaaceized by sepaable piecewise-linea convex value funcion appoximaions {Ṽ : T }, we can define he decision and sae ansiion funcions as X, d, R+1, d = agmin x, +1 Y,d c x + Ṽ We noe ha alhough he value funcion appoximaions {Ṽ : T } ae sepaable, he cumulaive cos funcions {F, d,..., d T : T } fo policy ae no necessaily sepaable. Fuhemoe, we have V = E { } F, D,..., D T fo he opimal policy by he pincipal of opimaliy see Pueman 1994, bu we do no necessaily have Ṽ = E { F, D,..., D T } fo he policy chaaceized by he value funcion appoximaions {Ṽ : T }. Godfey & Powell 2002a give a sampling-based algoihm ha can be used o obain a good se of value funcion appoximaions. The quesion of whehe hese subopimal policies yield highqualiy soluions is ouside he scope of his pape. We efe he eade o Godfey & Powell 2002a, Godfey & Powell 2002b and Topaloglu & Powell 2006 whee he expeimenal wok indicaes ha his class of policies bea sandad benchmaks by significan magins. Hee, we assume ha we aleady have a good policy, and we ae ineesed in compuing he change in F 1 1, d 1,..., d T induced by changing an elemen of he sae veco 1 o he load availabiliy veco d 1. We make his quesion pecise in he nex wo secions. Howeve, befoe going ino he specific deails, we can summaize he conens of he nex wo secions as follows. 1 We noe ha if we use a policy chaaceized by sepaable piecewise-linea convex value funcion appoximaions, hen poblem 10 is a min-cos newok flow poblem. 2 We use he well-known elaionships beween he sensiiviy analyses of min-cos newok flow poblems and min-cos flow augmening ees o find how he soluion of poblem 10 a ime peiod 1 changes in esponse o an addiional vehicle o an addiional load inoduced ino he sysem. 3 We find how he sae veco a ime peiod 2 changes in esponse o he change in he soluion of poblem 10 a ime peiod 1. 4 Finally, we find how he soluion of poblem 10 a ime peiod 2 changes in esponse o he change in he sae veco a ime peiod 2. We epea he same agumen in a ecusive fashion fo he subsequen ime peiods. In Secion 2, we sa by consideing poblems wih a single vehicle ype. We genealize he ideas o muliple vehicle ypes in Secion 3. 2 Poblems wih a Single Vehicle Type In his secion, we assume ha K = 1 and dop he vehicle ype supescip, in which case 9 becomes Ṽ = i I Ṽ i i. Leing R be he oal numbe of available vehicles, he elevan 7

8 domain of Ṽ i is {0, 1,..., R}. Theefoe, assuming ha Ṽ i 0 = 0 wihou loss of genealiy, we can epesen Ṽ i by a sequence of numbes {ṽ i q : q = 1,..., R} whee ṽ i q is he slope of Ṽi q ove q 1, q. Tha is, we have ṽ i q = Ṽ i q Ṽ i q 1. In his case, poblem 10 can explicily be wien as min c ijl x ijl + R ṽj,+1q z j,+1 q 11 x, +1,z +1 i,j I l L j I q=1 subjec o x ijl = i fo all i I 12 j I l L x ijl j,+1 = 0 fo all j I 13 i I l L j,+1 R z j,+1 q = 0 fo all j I 14 q=1 x ijl d ijl fo all i, j I, l L 15 z j,+1 q 1 fo all j I, q = 1,..., R 16 x ijl, j,+1, z j,+1 q Z + fo all i, j I, l L, q = 1,..., R, 17 whee we use a sandad echnique o embed he piecewise-linea convex funcions {Ṽ j,+1 : j I} ino poblem above hough he decision vaiables {z j,+1 q : j I, q = 1,..., R}. In paicula, due o he convexiy of Ṽ j,+1, we have ṽ j,+1 1 ṽ j, ṽ j,+1 R. Since he objecive funcion is minimized, noing consains 14, 16 and 17, we mus have R q=1 ṽ j,+1 q z j,+1q = j,+1 q=1 ṽj,+1 q = Ṽ j,+1 j,+1 in he opimal soluion. Theefoe, he second em in 11 compues j I Ṽ j,+1 j,+1 see Nemhause & Wolsey Alhough consains 13 and 14 can be combined ino i I l L x ijl R q=1 z j,+1q = 0, we leave hem sepaae o emphasize ha 13 handles he sae ansiion, wheeas 14 handles he compuaion of he value funcion appoximaion. I is easy o see ha poblem 11 is he min-cos newok flow poblem in Figue 1. In his figue, we assume ha I = {a, b, c} and L = {n, m}. Consains 12, 13 and 14 especively coespond o he flow balance consains fo he whie, gay and black nodes. The ses of decision vaiables {x ijl : i, j I, l L}, { j,+1 : j I} and {z j,+1 q : j I, q = 1,..., R} especively coespond o he acs ha leave he whie, gay and black nodes. 2.1 Policy gadiens wih espec o vehicle availabiliies In his secion, we develop a mehod o compue how much he oal cos unde policy would change if an addiional vehicle wee inoduced ino he sysem. We le { x : T } and { : T } be he sequences of decisions and saes visied by he sysem unde policy and load ealizaion d = {d : T }. Tha is, {x x : T } and { = X, d, +1 = R+1 : T } ae ecusively compued by, d, wih 1 =

9 a a n b n b sink node c c m xijl j, + 1 z j, + 1 q Figue 1: Poblem 11 is a min-cos newok flow poblem. The pah in bold acs epesens a possible min-cos flow augmening pah fom node a on he lef side o he sink node. Such min-cos flow augmening pahs will be useful in Secion 2.1. Then, noing 5, F, d,..., d T becomes he oal cos incued a ime peiods {,..., T } unde policy and load ealizaion d. Leing e i be he I -dimensional uni veco wih a 1 in he elemen coesponding o i I, ou objecive in his secion is o compue Φ e i, d = F + e i, d,..., d T F, d,..., d T 19 fo all i I, T. Then, E { Φ 1 e i, D } ells us how much he oal expeced cos unde policy would change by inoducing an addiional vehicle a locaion i a he fis ime peiod. We noe ha Φ e i, d can be compued by wo simulaions of policy unde load ealizaion d, one of which sas wih he sae veco and he ohe wih + e i. Howeve, doing his fo all i I, T and fo muliple load ealizaions can ge ime consuming. Ou objecive is o be able o compue Φ e i, d fo all i I, T fom a single simulaion. Using 5 and 18, 19 can be wien as Φ e i, d = c {X + e i, d X, d } + F+1 R +1 + e i, d, d +1,..., d T F +1 R +1, d, d +1,..., d T = c {X + F +1 + e i, d x R +1 } + e i, d, d +1,..., d T F +1 +1, d +1,..., d T. 20 As will be clea sholy, compuing X compuing Φ e i, d. Since we have + e i, d x and R +1 + e i, d +1 is key o x, +1 = agmin c x + Ṽ +1 +1, 21 x, +1 Y,d 9

10 X + e i, d x and R+1 + e i, d +1 ae elaed o how he soluion of poblem 11 changes when he igh side of consains 12 is inceased fom Conside poblem 21 and is newok epesenaion in Figue 1. o + e i. Se he flows on he acs in his newok such ha hese flows coespond o he opimal soluion x, +1. Le P e i, d be he min-cos flow augmening pah fom node i I on he lef side o he sink node in his figue. One possible flow augmening pah when i = a is shown in bold acs. We define he veco ξ e i, d = {ξıjl e i, d : ı, j I, l L} as +1 if he ac coesponding o vaiable x ıjl is a fowad ac in P e i, d ξıjl e i, d = 1 if he ac coesponding o vaiable x ıjl is a backwad ac in P e i, d 0 if he ac coesponding o vaiable x ıjl is no in P e i, d. Similaly, we define he veco δ+1 e i, d = {δj,+1 e i, d : j I} as +1 if he ac coesponding o vaiable j,+1 is a fowad ac in P e i, d δj,+1e i, d = 1 if he ac coesponding o vaiable j,+1 is a backwad ac in P e i, d 0 if he ac coesponding o vaiable j,+1 is no in P e i, d. Fo example, fo he flow augmening pah in Figue 1, we have ξ abn e a, d = +1, ξ cbn e a, d = 1, ξ ccme a, d = +1 and δ c,+1 e a, d = +1. The following esul chaaceizes how he soluion of poblem 21 changes when he numbe of vehicles available a locaion i is inceased by Lemma 1 The following esuls hold. 1 We have X + e i, d = x + ξ e i, d and R +1 + e i, d = +1 + δ +1 e i, d. 2 One elemen of he veco δ +1 e i, d is equal o +1 and he ohe elemens ae equal o 0. Poof The fis pa is a diec esul of Theoem 1 in Powell The second pa holds because any acyclic pah fom node i I on he lef side of Figue 1 o he sink node aveses exacly one of he acs coesponding o one of he vaiables { j,+1 : j I}. The second pa of he lemma implies ha if an addiional vehicle is inoduced a a ceain locaion a ime peiod, hen exacly one elemen of he sae veco a ime peiod + 1 will incease by 1. The following poposiion gives an efficien mehod o compue Φ e i, d. Poposiion 2 We have Φ e i, d = c ξ e i, d + Φ +1 δ +1 e i, d, d fo all i I, T wih he bounday condiion Φ T +1, d = 0. Poof Using Lemma 1, 20 can be wien as Φ e i, d = c ξ e i, d + F δ+1e i, d, d +1,..., d T F +1 +1, d +1,..., d T = c ξ e i, d + Φ +1δ +1e i, d, d. 10

11 The fis em in c ξ e i, d + Φ +1 δ +1 e i, d, d capues how much he cos incued a ime peiod changes in esponse o an addiional vehicle a locaion i a ime peiod, wheeas he second em capues how much he cos incued a ime peiods { + 1,..., T } changes in esponse o an addiional vehicle a locaion i a ime peiod. Thus, he idea is o sa wih he las ime peiod T and le Φ T e i, d = c T ξt e i, d fo all i I. Then, we move o ime peiod T 1. Since δt e i, d is always a posiive inege uni veco, Φ T 1 e i, d can easily be compued as c T 1 ξt 1 e i, d + Φ T δ T e i, d, d. We coninue in a simila fashion unil we each he fis ime peiod. We noe ha o evaluae he expeced cos impac of an addiional vehicle, we need o compue E { Φ e i, D } as opposed o Φ e i, d fo a paicula load ealizaion d. In his case, since compuing his expecaion is usually inacable, we can sample N load ealizaions, say d 1,..., d N, use he mehod descibed in his secion o compue Φ e i, d n fo all n = 1,..., N and use he sandad confidence ineval mehodology o esimae E { Φ e i, D }. By caying ou a pilo un ha uses a small numbe of load ealizaions, we can assess he numbe of load ealizaions ha ae needed o esimae E { Φ e i, D } wih a ceain pecision see Law & Kelon Finally, we noe ha a simila mehod o compue Φ e i, d = F e i, d,..., d T F, d,..., d T 23 can be developed by using min-cos flow deceasing ees. This will be useful in he nex secion. 2.2 Policy gadiens wih espec o load availabiliies Feigh caies coninuously face he poblem of evaluaing newly aiving loads o decide whehe o accep o ejec hem. In his secion, we develop a mehod o compue how much he oal cos unde policy would change if an addiional load wee inoduced ino he sysem. This infomaion can, in un, be used fo load evaluaion decisions. The class of policies we conside assume ha hee is no advance infomaion abou fuue load ealizaions. Fo his eason, we assume ha if is he cuen ime peiod, hen he addiional load ha is inoduced ino he sysem is a load ha needs o be seved a ime peiod. Leing e ijl and e ijl be he I 2 L T and I 2 L -dimensional uni vecos wih a 1 in he elemen coesponding o i, j I, l L, T and i, j I, l L, we wan o compue Ψ e ijl, d = F,d+e ijl, d + e ijl, d +1,..., d T F, d, d +1,..., d T, 24 which is he change in he oal cos of policy unde load ealizaion d in esponse o an addiional load of ype l on lane i, j a ime peiod. Using an agumen simila o he one in 20 and noing 11

12 a n a b b sink node c m n c xijl j, + 1 z j, + 1 q Figue 2: Q e a, e c, d is he min-cos flow augmening pah fom node c in he middle secion o node a on he lef side. ha,d+e ijl be wien as is a funcion of he load ealizaions up o bu no including ime peiod, 24 can Ψ e ijl, d = F = c {X + F +1, d + e ijl, d +1,..., d T F, d, d +1,..., d T, d + e ijl x } R +1, d + e ijl, d +1,..., d T F +1 +1, d +1,..., d T. 25 To compue Ψ e ijl, d, we now need o chaaceize X, d +e ijl x and R+1, d +e ijl +1. These quaniies ae elaed o how he soluion of he min-cos newok flow poblem 11 changes when he uppe bound on he decision vaiables x is inceased fom d o d + e ijl. Conside poblem 21 and is newok epesenaion in Figue 2. Se he flows on he acs in his newok such ha hese flows coespond o he opimal soluion x, +1. Le Q e i, e j, d be he min-cos flow augmening pah fom node j I in he middle secion o node i I on he lef side in his figue. Denoe he cos of his min-cos flow augmening pah by C e i, e j, d. One possible flow augmening pah when i = a, j = c is shown in dashed acs. We define he veco ξ e i, e j, d = {ξıjl e i, e j, d : ı, j I, l L} as +1 if he ac coesponding o vaiable x ıjl is a fowad ac in Q e i, e j, d ξıjl e i, e j, d = 1 if he ac coesponding o vaiable x ıjl is a backwad ac in Q e i, e j, d 0 if he ac coesponding o vaiable x ıjl is no in Q e i, e j, d. We also define he veco δ +1 e i, e j, d simila o 22, bu using he flow augmening pah Q e i, e j, d. Fo example, fo he flow augmening pah in Figue 2, we have ξ aan e a, e c, d = 1, ξ cbm e a, e c, d = +1, ξ ccn e a, e c, d = 1, δ a,+1 e a, e c, d = 1 and δ b,+1 e a, e c, d = +1. The following esul chaaceizes how he soluion of poblem 21 changes when he numbe of loads of ype l on lane i, j is inceased by 1. 12

13 Lemma 3 The following esuls hold. { 1 if a < b 1 Leing 1 {a<b} = we have 0 ohewise, X, d + e ijl = x { } + 1 {C e i,e j,d+c ijl <0} ξ e i, e j, d + e ijl R+1, d + e ijl = {C e i,e j,d+c ijl <0} δ+1 e i, e j, d. 2 The veco δ +1 e i, e j, d can be wien as δ +1 e i, e j, d = δ + +1 e i, e j, d δ +1 e i, e j, d whee one elemen of each of he vecos δ + +1 e i, e j, d and δ +1 e i, e j, d is equal o +1 and he ohe elemens ae equal o 0. Poof See he appendix. Theefoe, if C e i, e j, d + c ijl 0, hen an addiional load of ype l on lane i, j does no change he soluion of poblem 21. We noe ha he min-cos flow augmening pah Q e i, e j, d may no include any acs coesponding o one of he vaiables { j,+1 : j I}. In his case, we have δ +1 e i, e j, d = 0 and we can se δ + +1 e i, e j, d = δ +1 e i, e j, d in he second pa of Lemma 3. The following poposiion gives an efficien mehod o compue Ψ e ijl, d. Poposiion 4 Leing ζ { } e ijl, d = 1 {C e i,e j,d+c ijl <0} ξ e i, e j, d+e ijl fo noaional beviy, we have he following esuls. 1 If C e i, e j, d + c ijl 0 o δ+1 e i, e j, d = 0, hen we have Ψ e ijl, d = c ζ e ijl, d. 2 If F+1, d+1,..., d T is a sepaable funcion, C e i, e j, d + c ijl < 0 and δ+1 e i, e j, d 0, hen we have Ψ e ijl, d = c ζ e ijl, d + Φ +1δ + +1 e i, e j, d, d + Φ +1 δ +1 e i, e j, d, d 26 fo all i, j I, l L, T, whee Φ +1 e i, d is as defined in 19 and 23. Poof Unde he condiions saed in he fis pa, Lemma 3 implies ha R+1, d +e ijl = +1 and 25 becomes Ψ e ijl, d = c {X, d + e ijl x }. Then, he fis pa follows by he definiion of ζ e ijl, d and Lemma 3. By using Lemma 3, 25 becomes Ψ e ijl, d = c ζ e ijl, d + F δ+1 + e i, e j, d δ +1 e i, e j, d, d +1,..., d T F+1 +1, d +1,..., d T. 13

14 Since F +1, d+1,..., d T is sepaable, Lemma 8 in he appendix implies ha Ψ e ijl, d = c ζ e ijl, d + F δ+1 + e i, e j, d, d +1,..., d T F +1 +1, d +1,..., d T + F+1 +1 δ+1 e i, e j, d, d +1,..., d T F +1 +1, d +1,..., d T. The fis pa of he poposiion coesponds o he case whee an addiional load of ype l on lane i, j a ime peiod eihe does no change he decisions a ime peiod o does no change he sae veco a ime peiod + 1. The second pa coesponds o he case whee an addiional load of ype l on lane i, j a ime peiod does change he sae veco a ime peiod + 1. Given he fac ha δ + +1 e i, e j, d and δ +1 e i, e j, d ae posiive inege uni vecos, 26 can easily be compued once we know Φ +1 e i, d fo all i I. As noed in Secion 1, F +1, d+1,..., d T is no necessaily a sepaable funcion. Howeve, we popose using 26 as an appoximaion o Ψ e ijl, d even when F +1, d+1,..., d T is no sepaable. Ou compuaional expeimens show ha his appoximaion yields accuae esuls. We believe ha he accuacy of his appoximaion is due o he following eason. The expession in 26 capues he change in he oal cos of policy unde load ealizaion d in esponse o an addiional load of ype l on lane i, j a ime peiod. Among he hee ems on he igh side of 26, he fis em accuaely capues he change in he cos incued a ime peiod, wheeas he sum of he second and hid ems appoximaely capues he change in he cos incued a ime peiods { + 1,..., T }. Theefoe, accuaely capuing he change in he cos incued a he cuen ime peiod and appoximaely capuing he change in he cos incued a he fuue ime peiods appea o be adequae o obain a good appoximaion. Assuming ha C e a, e c, d+c acm < 0 and noing ha δ a,+1 e a, e c, d = 1, δ b,+1 e a, e c, d = +1 fo he min-cos flow augmening pah in Figue 2, he appoximaion in 26 can be inepeed as follows. The fis em gives he change in he immediae cos due o he change in he decisions a ime peiod. The second em gives he change in he fuue cos due o having an addiional vehicle a locaion b a ime peiod + 1. The hid em gives he change in he fuue cos due o having one less vehicle a locaion a a ime peiod Poblems wih Muliple Vehicle Types In his secion, we exend he ideas in Secion 2 o he case whee hee ae muliple vehicle ypes. Topaloglu & Powell 2006 noe ha if hee ae muliple vehicles ypes and policy is chaaceized by a se of sepaable piecewise-linea convex value funcion appoximaions, hen poblem 10 14

15 becomes a min-cos inege mulicommodiy newok flow poblem, and his inhibis exploiing popeies of min-cos flow augmening and deceasing ees as we did in he pevious secion. Noneheless, hey also show ha if each Ṽ is a linea funcion of he fom Ṽ = i I k K ṽk i i k, hen poblem 10 is a min-cos newok flow poblem. Fuhemoe, ove a limied domain, say [0, 1] o [0, 2], linea funcions appoximae piecewise-linea funcions quie well. Since he sum of he elemens of he I K -dimensional veco is always equal o he numbe of available vehicles, say R, if I K R, hen we expec he elemens of he veco o be mosly 0 s, 1 s o 2 s. In his case, using piecewise-linea appoximaions does no bing oo much advanage ove linea appoximaions. Fo hese easons, when woking on poblems wih muliple vehicle ypes, we use policies chaaceized by linea value funcion appoximaions. We now exend he esuls of Secion 2.1 o he case of muliple vehicle ypes. Noing 2, he objecive funcion of poblem 10 unde a policy defined by linea value funcion appoximaions is c x + Ṽ = c k ijl xk ijl + ṽj,+1 k x k ijl. i,j I l L k K j I k K i I l L Then, he decision funcion fo policy can be wien as X, d = agmin x i,j I l L k K subjec o 1, 3, 4, c k ijl + ṽk j,+1 x k ijl 27 which is he min-cos newok flow poblem in Figue 3. In his figue, we assume ha I = {a, b}, L = {n, m} and K = {f, g}. Consains 1 epesen he flow balance consains fo he gay nodes. Defining he addiional decision vaiables {w ijl : i, j I, l L} and spliing consains 3 ino wo ses of consains k K xk ijl w ijl = 0 and w ijl d ijl fo all i, j I, l L, he fis se epesens he flow balance consains fo he whie nodes in he middle secion. We le be a policy chaaceized by he linea value funcion appoximaions {Ṽ : T } wih he decision, sae ansiion and cumulaive cos funcions X,, R +1,, F,,...,. We also define {x : T } and { : T } simila o hei counepas in Secion 2.1. Leing e k i be he I K -dimensional uni veco wih a 1 in he elemen coesponding o i I, k K, we wan o compue Φ e k i, d = F + e k i, d,..., d T F, d,..., d T fo all i I, k K, T. Conside poblem 27 and is newok epesenaion in Figue 3. Se he flows on he acs in his newok such ha hese flows coespond o he opimal soluion x. Le P e k i, d be he min-cos flow augmening pah fom node i, k I K on he lef side o he sink node in his figue. One possible flow augmening pah when i, k = a, f is shown in bold acs. We define he 15

16 diffeen locaions a, f a, g d aan d aam d abn d abm sink node b, f d ban d bam diffeen vehicle ypes b, g k x ijl d bbn d bbm uppe bounds ae d ijl Figue 3: Poblem 27 is a min-cos newok flow poblem. veco ξ e k i, d = {ξκ ıjl ek i, d : ı, j I, l L, κ K} as +1 if he ac coesponding o vaiable x κ ξıjl κ ıjl is a fowad ac in P e k i, d ek i, d = 1 if he ac coesponding o vaiable x κ ıjl is a backwad ac in P e k i, d 0 if he ac coesponding o vaiable x κ ıjl is no in P e k i, d. We also define he veco δ+1 ek i, d = {δκ j,+1 ek i, d : j I, κ K} as δ κ j,+1e k i, d = ı I l L ξ κ ıjl ek i, d. 28 Fo example, fo he flow augmening pah in Figue 3, we have ξ f aan ef a, d = +1, ξ g aan ef a, d = 1, ξ g abm ef a, d = +1, δ f a,+1 ef a, d = +1 δ g a,+1 ef a, d = 1 and δ g b,+1 ef a, d = +1. The following esul chaaceizes how he soluion of poblem 27 changes when he numbe of vehicles of ype k available a locaion i is inceased by 1. Lemma 5 The following esuls hold. 1 We have X + e k i, d = x + ξ e k i, d and R +1 + e k i, d = +1 + δ +1 ek i, d. 2 Thee exis wo disjoin subses of I K, say + +1 ek i, d and +1 ek i, d, such ha δ +1 ek i, d can be wien as δ +1e k i, d = κ ej j,κ + +1 ek i,d κ ej. j,κ +1 ek i,d 16

17 Poof In he fis pa, he fis equaliy is a diec esul of Theoem 1 in Powell 1989 and he second equaliy follows fom he definiion of δ+1 ek i, d in 28 and he sae ansiion consains 2. We show he second pa in he appendix. The second pa of he lemma shows ha an addiional vehicle of ype k a locaion i a ime peiod may change he sae veco a ime peiod +1 in a complicaed manne, bu each componen of he sae veco a ime peiod + 1 changes by a mos 1. The following poposiion gives an efficien mehod o compue Φ e k i, d. Poposiion 6 If F +1, d+1,..., d T is a sepaable funcion, hen we have Φ e k i, d = c ξ e k i, d + Φ j,κ + +1 ek i,d +1e κ j, d + fo all i I, k K, T wih he bounday condiion Φ T +1, d = 0. Φ j,κ +1 ek i,d +1 e κ j, d 29 Poof Using Lemma 5, we have Φ e k i, d = c ξ e k i, d + F +1 = c ξ e k i, d + + j,κ + +1 ek i,d j,κ +1 ek i,d +1 + κ j e j,κ + +1 ek i,d κ e j,κ +1 ek i,d j, d +1,..., d T, d +1,..., d T F+1 { F e κ } j, d +1,..., d T F +1 +1, d +1,..., d T { F e κ j, d +1,..., d T F +1 +1, d +1,..., d T }, whee he second equaliy uses he sepaabiliy assumpion and Lemma 8 in he appendix. On he igh side of 29, he fis em capues how much he cos incued a ime peiod changes in esponse o an addiional vehicle of ype k a locaion i a ime peiod, wheeas he second and hid ems capue how much he cos incued a ime peiods { + 1,..., T } changes in esponse o an addiional vehicle of ype k a locaion i a ime peiod. F+1, d+1,..., d T is no necessaily a sepaable funcion. Howeve, we popose using 29 as an appoximaion o Φ e k i, d even when F +1, d+1,..., d T is no sepaable. 4 Compuaional Expeimens This secion focuses on he esuls of Secions 2.2 and 3, and numeically esablishes he accuacy of he mehods poposed o compue Ψ e ijl, d and Φ e k i, d. In paicula, we use a vaiey of es poblems o show ha 26 and 29 can appoximae Ψ e ijl, d and Φ e k i, d accuaely even 17

18 when F +1, d+1,..., d T is no a sepaable funcion. The mehod poposed o compue Φ e i, d in Secion 2.1 is exac and does no equie numeical validaion. Ou es poblems involve 40 locaions and 41 movemen modes 40 load ypes and one movemen mode fo empy eposiioning. We label ou es poblems by T, D, K, R, e, whee T is he lengh of he planning hoizon, D is he expeced numbe of loads ove he planning hoizon, K is he numbe of vehicle ypes, R is he numbe of available vehicles and e is he empy eposiioning cos applied on a pe-mile basis. Accuacy of he policy gadiens wih espec o load availabiliies. We sa by esing he accuacy of he mehod poposed o compue Ψ e ijl, d. Ou expeimenal seup is as follows. Fo each es poblem, we fis obain a good vehicle allocaion policy by using he sampling-based mehod of Godfey & Powell 2002a. Having obained a policy, we sample N load ealizaions, say d 1,..., d N. Fo each load ealizaion d n, we appoximae Ψ 1 e ijl, d n fo all i, j I, l L by using 26. We le { Ψ 1 e ijl, d n : i, j I, l L, n = 1,..., N} be hese appoximaions. Since he mehod given in Secion 2.2 equies simulaing he behavio of policy unde load ealizaion d n, a his poin we can also compue F1 1, d n 1, dn 2,..., dn T as T c x n. We hen physically incease he numbe of loads of ype l on lane i, j a ime peiod 1 by 1 and compue F1 1, d n 1 +e ijl, d n 2,..., dn T by simulaing he behavio of policy unde load ealizaion d n +e ijl1. In his way, we can accuaely compue Ψ 1 e ijl, d n in a bue foce fashion as F1 1, d n 1 +e ijl, d n 2,..., dn T F 1 1, d n 1, dn 2,..., dn T. Ou aim is o compae he appoximaion Ψ 1 e ijl, d n ha is compued hough 26 wih Ψ 1 e ijl, d n ha is compued in a bue foce fashion. Table 1 summaizes ou findings. The fis se of columns give he aveage pecen deviaion and he coefficien of coelaion beween {Ψ 1 e ijl, d n : i, j I, l L, n = 1,..., N} and { Ψ 1 e ijl, d n : i, j I, l L, n = 1,..., N}. The second se of columns give a hisogam fo he pecen deviaions ha shows wha facion of he pecen deviaions is less han 2.5%, 5%, 10% and 25%. The nex column gives he aveage ime o compue { Ψ 1 e ijl, d n : i, j I, l L} fo a paicula load ealizaion d n. This ime includes he ime spen simulaing he behavio of policy unde load ealizaion d n. The las se of columns give summay saisics fo he coefficiens of vaiaion of {Ψ 1 e ijl, D : i, j I, l L}. We esimae he coefficien of vaiaion of Ψ 1 e ijl, D as µ ijl /σ ijl whee µ ijl and σ 2 ijl ae he sample mean and sample vaiance of {Ψ 1 e ijl, d n : n = 1,..., N}. Using his esimae of coefficien of vaiaion, one can have an idea of how many load ealizaions ae needed o esimae E { Ψ 1 e ijl, D } wih a ceain pecision see Law & Kelon Since µ ijl /σ ijl depends on i, j I, l L, we give he mean, and he 20-h and 80-h peceniles of {µ ijl /σ ijl : i, j I, l L}. We noe ha he coefficien of vaiaion esimaes ae highly poblem-specific and one should no daw geneal conclusions fom hem. 18

19 Co. Avg. Hisogam Time Coeff. of vaiaion Poblem coeff. % dev. 2.5% 5% 10% 25% sec. Avg. 20 p. 80 p. 10, 1000, 1, 100, , 1000, 1, 100, , 1000, 1, 100, , 1000, 1, 200, , 1000, 1, 200, , 1000, 1, 200, , 3000, 1, 100, , 3000, 1, 100, , 3000, 1, 100, , 3000, 1, 200, , 3000, 1, 200, , 3000, 1, 200, Table 1: Accuacy of he policy gadiens wih espec o load availabiliies. Pecen deviaion is 100 Ψ 1 e ijl, d n Ψ 1 e ijl, d n / Ψ 1 e ijl, d n and we ignoe he daa poins wih Ψ 1 e ijl, d n = 0. The high coefficien of coelaion and he low aveage pecen deviaion figues in Table 1 show ha {Ψ 1 e ijl, d n : i, j I, l L, n = 1,..., N} and { Ψ 1 e ijl, d n : i, j I, l L, n = 1,..., N} ae in close ageemen. The hisogams show ha, abou 90% of he ime, ou appoximaions ae wihin 25% of he ue value. If we wee woking on poblems wih deeminisic load aivals, hen Ψ 1 e ijl, d could also be appoximaed by using he opimal value of he dual vaiable associaed wih he load availabiliy consain x ijl1 d ijl1 in he sae-ime newok fomulaion of he poblem. Powell 1989 epos ha, 10% of he ime, appoximaing Ψ 1 e ijl, d by using he dual soluion bings an eo of 50% o moe. Theefoe, ou mehod can appoximae Ψ 1 e ijl, d noiceably bee han he dual soluion of he sae-ime newok fomulaion. Accuacy of he policy gadiens wih espec o vehicle availabiliies. We now compae he appoximaions obained hough 29, say { Φ 1 ek i, dn : i I, k K, n = 1,..., N}, wih he values of {Φ 1 ek i, dn : i I, k K, n = 1,..., N} obained in a bue foce fashion by inceasing he numbe of vehicles of ype k a locaion i by 1 and simulaing he behavio of policy unde load ealizaion d n. The esuls in Table 2 indicae ha 29 yields accuae esuls. 5 Conclusions We pesened efficien mehods o assess he sensiiviy of a sochasic dynamic flee managemen model o flee size and load availabiliy. Numeical expeimens indicaed ha hese mehods ae accuae and compuaionally acable. Infomaion abou he cos impac of an addiional vehicle o an addiional load can, in un, be used when making flee sizing, load evaluaion and picing decisions. Using he mehod descibed in Secion 2.2 fo load picing is he opic of anohe pape see Topaloglu & Powell

20 Co. Avg. Hisogam Time Coeff. of vaiaion Poblem coeff. % dev. 2.5% 5% 10% 25% sec. Avg. 20 p. 80 p. 10, 1000, 20, 100, , 1000, 20, 100, , 1000, 20, 100, , 1000, 20, 200, , 1000, 20, 200, , 1000, 20, 200, , 1000, 40, 100, , 1000, 40, 100, , 1000, 40, 100, , 1000, 40, 200, , 1000, 40, 200, , 1000, 40, 200, , 3000, 20, 100, , 3000, 20, 100, , 3000, 20, 100, , 3000, 20, 200, , 3000, 20, 200, , 3000, 20, 200, , 3000, 40, 100, , 3000, 40, 100, , 3000, 40, 100, , 3000, 40, 200, , 3000, 40, 200, , 3000, 40, 200, Appendix Table 2: Accuacy of he policy gadiens wih espec o vehicle availabiliies. This secion pesens he omied poofs. The following esul is useful when poving Lemma 3. Lemma 7 If C e i, e j, d + c ijl 0, hen we have X, d + e ijl = x. Poof of Lemma 7 Conside poblem 21 and is newok epesenaion in Figue 2. In his poblem, d ijl acs as an uppe bound on he decision vaiable x ijl and a min-cos newok flow poblem wih uppe bounds can be conveed o an equivalen poblem wihou uppe bounds by he ansfomaion shown in Figue 4 see Vandebei Theefoe, if d ijl is inceased by 1, hen he change in he opimal soluion of poblem 21 is given by a min-cos flow augmening pah fom node j o node i, j, l in Figue 4.b. We denoe his min-cos flow augmening pah by Q. Since node i, j, l has exacly wo inbound acs, hee ae wo possible cases o conside fo Q. 1 Eihe Q includes only he bold ac ha connecs node j o node i, j, l. In his case, he cos of Q is 0. 2 O Q connecs node j o node i, and hen, node i o node i, j, l. We le C e i, e j, d be he cos of he min-cos flow augmening pah fom node j o node i in Figue 4.a. Then, he cos of he min-cos flow augmening pah fom node j o node i in Figue 4.b is also C e i, e j, d. Hence, fo he second case, he cos of Q is C e i, e j, d + c ijl. 20

21 i [ c ijl, dijl ] a. j b. i c i, j, l [, ] ijl d ijl [ 0, ] j j, +1 + d ijl j, +1 Figue 4: a. The cos and he uppe bound fo he bold ac ha connecs node i o node j ae c ijl and d ijl. b. The min-cos newok flow poblem in Figue 4.a can be conveed o one wihou he afoemenioned uppe bound by a simple ansfomaion. We inoduce an exa node i, j, l wih supply d ijl and se he supply of node j o +d ijl. Since Q is he min-cos flow augmening pah, if C e i, e j, d + c ijl > 0, hen he fis case mus hold fo Q. Thus, we have X, d + e ijl = X, d. We conclude by noing ha he possibiliy of having C e i, e j, d + c ijl = 0 is uled ou by he andom peubaion of he coss so ha poblem 21 does no have alenaive opima. Poof of Lemma 3 To show he fis pa, we conside wo cases depending on he sign of C e i, e j, d + c ijl. 1 If C e i, e j, d + c ijl 0, hen by Lemma 7, an addiional load of ype l on lane i, j does no change he soluion of poblem 21. Theefoe, X, d + e ijl = x and R+1, d + e ijl = +1 hold. 2 Following an agumen simila o he poof of Lemma 7, if we inoduce an addiional load of ype l on lane i, j, hen he change in he soluion of poblem 21 is given by he min-cos flow augmening pah fom node j o i, j, l in Figue 4.b. Le his flow augmening pah be Q. If C e i, e j, d + c ijl < 0, hen he second case in he poof of Lemma 7 holds. Theefoe, Q fis connecs node j o node i, and hen, node i o node i, j, l. This means ha Q is equal o he min-cos flow augmening pah Q e i, e j, d appended by he ac ha connecs 21

22 node i o node i, j, l. Then, he esul follows. The second pa holds because any acyclic pah fom node j I in he middle secion of Figue 4.a o Figue 4.b o node i I on he lef side aveses eihe zeo o wo of he acs coesponding o he vaiables { j,+1 : j I}. If a pah aveses wo of hese acs, hen one of hese acs is a fowad ac and he ohe is a backwad ac in he pah. Poof of Lemma 5 We now show he second pa. We fis noe ha any acyclic pah fom node i, k I K on he lef side of Figue 3 o he sink node can only visi he nodes {i, k : k K}. Assume ha he esul does no hold. This means ha δ+1 ek i, d canno be wien as a veco whose elemens ae +1, 1 o 0, and hence, hee exis j I, k K such ha δj k,+1 ek i, d 2. Assume ha δj k,+1 ek i, d 2. Since we have δ k j,+1 ek i, d = ı I l L ξ k ıj l ek i, d, hee exis i, i I and l, l L such ha ξi k j l ek i, d = +1 and ξk i j l ek i, d = +1. Because of ou iniial obsevaion, we mus have i = i = i. Bu, having ξij k l ek i, d = +1 and ξk ij l ek i, d = +1 implies ha, on he min-cos flow augmening pah fom node i, k o he sink node, hee ae wo fowad acs ha leave node i, k. This conadics he fac ha he min-cos flow augmening pah is acyclic. One can also each a conadicion by assuming ha δj k,+1 ek i, d 2. Poofs of Poposiions 4 and 6 use he following esul. Lemma 8 Fo a sepaable funcion G : R n R, we have n n { G x + α i e i Gx = G x + αi e i Gx } i=1 i=1 whee α i R fo all i = 1,..., n and e i is he n-dimensional uni veco wih a 1 in he i-h elemen. Poof of Lemma 8 Leing Gx = n i=1 g ix i whee g i : R R fo all i = 1,..., n, we have n n { G x + α i e i Gx = gi x i + α i g i x i } i=1 i=1 = n i 1 g j x j + g i x i + α i + i=1 j=1 n g j x j j=i+1 n g j x j. j=1 7 Acknowledgemens The auhos gaefully acknowledge he commens of wo anonymous efeees ha ighened he pesenaion significanly. The wok of he fis auho was suppoed in pa by Naional Science 22

23 Foundaion gan DMI The wok of he second auho was suppoed in pa by Ai Foce Office of Scienific Reseach gan AFOSR-FA and Naional Science Foundaion gan CMS Refeences Cavalho, T. A. & Powell, W. B. 2000, A muliplie adjusmen mehod fo dynamic esouce allocaion poblems, Tanspoaion Science 34, Cainic, T., Gendeau, M. & Dejax, P. 1993, Dynamic and sochasic models fo he allocaion of empy conaines, Opeaions Reseach 41, Danzig, G. & Fulkeson, D. 1954, Minimizing he numbe of ankes o mee a fixed schedule, Naval Reseach Logisics Quaely 1, Dejax, P. & Cainic, T. 1987, A eview of empy flows and flee managemen models in feigh anspoaion, Tanspoaion Science 21, Fanzeskakis, L. & Powell, W. B. 1990, A successive linea appoximaion pocedue fo sochasic dynamic vehicle allocaion poblems, Tanspoaion Science 241, Glasseman, P. 1991, Gadien Esimaion via Peubaion Analysis, Kluwe Academic Publishes, Nowell, Massachuses. Godfey, G. A. & Powell, W. B. 2002a, An adapive, dynamic pogamming algoihm fo sochasic esouce allocaion poblems I: Single peiod avel imes, Tanspoaion Science 361, Godfey, G. A. & Powell, W. B. 2002b, An adapive, dynamic pogamming algoihm fo sochasic esouce allocaion poblems II: Muli-peiod avel imes, Tanspoaion Science 361, Hane, C., Banha, C., Johnson, E., Masen, R., Nemhause, G. & Sigismondi, G. 1995, The flee assignmen poblem: Solving a lage-scale inege pogam, Mah. Pog. 70, Ho, Y.-C. & Cao, X.-R. 1991, Peubaion Analysis of Discee Even Dynamic Sysems, Kluwe Academic Pess. Holmbeg, K., Jobon, M. & Lundgen, J. T. 1998, Impoved empy feigh ca disibuion, Tanspoaion Science 32, Law, A. L. & Kelon, W. D. 2000, Simulaion Modeling and Analysis, McGaw-Hill, Boson, MA. Nemhause, G. & Wolsey, L. 1988, Inege and Combinaoial Opimizaion, John Wiley & Sons, Inc., Chichese. Powell, W. B. 1989, A eview of sensiiviy esuls fo linea newoks and a new appoximaion o educe he effecs of degeneacy, Tanspoaion Science 234, Powell, W. B., Jaille, P. & Odoni, A. 1995, Sochasic and dynamic newoks and ouing, in C. Monma, T. Magnani & M. Ball, eds, Handbook in Opeaions Reseach and Managemen Science, Volume on Newoks, Noh Holland, Amsedam, pp Pueman, M. L. 1994, Makov Decision Pocesses, John Wiley and Sons, Inc., New Yok. Topaloglu, H. & Powell, W. B. 2005, Incopoaing picing decisions ino he sochasic dynamic flee managemen poblem, Technical epo, Conell Univesiy, School of Opeaions Reseach and Indusial Engineeing. Topaloglu, H. & Powell, W. B. 2006, Dynamic pogamming appoximaions fo sochasic, imesaged inege mulicommodiy flow poblems, INFORMS Jounal on Compuing 181, o appea. Vandebei, R. 1997, Linea Pogamming: Foundaions and Exensions, Kluwe s Inenaional Seies. 23