REVISTA INVESTIGACION OPERACIONAL VOL. 31, No.2, , 2010


 Eugene Freeman
 1 years ago
 Views:
Transcription
1 REVISTA INVESTIGACION OPERACIONAL VOL. 3, No., 5970, 00 AN ALGORITHM TO OBTAIN AN OPTIMAL STRATEGY FOR THE MARKOV DECISION PROCESSES, WITH PROBABILITY DISTRIBUTION FOR THE PLANNING HORIZON. Gouliois E. Joh Deparme of saisics ad isurace sciece Uiversiy of Pireas, 80 Karaoli a Dimiriou Sree, 8534 Piraeus, Greece ABSTRACT I his paper we formulae Markov Decisio Processes wih Radom Horizo. We show he opimaliy equaio for his problem, however here may o exis opimal saioary sraegies. For he MDP (Markov Decisio Process, wih probabiliy disribuio for he plaig horizo wih ifiie suppor, we show Turpike Plaig Horizo Theorem. We develop a algorihm obaiig a opimal firs sage decisio. We give some umerical examples. MSC: 90C40, 90B50, 90C39, 90C5 KEY WORDS: MDPs, opimizaio, probabiliies ad decisio makig, operaio research RESUMEN E ese rabajo formulamos u Proceso de Decisió Markoviao co Horizoe Aleaorio. Desarrollamos la ecuació de opimalidad para ese problema, si embargo puede o exisir esraegias opimales esacioarias. Para el MDP (Proceso de Decisió Markoviao, co disribució de probabilidad para horizoe de plaeamieo co sopore ifiio, demosramos el Teorema de Horizoe de Plaeamieo de Turpike. Desarrollamos u algorimo para obeer ua decisió de primera eapa opimal. Damos alguos ejemplos uméricos.. INTRODUCTION A muliperiod opimizaio problem is ofe modeled as a ifiie horizo problem whe is horizo is log sufficiely. We do o ecessarily kow he horizo of he problem i advace sice we ca o predic he fuure precisely. For example, we imagie vaguely ha a drasic chage of a projec may occur some day, ad we oly believe whe is chage will occur uder a cerai probabiliy disribuio. Thus i is o appropriae ha we simply model he problem as a ifiie or a fixed fiie horizo case. If he plaig horizo chages may cause a remarkable chage of opimal sraegy, ad he oal reward may differ much. Hece, i is ecessary o make a decisio cosiderig he probabiliy of he ime a which he projec will ed. We formulae hese problems usig MDPs i which probabiliy disribuios for he plaig horizo are give i advace, ha is, MDP wih Radom Horizo. I his paper we will cosider o homogeeous MDPs. I is kow oe ypical example wih a geomerically disribued plaig horizo, which is equivale o a ordiary discoued MDP (Ross [9]. We ca oice ha he discou rae represes a evaluaio of uceraiy expeced o be happeed i he fuure. Numerous researches have bee made for he ype of MDP which has variable discou raes (Whie [], Puerma [8], Sodik []. I he MDP wih radom horizo here may o exis a opimal saioary sraegy. Whe he suppor of he probabiliy disribuio for he plaig horizo is fiie, we ca easily ge a opimal sraegy by solvig he correspodig opimaliy equaio. Whe he suppor of he probabiliy disribuio for he plaig horizo is ifiie, i is difficul o solve he problem. So we adop a rollig horizo sraegy o obai a opimal sraegy, ha is, firs we obai he Turpike Plaig horizo for MDP ad solve he problem uder is horizo. Shapiro [] shows he exisece of he Turpike Plaig Horizo for he homogeeous discoued MDP. 59
2 This paper is relaed o he researches of Bea ad Smih []. They reas deermiisic decisio problems. I addiio Hopp, Bea ad Smih [7] cosiders he codiio for he exisece of a opimal sraegy for he o homogeeous o discoued MDP uder a weak ergodiciy assumpio. I Secio, he model is described i deail ad some assumpios are provided. We derive he opimaliy equaio for he MDP wih radom horizo. I his secio he opimal problem is formulaed as MDP. I Secio 3 we describe he srucure ad aure of a opimal sraegy i he case ha he suppor of he probabiliy disribuio for he plaig horizo is ifiie. I secio 4 we give a algorihm for solvig he problem based o he aure derived i secio 3. I secio 5 some coclusios are provided.. MODEL DESCRIPTION AND ASSUMPTIONS. Le (Ω,F,P deoe he uderlyig probabiliy space. Le T { 0,,, } K be he se of oegaive iegers. We cosider a discree ime o homogeeous Markov Decisio Model wih (i Couable sae space S, (ii measurable acio space A edowed wih σfield, A coaiig all oepoi subses of A. (iii ses of acio A( s available a s S, where A( s is a eleme of A, (iv rasiio probabiliies { p ( j, i α } a sae, T,where p ( j, i α is oegaive ad measurable i a, ad for each i S, a A( s, p ( j, i a, T, j S (v ses of reward fucios { (, } r i a a sage, T r i a is measurable i a, c i a whe he projec ed a sage, T, where c i a is measurable i a. The salvage cos is he icurred cos o sop he projec ad may deped o he sae ad acio a ha sage. (vi ses of salvage cos fucios { (, } he fucio (,, where he fucio (, Assumpio..For each sage, reward fucios ad salvage cos fucios are assumed o be bouded, ha is, r s, a R c s, a C <+ ( <+, ( Le a fucio a : S A, T ( a, T (,,, be a decisio fucio wih a ( s A( s. The sequece is called a sraegy. Le Δ deoe he se of all sraegies. We also use he oaio 0 K o represe firs decisios i. I his model, we also se, f wih which he projec ed a sage, T (vii a probabiliy disribuio Also we cosider a absorbig sae The we add ex hree o he above (iii, (iv ad (v, (iii A( s { a' }, (iv for ay j S, all T, (, 0 (v for all T r s, a o, (. s ' represeig he ed sae of projec ad le S' S { s' }. p j s a, ( p s' s, a, 60
3 Le H ( S A S be he space of hisories up o he sage T { }, where A A { a }. If a sraegy Δ ad iiial sae s are specified, rasiio probabiliies are deermied compleely. Accordigly a probabiliy measure P s is iduced. Le x deoe he sae of process a sae, M deoe he radom plaig horizo wih disribuio ϕ, A deoe he acio ake a sae, he he expeced oal reward for he horizo problem is give by ( ( r X, A whe X S, < M R( X, A c X, A whe X S, M 0 whe X S. (. Cosiderig he horizo problem, for a fixed, whe he process sars wih a iiial sae s uder a sraegy, he expeced oal reward for his problem is give by (,, (, V s E R X A, (. where s s 0 E is he correspodig expecaio operaor. We oice ha he expeced reward V ( s,, depeds oly o he firs decisios i each. Now, we ca describe he horizo ad Mhorizo opimal decisio problem. The horizo problem is defied as sup V( s,, E R ( X, A for each s. (.3 0 A sraegy ( Δ is called a opimal sraegy for he horizo problem if for each s S, V s,, sup V s,,. ( ( ( For he radom Mhorizo problem, we se (, (, V s E R X A. (.4 s 0 I should be also oed ha a opimal sraegy for horizo problem depeds oly o he firs decisios i each. Similarly a sraegy Δ is called a opimal sraegy for he radom Mhorizo problem if for each s S V s, sup V s,., ( ( Le ε be a arbirary oegaive cosa. The a sraegy ( for he horizo problem if for each s S, V s, (, V s,, ε ( ε ( ( is called ε  opimal sraegy. (.5 Now we cosider he opimaliy equaio for he MDP wih radom horizo. Le b be a probabiliy which he projec is sill coiuig a sage ( + uder codiio ha i has coiued uil sage, ha is, 6
4 κ κ b φ. (.6 φ κ κ Whe he process is i sae i ad acio a is used a sage, he expeced reward we ge is d i, a b r i, a + b c i, a. (.7 ( ( ( ( Now, le v ( i deoe he maximal value which we ca ge afer he sage whe he process is i sae i, a sage. Therefore we ca ge he opimaliy equaio as follows, v ( max (, (, i d i a + b p j ia v ( j a A +. (.8 j S Whe he suppor of he probabiliy disribuio for he plaig horizo is fiie, we ca easily obai he soluio of he problem as i a ordiary fiie horizo, by applyig he backward iducio mehod o he opimaliy equaio (.8 wih seig v ( i max c ( i, a, for all i S, (.9 a A( i where is a maximal value of he suppor of { φ, T}. 3. OPTIMAL STRATEGIES WHEN THE SUPPORT OF THE PROBABILITY DISTRIBUTION FOR THE PLANNING HORIZON IS INFINITE. I his secio we discuss he MDPs wih radom horizo which have he ifiie suppor of he probabiliy disribuio for he plaig horizo. We discuss he problem based o he idea ha if he opimal sraegies for he fiie horizo problem approach a paricular sraegy for he ifiie suppor problem, we will cosider ha sraegy as he opimal oe. Works of Hopp, Bea ad Smih [7], Bes ad Sehi [3] are based o his idea, oo. We ow defie a meric opology o he se of all sraegies Δ. The meric ρ below is he same oe which Bea ad Smih [] uses ρ (, σ (,, where σ (, ( ( ( ( ' a x a x x S ' 0 a x a x x S The ρ meric has he propery ha ay wo sraegies ha agree i he firs M policies, for ay M, are cosidered closer ha ay wo sraegies ha do o. Now we defie Defiiio 3..A sraegy % Δ is periodic forecas horizo (PFH opimal if for some subsequece i he ρ meric as m. of he iegers { M m}, ( % m M m Proposiio 3..(, ρ Δ is a meric space. If ρ( ', < ε < for all log ε α α. See Bea ad Smih [] 6
5 Assumpio 3..We assume ha he sraegy Space, Δ, is compac i meric space geeraed by ρ. This assumpio precludes he possibiliy of a sequece of feasible sraegies covergig o a ifeasible sraegy. For furher discussio of a relaed problem, see Bea ad Smih []. ' ' Le Δ { a} for all T. We defie he discree opology p (, a a σ ( a, a o hem. The he heorem below holds. Theorem 3.. Δ is compac if ad oly if T, Δ are fiie ses. If Δ is compac, he each cylider subse of Δ is compac. See [3] Theorem 3.3.A periodic forecas horizo opimal sraegy exiss for he ohomogeeous Markov decisio process. { m } Compacess of Δ implies ha he sequece ( M m has a coverge subsequece. The limi of such a sequece is PFH opimal by defiiio (3.. Whe S ad A are fiie ses, a compacess of Δ is esured. From he defiiio of V( s, a, we have he followig proposiio. Proposiio 3.3.Whe he expecaio of he plaig horizo is fiie, he oal expeced reward is fiie. Sice he expecaio of he plaig horizo, E( M, is fiie, ϕ <+. (3. The, Δ, N V ( s,, M Es R( X, A 0 E s r ( X, A M P[ M ] E s c ( X, A M P[ M ] > > + κ κ. κ < max { RC, } ϕ max { RC, } ( ϕ Thus from (3., ( V s,, M <+. Assumpio 3.4. The expecaio of he plaig horizo is fiie. Now we ca discuss he exisece of opimal sraegy for he MDP wih radom horizo ad ifiie suppor. Lemma 3.5. V ( s, is coiuous i Δ. 63
6 For ay 0 ε >, here exiss Λ, such ha max { RC, } ε <. b κ Λ+ κ Therefore we ge a v such ha Λ log v. The for ay ' M M ' V ( s, V ( s, ' Es R ( X, A Es R ( X, A ρ, ' < v, Δ such ha ( E s r ( X, A M P[ M ] E s c ( X, A M P[ M ] > > + Μ+ Μ+ ' ' s, s, M+ M+ max RC, b κ < ε. M+ κ ( [ ] ( [ ] E r X A M > P M > E c X A M P M { } Le ow ( { } Noe ha sice ( Δ be a opimal sraegy for horizo problem ad Δ be a se of cluser pois of all he sequeces ( Δ α Δ, T V s,, is coiuous i ad Δ is compac, Δ is a oempy se., ha is, a se of PFH opimal sraegies ad le { }. Theorem 3.6 (exisece. Uder assumpios., 3. ad 3.4 here exiss a PFH opimal sraegy for he MDP wih radom horizo. Sice Δ is compac, V ( s, is uiformly coiuous o Δ. Thus here exiss a sraegy such ha V( s, max V ( s, wih radom horizo Δ. Δ. Therefore here exiss a PFH opimal sraegy for he MDP Lemma 3.7. lim max (,, max (, Le max {, } V s V s K RC b κ, he we have + κ ( ( ( + K as, lim max V( s,, max V ( s, max V s, K max V s,, max V s, K Thus sice 0 Le Δ deoes a se of all opimal sraegies Δ for he MDP wih radom horizo. Lemma 3.8. Le Δ Δ { ( } Δ. From he defiiio here exiss a sequece of sraegies m( j lim ( m( j. j Thus ( ( ( ( ( lim V s, m j, m j V s,. j j T such ha 64
7 , Sice from lemma (3.7 V ( s, max V( s, Δ. There may o ecessarily exiss a saioary deermiisic sraegy or saioary radomized sraegy for he MDP wih radom horizo. There may o exis eve a ε  opimal radomized aioary sraegy. We show a example. Example. Cosider he homogeeous model wih S {, } ad A { ab, } p( s, a p( s, b, for s,. r(, a, r( b r( a, (,, 0 c( s, x 0, for, s, x ab,. r, b.. Le We deoe he probabiliy disribuio for he plaig horizo { ϕ } as follows, β ( whe 0 ( ( whe ( β ( β β ( whe φ β β ha is, he geomeric disribuio of which parameer chages o β from β a sage. I his model, i is clear ha acio b is opimal a sae. Thus here are wo cadidaes for opimal deermiisic saioary sraegy as follows, ' : keep your sae (use acio a sae ad acio b a sae, '' :move o sae ad keep i (use oly acio b. We shall examie a opimal radomized saioary sraegy for his model. A radomized saioary sraegy is defied as a a a τ (, ( Whe, his model is equivale o he MDP wih discou rae β. Therefore he expeced v s, s,, is he uique soluio of he sysem of he followig liear equaios, reward (, { } { } (, ( β α( (, ( α (, (, ( β τ. (, + ( τ ( + (, v + v + v v v v Solvig he equaios, we have,. (a (, β (, β ( α( β + τ ( β a a( τ( β + ( a( τ( β aτ ( β + ( τ a( τ( β v v Similarly, 65
8 { } { } (, ( β α( (, ( α (, (, ( β τ (, + ( τ ( + (, v v + v v v v so ha v, π β α + v, + α v, { } ( ( ( ( ( ( 0 a( β a ( β ( α( τ( β ( ( + + ( a a v ( ( a( ( a ( v ( + β + τ, + β + τ, Sice from (a we have ϑv (, ϑv (, 0, 0, v(, v(,, ϑτ ϑτ ad we obai ϑv 0 (, 0. ϑτ Therefore i is see formally ha acio b is opimal a sae. 3 Now fix β, β so ha he expeced rewards of deermiisic saioary sraegies u ', 5 5 are v0 (, ' K, v0 (, '' K, ad he expeced reward of radomized saioary sraegy π is 0 ( 5a a v0 (,, 55 3a ( 5 0 which is maximized a a. The expeced reward associaed wih his 3 v 0 (, ( α Give iiial sae, he radomized saioary sraegy saioary sraegies. associaed wih. a is u '' a is he bes amog all Defie he sraegy as follows, ' if 0 '' if The expeced reward of his sraegy is v0 (, From he fac meioed above, i is see ha for ε < here does o exis a ε  opimal radomized saioary sraegy. We coiue ow wih showig ha a heorem similar o Turpike Plaig Horizo Theorem which Shapiro []shows for he homogeeous discoued MDP holds for his MDP wih radom horizo. Because, here may o ecessarily exiss a saioary deermiisic sraegy or saioary radomized sraegy for MDP wih radom horizo, a opimal sraegy we wish o kow may be o saioary, so i s difficul o ge i direcly. 66
9 We iroduce he followig wo oaios, F { : Δ } : a se of opimal decisios a he firs sae for he radom M horizo problem, ad F :a se of opimal decisios a he firs sae for he  horizo problem. { : } ( ( Theorem 3.9 (Turpike Plaig Horizo Theorem. There exiss some L such ha for ay L F F., ( Assume as he corary ha here does o exis such a umber L. The here exiss a ieger M such ha he firs decisio of some opimal sraegy for he M horizo problem is o coaied i F,ad here exiss a ieger M ( > similarly, so we obai a sequece of sraegies M { ( M i } such ha ( M i F { ( m( M i } such ha is limi is Δ.Thus for sufficie large ( i (, p ( m( M < ε, so ( ( i m Mi. Therefore for all i. Sice Δ is compac, here exiss a subsequece 67 m M, F. O he oher had, from defiiio Δ %, ad from he lemma 3.8 Δ. Thus F, which is a coradicio. From he above heorem we ca make a firs opimal decisio by solvig he sufficie large  horizo problem. I should be oed ha here exiss a opimal rollig sraegy. 4. ALGORITHM FOR FINDING AN OPTIMAL FIRST DECISION Alhough he Turpike Plaig Theorem i he above secio saes he exisece of he urpike horizo, he heorem shows o way for fidig i. Hece i his secio we ivesigae a algorihm for fidig a opimal firs decisio or ε opimal firs decisio. If we ca fid a opimal firs decisio, ex we pay aeio o he secod sage, ha is, we cosider he secod sage as he firs sage, ad he apply he same algorihm o i. By meas of coiuig his procedure a hird, fourh, sage, we ca fid a sequece of opimal decisios oe by oe, ha is, a opimal rollig sraegy. Above procedures correspods o ideifyig he PFH opimal sraegy gradually, ha is, makig he eighborhood of PFHopimal sraegy small. Le Δ ˆ deoes a se of sraegies such ha is firs decisio is o icluded i F(, ha is ˆ Δ { Δ : F( }. ˆ Theorem 4.. For ay Δ, if saisfies he followig codiio (, ' ( max (,, (,, > max {, }, (4. V s V s R C b κ is o opimal for he problem wih ifiie suppor. Le ' Δ be a sraegy saisfyig a codiio (. Se max {, } + κ K RC b κ, he from he codiio (, ( ( + κ ' max V s,, V s,, > K (4. ad
10 ( ( max V s,, K max V s,. (4.3 Therefore from (4. ad (4.3 ( ( max V s, V s, ', > K. Thus ' Δ is o opimal. Remark 4.. If F( is sigleo ad a codiio of heorem 4. holds for ay ( is a opimal firs decisio. ˆ Δ, F From he above heorem we ca fid a firs decisio which is o opimal ad he remove i. I cosequece we propose a algorihm which decreases he umber of decisios possible o be opimal by ieraig he above check. The followig algorihm fids eiher a opimal firs decisio or a ε opimal decisio. Algorihm 4.3 Sep. Se. u max R, C b κ. Sep.Le { } κ Sep 3. a A + a, compue ξ max V( s,, V ( s,, a, where, α α. If ξ > ad a F is sigleo, Sop. Is decisio is a opimal firs oe. Sep 4. If ε, Sop. Is decisio is a εopimal firs oe. Sep 5. +, ad go o Sep. Remark 4.4. From he heorem 3.6 he above algorihm sops i a fiie umber of seps. Remark 4.5. If Δ is sigleo, he above algorihm ca fid a opimal firs decisio i a fiie umber of seps. I is discussed by Bes ad Sehi [3] ha Δ is o rarely sigleo. As a umerical example, we cosider a followig iveory problem. A iem has a lifeime disribuio a accou of is lifecycle or appearace of a ew iem. We cosider ha his disribuio correspods o he radom horizo previously saed. We deoe is disribuio by { ϕ } projec ed, all remaiig iems may be se back a a salvage cos per ui. Whe he Here we assume ha oeperiod demad, η, follows i.i.d. Poisso disribuio. Le a deoes he amou of order. So he amou of sock saisfies a followig relaio, s s + a η, (4.4 where he iiial sock, s 0, is eve. We assume ha S s S, ha is, a upper boud ad a lower boud of he sock is give. The cos we cosider are followig, k( a : he order cos i he period whe a iems are ordered, ( c s : he holdig cos i he period whe s 0, he backloggig cos i he period whe s < 0, ( r x : he icome i he period whe Accordigly he problem is o maximize he oal expeced reward: 68 x iems are sold, where x max{ mi { η, s },0}.
11 a Maximize Es { ( ( ( } 0 r X κ Α c S. (4.5 Now we assume ha he daa are as follows, s 0 5, S 5, S 0. The expeced value of he demads i oeperiod is 7. 0x ( x α ( α 0 s ( s 0 r ( x k ( a c ( s. (4.6 0 ( x < 0 0 ( α < 0 4s ( s< 0 The le he salvage cos per ui be 7. Table. Opimal Firs Decisios ad Turpike Plaig Horizos CV Mea (.5,.75 (0.886,3. (0.589, (.5,3.49 (.8,4.9 (.34,4.66 (0.99,5.07 (0.55, (3.88,6. (3.00,7.00 (.3,7.69 (.68,8.3 (.09,8.9 (0.58, (6.3,3.9 (4.90,5. (3.76,6. (.65,7.3 (.57,8.4 (0.53, (8.58,.4 (6.88,3. (5.4,4.8 (3.64,6.4 (.07,7.9 (0.509, (.,8.9 (8.86,3. (6.73,33.3 (4.64,35.4 (.56,37.4 (0.506, (6.0,44.0 (.9,47. (9.73,50.3 (6.63,53.4 (3.56,56.4 (0.504, (6.0,74.0 (0.8,79. (5.7,84.3 (0.6,89.4 (5.56,94.4 (0.503, 99.5 (upper opimal firs decisio (middle Turpike Plaig Horizo (lower (λ, λ We examie how he probabiliy disribuio for he plaig horizo cause he chage of he firs opimal decisios. We use he followig composie disribuio of Poisso disribuios, P[ N ] 0.5Pλ [ N ] + 0.5P [ N ] λ, (4.7 which eables us o arrage various combiaios of values of he mea ad coefficie of variaio of he disribuio by chagig λ ad λ. We calculae he opimal decisios for amou of orders a he firs sage ad he Turpike plaig horizos for he cases i which meas are,3,5,0,5,0,30,50, ad coefficies of variaio are 0.5,0.6,0.7,0.8,0.9,.0. The resuls of calculaios are show i Table. From Table.we ca see wo edecies i his iveory problem, oe is ha qualiy of order a he firs sage icreases as he mea horizo icreases, ad he oher is ha i decreases as he coefficie of variaio icreases. The umerical resul shows he ieresig behaviour ha whe he coefficie of variaio is.0, he firs opimal decisios are always 5. I his umerical example, whe he coefficie of variaio is.0, λ becomes very small for each emas, which suggess he probabiliy ha he projec will ed soo is fairly large. Thus he firs decisio for amou of order is expeced o become small. From heses resuls he opimal firs decisios are cosidered o deped o he shape of he probabiliy disribuio for he plaig horizo much. 5. CONCLUSIONS The purpose of his paper is o aalyze a opimal sraegy for he MDP wih radom horizo, ad purpose he algorihm o obai i umerically by Turpike Plaig Horizo approach. For he processes here may o exis opimal saioary sraegies, so we evaluae rollig sraegies, derived by 69
12 usig he resul of Turpike Horizo Theorem. We develop a algorihm obaiig a opimal firs sage decisio, ad some umerical experimes. As a resul of umerical experimes, we ake ha he opimal firs decisios deped o he shape of he probabiliy disribuio for he plaig horizo. REFERENCES RECEIVED OCTOBER 008 REVISED NOVEMBER 009 [] ALDEN,J.M. ad SMITH R.(99: Rollig horizo procedures i ohomogeeous Markov decisio processes. Operaios Research, 40, [] BEAN, J.,ad SMITH R.(984: Codiios for he exisece of plaig horizos. Mah. of Operaios Research, 9, [3] BES C. ad SETH, S. (984: Coceps of forecas ad decisio horizos : applicaios o dyamic sochasic opimizaio problems. Mah., of Operaios Research, 3, [4] GOULIONIS E.J., (004: Periodic policies for parially observable Markov decisio processes, Workig paper No. 30, 58, Uiversiy of Piraeus 004. [5] GOULIONIS, E.J. (006: Α replaceme policy uder Markovia deerioraio. Mahemaical ispecio, 63, [6] GOULIONIS E.J. (005: P.O.M.D.Ps wih uiformly disribued sigal processes. Spoydai, 55, [7] HOPP,W.J,, BEAN J.C ad SMITH, R (987: A ew opimaliy crierio for ohomogeeous Markov decisio processes. Operaios Research, 35, [8] PUTERMAN, M.L (994: Discree Sochasic Dyamic Programmig. Joh Wiley ad Sos, New York. [9] ROSS,S.M. (984: Iroducio o Sochasic Dyamic Programmig. Academic Press, New York. [0] SETHI S. ad BHASKARAN,S. (985: Codiios for he Exisece of Decisio Horizos for Discoued Problems i a Sochasic Evirome. Operaios Research Leers, 4, [] SHAPIRO,J.F. (968: Turpike plaig horizos for a Markovia decisio model. Maageme Sci., 4, [] SONDIK.J.E. (978: Οpimal corol of parially observable Markov decisio processes over ifiie horizo. Qperaios Research, 6, [3] WHITE,D.J. (987: Ifiie horizo Markov decisio processes wih ukow variables discou facors. Euro. J. of Qperaios Research,.8,
ON THE RISKNEUTRAL VALUATION OF LIFE INSURANCE CONTRACTS WITH NUMERICAL METHODS IN VIEW ABSTRACT KEYWORDS 1. INTRODUCTION
ON THE RISKNEUTRAL VALUATION OF LIFE INSURANCE CONTRACTS WITH NUMERICAL METHODS IN VIEW BY DANIEL BAUER, DANIELA BERGMANN AND RÜDIGER KIESEL ABSTRACT I rece years, markecosise valuaio approaches have
More informationUsing Kalman Filter to Extract and Test for Common Stochastic Trends 1
Usig Kalma Filer o Exrac ad Tes for Commo Sochasic Treds Yoosoo Chag 2, Bibo Jiag 3 ad Joo Y. Park 4 Absrac This paper cosiders a sae space model wih iegraed lae variables. The model provides a effecive
More informationThe effect of the increase in the monetary base on Japan s economy at zero interest rates: an empirical analysis 1
The effec of he icrease i he moeary base o Japa s ecoomy a zero ieres raes: a empirical aalysis 1 Takeshi Kimura, Hiroshi Kobayashi, Ju Muraaga ad Hiroshi Ugai, 2 Bak of Japa Absrac I his paper, we quaify
More informationWhen Two Anomalies meet: PostEarningsAnnouncement. Drift and ValueGlamour Anomaly
Whe Two Aomalies mee: PosEarigsAouceme Drif ad ValueGlamour Aomaly By Zhipeg Ya* & Ya Zhao** This Draf: Sepember 009 Absrac I his paper, we ivesigae wo promie marke aomalies documeed i he fiace ad accouig
More informationCostSensitive Learning by CostProportionate Example Weighting
CosSensiive Learning by CosProporionae Example Weighing Bianca Zadrozny, John Langford, Naoki Abe Mahemaical Sciences Deparmen IBM T. J. Wason Research Cener Yorkown Heighs, NY 0598 Absrac We propose
More informationANALYSIS AND COMPARISONS OF SOME SOLUTION CONCEPTS FOR STOCHASTIC PROGRAMMING PROBLEMS
ANALYSIS AND COMPARISONS OF SOME SOLUTION CONCEPTS FOR STOCHASTIC PROGRAMMING PROBLEMS R. Caballero, E. Cerdá, M. M. Muñoz and L. Rey () Deparmen of Applied Economics (Mahemaics), Universiy of Málaga,
More informationToday s managers are very interested in predicting the future purchasing patterns of their customers, which
Vol. 24, No. 2, Spring 25, pp. 275 284 issn 7322399 eissn 1526548X 5 242 275 informs doi 1.1287/mksc.14.98 25 INFORMS Couning Your Cusomers he Easy Way: An Alernaive o he Pareo/NBD Model Peer S. Fader
More informationFollow the Leader If You Can, Hedge If You Must
Journal of Machine Learning Research 15 (2014) 12811316 Submied 1/13; Revised 1/14; Published 4/14 Follow he Leader If You Can, Hedge If You Mus Seven de Rooij seven.de.rooij@gmail.com VU Universiy and
More informationA Simple Introduction to Dynamic Programming in Macroeconomic Models
Economics Deparmen Economics orking Papers The Universiy of Auckland Year A Simple Inroducion o Dynamic Programming in Macroeconomic Models Ian King Universiy of Auckland, ip.king@auckland.ac.nz This paper
More informationHOW MANY TIMES SHOULD YOU SHUFFLE A DECK OF CARDS? 1
1 HOW MANY TIMES SHOULD YOU SHUFFLE A DECK OF CARDS? 1 Brad Ma Departmet of Mathematics Harvard Uiversity ABSTRACT I this paper a mathematical model of card shufflig is costructed, ad used to determie
More informationDynamic Contracting: An Irrelevance Result
Dynamic Conracing: An Irrelevance Resul Péer Eső and Balázs Szenes Sepember 5, 2013 Absrac his paper considers a general, dynamic conracing problem wih adverse selecion and moral hazard, in which he agen
More informationThe U.S. Treasury Yield Curve: 1961 to the Present
Finance and Economics Discussion Series Divisions of Research & Saisics and Moneary Affairs Federal Reserve Board, Washingon, D.C. The U.S. Treasury Yield Curve: 1961 o he Presen Refe S. Gurkaynak, Brian
More informationMAXIMUM LIKELIHOODESTIMATION OF DISCRETELY SAMPLED DIFFUSIONS: A CLOSEDFORM APPROXIMATION APPROACH. By Yacine AïtSahalia 1
Ecoometrica, Vol. 7, No. 1 (Jauary, 22), 223 262 MAXIMUM LIKELIHOODESTIMATION OF DISCRETEL SAMPLED DIFFUSIONS: A CLOSEDFORM APPROXIMATION APPROACH By acie AïtSahalia 1 Whe a cotiuoustime diffusio is
More informationImproved Techniques for Grid Mapping with RaoBlackwellized Particle Filters
1 Improved Techniques for Grid Mapping wih RaoBlackwellized Paricle Filers Giorgio Grisei Cyrill Sachniss Wolfram Burgard Universiy of Freiburg, Dep. of Compuer Science, GeorgesKöhlerAllee 79, D79110
More informationConsistency of Random Forests and Other Averaging Classifiers
Joural of Machie Learig Research 9 (2008) 20152033 Submitted 1/08; Revised 5/08; Published 9/08 Cosistecy of Radom Forests ad Other Averagig Classifiers Gérard Biau LSTA & LPMA Uiversité Pierre et Marie
More informationAnchoring Bias in Consensus Forecasts and its Effect on Market Prices
Finance and Economics Discussion Series Divisions of Research & Saisics and Moneary Affairs Federal Reserve Board, Washingon, D.C. Anchoring Bias in Consensus Forecass and is Effec on Marke Prices Sean
More informationAre Under and Overreaction the Same Matter? A Price Inertia based Account
Are Under and Overreacion he Same Maer? A Price Ineria based Accoun Shengle Lin and Sephen Rasseni Economic Science Insiue, Chapman Universiy, Orange, CA 92866, USA Laes Version: Nov, 2008 Absrac. Theories
More informationDoes Britain or the United States Have the Right Gasoline Tax?
Does Briain or he Unied Saes Have he Righ Gasoline Tax? Ian W.H. Parry and Kenneh A. Small March 2002 (rev. Sep. 2004) Discussion Paper 02 12 rev. Resources for he uure 1616 P Sree, NW Washingon, D.C.
More informationAsymptotic Growth of Functions
CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll
More informationFIRST PASSAGE TIMES OF A JUMP DIFFUSION PROCESS
Adv. Appl. Prob. 35, 54 531 23 Prined in Norhern Ireland Applied Probabiliy Trus 23 FIRST PASSAGE TIMES OF A JUMP DIFFUSION PROCESS S. G. KOU, Columbia Universiy HUI WANG, Brown Universiy Absrac This paper
More informationSOME GEOMETRY IN HIGHDIMENSIONAL SPACES
SOME GEOMETRY IN HIGHDIMENSIONAL SPACES MATH 57A. Itroductio Our geometric ituitio is derived from threedimesioal space. Three coordiates suffice. May objects of iterest i aalysis, however, require far
More informationRamseytype theorems with forbidden subgraphs
Ramseytype theorems with forbidde subgraphs Noga Alo Jáos Pach József Solymosi Abstract A graph is called Hfree if it cotais o iduced copy of H. We discuss the followig questio raised by Erdős ad Hajal.
More informationOUTOFBAG ESTIMATION. Leo Breiman* Statistics Department University of California Berkeley, CA. 94708 leo@stat.berkeley.edu
1 OUTOFBAG ESTIMATION Leo Breiman* Saisics Deparmen Universiy of California Berkeley, CA. 94708 leo@sa.berkeley.edu Absrac In bagging, predicors are consruced using boosrap samples from he raining se
More informationParttime Work, Wages and Productivity: Evidence from Matched Panel Data
Parime Work, Wages and Produciviy: Evidence from Mached Panel Daa Alessandra Caaldi (Universià di Roma "La Sapienza" and SBSEM) Sephan Kampelmann (Universié de Lille, CLERSE, SBSEM) François Rycx (Universié
More informationWhen Should Public Debt Be Reduced?
I M F S T A F F D I S C U S S I ON N O T E When Should Public Deb Be Reduced? Jonahan D. Osry, Aish R. Ghosh, and Raphael Espinoza June 2015 SDN/15/10 When Should Public Deb Be Reduced? Prepared by Jonahan
More informationAsymmetry of the exchange rate passthrough: An exercise on the Polish data 1
Asymmery of he exchange rae passhrough: An exercise on he Polish daa Jan Przysupa Ewa Wróbel 3 Absrac We propose a complex invesigaion of he exchange rae passhrough in a small open economy in ransiion.
More informationI M F S T A F F D I S C U S S I O N N O T E
I M F S T A F F D I S C U S S I O N N O T E February 29, 2012 SDN/12/01 Two Targes, Two Insrumens: Moneary and Exchange Rae Policies in Emerging Marke Economies Jonahan D. Osry, Aish R. Ghosh, and Marcos
More informationCentral Bank Communication: Different Strategies, Same Effectiveness?
Cenral Bank Communicaion: Differen Sraegies, Same Effeciveness? Michael Ehrmann and Marcel Frazscher * European Cenral Bank Michael.Ehrmann@ecb.in, Marcel.Frazscher@ecb.in November 2004 Absrac The paper
More informationExchange Rate PassThrough into Import Prices: A Macro or Micro Phenomenon? Abstract
Exchange Rae PassThrough ino Impor Prices: A Macro or Micro Phenomenon? Absrac Exchange rae regime opimaliy, as well as moneary policy effeciveness, depends on he ighness of he link beween exchange rae
More informationStatistica Siica 6(1996), 31139 EFFECT OF HIGH DIMENSION: BY AN EXAMPLE OF A TWO SAMPLE PROBLEM Zhidog Bai ad Hewa Saraadasa Natioal Su Yatse Uiversity Abstract: With the rapid developmet of moder computig
More information