Lecture 8 Dynamic Optimization

Transcription

1 Lecure 8 Dynamic Opimizaion Many economic problems involve making choices over ime. Examples include: 1) Individuals a) People mus plan heir savings and invesmen decisions o prepare hemselves for reiremen. b) Individuals plan heir invesmens in human capial (educaion, job raining) before and during heir careers. 2) Firms make decisions over ime abou mainaining heir socks of physical capial (buildings, equipmen). 3) Fossil fuels a) Oil, gas and coal companies mus plan exracion, and exploraion and developmen of new reserves over ime. b) Policymakers mus deermine ime pahs of policy insrumens for conrolling carbon emissions o respond o he hreas of global climae change. Recall ha carbon accumulaes in he amosphere. 4) Fores managers mus choose how long o le rees grow before hey are harvesed. his decision may also be affeced by moivaions o preserve wildlife socks and he flow of recreaion services a fores provides. 5) Fishery managers mus choose cach quoas ha are deermined in par by he naural growh and decline of fish socks. Each of hese examples has several common characerisics: 1) All involve managing he sock of an asse over ime reiremen savings, producion capial, oil reserves, concenraion of carbon in he upper amosphere, fores socks, and fish populaions. 2) In each case decisions in one ime period will affec opporuniies and payoffs in he fuure. Examples: Increased exracion of oil wih no addiions of new reserves will leave less oil for fuure exracion and consumpion. Increased fish harvess may resul in a lower growh rae, leaving a reduced fish sock in he fuure. 3) All decisions are funcions insead of single values. hese decisions are ime pahs of acions over some ime frame. he Elemens of a Dynamic Decision Problem: Managing Reiremen Savings An individual reires a dae = and will die a dae =. o prepare for reiremen she has saved s over her career. She does no work during her reiremen years so her income during his ime comes from ineres on her savings paid a rae i. Her savings and ineres income over he ime frame =,..., is allocaed beween consumpion c and new savings. he sock she is rying o manage is her reiremen capial, which changes beween periods according o s 1 s = is + c, 8.1

2 where s is her sock of savings in ime. Her problem is o opimally plan her consumpion during her reiremen years; ha is, she chooses ( c, c1,..., c ). Suppose ha she ges uiliy from consumpion according o uc ( ), and ha her goal is o maximize he presen value of her uiliy sream over her reiremen years. ha is, she solves uc ( ) max, { c } (1 ) = = + r where r is he rae a which she discouns fuure uiliy; ha is, 1 ( 1 r) + is he discoun facor applied o uiliy received in ime. Here choice of consumpion pah is consrained by how her sock of reiremen savings changes over ime, s + 1 s = is c, =,..., 1, as well as condiions abou how much savings she begins wih and how much she dies wih; ha is, s s s = = and = s. We may require s = if she inends o leave nohing behind and canno pass away in deb. Or, if she has some arge for he amoun she wans o leave o her children, s >. We have saed he reiree s problem in discree ime, bu i may be more convenien (or appropriae) o analyze her problem in coninuous ime. If so, hen her objecive is saed as max r e u ( c ( )) d, c () subjec o he descripion of how her sock of savings evolves, s () = is () c (), and iniial and erminal condiions, s() = s and s ( ) = s. Noe ha: 1) r e is he coninuous ime discouning facor. 2) s ( ) is he derivaive of s ( ) wih respec o ime. 3) By convenion, for any variable x, x is used o denoe x in ime in discree-ime problems, while x( ) is used o denoe he same hing in coninuous ime problems. 8.2

3 In his lecure we will invesigae wo mehods for solving problems like hese. he mehod of opimal conrol, which we will look a firs and more in deph, will be applied o coninuous-ime problems. he mehod of dynamic programming will be applied o discree-ime problems. Elemens of a One-dimensional Opimal Conrol Problem he sae equaion Consider he following firs-order differenial equaion dx x() f(, x(), u()), d = = [, ], where: i) is ime, he inerval [, ] is he planning period and is he ime (planning) horizon. may be finie or infinie, and eiher exogenous our endogenous. ii) x( ) is he sae variable, which describes he sae of he sysem a ime. Usually, x( ) is he level of he sock ha is being managed. iii) u ( ) is he conrol variable. his is he choice variable (exracion rae, oupu choice, invesmen, ec.). Given an iniial sae, x() = x, which is usually assumed o be exogenous, he funcion f governs how he sae (sock) evolves wih he choice of conrol. Because of his, x () = f(, x(), u()), is called he sae equaion or equaion of moion. Examples: 1) A non-renewable resource in fixed supply. x( ) is he size of he reserve in, u ( ) is exracion in, and he sae equaion is x () = u(). 2) An aquifer. x( ) is he volume of waer in he aquifer in, u ( ) is wihdrawal in, and R is he naural rae of recharge. he sae equaion is x () = R u(). 3) A fishery. x( ) is he sock of fish in, u ( ) is harves in, f(x()) is he naural growh of he fish sock in. he sae equaion is x () = f( x()) u(). 4) A cumulaive polluan. x( ) is he sock of he polluan in, u ( ) is emissions in, δ (,1) is he proporion of he polluan ha decays naurally in every period. he sae equaion is 8.3

4 x () = u() δ x(). 5) Consumpion during reiremen. x( ) is he sock of savings in, u ( ) is consumpion financed from savings in, and i is he rae of reurn on savings. he sae equaion is x () = ix() u(). 6) A firm s capial sock. x( ) is he firm s capial sock in, u ( ) is new invesmen in, and d is he consan rae of depreciaion of he capial sock. he sae equaion is x () = u() dx(). Admissible conrols In mos economic applicaions, we will need o resric he level of a conrol, a leas implicily. Obviously producion, emissions, harves and he like canno be negaive. here may also be upper-bound consrains on hese conrols ha have o do wih capaciy consrains. For example, exracion of an ore by a mining operaion wih fixed capaciy will be consrained by u () [, u], where u ( ) is exracion in and u is exracion capaciy. Le U denoe a conrol se in. his defines he range of possible values of he conrol u ( ) in. hen: he class of admissible conrols U is he class of all piece-wise coninuous real funcions u() defined over [, ], and saisfying u () U for all. he definiion of admissible conrols highlighs wo imporan characerisics of conrol problems. 1) he opimal conrol in a paricular ime period may be on a boundary of he conrol se. he opimal conrol mehod provides ways o deal wih boundary soluions. 2) An admissible conrol does no need o be coninuous, only ha if i is disconinuous i is disconinuous a a finie number of poins. An Illusraion: Suppose a fishing concern has a fixed capaciy u over some finie planning period [, ]. he conrol u ( ) is harves and he conrol se is U = [, u] for all in he planning period. he following graph illusraes wo conrols, one of which is coninuous, he oher only piece-wise coninuous. Boh are admissible conrols. 8.4

5 u () u u () u () Alhough conrols need no be coninuous, saes x( ) mus be, bu hey may be only piece-wise coninuously differeniable; ha is coninuously differeniable excep a a finie number of poins. For example, he applicaion of conrol uabove () may generae he following x( ). x() Noe ha x( ) is coninuous, and coninuously differeniable excep a daes 1, 2, and 3. Iniial and erminal condiions Wih he applicaion of a paricular conrol u ( ) and he sae equaion x ( ) = f(, s( ), u( )), he x() pah is only fully described wih an iniial condiion x () and a erminal condiion, which is a saemen abou how he sysem is compleed. 8.5

6 Alhough iniial condiions can be variable, hey usually are fixed because hey usually describe he level of a sock a planner sars a planning period wih. In our problem of consumpion during reiremen we specified how much savings he individual brough ino he beginning of her reiremen. Fixed iniial condiions are usually wrien as x() = x, where x is an exogenous consan. here is much more variabiliy in he kinds of erminal condiions we are likely o encouner. he reason for his is ha we mus say somehing abou boh he erminal sae x( ) and he erminal dae. Boh may be fixed, one may be fixed while he oher can vary, or boh may be variable. Possible erminal condiions 1) Fixed ime, fixed endpoin boh x( ) and are fixed. For example, a fishery regulaor seeks o manage harvess over one year s ime and does so in par o make sure ha here is a cerain amoun of he fish sock lef. hen is fixed a (welve monhs, 52 weeks, whaever is used as he uni of ime) and x( ) is consrained o be some value x. In he graph below are possible pahs of he fish sock x( ), wih iniial sock x() = x and erminal condiion =, x( ) = x. x() x x 2) Fixed ime, variable endpoin is fixed, while x( ) is free. In hese problems he ime horizon is fixed, bu he erminal sae is free. For example, a mine manager is charged wih planning exracion over a fixed horizon. He is free o leave ore in he ground a he end of his erm, perhaps because i may no be economical o exrac i all. He sars he planning period wih x in he 8.6

7 ground and here is no new developmen of addiional reserves during [, ]. wo possible pahs for x( ) are graphed below. x() x x ( ) > x ( ) = o be compleely rigorous, his is a fixed-ime, runcaed end-poin siuaion, because x() is no compleely free. I mus be non-negaive, so we really need he consrain x ( ). 3) Variable ime, fixed endpoin is variable, bu x( ) is fixed. Suppose an environmenal regulaor is charged wih sabilizing he concenraion of a sock polluan, bu is given complee discreion over when he arge concenraion is o be reached. Here x( ) = x he arge concenraion and is variable. Suppose ha x x() < x. wo possible pahs for x( ) are shown below. x (1) (2) x

8 For pah (1), x is reached raher quickly because he polluan is allowed o accumulae quickly a firs. For pah (2), accumulaion is more rapid a laer daes. 4) Variable ime, variable endpoin (erminal curve) and x( ) are variable bu mus saisfy x = φ( ), where φ is given. Suppose a manager wans o build up he capial sock of a firm wih he inenion of selling he firm a a laer dae. he manager would like o ake some ime o build he capial sock, bu he longer he wais o sell he lower he presen value of he firm. his rade-off is summarized by x = φ( ). he iniial capial sock is x. wo pahs are shown. x() φ( ) x he objecive funcional he fundamenal problem of opimal conrol is o find a conrol u ( ), [, ], o maximize a cerain objecive, subjec o consrains on he evoluion of he sae variable x( ). (hese consrains are he iniial condiion, he sae equaion, and he erminal condiion). In mos economic applicaions he objecive is o opimize some sream of payoffs (coss, profi, welfare, ec.). Le gx (, ( ), u ( )) denoe an insananeous payoff in. hen he sream of payoffs over he planning inerval [, ] is gx (, (), u ()) d. 8.8

9 his is called he objecive funcional. Usually in economic applicaions he objecive is o opimize he presen value of he sream of payoffs. If gx (, (), u ()) is he curren value (undiscouned) payoff in, he objecive funcional is r gx (, ( ), u ( )) d= e gx (, ( ), u ( )) d. Examples: A mine operaor plans exracion, u ( ), of a mineral resource over [, ]. Exracion coss a consan c per uni, and he exraced ore sells a a compeiive price ha evolves according o p(). he mine operaor wans o choose an exracion pah o maximize he presen value of he sream of profis from exracion. he objecive funcional is [ () () ()] r e p u cu d. he EPA wans o manage emissions, u ( ), of a sock polluan over [, ]. he sock of he polluan, x( ), causes damage d( x( )), and he aggregae coss of conrolling emissions are cu ( ( )). he EPA wans o choose a ime pah of emissions o minimize he presen value of he sream of damages plus conrol coss. he objecive funcional is [ + ] r e d( x ( )) cu ( ( )) d. r Noes: In he firs example, gx u e [ pu cu] (, (), ()) = () () (). Noe ha he sae variable does r no ener he objecive funcion, and ime eners independenly in p( ) and e. In he second r example gx (, ( ), u ( )) = e [ d( x ( )) cu ( ( ))]. Here, he objecive funcional includes he sae r variable, and ime eners independenly only as par of he discoun facor e. Scrap-value objecive funcional Besides opimizing he presen value of a sream of payoffs, oher forms of objecive funcional are possible. Anoher form arises when a decision-maker is also concerned abou he value of he sae a he end of he planning period, ofen because he/she inends o sell he remaining sock of whaever he/she is managing when he planning period is over. hese are called scrapvalue (salvage-value) problems. Le vx ( ( ), ) denoe he scrap value of x in. hen, he objecive funcional is gx (, (), u ()) d+ vx ( ( ), ). Example: Over [, ] a firm invess u ( ) in is capial sock x( ). he capial sock generaes profi according o π ( x( )), and he cos of invesing u ( ) is cu ( ( )). he firm chooses a ime- 8.9

10 pah of invesmen o maximize i discouned sream of profis, plus he presen value of he salvage value of is capial sock remaining in. Le he curren salvage value of x( ) be vx ( ( ), ). he firm s objecive funcional is [ π ] r r e ( x( )) c( u( )) d + e v( x( ), ). he fundamenal problem of opimal conrol We now have all he elemens necessary o give a formal saemen of a one-dimensional conrol problem. I is o choose a conrol u ( ), [, ], o maximize gx (, ( ), u ( )) d () subjec o x = fx (, ( ), u ( )) x() = x x ( ), some erminal condiion u () U, [, ]. he soluion (provided one exiss) is u ( ), [, ], which provides a ime pah of he sae, x (), [, ], as he soluion o x () = f(, x(), u ()) x() = x x ( ), some condiion. he maximum value for he problem is gx (, (), u ()) d. 8.1

11 he Maximum Principle We can now urn our aenion o characerizing opimal conrols. o do so we firs inroduce anoher variable, λ ( ), called a cosae variable (or adjoin variable). his variable will serve a role similar o Lagrange mulipliers in saic consrained opimizaion problems. As such i is a value variable, which provides a shadow price for he sae variable. he cosae variable eners our conrol problem wih he Hamilonian funcion, Hx (, ( ), u ( ), λ( )) = gx (, ( ), u ( )) + λ( ) f( x, ( ), u ( )). (From now on le s drop he () aached o x, u, and λ, excep when we need i). Necessary condiions he maximum principle is a se a necessary condiions ha deermine an opimal conrol, he associaed response, as well as he pah of he adjoin variable. hey are: i) max u Hsu (,,, λ), [, ] ii) H g f λ = = + λ x x x H iii) x = = f(, x, u) λ iv) a ransversaliy condiion. max u Hsu (,,, λ), [, ], indicaes ha he conrol u mus maximize he Hamilonian a every insan of ime. Assuming ha H is coninuously differeniable, u is unbounded, or we insis on inerior soluions for u, and an appropriae second-order condiion is saisfied (more on his laer), we may replace max u H wih H u =. Equaion ii) governs he pah of he cosae variable, while iii) is simply a re-saemen of he sae equaion x = f(, x, u). ransversaliy condiions govern he sae and is value a, so hese condiions vary according o he erminal condiion ha is assumed. ransveraliy condiions 1. Fixed, fixed x( ) = x. No ransversaliy condiion is required. 2. Fixed, free x( ). he ransversaliy condiion is λ ( ) =, indicaing ha he value of he sae a mus be zero. 8.11

12 3. Fixed, x( ) x. he ransversaliy condiions are λ( ), x( ) x, λ( )[ x( ) x ] =. Suppose ha x =. hen his condiion means ha eiher he sock is exhaused, x ( ) =, or if here is some remaining is value mus be zero, λ ( ) =. 4. Fixed x( ) = x, is free. he ransversaliy condiion is [ H ] = =, indicaing ha he program ends when here is no value o keeping i going. 5. and x( ) are free bu governed by x( ) = φ( ). he ransversaliy condiion is [ H λφ ] = =.. Sufficien condiions For he conrol problem maximize gxud (,, ) and associaed Hamilonian, he maximum principle condiions, subjec o x = f(, x, u) [1] x() = x a erminal condiion, H( xu,,, λ) = gxu (,, ) + λ f( xu,, ), [2] maxu H λ = H x (,, ) x = f x u and ransversaliy condiion, [3] are only necessary condiions. For hem o be necessary and sufficien o idenify an opimal conrol u and associaed sae x, we mus make sure ha one of a number of sufficien condiions are saisfied. Mangasarian (loosely). If gxu (,, ) is concave in x and u, and f ( xu,, ) is linear in x and u, he necessary condiions [3] are also sufficien o idenify a soluion o [1]. If gxu (,, ) is sricly concave in x and u, he soluion o [1] is unique. 8.12

13 If f ( xu,, ) is non-linear bu concave in x or u, and i is rue ha wih an opimal soluion λ(), [, ], he necessary condiions [3] are also sufficien. Arrow (a weaker condiion). Le * u = u * (, x, λ). Define * u maximize Hxuλ (,,, ) given, x and λ. hen, * * * H(, x, λ) = Hxu (,,, λ) = gxu (,, ) + λ fxu (,, ). hen, he necessary condiions [3] are also sufficien o idenify a soluion o [1] if concave in x. H (, x, λ ) is Applicaion of he Maximum Principle: Consumpion during reiremen Le s reurn o our problem of a reiree managing her savings during her reiremen. Recall ha she begins her reiremen a = wih savings s. She can generae income from her savings a rae i. From her savings and ineres income she finances consumpion c. She will die a ime and she canno die in deb. hen, he condiions governing her pah of savings s are s = is c, s() = s, s( ), and is fixed. Her uiliy from consumpion is uc ( ), wih u ( c) > and u ( c) <. She wishes o maximize he presen value of consumpion during her reiremen years, and she has a rae of ime discoun r. Her conrol problem is o choose c ( ), [, ], o solve r maximize e u( c) d subjec o s = is c [4] s() = s s ( ), fixed. he Hamilonian for his problem is r Hsc (,,, λ) = e uc ( ) + λ( is c). [5] he necessary condiions are r H = e u () c λ = (max c H ) [6] c λ = H λ = iλ (cosae equaion) [7] s s = is c (sae equaion) [8] λ( ), s( ), λ( ) s( ) =. (ransversaliy condiion) [9] 8.13

14 r Since e u( c) is concave in c and he sae equaion is linear in c and s, hese condiions are also sufficien o idenify a soluion o [4]. r Equaion [6] reveals he sign of he cosae variable λ. Since e u ( c) >, [6] reveals ha λ >, [, ]. We should noe ha λ mus be a presen value. Again from [6], since marginal uiliy is discouned (a presen value), λ mus also be a presen value in order for his condiion o be a valid comparison. he inerpreaion of λ is ha i is he presen value of he shadow price of he reiree s savings i is he presen value of he marginal value she places on her savings. Since λ >, [, ], λ ( ) s( ) = from he ransversaliy condiion [9] implies s ( ) = ; ha is, her savings will be exhaused by he dae of her deah. r Rewrie [6] as u ( c) = λe. Now marginal uiliy and he shadow price are in curren (-period) values. his condiion says ha her opimal consumpion (opimal conrol) a any dae is where her marginal uiliy from consumpion (her marginal benefi of consumpion) is equal o he shadow price of her savings (her subjecive marginal cos of reducing her savings in order o increase curren consumpion). Now le s ry o say somehing abou how her consumpion changes over her reiremen years. Firs combine [6] and [7] o obain r λ = ie u (). c [1] Now differeniae [6] wih respec o ime and rearrange erms o obain r r λ = re u () c + e u (). c c [11] Combine [1] and [11] and eliminae r e : ( ) iu () c = ru () c + u c c. Combine erms and rearrange o obain u () c c = ( i r ). u () c Since u u >, sign c = sign ( i r), indicaing ha he qualiaive direcion of consumpion depends on he relaive levels of he rae of reurn on her savings and her rae of ime preference (he rae a which she discouns fuure uiliy). If he wo raes are he same her consumpion is consan hrough ime. If she is relaively paien ( i > r), her consumpion rises ( c > ) hroughou [, ]. If she is relaively impaien ( i< r), her consumpion seadily decreases ( c < ) over her reiremen years. 8.14

15 he imporance of boh he marke rae of ineres i and her subjecive rae of ime discoun r is an ineresing characerisic of his model. Le s examine his from he perspecive of he cosae variable λ. he cosae equaion [6], λ = iλ, is a simple linear homogenous differenial equaion wih soluion λ = λ i e. In general, λ is an arbirary consan, bu by seing = we have λ() = λ, indicaing ha λ is he shadow value of her savings a = ; ha is, λ is he marginal value of s. i he equaion λ = λe indicaes ha he presen value of his shadow price decreases over ime. i However, λe = λ indicaes ha he curren value of he shadow price is consan a λ over [, ]. Noe ha he pah of λ is governed by he marke reurn on her savings i. However, [6], which r we have rewrien as u ( c) = λe, indicaes ha her subjecive rae of discoun r is key in r deermining her opimal consumpion in every ime period. Combine u ( c) = λe and λ = λe o obain ( r i) u () c = λ e, which clearly indicaes ha opimal consumpion depends on he relaionship beween her subjecive rae of ime preference and he marke rae of ineres. Of course we noed his fac already, bu from a differen perspecive. i Curren-value Hamilonian Consider he problem r maximize e π (, x, u) d subjec o x = f(, x, u) x() = x a erminal condiion. r Le π indicae insananeous profi. Noe ha π ( ) is a curren-value expression and π ( ) e is he presen value of π. Wrie he Hamilonian, r H = π(, x, u) e + λf(, x, u). 8.15

16 As we ve noed, λ is he presen value of he shadow price of x. Since λ is a presen value and π is a curren value, confusion can arise when inerpreing he Hamilonian and he necessary condiions. herefore, i is someimes convenien o make all values curren values. r We do his by creaing a new cosae variable µ such ha µ e = λ. Noe ha µ is a curren value shadow price. Incorporaing µ ino he Hamilonian gives us r c r H = [ π (, x, u) + µ f(, x, u)] e = H e. c H = π ( xu,, ) + µ f( xu,, ) is called he curren-value Hamilonian. Necessary condiions 1) Since r e is independen of u, max u H c max H. u herefore his necessary condiion is he same wheher we use he curren value Hamilonian or no. 2) he cosae equaion is λ = H x. ransform each side of his equaion using he r new cosae variable. Since λ = µ e, Furhermore. r r λ rµ e = + µ e c r Hx = Hxe. herefore λ = Hx is idenical o or rµ e + µ e = H e r r c r x µ rµ = H x c., his is he cosae equaion for he curren value cosae variable. 3) he condiion for he sae equaion is x = H = f(, x, u), λ c c bu since H = H, we may use x = H = f(, x, u) insead. λ µ µ 8.16

17 r 4) ransversaliy condiions are revised by simply using λ = µ e. For examples: i) Fixed, free x( ) requires r λ ( ) = µ ( ) e =. ii) Fixed, x( ) x, requires which is idenical o λ( ), x( ) x, λ( )[ x( ) x ] =, r r µ ( e ), x ( ) x, µ ( e ) [ x ( ) x] =. Economic Inerpreaion of he Maximum Principle Each elemen of he maximum principle has an inuiive economic inerpreaion. In his secion we will develop hese inerpreaions wih a simple conrol model of a profi-maximizing firm. he sae variable is is capial sock K. he conrol variable is u, which represens some decision of he firm ha affecs he firms profi and is capial sock. Le π denoe he firm s profi and suppose ha is conrol problem is maximize u r e π (, K, u) d subjec o K = f(, K, u) K() = K K ( ) free, given. he Hamilonian is H= π (, Ku, ) + λ f(, Ku, ). As we have been doing all along, he cosae variable is inerpreed as he shadow price of he capial sock. I is he marginal valuaion of he firm s capial sock a a given poin in ime. Now consider he Hamilonian in is enirey. I is made up of wo componens π ( Ku,, ) and λ f (, Ku, ). he firs componen is he curren profi from decision u, given curren capial K. he second componen is λk. K is he change in he capial sock from decision u, given K; hence, λk is he change in capial value from decision u. While π ( Ku,, ) is he curren profi effec of u, λk is he fuure profi effec of u hrough he change in he capial sock. he Hamilonian, herefore, accouns for he effecs on curren and fuure profis of he decision u. 8.17

18 he maximum principle requires max u H. Assume ha his can be deermined by or H u = π + λ f =, u u u π = λ f. u Wha his means is ha a each poin of ime, he firm mus make he decision u o balance he marginal increase in curren profi agains marginal decrease in fuure profi hrough he change in he capial sock. he maximum principle also requires saisfacion of wo equaions of moion. One of hese, he sae equaion K = f(, K, u), is exogenously given and simply specifies how he decision u affecs he change in he capial sock. In conras he cosae equaion, λ = H = π λf K K is endogenous. λ is he increase in capial value a some. Rewrie he cosae equaion as λ = πk + λfk. Imagine adding a uni of capial: λ is he decrease in he value of he capial sock (because i is marginally less scarce), while π K + λ f K is he increase in curren and fuure profis. hus, opimal capial accumulaion a occurs when he marginal decrease in he value of he capial sock is equal o he marginal conribuion o curren and fuure profi. K, he remaining condiion of he maximum principle is a ransversaliy condiion. Suppose ha K ( ) is free bu is fixed. he ransversaliy condiion is λ ( ) =,, indicaing ha he shadow price of capial be driven o zero a he erminal dae. A he erminal dae, any capial ha is lef has no value because i is oo lae o use i o produce more profi. Suppose insead ha K ( ) K min > for some reason. he ransversaliy condiion is λ( ) and λ( )[ K( ) K min ] =. hen, if he capial sock has value a some fixed, K ( ) = K min, indicaing ha he firm should use as much capial as i is allowed. On he oher hand, if K ( ) > Kmin, hen λ( ) =, indicaing ha he excess capial has no value. Lasly, consider a variable erminal ime problem. Suppose K ( ) = K, some pre-specified value, bu is free. hen, he ransversaliy condiion is [ H ] = = ; ha is, should be chosen so ha a ha a he sum of fuure and curren profi is zero. In oher words, K should no be reached as long as here are curren and/or fuure profis o be had. 8.18

19 Auonomous Problems Consider he problem maximize gxud (, ) u subjec o x = f( x, u) x() = x a erminal condiion. Problems of his sor are called auonomous problems, because ime does no ener he objecive funcional or sae equaion independenly. his means ha he Hamilonian, H( x, u, λ) = g( x, u) + λ f( x, u), as well as he necessary condiions, do no involve explicily. his fac makes auonomous problems much easier o solve. Anoher consequence of auonomous problems is ha he Hamilonian evaluaed along opimal pahs u, x, and λ is consan hrough ime. o see his differeniae H (, x, u, λ ) wih respec o ime: dh H H H H = + x + u + λ. d x u λ Opimaliy requires H u = ; hence, ( H / u) u =. Furhermore, opimaliy requires x = H λ and λ = H x, implying herefore, H x dh d H x + λ = λx + x λ =. λ H =. If we are looking a an auonomous problem wih maximized Hamilonian H = H( x, u, λ ), H = and dh d =. hus, he Hamilonian evaluaed along is opimal pah is consan hrough ime. Suppose we have an auonomous problem wih a variable erminal ime. hen he ransversaliy condiion is [ H ] =. Since he problem is auonomous H = a every poin in ime. Now suppose ha he per-period payoff gxuis (, ) discouned, so ha our conrol problem is 8.19

20 r maximize u gxue (, ) d subjec o x = f( x, u) x() = x a erminal condiion. Alhough now eners he objecive funcional independenly, economiss end o view his kind of problem as auonomous as well. he reason is ha we can ake he discoun facor ou of consideraion by using he curren-value Hamilonian c H = g( x, u) + µ f( x, u). Clearly, wih x and u c, dh d =, implying ha ime. c H along is opimal pah is consan hrough 8.2

21 Anoher example: Opimal exracion of a nonrenewable resource and environmenal damage. Suppose ha a sociey explois a nonrenewable resource for consumpion, bu ha exracion of he resource generaes a flow polluan. he sock of he resource is denoed x wih iniial sock x. here is no exploraion and discoveries of new deposis. hen, if y denoes exracion, he equaion of moion for he sock is simply x = y. Le b(y) denoe ne consumpion benefis of he exraced resource. Benefis are increasing, sricly concave, and quadraic in exracion; ha is, b >, b <, andb =. Exracion causes damage according o d( y) = dy, where d is he consan marginal damage from exracion. he goal of a social planner is o choose an exracion pah o maximize he presen value of he flow of ne consumpion benefis less environmenal damages over he inerval [, ] wih a fixed erminal dae. Formally, maximize y r [ by ( ) dye ] d subjec o x = y [12] x() = x x ( ), fixed. r he Hamilonian is H = [ b( y) dy] e λ y. Define Hamilonian, r µ e λ = and wrie he curren-value c H = b( y) dy µ y. [13] Noe ha H c is sricly concave in y and he sae equaion is linear in y and x; herefore, he following necessary condiions are also sufficien o solve [12]. H = b ( y) d µ = [14] c y d r c r r r [ e ] Hxe e r e r d µ = = µ µ = µ = µ (cosae equaion) [15] = = µ (sae equaion) [16] c H x y ( ) r r µ e, x ( ), µ ( xe ) ( ). (ransversaliy) [17] he curren-value cosae variable µ is he shadow price of he resource sock, ofen referred o as he user cos of he resource (from he fac ha a uni exraced in one period is unavailable for use in he fuure). Equaion [14] says ha a any poin in ime, exracion akes place o he poin where curren marginal benefi minus marginal damage is equal o he user cos of he resource sock. 8.21

22 Equaion [15] says ha he rae of appreciaion of he resource sock is equal o he ineres rae r. his is he sandard Hoelling resul for nonrenewable resource exracion. Since µ = rµ is a r simple linear homogeneous differenial equaion, is soluion is µ = µ e, or r µ e µ =. [18] In [18], µ is he shadow price of he iniial sock of he resource. Equaion [18] says ha he presen value of he resource is consan hrough ime. Now urn o he ransversaliy condiion. One possibiliy is ha x ( ) and µ ( ) =. Using r r [18], µ ( e ) = µ = ; hence, µ = µ e = for all [, ]. In his case he resource is no really scarce. Wih µ =, [, ], equaion [14] reduces o he basic saic condiion ha marginal consumpion benefi should be equal o marginal damage in every ime period. Furhermore, since b ( y) d = in each ime period, opimal exracion is consan hrough ime. his problem is more ineresing when x ( ) = and µ ( ) >. In his case, µ > in every period. Given µ >, les combine [14] and [15] o say somehing abou he opimal exracion pah. Noe firs ha µ = rµ >, indicaing ha he curren-value shadow price rises hrough ime. Now differeniae [14] wih respec o ime o obain µ = by. [19] Since µ > and b <, hen y <. hus, exracion is decreasing over ime. Use [14] and [15] o wrie µ = rb ( ( y) d). Equae his o [19] and rearrange erms o obain Differeniae [2] wih respec o ime o obain rb ( ( y) d) y =. [2] b rb y y = = ry <. b [Recall ha since b is a quadraic funcion, b is a consan]. he inequaliy follows because y <. herefore, he ime pah of exracion is decreasing and concave. 8.22

23 y Since he sock of he resource is o be exhaused by he erminal dae, he area under he exracion pah is equal o he iniial sock of he resource; ha is, yd = x. Now le s consider how environmenal damage may affec he exracion pah. Opimal exracion and he shadow price will be a funcion of he marginal damage parameer d. Rewrie [14] using his fac: b ( y( d)) d µ ( d). [21] Noe ha his mus hold as an ideniy. Use [21] o rewrie [2]: Since y = rµ / b, sign[ y ] = sign [ µ ]. d d rµ ( d) yd ( ) =. [22] b d d Recall ha µ is he shadow value of he resource in he ground so o speak. If exracion and consumpion of he resource causes greaer environmenal damage (higher d), he value of he resource should be lower. herefore, i seems reasonable o expec ha µ d <. If his is rue, hen y d >. Since y <, y d > implies ha higher damage makes he exracion pah decline less seeply. In he graph below we have wo exracion pahs, one ha assumes polluion damage from exracion and consumpion of he resource, while he oher assumes eiher ha here is no 8.23

24 polluion damage, or he social planner ignores polluion damage. Recall ha he areas under boh exracion pahs have o be equal because yd = x. herefore, because polluion damage generaes a less seep exracion pah, exracion mus be lower in earlier periods and higher in laer periods. y Exracion wih polluion damage. Exracion wihou polluion damage. he reason for shifing exracion o he fuure in he presence of polluion damage is o reduce he presen value of his damage. Noe ha oal undiscouned damage, dy d dx ( ) =, is consan. his can be changed. However, he presen value of damages, ( ) r dy e d, can be reduced by shifing exracion (and polluion) ino he fuure. 8.24

25 Muli-Dimensional Conrol Problems hus far we ve considered one-dimensional conrol problems (one conrol, one sae). he maximum principle is easily exended o muli-dimensional problems. Suppose a problem has n sae variables, ( x1, x2.., x n), and m conrol variables, ( u1, u2,... u m). Since we have n sae variables we mus have n sae equaions x 1 = f1(, x1,..., xn, u1,..., um) x 2 = f2(, x1,..., xn, u1,..., um) x = f (, x,..., x, u,..., u ). n n 1 n 1 m More compacly, x i = fi(, x1,..., xn, u1,..., um), i = 1,..., n. Each sae has an iniial posiion xi() = xi, i = 1,..., n, as well as a erminal condiion. Suppose ha is fixed and he x ( ), i = 1,..., n, are free. Consider he conrol problem i maximize gx (,,..., x, u,..., u ) d uj, j= 1,..., m 1 n 1 m subjec o x = f (, x,..., x, u,..., u ), i = 1,..., n i i 1 n 1 m x () = x, i = 1,..., n i i x ( ), i = 1,..., n, are free, given. i Since we have n sae equaions, we need o inroduce n cosae variables, λ i, i = 1,..., n. he Hamilonian is H gx (,,..., x, u,..., u ) λ f ( x,,..., x, u,..., u ). = + n 1 n 1 m i= 1 i i 1 n 1 m he necessary condiions are max u j H, j = 1,..., m λi = H xi, i = 1,..., n x i = H λi = fi( ), i = 1,..., n λ ( ) =, i = 1,..., n. i Noe ha he maximum principle for muli-dimensional problems is a sraighforward exension of he one-dimension case. In pracice however, analyic soluions o muli-dimensional problems are very difficul o obain. Usually, if an analys is ineresed in specific soluions (as opposed o simply drawing qualiaive conclusions from he necessary condiions), he or she will rely on numerical echniques. 8.25

26 Infinie Horizon Models For infinie horizon models he planning period is [, ]. Consider he following conrol problem: r max π ( x, ue ) d u s.. x = f( x, u) x() = x he curren-value Hamilonian is he necessary condiions are (,, µ ) π (, ) µ (, ) c H x u x u f x u = +. c Hu ( x, u, µ ) = d ( µ e ) = H ( x, u, µ ) e d x = f( x, u). r c r x Noe ha in he saemen of he problem here is no erminal condiion, and he necessary condiions do no include a ransversaliy condiion. In infinie horizon problems, he seady sae ofen replaces he erminal condiion. Moreover, a ransversaliy condiion is replaced by he assumpion ha he opimal soluion approaches a seady sae. In a seady sae he sae and cosae variables are consan; ha is, x = µ =. If a seady sae exiss and is unique, we can use x = µ = and he necessary condiions o idenify seady sae s s s values ( x, u, µ ). Given ha we can idenify a seady sae, we can examine pahs o he seady sae of x, u, and µ. We will do so wih a phase diagram. Example: Conrolling a sock polluan A sock polluan is a polluan ha accumulaes over ime, and he damage i causes a a poin in ime is a funcion of how much has accumulaed o ha poin. For example, greenhouse gases like carbon dioxide are sock polluans. Noaion: S sock of he polluan in he amosphere: his is he sae variable E poenial aggregae emissions: his is aggregae emissions wihou any conrol A abaed emissions, i.e., emissions reducion: his is he conrol variable 8.26

27 (E-A) aggregae flow of emissions g percenage of he sock of he polluan ha decays in a ime period, g (,1). he sae equaion is S = E A gs. Le: D(S) aggregae damage funcion wih D > and D >. C(A) aggregae abaemen coss wih C > and C >. i discoun rae held consan over [, ]. he planning objecive is o minimize he presen value of he flow of oal coss damage plus abaemen coss: minimize A i [ DS ( ) + CA ( )] e d subjec o S = E A gs S() = S (he iniial sock of S). he curren-value Hamilonian is c H = D( S) + C( A) + µ ( E A gs). Since D and C are sricly convex, he Hamilonian is sricly convex in ( S, A ). herefore, he following necessary condiions are also sufficien. H = C ( A ) µ = [23] c A i c i i i i d( µ e ) d = H e µ e iµ e = ( D ( S) µ g) e S µ = D ( S) + µ ( i+ g) [24] S = E A gs [25] he curren-value cosae variable is he marginal reducion in fuure damages and abaemen coss from abaing one uni now. hus, µ = C ( A) balances he marginal benefi of curren abaemen agains marginal coss. Since µ is he shadow value of abaing a uni of emissions, we can inerpre i as he opimal ax on emissions. Graphically: 8.27

28 µ C ( A) µ A A he Seady Sae o idenify he seady sae we use he firs hree firs order condiions and se S = µ =. ake he cosae equaion [24] firs and solve which yields µ = D ( S) + µ ( i+ g) =, µ = D ( S) ( i+ g). [26] Now consider he sae equaion: hen, wih S = E A gs =. [27] ( A) µ = C, [28] we have a sysem of hree equaions, [26]-[28], wih hree unknowns ( S, A, µ ). he soluion o s s s [26]-[28] is he seady sae ( S, A, µ ). o illusrae he soluion in wo dimensions le s focus on µ and S for which µ =. From [26], noe ha µ S = D ( S) ( i+ g) >. Graphically, 8.28

29 µ ( µ = ) S he sae equaion [27], S = E A gs =, akes a lile more work because i doesn involve µ direcly. o ge µ ino S we need boh [27] and [28]. Noe firs ha S = E A gs = implies A= E gs [29] in he seady sae. Combine [28] and [29] o obain µ = C ( E gs). [3] Equaion [3] collecs all combinaions of µ and S for which S =. Noe ha µ / S = C ( E gs) <. Graphically: µ ( µ = ) s µ s S ( S = ) he seady-sae value of µ and S are deermined as he simulaneous soluion o S and µ = D ( S) ( i+ g) [ µ = ], [26] µ = C ( E gs) [ S = ]. [3] 8.29

30 he seady-sae level of abaemen is deermined easily wih he sae equaion S S s = E A gs = ; ha is, A = E gs. Pahs o he Seady Sae A Phase Diagram One of he mos ineresing aspecs abou seady-sae analysis is he analysis of how µ, S, and A move oward he seady sae. o do his we consruc a phase diagram by examining how µ moves when he sysem is away from µ = and doing he same wih S. Begin wih µ and he cosae equaion [24]: µ = µ ( S, µ ) = D ( S) + µ ( i+ g). Noe ha µ S = D ( S) <. [31] Consider a pair ( S, µ ) such ha µ =. ha is, µ = µ ( S, µ ) = D ( S) + µ ( i+ g) =. Leave µ = µ, bu consider S 1 1 S > S, µ ( S, µ ) <. Graphically: < S. Since µ S < and µ ( S, µ ) =, hen ( S, ). For µ µ > µ ( µ > ) ( µ = ) µ ( µ < ) S S 1 S S hus, for pairs ( µ, S) above µ =, µ >. For every ( µ, S) below µ =, µ <. Now consider S away from S =. ake he sae equaion S = E A gs. We need his in erms of µ, so consider µ = C ( A). Since C is sricly increasing ( C > ), i has an inverse 1 ha is also sricly increasing. Le ( C ) = h. hen 8.3

31 wih S µ = h ( µ ) <. ake a pair ( S, µ ) such ha S = S ( S, µ ) = E h( µ ) gs, SS (, µ ) = E h( µ ) gs =. Fix S = S and consider µ < µ. Since S µ < and SS (, µ ) =, hen 1 1 µ > µ, SS (, µ ) <. Graphically: (, ) >. For SS µ µ 1 µ ( S < ) µ µ S ( S > ) ( S = ) S herefore, for any ( µ, S) above S =, S <. For any ( µ, S ) below S =, S >. Combining our findings abou µ and S away from u = S = yields he following graph. 8.31

32 µ ( µ = ) ( µ >, S < ) ( µ >, S > ) s µ ( µ <, S < ) ( µ <, S > ) S s ( S = ) S he direcional arrows (like ) ell us he qualiaive movemen of ( µ, S) away from or oward he seady sae in each of he four regions of he diagram. Given some iniial sock of polluion, s S S, we can draw qualiaive conclusions abou he pah of ( µ, S) oward he seady sae. his will be paricularly revealing for he µ pah, because his will be he pah of he opimal ax on emissions. s In he following graph we assume ha S < S. 8.32

33 µ ( µ = ) (a) s µ (b) (c) ( S = ) S S s S I have drawn he pahs (a), (b) and (c), bu only pah (b) reaches he seady sae. Noe ha a ax policy ha originaes above µ s s µ () > µ as for pah (a) canno reach he seady sae. herefore he opimal iniial ax mus be below he seady sae ax. However, i canno be oo low like µ () for pah (c). Pah (b) converges o he seady sae. Noe he policy prescripion: se he iniial emissions ax s lower han he seady sae ax and increase i oward µ as ime goes by. his also implies ha opimal abaemen sars ou relaively low and increases as he ax is increased [use µ = C ( A) ]. Abaemen is higher in laer periods, because in presen value erms i is cheaper o push i off ino he fuure. Of course, doing so has o be balanced agains increased damage in earlier periods. Using he same phase-diagram we can draw he opposie conclusions if he iniial sock is greaer han he seady-sae sock. In his case he iniial ax is se higher han he seady sae ax and he ax is decreased oward he seady sae as ime goes by. Consequenly, abaemen sars ou relaively high and is gradually decreased as he sysem moves oward he seady sae. 8.33

34 Dynamic Programming Consider he following discree-ime dynamic opimizaion problem max gxu (,, ) { u } = x = + 1 subjec o f( x, u, ) x = x given, where g is he per period payoff, x is he sae variable, u is he conrol variable, and x+ 1 = f( x, u, ) is he discree-ime sae equaion. Define Vn( x-( n-1) ) = maximize g( x, u, ) { } = ( n 1) u = -( n-1) subjec o x = f( x, u, ), = - ( n-1),,. + 1 Vn() is he maximized value of he program wih n periods remaining from ( n 1) o given he sae a he ouse of hese n periods, x ( n 1). Aside: ime can be a lile confusing here. An illusraion migh help. Suppose ha = 4. Since he program sars a =, here are 5 periods under consideraion. Suppose we begin wih n = 3 periods o go, which implies ha we are beginning in = ( n 1) = = 2. he hree periods o go wih = 4 are he 2 nd, 3 rd, and 4 h periods. o specify Vn( x -( n-1) ) we may sar wih 1 period o go and work our way backward hrough ime. Wih one period o go, V ( x ) = max g( x, u, ). 1 u his is a simple saic opimizaion problem. Suppose ha he soluion is * 1( ) = (, (, ), ). V x g x u x u * ( x, ). hen Now consider wo periods o go ( n= 2, = 1). he choice of u 1 affecs no only he payoff in 1, bu also V ( ) 1 x hrough he sae equaion x = f( x 1, u 1, 1). herefore, from he perspecive of 1, he maximized payoff in is V1( f( x 1, u 1, 1)). he maximized value of he program wih 2 periods remaining is [ ] V ( x ) = max g( x, u, 1) + V ( f( x, u, 1)) u

35 he soluion is u 1( x 1, 1), which upon subsiuion back ino he curren and fuure payoff funcions yields V2( x 1). o solve his problem we only choose u 1( x 1), because u( x ) has already been deermined. Furhermore, his specificaion makes i clear ha he choice u 1 mus balance he conribuion o curren payoff, gx ( 1, u 1, 1), agains is affec on he fuure payoff, V ( f( x, u, 1)) Wih hree periods o go ( n= 3, = 2) we have 2 2 [ ] V ( x ) = max g( x, u, 2) + V ( x ) u s.. x = f( x, u, 2) Wih an arbirary n periods o go we have [ gx u V f x u ] = max (,, 2) + ( (,, 2)). u V ( x ) = max g( x, u, ( n 1)) + V ( x ) n ( n 1) ( n 1) ( n 1) ( n 2) ( n 2) u ( n 1) s.. x = f( x, u, ( n 1)). ( n 2) ( n 1) ( n 1) Le s = ( n 1) o simplify he subscrips. hen, [ ] [ ] V ( x ) = max g( x, u, s) + V ( x ) = max g( x, u, s) + V ( f( x, u, s)). n s s s s + 1 s + 1 s s s + 1 u u s s s s.. x = f( x, u, s) s+ 1 s s s his is he fundamenal relaion of dynamic programming. I is called Bellman s equaion or he Dynamic Programming Equaion. Suppose ha gx ( s, us, s) + Vs+ 1( f( xs, us, s)) is sricly concave and differeniable in u s, and ha u s is unconsrained. hen, u s is deermined by he firs-order condiion V x g + s+ 1 s+ 1 = us xs+ 1 us. Noice again ha he choice of u s in any period s balances he conribuion of u o curren payoff, g us, agains is affec on he maximized value fuure payoffs, V s + 1, hough is affec on he sae, x s

36 Example Suppose we wish o solve 3 2 max (1 x (.1) u ) { u } = subjec o x+ 1 = x + u x =, u. Noe ha applicaion of he conrol u in a ime period is cosly, bu i adds o he sock x from which value is obained in he fuure. We may hink of u as an invesmen in a capial sock x ha generaes a reurn of 1 per uni. he value funcion wih one period o go is V ( x ) = max [1 x (.1) u ] u Since 1 x3 (.1) u3 is decreasing in u3, u 3 = and V1( x3) = 1 x3. Wih wo periods o go, which implies V ( x ) = max 1 x (.1) u + V ( x ) u 2 s.. V( x ) = 1x x = x + u 3 2 2, V ( x ) = max 1 x (.1) u + 1( x + u ) u 2 Noe ha he funcion in brackes is sricly concave in u 2. herefore u 2 is he soluion o (.2) u2 + 1 =, implying u 2 = 5. Subsiuing his back ino V2( x 2) yields 2 V2( x2) = 1 x2 (.1)(5) + 1( x2 + 5) = 2x Wih hree periods o go: which implies V ( x ) = max 1 x (.1) u + V ( x ) u 1 s.. V ( x ) = 2x x = x + u,

37 V ( x ) = max 1 x (.1) u + 2( x + u ) u 1 he maximizer 1 u is deermined from ( ) u =, implying u 1 = 1, and In he firs period ( = ), V ( x ) = max 1 x (.1)(1) + 2( x + 1) + 25 = 3x u1 2 V4( x) = max 1 x (.1) u V3( x1) u + s.. V3( x1) = 3x x1 = x + u x =. Combining he hree consrains yields x1 = uand V3( x1) = 3u Subsiuing hese ino V4( x ) gives us V ( x ) = max (.1) u + 3u u he firs-order condiion for deermining u is (.2) u + 3 =, implying u = 15. he opimal pahs of u and x * * * are: x+ 1 = x + u, x =. u * * * x+ 1 = x + u, x = 15 x = x + u = u ( 1 ) ( x2 = x1+ u1) 3 3 ( x = x + u )

38 Exercises [1] Consider he following problem wih conrol u [,1], and x : maximize (1 uxd ) subjec o x = ux x() = x fixed, x( ) free. [a] Se up he Hamilonian and sae all of he necessary condiions. Noe ha he derivaive of he Hamilonian wih respec o u is independen of u. [b] Le λ denoe he adjoin variable for his problem. Show: i) If λ > 1, hen u = 1 and λ <. ii) If λ < 1, hen u = and λ = 1. ii) If x ( ) >, hen u ( ) =. iv) If x ( ) =, hen u ( ) is indeerminan. [2] Consider he problem wih conrol u >, and sae x> : 2 maximize (1 u ) subjec o x = ux x() = x xd fixed, x( ) free. [a] Se up he Hamilonian and sae all of he necessary condiions. [b] Show ha a sufficien condiion is saisfied. [c] Show ha λ <, u <, and x >. [3] From a sock of capial k, oupu is produced by sociey a rae y(k), wih y > and y <. Oupu can be consumed a rae c, yielding uiliy u(c), or i can be invesed a rae i. he capial sock depreciaes a a consan rae b. Sociey s problem is o maximize he presen value of he flow of uiliy over a fixed planning period by choosing he pah of consumpion. Specify his problem in he opimal conrol forma, check o see ha a sufficien condiion is saisfied, and derive he necessary condiions. 8.38

39 [4] he owner of a nonrenewable resource in fixed supply wans o plan exracion of he resource o maximize he presen value of he sream of profis. Le x denoe he size of he reserve in, wih x() = x and x ( ). Le y denoe unis of he resource exraced, which sell for a price p ha is expeced o remain consan over [, ]. Exracion coss are c(y), wih c > and c >, for y >. o make his an ineresing problem, suppose ha c () < p. he owner uses a consan ineres rae r. [a] Specify his problem in he opimal conrol forma, check o see ha a sufficien condiion is saisfied, and derive he necessary condiions. [b] Inerpre he necessary condiions, paying paricular aenion o he inerpreaion of he adjoin variable. Hin: he adjoin variable in his conex is ofen referred o as he marginal user cos of he resource. [c] Show: i) price minus marginal cos rises a he rae of ineres. (Hoelling) ii) exracion declines over ime. [d] Now suppose ha he cos of exracion is given by c(y, x), wih c x (y, x) <. Show in his case ha price minus marginal cos rises more slowly han he rae of ineres. [5] A social planner wans o plan exracion y of a nonrenewable resource o maximize he presen value of social welfare over a fixed planning period [, ]. Social welfare from exracion (ne of exracion coss) is b(y), wih b > and b <, for y >. Also assume ha b(y) is a quadraic so ha b =. he sock of he resource a an insan in ime is x, wih iniial sock x() = x. here is no exploraion or discovery of new deposis. herefore, he equaion of moion for he sock is simply x = y. he social planner uses of consan discoun rae r in her calculaions. [a] Formulae he planner s problem as an opimal conrol problem, form he curren-value Hamilonian, and derive he necessary condiions. [b] Provide complee economic inerpreaions of he necessary condiions. [c] Show ha he curren-value shadow price of he resource is declining hrough ime, and ha exracion is declining and sricly concave over ime as long as r >. [d] Suppose ha r =. Show ha he curren-value shadow price of he resource and exracion are consan hrough ime. [e] Provide a graphical comparison of he exracion pahs wih r > and r =. Is exracion in early periods greaer han, less han, or he same when r =? Is exracion in laer periods greaer han, less han, or he same when r =? Provide an explanaion and inerpreaion of your findings. 8.39

40 [6] An economy produces a consumpion good c wih energy e according o a sricly increasing and concave producion funcion c(e). Energy is produced from a non-renewable resource. In energy-equivalen unis, he sock of he resource is s and he iniial sock is s. Unforunaely, energy use produces polluion p according o a sricly decreasing and convex producion funcion p(e). he polluan is a flow ha does no accumulae over ime. A social planner wans o choose a pah of energy consumpion o maximize he presen value of he flow of social welfare over a specified planning period [, ]. he social welfare funcion is U(c(e), p(e)), which has he following characerisics: U c >, U cc <, U p <, U pp >, and U cp =. he social planner uses a consan ineres rae r in is calculaions. [a] Use he mehod of opimal conrol o characerize he opimal pah of energy consumpion. Be sure o include complee saemens of he planner s objecive and he necessary condiions for a soluion o he problem. [b] Provide economic inerpreaions of he necessary condiions you derived in [a]. Pay paricular aenion o he inerpreaion of he adjoin variable. [7] A sock of capial x a an insan in ime allows a firm o earn revenue rx ( ), and he firm can inves u in new capial a cos c(u). Assume r >, r <, c >, and c >. he porion of he sock of capial ha depreciaes a any poin in ime is bx, where b is a posiive consan. he firm discouns fuure profi a rae ρ, and wishes o plan invesmen o maximize he presen value of he firm s profis over some planning period. [a] Sae he firm s opimal conrol problem and check o see ha a sufficien condiion is saisfied. [b] Sae he curren-value Hamilonian and he necessary condiions. For he purposes of his problem you may skip he ransversaliy condiion. [c] Show ha he curren marginal value of he capial sock is increasing (decreasing) over ime if he rae of new invesmen is increasing (decreasing) over ime. [d] Suppose ha he firm has an infinie planning horizon. Characerize he seady-sae values of he curren-value shadow price, capial sock, and rae of invesmen. Provide economic inerpreaions of he condiions ha characerize he seady-sae. [e] Consruc a phase-diagram o illusrae possible pahs o he seady sae. 8.4