Applied Intertemporal Optimization

. Applied Ineremporal Opimizaion Klaus Wälde Universiy of Mainz CESifo, Universiy of Brisol, UCL Louvain la Neuve www.waelde.com

These lecure noes can freely be downloaded from www.waelde.com/aio. A prin version can also be bough a his sie. @BOOK{Waelde10, } auhor = {Klaus Wälde}, year = 2010, ile = {Applied Ineremporal Opimizaion}, publisher = {Mainz Universiy Guenberg Press}, address = {Available a www.waelde.com/aio} Guenberg Press Copyrigh (c) 2010 by Klaus Wälde www.waelde.com ISBN 978-3-00-032428-4

1 Acknowledgmens This book is he resul of many lecures given a various insiuions, including he Bavarian Graduae Program in Economics, he Universiies of Dormund, Dresden, Frankfur, Glasgow, Louvain-la-Neuve, Mainz, Munich and Würzburg and he European Commission in Brussels. I would herefore rs like o hank he sudens for heir quesions and discussions. These conversaions revealed he mos imporan aspecs in undersanding model building and in formulaing maximizaion problems. I also pro ed from many discussions wih economiss and mahemaicians from many oher places. The high download gures a repec.org show ha he book is also widely used on an inernaional level. I especially would like o hank Chrisian Bayer and Ken Sennewald for many insighs ino he more suble issues of sochasic coninuous ime processes. I hope I succeeded in incorporaing hese insighs ino his book. I would also like o hank MacKichan for heir repeaed, quick and very useful suppor on ypeseing issues in Scieni c Word.

2 Overview The basic srucure of his book is simple o undersand. I covers opimizaion mehods and applicaions in discree ime and in coninuous ime, boh in worlds wih cerainy and worlds wih uncerainy. discree ime coninuous ime deerminisic seup Par I Par II sochasic seup Par III Par IV Table 0.0.1 Basic srucure of his book Soluion mehods Pars and chapers Subsiuion Lagrange Ch. 1 Inroducion Par I Deerminisic models in discree ime Ch. 2 Two-period models and di erence equaions 2.2.1 2.3 Ch. 3 Muli-period models 3.8.2 3.1.2, 3.7 Par II Deerminisic models in coninuous ime Ch. 4 Di erenial equaions Ch. 5 Finie and in nie horizon models 5.2.2, 5.6.1 Ch. 6 In nie horizon models again Par III Sochasic models in discree ime Ch. 7 Sochasic di erence equaions and momens Ch. 8 Two-period models 8.1.4, 8.2 Ch. 9 Muli-period models 9.5 9.4 Par IV Sochasic models in coninuous ime Ch. 10 Sochasic di erenial equaions, rules for di erenials and momens Ch. 11 In nie horizon models Ch. 12 Noaion and variables, references and index Table 0.0.2 Deailed srucure of his book

Each of hese four pars is divided ino chapers. As a quick reference, he able below provides an overview of where o nd he four soluion mehods for maximizaion problems used in his book. They are he subsiuion mehod, Lagrange approach, opimal conrol heory and dynamic programming. Whenever we employ hem, we refer o hem as Solving by and hen eiher subsiuion, he Lagrangian, opimal conrol or dynamic programming. As di erences and comparaive advanages of mehods can mos easily be undersood when applied o he same problem, his able also shows he mos frequenly used examples. Be aware ha hese are no he only examples used in his book. Ineremporal pro maximizaion of rms, capial asse pricing, naural volailiy, maching models of he labour marke, opimal R&D expendiure and many oher applicaions can be found as well. For a more deailed overview, see he index a he end of his book. 3 Applicaions (selecion) opimal Dynamic Uiliy Cenral General Budge conrol programming maximizaion planner equilibrium consrains 2.1, 2.2 2.3.2 2.4 2.5.5 3.3 3.1, 3.4, 3.8 3.2.3, 3.7 3.6 5 5.1, 5.3, 5.6.1 5.6.3 6 6.1 6.4 4.4.2 8.1.4, 8.2 8.1 9.1, 9.2, 9.3 9.1, 9.4 9.2 11 11.1, 11.3 11.5.1 10.3.2 Table 0.0.2 Deailed srucure of his book (coninued)

4

Conens 1 Inroducion 1 I Deerminisic models in discree ime 7 2 Two-period models and di erence equaions 11 2.1 Ineremporal uiliy maximizaion...................... 11 2.1.1 The seup................................ 11 2.1.2 Solving by he Lagrangian....................... 13 2.2 Examples.................................... 14 2.2.1 Opimal consumpion.......................... 14 2.2.2 Opimal consumpion wih prices................... 16 2.2.3 Some useful de niions wih applicaions............... 17 2.3 The idea behind he Lagrangian........................ 21 2.3.1 Where he Lagrangian comes from I.................. 22 2.3.2 Shadow prices.............................. 24 2.4 An overlapping generaions model....................... 26 2.4.1 Technologies............................... 27 2.4.2 Households............................... 28 2.4.3 Goods marke equilibrium and accumulaion ideniy........ 28 2.4.4 The reduced form............................ 29 2.4.5 Properies of he reduced form..................... 30 2.5 More on di erence equaions.......................... 32 2.5.1 Two useful proofs............................ 32 2.5.2 A simple di erence equaion...................... 32 2.5.3 A slighly less bu sill simple di erence equaion.......... 35 2.5.4 Fix poins and sabiliy......................... 36 2.5.5 An example: Deriving a budge consrain.............. 37 2.6 Furher reading and exercises......................... 39 3 Muli-period models 45 3.1 Ineremporal uiliy maximizaion...................... 45 3.1.1 The seup................................ 45 5

6 Conens 3.1.2 Solving by he Lagrangian....................... 46 3.2 The envelope heorem............................. 46 3.2.1 The heorem............................... 47 3.2.2 Illusraion............................... 47 3.2.3 An example............................... 48 3.3 Solving by dynamic programming....................... 49 3.3.1 The seup................................ 49 3.3.2 Three dynamic programming seps.................. 50 3.4 Examples.................................... 52 3.4.1 Ineremporal uiliy maximizaion wih a CES uiliy funcion... 52 3.4.2 Wha is a sae variable?........................ 55 3.4.3 Opimal R&D e or.......................... 57 3.5 On budge consrains............................. 59 3.5.1 From ineremporal o dynamic.................... 60 3.5.2 From dynamic o ineremporal.................... 60 3.5.3 Two versions of dynamic budge consrains............. 62 3.6 A decenralized general equilibrium analysis................. 62 3.6.1 Technologies............................... 62 3.6.2 Firms.................................. 63 3.6.3 Households............................... 63 3.6.4 Aggregaion and reduced form..................... 64 3.6.5 Seady sae and ransiional dynamics................ 65 3.7 A cenral planner................................ 66 3.7.1 Opimal facor allocaion........................ 66 3.7.2 Where he Lagrangian comes from II................. 67 3.8 Growh of family size.............................. 68 3.8.1 The seup................................ 68 3.8.2 Solving by subsiuion......................... 69 3.8.3 Solving by he Lagrangian....................... 69 3.9 Furher reading and exercises......................... 70 II Deerminisic models in coninuous ime 75 4 Di erenial equaions 79 4.1 Some de niions and heorems......................... 79 4.1.1 De niions................................ 79 4.1.2 Two heorems.............................. 80 4.2 Analyzing ODEs hrough phase diagrams................... 81 4.2.1 One-dimensional sysems........................ 81 4.2.2 Two-dimensional sysems I - An example............... 84 4.2.3 Two-dimensional sysems II - The general case............ 86 4.2.4 Types of phase diagrams and xpoins................ 90

Conens 7 4.2.5 Mulidimensional sysems....................... 92 4.3 Linear di erenial equaions.......................... 92 4.3.1 Rules on derivaives.......................... 92 4.3.2 Forward and backward soluions of a linear di erenial equaion.. 94 4.3.3 Di erenial equaions as inegral equaions.............. 97 4.4 Examples.................................... 97 4.4.1 Backward soluion: A growh model.................. 97 4.4.2 Forward soluion: Budge consrains................. 98 4.4.3 Forward soluion again: capial markes and uiliy......... 100 4.5 Linear di erenial equaion sysems...................... 102 4.6 Furher reading and exercises......................... 102 5 Finie and in nie horizon models 107 5.1 Ineremporal uiliy maximizaion - an inroducory example....... 107 5.1.1 The seup................................ 107 5.1.2 Solving by opimal conrol....................... 108 5.2 Deriving laws of moion............................ 109 5.2.1 The seup................................ 109 5.2.2 Solving by he Lagrangian....................... 109 5.2.3 Hamilonians as a shorcu....................... 111 5.3 The in nie horizon............................... 111 5.3.1 Solving by opimal conrol....................... 111 5.3.2 The boundedness condiion...................... 113 5.4 Boundary condiions and su cien condiions................ 113 5.4.1 Free value of he sae variable a he endpoin........... 114 5.4.2 Fixed value of he sae variable a he endpoin........... 114 5.4.3 The ransversaliy condiion...................... 114 5.4.4 Su cien condiions.......................... 115 5.5 Illusraing boundary condiions........................ 116 5.5.1 A rm wih adjusmen coss..................... 116 5.5.2 Free value a he end poin....................... 119 5.5.3 Fixed value a he end poin...................... 119 5.5.4 In nie horizon and ransversaliy condiion............. 120 5.6 Furher examples................................ 120 5.6.1 In nie horizon - opimal consumpion pahs............. 120 5.6.2 Necessary condiions, soluions and sae variables.......... 125 5.6.3 Opimal growh - he cenral planner and capial accumulaion... 126 5.6.4 The maching approach o unemploymen.............. 129 5.7 The presen value Hamilonian......................... 132 5.7.1 Problems wihou (or wih implici) discouning........... 132 5.7.2 Deriving laws of moion........................ 133 5.7.3 The link beween CV and PV..................... 134

8 Conens 5.8 Furher reading and exercises......................... 135 6 In nie horizon again 141 6.1 Ineremporal uiliy maximizaion...................... 141 6.1.1 The seup................................ 141 6.1.2 Solving by dynamic programming................... 141 6.2 Comparing dynamic programming o Hamilonians............. 145 6.3 Dynamic programming wih wo sae variables............... 145 6.4 Nominal and real ineres raes and in aion................. 148 6.4.1 Firms, he cenral bank and he governmen............. 148 6.4.2 Households............................... 149 6.4.3 Equilibrium............................... 149 6.5 Furher reading and exercises......................... 152 6.6 Looking back.................................. 155 III Sochasic models in discree ime 157 7 Sochasic di erence equaions and momens 161 7.1 Basics on random variables........................... 161 7.1.1 Some conceps.............................. 161 7.1.2 An illusraion............................. 162 7.2 Examples for random variables........................ 162 7.2.1 Discree random variables....................... 163 7.2.2 Coninuous random variables..................... 163 7.2.3 Higher-dimensional random variables................. 164 7.3 Expeced values, variances, covariances and all ha............. 164 7.3.1 De niions................................ 164 7.3.2 Some properies of random variables................. 165 7.3.3 Funcions on random variables..................... 166 7.4 Examples of sochasic di erence equaions.................. 168 7.4.1 A rs example............................. 168 7.4.2 A more general case.......................... 172 8 Two-period models 175 8.1 An overlapping generaions model....................... 175 8.1.1 Technology............................... 175 8.1.2 Timing.................................. 175 8.1.3 Firms.................................. 176 8.1.4 Ineremporal uiliy maximizaion.................. 176 8.1.5 Aggregaion and he reduced form for he CD case......... 179 8.1.6 Some analyical resuls......................... 180 8.1.7 CRRA and CARA uiliy funcions.................. 182

Conens 9 8.2 Risk-averse and risk-neural households.................... 183 8.3 Pricing of coningen claims and asses.................... 187 8.3.1 The value of an asse.......................... 187 8.3.2 The value of a coningen claim.................... 188 8.3.3 Risk-neural valuaion......................... 188 8.4 Naural volailiy I............................... 189 8.4.1 The basic idea.............................. 189 8.4.2 A simple sochasic model....................... 190 8.4.3 Equilibrium............................... 192 8.5 Furher reading and exercises......................... 193 9 Muli-period models 197 9.1 Ineremporal uiliy maximizaion...................... 197 9.1.1 The seup wih a general budge consrain.............. 197 9.1.2 Solving by dynamic programming................... 198 9.1.3 The seup wih a household budge consrain............ 199 9.1.4 Solving by dynamic programming................... 200 9.2 A cenral planner................................ 201 9.3 Asse pricing in a one-asse economy..................... 202 9.3.1 The model................................ 203 9.3.2 Opimal behaviour........................... 203 9.3.3 The pricing relaionship........................ 204 9.3.4 More real resuls............................ 205 9.4 Endogenous labour supply........................... 207 9.5 Solving by subsiuion............................. 209 9.5.1 Ineremporal uiliy maximizaion.................. 209 9.5.2 Capial asse pricing.......................... 210 9.5.3 Sicky prices............................... 212 9.5.4 Opimal employmen wih adjusmen coss............. 213 9.6 An explici ime pah for a boundary condiion............... 215 9.7 Furher reading and exercises......................... 216 IV Sochasic models in coninuous ime 221 10 SDEs, di erenials and momens 225 10.1 Sochasic di erenial equaions (SDEs).................... 225 10.1.1 Sochasic processes.......................... 225 10.1.2 Sochasic di erenial equaions.................... 228 10.1.3 The inegral represenaion of sochasic di erenial equaions... 232 10.2 Di erenials of sochasic processes...................... 232 10.2.1 Why all his?.............................. 233 10.2.2 Compuing di erenials for Brownian moion............. 233

10 Conens 10.2.3 Compuing di erenials for Poisson processes............. 236 10.2.4 Brownian moion and a Poisson process................ 238 10.3 Applicaions................................... 239 10.3.1 Opion pricing.............................. 239 10.3.2 Deriving a budge consrain...................... 240 10.4 Solving sochasic di erenial equaions.................... 242 10.4.1 Some examples for Brownian moion................. 242 10.4.2 A general soluion for Brownian moions............... 245 10.4.3 Di erenial equaions wih Poisson processes............. 247 10.5 Expecaion values............................... 249 10.5.1 The idea................................. 249 10.5.2 Simple resuls.............................. 251 10.5.3 Maringales............................... 254 10.5.4 A more general approach o compuing momens.......... 255 10.6 Furher reading and exercises......................... 260 11 In nie horizon models 267 11.1 Ineremporal uiliy maximizaion under Poisson uncerainy....... 267 11.1.1 The seup................................ 267 11.1.2 Solving by dynamic programming................... 268 11.1.3 The Keynes-Ramsey rule........................ 270 11.1.4 Opimal consumpion and porfolio choice.............. 272 11.1.5 Oher ways o deermine ~c....................... 275 11.1.6 Expeced growh............................ 277 11.2 Maching on he labour marke: where value funcions come from..... 278 11.2.1 A household............................... 278 11.2.2 The Bellman equaion and value funcions.............. 279 11.3 Ineremporal uiliy maximizaion under Brownian moion......... 280 11.3.1 The seup................................ 280 11.3.2 Solving by dynamic programming................... 281 11.3.3 The Keynes-Ramsey rule........................ 282 11.4 Capial asse pricing.............................. 283 11.4.1 The seup................................ 283 11.4.2 Opimal behaviour........................... 284 11.4.3 Capial asse pricing.......................... 286 11.5 Naural volailiy II............................... 286 11.5.1 An real business cycle model...................... 286 11.5.2 A naural volailiy model....................... 290 11.5.3 A numerical approach......................... 290 11.6 Furher reading and exercises......................... 291 12 Miscellanea, references and index 299

Chaper 1 Inroducion This book provides a oolbox for solving dynamic maximizaion problems and for working wih heir soluions in economic models. Maximizing some objecive funcion is cenral o Economics, i can be undersood as one of he de ning axioms of Economics. When i comes o dynamic maximizaion problems, hey can be formulaed in discree or coninuous ime, under cerainy or uncerainy. Various maximizaion mehods will be used, ranging from he subsiuion mehod, via he Lagrangian and opimal conrol o dynamic programming using he Bellman equaion. Dynamic programming will be used for all environmens, discree, coninuous, cerain and uncerain, he Lagrangian for mos of hem. The subsiuion mehod is also very useful in discree ime seups. The opimal conrol heory, employing he Hamilonian, is used only for deerminisic coninuous ime seups. An overview was given in g. 0.0.2 on he previous pages. The general philosophy behind he syle of his book says ha wha maers is an easy and fas derivaion of resuls. This implies ha a lo of emphasis will be pu on examples and applicaions of mehods. While he idea behind he general mehods is someimes illusraed, he focus is clearly on providing a soluion mehod and examples of applicaions quickly and easily wih as lile formal background as possible. This is why he book is called applied ineremporal opimizaion. Conens of pars I o IV This book consiss of four pars. In his rs par of he book, we will ge o know he simples and herefore maybe he mos useful srucures o hink abou changes over ime, o hink abou dynamics. Par I deals wih discree ime models under cerainy. The rs chaper inroduces he simples possible ineremporal problem, a wo-period problem. I is solved in a general way and for many funcional forms. The mehods used are he Lagrangian and simple subsiuion. Various conceps like he ime preference rae and he ineremporal elasiciies of subsiuion are inroduced here as well, as hey are widely used in he lieraure and are used frequenly hroughou his book. For hose who wan o undersand he background of he Lagrangian, a chaper is included ha shows he link beween Lagrangians and solving by subsiuion. This will also give us he 1

2 Chaper 1. Inroducion opporuniy o explain he concep of shadow prices as hey play an imporan role e.g. when using Hamilonians or dynamic programming. The wo-period opimal consumpion seup will hen be pu ino a decenralized general equilibrium seup. This allows us o undersand general equilibrium srucures in general while, a he same ime, we ge o know he sandard overlapping generaions (OLG) general equilibrium model. This is one of he mos widely used dynamic models in Economics. Chaper 2 concludes by reviewing some aspecs of di erence equaions. Chaper 3 hen covers in nie horizon models. We solve a ypical maximizaion problem rs by using he Lagrangian again and hen by dynamic programming. As dynamic programming regularly uses he envelope heorem, his heorem is rs reviewed in a simple saic seup. Examples for in nie horizon problems, a general equilibrium analysis of a decenralized economy, a ypical cenral planner problem and an analysis of how o rea family or populaion growh in opimizaion problems hen complee his chaper. To complee he range of maximizaion mehods used in his book, he presenaion of hese examples will also use he mehod of solving by insering. Par II covers coninuous ime models under cerainy. Chaper 4 rs looks a differenial equaions as hey are he basis of he descripion and soluion of maximizaion problems in coninuous ime. Firs, some useful de niions and heorems are provided. Second, di erenial equaions and di erenial equaion sysems are analyzed qualiaively by he so-called phase-diagram analysis. This simple mehod is exremely useful for undersanding di erenial equaions per se and also for laer purposes for undersanding qualiaive properies of soluions o maximizaion problems and properies of whole economies. Linear di erenial equaions and heir economic applicaions are hen nally analyzed before some words are spen on linear di erenial equaion sysems. Chaper 5 presens a new mehod for solving maximizaion problems - he Hamilonian. As we are now in coninuous ime, wo-period models do no exis. A disincion will be drawn, however, beween nie and in nie horizon models. In pracice, his disincion is no very imporan as, as we will see, opimaliy condiions are very similar for nie and in nie maximizaion problems. Afer an inroducory example on maximizaion in coninuous ime by using he Hamilonian, he simple link beween Hamilonians and he Lagrangian is shown. The soluion o maximizaion problems in coninuous ime will consis of one or several di erenial equaions. As a unique soluion o di erenial equaions requires boundary condiions, we will show how boundary condiions are relaed o he ype of maximizaion problem analyzed. The boundary condiions di er signi canly beween nie and in nie horizon models. For he nie horizon models, here are iniial or erminal condiions. For he in nie horizon models, we will ge o know he ransversaliy condiion and oher relaed condiions like he No-Ponzi-game condiion. Many examples and a comparison beween he presen-value and he curren-value Hamilonian conclude his chaper. Chaper 6 solves he same kind of problems as chaper 5, bu i uses he mehod of dynamic programming. The reason for doing his is o simplify undersanding of dynamic programming in sochasic seups in Par IV. Various aspecs speci c o he use

of dynamic programming in coninuous ime, e.g. he srucure of he Bellman equaion, can already be reaed here under cerainy. This chaper will also provide a comparison beween he Hamilonian and dynamic programming and look a a maximizaion problem wih wo sae variables. An example from moneary economics on real and nominal ineres raes concludes he chaper. In par III, he world becomes sochasic. Pars I and II provided many opimizaion mehods for deerminisic seups, boh in discree and coninuous ime. All economic quesions ha were analyzed were viewed as su cienly deerminisic. If here was any uncerainy in he seup of he problem, we simply ignored i or argued ha i is of no imporance for undersanding he basic properies and relaionships of he economic quesion. This is a good approach o many economic quesions. Generally speaking, however, real life has few cerain componens. Deah is cerain, bu when? Taxes are cerain, bu how high are hey? We know ha we all exis - bu don ask philosophers. Par III (and par IV laer) will ake uncerainy in life seriously and incorporae i explicily in he analysis of economic problems. We follow he same disincion as in par I and II - we rs analyse he e ecs of uncerainy on economic behaviour in discree ime seups in par III and hen move o coninuous ime seups in par IV. Chaper 7 and 8 are an exended version of chaper 2. As we are in a sochasic world, however, chaper 7 will rs spend some ime reviewing some basics of random variables, heir momens and disribuions. Chaper 7 also looks a di erence equaions. As hey are now sochasic, hey allow us o undersand how disribuions change over ime and how a disribuion converges - in he example we look a - o a limiing disribuion. The limiing disribuion is he sochasic equivalen o a x poin or seady sae in deerminisic seups. Chaper 8 looks a maximizaion problems in his sochasic framework and focuses on he simples case of wo-period models. A general equilibrium analysis wih an overlapping generaions seup will allow us o look a he new aspecs inroduced by uncerainy for an ineremporal consumpion and saving problem. We will also see how one can easily undersand dynamic behaviour of various variables and derive properies of longrun disribuions in general equilibrium by graphical analysis. One can for example easily obain he range of he long-run disribuion for capial, oupu and consumpion. This increases inuiive undersanding of he processes a hand remendously and helps a lo as a guide o numerical analysis. Furher examples include borrowing and lending beween risk-averse and risk-neural households, he pricing of asses in a sochasic world and a rs look a naural volailiy, a view of business cycles which sresses he link beween joinly endogenously deermined shor-run ucuaions and long-run growh. Chaper 9 is hen similar o chaper 3 and looks a muli-period, i.e. in nie horizon, problems. As in each chaper, we sar wih he classic ineremporal uiliy maximizaion problem. We hen move on o various imporan applicaions. The rs is a cenral planner sochasic growh model, he second is capial asse pricing in general equilibrium and how i relaes o uiliy maximizaion. We coninue wih endogenous labour supply and he 3

4 Chaper 1. Inroducion maching model of unemploymen. The nex secion hen covers how many maximizaion problems can be solved wihou using dynamic programming or he Lagrangian. In fac, many problems can be solved simply by insering, despie uncerainy. This will be illusraed wih many furher applicaions. A nal secion on nie horizons concludes. Par IV is he nal par of his book and, logically, analyzes coninuous ime models under uncerainy. The choice beween working in discree or coninuous ime is parly driven by previous choices: If he lieraure is mainly in discree ime, sudens will nd i helpful o work in discree ime as well. The use of discree ime mehods seem o hold for macroeconomics, a leas when i comes o he analysis of business cycles. On he oher hand, when we alk abou economic growh, labour marke analyses and nance, coninuous ime mehods are very prominen. Whaever he radiion in he lieraure, coninuous ime models have he huge advanage ha hey are analyically generally more racable, once some iniial invesmen ino new mehods has been digesed. As an example, some papers in he lieraure have shown ha coninuous ime models wih uncerainy can be analyzed in simple phase diagrams as in deerminisic coninuous ime seups. See ch. 10.6 and ch. 11.6 on furher reading for references from many elds. To faciliae access o he magical world of coninuous ime uncerainy, par IV presens he ools required o work wih uncerainy in coninuous ime models. I is probably he mos innovaive par of his book as many resuls from recen research ow direcly ino i. This par also mos srongly incorporaes he cenral philosophy behind wriing his book: There will be hardly any discussion of formal mahemaical aspecs like probabiliy spaces, measurabiliy and he like. While some will argue ha one can no work wih coninuous ime uncerainy wihou having sudied mahemaics, his chaper and he many applicaions in he lieraure prove he opposie. The objecive here is o clearly make he ools for coninuous ime uncerainy available in a language ha is accessible for anyone wih an ineres in hese ools and some feeling for dynamic models and random variables. The chapers on furher reading will provide links o he more mahemaical lieraure. Maybe his is also a good poin for he auhor of his book o hank all he mahemaicians who helped him gain access o his magical world. I hope hey will forgive me for beraying heir secres o hose who, maybe in heir view, were no appropriaely iniiaed. Chaper 10 provides he background for opimizaion problems. As in par II where we rs looked a di erenial equaions before working wih Hamilonians, here we will rs look a sochasic di erenial equaions. Afer some basics, he mos ineresing aspec of working wih uncerainy in coninuous ime follows: Io s lemma and, more generally, change-of-variable formulas for compuing di erenials will be presened. As an applicaion of Io s lemma, we will ge o know one of he mos famous resuls in Economics - he Black-Scholes formula. This chaper also presens mehods for how o solve sochasic di erenial equaions or how o verify soluions and compue momens of random variables described by a sochasic process. Chaper 11 hen looks once more a maximizaion problems. We will ge o know

he classic ineremporal uiliy maximizaion problem boh for Poisson uncerainy and for Brownian moion. The chaper also shows he link beween Poisson processes and maching models of he labour marke. This is very useful for working wih exensions of he simple maching model ha allows for savings. Capial asse pricing and naural volailiy conclude he chaper. From simple o complex seups Given ha cerain maximizaion problems are solved many imes - e.g. uiliy maximizaion of a household rs under cerainy in discree and coninuous ime and hen under uncerainy in discree and coninuous ime - and using many mehods, he seps how o compue soluions can be easily undersood: Firs, he discree deerminisic wo-period approach provides he basic inuiion or feeling for a soluion. Nex, in nie horizon problems add one dimension of complexiy by aking away he simple boundary condiion of nie horizon models. In a hird sep, uncerainy adds expecaions operaors and so on. By gradually working hrough increasing seps of sophisicaion and by linking back o simple bu srucurally idenical examples, inuiion for he complex seups is buil up as much as possible. This approach hen allows us o nally undersand he beauy of e.g. Keynes-Ramsey rules in coninuous ime under Poisson uncerainy or Brownian moion. Even more moivaion for his book Why each a course based on his book? Is i no boring o go hrough all hese mehods? In a way, he answer is yes. We all wan o undersand cerain empirical regulariies or undersand poenial fundamenal relaionships and make exciing new empirically esable predicions. In doing so, we also all need o undersand exising work and evenually presen our own ideas. I is probably much more boring o be hindered in undersanding exising work and be almos cerainly excluded from presening our own ideas if we always spend a long ime rying o undersand how cerain resuls were derived. How did his auhor ge from equaion (1) and (2) o equaion (3)? The major advanage of economic analysis over oher social sciences is is srong foundaion in formal models. These models allow us o discuss quesions much more precisely as expressions like marginal uiliy, ime preference rae or Keynes-Ramsey rule reveal a lo of informaion in a very shor ime. I is herefore exremely useful o rs spend some ime in geing o know hese mehods and hen o ry o do wha Economics is really abou: undersand he real world. Bu, before we really sar, here is also a second reason - a leas for some economiss - o go hrough all hese mehods: They conain a cerain ype of ruh. A proof is rue or false. The derivaion of some opimal behaviour is rue or false. A predicion of general equilibrium behaviour of an economy is ruh. Unforunaely, i is only ruh in an analyical sense, bu his is a leas some ype of ruh. Beer han none. 5

6 Chaper 1. Inroducion The audience for his book Before his book came ou, i had been esed for a leas en years in many courses. There are wo ypical courses which were based on his book. A hird year Bachelor course (for ambiious sudens) can be based on pars I and II, i.e. on maximizaion and applicaions in discree and coninuous ime under cerainy. Such a course ypically ook 14 lecures of 90 minues each plus he same number of uorials. I is also possible o presen he maerial also in 14 lecures of 90 minues each plus only 7 uorials. Presening he maerial wihou uorials requires a lo of energy from he sudens o go hrough he problem ses on heir own. One can, however, discuss some seleced problem ses during lecures. The oher ypical course which was based on his book is a rs-year PhD course. I would review a few chapers of par I and par II (especially he chapers on dynamic programming) and cover in full he sochasic maerial of par III and par IV. I also requires foureen 90 minue sessions and exercise classes help even more, given he more complex maerial. Bu he same ype of arrangemens as discussed for he Bachelor course did work as well. Of course, any oher combinaion is feasible. From my own experience, eaching par I and II in a hird year Bachelor course allows eaching of par III and IV a he Maser level. Of course, Maser courses can be based on any pars of his book, rs-year PhD courses can sar wih par I and II and second-year eld courses can use maerial of par III or IV. This all depends on he ambiion of he programme, he inenion of he lecurer and he needs of he sudens. Apar from being used in classroom, many PhD sudens and advanced Bachelor or Maser sudens have used various pars of previous versions of his ex for sudying on heir own. Given he deailed sep-by-sep approach o problems, i urned ou ha i was very useful for undersanding he basic srucure of, say, a maximizaion problem. Once his basic srucure was undersood, many exensions o more complex problems were obained - some of which hen even found heir way ino his book.

Par I Deerminisic models in discree ime 7

This book consiss of four pars. In his rs par of he book, we will ge o know he simples and herefore maybe he mos useful srucures o hink abou changes over ime, o hink abou dynamics. Par I deals wih discree ime models under cerainy. The rs chaper inroduces he simples possible ineremporal problem, a wo-period problem. I is solved in a general way and for many funcional forms. The mehods used are he Lagrangian and simple subsiuion. Various conceps like he ime preference rae and he ineremporal elasiciies of subsiuion are inroduced here as well, as hey are widely used in he lieraure and are used frequenly hroughou his book. For hose who wan o undersand he background of he Lagrangian, a chaper is included ha shows he link beween Lagrangians and solving by subsiuion. This will also give us he opporuniy o explain he concep of shadow prices as hey play an imporan role e.g. when using Hamilonians or dynamic programming. The wo-period opimal consumpion seup will hen be pu ino a decenralized general equilibrium seup. This allows us o undersand general equilibrium srucures in general while, a he same ime, we ge o know he sandard overlapping generaions (OLG) general equilibrium model. This is one of he mos widely used dynamic models in Economics. Chaper 2 concludes by reviewing some aspecs of di erence equaions. Chaper 3 hen covers in nie horizon models. We solve a ypical maximizaion problem rs by using he Lagrangian again and hen by dynamic programming. As dynamic programming regularly uses he envelope heorem, his heorem is rs reviewed in a simple saic seup. Examples for in nie horizon problems, a general equilibrium analysis of a decenralized economy, a ypical cenral planner problem and an analysis of how o rea family or populaion growh in opimizaion problems hen complee his chaper. To complee he range of maximizaion mehods used in his book, he presenaion of hese examples will also use he mehod of solving by insering. 9

10

Chaper 2 Two-period models and di erence equaions Given ha he idea of his book is o sar from simple srucures and exend hem o he more complex ones, his chaper sars wih he simples ineremporal problem, a wo-period decision framework. This simple seup already allows us o illusrae he basic dynamic rade-o s. Aggregaing over individual behaviour, assuming an overlappinggeneraions (OLG) srucure, and puing individuals in general equilibrium provides an undersanding of he issues involved in hese seps and in idenical seps in more general seings. Some revision of properies of di erence equaions concludes his chaper. 2.1 Ineremporal uiliy maximizaion 2.1.1 The seup Le here be an individual living for wo periods, in he rs she is young, in he second she is old. Le her uiliy funcion be given by U = U c y ; c o +1 U (c ; c +1 ) ; (2.1.1) where consumpion when young and old are denoed by c y and c o +1 or c and c +1 ; respecively, when no ambiguiy arises. The individual earns labour income w in boh periods. I could also be assumed ha she earns labour income only in he rs period (e.g. when reiring in he second) or only in he second period (e.g. when going o school in he rs). Here, wih s denoing savings, her budge consrain in he rs period is c + s = w (2.1.2) and in he second i reads c +1 = w +1 + (1 + r +1 ) s : (2.1.3) 11

12 Chaper 2. Two-period models and di erence equaions Ineres paid on savings in he second period are given by r +1 : All quaniies are expressed in unis of he consumpion good (i.e. he price of he consumpion good is se equal o one. See ch. 2.2.2 for an exension where he price of he consumpion good is p :). This budge consrain says somehing abou he assumpions made on he iming of wage paymens and savings. Wha an individual can spend in period wo is principal and ineres on her savings s of he rs period. There are no ineres paymens in period one. This means ha wages are paid and consumpion akes place a he end of period 1 and savings are used for some producive purposes (e.g. rms use i in he form of capial for producion) in period 2. Therefore, reurns r +1 are deermined by economic condiions in period 2 and have he index + 1: Timing is illusraed in he following gure. w s = w c c w +1 c +1 (1 + r +1 )s + 1 Figure 2.1.1 Timing in wo-period models These wo budge consrains can be merged ino one ineremporal budge consrain by subsiuing ou savings, w + (1 + r +1 ) 1 w +1 = c + (1 + r +1 ) 1 c +1 : (2.1.4) I should be noed ha by no resricing savings o be non-negaive in (2.1.2) or by equaing he presen value of income on he lef-hand side wih he presen value of consumpion on he righ-hand side in (2.1.4), we assume perfec capial markes: individuals can save and borrow any amoun hey desire. Equaion (2.2.14) provides a condiion under which individuals save. Adding he behavioural assumpion ha individuals maximize uiliy, he economic behaviour of an individual is described compleely and one can derive her consumpion and saving decisions. The problem can be solved by a Lagrange approach or by simply insering. The laer will be done in ch. 2.2.1 and 3.8.2 in deerminisic seups or exensively in ch. 8.1.4 for a sochasic framework. Insering ransforms an opimizaion problem wih a consrain ino an unconsrained problem. We will use a Lagrange approach now. The maximizaion problem reads max c; c +1 (2.1.1) subjec o he ineremporal budge consrain (2.1.4). The household s conrol variables are c and c +1 : As hey need o be chosen so ha hey saisfy he budge consrain, hey can no be chosen independenly of each oher. Wages and ineres raes are exogenously given o he household. When

2.1. Ineremporal uiliy maximizaion 13 choosing consumpion levels, he reacion of hese quaniies o he decision of our household is assumed o be zero - simply because he household is oo small o have an e ec on economy-wide variables. 2.1.2 Solving by he Lagrangian We will solve his maximizaion problem by using he Lagrange funcion. This funcion will now be presened simply and is srucure will be explained in a recipe sense, which is he mos useful one for hose ineresed in quick applicaions. For hose ineresed in more background, ch. 2.3 will show he formal principles behind he Lagrangian. The Lagrangian for our problem reads L = U (c ; c +1 ) + w + (1 + r +1 ) 1 w +1 c (1 + r +1 ) 1 c +1, (2.1.5) where is a parameer called he Lagrange muliplier. The Lagrangian always consiss of wo pars. The rs par is he objecive funcion, he second par is he produc of he Lagrange muliplier and he consrain, expressed as he di erence beween he righ-hand side and he lef-hand side of (2.1.4). Technically speaking, i makes no di erence wheher one subracs he lef-hand side from he righ-hand side as here or vice versa - righhand side minus lef-hand side. Reversing he di erence would simply change he sign of he Lagrange muliplier bu no change he nal opimaliy condiions. Economically, however, one would usually wan a posiive sign of he muliplier, as we will see in ch. 2.3. The rs-order condiions are L c = U c (c ; c +1 ) = 0; L c+1 = U c+1 (c ; c +1 ) [1 + r +1 ] 1 = 0; L = w + (1 + r +1 ) 1 w +1 c (1 + r +1 ) 1 c +1 = 0: Clearly, he las rs-order condiion is idenical o he budge consrain. Noe ha here are hree variables o be deermined, consumpion for boh periods and he Lagrange muliplier. Having a leas hree condiions is a necessary, hough no su cien (hey migh, generally speaking, be linearly dependen) condiion for his o be possible. The rs wo rs-order condiions can be combined o give U c (c ; c +1 ) = (1 + r +1 ) U c+1 (c ; c +1 ) = : (2.1.6) Marginal uiliy from consumpion oday on he lef-hand side mus equal marginal uiliy omorrow, correced by he ineres rae, on he righ-hand side. The economic meaning of his correcion can bes be undersood when looking a a version wih nominal budge consrains (see ch. 2.2.2). We learn from his maximizaion ha he modi ed rs-order condiion (2.1.6) gives us a necessary equaion ha needs o hold when behaving opimally. I links consumpion c oday o consumpion c +1 omorrow. This equaion ogeher wih he budge consrain

14 Chaper 2. Two-period models and di erence equaions (2.1.4) provides a wo-dimensional sysem: wo equaions in wo unknowns, c and c +1. These equaions herefore allow us in principle o compue hese endogenous variables as a funcion of exogenously given wages and ineres raes. This would hen be he soluion o our maximizaion problem. The nex secion provides an example where his is indeed he case. 2.2 Examples 2.2.1 Opimal consumpion The seup This rs example allows us o solve explicily for consumpion levels in boh periods. Le preferences of households be represened by U = ln c + (1 ) ln c +1 : (2.2.1) This uiliy funcion is ofen referred o as Cobb-Douglas or logarihmic uiliy funcion. Uiliy from consumpion in each period, insananeous uiliy, is given by he logarihm of consumpion. Insananeous uiliy is someimes also referred o as feliciy funcion. As ln c has a posiive rs and negaive second derivaive, higher consumpion increases insananeous uiliy bu a a decreasing rae. Marginal uiliy from consumpion is decreasing in (2.2.1). The parameer capures he imporance of insananeous uiliy in he rs relaive o insananeous uiliy in he second. Overall uiliy U is maximized subjec o he consrain we know from (2.1.4) above, where we denoe he presen value of labour income by W = c + (1 + r +1 ) 1 c +1 ; (2.2.2) W w + (1 + r +1 ) 1 w +1 : (2.2.3) Again, he consumpion good is chosen as numeraire good and is price equals uniy. Wages are herefore expressed in unis of he consumpion good. Solving by he Lagrangian The Lagrangian for his problem reads L = ln c + (1 ) ln c +1 + W c (1 + r +1 ) 1 c +1 : The rs-order condiions are L c = (c ) 1 = 0; L c+1 = (1 ) (c +1 ) 1 [1 + r +1 ] 1 = 0; and he budge consrain (2.2.2) following from L = 0:

2.2. Examples 15 The soluion Dividing rs-order condiions gives c = c +1 1 c 1 (1 + r +1) 1 c +1 : = 1 + r +1 and herefore This equaion corresponds o our opimaliy rule (2.1.6) derived above for he more general case. Insering ino he budge consrain (2.2.2) yields W = 1 + 1 (1 + r +1 ) 1 c +1 = 1 1 (1 + r +1) 1 c +1 which is equivalen o I follows ha c +1 = (1 ) (1 + r +1 ) W : (2.2.4) c = W : (2.2.5) Apparenly, opimal consumpion decisions imply ha consumpion when young is given by a share of he presen value W of life-ime income a ime of he individual under consideraion. Consumpion when old is given by he remaining share 1 plus ineres paymens, c +1 = (1 + r +1 ) (1 ) W : Equaions (2.2.4) and (2.2.5) are he soluion o our maximizaion problem. These expressions are someimes called closedform soluions. A (closed-form) soluion expresses he endogenous variable, consumpion in our case, as a funcion only of exogenous variables. Closed-form soluion is a di eren word for closed-loop soluion. For furher discussion see ch. 5.6.2. Assuming preferences as in (2.2.1) makes modelling someimes easier han wih more complex uiliy funcions. A drawback here is ha he share of lifeime income consumed in he rs period and herefore he savings decision is independen of he ineres rae, which appears implausible. A way ou is given by he CES uiliy funcion (see below a (2.2.10)) which also allows for closed-form soluions for consumpion (for an example in a sochasic seup, see exercise 6 in ch. 8.1.4). More generally speaking, such a simpli caion should be jusi ed if some aspec of an economy ha is fairly independen of savings is he focus of some analysis. Solving by subsiuion Le us now consider his example and see how his maximizaion problem could have been solved wihou using he Lagrangian. The principle is simply o ransform an opimizaion problem wih consrains ino an opimizaion problem wihou consrains. This is mos simply done in our example by replacing he consumpion levels in he objecive funcion (2.2.1) by he consumpion levels from he consrains (2.1.2) and (2.1.3). The unconsrained maximizaion problem hen reads max U by choosing s ; where U = ln (w s ) + (1 ) ln (w +1 + (1 + r +1 ) s ) :

16 Chaper 2. Two-period models and di erence equaions This objecive funcion shows he rade-o faced by anyone who wans o save nicely. High savings reduce consumpion oday bu increase consumpion omorrow. The rs-order condiion, i.e. he derivaive wih respec o saving s ; is simply 1 = (1 + r +1 ) : w s w +1 + (1 + r +1 ) s When his is solved for savings s ; we obain w +1 + s = (w s ) 1 1 + r +1, w +1 1 = w 1 + r +1 s = (1 ) w w +1 1 + r +1 = w W ; s 1, where W is life-ime income as de ned afer (2.2.2). To see ha his is consisen wih he soluion by Lagrangian, compue rs-period consumpion and nd c = w s = W - which is he soluion in (2.2.5). Wha have we learned from using his subsiuion mehod? We see ha we do no need sophisicaed ools like he Lagrangian as we can solve a normal unconsrained problem - he ype of problem we migh be more familiar wih from saic maximizaion seups. Bu he seps o obain he nal soluion appear somewha more curvy and less elegan. I herefore appears worhwhile o become more familiar wih he Lagrangian. 2.2.2 Opimal consumpion wih prices Consider now again he uiliy funcion (2.1.1) and maximize i subjec o he consrains p c + s = w and p +1 c +1 = w +1 + (1 + r +1 ) s : The di erence o he inroducory example in ch. 2.1 consiss in he inroducion of an explici price p for he consumpion good. The rs-period budge consrain now equaes nominal expendiure for consumpion (p c is measured in, say, Euro, Dollar or Yen) plus nominal savings o nominal wage income. The second period consrain equally equaes nominal quaniies. Wha does an opimal consumpion rule as in (2.1.6) now look like? The Lagrangian is, now using an ineremporal budge consrain wih prices, L = U (c ; c +1 ) + W p c (1 + r +1 ) 1 p +1 c +1 : The rs-order condiions for c and c +1 are L c = U c (c ; c +1 ) p = 0; L c+1 = U c+1 (c ; c +1 ) (1 + r +1 ) 1 p +1 = 0 and he ineremporal budge consrain. Combining hem gives U c (c ; c +1 ) p = U c +1 (c ; c +1 ) p +1 [1 + r +1 ] 1, U c (c ; c +1 ) U c+1 (c ; c +1 ) = p p +1 [1 + r +1 ] 1 : (2.2.6)

2.2. Examples 17 This equaion says ha marginal uiliy of consumpion oday relaive o marginal uiliy of consumpion omorrow equals he relaive price of consumpion oday and omorrow. This opimaliy rule is idenical for a saic 2-good decision problem where opimaliy requires ha he raio of marginal uiliies equals he relaive price. The relaive price here is expressed in a presen value sense as we compare marginal uiliies a wo poins in ime. The price p is herefore divided by he price omorrow, discouned by he nex period s ineres rae, p +1 [1 + r +1 ] 1 : In conras o wha someimes seems common pracice, we will no call (2.2.6) a soluion o he maximizaion problem. This expression (frequenly referred o as he Euler equaion) is simply an expression resuling from rs-order condiions. Sricly speaking, (2.2.6) is only a necessary condiion for opimal behaviour - and no more. As de ned above, a soluion o a maximizaion problem is a closed-form expression as for example in (2.2.4) and (2.2.5). I gives informaion on levels - and no jus on changes as in (2.2.6). Being aware of his imporan di erence, in wha follows, he erm solving a maximizaion problem will neverheless cover boh analyses. Those which sop a he Euler equaion and hose which go all he way owards obaining a closed-form soluion. 2.2.3 Some useful de niions wih applicaions In order o be able o discuss resuls in subsequen secions easily, we review some de niions here ha will be used frequenly in laer pars of his book. We are mainly ineresed in he ineremporal elasiciy of subsiuion and he ime preference rae. While a lo of his maerial can be found in micro exbooks, he noaion used in hese books di ers of course from he one used here. As his book is also inended o be as self-conained as possible, his shor review can serve as a reference for subsequen explanaions. We sar wih he Marginal rae of subsiuion (MRS) Le here be a consumpion bundle (c 1 ; c 2 ; ::::; c n ) : Le uiliy be given by u (c 1 ; c 2 ; ::::; c n ) which we abbreviae o u (:) : The MRS beween good i and good j is hen de ned by MRS ij (:) @u (:) =@c i @u (:) =@c j : (2.2.7) I gives he increase of consumpion of good j ha is required o keep he uiliy level a u (c 1 ; c 2 ; ::::; c n ) when he amoun of i is decreased marginally. By his de niion, his amoun is posiive if boh goods are normal goods - i.e. if boh parial derivaives in (2.2.7) are posiive. Noe ha de niions used in he lieraure can di er from his one. Some replace decreased by increased (or - which has he same e ec - replace increase by decrease ) and hereby obain a di eren sign. Why his is so can easily be shown: Consider he oal di erenial of u (c 1 ; c 2 ; ::::; c n ) ; keeping all consumpion levels apar from c i and c j x. This yields du (c 1 ; c 2 ; ::::; c n ) = @u (:) @c i dc i + @u (:) @c j dc j :

18 Chaper 2. Two-period models and di erence equaions The overall uiliy level u (c 1 ; c 2 ; ::::; c n ) does no change if du (:) = 0, dc j dc i = @u (:) @u (:) = MRS ij (:) : @c i @c j Equivalen erms As a reminder, he equivalen erm o he MRS in producion heory is he marginal rae of echnical subsiuion MRT S ij (:) = @f(:)=@x i @f(:)=@x j where he uiliy funcion was replaced by a producion funcion and consumpion c k was replaced by facor inpus x k. On a more economy-wide level, here is he marginal rae of ransformaion MRT ij (:) = @G(:)=@y i @G(:)=@y j where he uiliy funcion has now been replaced by a ransformaion funcion G (maybe beer known as producion possibiliy curve) and he y k are oupu of good k. The marginal rae of ransformaion gives he increase in oupu of good j when oupu of good i is marginally decreased. (Ineremporal) elasiciy of subsiuion Though our main ineres is a measure of ineremporal subsiuabiliy, we rs de ne he elasiciy of subsiuion in general. As wih he marginal rae of subsiuion, he de niion implies a cerain sign of he elasiciy. In order o obain a posiive sign (wih normal goods), we de ne he elasiciy of subsiuion as he increase in relaive consumpion c i =c j when he relaive price p i =p j decreases (which is equivalen o an increase of p j =p i ). Formally, we obain for he case of wo consumpion goods ij d(c i=c j ) d(p j =p i ) p j =p i c i =c j. This de niion can be expressed alernaively (see ex. 6 for deails) in a way which is more useful for he examples below. We express he elasiciy of subsiuion by he derivaive of he log of relaive consumpion c i =c j wih respec o he log of he marginal rae of subsiuion beween j and i, ij d ln (c i=c j ) d ln MRS ji : (2.2.8) Insering he marginal rae of subsiuion MRS ji from (2.2.7), i.e. exchanging i and j in (2.2.7), gives ij = d ln (c i=c j ) = u c j =u ci d (c i =c j ) : d ln u cj =u ci c i =c j d u cj =u ci The advanage of an elasiciy when compared o a normal derivaive, such as he MRS, is ha an elasiciy is measureless. I is expressed in percenage changes. (This can be bes seen in he following example and in ex. 6 where he derivaive is muliplied by p j=p i c i =c j :) I can boh be applied o saic uiliy or producion funcions or o ineremporal uiliy funcions.

2.2. Examples 19 The ineremporal elasiciy of subsiuion for a uiliy funcion u (c ; c +1 ) is hen simply he elasiciy of subsiuion of consumpion a wo poins in ime, ;+1 u c =u c+1 c +1 =c d (c +1 =c ) d u c =u c+1 : (2.2.9) Here as well, in order o obain a posiive sign, he subscrips in he denominaor have a di eren ordering from he one in he numeraor. The ineremporal elasiciy of subsiuion for logarihmic and CES uiliy funcions For he logarihmic uiliy funcion U = ln c +(1 an ineremporal elasiciy of subsiuion of one, where he las sep used ;+1 = d (c +1 =c ) = d c = 1 c +1 c = 1 c +1 d (c +1 =c ) = 1; c +1 =c d c = 1 c +1 1 d (c +1 =c ) d (c +1 =c ) = 1 ) ln c +1 from (2.2.1), we obain : I is probably worh noing a his poin ha no all exbooks would agree on he resul of plus one. Following some oher de niions, a resul of minus one would be obained. Keeping in mind ha he sign is jus a convenion, depending on increase or decrease in he de niion, his should no lead o confusions. When we consider a uiliy funcion where insananeous uiliy is no logarihmic bu of CES ype U = c 1 + (1 ) c 1 +1 ; (2.2.10) he ineremporal elasiciy of subsiuion becomes ;+1 [1 ] c = (1 ) (1 ) c+1 d (c +1 =c ) c +1 =c d [1 ] c = (1 ) (1 ) c De ning x (c +1 =c ) ; we obain d [1 d (c +1 =c ) ] c = (1 ) (1 ) c +1 = 1 = 1 +1 d (c +1 =c ) d c = 1 1 x 1 1 : =c +1 : (2.2.11) dx 1= Insering his ino (2.2.11) and cancelling erms, he elasiciy of subsiuion urns ou o be ;+1 c =c+1 1 1 c+1 = 1 c +1 =c : This is where he CES uiliy funcion (2.2.10) has is name from: The ineremporal elasiciy (E) of subsiuion (S) is consan (C). c dx

20 Chaper 2. Two-period models and di erence equaions The ime preference rae Inuiively, he ime preference rae is he rae a which fuure insananeous uiliies are discouned. To illusrae, imagine a discouned income sream x 0 + 1 2 1 1 + r x 1 + x 2 + ::: 1 + r where discouning akes place a he ineres rae r: Replacing income x by insananeous uiliy and he ineres rae by, would be he ime preference rae. Formally, he ime preference rae is he marginal rae of subsiuion of insananeous uiliies (no of consumpion levels) minus one, T P R MRS ;+1 1: As an example, consider he following sandard uiliy funcion which we will use very ofen in laer chapers, U 0 = 1 =0 u (c ) ; 1 ; > 0: (2.2.12) 1 + Le be a posiive parameer and he implied discoun facor, capuring he idea of impaience: By muliplying insananeous uiliy funcions u (c ) by, fuure uiliy is valued less han presen uiliy. This uiliy funcion generalizes (2.2.1) in wo ways: Firs and mos imporanly, here is a much longer planning horizon han jus wo periods. In fac, he individual s overall uiliy U 0 sems from he sum of discouned insananeous uiliy levels u (c ) over periods 0; 1; 2;... up o in niy. The idea behind his objecive funcion is no ha individuals live forever bu ha individuals care abou he wellbeing of subsequen generaions. Second, he insananeous uiliy funcion u (c ) is no logarihmic as in (2.2.1) bu of a more general naure where one would usually assume posiive rs and negaive second derivaives, u 0 > 0; u 00 < 0. The marginal rae of subsiuion is hen MRS ;+1 (:) = @U 0 (:) =@u (c ) @U 0 (:) =@u (c +1 ) = (1= (1 + )) +1 = 1 + : (1= (1 + )) The ime preference rae is herefore given by : Now ake for example he uiliy funcion (2.2.1). Compuing he MRS minus one, we have = 1 1 = 2 1 1 : (2.2.13) The ime preference rae is posiive if > 0:5: This makes sense for (2.2.1) as one should expec ha fuure uiliy is valued less han presen uiliy. As a side noe, all ineremporal uiliy funcions in his book will use exponenial discouning as in (2.2.12). This is clearly a special case. Models wih non-exponenial or hyperbolic discouning imply fundamenally di eren dynamic behaviour and ime inconsisencies. See furher reading for some references.

2.3. The idea behind he Lagrangian 21 Does consumpion increase over ime? This de niion of he ime preference rae allows us o provide a precise answer o he quesion wheher consumpion increases over ime. We simply compue he condiion under which c +1 > c by using (2.2.4) and (2.2.5), c +1 > c, (1 ) (1 + r +1 ) W > W, 1 + r +1 > 1, r +1 > 1 +, r +1 > : 1 Consumpion increases if he ineres rae is higher han he ime preference rae. The ime preference rae of he individual (being represened by ) deermines how o spli he presen value W of oal income ino curren and fuure use. If he ineres rae is su cienly high o overcompensae impaience, i.e. if (1 ) (1 + r) > in he rs line, consumpion rises. Noe ha even hough we compued he condiion for rising consumpion for our special uiliy funcion (2.2.1), he resul ha consumpion increases when he ineres rae exceeds he ime preference rae holds for more general uiliy funcions as well. We will ge o know various examples for his in subsequen chapers. Under wha condiions are savings posiive? Savings are from he budge consrain (2.1.2) and he opimal consumpion resul (2.2.5) given by 1 s = w c = w w + w +1 = w 1 1 + r +1 1 + r +1 where he las equaliy assumed an invarian wage level, w = w +1 w. Savings are posiive if and only if s > 0, 1 >, 1 + r +1 > 1 + r +1 1, r +1 > 2 1 1, r +1 > (2.2.14) This means ha savings are posiive if ineres rae is larger han ime preference rae. Clearly, his resul does no necessarily hold for w +1 > w : 2.3 The idea behind he Lagrangian So far, we simply used he Lagrange funcion wihou asking where i comes from. This chaper will o er a derivaion of he Lagrange funcion and also an economic inerpreaion of he Lagrange muliplier. In maximizaion problems employing a uiliy funcion, he Lagrange muliplier can be undersood as a price measured in uiliy unis. I is ofen called a shadow price.

22 Chaper 2. Two-period models and di erence equaions 2.3.1 Where he Lagrangian comes from I The maximizaion problem Le us consider a maximizaion problem wih some objecive funcion and a consrain, max F (x 1 ; x 2 ) subjec o g(x 1; x 2 ) = b: (2.3.1) x 1 ;x 2 The consrain can be looked a as an implici funcion, i.e. describing x 2 as a funcion of x 1 ; i.e. x 2 = h (x 1 ) : Using he represenaion x 2 = h (x 1 ) of he consrain, he maximizaion problem can be wrien as max x 1 F (x 1; h (x 1 )) : (2.3.2) The derivaives of implici funcions As we will use implici funcions and heir derivaives here and in laer chapers, we brie y illusrae he underlying idea and show how o compue heir derivaives. Consider a funcion f (x 1 ; x 2 ; :::; x n ) = 0: The implici funcion heorem says - saed simply - ha his funcion f (x 1 ; x 2 ; :::; x n ) = 0 implicily de nes (under suiable assumpions concerning he properies of f (:) - see exercise 7) a funcional relaionship of he ype x 2 = h (x 1 ; x 3 ; x 4 ; :::; x n ) : We ofen work wih hese implici funcions in Economics and we are also ofen ineresed in he derivaive of x 2 wih respec o, say, x 1 : In order o obain an expression for his derivaive, consider he oal di erenial of f (x 1 ; x 2 ; :::; x n ) = 0; df (:) = @f (:) @f (:) @f (:) dx 1 + dx 2 + ::: + dx n = 0: @x 1 @x 2 @x n When we keep x 3 o x n consan, we can solve his o ge dx 2 dx 1 = @f (:) =@x 1 @f (:) =@x 2 : (2.3.3) We have hereby obained an expression for he derivaive dx 2 =dx 1 wihou knowing he funcional form of he implici funcion h (x 1 ; x 3 ; x 4 ; :::; x n ) : For illusraion purposes, consider he following gure.

2.3. The idea behind he Lagrangian 23 Figure 2.3.1 The implici funcion visible a z = 0 The horizonal axes plo x 1 and, o he back, x 2 : The verical axis plos z: The increasing surface depics he graph of he funcion z = g (x 1 ; x 2 ) b: When his surface crosses he horizonal plane a z = 0; a curve is creaed which conains all he poins where z = 0: Looking a his curve illusraes ha he funcion z = 0, g (x 1 ; x 2 ) = b implicily de nes a funcion x 2 = h (x 1 ) : See exercise 7 for an explici analyical derivaion of such an implici funcion. The derivaive dx 2 =dx 1 is hen simply he slope of his curve. The analyical expression for his is - using (2.3.3) - dx 2 =dx 1 = (@g (:) =@x 1 ) = (@g (:) =@x 2 ) : Firs-order condiions of he maximizaion problem The maximizaion problem we obained in (2.3.2) is an example for he subsiuion mehod: The budge consrain was solved for one conrol variable and insered ino he objecive funcion. The resuling maximizaion problem is one wihou consrain. The problem in (2.3.2) now has a sandard rs-order condiion, df = @F + @F dh = 0: (2.3.4) dx 1 @x 1 @x 2 dx 1 Taking ino consideraion ha from he implici funcion heorem applied o he consrain, dh = dx 2 = @g(x 1; x 2 )=@x 1 ; (2.3.5) dx 1 dx 1 @g(x 1 ; x 2 )=@x 2 he opimaliy condiion (2.3.4) can be wrien as @F @F=@x 2 @g(x 1 ;x 2 ) @x 1 @g=@x 2 @x 1 he Lagrange muliplier @F=@x 2 @g=@x 2 and obain = 0: Now de ne @F @x 1 @g(x 1; x 2 ) @x 1 = 0: (2.3.6)

24 Chaper 2. Two-period models and di erence equaions As can be easily seen, his is he rs-order condiion of he Lagrangian L = F (x 1; x 2 ) + [b g (x 1; x 2 )] (2.3.7) wih respec o x 1 : Now imagine we wan o underake he same seps for x 2 ; i.e. we sar from he original problem (2.3.1) bu subsiue ou x 1 : We would hen obain an unconsrained problem as in (2.3.2) only ha we maximize wih respec o x 2 : Coninuing as we jus did for x 1 would yield he second rs-order condiion @F @g(x 1; x 2 ) = 0: @x 2 @x 2 We have hereby shown where he Lagrangian comes from: Wheher one de nes a Lagrangian as in (2.3.7) and compues he rs-order condiion or one compues he rsorder condiion from he unconsrained problem as in (2.3.4) and hen uses he implici funcion heorem and de nes a Lagrange muliplier, one always ends up a (2.3.6). The Lagrangian-roue is obviously faser. 2.3.2 Shadow prices The idea We can now also give an inerpreaion of he meaning of he mulipliers. Saring from he de niion of in (2.3.6), we can rewrie i according o @F=@x 2 @g=@x 2 = @F @g = @F @b : One can undersand ha he rs equaliy can cancel he erm @x 2 by looking a he @f(x de niion of a (parial) derivaive: 1 ;:::;x n) @x i = lim 4x i!0 f(x 1 ;:::;x i +4x i ;:::;x n) f(x 1 ;:::;x n) lim 4xi!0 4x i : The second equaliy uses he equaliy of g and b from he consrain in (2.3.1). From hese ransformaions, we see ha equals he change in F as a funcion of b. I is now easy o come up wih examples for F or b: How much does F increase (e.g. your uiliy) when your consrain b (your bank accoun) is relaxed? How much does he social welfare funcion change when he economy has more capial? How much do pro s of rms change when he rm has more workers? This is called shadow price and expresses he value of b in unis of F:

2.3. The idea behind he Lagrangian 25 A derivaion A more rigorous derivaion is as follows (cf. Inriligaor, 1971, ch. 3.3). Compue he derivaive of he maximized Lagrangian wih respec o b, @L(x 1 (b) ; x 2 (b)) @b = @ @b (F (x 1(b); x 2 (b)) + (b) [b g (x 1(b); x 2 (b))] @x 1 = F x 1 @b + F @x 2 x 2 @b + 0 (b) [b = (b) g(:)] + (b) 1 g x 1 @x 1 @b @x 2 g x 2 @b The las equaliy resuls from rs-order condiions and he fac ha he budge consrain holds. As L(x 1; x 2) = F (x 1; x 2) due o he budge consrain holding wih equaliy, An example (b) = @L(x 1; x 2) @b = @F (x 1; x 2) @b The Lagrange muliplier is frequenly referred o as shadow price. As we have seen, is uni depends on he uni of he objecive funcion F: One can hink of price in he sense of a price in a currency, for example in Euro, only if he objecive funcion is some nominal expression like pro s or GDP. Oherwise i is a price expressed for example in uiliy erms. This can explicily be seen in he following example. Consider a cenral planner ha maximizes social welfare u (x 1 ; x 2 ) subjec o echnological and resource consrains, max u (x 1 ; x 2 ) subjec o x 1 = f (K 1; L 1 ) ; x 2 = g (K 2; L 2 ) ; K 1 + K 2 = K; L 1 + L 2 = L: Technologies in secors 1 and 2 are given by f (:) and g (:) and facors of producion are capial K and labour L: Using as mulipliers p 1 ; p 2 ; w K and w L ; he Lagrangian reads L = u (x 1 ; x 2 ) + p 1 [f (K 1; L 1 ) x 1 ] + p 2 [g (K 2 ; L 2 ) x 2 ] + w K [K K 1 K 2 ] + w L [L L 1 L 2 ] (2.3.8) and rs-order condiions are @L = @u p 1 = 0; @x 1 @x 1 @L @f = p 1 @K 1 @K 1 w K = 0; @L @f = p 1 @L 1 @L 1 w L = 0; @L = @u @x 2 @x 2 p 2 = 0; (2.3.9) @L @g = p 2 @K 2 @K 2 w K = 0; @L @g = p 2 @L 2 @L 2 w L = 0:

26 Chaper 2. Two-period models and di erence equaions Here we see ha he rs muliplier p 1 is no a price expressed in some currency bu he derivaive of he uiliy funcion wih respec o good 1, i.e. marginal uiliy. By conras, if we looked a he muliplier w K only in he hird rs-order condiion, w K = p 1 @f=@k 1 ; we would hen conclude ha i is a price. Then insering he rs rs-order condiion, @u=@x 1 = p 1 ; and using he consrain x 1 = f (K 1; L 1 ) shows however ha i really sands for he increase in uiliy when he capial sock used in producion of good 1 rises, w K @f = p 1 = @u @f = @u : @K 1 @x 1 @K 1 @K 1 Hence w K and all oher mulipliers are prices in uiliy unis. I is now also easy o see ha all shadow prices are prices expressed in some currency if he objecive funcion is no uiliy bu, for example GDP. Such a maximizaion problem could read max p 1 x 1 + p 2 x 2 subjec o he consrains as above. Finally, reurning o he discussion afer (2.1.5), he rs-order condiions show ha he sign of he Lagrange muliplier should be posiive from an economic perspecive. If p 1 in (2.3.9) is o capure he value aached o x 1 in uiliy unis and x 1 is a normal good (uiliy increases in x 1 ; i.e. @u=@x 1 > 0), he shadow price should be posiive. If we had represened he consrain in he Lagrangian (2.3.8) as x 1 f (K 1; L 1 ) raher han righ-hand side minus lef-hand side, he rs-order condiion would read @u=@x 1 + p 1 = 0 and he Lagrange muliplier would have been negaive. If we wan o associae he Lagrange muliplier o he shadow price, he consrains in he Lagrange funcion should be represened such ha he Lagrange muliplier is posiive. 2.4 An overlapping generaions model We will now analyze many households joinly and see how heir consumpion and saving behaviour a ecs he evoluion of he economy as a whole. We will ge o know he Euler heorem and how i is used o sum facor incomes o yield GDP. We will also undersand how he ineres rae in he household s budge consrain is relaed o marginal produciviy of capial and he depreciaion rae. All his joinly yields ime pahs of aggregae consumpion, he capial sock and GDP. We will assume an overlapping-generaions srucure (OLG). A model in general equilibrium is described by fundamenals of he model and marke and behavioural assumpions. Fundamenals are echnologies of rms, preferences of households and facor endowmens. Adding clearing condiions for markes and behavioural assumpions for agens complees he descripion of he model.

2.4. An overlapping generaions model 27 2.4.1 Technologies The rms Le here be many rms who employ capial K and labour L o produce oupu Y according o he echnology Y = Y (K ; L) : (2.4.1) Assume producion of he nal good Y (:) is characerized by consan reurns o scale. We choose Y as our numeraire good and normalize is price o uniy, p = 1: While his is no necessary and we could keep he price p all hrough he model, we would see ha all prices, like for example facor rewards, would be expressed relaive o he price p : Hence, as a shorcu, we se p = 1: We now, however, need o keep in mind ha all prices are henceforh expressed in unis of his nal good. Wih his normalizaion, pro s are given by = Y w K K w L L. Leing rms ac under perfec compeiion, he rsorder condiions from pro maximizaion re ec he fac ha he rm akes all prices as parameric and se marginal produciviies equal o real facor rewards, @Y @K = w K ; @Y @L = wl : (2.4.2) In each period hey equae ; he marginal produciviy of capial, o he facor price w K for capial and he marginal produciviy of labour o labour s facor reward w L. Euler s heorem Euler s heorem shows ha for a linear-homogeneous funcion f (x 1 ; x 2 ; :::; x n ) he sum of parial derivaives imes he variables wih respec o which he derivaive was compued equals he original funcion f (:) ; f (x 1 ; x 2 ; :::; x n ) = @f (:) @f (:) @f (:) x 1 + x 2 + ::: + x n : (2.4.3) @x 1 @x 2 @x n Provided ha he echnology used by rms o produce Y has consan reurns o scale, we obain from his heorem ha Y = @Y K + @Y L: (2.4.4) @K @L Using he opimaliy condiions (2.4.2) of he rm for he applied version of Euler s heorem (2.4.4) yields Y = w K K + w L L: (2.4.5) Toal oupu in his economy, Y ; is idenical o oal facor income. This resul is usually given he economic inerpreaion ha under perfec compeiion all revenue in rms is used o pay facors of producion. As a consequence, pro s of rms are zero.

28 Chaper 2. Two-period models and di erence equaions 2.4.2 Households Individual households Households live again for wo periods. The uiliy funcion is herefore as in (2.2.1) and given by U = ln c y + (1 ) ln c o +1: (2.4.6) I is maximized subjec o he ineremporal budge consrain w L = c y + (1 + r +1 ) 1 c o +1: This consrain di ers slighly from (2.2.2) in ha people work only in he rs period and reire in he second. Hence, here is labour income only in he rs period on he lef-hand side. Savings from he rs period are used o nance consumpion in he second period. Given ha he presen value of lifeime wage income is w L ; we can conclude from (2.2.4) and (2.2.5) ha individual consumpion expendiure and savings are given by Aggregaion c y = w L ; c o +1 = (1 ) (1 + r +1 ) w L ; (2.4.7) s = w L c y = (1 ) w L : (2.4.8) We assume ha in each period L individuals are born and die. Hence, he number of young and he number of old is L as well. As all individuals wihin a generaion are idenical, aggregae consumpion wihin one generaion is simply he number of, say, young imes individual consumpion. Aggregae consumpion in is herefore given by C = Lc y + Lc o : Using he expressions for individual consumpion from (2.4.7) and noing he index (and no + 1) for he old yields C = Lc y + Lc o = w L + (1 ) (1 + r ) w L 1 L: 2.4.3 Goods marke equilibrium and accumulaion ideniy The goods marke equilibrium requires ha supply equals demand, Y = C + I ; where demand is given by consumpion plus gross invesmen. Nex period s capial sock is - by an accouning ideniy - given by K +1 = I +(1 ) K : Ne invesmen, amouning o he change in he capial sock, K +1 K, is given by gross invesmen I minus depreciaion K, where is he depreciaion rae, K +1 K = I K. Replacing gross invesmen by he goods marke equilibrium, we obain he resource consrain K +1 = (1 ) K + Y C : (2.4.9) For our OLG seup, i is useful o rewrie his consrain slighly, Y + (1 ) K = C + K +1 : (2.4.10)

2.4. An overlapping generaions model 29 In his formulaion, i re ecs a broader goods marke equilibrium where he lef-hand side shows supply as curren producion plus capial held by he old. The old sell capial as i is of no use for hem, given ha hey will no be able o consume anyhing in he nex period. Demand for he aggregae good is given by aggregae consumpion (i.e. consumpion of he young plus consumpion of he old) plus he capial sock o be held nex period by he currenly young. 2.4.4 The reduced form For he rs ime in his book we have come o he poin where we need o nd wha will be called a reduced form. Once all maximizaion problems are solved and all consrains and marke equilibria are aken ino accoun, he objecive consiss of undersanding properies of he model, i.e. undersanding is predicions. This is usually done by rs simplifying he srucure of he sysem of equaions coming ou of he model as much as possible. In he end, afer insering and reinsering, a sysem of n equaions in n unknowns resuls. The sysem where n is he smalles possible is wha will be called he reduced form. Ideally, here is only one equaion lef and his equaion gives an explici soluion of he endogenous variable. In saic models, an example would be L X = L; i.e. employmen in secor X is given by a uiliy parameer imes he oal exogenous labour supply L: This would be an explici soluion. If we are lef wih jus one equaion bu we obain on an implici soluion, we would obain somehing like f (L X ; ; L) = 0: Deriving he reduced form We now derive, given he resuls we have obained so far, how large he capial sock in he nex period is. Spliing aggregae consumpion ino consumpion of he young and consumpion of he old and using he oupu-facor reward ideniy (2.4.5) for he resource consrain in he OLG case (2.4.10), we obain w K K + w L L + (1 ) K = C y + C o + K +1 : De ning he ineres rae r as he di erence beween facor rewards w K for capial and he depreciaion rae ; r w K ; (2.4.11) we nd r K + w L L + K = C y + C o + K +1 : The ineres rae de niion (2.4.11) shows he ne income of capial owners per uni of capial. They earn he gross facor rewards w K bu, a he same ime, hey experience a loss from depreciaion. Ne income herefore only amouns o r : As he old consume he capial sock plus ineres c o L = (1 + r )K, we obain K +1 = w L L C y = s L: (2.4.12)

30 Chaper 2. Two-period models and di erence equaions which is he aggregae version of he savings equaion (2.4.8). Hence, we have found ha savings s of young a is he capial sock a + 1: Noe ha equaion (2.4.12) is ofen presen on inuiive grounds. The old in period have no reason o save as hey will no be able o use heir savings in + 1: Hence, only he young will save and he capial sock in + 1; being made up from savings in he previous period, mus be equal o he savings of he young. The one-dimensional di erence equaion In our simple dynamic model considered here, we obain he ideal case where we are lef wih only one equaion ha gives us he soluion for one variable, he capial sock. Insering he individual savings equaion (2.4.8) ino (2.4.12) gives wih he rs-order condiion (2.4.2) of he rm K +1 = (1 ) w L L = (1 ) @Y (K ; L) L: (2.4.13) @L The rs equaliy shows ha a share 1 of labour income urns ino capial in he nex period. Ineresingly, he depreciaion rae does no have an impac on he capial sock in period + 1. Economically speaking, he depreciaion rae a ecs he wealh of he old bu - wih logarihmic uiliy - no he saving of he young. 2.4.5 Properies of he reduced form Equaion (2.4.13) is a non-linear di erence equaion in K : All oher quaniies in his equaion are consan. This equaion deermines he enire pah of capial in his dynamic economy, provided we have an iniial condiion K 0. We have herefore indeed solved he maximizaion problem and reduced he general equilibrium model o one single equaion. From he pah of capial, we can compue all oher variables which are of ineres for our economic quesions. Whenever we have reduced a model o is reduced form and have obained one or more di erence equaions (or di erenial equaions in coninuous ime), we would like o undersand he properies of such a dynamic sysem. The procedure is in principle always he same: We rs ask wheher here is some soluion where all variables (K in our case) are consan. This is hen called a seady sae analysis. Once we have undersood he seady sae (if here is one), we wan o undersand how he economy behaves ou of he seady sae, i.e. wha is ransiional dynamics are. Seady sae by In he seady sae, he capial sock is consan, K = K +1 = K, and deermined K = (1 ) @Y (K ; L) L: (2.4.14) @L

2.4. An overlapping generaions model 31 All oher variables like aggregae consumpion, ineres raes, wages ec. are consan as well. Consumpion when young and when old can di er, as in a seup wih nie lifeimes, he ineres rae in he seady sae does no need o equal he ime preference rae of households. Transiional dynamics Dynamics of he capial sock are illusraed in gure 2.4.1. The gure plos he capial sock in period on he horizonal axis. The capial sock in he nex period, K +1 ; is ploed on he verical axis. The law of moion for capial from (2.4.13) hen shows up as he curve in his gure. The 45 line equaes K +1 o K : We sar from our iniial condiion K 0. Equaion (2.4.13) or he curve in his gure hen deermines he capial sock K 1 : This capial sock is hen viewed as K so ha, again, he curve gives us K +1 ; which is, given ha we now sared in 1; he capial sock K 2 of period 2: We can coninue doing so and see graphically ha he economy approaches he seady sae K which we compued in (2.4.14). K +1 N K = K +1 6 - - 6 K +1 = (1 )w L (K )L 0 K 0 K N Figure 2.4.1 Convergence o he seady sae Summary We sared wih a descripion of echnologies in (2.4.1), preferences in (2.4.6) and facor endowmen given by K 0 : Wih behavioural assumpions concerning uiliy and pro maximizaion and perfec compeiion on all markes plus a descripion of markes in (2.4.3) and some juggling of equaions, we ended up wih a one-dimensional di erence equaion (2.4.13) which describes he evoluion of he economy over ime and seady sae in he long-run. Given his formal analysis of he model, we could now sar answering economic quesions.

32 Chaper 2. Two-period models and di erence equaions 2.5 More on di erence equaions The reduced form in (2.4.13) of he general equilibrium model urned ou o be a nonlinear di erence equaion. We derived is properies in a fairly inuiive manner. However, one can approach di erence equaions in a more sysemaic manner, which we will do in his chaper. 2.5.1 Two useful proofs Before we look a di erence equaions, we provide wo resuls on sums which will be useful in wha follows. As he proof of his resul also has an esheic value, here will be a second proof of anoher resul o be done in he exercises. Lemma 2.5.1 For any a 6= 1; Proof. The lef hand side is given by Muliplying his sum by a yields Now subrac (2.5.2) from (2.5.1) and nd T i=1a i = a 1 at 1 a ; T i=0a i = 1 at +1 1 a T i=1a i = a + a 2 + a 3 + : : : + a T 1 + a T : (2.5.1) a T i=1a i = a 2 + a 3 + : : : + a T + a T +1 : (2.5.2) (1 a) T i=1a i = a a T +1, T i=1a i = a 1 at 1 a : (2.5.3) Lemma 2.5.2 T i=1ia i = 1 a 1 at 1 a 1 a Proof. The proof is lef as exercise 9. T a T +1 2.5.2 A simple di erence equaion One of he simples di erence equaions is x +1 = ax ; a > 0: (2.5.4) This equaion appears oo simple o be worh analysing. We do i here as we ge o know he sandard seps in analyzing di erence equaions which we will also use for more complex di erence equaions. The objecive here is herefore no his di erence equaion as such bu wha is done wih i.

2.5. More on di erence equaions 33 Solving by insering The simples way o nd a soluion o (2.5.4) consiss of insering and reinsering his equaion su cienly ofen. Doing i hree imes gives x 1 = ax 0 ; x 2 = ax 1 = a 2 x 0 ; x 3 = ax 2 = a 3 x 0 : When we look a his soluion for = 3 long enough, we see ha he general soluion is x = a x 0 : (2.5.5) This could formally be proven by eiher inducion or by veri caion. In his conex, we can make use of he following De niion 2.5.1 A soluion of a di erence equaion is a funcion of ime which, when insered ino he original di erence equaion, sais es his di erence equaion. Equaion (2.5.5) gives x as a funcion of ime only. Verifying ha i is a soluion indeed jus requires insering i wice ino (2.5.4) o see ha i sais es he original di erence equaion. Examples for soluions The sequence of x given by his soluion, given di eren iniial condiions x 0 ; are shown in he following gure for a > 1: The parameer values chosen are a = 2, x 0 2 f0:5; 1; 2g and runs from 0 o 10. 14 12 10 8 6 4 2 0 0 2 4 6 8 10 Figure 2.5.1 Soluions o a di erence equaion for a > 1

34 Chaper 2. Two-period models and di erence equaions Long-erm behaviour We can now ask wheher x approaches a consan when ime goes o in niy. This gives 8 9 8 < 0 = < 0 < a < 1 lim x = x 0 lim!1!1 a = x 0 : ;, a = 1 : : 1 a > 1 Hence, x approaches a consan only when a < 1: For a = 1; i says a is iniial value x 0 : A graphical analysis For more complex di erence equaions, i ofen urns ou o be useful o analyze heir behaviour in a phase diagram. Even hough his simple di erence equaion can be undersood easily analyically, we will illusrae is properies in he following gure. Here as well, his allows us o undersand how analyses of his ype can also be underaken for more complex di erence equaions. N N x +1 x +1 x = x +1 x +1 = ax x +1 = ax 6 - x = x +1-6 x 0 x Figure 2.5.2 A phase diagram for a < 1 on he lef and a > 1 in he righ panel x 0 x N N The principle of a phase diagram is simple. The horizonal axis plos x ; he verical axis plos x +1 : There is a 45 line which serves o equae x o x +1 and here is a plo of he di erence equaion we wan o undersand. In our curren example, we plo x +1 as ax ino he gure. Now sar wih some iniial value x 0 and plo his on he horizonal axis as in he lef panel. The value for he nex period, i.e. for period 1; can hen be read o he verical axis by looking a he graph of ax : This value for x 1 is hen copied ono he horizonal axis by using he 45 line. Once on he horizonal axis, we can again use he graph of ax o compue he nex x +1 : Coninuing o do so, he lef panel shows how x evolves over ime, saring a x 0 : In his case of a < 1; we see how x approaches zero. When we graphically illusrae he case of a > 1; he evoluion of x is as shown in he righ panel.

2.5. More on di erence equaions 35 2.5.3 A slighly less bu sill simple di erence equaion We now consider a slighly more general di erence equaion. Compared o (2.5.4), we jus add a consan b in each period, Solving by insering x +1 = ax + b; a > 0: (2.5.6) We solve again by insering. In conras o he soluion for (2.5.4), we sar from an iniial value of x : Hence, we imagine we are in ( as oday) and compue wha he level of x will be omorrow and he day afer omorrow ec. We nd for x +2 and x +3 ha x +2 = ax +1 + b = a [ax + b] + b = a 2 x + b [1 + a] ; x +3 = a 3 x + b 1 + a + a 2 : This suggess ha he general soluion is x +n = a n x + b n 1 i=0 ai = a n x + b an 1 a 1 : The las equaliy used he rs lemma from ch. 2.5.1. N x +1 Limi for n! 1 and a < 1 The limi for n going o in niy and a < 1 is given by A graphical analysis x = x +1?? x +1 = ax + b - b - 6 lim x +n = lim n!1 n!1 an x + b an 1 a 1 = b 1 a : (2.5.7) x +1 N x +1 = ax + b - - 6 6 x = x +1 x x N x x N Figure 2.5.3 Phase diagrams of (2.5.6) for posiive (lef panel) and negaive b (righ panel) and higher a in he righ panel

36 Chaper 2. Two-period models and di erence equaions The lef panel in g. 2.5.3 sudies he evoluion of x for he sable case, i.e. where 0 < a < 1 and b > 0: Saring oday in wih x ; we end up in x. As we chose a smaller han one and a posiive b; x is posiive as (2.5.7) shows. We will reurn o he righ panel in a momen. 2.5.4 Fix poins and sabiliy De niions We can use hese examples o de ne wo conceps ha will also be useful a laer sages. De niion 2.5.2 (Fixpoin) A xpoin x of a funcion f (x) is de ned by x = f (x ) : (2.5.8) For di erence equaions of he ype x +1 = f (x ) ; he xpoin x of he funcion f (x ) is also he poin where x says consan, i.e. x +1 = x. This is usually called he long-run equilibrium poin of some economy or is seady or saionary sae. Whenever an economic model, represened by is reduced form, is analyzed, i is generally useful o rs ry and nd ou wheher xpoins exis and wha heir economic properies are. For he di erence equaion from he las secion, we obain x +1 = x x () x = ax + b () x = b 1 a : Once a xpoin has been ideni ed, one can ask wheher i is sable. De niion 2.5.3 (Global sabiliy) A xpoin x is globally sable if, saring from an iniial value x 0 6= x ; x converges o x. The concep of global sabiliy usually refers o iniial values x 0 ha are economically meaningful. An iniial physical capial sock ha is negaive would no be considered o be economically meaningful. unsable De niion 2.5.4 (Local sabiliy and insabiliy) A xpoin x is if, locally sable diverges from saring from an iniial value x + "; where " is small, x x converges o. For illusraion purposes consider he xpoin x in he lef panel of g. 2.5.3 - i is globally sable. In he righ panel of he same gure, i is unsable. As can easily be seen, he insabiliy follows by simply leing he x +1 line inersec he 45 -line from below. In erms of he underlying di erence equaion (2.5.6), his requires b < 0 and a > 1:

2.5. More on di erence equaions 37 Clearly, economic sysems can be much more complex and generae several xpoins. Imagine he link beween x +1 and x is no linear as in (2.5.6) bu nonlinear, x +1 = f (x ) : Unforunaely for economic analysis, a nonlinear relaionship is he much more realisic case. The nex gure provides an example for some funcion f (x ) ha implies an unsable x u and a locally sable xpoin x s. x +1 N? - - 6? x 01 x u x 02 x s x 03 N x Figure 2.5.4 A locally sable xpoin x s and an unsable xpoin x u 2.5.5 An example: Deriving a budge consrain A frequenly encounered di erence equaion is he budge consrain. We have worked wih budge consrains a various poins before bu we have hardly hough abou heir origin. We more or less simply wroe hem down. Budge consrains, however, are ricky objecs, a leas when we hink abou general equilibrium seups. Wha is he asse we save in? Is here only one asse or are here several? Wha are he prices of hese asses? How does i relae o he price of he consumpion good, i.e. do we express he value of asses in real or nominal erms? This secion will derive a budge consrain. We assume ha here is only one asse. The price of one uni of he asse will be denoed by v : Is relaion o he price p of he consumpion good will be lef unspeci ed, i.e. we will discuss he mos general seup which is possible for he one-asse case. The derivaion of a budge consrain is in principle sraighforward. One de nes he wealh of he household (aking ino accoun which ypes of asses he household can hold for saving purposes and wha heir prices are), compues he di erence beween wealh oday and omorrow (his is where he di erence equaion aspec comes in) and uses an equaion which relaes curren savings o curren changes in he number of asses. In a nal sep, one will naurally nd ou how he ineres rae appearing in budge consrains relaes o more fundamenal quaniies like value marginal producs and depreciaion raes.

38 Chaper 2. Two-period models and di erence equaions A real budge consrain The budge consrain which resuls depends on he measuremen of wealh. We sar wih he case where we measure wealh in unis of share, or number of machines k : Savings of a household who owns k shares are given by capial income (ne of depreciaion losses) plus labour income minus consumpion expendiure, s w K k v k + w L p c : This is an ideniy resuling from bookkeeping of ows a he household level. Savings in are used for buying new asses in for which he period- price v needs o be paid, s v = k +1 k : (2.5.9) We can rewrie his equaion slighly, which will simplify he inerpreaion of subsequen resuls, as k +1 = (1 ) k + wk k + w L p c v : Wealh in he nex period expressed in number of socks (and hence no in nominal erms) is given by wealh which is lef over from he curren period, (1 ) k, plus new acquisiions of socks which amoun o gross capial plus labour income minus consumpion expendiure divided by he price of one sock. Collecing he k erms and de ning an ineres rae r wk v gives a budge consrain for wealh measured by k ; k +1 = 1 + wk v k + wl v p v c = (1 + r ) k + wl v p v c : (2.5.10) This is a di erence equaion in k bu no ye a di erence equaion in nominal wealh a : Rearranging such ha expendiure is on he lef- and disposable income on he righhand side yields p c + v k +1 = v k + w K v k + w L : This equaion also lends iself o a simple inerpreaion: On he lef-hand side is oal expendiure in period ; consising of consumpion expendiure p c plus expendiure for buying he number of capial goods, k +1, he household wans o hold in + 1: As his expendiure is made in ; oal expendiure for capial goods amouns o v k +1. The righhand side is oal disposable income which splis ino income v k from selling all capial inheried from he previous period, capial income w K v k and labour income w L : This is he form budge consrains are ofen expressed in capial asse pricing models. Noe ha his is in principle also a di erence equaion in k :

2.6. Furher reading and exercises 39 A nominal budge consrain In our one-asse case, nominal wealh a of a household is given by he number k of socks he household owns (say he number of machines i owns) imes he price v of one sock (or machine), a = v k : Compuing he rs di erence yields a +1 a = v +1 k +1 v k = v +1 (k +1 k ) + (v +1 v ) k ; (2.5.11) where he second line added v +1 k v +1 k. Wealh changes depend on he acquisiion v +1 (k +1 k ) of new asses and on changes in he value of asses ha are already held, (v +1 v ) k. Insering (2.5.9) ino (2.5.11) yields a +1 a = v +1 s + (v +1 v ) k v = v +1 w K a a + w L v+1 p c + 1 a, v v v a +1 = v +1 1 + wk a + w L p c : (2.5.12) v v Wha does his equaion ell us? Each uni of wealh a (say Euro, Dollar, Yen...) gives 1 unis a he end of he period as % is los due o depreciaion plus dividend paymens w K =v. Wealh is augmened by labour income minus consumpion expendiure. This end-of-period wealh is expressed in wealh a +1 a he beginning of he nex period by dividing i hrough v (which gives he number k of socks a he end of he period) and muliplying i by he price v +1 of socks in he nex period. We have hereby obained a di erence equaion in a : This general budge consrain is fairly complex, however, which implies ha in pracice i is ofen expressed di erenly. One possibiliy consiss of choosing he capial good as he numeraire good and seing v 1 8: This simpli es (2.5.12) o a +1 = (1 + r ) a + w L p c : (2.5.13) The simpli caion in his expression consiss also in he de niion of he ineres rae r as r w K : 2.6 Furher reading and exercises Raes of subsiuion are discussed in many books on Microeconomics; see e.g. Mas-Colell, Whinson and Green (1995) or Varian (1992). The de niion of he ime preference rae is no very explici in he lieraure. An alernaive formulaion implying he same de niion as he one we use here is used by Buier (1981, p. 773). He de nes he pure rae of ime preference as he marginal rae of subsiuion beween consumpion in wo periods

40 Chaper 2. Two-period models and di erence equaions when equal amouns are consumed in boh periods, minus one. A derivaion of he ime preference rae for a wo-period model is in appendix A.1 of Bossmann, Kleiber and Wälde (2007). The OLG model goes back o Samuelson. For presenaions in exbooks, see e.g. Blanchard and Fischer (1989), Azariadis (1993) or de la Croix and Michel (2002). Applicaions of OLG models are more han numerous. For an example concerning bequess and wealh disribuions, see Bossmann, Kleiber and Wälde (2007). See also Galor and Moav (2006) and Galor and Zeira (1993). The presenaion of he Lagrangian is inspired by Inriligaor (1971, p. 28-30). Treamens of shadow prices are available in many oher exbooks (Dixi, 1989, ch. 4; Inriligaor, 1971, ch. 3.3). More exensive reamens of di erence equaions and he implici funcion heorem can be found in many inroducory mahemaics for economiss books. There is an ineresing discussion on he empirical relevance of exponenial discouning. An early analysis of he implicaions of non-exponenial discouning is by Sroz (1955/56). An overview is provided by Frederick e al. (2002). An analysis using sochasic coninuous ime mehods is by Gong e al. (2007).

2.6. Furher reading and exercises 41 Exercises chaper 2 Applied Ineremporal Opimizaion Opimal consumpion in wo-period discree ime models 1. Opimal choice of household consumpion Consider he following maximizaion problem, subjec o Solve i by using he Lagrangian. max U = v (c ) + 1 c ;c +1 1 + v (c +1) (2.6.1) w + (1 + r) 1 w +1 = c + (1 + r) 1 c +1 : (a) Wha is he opimal consumpion pah? (b) Under wha condiions does consumpion rise? (c) Show ha he rs-order condiions can be wrien as u 0 (c ) =u 0 (c +1 ) = [1 + r] : Wha does his equaion ell you? 2. Solving by subsiuion Consider he maximizaion problem of secion 2.2.1 and solve i by insering. Solve he consrain for one of he conrol variables, inser his ino he objecive funcion and compue rs-order condiions. Show ha he same resuls as in (2.2.4) and (2.2.5) are obained. 3. Capial marke resricions Now consider he following budge consrain. This is a budge consrain ha would be appropriae if you wan o sudy he educaion decisions of households. The parameer b amouns o schooling coss. Inheriance of his individual under consideraion is n. U = ln c + (1 ) ln c +1 subjec o b + n + (1 + r) 1 w +1 = c + (1 + r) 1 c +1 : (a) Wha is he opimal consumpion pro le under no capial marke resricions? (b) Assume loans for nancing educaion are no available, hence savings need o be posiive, s 0. Wha is he consumpion pro le in his case?

42 Chaper 2. Two-period models and di erence equaions 4. Opimal invesmen Consider a monopolis invesing in is echnology. Technology is capured by marginal coss c. The chief accounan of he rm has provided he manager of he rm wih he following informaion, = 1 + R 2 ; = p (x ) x c x I ; c +1 = c f (I 1 ) : Assume you are he manager. Wha is he opimal invesmen sequence I 1, I 2? 5. A paricular uiliy funcion Consider he uiliy funcion U = c + c +1 ; where 0 < < 1: Maximize U subjec o an arbirary budge consrain of your choice. Derive consumpion in he rs and second period. Wha is srange abou his uiliy funcion? 6. Ineremporal elasiciy of subsiuion Consider he uiliy funcion U = c 1 + c 1 +1 : (a) Wha is he ineremporal elasiciy of subsiuion? (b) How can he de niion in (2.2.8) of he elasiciy of subsiuion be ransformed ino he maybe beer known de niion Wha does ij sand for in words? 7. An implici funcion Consider he consrain x 2 x 1 x 1 = b. ij = d ln (c i=c j ) d ln u cj =u ci = p j=p i c i =c j d (c i =c j ) d (p j =p i )? (a) Convince yourself ha his implicily de nes a funcion x 2 = h (x 1 ) : Can he funcion h (x 1 ) be made explici? (b) Convince yourself ha his implicily de nes a funcion x 1 = k (x 2 ) : Can he funcion k (x 2 ) be made explici? (c) Think of a consrain which does no de ne an implici funcion. 8. General equilibrium Consider he Diamond model for a Cobb-Douglas producion funcion of he form Y = K L 1 and a logarihmic uiliy funcion u = ln c y + ln c o +1. (a) Derive he di erence equaion for K : (b) Draw a phase diagram. (c) Wha are he seady sae consumpion level and capial sock?

2.6. Furher reading and exercises 43 9. Sums (a) Proof he saemen of he second lemma in ch. 2.5.1, T i=1ia i = 1 a 1 at 1 a 1 a The idea is idenical o he rs proof in ch. 2.5.1. (b) Show ha k 1 s=0c k 1 s 4 s = ck 4 k c 4 : T a T +1 : Boh parameers obey 0 < c 4 < 1 and 0 < v < 1: Hin: Rewrie he sum as c k 4 1 k s=0 1 (=c 4 ) s and observe ha he rs lemma in ch. 2.5.1 holds for a which are larger or smaller han 1. 10. Di erence equaions Consider he following linear di erence equaion sysem y +1 = a y + b; a < 0 < b; (a) Wha is he xpoin of his equaion? (b) Is his poin sable? (c) Draw a phase diagram.

44 Chaper 2. Two-period models and di erence equaions

Chaper 3 Muli-period models This chaper looks a decision processes where he ime horizon is longer han wo periods. In mos cases, he planning horizon will be in niy. In such a conex, Bellman s opimaliy principle is very useful. Is i, however, no he only way o solve maximizaion problems wih in nie ime horizon? For comparison purposes, we herefore sar wih he Lagrange approach, as in he las secion. Bellman s principle will be inroduced aferwards when inuiion for he problem and relaionships will have been increased. 3.1 Ineremporal uiliy maximizaion 3.1.1 The seup The objecive funcion is given by he uiliy funcion of an individual, where again as in (2.2.12) U = 1 = u (c ) ; (3.1.1) (1 + ) 1 ; > 0 (3.1.2) is he discoun facor and is he posiive ime preference rae. We know his uiliy funcion already from he de niion of he ime preference rae, see (2.2.12). The uiliy funcion is o be maximized subjec o a budge consrain. The di erence o he formulaion in he las secion is ha consumpion does no have o be deermined for wo periods only bu for in niely many. Hence, he individual does no choose one or wo consumpion levels bu an enire pah of consumpion. This pah will be denoed by fc g : As ; fc g is a shor form of fc ; c +1 ; :::g : Noe ha he uiliy funcion is a generalizaion of he one used above in (2.1.1), bu is assumed o be addiively separable. The corresponding wo period uiliy funcion was used in exercise se 1, cf. equaion (2.6.1). The budge consrain can be expressed in he ineremporal version by 1 = (1 + r) ( ) e = a + 1 = (1 + r) ( ) w ; (3.1.3) 45

46 Chaper 3. Muli-period models where e = p c : I saes ha he presen value of expendiure equals curren wealh a plus he presen value of labour income w. Labour income w and he ineres rae r are exogenously given o he household, is wealh level a is given by hisory. The only quaniy ha is lef o be deermined is herefore he pah fc g : Maximizing (3.1.1) subjec o (3.1.3) is a sandard Lagrange problem. 3.1.2 Solving by he Lagrangian The Lagrangian reads L = 1 = u (c ) + h 1= (1 + r) ( ) e a 1= (1 + r) ( ) w i ; where is he Lagrange muliplier. Firs-order condiions are L c = u 0 (c ) + [1 + r] ( ) p = 0; < 1; (3.1.4) L = 0; (3.1.5) where he laer is, as in he OLG case, he budge consrain. Again, we have as many condiions as variables o be deermined: here are in niely many condiions in (3.1.4), one for each c and one condiion for in (3.1.5). Do hese rs-order condiions ell us somehing? Take he rs-order condiion for period and for period + 1. They read Dividing hem gives u 0 (c ) = [1 + r] +1 u 0 (c +1 ) = [1 + r] ( ) p ; (+1 ) p +1 : 1 u 0 (c ) u 0 (c +1 ) = (1 + r) p p +1 () u0 (c ) u 0 (c +1 ) = p (1 + r) 1 p +1 : (3.1.6) Rearranging allows us o see an inuiive inerpreaion: Comparing he insananeous gain in uiliy u 0 (c ) wih he fuure gain, discouned a he ime preference rae, u 0 (c +1 ) ; mus yield he same raio as he price p ha has o be paid oday relaive o he price ha has o be paid in he fuure, also appropriaely discouned o is presen value price (1 + r) 1 p +1 : This inerpreaion is idenical o he wo-period inerpreaion in (2.2.6) in ch. 2.2.2. If we normalize prices o uniy, (3.1.6) is jus he expression we obained in he soluion for he wo-period maximizaion problem in (2.6.1). 3.2 The envelope heorem We saw how he Lagrangian can be used o solve opimizaion problems wih many ime periods. In order o undersand how dynamic programming works, i is useful o undersand a heorem which is frequenly used when employing he dynamic programming mehod: he envelope heorem.

3.2. The envelope heorem 47 3.2.1 The heorem In general, he envelope heorem says Theorem 3.2.1 Le here be a funcion g (x; y) : Choose x such ha g (:) is maximized for a given y: (Assume g (:) is such ha a smooh inerior soluion exiss.) Le f(y) be he resuling funcion of y; f (y) max g (x; y) : x Then he derivaive of f wih respec o y equals he parial derivaive of g wih respec o y, if g is evaluaed a ha x = x(y) ha maximizes g; d f (y) d y = @ g (x; y) @ y : x=x(y) Proof. f (y) is consruced by @ g (x; y) = 0: This implies a cerain x = x (y) ; @x provided ha second order condiions hold. Hence, f (y) = max x g (x; y) = g (x (y) ; y) : : The rs erm of he rs erm is zero. Then, d f(y) d y = @ g(x(y);y) @ x d x(y) d y + @ g(x(y);y) @ y 3.2.2 Illusraion The plane depics he funcion g (x; y). The maximum of his funcion wih respec o x is shown as max x g (x; y), which is f (y). Given his gure, i is obvious ha he derivaive of f (y) wih respec o y is he same as he parial derivaive of g (:) wih respec o y a he poin where g (:) has is maximum wih respec o x: The parial derivaive @g @y is he derivaive when going in he direcion of y. Choosing he highes poin of g (:) wih respec o x, his direcional derivaive mus be he same as df(y) a he back of he dy gure.

48 Chaper 3. Muli-period models f( y) gxy (, ) ( ) max gxy, x x ( ) g( x( y), y) dx g( x( y), y) df y dy = + x dy g( x( y), y = ) y y y Figure 3.2.1 Illusraing he envelope heorem 3.2.3 An example There is a cenral planner of an economy. The social welfare funcion is given by U (A; B), where A and B are consumpion goods. The echnologies available for producing hese goods are A = A (cl A ) and B = B (L B ) : The amoun of labour used for producing one or he oher good is denoed by L A and L B and c is a produciviy parameer in secor A: The economy s resource consrain is L A + L B = L: The planner is ineresed in maximizing he social welfare level and allocaes labour according o max LA U (A (cl A ) ; B (L L A )) : The opimaliy condiion is @U @A A0 c @U @B B0 = 0: (3.2.1) This makes opimal employmen L A a funcion of c, L A = L A (c) : (3.2.2) The cenral planner now asks wha happens o he social welfare level when he echnology parameer c increases and she sill maximizes he social welfare. The laer requires ha (3.2.1) coninues o hold and he maximized social welfare funcion wih (3.2.2) and

3.3. Solving by dynamic programming 49 he resource consrain reads U (A (cl A (c)) ; B (L heorem, he answer is L A (c))). Wihou using he envelope d dc U (A (cl A (c)) ; B (L L A (c))) @U (:) = @A A0 [L A (c) + cl 0 @U (:) A (c)] + @B B0 [ L 0 A (c)] = @U (:) @A A0 L A (c) > 0; where he las equaliy follows from insering he opimaliy condiion (3.2.1). Economically, his resul means ha he e ec of beer echnology on overall welfare is given by he direc e ec in secor A: The indirec e ec hrough he reallocaion of labour vanishes as, due o he rs-order condiion (3.2.1), he marginal conribuion of each worker is idenical across secors. Clearly, his only holds for a small change in c: If one is ineresed in nding an answer by using he envelope heorem, one would sar by de ning a funcion V (c) max LA U (A (cl A ) ; B (L L A )) : Then, according o he envelope heorem, d dc V (c) = @ @c U (A (cl A) ; B (L L A )) = @U (:) @A A0 L A LA =L A (c) = LA =L A (c) @U (:) @A A0 L A (c) > 0: Apparenly, boh approaches yield he same answer. Applying he envelope heorem gives he answer faser. 3.3 Solving by dynamic programming 3.3.1 The seup We will now ge o know how dynamic programming works. Le us sudy a maximizaion problem which is similar o he one in ch. 3.1.1. We will use he same uiliy funcion as in (3.1.1), reproduced here for convenience, U = 1 = u (c ). The consrain, however, will be represened in a more general way han in (3.1.3). We sipulae ha here is a variable x which evolves according o x +1 = f (x ; c ) : (3.3.1) This variable x could represen wealh and his consrain could hen represen he budge consrain of he household. This di erence equaion could also be non-linear, however, as for example in a cenral planner problem where he consrain is a resource consrain as in (3.9.3). In his case, x would sand for capial. Anoher sandard example for x as a sae variable would be environmenal qualiy. Here we will rea he general case rs before we go on o more speci c examples furher below.

50 Chaper 3. Muli-period models The consumpion level c and - more generally speaking - oher variables whose value is direcly chosen by individuals, e.g. invesmen levels or shares of wealh held in di eren asses, are called conrol variables. Variables which are no under he direc conrol of individuals are called sae variables. In many maximizaion problems, sae variables depend indirecly on he behaviour of individuals as in (3.3.1). Sae variables can also be compleely exogenous like for example he TFP level in an exogenous growh model or prices in a household maximizaion problem. Opimal behaviour is de ned by max fc g U subjec o (3.3.1), i.e. he highes value U can reach by choosing a sequence fc g fc ; c +1 ; :::g and by saisfying he consrain (3.3.1). The value of his opimal behaviour or opimal program is denoed by V (x ) max U subjec o x +1 = f (x ; c ) : (3.3.2) fc g V (x ) is called he value funcion. I is a funcion of he sae variable x and no of he conrol variable c. The laer poin is easy o undersand if one akes ino accoun ha he conrol variable c is, when behaving opimally, a funcion of he sae variable x : The value funcion V (:) could also be a funcion of ime (e.g. in problems wih nie horizon) bu we will no discuss his furher as i is of no imporance in he problems we will encouner. Generally speaking, x and c could be vecors and ime could hen be par of he sae vecor x : The value funcion is always a funcion of he saes of he sysem or of he maximizaion problem. 3.3.2 Three dynamic programming seps Given his descripion of he maximizaion problem, solving by dynamic programming essenially requires us o go hrough hree seps. This hree-sep approach will be followed here, laer in coninuous ime, and also in models wih uncerainy. DP1: Bellman equaion and rs-order condiions The rs sep esablishes he Bellman equaion and compues rs-order condiions. The objecive funcion U in (3.1.1) is addiively separable which means ha i can be wrien in he form U = u(c ) + U +1 : (3.3.3) Bellman s idea consiss of rewriing he maximizaion problem in he opimal program (3.3.2) as V (x ) max fu(c ) + V (x +1 )g (3.3.4) c subjec o x +1 = f (x ; c ) : Equaion (3.3.4) is known as he Bellman equaion. In his equaion, he problem wih poenially in niely many conrol variables fc g was broken down in many problems

3.3. Solving by dynamic programming 51 wih one conrol variable c. Noe ha here are wo seps involved: Firs, he addiive separabiliy of he objecive funcion is used. Second, more imporanly, U +1 is replaced by V (x +1 ). This says ha we assume ha he opimal problem for omorrow is solved and we should worry abou he maximizaion problem of oday only. We can now compue he rs-order condiion which is of he form u 0 (c ) + V 0 (x +1 ) @f (x ; c ) @c = 0: (3.3.5) I ells us ha increasing consumpion c has advanages and disadvanages. The advanages consis in higher uiliy oday, which is re eced here by marginal uiliy u 0 (c ) : The disadvanages come from lower overall uiliy - he value funcion V - omorrow. The reducion in overall uiliy amouns o he change in x +1 ; i.e. he derivaive @f (x ; c ) =@c ; imes he marginal value of x +1 ; i.e. V 0 (x +1 ) : As he disadvanage arises only omorrow, his is discouned a he rae : One can alk of a disadvanage of higher consumpion oday on overall uiliy omorrow as he derivaive @f (x ; c ) =@c needs o be negaive, oherwise an inerior soluion as assumed in (3.3.5) would no exis. In principle, his is he soluion of our maximizaion problem. Our conrol variable c is by his expression implicily given as a funcion of he sae variable, c = c (x ) ; as x +1 by he consrain (3.3.1) is a funcion of x and c. As all sae variables in are known, he conrol variable is deermined by his opimaliy condiion. Hence, as V is well-de ned above, we have obained a soluion. As we know very lile abou he properies of V a his sage, however, we need o go hrough wo furher seps in order o eliminae V (o be precise, is derivaive V 0 (x +1 ) ; i.e. he cosae variable of x +1 ) from his rs-order condiion and obain a condiion ha uses only funcions (like e.g. he uiliy funcion or he echnology for producion in laer examples) of which properies like signs of rs and second derivaives are known. We obain more informaion abou he evoluion of his cosae variable in he second dynamic programming sep. DP2: Evoluion of he cosae variable The second sep of he dynamic programming approach sars from he maximized Bellman equaion. The maximized Bellman equaion is obained by replacing he conrol variables in he Bellman equaion, i.e. he c in (3.3.4), in he presen case, wih he opimal conrol variables as given by he rs-order condiion, i.e. by c (x ) resuling from (3.3.5). Logically, he max operaor disappears (as we inser he c (x ) which imply a maximum) and he maximized Bellman equaion reads V (x ) = u (c (x )) + V (f (x ; c(x ))) : The derivaive wih respec o x reads V 0 (x ) = u 0 (c (x )) dc (x ) + V 0 dc (x ) (f (x ; c (x ))) f x + f c : dx dx

52 Chaper 3. Muli-period models This sep shows why i is imporan ha we use he maximized Bellman equaion here: Now conrol variables are a funcion of sae variable and we need o compue he derivaive of c wih respec o x when compuing he derivaive of he value funcion V (x ) : Insering he rs-order condiion simpli es his equaion o V 0 (x ) = V 0 (f (x ; c (x ))) f x = V 0 (x +1 ) f x (3.3.6) This equaion is a di erence equaion for he cosae variable, he derivaive of he value funcion wih respec o he sae variable, V 0 (x ). The cosae variable is also called he shadow price of he sae variable x : If we had more sae variables, here would be a cosae variable for each sae variable. I says how much an addiional uni of he sae variable (say e.g. of wealh) is valued: As V (x ) gives he value of opimal behaviour beween and he end of he planning horizon, V 0 (x ) says by how much his value changes when x is changed marginally. Hence, equaion (3.3.6) describes how he shadow price of he sae variable changes over ime when he agen behaves opimally. If we had used he envelope heorem, we would have immediaely ended up wih (3.3.6) wihou having o inser he rs-order condiion. DP3: Insering rs-order condiions Now inser he rs-order condiion (3.3.5) wice ino (3.3.6) o replace he unknown shadow price and o nd an opimaliy condiion depending on u only. This will hen be he Euler equaion. We do no do his here explicily as many examples will go hrough his sep in deail in wha follows. 3.4 Examples 3.4.1 Ineremporal uiliy maximizaion wih a CES uiliy funcion The individual s budge consrain is given in he dynamic formulaion a +1 = (1 + r ) (a + w c ) : (3.4.1) Noe ha his dynamic formulaion corresponds o he ineremporal version in he sense ha (3.1.3) implies (3.4.1) and (3.4.1) wih some limi condiion implies (3.1.3). This will be shown formally in ch. 3.5.1. The budge consrain (3.4.1) can be found in many papers and also in some exbooks. The iming as implici in (3.4.1) is illusraed in he following gure. All evens ake place a he beginning of he period. Our individual owns a cerain amoun of wealh a a he beginning of and receives here wage income w and spends c on consumpion also a he beginning. Hence, savings s can be used during for producion and ineres is paid on s which in urn gives a +1 a he beginning of period + 1.

3.4. Examples 53 a + 1 = 1 + r s c +1 s = a + w c ( ) c +1 Figure 3.4.1 The iming in an in nie horizon discree ime model The consisency of (3.4.1) wih echnologies in general equilibrium is no self-eviden. We will encouner more convenional budge consrains of he ype (2.5.13) furher below. As (3.4.1) is widely used, however, we now look a dynamic programming mehods and ake his budge consrain as given. The objecive of he individual is o maximize her uiliy funcion (3.1.1) subjec o he budge consrain by choosing a pah of consumpion levels c ; denoed by fc g ; 2 [; 1] : We will rs solve his wih a general insananeous uiliy funcion and hen inser he CES version of i, i.e. 1 u (c ) = c1 1 : (3.4.2) The value of he opimal program fc g is, given is iniial endowmen wih a, de ned as he maximum which can be obained subjec o he consrain, i.e. V (a ) max U (3.4.3) fc g subjec o (3.4.1). I is called he value funcion. Is only argumen is he sae variable a : See ch. 3.4.2 for a discussion on sae variables and argumens of value funcions. DP1: Bellman equaion and rs-order condiions We know ha he uiliy funcion can be wrien as U = u (c ) + U +1 : Now assume ha he individual behaves opimally as from +1: Then we can inser he value funcion. The uiliy funcion reads U = u (c ) + V (a +1 ) : Insering his ino he value funcion, we obain he recursive formulaion V (a ) = max c fu (c ) + V (a +1 )g ; (3.4.4) known as he Bellman equaion. Again, his breaks down a many-period problem ino a wo-period problem: The objecive of he individual was max fc g (3.1.1) subjec o (3.4.1), as shown by he value funcion in equaion (3.4.3). The Bellman equaion (3.4.4), however, is a wo period decision problem, he rade-o beween consumpion oday and more wealh omorrow (under he assumpion ha he funcion V is known). This is wha is known as Bellman s

54 Chaper 3. Muli-period models principle of opimaliy: Whaever he decision oday, subsequen decisions should be made opimally, given he siuaion omorrow. Hisory does no coun, apar from is impac on he sae variable(s). We now derive a rs-order condiion for (3.4.4). I reads d u (c ) + d V (a +1 ) = u 0 (c ) + V 0 (a +1 ) da +1 = 0: dc dc dc Since da +1 =dc = (1 + r ) by he budge consrain (3.4.1), his gives u 0 (c ) (1 + r ) V 0 (a +1 ) = 0: (3.4.5) Again, his equaion makes consumpion a funcion of he sae variable, c = c (a ) : Following he rs-order condiion (3.3.5) in he general example, we wroe c = c (x ) ; i.e. consumpion c changes only when he sae variable x changes. Here, we wrie c = c (a ) ; indicaing ha here can be oher variables which can in uence consumpion oher han wealh a : An example for such an addiional variable in our seup would be he wage rae w or ineres rae r ; which afer all is visible in he rs-order condiion (3.4.5). See ch. 3.4.2 for a more deailed discussion of sae variables. Economically, (3.4.5) ells us as before in (3.3.5) ha, under opimal behaviour, gains from more consumpion oday are jus balanced by losses from less wealh omorrow. Wealh omorrow falls by 1 + r, his is evaluaed by he shadow price V 0 (a +1 ) and everyhing is discouned by :, DP2: Evoluion of he cosae variable Using he envelope heorem, he derivaive of he maximized Bellman equaion reads V 0 (a ) = V 0 (a +1 ) @a +1 @a : (3.4.6) We compue he parial derivaive of a +1 wih respec o a as he funcional relaionship of c = c (a ) should no (because of he envelope heorem) be aken ino accoun. From he budge consrain we know ha @a +1 @a = 1 + r : Hence, he evoluion of he shadow price/ he cosae variable under opimal behaviour is described by This is he analogon o (3.3.6). DP3: Insering rs-order condiions V 0 (a ) = [1 + r ] V 0 (a +1 ) : Le us now be explici abou how o inser rs-order condiions ino his equaion. We can inser he rs-order condiion (3.4.5) on he righ-hand side. We can also rewrie he

3.4. Examples 55 rs-order condiion (3.4.5), by lagging i by one period, as (1 + r 1 ) V 0 (a ) = u 0 (c 1 ) and can inser his on he lef-hand side. This gives u 0 (c 1 ) 1 (1 + r 1 ) 1 = u 0 (c ), u 0 (c ) = [1 + r ] u 0 (c +1 ) : (3.4.7) This is he same resul as he one we obained when we used he Lagrange mehod in equaion (3.1.6). I is also he same resul as for he wo-period saving problem which we found in OLG models - see e.g. (2.2.6) or (2.6.1) in he exercises. This migh be surprising as he planning horizons di er considerably beween a 2- and an in nie-period decision problem. Apparenly, wheher we plan for wo periods or for many more, he change beween wo periods is always he same when we behave opimally. I should be kep in mind, however, ha consumpion levels (and no changes) do depend on he lengh of he planning horizon. The CES and logarihmic version of he Euler equaion Le us now inser he CES uiliy funcion from (3.4.2) ino (3.4.7). Compuing marginal uiliy gives u 0 (c ) = c and we obain a linear di erence equaion in consumpion, c +1 = ( [1 + r ]) 1= c : (3.4.8) Noe ha he logarihmic uiliy funcion u (c ) = ln c ; known for he wo-period seup from (2.2.1), is a special case of he CES uiliy funcion (3.4.2). Leing approach uniy, we obain c 1 1!1 lim u (c ) = lim!1 1 = ln c where he las sep used L Hôspial s rule: The derivaive of he numeraor wih respec o is Hence, d d c1 1 = d d e(1 ) ln c 1 = e (1 ) ln c ( ln c ) = c 1 ( ln c ) : lim!1 c 1 1 1 = lim!1 c 1 ( ln c ) 1 = ln c : (3.4.9) When he logarihmic uiliy funcion is insered ino (3.4.7), one obains an Euler equaion as in (3.4.8) wih se equal o one. 3.4.2 Wha is a sae variable? Dynamic programming uses he concep of a sae variable. In he general version of ch. 3.3, here is clearly only one sae variable. I is x and is evoluion is described in (3.3.1). In he economic example of ch. 3.4.1, he quesion of wha is a sae variable is less obvious.

56 Chaper 3. Muli-period models In a sric formal sense, everyhing is a sae variable. Everyhing means all variables which are no conrol variables are sae variables. This very broad view of sae variables comes from he simple de niion ha everyhing (apar from parameers) which deermines conrol variables is a sae variable. We can undersand his view by looking a he explici soluion for he conrol variables in he wo-period example of ch. 2.2.1. We reproduce (2.2.3), (2.2.4) and (2.2.5) for ease of reference, W = w + (1 + r +1 ) 1 w +1 ; c +1 = (1 ) (1 + r +1 ) W ; c = W : We did no use he erms conrol and sae variable here bu we could of course solve his wo-period problem by dynamic programming as well. Doing so would allow us o undersand why everyhing is a sae variable. Looking a he soluion for c +1 shows ha i is a funcion of r +1 ; w and w +1 : If we wan o make he saemen ha he conrol variable is a funcion of he sae variables, hen clearly r +1 ; w and w +1 are sae variables. Generalizing his for our muli-period example from ch. 3.4.1, he enire pahs of r and w are sae variables, in addiion o wealh a : As we are in a deerminisic world, we know he evoluion of variables r and w and we can reduce he pah of r and w by he levels of r and w plus he parameers of he process describing heir evoluion. Hence, he broad view for sae variables applied o ch. 3.4.1 requires us o use r ; w ; a as sae variables. This broad (and ulimaely correc) view of sae variables is he reason why he rsorder condiion (3.4.5) is summarized by c = c (a ) : The index capures all variables which in uence he soluion for c apar from he explici argumen a : In a more pracical sense - as opposed o he sric sense - i is highly recommended o consider only he variable which is indirecly a eced by he conrol variable as (he relevan) sae variable. Wriing he value funcion as V = V (a ; w ; r ) is possible bu highly cumbersome from a noaional poin of view. Wha is more, going hrough he dynamic programming seps does no require more han a as a sae variable as only he shadow price of a is required o obain an Euler equaion and no he shadow price of w or r. To remind us ha more han jus a has an impac on opimal conrols, we should, however, always wrie c = c (a ) as a shorcu for c = c (a ; w ; r ; :::) : The conclusion of all his, however, is more cauious: When encounering a new maximizaion problem and when here is uncerainy abou how o solve i and wha is a sae variable and wha is no, i is always he bes choice o include more raher han less variables as argumens of he value funcion. Dropping some argumens aferwards is simpler han adding addiional ones.

3.4. Examples 57 3.4.3 Opimal R&D e or In his second example, a research projec has o be nished a some fuure known poin in ime T. This research projec has a cerain value a poin T and we denoe i by D like disseraion. In order o reach his goal, a pah of a cerain lengh L needs o be compleed. We can hink of L as a cerain number of pages, a cerain number of papers or - probably beer - a cerain qualiy of a xed number of papers. Going hrough his process of walking and wriing is cosly, i requires e or e a each poin in ime : Summing over hese cos - hink of hem as hours worked per day - evenually yields he desired amoun of pages, T =f (e ) = L; (3.4.10) where f(:) is he page of qualiy producion funcion: More e or means more oupu, f 0 (:) > 0; bu any increase in e or implies a lower increase in oupu, f 00 (:) < 0: The objecive funcion of our suden is given by U = T D T = e : (3.4.11) The value of he compleed disseraion is given by D and is presen value is obained by discouning a he discoun facor : Toal uiliy U sems from his presen value minus he presen value of research cos e. The maximizaion problem consiss in maximizing (3.4.11) subjec o he consrain (3.4.10) by choosing an e or pah fe g : The quesion now arises how hese coss are opimally spread over ime. How many hours should be worked per day? An answer can be found by using he Lagrange-approach wih (3.4.11) as he objecive funcion and (3.4.10) as he consrain. However, her we will use he dynamic programming approach. Before we can apply i, we need o derive a dynamic budge consrain. We herefore de ne M 1 =1f(e ) as he amoun of he pages ha have already been wrien by oday. This hen implies M +1 M = f(e ): (3.4.12) The increase in he number of compleed pages beween oday and omorrow depends on e or-induced oupu f (e ) oday. We can now apply he hree dynamic programming seps. DP1: Bellman equaion and rs-order condiions The value funcion can be de ned by V (M ) max fe g U subjec o he consrain. We follow he approach discussed in ch. 3.4.2 and explicily use as sae variable M only, he only sae variable relevan for derivaions o come. In oher words, we explicily suppress ime as an argumen of V (:) : The reader can go hrough he derivaions by

58 Chaper 3. Muli-period models using a value funcion speci ed as V (M ; ) and nd ou ha he same resul will obain. The objecive funcion U wrien recursively reads U = T (+1) D 1 T= e = T (+1) D 1 e + T=+1 e = T (+1) D T=+1 (+1) e e = U +1 e : Assuming ha he individual behaves opimally as from omorrow, his reads U = e + V (M +1 ) and he Bellman equaion reads V (M ) = max e f e + V (M +1 )g : (3.4.13) The rs-order condiion is 1 + V 0 (M +1 ) dm +1 de = 0; which, using he dynamic budge consrain, becomes 1 = V 0 (M +1 ) f 0 (e ) : (3.4.14) Again as in (3.3.5), implicily and wih (3.4.12), his equaion de nes a funcional relaionship beween he conrol variable and he sae variable, e = e (M ) : One uni of addiional e or reduces insananeous uiliy by 1 bu increases he presen value of overall uiliy omorrow by V 0 (M +1 ) f 0 (e ) : DP2: Evoluion of he cosae variable To provide some variaion, we will now go hrough he second sep of dynamic programming wihou using he envelope heorem. Consider he maximized Bellman equaion, where we inser e = e (M ) and (3.4.12) ino he Bellman equaion (3.4.13), The derivaive wih respec o M is V (M ) = e (M ) + V (M + f(e (M ))) : V 0 (M ) = e 0 (M ) + V 0 (M + f(e (M ))) d [M + f (e (M ))] dm = e 0 (M ) + V 0 (M +1 ) [1 + f 0 (e (M )) e 0 (M )] : Using he rs-order condiion (3.4.14) simpli es his derivaive o V 0 (M ) = V 0 (M +1 ) : Expressed for + 1 gives V 0 (M +1 ) = V 0 (M +2 ) (3.4.15) DP3: Insering rs-order condiions The nal sep insers he rs-order condiion (3.4.14) wice o replace V 0 (M +1 ) and V 0 (M +2 ) ; 1 (f 0 (e )) 1 = (f 0 (e +1 )) 1, f 0 (e +1 ) = : (3.4.16) f 0 (e ) The inerpreaion of his Euler equaion is now simple. As f 00 (:) < 0 and < 1; e or e increases under opimal behaviour, i.e. e +1 > e : Opimal wriing of a disseraion implies more work every day.

3.5. On budge consrains 59 Wha abou levels? The opimaliy condiion in (3.4.16) speci es only how e or e changes over ime, i does no provide informaion on he level of e or required every day. This is a propery of all expressions based on rs-order condiions of ineremporal problems. They only give informaion abou changes of levels, no abou levels hemselves. However, he basic idea for how o obain informaion abou levels can be easily illusraed. Assume f (e ) = e ; wih 0 < < 1: Then (3.4.16) implies (wih being replaced by ) e +1=e 1 1 =, e +1 = 1=(1 ) e : Solving his di erence equaion yields e = ( 1)=(1 ) e 1 ; (3.4.17) where e 1 is he (a his sage sill) unknown iniial e or level. Saring in = 1 on he rs day, insering his soluion ino he ineremporal consrain (3.4.10) yields T =1f ( 1)=(1 ) e 1 = T =1 ( 1)=(1 ) e 1 = L: This gives us he iniial e or level as (he sum can be solved by using he proofs in ch. 2.5.1) 1= L e 1 = : T =1 ( 1)=(1 ) Wih (3.4.17), we have now also compued he level of e or every day. Behind hese simple seps, here is a general principle. Modi ed rs-order condiions resuling from ineremporal problems are di erence equaions, see for example (2.2.6), (3.1.6), (3.4.7) or (3.4.16) (or di erenial equaions when we work in coninuous ime laer). Any di erence (or also di erenial) equaion when solved gives a unique soluion only if an iniial or erminal condiion is provided. Here, we have solved he di erence equaion in (3.4.16) assuming some iniial condiion e 1. The meaningful iniial condiion hen followed from he consrain (3.4.10). Hence, in addiion o he opimaliy rule (3.4.16), we always need some addiional consrain which allows us o compue he level of opimal behaviour. We reurn o his poin when looking a problems in coninuous ime in ch. 5.4. 3.5 On budge consrains We have encounered wo di eren (bu relaed) ypes of budge consrains so far: dynamic ones and ineremporal ones. Consider he dynamic budge consrain derived in (2.5.13) as an example. Using e p c for simpliciy, i reads a +1 = (1 + r ) a + w e : (3.5.1) This budge consrain is called dynamic as i only akes wha happens beween he wo periods and + 1;ino accoun. In conras, an ineremporal budge consrain akes

60 Chaper 3. Muli-period models wha happens beween any saring period (usually ) and he end of he planning horizon ino accoun. In his sense, he ineremporal budge consrain is more comprehensive and conains more informaion (as we will also see formally below whenever we alk abou he no-ponzi game condiion). An example for an ineremporal budge consrain was provided in (3.1.3), replicaed here for ease of reference, 1 = (1 + r) ( ) e = a + 1 = (1 + r) ( ) w : (3.5.2) 3.5.1 From ineremporal o dynamic We will now ask abou he link beween dynamic and ineremporal budge consrains. Le us choose he simpler link o sar wih, i.e. he link from he ineremporal o he dynamic version. As an example, ake (3.5.2). We will now show ha his ineremporal budge consrain implies which was used before, for example in (3.4.1). Wrie (3.5.2) for he nex period as Express (3.5.2) as a +1 = (1 + r ) (a + w c ) ; (3.5.3) 1 =+1 (1 + r) ( 1) e = a +1 + 1 =+1 (1 + r) ( 1) w : (3.5.4) e + 1 =+1 (1 + r) ( ) e = a + w + 1 =+1 (1 + r) ( ) w, e + (1 + r) 1 1 =+1 (1 + r) ( 1) e = a + w + (1 + r) 1 1 =+1 (1 + r) ( 1) w, 1 =+1 (1 + r) ( 1) e = (1 + r) (a + w e ) + 1 =+1 (1 + r) ( 1) w : Inser (3.5.4) and nd he dynamic budge consrain (3.5.3). 3.5.2 From dynamic o ineremporal Le us now ask abou he link from he dynamic o he ineremporal budge consrain. How can we obain he ineremporal version of he budge consrain (3.5.1)? Technically speaking, his simply requires us o solve a di erence equaion: In order o solve (3.5.1) recursively, we rewrie i as w a = a +1 + e ; a +i = a +i+1 + e +i w +i : 1 + r 1 + r +i

3.5. On budge consrains 61 Insering su cienly ofen yields a = = = a +2 +e +1 w +1 1+r +1 + e w = a +2 + e +1 w +1 1 + r (1 + r +1 ) (1 + r ) + e w 1 + r a +3 +e +2 w +2 1+r +2 + e +1 w +1 (1 + r +1 ) (1 + r ) + e w 1 + r a +3 + e +2 w +2 (1 + r +2 ) (1 + r +1 ) (1 + r ) + e +1 w +1 (1 + r +1 ) (1 + r ) + e w 1 + r a +i = ::: = lim i!1 (1 + r +i 1 ) (1 + r +1 ) (1 + r ) + 1 i=0 (1 + r +i ) (1 + r +1 ) (1 + r ) : The expression in he las line is hopefully insrucive bu somewha cumbersome. We can wrie i in a more concise way as a = lim a +i i!1 i 1 e +i w +i e +i s=0 (1 + r +s ) + 1 i=0 i s=0 (1 + r +s ) where indicaes a produc, i.e. i s=0 (1 + r +i ) = 1 + r for i = 0 and i s=0 (1 + r +i ) = (1 + r +i ) (1 + r ) for i > 0: For i = 1; i s=0 (1 + r +i ) = 1 by de niion. Leing he limi be zero, a sep explained in a second, we obain e +i w +i a = 1 i=0 i s=0 (1 + r +s ) = 1 = e w s=0 (1 + r +s ) 1 = w +i e w R where he las bu one equaliy is subsiued + i by : We can wrie his as 1 e = Wih a consan ineres rae, his reads w = a + 1 = : (3.5.5) R R 1 = (1 + r) ( +1) e = a + 1 = (1 + r) ( +1) w : (3.5.6) Equaion (3.5.5) is he ineremporal budge consrain ha resuls from a dynamic budge consrain as speci ed in (3.5.1) using he addiional condiion ha a lim +i i!1 = 0. i 1 s=0 (1+r +s) Noe ha he assumpion ha his limi is zero has a sandard economic inerpreaion. I is usually called he no-ponzi game condiion. To undersand he inerpreaion more easily, jus focus on he case of a consan ineres rae. The condiion hen reads lim i!1 a +i = (1 + r) i = 0: The erm a +i = (1 + r) i is he presen value in of wealh a +i held in + i: The condiion says ha his presen value mus be zero. Imagine an individual ha nances expendiure e by increasing deb, i.e. by leing a +i becoming more and more negaive. This condiion simply says ha an individual s long-run deb level, i.e. a +i for i going o in niy mus no increase oo quickly - he presen value mus be zero. Similarly, he condiion also requires ha an individual should no hold posiive wealh in he long run whose presen value is no zero. Noe ha his condiion is ful lled, for example, for any consan deb or wealh level.

62 Chaper 3. Muli-period models 3.5.3 Two versions of dynamic budge consrains Noe ha we have also encounered wo subspecies of dynamic budge consrains. The one from (3.5.1) and he one from (3.5.3). The di erence beween hese wo consrains is due o more basic assumpions abou he iming of evens as was illusraed in g.s 2.1.1 and 3.4.1. These wo dynamic consrains imply wo di eren versions of ineremporal budge consrains. The version from (3.5.1) leads o (3.5.6) and he one from (3.5.3) leads (wih a similar no-ponzi game condiion) o (3.5.2). Comparing (3.5.6) wih (3.5.2) shows ha he presen values on boh sides of (3.5.6) discouns one ime more han in (3.5.2). The economic di erence again lies in he iming, i.e. wheher we look a values a he beginning or end of a period. The budge consrain (3.5.1) is he naural budge consrain in he sense ha i can be derived easily as above in ch. 2.5.5 and in he sense ha i easily aggregaes o economy wide resource consrains. We will herefore work wih (3.5.1) and he corresponding ineremporal version (3.5.6) in wha follows. The reason for no working wih hem righ from he beginning is ha he ineremporal version (3.5.2) has some inuiive appeal and ha is dynamic version (3.5.3) is widely used in he lieraure. 3.6 A decenralized general equilibrium analysis We have so far analyzed maximizaion problems of households in parial equilibrium. In wo-period models, we have analyzed how households can be aggregaed and wha we learn abou he evoluion of he economy as a whole. We will now do he same for in nie horizon problems. As we did in ch. 2.4, we will rs specify echnologies. This shows wha is echnologically feasible in his economy. Which goods are produced, which goods can be sored for saving purposes, is here uncerainy in he economy semming from producion processes? Given hese echnologies, rms maximize pro s. Second, household preferences are presened and he budge consrain of households is derived from he echnologies presened before. This is he reason why echnologies should be presened before households are inroduced: budge consrains are endogenous and depend on knowledge of wha households can do. Opimaliy condiions for households are hen derived. Finally, aggregaion over households and an analysis of properies of he model using he reduced form follows. 3.6.1 Technologies The echnology is a simple Cobb-Douglas echnology Y = AK L 1 : (3.6.1) Capial K and labour L is used wih a given oal facor produciviy level A o produce oupu Y : This good can be used for consumpion and invesmen and equilibrium on he

3.6. A decenralized general equilibrium analysis 63 goods marke requires Y = C + I : (3.6.2) Gross invesmen I is urned ino ne invesmen by aking depreciaion ino accoun, K +1 = (1 ) K + I : Taking hese wo equaions ogeher gives he resource consrain of he economy, K +1 = (1 ) K + Y C : (3.6.3) As his consrain is simply a consequence of echnologies and marke clearing, i is idenical o he one used in he OLG seup in (2.4.9). 3.6.2 Firms Firms maximize pro s by employing opimal quaniies of labour and capial, given he echnology in (3.6.1). Firs-order condiions are @Y @K = w K ; @Y @L = wl (3.6.4) as in (2.4.2), where we have again chosen he consumpion good as numeraire. 3.6.3 Households Preferences of households are described as in he ineremporal uiliy funcion (3.1.1). As he only way households can ransfer savings from one period o he nex is by buying invesmen goods, an individual s wealh is given by he number of machines k, she owns. Clearly, adding up all individual wealh socks gives he oal capial sock, L k = K : Wealh k increases over ime if he household spends less on consumpion han wha i earns hrough capial plus labour income, correced for he loss in wealh each period caused by depreciaion, k +1 k = w K k k + w L c, k +1 = 1 + w K k + w L c : If we now de ne he ineres rae o be given by we obain our budge consrain r w K ; (3.6.5) k +1 = (1 + r ) k + w L c : (3.6.6) Noe ha he derivaion of his budge consrain was simpli ed in comparison o ch. 2.5.5 as he price v of an asse is, as we measure i in unis of he consumpion good which is raded on he same nal marke (3.6.2), given by 1. More general budge consrains will become prey complex as soon as he price of he asse is no normalized.

64 Chaper 3. Muli-period models This complexiy is needed when i comes e.g. o capial asse pricing - see furher below in ch. 9.3. Here, however, his simple consrain is perfec for our purposes. Given he preferences and he consrain, he Euler equaion for his maximizaion problem is given by (see exercise 5) u 0 (c ) = [1 + r +1 ] u 0 (c +1 ) : (3.6.7) Srucurally, his is he same expression as in (3.4.7). The ineres rae, however, refers o + 1, due o he change in he budge consrain. Remembering ha = 1= (1 + ), his shows ha consumpion increases as long as r +1 >. 3.6.4 Aggregaion and reduced form Aggregaion To see ha individual consrains add up o he aggregae resource consrain, we simply need o ake ino accoun ha individual income adds up o oupu, w K K +w L L = Y. Remember ha we are familiar wih he laer from (2.4.4). Now sar from (3.6.6) and use (3.6.5) o obain, K +1 = L k +1 = 1 + w K L k + w L L C = (1 ) K + Y C : The opimal behaviour of all households aken ogeher can be gained from (3.6.7) by summing over all households. This is done analyically correcly by rs applying he inverse funcion of u 0 o his equaion and hen summing individual consumpion levels over all households (see exercise 6 for deails). Applying he inverse funcion again gives where C is aggregae consumpion in : Reduced form u 0 (C ) = [1 + r +1 ] u 0 (C +1 ) ; (3.6.8) We now need o undersand how our economy evolves in general equilibrium. Our rs equaion is (3.6.8), elling us how consumpion evolves over ime. This equaion conains consumpion and he ineres rae as endogenous variables. Our second equaion is herefore he de niion of he ineres rae in (3.6.5) which we combine wih he rs-order condiion of he rm in (3.6.4) o yield r = @Y @K : (3.6.9) This equaion conains he ineres rae and he capial sock as endogenous variables.

3.6. A decenralized general equilibrium analysis 65 Our nal equaion is he resource consrain (3.6.3), which provides a link beween capial and consumpion. Hence, (3.6.8), (3.6.9) and (3.6.3) give a sysem in hree equaions and hree unknowns. When we inser he ineres rae ino he opimaliy condiion for consumpion, we obain as our reduced form h i u 0 (C ) = 1 + @Y +1 @K +1 u 0 (C +1 ) ; (3.6.10) K +1 = (1 ) K + Y C : This is a wo-dimensional sysem of non-linear di erence equaions which gives a unique soluion for he ime pah of capial and consumpion, provided we have wo iniial condiions K 0 and C 0. 3.6.5 Seady sae and ransiional dynamics When rying o undersand a sysem like (3.6.10), he same principles can be followed as wih one-dimensional di erence equaions. Firs, one ries o idenify a xed poin, i.e. a seady sae, and hen one looks a ransiional dynamics. Seady sae In a seady sae, all variables are consan. Seing K +1 = K = K and C +1 = C = C; we obain 1 = 1 + @Y @K, @Y = + ; C = Y K; @K where he, sep used he link beween and from (3.1.2). In he seady sae, he marginal produciviy of capial is given by he ime preference rae plus he depreciaion rae. Consumpion equals oupu minus depreciaion, i.e. minus replacemen invesmen. These wo equaions deermine wo variables K and C: he rs deermines K; he second deermines C: Transiional dynamics Undersanding ransiional dynamics is no as sraighforward as undersanding he seady sae. Is analysis follows he same idea as in coninuous ime, however, and we will analyze ransiional dynamics in deail here. Having said his, we should acknowledge he fac ha ransiional dynamics in discree ime can quickly become more complex han in coninuous ime. As an example, chaoic behaviour can occur in one-dimensional di erence equaions while one needs a leas a hree-dimensional di erenial equaion sysem o obain chaoic properies in coninuous ime. The lieraure on chaos heory and exbooks on di erence equaions provide many examples.

66 Chaper 3. Muli-period models 3.7 A cenral planner 3.7.1 Opimal facor allocaion One of he mos solved maximizaion problems in Economics is he cenral planner problem. The choice by a cenral planner given a social welfare funcion and echnological consrains provides informaion abou he rs-bes facor allocaion. This is a benchmark for many analyses in normaive economics. We consider he probably mos simple case of opimal facor allocaion in a dynamic seup. The maximizaion problem Le preferences be given by U = 1 = u (C ) ; (3.7.1) where C is he aggregae consumpion of all households a a poin in ime : This funcion is maximized subjec o a resource accumulaion consrain which reads K +1 = Y (K ; L ) + (1 ) K C (3.7.2) for all : The producion echnology is given by a neoclassical producion funcion Y (K ; L ) wih sandard properies. The Lagrangian This seup di ers from he ones we go o know before in ha here is an in nie number of consrains in (3.7.2). This consrain holds for each poin in ime : As a consequence, he Lagrangian is formulaed wih in niely many Lagrangian mulipliers, L = 1 = u (C ) + 1 = f [K +1 Y (K ; L ) (1 ) K + C ]g : (3.7.3) The rs par of he Lagrangian is sandard, 1 = u (C ), i jus copies he objecive funcion. The second par consiss of a sum from o in niy, one consrain for each poin in ime, each muliplied by is own Lagrange muliplier. In order o make he maximizaion procedure clearer, we rewrie he Lagrangian as L = 1 = u (C ) + s 2 = [K +1 Y (K ; L ) (1 ) K + C ] + s 1 [K s Y (K s 1 ; L s 1 ) (1 ) K s 1 + C s 1 ] + s [K s+1 Y (K s ; L s ) (1 ) K s + C s ] + 1 =s+1 [K +1 Y (K ; L ) (1 ) K + C ] ; where we simply explicily wrie ou he sum for s 1 and s: Now maximize he Lagrangian boh wih respec o he conrol variable C s ; he mulipliers s and he sae variables K s. Maximizaion wih respec o K s migh appear

3.7. A cenral planner 67 unusual a his sage; we will see a jusi caion for his in he nex chaper. Firs-order condiions are L Cs = s u 0 (C s ) + s = 0 () s = u 0 (C s ) s ; (3.7.4) @Y L Ks = s 1 s + 1 = 0 () s 1 = 1 + @Y ; (3.7.5) @K s s @K s L s = 0: (3.7.6) u Combining he rs and second rs-order condiion gives 0 (C s 1 ) s 1 u 0 (C s) = 1 + @Y s @K s : This is equivalen o u 0 (C s ) u 0 (C s+1 ) = 1 1 : (3.7.7) 1+ @Y @K s+1 This expression has he same inerpreaion as (3.1.6) or (3.4.7) for example. When we replace s by ; his equaion wih he consrain (3.7.2) is a wo-dimensional di erence equaion sysem which allows us o deermine he pahs of capial and consumpion, given wo boundary condiions, which he economy will follow when facor allocaion is opimally chosen. The seady sae of such an economy is found by seing C s = C s+1 and K s = K s+1 in (3.7.7) and (3.7.2). This example also allows us o reurn o he discussion abou he link beween he sign of shadow prices and he Lagrange muliplier a he end of ch. 2.3.2. Here, he consrains in he Lagrangian are represened as lef-hand side minus righ-hand side. As a consequence, he Lagrange mulipliers are negaive, as he rs-order condiions (3.7.4) show. Apar from he fac ha he Lagrange muliplier here now sands for minus he shadow price, his does no play any role for he nal descripion of opimaliy in (3.7.7). 3.7.2 Where he Lagrangian comes from II Le us now see how we can derive he same expression as ha in (3.7.7) wihou using he Lagrangian. This will allow us o give an inuiive explanaion for why we maximized he Lagrangian in he las chaper wih respec o boh he conrol and he sae variable. Maximizaion wihou Lagrange Inser he consrain (3.7.2) ino he objecive funcion (3.7.1) and nd U = 1 = u (Y (K ; L ) + (1 ) K K +1 )! max fk g This is now maximized by choosing a pah fk g for capial. Choosing he sae variable implicily pins down he pah fc g of he conrol variable consumpion and one can herefore hink of his maximizaion problem as one where consumpion is opimally chosen.

68 Chaper 3. Muli-period models Now rewrie he objecive funcion as Choosing K s opimally implies U = s 2 = u (Y (K ; L ) + (1 ) K K +1 ) + s 1 u (Y (K s 1 ; L s 1 ) + (1 ) K s 1 K s ) + s u (Y (K s ; L s ) + (1 ) K s K s+1 ) + 1 =s+1 u (Y (K ; L ) + (1 ) K K +1 ) : s 1 u 0 (Y (K s 1 ; L s 1 ) + (1 ) K s 1 K s ) @Y + s u 0 (Ks ; L s ) (Y (K s ; L s ) + (1 ) K s K s+1 ) + 1 = 0: @K s Reinsering he consrain (3.7.2) and rearranging gives u 0 (C s 1 ) = u 0 (C s ) 1 + @Y (K s; L s ) @K s : This is he sandard opimaliy condiion for consumpion which we obained in (3.7.7). As s can sand for any poin in ime beween and in niy, we could replace s by, or + 1: Back o he Lagrangian When we now go back o he maximizaion procedure where he Lagrangian was used, we see ha he parial derivaive of he Lagrangian wih respec o K in (3.7.5) capures how changes over ime. The simple reason why he Lagrangian is maximized wih respec o K is herefore ha an addiional rs-order condiion is needed as needs o be deermined as well. In saic maximizaion problems wih wo consumpion goods and one consrain, he Lagrangian is maximized by choosing consumpion levels for boh consumpion goods and by choosing he Lagrange muliplier. In he Lagrange seup above in (3.7.4) o (3.7.6), we choose boh endogenous variables K and C plus he muliplier and hereby deermine opimal pahs for all hree variables. Hence, i is a echnical - mahemaical - reason ha K is chosen : deermining hree unknowns simply requires hree rs-order condiions. Economically, however, he conrol variable C is economically chosen while he sae variable K adjuss indirecly as a consequence of he choice of C : 3.8 Growh of family size 3.8.1 The seup Le us now consider an exension o he models considered so far. Le us imagine here is a family consising of n members a poin in ime and le consumpion c per individual

3.8. Growh of family size 69 family member be opimally chosen by he head of he family. The objecive funcion for his family head consiss of insananeous uiliy u (:) per family member imes he number of members, discouned a he usual discoun facor ; U = 1 = u (c ) n : Le us denoe family wealh by ^a. I is he produc of individual wealh imes he number of family members, ^a n a. The budge consrain of he household is hen given by ^a +1 = (1 + r ) ^a + n w n c Toal labour income is given by n imes he wage w and family consumpion is n c : 3.8.2 Solving by subsiuion We solve his maximizaion problem by subsiuion. We rewrie he objecive funcion and inser he consrain wice, s 1 = u (c ) n + s u (c s ) n s + s+1 u (c s+1 ) n s+1 + 1 =s+2 u (c ) n (1 + = = s 1 u (c ) n + s rs ) ^a s + n s w s ^a s+1 u n s n s (1 + + s+1 rs+1 ) ^a s+1 + n s+1 w s+1 ^a s+2 u n s+1 + 1 =s+2 u (c ) n : n s+1 Now compue he derivaive wih respec o ^a s+1. This gives (1 + u 0 rs ) ^a s + n s w s ^a s+1 ns (1 + = u 0 rs+1 ) ^a s+1 + n s+1 w s+1 ^a s+2 1 + rs+1 n s+1 : n s n s n s+1 n s+1 When we replace he budge consrain by consumpion again and cancel he n s and n s+1, we obain u 0 (c s ) = [1 + r s+1 ] u 0 (c s+1 ) : (3.8.1) The ineresing feaure of his rule is ha being par of a family whose size n can change over ime does no a ec he growh of individual consumpion c s : I follows he same rule as if individuals maximized uiliy independenly of each oher and wih heir personal budge consrains. 3.8.3 Solving by he Lagrangian The Lagrangian for his seup wih one budge consrain for each poin in ime requires an in nie number of Lagrange mulipliers ; one for each : I reads L = 1 = u (c ) n + [(1 + r ) ^a + n l w n c ^a +1 ] :

70 Chaper 3. Muli-period models We rs compue he rs-order condiions for consumpion and hours worked for one poin in ime s; L cs = s @ @c s u (c s ) s = 0; As discussed in 3.7, we also need o compue he derivaive wih respec o he sae variable. I is imporan o compue he derivaive wih respec o family wealh ^a as his is he rue sae variable of he head of he family. (Compuing he derivaive wih respec o individual wealh a would also work bu would lead o an incorrec resul, i.e. a resul ha di ers from 3.8.1.) This derivaive is L^as = s 1 + s (1 + r s ) = 0, s 1 = (1 + r s ) s : Opimal consumpion hen follows by replacing he Lagrange mulipliers, u 0 (c s 1 ) = [1 + r s ] u 0 (c s ) : This is idenical o he resul we obain by insering in (3.8.1). 3.9 Furher reading and exercises For a much more deailed background on he elasiciy of subsiuion, see Blackorby and Russell (1989). They sudy he case of more han wo inpus and sress ha he Morishima elasiciy is o be preferred o he Allan/ Uzawa elasiciy. The dynamic programming approach was developed by Bellman (1957). Maximizaion using he Lagrange mehod is widely applied by Chow (1997). The example in ch. 3.4.3 was originally inspired by Grossman and Shapiro (1986).

3.9. Furher reading and exercises 71 Exercises chaper 3 Applied Ineremporal Opimizaion Dynamic programming in discree deerminisic ime 1. The envelope heorem I Le he uiliy funcion of an individual be given by U = U (C; L) ; where consumpion C increases uiliy and supply of labour L decreases uiliy. Le he budge consrain of he individual be given by wl = C: Le he individual maximize uiliy wih respec o consumpion and he amoun of labour supplied. (a) Wha is he opimal labour supply funcion (in implici form)? How much does an individual consume? Wha is he indirec uiliy funcion? (b) Under wha condiions does an individual increase labour supply when wages rise (no analyical soluion required)? (c) Assume higher wages lead o increased labour supply. Does disuiliy arising from increased labour supply compensae uiliy from higher consumpion? Does uiliy rise if here is no disuiliy from working? Sar from he indirec uiliy funcion derived in a) and apply he proof of he envelope heorem and he envelope heorem iself. 2. The envelope heorem II (a) Compue he derivaive of he Bellman equaion (3.4.6) wihou using he envelope heorem. Hin: Compue he derivaive wih respec o he sae variable and hen inser he rs-order condiion. (b) Do he same wih (3.4.15) 3. The addiively separable objecive funcion (a) Show ha he objecive funcion can be wrien as in (3.3.3).

72 Chaper 3. Muli-period models (b) Find ou wheher (3.3.3) implies he objecive funcion. (I does no.) 4. Ineremporal and dynamic budge consrains (a) Show ha he ineremporal budge consrain T = 1 1 k= e = a + T = 1 + r k implies he dynamic budge consrain 1 k= 1 1 + r k i (3.9.1) a +1 = (a + i e ) (1 + r ) : (3.9.2) (b) Under which condiions does he dynamic budge consrain imply he ineremporal budge consrain? (c) Now consider a +1 = (1 + r ) a +w does i imply? e : Wha ineremporal budge consrain 5. The sandard saving problem Consider he objecive funcion from (3.1.1), U = 1 = u (c ), and maximize i by choosing a consumpion pah fc g subjec o he consrain (3.6.6), k +1 = (1 + r ) k + w L c : The resul is given by (3.6.7). (a) Solve his problem by dynamic programming mehods. (b) Solve his by using he Lagrange approach. Choose a muliplier for an in nie sequence of consrains. 6. Aggregaion of opimal consumpion rules Consider he opimaliy condiion u 0 (c ) = (1 + r +1 ) u 0 (c +1 ) in (3.6.7) and derive he aggregae version (3.6.8). Find he assumpions required for he uiliy funcion for hese seps o be possible. 7. A benevolen cenral planner You are he omniscien omnipoen benevolen cenral planner of an economy. You wan o maximize a social welfare funcion U = 1 = u (C ) for your economy by choosing a pah of aggregae consumpion levels fc g subjec o a resource consrain K +1 K = Y (K ; L ) K C (3.9.3) (a) Solve his problem by dynamic programming mehods.

3.9. Furher reading and exercises 73 (b) Discuss how he cenral planner resul is relaed o he decenralized resul from exercise 5. (c) Wha does he resul look like for a uiliy funcion which is logarihmic and for one which has consan elasiciy of subsiuion, u (C ()) = ln C () and u (C ()) = C ()1 1? (3.9.4) 1 8. Environmenal economics Imagine you are an economis only ineresed in maximizing he presen value of your endowmen. You own a renewable resource, for example a piece of fores. The amoun of wood in your fores a a poin in ime is given by x. Trees grow a b(x ) and you harves a he quaniy c. (a) Wha is he law of moion for x? (b) Wha is your objecive funcion if prices a per uni of wood is given by p, your horizon is in niy and you have perfec informaion? (c) How much should you harves per period when he ineres rae is consan? Does his change when he ineres rae is ime-variable? 9. The 10k run - Formulaing and solving a maximizaion problem You consider paricipaion in a 10k run or a marahon. The even will ake place in M monhs. You know ha your ness needs o be improved and ha his will be cosly: i requires e or a 0 which reduces uiliy u (:) : A he same ime, you enjoy being fas, i.e. uiliy increases he shorer your nish ime l: The higher your e or, he shorer your nish ime. (a) Formulae a maximizaion problem wih 2 periods. E or a ecs he nish ime in M monhs. Specify a uiliy funcion and discuss a reasonable funcional form which capures he link beween nish ime l and e or a 0 : (b) Solve his maximizaion problem by providing and discussing he rs-order condiion. 10. A cenral planner Consider he objecive funcion of a cenral planner, The consrain is given by U 0 = 1 =0 u (C ) : (3.9.5) K +1 = Y (K ; L ) + (1 ) K C : (3.9.6) (a) Explain in words he meaning of he objecive funcion and of he consrain.

74 Chaper 3. Muli-period models (b) Solve he maximizaion problem by rs insering (3.9.6) ino (3.9.5) and hen u by opimally choosing K. Show ha he resul is 0 (C ) = 1+ @Y (K +1;L +1 ) u 0 (C +1 ) @K +1 and discuss his in words. (c) Discuss why using he Lagrangian also requires maximizing wih respec o K even hough K is a sae variable.

Par II Deerminisic models in coninuous ime 75

Par II covers coninuous ime models under cerainy. Chaper 4 rs looks a differenial equaions as hey are he basis of he descripion and soluion of maximizaion problems in coninuous ime. Firs, some useful de niions and heorems are provided. Second, di erenial equaions and di erenial equaion sysems are analyzed qualiaively by he so-called phase-diagram analysis. This simple mehod is exremely useful for undersanding di erenial equaions per se and also for laer purposes for undersanding qualiaive properies of soluions o maximizaion problems and properies of whole economies. Linear di erenial equaions and heir economic applicaions are hen nally analyzed before some words are spen on linear di erenial equaion sysems. Chaper 5 presens a new mehod for solving maximizaion problems - he Hamilonian. As we are now in coninuous ime, wo-period models do no exis. A disincion will be drawn, however, beween nie and in nie horizon models. In pracice, his disincion is no very imporan as, as we will see, opimaliy condiions are very similar for nie and in nie maximizaion problems. Afer an inroducory example on maximizaion in coninuous ime by using he Hamilonian, he simple link beween Hamilonians and he Lagrangian is shown. The soluion o maximizaion problems in coninuous ime will consis of one or several di erenial equaions. As a unique soluion o di erenial equaions requires boundary condiions, we will show how boundary condiions are relaed o he ype of maximizaion problem analyzed. The boundary condiions di er signi canly beween nie and in nie horizon models. For he nie horizon models, here are iniial or erminal condiions. For he in nie horizon models, we will ge o know he ransversaliy condiion and oher relaed condiions like he No-Ponzi-game condiion. Many examples and a comparison beween he presen-value and he curren-value Hamilonian conclude his chaper. Chaper 6 solves he same kind of problems as chaper 5, bu i uses he mehod of dynamic programming. The reason for doing his is o simplify undersanding of dynamic programming in sochasic seups in Par IV. Various aspecs speci c o he use of dynamic programming in coninuous ime, e.g. he srucure of he Bellman equaion, can already be reaed here under cerainy. This chaper will also provide a comparison beween he Hamilonian and dynamic programming and look a a maximizaion problem wih wo sae variables. An example from moneary economics on real and nominal ineres raes concludes he chaper. 77

78

Chaper 4 Di erenial equaions There are many excellen exbooks on di erenial equaions. This chaper will herefore be relaively shor. Is objecive is more o recap basic conceps augh in oher courses and o serve as a background for laer applicaions. 4.1 Some de niions and heorems 4.1.1 De niions The following de niions are sandard and follow Brock and Malliaris (1989). De niion 4.1.1 An ordinary di erenial equaion sysem (ODE sysem) is of he ype dx () d _x () = f (; x ()) ; (4.1.1) where lies beween some saring poin and in niy, 2 [ 0 ; 1[ ; x can be a vecor, x 2 R n and f maps from R n+1 ino R n : When x is no a vecor, i.e. for n = 1; (4.1.1) obviously is an ordinary di erenial equaion. An auonomous di erenial equaion is an ordinary di erenial equaion where f(:) is independen of ime ; dx () _x () = f (x ()) : (4.1.2) d The di erence beween a di erenial equaion and a normal algebraic equaion obviously lies in he fac ha di erenial equaions conain derivaives of variables like _x (). An example of a di erenial equaion which is no an ODE is he parial di erenial equaion. A linear example is a (x; ) @p (x; ) @x + b (x; ) @p (x; ) @ = c (x; ) ; where a (:) ; b (:) ; c (:) and p (:) are funcions wih nice properies. While in an ODE, here is one derivaive (ofen wih respec o ime), a parial di erenial equaion conains 79

80 Chaper 4. Di erenial equaions derivaives wih respec o several variables. Parial di erenial equaions can describe e.g. a densiy and how i changes over ime. Oher ypes of di erenial equaions include sochasic di erenial equaions (see ch. 9), implici di erenial equaions (which are of he ype g ( _x ()) = f (; x ())), delay di erenial equaions ( _x () = f (x ( ))) and many oher more. De niion 4.1.2 An iniial value problem is described by where x 0 is he iniial condiion. A erminal value problem is of he form where x T is he erminal condiion. _x = f(; x); x ( 0 ) = x 0; 2 [ 0 ; T ] ; _x = f (; x) ; x (T ) = x T ; 2 [ 0 ; T ] ; 4.1.2 Two heorems Theorem 4.1.1 Exisence (Brock and Malliaris, 1989) If f (; x) is a coninuous funcion on recangle L = f(; x)j j 0 j a; jx x 0 j bg hen here exiss a coninuous di ereniable soluion x () on inerval j 0 j a ha solves iniial value problem _x = f (; x) ; x ( 0 ) = x 0 : (4.1.3) This heorem only proves ha a soluion exiss. I is sill possible ha here are many soluions. Theorem 4.1.2 Uniqueness If f and @f=@x are coninuous funcions on L; he iniial value problem (4.1.3) has a unique soluion for 2 0 ; 0 + min a; b max jf (; x)j If his condiion is me, an ODE wih an iniial or erminal condiion has a unique soluion. More generally speaking, a di erenial equaion sysem consising of n ODEs ha saisfy hese condiions (which are me in he economic problems we encouner here) has a unique soluion provided ha here are n boundary condiions. Knowing abou a unique soluion is useful as one knows ha changes in parameers imply unambiguous predicions abou changes in endogenous variables. If he governmen changes some ax, we can unambiguously predic wheher employmen goes up or down.

4.2. Analyzing ODEs hrough phase diagrams 81 4.2 Analyzing ODEs hrough phase diagrams This secion will presen ools ha allow us o deermine properies of soluions of differenial equaions and di erenial equaion sysems. The analysis will be qualiaive in his chaper as mos economic sysems are oo non-linear o allow for an explici general analyic soluion. Explici soluions for linear di erenial equaions will be reaed in ch. 4.3. 4.2.1 One-dimensional sysems We sar wih a one-dimensional di erenial equaion _x () = f (x ()), where x 2 R and > 0: This will also allow us o review he conceps of xpoins, local and global sabiliy and insabiliy as used already when analyzing di erence equaions in ch. 2.5.4. Unique xpoin Le f (x) be represened by he graph in he following gure, wih x () being shown on he horizonal axis. As f (x) gives he change of x () over ime, _x () is ploed on he verical axis. N _x() N N x N N x() N Figure 4.2.1 Qualiaive analysis of a di erenial equaion As in he analysis of di erence equaions in ch. 2.5.4, we rs look for he xpoin of he underlying di erenial equaion. A xpoin is de ned similarly in spiri bu - hinking now in coninuous ime - di erenly in deail.

82 Chaper 4. Di erenial equaions De niion 4.2.1 A xpoin x is a poin where x () does no change. In coninuous ime, his means _x () = 0 which, from he de niion (4.1.2) of he di erenial equaion, requires f (x ) = 0: The requiremen ha x () does no change is he similariy in spiri o he de niion in discree ime. The requiremen ha f (x ) = 0 is he di erence in deail: in discree ime as in (2.5.8) we required f (x ) = x. Looking a he above graph of f (x) ; we nd x a he poin where f (x) crosses he horizonal line. We hen inquire he sabiliy of he xpoin. When x is o he lef of x ; f (x) > 0 and herefore x increases, _x () > 0: This increase of x is represened in his gure by he arrows on he horizonal axis. Similarly, for x > x ; f (x) < 0 and x () decreases. We have herefore found ha he xpoin x is globally sable and have also obained a feeling for he behaviour of x () ; given some iniial condiions. We can now qualiaively plo he soluions wih ime on he horizonal axis. As he discussion has jus shown, he soluion x () depends on he iniial value from which we sar, i.e. on x (0) : For x (0) > x ; x () decreases, for x (0) < x ; x () increases: any changes over ime are monoonic. There is one soluion for each iniial condiion. The following gure shows hree soluions of _x () = f (x ()), given hree di eren iniial condiions. Figure 4.2.2 Qualiaive soluions of an ODE for hree di eren iniial condiions Muliple xpoins and equilibria Of course, more sophisicaed funcions han f (x) can be imagined. Now consider a di erenial equaion _x () = g (x ()) where g (x) is non-monoonic as ploed in he nex gure.

4.2. Analyzing ODEs hrough phase diagrams 83 Figure 4.2.3 Muliple equilibria As his gure shows, here are four xpoins. Looking a wheher g (x) is posiive or negaive, we know wheher x () increases or decreases over ime. This allows us o plo arrows on he horizonal axis as in he previous example. The di erence o before consiss of he fac ha some xpoins are unsable and some are sable. De niion 4.2.1 showed us ha he concep of a xpoin in coninuous ime is slighly di eren from discree ime. However, he de niions of sabiliy as hey were inroduced in discree ime can be direcly applied here as well. Looking a x 1; any small deviaion of x from x 1 implies an increase or decrease of x: The xpoin x 1 is herefore unsable, given he de niion in ch. 2.5.4. Any small deviaion x 2; however, implies ha x moves back o x 2: Hence, x 2 is (locally) sable. The xpoin x 3 is also unsable, while x 4 is again locally sable: While x converges o x 4 for any x > x 4 (in his sense x 4 could be called globally sable from he righ), x converges o x 4 from he lef only if x is no smaller han or equal o x 3: Figure 4.2.4 Qualiaive soluions of an ODE for di eren iniial condiions II

84 Chaper 4. Di erenial equaions If an economy can be represened by such a di erenial equaion _x () = g (x ()), one would call xpoins long-run equilibria. There are sable equilibria and unsable equilibria and i depends on he underlying sysem (he assumpions ha implied he di erenial equaion _x = g (x)) which equilibrium would be considered o be he economically relevan one. As in he sysem wih one xpoin, we can qualiaively plo soluions of x () over ime, given di eren iniial values for x (0). This is shown in g. 4.2.4 which again highlighs he sabiliy properies of xpoins x 1 o x 4: 4.2.2 Two-dimensional sysems I - An example We now exend our qualiaive analysis of di erenial equaions o wo-dimensional sysems. This laer case allows for an analysis of more complex sysems han simple onedimensional di erenial equaions. In almos all economic models wih opimal saving decisions, a reduced form consising of a leas wo di erenial equaions will resul. We sar here wih an example before we analyse wo-dimensional sysems more generally in he nex chaper. The sysem Consider he following di erenial equaion sysem, _x 1 = x 1 x 2 ; _x 2 = b + x 1 1 x 2 ; 0 < < 1 < b: Assume ha for economic reasons we are ineresed in properies for x i > 0: Fixpoin The rs quesion is as always wheher here is a xpoin a all. In a wo-dimensional sysem, a xpoin x = (x 1; x 2) is wo-dimensional as well. The xpoin is de ned such ha boh variables do no change over ime, i.e. _x 1 = _x 2 = 0: If such a poin exiss, i mus saisfy _x 1 = _x 2 = 0, (x 1) = x 2; x 2 = b + (x 1) 1 : By insering he second equaion ino he rs, x 1 is deermined by (x 1) = b+(x 1) 1 and x 2 follows from x 2 = (x 1) : Analyzing he properies of he equaion (x 1) = b + (x 1) 1 would hen show ha x 1 is unique: The lef-hand side increases monoonically from 0 o in niy for x 1 2 [0; 1[ while he righ-hand side decreases monoonically from in niy o b: Hence, here mus be an inersecion poin and here can be only one as funcions are monoonic. As x 1 is unique, so is x 2 = (x 1).

4.2. Analyzing ODEs hrough phase diagrams 85 Zero-moion lines and pairs of arrows Having derived he xpoin, we now need o undersand he behaviour of he sysem more generally. Wha happens o x 1 and x 2 when (x 1 ; x 2 ) 6= (x 1; x 2)? To answer his quesion, he concep of zero-moion lines is very useful. A zero-moion line is a line for a variable x i which marks he poins for which he variable x i does no change, i.e. _x i = 0: For our wo-dimensional di erenial equaion sysem, we obain wo zero-moion lines, _x 1 0, x 2 x 1 ; _x 2 0, x 2 b + x 1 1 : (4.2.1) In addiion o he equaliy sign, we also analyse here a he same ime for which values x i rises. Why his is useful will soon become clear. We can now plo he curves where _x i = 0 in a diagram. In conras o he one-dimensional graphs in he previous chaper, we now have he variables x 1 and x 2 on he axes (and no he change of one variable on he verical axis). The inersecion poin of he wo zero-moion lines gives he x poin x = (x 1; x 2) which we derived analyically above. Figure 4.2.5 Firs seps owards a phase diagram In addiion o showing where variables do no change, he zero-moion lines also delimi regions where variables do change. Looking a (4.2.1) again shows (why we used he and no he = sign and) ha he variable x 1 increases whenever x 2 < x 1. Similarly, he variable x 2 increases, whenever x 2 < b + x1 1 : The direcions in which variables change can hen be ploed ino his diagram by using arrows. In his diagram, here is a pair of arrows per region as wo direcions (one for x 1 ; one for x 2 ) need o be indicaed. This is in principle idenical o he arrows we used in he analysis of he one-dimensional sysems. If he sysem nds iself in one of hese four regions, we know qualiaively, how variables change over ime: Variables move o he souh-eas in region I, o he norh-eas in region II, o he norh-wes in region III and o he souh-wes in region IV.

86 Chaper 4. Di erenial equaions Trajecories Given he zero-moion lines, he xpoin and he pairs of arrows, we are now able o draw rajecories ino his phase diagram. We will do so and analyse he implicaions of pairs of arrows furher once we have generalized he derivaion of a phase diagram. 4.2.3 Two-dimensional sysems II - The general case Afer his speci c example, we will now look a a more general di erenial equaion sysem and will analyse i by using a phase diagram. The sysem Consider wo di erenial equaions where funcions f (:) and g (:) are coninuous and di ereniable, _x 1 = f (x 1 ; x 2 ) ; _x 2 = g (x 1 ; x 2 ) : (4.2.2) For he following analysis, we will need four assumpions on parial derivaives; all of hem are posiive apar from f x1 (:) ; f x1 (:) < 0; f x2 (:) > 0; g x1 (:) > 0; g x2 (:) > 0: (4.2.3) Noe ha, provided we are willing o make he assumpions required by he heorems in ch. 4.1.2, we know ha here is a unique soluion o his di erenial equaion sysem, i.e. x 1 () and x 2 () are unambiguously deermined given wo boundary condiions. Fixpoin The rs quesion o be ackled is wheher here is an equilibrium a all. Is here a xpoin x such ha _x 1 = _x 2 = 0? To his end, se f (x 1 ; x 2 ) = 0 and g (x 1 ; x 2 ) = 0 and plo he implicily de ned funcions in a graph. x 2 * x 2 f ( x x ) 0 1, 2 = g ( x x ) 0 1, 2 = * x 1 x 1 Figure 4.2.6 Zero moion lines wih a unique seady sae

4.2. Analyzing ODEs hrough phase diagrams 87 By he implici funcion heorem - see (2.3.3) - and he assumpions made in (4.2.3), one zero moion line is increasing and one is decreasing. If we are furher willing o assume ha funcions are no monoonically approaching an upper and lower bound, we know ha here is a unique xpoin (x 1; x 2) x : General evoluion Now we ask again wha happens if he sae of he sysem di ers from x ; i.e. if eiher x 1 or x 2 or boh di er from heir seady sae values. To nd an answer, we have o deermine he sign of f (x 1 ; x 2 ) and g (x 1; x 2 ) for some (x 1 ; x 2 ) : Given (4.2.2); x 1 would increase for a posiive f (:) and x 2 would increase for a posiive g (:) : For any known funcions f (x 1 ; x 2 ) and g (x 1; x 2 ) ; one can simply plo a 3-dimensional gure wih x 1 and x 2 on he axes in he plane and wih ime derivaives on he verical axis. Figure 4.2.7 A hree-dimensional illusraion of wo di erenial equaions and heir zeromoion lines The whie area in his gure is he horizonal plane, i.e. where _x 1 and _x 2 are zero. The dark surface illusraes he law of moion for x 1 as does he grey surface for x 2 : The inersecion of he dark surface wih he horizonal plane gives he loci on which x 1 does no change. The same is rue for he grey surface and x 2 : Clearly, a he inersecion poin of hese zero-moion lines we nd he seady sae x : When working wih wo-dimensional gures and wihou he aid of compuers, we sar from he zero-moion line for, say, x 1 and plo i ino a normal gure.

88 Chaper 4. Di erenial equaions x 2 N (~x 1; ~x 2 ) r - - - f(x 1 ; x 2 ) = 0 - - Figure 4.2.8 Sep 1 in consrucing a phase diagram x 1 N Now consider a poin (~x 1 ; ~x 2 ) : As we know ha _x 1 = 0 on f (x 1 ; x 2 ) = 0 and ha from (4.2.3) f x2 (:) > 0; we know ha moving from ~x 1 on he zero-moion line verically o (~x 1 ; ~x 2 ) ; f (:) is increasing. Hence, x 1 is increasing, _x 1 > 0; a (~x 1 ; ~x 2 ) : As his line of reasoning holds for any (~x 1 ; ~x 2 ) above he zero-moion line, x 1 is increasing everywhere above f (x 1 ; x 2 ) = 0: As a consequence, x 1 is decreasing everywhere below he zero-moion line. This movemen is indicaed by he arrows in he above gure. x 2 N 6 g(x 1 ; x 2 ) = 0 6 r? (~x 1; ~x 2 )? 6? 6? Figure 4.2.9 Sep 2 in consrucing a phase diagram x 1 N Le us now consider he second zero-moion line, _x 2 = 0, g (x 1 ; x 2 ) = 0; and look again a he poin (~x 1 ; ~x 2 ) : When we sar from he poin ~x 2 on he zero moion line and move owards (~x 1 ; ~x 2 ) ; g (x 1 ; x 2 ) is decreasing, given he derivaive g x1 (:) > 0 from (4.2.3). Hence, for any poins o he lef of g (x 1 ; x 2 ) = 0; x 2 is decreasing. Again, his is shown in he above gure.

4.2. Analyzing ODEs hrough phase diagrams 89 Fixpoins and rajecories We can now represen he direcions in which x 1 and x 2 are moving ino a single phase-diagram by ploing one arrow each ino each of he four regions limied by he zero-moion lines. Given ha he arrows can eiher indicae an increase or a decrease for boh x 1 and x 2 ; here are wo imes wo di eren combinaions of arrows, i.e. four regions. When we add some represenaive rajecories, a complee phase diagram resuls. Figure 4.2.10 A saddle pah Adding rajecories is relaively easy when paying aenion o he zero-moion lines. When rajecories are ploed far away from zero-moion lines, he arrow pairs indicae wheher he movemen is owards he norh-eas, he souh-eas or he oher wo possible direcions. A poin A in he phase diagram for he saddle pah, he movemen is owards he norh-wes. Noe ha he arrows represen rs derivaives only. As second derivaives are no aken ino accoun (usually), we do no know wheher he rajecory moves more and more o he norh or more and more o he wes. The arrow-pairs are also consisen wih wave-like rajecories, as long as he movemen is always owards he norh-wes. Precise informaion on he shape of he rajecories is available when we look a he poins where rajecories cross he zero-moion lines. On a zero-moion line, he variable o which his zero-moion line belongs does no change. Hence all rajecories cross zero-moion lines eiher verically or horizonally. When we look a poin B above, he rajecory moves from he norh-wes o he norh-eas region. The variable x 1 changes direcion and begins o rise afer having crossed is zero-moion line verically. An example where a zero-moion line is crossed horizonally is poin C: To he lef of poin C; x 1 rises and x 2 falls. On he zero-moion line for x 2 ; x 2 does no change bu x 1 coninues o rise. To he righ of C; boh x 1 and x 2 increase. Similar changes in direcion can be observed a oher inersecion poins of rajecories wih zero-moion lines.

90 Chaper 4. Di erenial equaions 4.2.4 Types of phase diagrams and xpoins Types of xpoins As has become clear by now, he parial derivaives in (4.2.3) are crucial for he slope of he zero-moion lines and for he direcion of movemens of variables x 1 and x 2 : Depending on he signs of he parial derivaives, various phase diagrams can occur. As here are wo possible direcions for each variable, hese phase diagrams can be classi ed ino four ypical groups, depending on he properies of heir xpoin. De niion 4.2.2 A xpoin is called a 9 8 >= >< and cener saddle poin focus node 8 >< >: on () >; >: zero wo all all 9 >= >; rajecories pass hrough he xpoin a leas one rajecory, boh variables are non-monoonic all rajecories, one or boh variables are monoonic A node and a focus can be eiher sable or unsable. Illusraion Here is now an overview of some ypical phase diagrams. Figure 4.2.11 Phase diagram for a node

4.2. Analyzing ODEs hrough phase diagrams 91 This rs phase diagram shows a node. A node is a xpoin hrough which all rajecories go and where he ime pahs implied by rajecories are monoonic for a leas one variable. As drawn here, i is a sable node, i.e. for any iniial condiions, he sysem ends up in he xpoin. An unsable node is a xpoin from which all rajecories sar. A phase diagram for an unsable node would look like he one above bu wih all direcions of moions reversed. Figure 4.2.12 Phase diagram for a focus A phase diagram wih a focus looks similar o one wih a node. The di erence lies in he non-monoonic pahs of he rajecories. As drawn here, x 1 or x 2 rs increase and hen decrease on some rajecories. Figure 4.2.13 Phase diagram for a cener

92 Chaper 4. Di erenial equaions A circle is a very special case for a di erenial equaion sysem. I is rarely found in models wih opimizing agens. The sandard example is he predaor-prey model, _x = x xy, _y = y + xy, where,,, and are posiive consans. This is also called he Loka-Volerra model. No closed-form soluion has been found so far. Limiaions I should be noed ha a phase diagram analysis allows us o idenify a saddle poin. If no saddle poin can be ideni ed, i is generally no possible o disinguish beween a node, focus or cener. In he linear case, more can be deduced from a graphical analysis. This is generally no necessary, however, as here is a closed-form soluion. The de niion of various ypes of xpoins is hen based on Eigenvalues of he sysem. See ch. 4.5. 4.2.5 Mulidimensional sysems If we have higher-dimensional problems where x 2 R n and n > 2; phase diagrams are obviously di cul o draw. In he hree-dimensional case, ploing zero moion surfaces someimes helps o gain some inuiion. A graphical soluion will generally, however, no allow us o idenify equilibrium properies like saddle-pah or saddle-plane behaviour. 4.3 Linear di erenial equaions This secion will focus on a special case of he general ODE de ned in (4.1.1). The special aspec consiss of making he funcion f (:) in (4.1.1) linear in x () : By doing his, we obain a linear di erenial equaion, _x () = a () x () + b () ; (4.3.1) where a () and b () are funcions of ime. This is he mos general case for a one dimensional linear di erenial equaion. 4.3.1 Rules on derivaives Before analyzing (4.3.1) in more deail, we rs need some oher resuls which will be useful laer. This secion herefore rs presens some rules on how o compue derivaives. I is by no means inended o be comprehensive or go in any deph. I presens rules which have shown by experience o be of imporance. Inegrals De niion 4.3.1 A funcion F (x) R f (x) dx is he inde nie inegral of a funcion f (x) if d dx F (x) = d Z f (x) dx = f (x) (4.3.2) dx

4.3. Linear di erenial equaions 93 This de niion implies ha here are in niely many inegrals of f (x) : If F (x) is an inegral, hen F (x) + c; where c is a consan, is an inegral as well. Leibniz rule We presen here a rule for compuing he derivaive of an inegral funcion. Le here be a funcion z (x) wih argumen x; de ned by he inegral z (x) Z b(x) a(x) f (x; y) dy; where a (x) ; b (x) and f (x; y) are di ereniable funcions. Noe ha x is he only argumen of z; as y is inegraed ou on he righ-hand side. Then, he Leibniz rule says ha he derivaive of his funcion wih respec o x is d dx z (x) = b0 (x) f (x; b (x)) a 0 (x) f (x; a (x)) + The following gure illusraes his rule. Z b(x) a(x) @ f (x; y) dy: (4.3.3) @x ( x y) f, z ( x + x ) ( x y) f, z( x) z( x) a ( x) b( x) y Figure 4.3.1 Illusraion of he di ereniaion rule Le x increase by a small amoun. Then he inegral changes a hree margins: The upper bound, he lower bound and he funcion f (x; y) iself. As drawn here, he upper bound b (x) and he funcion f (x; y) increase and he lower bound a (x) decreases in x. As a consequence, he area below he funcion f (:) beween bounds a (:) and b (:) increases because of hree changes: he increase o he lef because of a (:) ; he increase o he righ because of b (:) and he increase upwards because of f (:) iself. Clearly, his gure changes when he derivaives of a (:), b (:) and f (:) wih respec o x have a di eren sign han he ones assumed here.

94 Chaper 4. Di erenial equaions Anoher derivaive wih an inegral Now consider a funcion of he ype y = R b f (x (i)) di: Funcions of his ype will be a encounered frequenly as objecive funcions, e.g. ineremporal uiliy or pro funcions. Wha is he derivaive of y wih respec o x (i)? I is given by @y=@x (i) = f 0 (x (i)) : The inegral is no par of his derivaive as he derivaive is compued for one speci c x (i) and no for all x (i) wih i lying beween a and b: Noe he analogy o maximizing a sum as e.g. in (3.1.4). The inegraion variable i here corresponds o he summaion index in (3.1.4). When one speci c poin i is chosen (say i = (a + b) =2), all derivaives of he oher f (x (i)) wih respec o his speci c x (i) are zero. Inegraion by pars (for inde nie and de nie inegrals) Proposiion 4.3.1 For wo di ereniable funcions u (x) and v (x) ; Z Z u 0 (x) v (x) dx = u (x) v (x) u (x) v 0 (x) dx: (4.3.4) Proof. We sar by observing ha (u (x) v (x)) 0 = u 0 (x) v (x) + u (x) v 0 (x) ; where we used he produc rule. Inegraing boh sides by applying R dx; gives Z Z u (x) v (x) = u 0 (x) v (x) dx + u (x) v 0 (x) dx: Rearranging gives (4.3.4). Equivalenly, one can show (see he exercise 8) ha Z b a _xyd = [xy] b a Z b a x _yd: (4.3.5) 4.3.2 Forward and backward soluions of a linear di erenial equaion We now reurn o our linear di erenial equaion _x () = a () x () + b () from (4.3.1). I will now be solved. Generally speaking, a soluion o a di erenial equaion is a funcion x () which sais es his equaion. A soluion can be called a ime pah of x when represens ime.

4.3. Linear di erenial equaions 95 General soluion of he non-homogeneous equaion The di erenial equaion in (4.3.1) has, as all di erenial equaions, an in nie number of soluions. The general soluion reads Z x () = e R a()d ~x + e R a()d b () d : (4.3.6) Here, ~x is some arbirary consan. As his consan is arbirary, (4.3.6) indeed provides an in nie number of soluions o (4.3.1). To see ha (4.3.6) is a soluion o (4.3.1) indeed, remember he de niion of wha a soluion is. A soluion is a ime pah x () which sais es (4.3.1). Hence, we simply need o inser he ime pah given by (4.3.6) ino (4.3.1) and check wheher (4.3.1) hen holds. To his end, compue he ime derivaive of x(), d d x () = er a()d a () ~x + Z e R a()d b () d + e R a()d e R a()d b () ; R where we have used he de niion of he inegral in (4.3.2), d dx f (x) dx = f R (x) : Noe ha we do no have o apply he produc or chain rule since, again by (4.3.2), d dx g (x) h (x) dx = g (x) h (x) : Insering (4.3.6) gives _x () = a () x () + b () : Hence, (4.3.6) is a soluion o (4.3.1). Deermining he consan ~x To obain one paricular soluion, some value x ( 0 ) a some poin in ime 0 has o be xed. Depending on wheher 0 lies in he fuure (where 0 is usually denoed by T ) or in he pas, < T or 0 <, he equaion is solved forward or backward. backward forward 0 T Figure 4.3.2 Illusraing backward soluion (iniial value problem) and forward soluion (boundary value problem) We sar wih he backward soluion, i.e. where 0 <. Le he iniial condiion be x ( 0 ) = x 0. Then he soluion of (4.3.1) is R Z R a()d x () = e 0 x 0 + a(u)du 0 b () d R = x 0 e a()d 0 + e Z 0 0 e R a(u)du b () d: (4.3.7)

96 Chaper 4. Di erenial equaions Some inuiion for his soluion can be gained by considering special cases. Look rs a he case where b () = 0 for all (and herefore all ). The variable x () hen grows a a variable growh rae a () ; _x () =x () = a () from (4.3.1). The soluion o his ODE is R x () = x 0 e a()d 0 x 0 e a[ 0] R a()d where a 0 0 is he average growh rae of a beween 0 and : The soluion x 0 e a[ 0] has he same srucure as he soluion for a consan a - his ODE implies an exponenial increase of x () : Looking a a hen shows ha his exponenial increase now akes place a he average of a () over he period we are looking a. Now allow for a posiive b () : The soluion (4.3.7) says ha when a b () is added in ; he e ec on x () is given by he iniial b () imes an exponenial increase facor e R a(u)du ha akes he increase from o ino accoun. As a b () is added a each ; he ouer inegral R 0 :d sums over all hese individual conribuions. The forward soluion is required if 0 ; which we rename T for ease of disincion, lies in he fuure of, T > : Wih a erminal condiion x (T ) = x T, he soluion hen reads x () = x T e R T a()d Z T e R a(u)du b () d: (4.3.8) A similar inuiive explanaion as afer (4.3.7) can be given for his equaion. Veri caion We now show ha (4.3.7) and (4.3.8) are indeed a soluion for (4.3.1). Leibniz rule from (4.3.3), he ime derivaive of (4.3.7) is given by R _x () = e a()d 0 a () x 0 + e R a(u)du b () + When we pull ou a () and reinser (4.3.7), we nd R Z _x () = a () e a()d 0 x 0 + e R a(u)du b () d 0 = a () x () + b () : Z 0 e R a(u)du a () b () d: + b () Using he This shows us ha our funcion x () in (4.3.7) is in fac a soluion of (4.3.1) as x () in (4.3.7) sais es (4.3.1). The ime derivaive for he forward soluion in (4.3.8) is _x () = e R T Z T a()d a () x T + b () Z = a () e R T T a()d x T e R a(u)du b () d = a () x () + b () : e R a(u)du b () da () + b () Here, (4.3.8) was also reinsered ino he second sep. This shows ha (4.3.8) is also a soluion of (4.3.1).

4.4. Examples 97 4.3.3 Di erenial equaions as inegral equaions Any di erenial equaion can be wrien as an inegral equaion. While we will work wih he usual di erenial equaion represenaion mos of he ime, we inroduce he inegral represenaion here as i will be used frequenly laer when compuing momens in sochasic seups. Undersanding he inegral version of di erenial equaions in his deerminisic seup allows for an easier undersanding of inegral represenaions of sochasic di erenial equaions laer. The principle The non-auonomous di erenial equaion _x = f (; x) can be wrien equivalenly as an inegral equaion. To his end, wrie his equaion as dx = f (; x) d or, afer subsiuing s for, as dx = f (s; x) ds: Now apply he inegral R on boh sides. This 0 gives he inegral version of he di erenial equaion _x = f (; x) which reads An example Z 0 dx = x () x (0) = Z The di erenial equaion _x = a () x is equivalen o x () = x 0 + Z 0 0 f (s; x) ds: a (s) x (s) ds: (4.3.9) Compuing he derivaive of his equaion wih respec o ime gives, using (4.3.3), _x () = a () x () again. The presence of an inegral in (4.3.9) should no lead one o confuse (4.3.9) wih a soluion of _x = a () x in he sense of he las secion. Such a soluion would read x () = x 0 e R 0 a(s)ds. 4.4 Examples 4.4.1 Backward soluion: A growh model Consider an example inspired by growh heory. Le he capial sock of he economy follow _ K = I K, gross invesmen minus depreciaion gives he ne increase of he capial sock. Le gross invesmen be deermined by a consan saving rae imes oupu, I = sy (K) and he echnology be given by a linear AK speci caion. The complee di erenial equaion hen reads _K = sak K = (sa ) K: Is soluion is (see ch. 4.3.2) K () = e (sa ). As in he qualiaive analysis above, we found a muliude of soluions, depending on he consan : If we specify an iniial condiion, say we know he capial sock a = 0; i.e. K (0) = K 0, hen we can x he consan by = K 0 and our soluion nally reads K () = K 0 e (sa ) :

98 Chaper 4. Di erenial equaions 4.4.2 Forward soluion: Budge consrains As an applicaion of di erenial equaions, consider he budge consrain of a household. As in discree ime in ch. 3.5.2, budge consrains can be expressed in a dynamic and an ineremporal way. We rs show here how o derive he dynamic version and hen how o obain he ineremporal version from solving he dynamic one. Deriving a nominal dynamic budge consrain Following he idea of ch. 2.5.5, le us rs derive he dynamic budge consrain. In conras o ch. 2.5.5 in discree ime, we will see how sraighforward a derivaion is in coninuous ime. We sar from he de niion of nominal wealh. We have only one asse here. Nominal wealh is herefore given by a = kv; where k is he household s physical capial sock and v is he value of one uni of he capial sock. One can alernaively hink of k as he number of shares held by he household. By compuing he ime derivaive, wealh of a household changes according o _a = _ kv + k _v: (4.4.1) If he household wans o save, i can buy capial goods. The household s nominal savings in are given by s = w K k+w pc; he di erence beween facor rewards for capial (value marginal produc imes capial owned), labour income and expendiure. Dividing savings by he value of a capial good, i.e. he price of a share, gives he number of shares bough (or sold if savings are negaive), _k = wk k + w v pc : (4.4.2) Insering his ino (4.4.1), he equaion for wealh accumulaion gives, afer reinroducing wealh a by replacing k by a=v; _a = w K k + w pc + k _v = w K + _v k + w pc = wk + _v a + w v De ning he nominal ineres rae as pc: we have he nominal budge consrain i wk + _v ; (4.4.3) v _a = ia + w pc: (4.4.4) This shows why i is wise o always derive a budge consrain. Wihou a derivaion, (4.4.3) is missed ou and he meaning of he ineres rae i in he budge consrain is no known.

4.4. Examples 99 Finding he ineremporal budge consrain We can now obain he ineremporal budge consrain from solving he dynamic one in (4.4.4). Using he forward soluion from (4.3.8), we ake a (T ) = a T as he erminal condiion lying wih T > in he fuure. We are in oday. The soluion is hen Z T a () = e R T i()d a T Z T D () w () d + a () = D (T ) a T + Z T e R i(u)du [w () p () c ()] d, D () p () c () d; where D () e R i(u)du de nes he discoun facor. As we have used he forward soluion, we have obained an expression which easily lends iself o an economic inerpreaion. Think of an individual who - a he end of his life - does no wan o leave any beques, i.e. a T = 0: Then, his ineremporal budge consrain requires ha curren wealh on he lef-hand side, consising of he presen value of life-ime labour income plus nancial wealh a () needs o equal he presen value of curren and fuure expendiure on he righ-hand side. Now imagine ha a T > 0 as he individual does plan o leave a beques. Then his beques is visible as an expendiure on he righ-hand side. Curren wealh plus he presen value of wage income on he lef mus hen be high enough o provide for he presen value D (T ) a T of his beques and he presen value of consumpion. Wha abou deb in T? Imagine here is a fairy godmoher who pays all debs lef a he end of a life. Wih a T < 0 he household can consume more han curren wealh a () and he presen value of labour income - he di erence is jus he presen value of deb, D (T ) a T < 0: R 1 R T Now le he fuure poin T in ime go o in niy. Expressing lim T!1 f () d as f () d, he budge consrain becomes Z 1 D () w () d + a () = lim T!1 D (T ) a T + Z 1 D () p () c () d Wha would a negaive presen value of fuure wealh now mean, i.e. lim T!1 D (T ) a T < 0? If here was such a fairy godmoher, having deb would allow he agen o permanenly consume above is income levels and pay for his di erence by accumulaing deb. As fairy godmohers rarely exis in real life - especially when we hink abou economic aspecs - economiss usually assume ha lim D (T ) a T = 0: (4.4.5) T!1 This condiion is ofen called solvency or no-ponzi game condiion. Noe ha he no- Ponzi game condiion is a di eren concep from he boundedness condiion in ch. 5.3.2 or he ransversaliy condiion in ch. 5.4.3.

100 Chaper 4. Di erenial equaions Real wealh We can also sar from he de niion of he household s real wealh, measured in unis of he consumpion good, whose price is p. Real wealh is hen a r = kv : The change in p real wealh over ime is hen (apply he log on boh sides and derive wih respec o ime), _a r a = k _ r k + _v v _p p : Insering he increase ino he capial sock obained from (4.4.2) gives _a r a = wk k + w r vk pc + _v v _p p = wk k + w pc p 1 vkp 1 + _v v _p p : Using he expression for real wealh a r ; _a r = w K k p + w p w K + _v = v c + _v v ar a r + w p _p p _p p ar = wk v ar + w p c = ra r + w p c + _v v ar _p p ar c: (4.4.6) Hence, he real ineres rae r is - by de niion - r = wk + _v v _p p : The di erence o (4.4.3) simply lies in he in aion rae: Nominal ineres rae minus in- aion rae gives real ineres rae. Solving he di erenial equaion (4.4.6) again provides he ineremporal budge consrain as in he nominal case above. Now assume ha he price of he capial good equals he price of he consumpion good, v = p: This is he case in an economy where here is one homogeneous oupu good as in (2.4.9) or in (9.3.4). Then, he real ineres rae is equal o he marginal produc of capial, r = w K =p: 4.4.3 Forward soluion again: capial markes and uiliy The capial marke no-arbirage condiion Imagine you own wealh of worh v () : You can inves i on a bank accoun which pays a cerain reurn r () per uni of ime or you can buy shares of a rm which cos v () and which yield dividend paymens () and are subjec o changes _v () in is worh. In a world of perfec informaion and assuming ha in some equilibrium agens hold boh asses, he wo asses mus yield idenical income sreams, r () v () = () + _v () : (4.4.7)

4.4. Examples 101 This is a linear di erenial equaion in v (). As jus moivaed, i can be considered as a no-arbirage condiion. Noe, however, ha i is srucurally equivalen o (4.4.3), i.e. his no-arbirage condiion can be seen o jus de ne he ineres rae r (). Whaever he inerpreaion of his di erenial equaion is, solving i forward wih a erminal condiion v (T ) = v T gives according o (4.3.8) Leing T go o in niy, we have Z v () = e R T T r()d v T + e R r(u)du () d: v () = Z 1 e R r(u)du () d + lim e R T r()d v T : T!1 This forward soluion sresses he economic inerpreaion of v () : The value of an asse depends on he fuure income sream - dividend paymens () - ha are generaed from owning his asse. Noe ha i is usually assumed ha here are no bubbles, i.e. he limi is zero so ha he fundamenal value of an asse is given by he rs erm. For a consan ineres rae and dividend paymens and no bubbles, he expression for v () simpli es o v = =r: The uiliy funcion Consider an ineremporal uiliy funcion as i is ofen used in coninuous ime models, U () = Z 1 e [ ] u (c ()) d: (4.4.8) This is he sandard expression which corresponds o (2.2.12) or (3.1.1) in discree ime. Again, insananeous uiliy is given by u (:) : I depends here on consumpion only, where households consume coninuously a each insan : Impaience is capured as before by he ime preference rae : Higher values aached o presen consumpion are capured by he discoun funcion e [ ] ; whose discree ime analog in (3.1.1) is : Using (4.3.3), di ereniaing wih respec o ime gives us a linear di erenial equaion, _U () = u (c ()) + Z 1 d e [ ] u (c ()) d = u (c ()) + U () : d This equaion says ha overall uiliy U () decreases, as ime goes by, by insananeous consumpion u (c ()) : When is over, he opporuniy is gone: we can no longer enjoy uiliy from consumpion a : Bu U () also has an increasing componen: as he fuure comes closer, we gain U () : Solving his linear di erenial equaion forward gives U () = e [T ] U (T ) + Z T e [ ] u (c ()) d:

102 Chaper 4. Di erenial equaions Leing T go o in niy, we have U () = Z T e [ ] u (c ()) d + lim T!1 e [T ] U (T ) : The second erm is relaed o he ransversaliy condiion. 4.5 Linear di erenial equaion sysems A di erenial equaion sysem consiss of wo or more di erenial equaions which are muually relaed o each oher. Such a sysem can be wrien as _x () = Ax () + b; where he vecor x () is given by x = (x 1 ; x 2 ; x 3 ; :::; x n ) 0 ; A is an n n marix wih elemens a ij and b is a vecor b = (b 1 ; b 2 ; b 3 ; :::; b n ) 0 : Noe ha elemens of A and b can be funcions of ime bu no funcions of x: Wih consan coe ciens, such a sysem can be solved in various ways, e.g. by deermining so-called Eigenvalues and Eigenvecors. These sysems eiher resul from economic models direcly or are he oucome of a linearizaion of some non-linear sysem around a seady sae. This laer approach plays an imporan role for local sabiliy analyses (compared o he global analyses we underook above wih phase diagrams). These local sabiliy analyses can be performed for sysems of almos arbirary dimension and are herefore more general and (for he local surrounding of a seady sae) more informaive han phase diagram analyses. Please see he references in furher reading on many exbooks ha rea hese issues. 4.6 Furher reading and exercises There are many exbooks ha rea di erenial equaions and di erenial equaion sysems. Any library search ool will provide many his. This chaper owes insighs o Gandolfo (1996) on phase diagram analysis and di erenial equaions and - iner alia - o Brock and Malliaris (1989), Braun (1975) and Chiang (1984) on di erenial equaions. See also Gandolfo (1996) on di erenial equaion sysems. The predaor-prey model is reaed in various biology exbooks. I can also be found on many sies on he Inerne. The Leibniz rule was aken from Fichenholz (1997) and can be found in many oher exbooks on di ereniaion and inegraion. The AK speci caion of a echnology was made popular by Rebelo (1991).

4.6. Furher reading and exercises 103 Exercises chaper 4 Applied Ineremporal Opimizaion Using phase diagrams 1. Phase diagram I Consider he following di erenial equaion sysem, _x 1 = f (x 1 ; x 2 ) ; _x 2 = g (x 1 ; x 2 ) : Assume f x1 (x 1 ; x 2 ) < 0; g x2 (x 1 ; x 2 ) < 0; dx 2 dx 1 < 0; f(x1 ;x 2 )=0 dx 2 dx 1 > 0: g(x1 ;x 2 )=0 (a) Plo a phase diagram for he posiive quadran. (b) Wha ype of xpoin can be ideni ed wih his seup? 2. Phase diagram II (a) Plo wo phase diagrams for by varying he parameer b. (b) Wha ype of xpoins do you nd? _x = xy a; _y = y b; a > 0: (4.6.1) (c) Solve his sysem analyically. Noe ha y is linear and can easily be solved. Once his soluion is plugged ino he di erenial equaion for x; his becomes a linear di erenial equaion as well. 3. Phase diagram III (a) Plo pahs hrough poins marked by a do in he gure below. (b) Wha ype of xpoins are A and B?

104 Chaper 4. Di erenial equaions y &y = 0 B A &x = 0 O x 4. Local sabiliy analysis Sudy local sabiliy properies of he xpoin of he di erenial equaion sysem (4.6.1). 5. Phase diagram and xpoin Grossman and Helpman (1991) presen a growh model wih an increasing number of varieies. The reduced form of his economy can be described by a wo-dimensional di erenial equaion sysem, _n () = L a v () ; _v () = v () 1 n () ; where 0 < < 1 and a > 0. Variables v () and n () denoe he value of he represenaive rm and he number of rms, respecively. The posiive consans and L denoe he ime preference rae and x labour supply. (a) Draw a phase diagram (for posiive n () and v ()) and deermine he xpoin. (b) Wha ype of xpoin do you nd? 6. Solving linear di erenial equaions Solve _y () + y () = ; y (s) = 17 for (a) > s; (b) < s: (c) Wha is he forward and wha is he backward soluion? How do hey relae o each oher?

4.6. Furher reading and exercises 105 7. Comparing forward and backward soluions Remember ha R z 2 z 1 f (z) dz = R z 1 z 2 f (z) dz for any well-de ned z 1 ; z 2 and f (z) : Replace T by 0 in (4.3.8) and show ha he soluion is idenical o he one in (4.3.7). Explain why his mus be he case. 8. Derivaives of inegrals Compue he following derivaives. (a) (b) (c) (d) R d y d y a R d y d y a d d y d d y R b a f (s) ds; f (s; y) ds; f (y) dy; R f (y) dy: (e) Show ha he inegraion by pars formula R b a _xyd = [xy]b a R b x _yd holds. a 9. Ineremporal and dynamic budge consrains Consider he ineremporal budge consrain which equaes he discouned expendiure sream o asse holdings plus a discouned income sream, Z 1 D r () E () d = A () + Z 1 D r () I () d; (4.6.2) where D r () = exp A dynamic budge consrain reads Z r (s) ds : (4.6.3) E () + _ A () = r () A () + I () : (4.6.4) (a) Show ha solving he dynamic budge consrain h yields he ineremporal i budge consrain if and only if lim T!1 A (T ) exp r () d = 0: (b) Show ha di ereniaing he ineremporal budge consrain yields he dynamic budge consrain. 10. A budge consrain wih many asses Consider an economy wih wo asses whose prices are v i (). A household owns n i () asses of each ype such ha oal wealh a ime of he household is given by a () = v 1 () n 1 () + v 2 () n 2 () : Each asse pays a ow of dividends i () : Le he household earn wage income w () and spend p () c () on consumpion per uni of ime. Show ha he household s budge consrain is given by R T _a () = r () a () + w () p () c ()

106 Chaper 4. Di erenial equaions where he ineres raes are de ned by r () () r 1 () + (1 ()) r 2 () ; r i () i () + _v i () v i () and () v 1 () n 1 () =a () is de ned as he share of wealh held in asse 1: 11. Opimal saving Le opimal saving and consumpion behaviour (see ch. 5, e.g. eq. (5.1.6)) be described by he wo-dimensional sysem _c = gc; _a = ra + w c; where g is he growh rae of consumpion, given e.g. by g = r or g = (r ) =: Solve his sysem for ime pahs of consumpion c and wealh a: 12. ODE sysems Sudy ransiional dynamics in a wo-counry world. (a) Compue ime pahs for he number n i () of rms in counry i: The laws of moion are given by (Grossman and Helpman, 1991; Wälde, 1996) _n i = n A + n B L i n i (L + ) ; i = A; B; L = L A + L B ; ; > 0: Hin: Eigenvalues are g = (1 ) L > 0 and = (L + ). (b) Plo he ime pah of n A. Choose appropriae iniial condiions.

Chaper 5 Finie and in nie horizon models One widely used approach o solve deerminisic ineremporal opimizaion problems in coninuous ime consiss of using he so-called Hamilonian funcion. Given a cerain maximizaion problem, his funcion can be adaped - jus like a recipe - o yield a sraighforward resul. The rs secion will provide an inroducory example wih a nie horizon. I shows how easy i can someimes be o solve a maximizaion problem. I is useful o undersand, however, where he Hamilonian comes from. A lis of examples can never be complee, so i helps o be able o derive he appropriae opimaliy condiions in general. This will be done in he subsequen secion. Secion 5.4 hen discusses wha boundary condiions for maximizaion problems look like and how hey can be moivaed. The in nie planning horizon problem is hen presened and solved in secion 5.3 which includes a secion on ransversaliy and boundedness condiions. Various examples follow in secion 5.5. Secion 5.7 nally shows how o work wih presen-value Hamilonians and how hey relae o curren-value Hamilonians (which are he ones used in all previous secions). 5.1 Ineremporal uiliy maximizaion - an inroducory example 5.1.1 The seup Consider an individual ha wans o maximize a uiliy funcion similar o he one encounered already in (4.4.8), U () = Z T e [ ] ln c () d: (5.1.1) The planning period sars in and sops in T : The insananeous uiliy funcion is logarihmic and given by ln c () : The ime preference rae is : The budge consrain of his individual equaes changes in wealh, _a () ; o curren savings, i.e. he di erence 107

108 Chaper 5. Finie and in nie horizon models beween capial and labour income, r () a ()+w () ; and consumpion expendiure c (), _a () = r () a () + w () c () : (5.1.2) The maximizaion ask consiss of maximizing U () subjec o his consrain by choosing a pah of conrol variables, here consumpion and denoed by fc ()g : 5.1.2 Solving by opimal conrol This maximizaion problem can be solved by using he presen-value or he curren-value Hamilonian. We will work wih he curren-value Hamilonian here and in wha follows. Secion 5.7 presens he presen-value Hamilonian and shows how i di ers from he curren-value Hamilonian. The curren-value Hamilonian reads H = ln c () + () [r () a () + w () c ()] ; (5.1.3) where () is a muliplier of he consrain. I is called he cosae variable as i corresponds o he sae variable a () : In maximizaion problems wih more han one sae variable, here is one cosae variable for each sae variable. The cosae variable could also be called Hamilon muliplier - similar o he Lagrange muliplier. We show furher below ha () is he shadow price of wealh. The meaning of he erms sae, cosae and conrol variables is he same as in discree ime seups. Omiing ime argumens, opimaliy condiions are @H @c = 1 c _ = = 0; (5.1.4) @H @a = r: (5.1.5) The rs-order condiion in (5.1.4) is a usual opimaliy condiion: he derivaive of he Hamilonian (5.1.3) wih respec o he consumpion level c mus be zero. The second opimaliy condiion - a his sage - jus comes ou of he blue. Is origin will be discussed in a second. Applying logs o he rs rs-order condiion, ln c = ln ; and compuing derivaives wih respec o ime yields _c=c = =: _ Insering ino (5.1.5) gives he Euler equaion _c c = r, _c = r : (5.1.6) c As his ype of consumpion problem was rs solved by Ramsey in 1928 wih some suppor by Keynes, a consumpion rule of his ype is ofen called Keynes-Ramsey rule. This rule is one of he bes-known and mos widely used in Economics. I says ha consumpion increases when he ineres rae is higher han he ime preference rae. One reason is ha a higher ineres rae implies - a unchanged consumpion levels - a quicker increase in wealh. This is visible direcly from he budge consrain (5.1.2). A quicker increase in wealh allows for a quicker increase in consumpion. The second reason is ha

5.2. Deriving laws of moion 109 a higher ineres rae can lead o a change in he consumpion level (as opposed o is growh rae). This channel will be analyzed in deail owards he end of ch. 5.6.1. Equaions (5.1.2) and (5.1.6) form a wo-dimensional di erenial equaion sysem in a and c: This sysem can be solved given wo boundary condiions. How hese condiions can be found will be reaed in ch. 5.4. 5.2 Deriving laws of moion This subsecion shows where he Hamilonian comes from. More precisely, i shows how he Hamilonian can be deduced from he opimaliy condiions resuling from a Lagrange approach. The Hamilonian can herefore be seen as a shorcu which is quicker han he Lagrange approach bu leads o (i needs o lead o) idenical resuls. 5.2.1 The seup Consider he objecive funcion U () = Z T which we now maximize subjec o he consrain e [ ] u (y () ; z () ; ) d (5.2.1) _y () = Q (y () ; z () ; ) : (5.2.2) The funcion Q (:) is lef fairly unspeci ed. I could be a budge consrain of a household, a resource consrain of an economy or some oher consrain. We assume ha Q (:) has nice properies, i.e. i is coninuous and di ereniable everywhere. The objecive funcion is maximized by an appropriae choice of he pah fz()g of conrol variables. 5.2.2 Solving by he Lagrangian This problem can be solved by using he Lagrangian L = Z T e [ ] u () d + Z T () [Q () _y ()] d: The uiliy funcion u (:) and he consrain Q (:) are presened as u () and Q (), respecively. This shorens noaion compared o full expressions in (5.2.1) and (5.2.2). The inuiion behind his Lagrangian is similar o he one behind he Lagrangian in he discree ime case in (3.7.3) in ch. 3.7, where we also looked a a seup wih many consrains. The rs par is simply he objecive funcion. The second par refers o he consrains. In he discree-ime case, each poin in ime had is own consrain wih is own Lagrange muliplier. Here, he consrain (5.2.2) holds for a coninuum of poins.

110 Chaper 5. Finie and in nie horizon models Hence, insead of he sum in he discree case we now have an inegral over he produc of mulipliers () and consrains. This Lagrangian can be rewrien as follows, L = = Z T e [ Z T ] u () + () Q () d e [ ] u () + () Q () d + Z T Z T () _y () d _ () y () d [ () y ()] T (5.2.3) where he las sep inegraed by pars and [ () y ()] T is he inegral funcion of () y () evaluaed a T minus is level a. Now assume ha we could choose no only he conrol variable z () ; bu also he sae variable y () a each poin in ime. The inuiion for his is he same as in discree ime in ch. 3.7.2. Hence, we maximize he Lagrangian (5.2.3) wih respec o y and z a one paricular 2 [; T ] ; i.e. we compue he derivaive wih respec o z () and y (). For he conrol variable z () ; we ge a rs-order condiion When we de ne we nd e [ ] u z () + () Q z () = 0, u z () + e [ ] () Q z () = 0: For he sae variable y () ; we obain () e [ ] () ; (5.2.4) u z () + () Q z () = 0: (5.2.5) e [ ] u y () + () Q y () + _ () = 0, u y () () e [ ] Q y () = e [ ] _ () : (5.2.6) Di ereniaing (5.2.4) wih respec o ime and resinsering (5.2.4) gives _ () = e [ ] () + e [ ] _ () = () + e [ ] _ () : Insering (5.2.6) and (5.2.4), we obain _ () = () u y () () Q y () : (5.2.7) Equaions (5.2.5) and (5.2.7) are he wo opimaliy condiions ha solve he above maximizaion problem joinly wih he consrain (5.2.2). We have hree equaions which x hree variables: The rs condiion (5.2.5) deermines he opimal level of he conrol variable z: As his opimaliy condiion holds for each poin in ime ; i xes an enire pah for z: The second opimaliy condiion (5.2.7) xes a ime pah for : By leing he cosae follow an appropriae pah, i makes sure, ha he level of he sae variable (which is no insananeously adjusable as he maximizaion of he Lagrangian would sugges) is as if i had been opimally chosen a each insan. Finally, he consrain (5.2.2) xes he ime pah for he sae variable y:

5.3. The in nie horizon 111 5.2.3 Hamilonians as a shorcu Le us now see how Hamilonians can be jusi ed. The opimal conrol problem coninues o be he one in ch.5.2.1. De ne he Hamilonian similar o (5.1.3), H = u () + () Q () : (5.2.8) In fac, his Hamilonian shows he general srucure of Hamilonians. Take he insananeous uiliy level (or any oher funcion behind he discoun erm in he objecive funcion) and add he cosae variable muliplied by he righ-hand side of he consrain. Opimaliy condiions are hen H z = 0; (5.2.9) _ = H y : (5.2.10) These condiions were already used in (5.1.4) and (5.1.5) in he inroducory example in he previous chaper 5.1 and in (5.2.5) and (5.2.7): When he derivaives H z and H y in (5.2.9) and (5.2.10) are compued from (5.2.8), his yields equaions (5.2.5) and (5.2.7). Hamilonians are herefore jus a shorcu ha allow us o obain resuls faser han in he case where Lagrangians are used. Noe for laer purposes ha boh and have ime as an argumen. There is an inerpreaion of he cosae variable which we simply sae a his poin (see ch. 6.2 for a formal derivaion): The derivaive of he objecive funcion wih respec o he sae variable a, evaluaed on he opimal pah, equals he value of he corresponding cosae variable a. Hence, jus as in he saic Lagrange case, he cosae variable measures he change in uiliy as a resul of a change in endowmen (i.e. in he sae variable). Expressing his formally, de ne he value funcion as V (y ()) max fz()g U () ; idenical in spiri o he value funcion in dynamic programming as we go o know i in discree ime. The derivaive of he value funcion, he shadow price V 0 (y ()) ; is hen he change in uiliy when behaving opimally resuling from a change in he sae y () : This derivaive equals he cosae variable, V 0 (y ()) =. 5.3 The in nie horizon 5.3.1 Solving by opimal conrol Seup In he in nie horizon case, he objecive funcion has he same srucure as before in e.g. (5.2.1) only ha he nie ime horizon T is replaced by an in nie ime horizon 1. The consrains are unchanged and he maximizaion problem reads max fz()g Z 1 e [ ] u (y () ; z () ; ) d;

112 Chaper 5. Finie and in nie horizon models subjec o _y () = Q (y () ; z () ; ) ; y () = y : (5.3.1) We need o assume for his problem ha he inegral R 1 e [ ] u (:) d converges for all feasible pahs of y () and z (), oherwise he opimaliy crierion mus be rede ned. This boundedness condiion is imporan only in his in nie horizon case. Consider he following gure for he nie horizon case. e ρ[τ ] u(y(τ),z(τ),τ) T Figure 5.3.1 Bounded objecive funcion for a nie horizon If individuals have a nie horizon (planning sars a and ends a T ) and he uiliy funcion u (:) is coninuous over he enire planning period (as drawn), he objecive funcion (he shaded area) is nie and he boundedness problem disappears. (As is clear from he gure, he condiion of a coninuous u (:) could be relaxed.) Clearly, making such an assumpion is no always innocuous and one should check, a leas afer having solved he maximizaion problem, wheher he objecive funcion indeed converges. This will be done in ch. 5.3.2. Opimaliy condiions The curren-value Hamilonian as in (5.2.8) is de ned by H = u () + () Q () : Opimaliy condiions are (5.2.9) and (5.2.10), i.e. @H @z = 0; _ = @H @y : Hence, we have idenical opimaliy condiions o he case of a nie horizon.

5.4. Boundary condiions and su cien condiions 113 5.3.2 The boundedness condiion When inroducing an in nie horizon objecive funcion, i was sressed righ afer (5.3.1) ha objecive funcions mus be nie for any feasible pahs of he conrol variable. Oherwise, overall uiliy U () would be in niely large and here would be no objecive for opimizing - we are already in niely happy! This problem of unlimied happiness is paricularly severe in models where conrol variables grow wih a consan rae, hink e.g. of consumpion in a model of growh. A pragmaic approach o checking wheher growh is no oo high is o rs assume ha i is no oo high, hen o maximize he uiliy funcion and aferwards check wheher he iniial assumpion is sais ed. As an example, consider he uiliy funcion U () = R 1 e [ ] u (c ()) d: The insananeous uiliy funcion u (c ()) is characerized by consan elasiciy of subsiuion as in (2.2.10), u (c ()) = c ()1 1 ; > 0: (5.3.2) 1 Assume ha consumpion grows wih a rae of g, where his growh rae resuls from uiliy maximizaion. Think of his g as represening e.g. he di erence beween he ineres rae and he ime preference rae, correced by ineremporal elasiciy of subsiuion, as will be found laer e.g. in he Keynes-Ramsey rule (5.6.8), i.e. _c=c = (r ) = g. Consumpion a is hen given by c () = c () e g[ ] : Wih his exponenial growh of consumpion, he uiliy funcion becomes Z 1 U () = (1 ) 1 c () 1 e [ ] e (1 )g[ ] d + 1 : This inegral is bounded if and only if he boundedness condiion (1 ) g < 0 holds. This can formally be seen by compuing he inegral explicily and checking under which condiions i is nie. Inuiively, his condiion makes sense: Insananeous uiliy from consumpion grows by a rae of (1 ) g: Impaience implies ha fuure uiliy is discouned by he rae : Only if his ime preference rae is large enough, he overall expression wihin he inegral, e [ ] C () 1 1 ; will fall in : 5.4 Boundary condiions and su cien condiions So far, maximizaion problems were presened wihou boundary condiions. Usually, however, boundary condiions are par of he maximizaion problem. Wihou boundary condiions, he resuling di erenial equaion sysem (e.g. (5.1.2) and (5.1.6) from he inroducory example in ch. 5.1) has an in nie number of soluions and he level of conrol and sae variables is no pinned down. We will now consider hree cases. All cases will laer be illusraed in he phase diagram of secion 5.5.

114 Chaper 5. Finie and in nie horizon models 5.4.1 Free value of he sae variable a he endpoin Many problems are of he form (5.2.1) and (5.2.2), where boundary values for he sae variable are given by y () = y ; y (T ) free. (5.4.1) The rs condiion is he usual iniial condiion. The second condiion allows he sae variable o be freely chosen for he end of he planning horizon. We can use he curren-value Hamilonian (5.2.8) o obain opimaliy condiions (5.2.9) and (5.2.10). In addiion o he boundary condiion y () = y from (5.4.1), we have (cf. Feichinger and Harl, 1986, p. 20), (T ) = 0: (5.4.2) Wih his addiional condiion, we have wo boundary condiions which allows us o solve our di erenial equaion sysem (5.2.9) and (5.2.10). This yields a unique soluion and an example for his will be discussed furher below in ch. 5.5. 5.4.2 Fixed value of he sae variable a he endpoin Now consider (5.2.1) and (5.2.2) wih one iniial and one erminal condiion, y () = y ; y (T ) = y T : (5.4.3) In order for his problem o make sense, we assume ha a feasible soluion exiss. This should generally be he case, bu i is no obvious: Consider again he inroducory example in ch. 5.1. Le he endpoin condiion be given by he agen is very rich in T ; i.e. a (T ) = very large. If a (T ) is oo large, even zero consumpion a each poin in ime, c () = 0 8 2 [; T ] ; would no allow wealh a o be as large as required by a (T ) : In his case, no feasible soluion would exis. We assume, however, ha a feasible soluion exiss. Opimaliy condiions are hen idenical o (5.2.9) and (5.2.10), plus iniial and boundary values (5.4.3). Again, wo di erenial equaions wih wo boundary condiions gives level informaion abou he opimal soluion and no jus informaion abou changes. The di erence beween his approach and he previous one is ha, now, y (T ) = y T is exogenously given, i.e. par of he maximizaion problem. Before he corresponding (T ) = 0 was endogenously deermined as a necessary condiion for opimaliy. 5.4.3 The ransversaliy condiion The analysis of he maximizaion problem wih an in nie horizon in ch. 5.3 also led o a sysem of wo di erenial equaions. One boundary condiion is provided by he iniial condiion in (5.3.1) for he sae variable. Hence, again, we need a second condiion o pin down he iniial level of he conrol variable, e.g. he iniial consumpion level.

5.4. Boundary condiions and su cien condiions 115 In he nie horizon case, we had erminal condiions of he ype K (T ) = K T or (T ) = 0 in (5.4.2) and (5.4.3). As no such erminal T is now available, hese condiions need o be replaced by alernaive speci caions. I is useful o draw a disincion beween absrac condiions and condiions which have a pracical value in he sense ha hey can be used o explicily compue he iniial consumpion level. A rs pracically useful condiion is he no-ponzi game condiion (4.4.5) resuling from consideraions concerning he budge consrain, Z T lim y (T ) exp r () d = 0 T!1 where r @H=@y. Noe ha his no-ponzi game condiion can be rewrien as lim e T (T ) y (T ) = lim (T ) y (T ) = 0; (5.4.4) T!1 T!1 where he fac ha _ = = r implies ha e = 0 e R T r()d was used. The formulaions in (5.4.4) of he no-ponzi game condiion is frequenly encounered as a second boundary condiion. We will use i laer in he example of ch. 5.6.1. A second useful way o deermine levels of variables is he exisence of a long-run seady sae. Wih a well-de ned seady sae, one generally analyses properies on he saddle pah which leads o his seady sae. On his saddle pah, he level of variables is deermined, a leas graphically. Numerical soluions also exis and someimes analyical closed-form soluions can be found. Ofen, an analysis of he seady sae alone is su - cien. An example where levels of variables are deermined when analysing ransiional dynamics on he saddle-pah leading o he seady sae is he cenral planner problem sudied in ch. 5.6.3. Concerning absrac condiions, a condiion occasionally encounered is he ransversaliy condiion (TVC), lim!1 f () [y () y ()]g = 0; where y () is he pah of y () for an opimal choice of conrol variables. There is a considerable lieraure on he necessiy and su ciency of he TVC and no aemp is made here o cover i. Various references o he lieraure on he TVC are in secion 5.8 on furher reading. 5.4.4 Su cien condiions So far, we have only presened condiions ha are necessary for a maximum. We do no ye know, however, wheher hese condiions are also su cien. Su ciency can be imporan, however, as i can be easily recalled when hinking of a saic maximizaion problem. Consider max x f (x) where f 0 (x ) = 0 is necessary for an inerior maximum. This is no su cien as f 0 (x + ") could be posiive for any " 6= 0. For our purposes, necessary condiions are su cien if eiher (i) he funcions u (:) and Q (:) in (5.2.1) and (5.2.2) are concave in y and z and if () is posiive for all ; (ii) Q (:) is linear in y and z for any () or (iii) Q (:) is convex and () is negaive for all :

116 Chaper 5. Finie and in nie horizon models The concaviy of he uiliy funcion u (:) and consrain Q (:) can easily be checked and obviously hold, for example, for sandard logarihmic and CES uiliy funcions (as in (2.2.10) or (3.9.4)) and for consrains conaining a echnology as in (5.6.12) below. The sign of he shadow price can be checked by looking a he rs order condiions as e.g. (5.1.4) or (5.6.13) laer. As we usually assume ha uiliy increases in consumpion, we see ha - for a problem o make economically sense - he shadow price is posiive. Lineariy in condiion (ii) is ofen ful lled when he consrain is e.g. a budge consrain. See furher reading on references wih a more formal reamen of su cien condiions. 5.5 Illusraing boundary condiions Le us now consider an example from microeconomics. We consider a rm ha operaes under adjusmen coss. This will bring us back o phase-diagram analysis, o an undersanding of he meaning of xed and free values of sae variables a he end poins for a nie planning horizon T; and o he meaning of ransversaliy condiions for he in nie horizon. 5.5.1 A rm wih adjusmen coss The maximizaion problem we are now ineresed in is a rm ha operaes under adjusmen coss. Capial can no be rened insananeously on a spo marke bu is insallaion is cosly. The crucial implicaion of his simple generalizaion of he sandard heory of producion implies ha rms all of a sudden have an ineremporal and no longer a saic opimizaion problem. As one consequence, facors of producion are hen no longer paid heir value marginal produc as in saic rm problems. The model A rm maximizes he presen value 0 of is fuure insananeous pro s (), 0 = Z T subjec o a capial accumulaion consrain 0 e r () d; (5.5.1) _K () = I () K () : (5.5.2) Gross invesmen I () minus depreciaion K () gives he ne increase of he rm s capial sock. Insananeous pro s are given by he di erence beween revenue and cos, () = pf (K ()) (I ()) : (5.5.3) Revenue is given by pf (K ()) where he producion echnology F (:) employs capial only. Oupu increases in capial inpu, F 0 (:) > 0; bu o a decreasing exen, F 00 (:) < 0.

5.5. Illusraing boundary condiions 117 The rm s coss are given by he cos funcion (I ()) : As he rm owns capial, i does no need o pay any renal coss for capial. Coss ha are capured by (:) are adjusmen coss which include boh he cos of buying and of insalling capial. The iniial capial sock is given by K (0) = K 0. The ineres rae r is exogenous o he rm. Maximizaion akes place by choosing a pah of invesmen fi ()g : The maximizaion problem is presened slighly di erenly from previous chapers. We look a he rm from he perspecive of a poin in ime zero and no - as before - from a poin in ime : This is equivalen o saying ha we normalize o zero. Boh ypes of objecive funcions are used in he lieraure. One wih normalizaion of oday o zero as in (5.5.1) and one wih a planning horizon saring in : Economically, his is of no major imporance. The presenaion wih represening oday as normally used in his book is slighly more general and is more useful when dynamic programming is he mehod chosen for solving he maximizaion problem. We now use a problem saring in 0 o show ha no major di erences in he soluion echniques arise. Soluion This rm obviously has an ineremporal problem, in conras o he rms we encounered so far. Before solving his problem formally, le us ask where his ineremporal dimension comes from. By looking a he consrain (5.5.2), his becomes clear: Firms can no longer insananeously ren capial on some spo marke. The rm s capial sock is now a sae variable and can only change slowly as a funcion of invesmen and depreciaion. As an invesmen decision oday has an impac on he capial sock omorrow, i.e. he decision oday a ecs fuure capial levels, here is an ineremporal link beween decisions and oucomes a di ering poins in ime. As discussed afer (5.2.8), he curren-value Hamilonian combines he funcion afer he discoun erm in he objecive funcion, here insananeous pro s () ; wih he consrain, here I () K () ; and uses he cosae variable (). This gives H = () + () [I () K ()] = pf (K ()) (I ()) + () [I () K ()] : Following (5.2.9) and (5.2.10), opimaliy condiions are H I = 0 (I ()) + () = 0; (5.5.4) _ () = r H K () = r pf 0 (K ()) + = (r + ) pf 0 (K ()) ; (5.5.5) The opimaliy condiion for in (5.5.5) shows ha he value marginal produc of capial, pf 0 (K ()) ; sill plays a role and i is sill compared o he renal price r of capial, bu here is no longer an equaliy as in saic models of he rm. We will reurn o his poin in exercise 2 of ch. 6.

118 Chaper 5. Finie and in nie horizon models Opimaliy condiions can be presened in a simpler way (i.e. wih fewer endogenous variables). Firs, solve he rs opimaliy condiion for he cosae variable and compue he ime derivaive, _ () = 00 (I ()) I _ () : Second, inser his ino he second opimaliy condiion (5.5.5) o nd 00 (I ()) _ I () = (r + ) 0 (I ()) pf 0 (K ()) : (5.5.6) This equaion, ogeher wih he capial accumulaion consrain (5.5.2), is a wo-dimensional di erenial equaion sysem ha can be solved, given he iniial condiion K 0 and one addiional boundary condiion. An example Now assume adjusmen coss are of he form (I) = vi + I 2 =2: (5.5.7) The price o be paid per uni of capial is given by he consan v and coss of insallaion are given by I 2 =2: This quadraic erm capures he idea ha insallaion coss are low and do no increase quickly, i.e. underproporionally o he new capial sock, a low levels of I bu increase overpropionally when I becomes large. Then, opimaliy requires (5.5.2) and, from insering (5.5.7) ino (5.5.6), _ I () = (r + ) (v + I ()) pf 0 (K ()) (5.5.8) A phase diagram using (5.5.2) and (5.5.8) is ploed in he following gure. As one can see, we can unambiguously deermine ha dynamic properies are represened by a saddle-pah sysem wih a saddle poin as de ned in def. 4.2.2. Figure 5.5.1 A rm wih adjusmen cos We now have o selec one of his in nie number of pahs ha sar a K 0 : Wha is he correc invesmen level I 0? This depends on how we choose he second boundary condiion.

5.5. Illusraing boundary condiions 119 5.5.2 Free value a he end poin One modelling opporuniy consiss of leaving he value of he sae variable a he end poin open as in ch. 5.4.1. The condiion is hen (T ) = 0 from (5.4.2) which in he conex of our example requires 0 (I (T )) = 0, I (T ) = 0: from he rs-order condiion (5.5.4) and he example for chosen in (5.5.7). In words, he rajecory where he invesmen level is zero a T is he opimal one. Le us now see how his informaion helps us o idenify he level of invesmen and capial, i.e. he corresponding rajecory in he phase diagram by looking a g. 5.5.1. Look a he rajecory saring a poin A rs. This rajecory crosses he zero-moion line for capial afer some ime and evenually his he horizonal axis where I = 0: Have we now found a rajecory which sais es all opimaliy condiions? No ye, as we do no know wheher he ime needed o go from A o C is exacly of lengh T: If we sared a B and wen o D; we would also end up a I = 0; also no knowing wheher he lengh is T: Hence, in order o nd he appropriae rajecory, a numerical soluion is needed. Such a soluion would hen compue various rajecories as he ones saring a A and B and compare he lengh required o reach he horizonal axis. When he rajecory requiring T o hi he horizonal axis is found, levels of invesmen and capial are ideni ed. There is also hope ha searching for he correc rajecory does no ake oo long. We know ha saring on he saddle pah which leads o he seady sae acually never brings us o he seady sae. Capial K () and invesmen I () approach he seady sae asympoically bu never reach i. As an example, he pah for invesmen would look qualiaively like he graph saring wih x 03 in g. 4.2.2 which approaches x from above bu never reaches i. If we sar very close o he saddle pah, le s say a B, i akes more ime o go owards he seady sae and hen o reurn han on he rajecory ha sars a A: Time o reach he horizonal line is in niy when we sar on he saddle pah (i.e. we never reach he horizonal line). As ime falls, he lower he iniial invesmen level, he correc pah can easily be found. 5.5.3 Fixed value a he end poin Le us now consider he case where he end poin requires a xed capial sock K T. The rs aspec o be checked is wheher K T can be reached in he planning period of lengh T: Maybe K T is simply oo large. Can we see his in our equaions? If we look a he invesmen equaion (5.5.2) only, any K T can be reached by seing he invesmen levels I (T ) jus high enough. When we look a period pro s in (5.5.3), however, we see ha here is an upper invesmen level above which pro s become negaive. If we wan o rule ou negaive pro s, invesmen levels are bounded from above a each poin in ime by () 0 and some values a he endpoin K T are no feasible. The maximizaion problem would have no soluion.

120 Chaper 5. Finie and in nie horizon models If we now look a a more opimisic example and le K T no be oo high, hen we can nd he appropriae rajecory in a similar way as before where I (T ) = 0: Consider he K T drawn in he gure 5.5.1. When he iniial invesmen level is a poin F; he level K T will be reached faser han on a rajecory ha sars beween F and E: As ime spen beween K 0 and K T is monoonically decreasing, he higher he iniial consumpion level, i.e. he furher he rajecory is away from he seady sae, he appropriae iniial level can again be easily found by numerical analysis. 5.5.4 In nie horizon and ransversaliy condiion Le us nally consider he case where he planning horizon is in niy, i.e. we replace T by 1. Which boundary condiion shall we use now? Given he discussion in ch. 5.4.3, one would rs ask wheher here is some ineremporal consrain. As his is no he case in his model (an example for his will be reaed shorly in ch. 5.6.1), one can add an addiional requiremen o he model analyzed so far. One could require he rm o be in a seady sae in he long run. This can be jusi ed by he observaion ha mos rms have a relaively consan size over ime. The soluion of an economic model of a rm should herefore be characerized by he feaure ha, in he absence of furher shocks, he rm size should remain consan. The only poin where he rm is in a seady sae is he inersecion poin of he zero-moion lines. The quesion herefore arises where o sar a K 0 if one wans o end up in he seady sae. The answer is clearly o sar on he saddle pah. As we are in a deerminisic world, a ransiion owards he seady sae would ake place and capial and invesmen approach heir long-run values asympoically. If here were any unanicipaed shocks which push he rm o his saddle pah, invesmen would insananeously be adjused such ha he rm is back on he saddle pah. 5.6 Furher examples This secion presens furher examples of ineremporal opimizaion problems which can be solved by employing he Hamilonian. The examples show boh how o compue opimaliy condiions and how o undersand he predicions of he opimaliy condiions. 5.6.1 In nie horizon - opimal consumpion pahs Le us now look a an example wih in nie horizon. We focus on he opimal behaviour of a consumer. The problem can be posed in a leas wo ways. In eiher case, one par of he problem is he ineremporal uiliy funcion U () = Z 1 e [ ] u (c ()) d: (5.6.1)

5.6. Furher examples 121 Due o he general insananeous uiliy funcion u (c ()), i is somewha more general han e.g. (5.1.1). The second par of he maximizaion problem is a consrain limiing he oal amoun of consumpion. Wihou such a consrain, maximizing (5.6.1) would be rivial (or meaningless): Wih u 0 (c ()) > 0 maximizing he objecive simply means seing c () o in niy. The way his consrain is expressed deermines he way in which he problem is solved mos sraighforwardly. Solving by Lagrangian The consrain o (5.6.1) is given by a budge consrain. The rs way in which his budge consrain can be expressed is, again, he ineremporal formulaion, Z 1 D r () E () d = a () + Z 1 D r () w () d; (5.6.2) where E () = p () c () and D r () = e R r(u)du : The maximizaion problem is hen given by: maximize (5.6.1) by choosing a pah fc()g subjec o he budge consrain (5.6.2). We build he Lagrangean wih as he ime-independen Lagrange muliplier Z 1 e [ L = ] u (c ()) d Z 1 D r () E () d a () Z 1 D r () w () d : Noe ha in conras o secion 5.2.2 where a coninuum of consrains implied a coninuum of Lagrange mulipliers (or, in an alernaive inerpreaion, a ime-dependen muliplier), here is only one consrain here. The opimaliy condiions are he consrain (5.6.2) and he parial derivaive wih respec o consumpion c () a one speci c poin in ime, i.e. he rs-order condiion for he Lagrangian, L c() = e [ ] u 0 (c ()) D r () p () = 0, D r () 1 e [ ] p () 1 = u 0 (c ()) 1 : (5.6.3) Noe ha his rs-order condiion represens an in nie number of rs-order condiions: one for each poin in ime beween and in niy. See ch. 4.3.1 for some background on how o compue a derivaive in he presence of inegrals. Applying logs o (5.6.3) yields Z r (u) du [ ] ln p () = ln ln u 0 (c ()) : Di ereniaing wih respec o ime gives he Keynes-Ramsey rule u 00 (c ()) _p _c () = r () u 0 (c ()) () p () : (5.6.4)

122 Chaper 5. Finie and in nie horizon models u 00 (c()) u 0 (c()) The Lagrange muliplier drops ou as i is no a funcion of ime. Noe ha is Arrow s measure of absolue risk aversion which is a measure of he curvaure of he uiliy funcion. In our seup of cerainy, i is more meaningful, however, o hink of he ineremporal elasiciy of subsiuion. Even hough we are in coninuous ime now, i can be de ned, as in (2.2.9), in discree ime, replacing he disance of 1 from o he nex period + 1 by a period of lengh : One could hen go hrough he same seps as afer (2.2.9) and obain idenical resuls for a CES and logarihmic insananeous uiliy funcion (see ex. 4). Wih a logarihmic uiliy funcion, u (c ()) = ln c (), u 0 (c ()) = 1 and c() u00 (c ()) = 1 c() 2 and he Keynes-Ramsey rule becomes Employing he Hamilonian _c () c () = r () _p () p () : (5.6.5) In conras o above, he uiliy funcion (5.6.1) here is maximized subjec o he dynamic (or ow) budge consrain, The soluion is obained by solving exercise 1. The ineres rae e ec on consumpion _a () = r () a () + w () p () c () : (5.6.6) As an applicaion for he mehods we have go o know so far, imagine here is a discovery of a new echnology in an economy. All of a sudden, compuers or cell-phones or he Inerne is available on a large scale. Imagine furher ha his implies an increase in he reurns on invesmen (i.e. he ineres rae r): For any Euro invesed, more comes ou han before he discovery of he new echnology. Wha is he e ec of his discovery on consumpion? To ask his more precisely: wha is he e ec of a change in he ineres rae on consumpion? To answer his quesion, we sudy a maximizaion problem as he one jus solved, i.e. he objecive funcion is (5.6.1) and he consrain is (5.6.2). We simplify he maximizaion problem, however, by assuming a CES uiliy funcion u (c ()) = c () 1 1 = (1 ) ; a consan ineres rae and a price being equal o one (imagine he consumpion good is he numeraire). This implies ha he budge consrain reads Z 1 e r[ ] c () d = a () + Z 1 e r[ ] w () d (5.6.7) and from insering he CES uiliy funcion ino (5.6.4), he Keynes-Ramsey rule becomes _c () c () = r : (5.6.8)

5.6. Furher examples 123 One e ec, he growh e ec, is sraighforward from (5.6.8) or also from (5.6.4). A higher ineres rae, ceeris paribus, increases he growh rae of consumpion. The second e ec, he e ec on he level of consumpion, is less obvious, however. In order o undersand i, we underake he following seps. Firs, we solve he linear di erenial equaion in c () given by (5.6.8). Following ch. 4.3, we nd c () = c () e r ( ) : (5.6.9) Consumpion, saring oday in wih a level of c () ; grows exponenially over ime a he rae (r ) = o reach he level c () a some fuure > : In he second sep, we inser his soluion ino he lef-hand side of he budge consrain (5.6.7) and nd Z 1 e r[ ] c () e r ( ) d = c () = c () Z 1 r e (r r )[ ] d 1 r i he (r r 1 )[ ] The simpli caion sems from he fac ha c () ; he iniial consumpion level, can be pulled ou of he erm R 1 e r[ ] c () d; represening he presen value of curren and fuure consumpion expendiure. Please noe ha c () could be pulled ou of he inegral also in he case of a non-consan ineres rae. Noe also ha we do no need o know wha he level of c () is, i is enough o know ha here is some c () in he soluion (5.6.9), whaever is level. Wih a consan ineres rae, he remaining inegral can be solved explicily. Firs r noe ha r mus be negaive. Consumpion growh would oherwise exceed he ineres rae and a boundedness condiion for he objecive funcion similar o he one in ch. 5.3.2 would evenually be violaed. (Noe, however, ha boundedness in ch. 5.3.2 refers o he uiliy funcion, here we focus on he presen value of consumpion.) Hence, we assume r > r, (1 ) r < : Therefore, for he presen value of consumpion expendiure we obain : c () r 1 r i he (r r 1 )[ ] = c () = c () r r r = 1 r [0 1] c () (1 ) r and, insered ino he budge consrain, his yields a closed-form soluion for consumpion, Z (1 ) r 1 c () = a () + e r[ ] w () d : (5.6.10) For he special case of a logarihmic uiliy funcion, he fracion in fron of he curly brackes simpli es o (as = 1).

124 Chaper 5. Finie and in nie horizon models Afer hese wo seps, we have wo resuls, boh visible in (5.6.10). One resul shows ha iniial consumpion c () is a fracion ou of wealh of he household. Wealh needs o be undersood in a more general sense han usual, however: I is nancial wealh a () plus, wha could be called human wealh (in an economic, i.e. maerial sense), he presen value of labour income, R 1 e r[ ] w () d: Going beyond oday and realizing ha his analysis can be underaken for any poin in ime, he relaionship (5.6.10) of course holds on any poin of an opimal consumpion pah. The second resul is a relaionship beween he level of consumpion and he ineres rae, our original quesion. We now need o undersand he derivaive dc () =dr in order o furher exploi (5.6.10). If we focus only on he erm in fron of he curly brackes, we nd for he change in he level of consumpion when he ineres rae changes dc () dr = 1 f:g R 0, R 1: The consumpion level increases when he ineres rae rises if is larger han one, i.e. if he ineremporal elasiciy of subsiuion 1 is smaller han uniy. This is probably he empirically more plausible case (compared o < 1) on he aggregae level. There is micro-evidence, however, where he ineremporal elasiciy of subsiuion can be much larger han uniy. This nding is summarized in he following gure. N lnc() r 2 > r 1 c 2 () c 1 () r 1 > 1 ime N Figure 5.6.1 The e ec of he ineres rae on consumpion growh and consumpion level for an ineremporal elasiciy of subsiuion smaller han one, i.e. > 1 The boundary condiion for he in nie horizon The seps we jus wen hrough are also an illusraion of how o use he no-ponzi game condiion as a condiion o obain level informaion in an in nie horizon problem. We herefore jus saw an example for he discussion in ch. 5.4.3 on ransversaliy condiions. Solving a maximizaion problem by Hamilonian requires a dynamic budge consrain, i.e. a di erenial equaion. The soluion is a Keynes-Ramsey rule, also a di erenial

5.6. Furher examples 125 equaion. These wo di erenial equaions require wo boundary condiions in order o obain a unique soluion. One boundary condiion is he iniial sock of wealh, he second boundary condiion is he no-ponzi game condiion. We jus saw how his second condiion can indeed be used o obain level informaion for he conrol and he sae variable: The No-Ponzi game condiion allows us o obain an ineremporal budge consrain of he ype we usually wan o work hrough solving he dynamic budge consrain - see ch. 4.4.2 on Finding he ineremporal budge consrain. (In he example we jus looked a, we did no need o derive an ineremporal budge consrain as i was already given in (5.6.2).) Using he Keynes-Ramsey rule in he way we jus did provides he iniial consumpion level c () : Hence, by using a boundary condiion for his in nie horizon problem, we were able o obain level informaion in addiion o informaion on opimal changes. Noe ha he principle used here is idenical o he one used in he analysis of level e ecs in ch. 3.4.3 on opimal R&D e or. 5.6.2 Necessary condiions, soluions and sae variables The previous example provides a good opporuniy o provide a more in-deph explanaion of some conceps ha were inroduced before. A disincion was drawn beween a soluion and a necessary condiion in ch. 2.2.2 when discussing (2.2.6). The saring poin of our analysis here is he Keynes-Ramsey rule (5.6.8) which is a necessary condiion for opimal behaviour. The soluion obained here is given in (5.6.10) and is he oucome of solving he di erenial equaion (5.6.8) and using he ineremporal budge consrain. Looking a hese expressions clearly shows ha (5.6.8), he oucome of modifying necessary condiions, conains much less informaion han he soluion in (5.6.10). The Keynes-Ramsey rule provides informaion abou he change of he conrol variable only while he soluion provides informaion abou he level. The soluion in (5.6.10) is a closed-form or closed-loop soluion. A closed-loop soluion is a soluion where he conrol variable is expressed as a funcion of he sae variable and R ime. In (5.6.10), he sae variable is a () and he funcion of ime is he inegral 1 e r[ ] w () d: Closed-loop soluions sand in conras o open-loop soluions where he conrol variable is a funcion of ime only. This disincion becomes meaningful only in a sochasic world. In a deerminisic world, any closed-loop soluion can be expressed as a funcion of ime only by replacing he sae-variable by he funcion of ime which describes is pah. When we solve he budge consrain saring a some a ( 0 ) wih 0 ; inser c () from (5.6.9) ino his soluion for a () and nally inser his soluion for a () ino (5.6.10), we would obain an expression for he conrol c () as a funcion of ime only. The soluion in (5.6.10) is also very useful for furher illusraing he quesion raised earlier in ch. 3.4.2 on wha is a sae variable?. De ning all variables which in uence he soluion for he conrol variable as sae variable, we clearly see from (5.6.10) ha

126 Chaper 5. Finie and in nie horizon models a () and he enire pah of w () ; i.e. w () for < 1 are sae variables. As we are in a deerminisic world, we can reduce he pah of w () o is iniial value in plus some funcion of ime. Wha his soluion clearly shows is ha a () is no he only sae variable. From solving he maximizaion problem using he Hamilonian as suggesed afer (5.6.6) or from comparing wih he similar seup in he inroducory example in ch. 5.1, i is su cien from a pracical perspecive, however, o ake only a () as explici sae variable ino accoun. The Keynes-Ramsey rule in (5.1.6) was obained using he shadow-price of wealh only - see (5.1.5) - bu no shadow-price for he wage was required in ch. 5.1. 5.6.3 Opimal growh - he cenral planner and capial accumulaion The seup This example sudies he classic cenral planner problem: Firs, here is a social welfare funcion like (5.1.1), expressed slighly more generally as Z 1 max fc()g e [ ] u (C ()) d: The generalizaion consiss of he in nie planning horizon and he general insananeous uiliy funcion (feliciy funcion) u (c ()). We will specify i in he mos common version, u (C) = C1 1 1 ; (5.6.11) where he ineremporal elasiciy of subsiuion is consan and given by 1=: Second, here is a resource consrain ha requires ha ne capial invesmen is given by he di erence beween oupu Y (K; L), depreciaion K and consumpion C; _K () = Y (K () ; L) K () C () : (5.6.12) This consrain is valid for and for all fuure poins in ime. Assuming for simpliciy ha he labour force L is consan, his compleely describes his cenral planner problem. The planner s choice variable is he consumpion level C () ; o be deermined for each poin in ime beween oday and he far fuure 1. The fundamenal rade-o lies in he uiliy increasing e ec of more consumpion visible from (5.6.11) and he ne-invesmen decreasing e ec of more consumpion visible from he resource consrain (5.6.12). As less capial implies less consumpion possibiliies in he fuure, he rade-o can also be described as lying in more consumpion oday vs. more consumpion in he fuure.

5.6. Furher examples 127 The Keynes-Ramsey rule Le us solve his problem by employing he Hamilonian consising of insananeous uiliy plus () muliplied by he relevan par of he consrain, H = u (C ()) + () [Y (K () ; L) K () C ()] : Opimaliy condiions are u 0 (C ()) = () ; (5.6.13) @H _ () = () @K = () () [Y K (K () ; L) ] : Di ereniaing he rs-order condiion (5.6.13) wih respec o ime gives u 00 (C ()) _ C () = _ () : Insering his and (5.6.13) ino he second condiion again gives, afer some rearranging, u 00 (C ()) u 0 (C ()) _ C () = Y K (K () ; L) : This is almos idenical o he opimaliy rule we obained on he individual level in (5.6.4). The only di erence lies in aggregae consumpion C insead of c and Y K (K () ; L) _p() insead of r () : Insead of he real ineres rae on he household level, we here p() have he marginal produciviy of capial minus depreciaion on he aggregae level. If we assumed a logarihmic insananeous uiliy funcion, u 00 (C ()) =u 0 (C ()) = 1=C () and he Keynes-Ramsey rule would be C _ () =C () = Y K (K () ; L), similar o (5.1.6) or (5.6.5). In our case of he more general CES speci caion in (5.6.11), we nd u 00 (C ()) =u 0 (C ()) = =C () such ha he Keynes-Ramsey rule reads _C () C () = Y K (K () ; L) : (5.6.14) This could be called he classic resul on opimal consumpion in general equilibrium. Consumpion grows if marginal produciviy of capial exceeds he sum of he depreciaion rae and he ime preference rae. The higher he ineremporal elasiciy of subsiuion 1=; he sronger consumpion growh reacs o he di erences Y K (K () ; L) : A phase diagram analysis The resource consrain of he economy in (5.6.12) plus he Keynes-Ramsey rule in (5.6.14) represen a wo-dimensional di erenial equaion sysem which, given wo boundary condiions, give a unique soluion for ime pahs C () and K () : These wo equaions can be analyzed in a phase diagram. This is probably he phase diagram augh mos ofen in Economics.

128 Chaper 5. Finie and in nie horizon models C N _C = 0 6 N? K 0-6 N C A N B N _K = 0 -? Figure 5.6.2 Opimal cenral planner consumpion K N Zero-moion lines for capial and labour, respecively, are given by _K () 0, C () Y (K () ; L) K () ; (5.6.15) _C () 0, Y K (K () ; L) + ; (5.6.16) when he inequaliy signs hold as equaliies. Zero moion lines are ploed in he above gure. When consumpion lies above he Y (K () ; L) K () line, (5.6.15) ells us ha capial increases, below his line, i decreases. When he marginal produciviy of capial is larger han + ; i.e. when he capial sock is su cienly small, (5.6.16) ells us ha consumpion increases. These laws of moion are also ploed in gure 5.6.2. This allows us o draw rajecories A; B and C which all saisfy (5.6.12) and (5.6.14). Hence, again, we have a muliude of soluions for a di erenial equaion sysem. As always, boundary condiions allow us o pick he single soluion o his sysem. One boundary condiion is he iniial value K 0 of he capial sock. The capial sock is a sae variable and herefore hisorically given a each poin in ime. I can no jump. The second boundary condiion should x he iniial consumpion level C 0 : Consumpion is a conrol or choice variable and can herefore jump or adjus o pu he economy on he opimal pah. The condiion which provides he second boundary condiion is, formally speaking, he ransversaliy condiion (see ch. 5.4.3). Taking his formal roue, one would have o prove ha saring wih K 0 a he consumpion level ha pus he economy on pah A would violae he TVC or some No-Ponzi game condiion. Similarly, i would have o be shown ha pah C or any pah oher han B violaes he TVC as well. Even hough no ofen admied, his is no ofen done in pracice. (For an excepion o he applicaion of he No-Ponzi game condiion, see ch. 5.6.1 or exercise 5 in ch. 5.) Whenever a saddle pah is found in a phase diagram, i is argued ha he saddle pah is he equilibrium pah and he iniial consumpion level is such ha he economy nds iself on he pah which approaches he seady sae. While his is a pracical approach, i is also formally sais ed as his pah sais es he TVC indeed.

5.6. Furher examples 129 5.6.4 The maching approach o unemploymen Maching funcions are widely used in, for example, Labour economics and Moneary economics. Here we will presen he background for heir use in labour marke modelling. Aggregae unemploymen The unemploymen rae in an economy is governed by wo facors: he speed wih which new employmen is creaed and he speed wih which exising employmen is desroyed. The number of new maches per uni of ime d is given by a maching funcion. I depends on he number of unemployed U, i.e. he number of hose poenially available o ll a vacancy, and he number of vacancies V, m = m (U; V ). The number of lled jobs ha are desroyed per uni of ime is given by he produc of he separaion rae s and employmen, sl. Combining boh componens, he evoluion of he number of unemployed over ime is given by _U = sl m (U; V ) : (5.6.17) De ning employmen as he di erence beween he size of he labour force N and he number of unemployed U, L = N U, we obain _U = s [N U] m (U; V ), _u = s [1 u] m (u; V=N) ; where we divided by he size of he labour force N in he las sep o obain he unemploymen rae u U=N: We also assumed consan reurns o scale in he maching funcion m (:) : De ning labour marke ighness by V=U; he maching funcion can furher be rewrien as m u; V N = m 1; V u = VU UV U m ; 1 u q () u (5.6.18) and we obain _u = s [1 u] q () u which is equaion (1.3) in Pissarides (2000). Clearly, from (5.6.17), one can easily obain he evoluion of employmen, again using he de niion L = N U; _L = m (N L; V ) sl: (5.6.19) Opimal behaviour of large rms Given his maching process, he choice variable of a rm i is no longer employmen L i bu he number of vacancies V i i creaes. There is an obvious similariy o he adjusmen cos seup in ch. 5.5.1. Here, he rm is no longer able o choose labour direcly bu only indirecly hrough vacancies. Wih adjusmen coss, he capial sock is chosen only indirecly hrough invesmen. The rm s objecive is o maximize is presen value, given by he inegral over discouned fuure pro s, max fv i ;K i g Z 1 e r[ ] i () d:

130 Chaper 5. Finie and in nie horizon models Pro s are given by i = Y (K i ; L i ) rk i wl i V i : Each vacancy implies coss measured in unis of he oupu good Y: The rm rens capial K i from he capial marke and pays ineres r and a wage rae w per workers. The employmen consrain is _L i = m (N L; V ) V i sl i : V The consrain now says - in conras o (5.6.19) - ha only a cerain share of all maches goes o he rm under consideraion and ha his share is given by he share of he rm s vacancies in oal vacancies, V i =V. Alernaively, his says ha he probabiliy ha a mach in he economy as a whole lls a vacancy of rm i is given by he number of maches in he economy as a whole divided by he oal number of vacancies. As under consan reurns o scale for m and using he de niion of q () implicily in (5.6.18), m (N L; V ) =V = m (U=V; 1) = q () ; we can wrie he rm s consrain as _L i = q () V i sl i : (5.6.20) Assuming small rms, his rae q () can safely be assumed o be exogenous for he rm s maximizaion problem. The curren-value Hamilonian for his problem reads H = i + i _ Li ; i.e. H = Y (K i ; L i ) rk i wl i V i + i [q () V i sl i ] ; and he rs-order condiions for capial and vacancies are H Ki = Y Ki (K i ; L i ) r = 0; H Vi = + i q () = 0: (5.6.21) The opimaliy condiion for he shadow price of labour is _ i = r i H Li = r i (Y Li (K i ; L i ) w i s) = (r + s) i Y Li (K i ; L i ) + w: The rs condiion for capial is he usual marginal produciviy condiion applied o capial inpu. Wih CRTS producion funcions, his condiion xes he capial o labour raio for each rm. This implies ha he marginal produc of labour is a funcion of he ineres only and herefore idenical for all rms, independen of he labour sock, i.e. he size of he rm. This changes he hird condiion o _ = (r + s) Y L (K; L) + w (5.6.22) which means ha he shadow prices are idenical for all rms. The rs-order condiion for vacancies, wrien as = q () says ha he marginal coss of a vacancy (which in his special case equal average and uni coss) mus be equal o revenue from a vacancy. This expeced revenue is given by he share q () of vacancies ha yield a mach imes he value of a mach. The value of a mach o he rm is given by ; he shadow price of labour. The link beween and he value of an addiional uni of labour (or of he sae variable, more generally speaking) is analyzed in ch. 6.2.

5.6. Furher examples 131 General equilibrium I appears as if he rs-order condiion for vacancies (5.6.21) was independen of he number of vacancies opened by he rm. In fac, given his srucure, he individual rm follows a bang-bang policy. Eiher he opimal number of vacancies is zero or in niy. In general equilibrium, however, a higher number of vacancies increases he rae q () in (5.6.20) wih which exising vacancies ge lled (remember ha = V =U). Hence, his second condiion holds in general equilibrium and we can compue by di ereniaing wih respec o ime _= = _q () =q () : The hird condiion (5.6.22) can hen be wrien as _q () q () = r + s q () [Y L (K; L) w] (5.6.23) which is an equaion depending on he number of vacancies and employmen only. This equaion, ogeher wih (5.6.19), is a wo-dimensional di erenial equaion sysem which deermines V and L; provided here are 2 boundary condiions for V and L and wages and he ineres rae are exogenously given. (In a more complee general equilibrium seup, wages would be deermined e.g. by Nash-bargaining. For his, we need o derive value funcions which is done in ch. 11.2). An example for he maching funcion Assume he maching funcion has CRTS and is of he CD ype, m = U V : As q () = m (N L; V ) =V; i follows ha _q() = _m _V : Given our Cobb-Douglas assumpion, we q() m V can wrie his as _q () q () = U _ U + V V V V = U _ V (1 ) _ U V : Hence, he vacancy equaion (5.6.23) becomes _ U U + (1 ) _ V V = r + s q () [Y L (K; L) w], (1 ) _ V V = r + s (N L) V 1 [Y L (K; L) w] + L s N L V (N L) 1 where he las equaliy used q () = (N L) V 1 and (5.6.17) wih m = U V : Again, his equaion wih (5.6.19) is a wo-dimensional di erenial equaion sysem which deermines V and L;as jus described in he general case.

132 Chaper 5. Finie and in nie horizon models 5.7 The presen value Hamilonian 5.7.1 Problems wihou (or wih implici) discouning The problem and is soluion Le he maximizaion problem be given by subjec o max z() Z T F (y () ; z () ; ) d (5.7.1) _y () = Q (y () ; z () ; ) (5.7.2) y () = y ; y (T ) free (5.7.3) where y () 2 R n ; z () 2 R m and Q = (Q 1 (y () ; z () ; ) ; Q 2 (:) ; ::: ; Q n (:)) T : A feasible pah is a pair (y () ; z ()) which sais es (5.7.2) and (5.7.3). z() is he vecor of conrol variables, y() is he vecor of sae variables. Then de ne he (presen-value) Hamilonian H P as where () is he cosae variable, Necessary condiions for an opimal soluion are and H P = F () + () Q () ; (5.7.4) () = ( 1 () ; 2 () ; ::: ; n ()) (5.7.5) H P z = 0; (5.7.6) _ () = H P y ; (5.7.7) (T ) = 0; (Kamien and Schwarz, p. 126) in addiion o (5.7.2) and (5.7.3). In order o have a maximum, we need second order condiions H zz < 0 o hold (Kamien and Schwarz). Undersanding is srucure The rs m equaions (5.7.6) (z () 2 R m ) solve he conrol variables z() as a funcion of sae and cosae variables, y() and (). Hence z() = z(y(); ()): The nex 2n equaions (5.7.7) and (5.7.2) (y () 2 R n ) consiue a 2n dimensional differenial equaion sysem. In order o solve i, one needs, in addiion o he n iniial condiions given exogenously for he sae variables by (5.7.3), n furher condiions for he cosae variables. These are given by boundary value condiions (T ) = 0: Su ciency of hese condiions resuls from heorem as e.g. in secion 5.4.4.

5.7. The presen value Hamilonian 133 5.7.2 Deriving laws of moion As in he secion on he curren-value Hamilonian, a derivaion of he Hamilonian saring from he Lagrange funcion can be given for he presen value Hamilonian as well. The maximizaion problem Le he objecive funcion be (5.7.1) ha is o be maximized subjec o he consrain (5.7.2) and, in addiion, a saic consrain G (y () ; z () ; ) = 0: (5.7.8) This maximizaion problem can be solved by using a Lagrangian. The Lagrangian reads L = = Z T Z T F (:) + () (Q (:) _y ()) () G (:); ) d F (:) + () Q (:) () G (:) d Z T () _y () d Using he inegraion by pars rule R b _xyd = a he las expression gives R b a x _yd + [xy]b a from (4.3.5), inegraing Z T and he Lagrangian reads () _y () d = Z T _ () y () d + [ () y ()] T L = Z T F (:) + () Q (:) + _ () y () [ () y ()] T : () G (:) d This is now maximized wih respec o y () and z (), boh he conrol and he sae variable. We hen obain condiions ha are necessary for an opimum. F z (:) + () Q z (:) () G z (:) = 0 (5.7.9) F y (:) + () Q y (:) + _ () () G y (:) = 0 (5.7.10) The las rs-order condiion can be rearranged o _ () = () Q y (:) F y (:) + () G y (:) (5.7.11) These necessary condiions will be he ones used regularly in maximizaion problems.

134 Chaper 5. Finie and in nie horizon models The shorcu As i is cumbersome o sar from a Lagrangian for each dynamic maximizaion problem, one can de ne he Hamilonian as a shorcu as Opimaliy condiions are hen H P = F (:) + () Q (:) () G (:) : (5.7.12) H P z = 0; (5.7.13) _ () = H P y : (5.7.14) which are he same as (5.7.9) and (5.7.11) above and (5.7.6) and (5.7.7) in he las secion. 5.7.3 The link beween CV and PV If we solve he problem (5.2.1) and (5.2.2) via he presen value Hamilonian, we would sar from H P () = e [ ] G (:) + () Q (:) ; (5.7.15) rs-order condiions would be @H P @z (T ) = 0; (5.7.16) = e [ ] G z (:) + Q z (:) = 0; (5.7.17) _ () = and we would be done wih solving his problem. Simpli caion @HP @y = e [ ] G y (:) Q y (:) (5.7.18) We can simplify he presenaion of rs-order condiions, however, by rewriing (5.7.17) as G z () + () Q z () = 0 (5.7.19) where we used he same de niion as in (5.2.4), () e [ ] () : (5.7.20) Noe ha he argumen of he cosae variables is always ime (and no ime ). When we use his de niion in (5.7.18), his rs-order condiion reads Replacing he lef-hand side by e [ ] _ () = G y (:) Q y (:) : e [ ] _ () = _ () e [ ] () = _ () () :

5.8. Furher reading and exercises 135 which follows from compuing he ime derivaive of he de niion (5.7.20), we ge _ = G y ()Q y : (5.7.21) Insering he de niion (5.7.20) ino he Hamilonian (5.7.15) gives e [ ] H P () = G (:) + Q (:) <> H c () = G (:) + Q (:) which de nes he link beween he presen value and he curren-value Hamilonian as H c () = e [ ] H P () Summary Hence, insead of rs-order condiions (5.7.17) and (5.7.18), we ge (5.7.19) and (5.7.21). As jus shown hese rs-order condiions are equivalen. 5.8 Furher reading and exercises The classic reference for he opimal consumpion behaviour of an individual household analyzed in ch. 5.6.1 is Ramsey (1928). The paper credis par of he inuiive explanaion o Keynes - which led o he name Keynes-Ramsey rule. See Arrow and Kurz (1969) for a deailed analysis. There are many exs ha rea Hamilonians as a maximizaion device. Some examples include Dixi (1990, p. 148), Inriligaor (1971), Kamien and Schwarz (1991), Leonard and Long (1992) or, in German, Feichinger and Harl (1986). Inriligaor provides a nice discussion of he disincion beween closed- and open-loop conrols in his ch. 11.3. The corresponding game-heory de niions for closed-loop and open-loop sraegies (which are in perfec analogy) are in Fudenberg and Tirole (1991). On su cien condiions, see Kamien and Schwarz (1991, ch. 3 and ch. 15). The lieraure on ransversaliy condiions includes Mangasarian (1966), Arrow (1968), Arrow and Kurz (1970), Araujo and Scheinkman (1983), Léonard and Long (1992, p. 288-289), Chiang (1992, p. 217, p. 252) and Kamihigashi (2001). Counerexamples ha he TVC is no necessary are provided by Michel (1982) and Shell (1969). See Buier and Sieber (2007) for a recen very useful discussion and an applicaion. The issue of boundedness was discovered a relaively long ime ago and received renewed aenion in he 1990s when he new growh heory was being developed. A more general reamen of his problem was underaken by von Weizsäcker (1965) who compares uiliy levels in unbounded circumsances by using overaking crieria. Expressions for explici soluions for consumpion have been known for a while. The case of a logarihmic uiliy funcion, i.e. where = 1 and where he fracion in fron of he curly brackes in (5.6.10) simpli es o, was obained by Blanchard (1985). Vissing-Jørgensen (2002) provides micro-evidence on he level of he ineremporal elasiciy of subsiuion. Maching models of unemploymen go back o Pissarides (1985). For a exbook reamen, see Pissarides (2000)

136 Chaper 5. Finie and in nie horizon models Exercises chaper 5 Applied Ineremporal Opimizaion Hamilonians 1. Opimal consumpion over an in nie horizon Solve he maximizaion problem subjec o by max c() Z 1 e [ ] ln c () d; p () c () + _ A () = r () A () + w () : (a) using he presen-value Hamilonian. Compare he resul o (5.6.5). (b) Use u (c ()) insead of ln c (). 2. Adjusmen coss Solve he adjusmen cos example for (I) = I: Wha do opimaliy condiions mean? Wha is he opimal end-poin value for K and I? 3. Consumpion over he life cycle The uiliy of an individual, born a s and living for T periods is given a ime by u (s; ) = Z s+t e [ The individual s budge consrain is given by where Z s+t D R () = exp ] ln (c (s; )) d: D R () c (s; ) d = h (s; ) + a (s; ) Z Z s+t r (u) du ; h (s; ) = D r () w (s; ) d: This deplorable individual would like o know how he can lead a happy life bu, unforunaely, has no sudied opimal conrol heory!

5.8. Furher reading and exercises 137 (a) Wha would you recommend him? Use a Hamilonian approach and disinguish beween changes of consumpion and he iniial level. Which informaion do you need o deermine he iniial consumpion level? Wha informaion would you expec his individual o provide you wih? In oher words, which of he above maximizaion problems makes sense? Why no he oher one? (b) Assume all prices are consan. Draw he pah of consumpion in a (,c()) diagram. Draw he pah of asse holdings a() in he same diagram, by guessing how you would expec i o look. (You could compue i if you wan) 4. Opimal consumpion (a) Derive he opimal allocaion of expendiure and consumpion over ime for by employing he Hamilonian. u (c ()) = c ()1 1 ; > 0 1 (b) Show ha his funcion includes he logarihmic uiliy funcion for = 1 (apply L Hôspial s rule). (c) Does he uiliy funcion u(c()) make sense for > 1? Why (no)? (d) Compue he ineremporal elasiciy of subsiuion for his uiliy funcion following he discussion afer (5.6.4). Wha is he ineremporal elasiciy of subsiuion for he logarihmic uiliy funcion u (c) = ln c? 5. A cenral planner You are responsible for he fuure well-being of 360 million Europeans and cenrally plan he EU by assigning a consumpion pah o each inhabian. Your problem consiss in maximizing a social welfare funcion of he form U i () = Z 1 subjec o he EU resource consrain e [ ] C 1 1 (1 ) 1 d _K = BK C (5.8.1) (a) Wha are opimaliy condiions? Wha is he consumpion growh rae? (b) Under which condiions is he problem well de ned (boundedness condiion)? Inser he consumpion growh rae and show under which condiions he uiliy funcion is bounded. (c) Wha is he growh rae of he capial sock? Compue he iniial consumpion level, by using he no-ponzi-game condiion (5.4.4).

138 Chaper 5. Finie and in nie horizon models (d) Under which condiions could you resign from your job wihou making anyone less happy han before? 6. Opimal consumpion levels (a) Derive a rule for he opimal consumpion level for a ime-varying ineres rae r () : Show ha (5.6.10) can be generalized o c () = 1 R 1 e R fw () + a ()g ; (1 )r(s) ds d where W () is human wealh. (b) Wha does his imply for he wealh level a ()? 7. Invesmen and he ineres rae (a) Use he resul of 5 c) and check under which condiions invesmen is a decreasing funcion of he ineres rae. (b) Perform he same analysis for a budge consrain _a = ra + w (5.8.1). 8. The Ramsey growh model (Cf. Blanchard and Fischer, ch. 2) Le he resource consrain now be given by _K = Y (K; L) C K: c insead of Draw a phase diagram. Wha is he long-run equilibrium? Perform a sabiliy analysis graphically and analyically (locally). Is he uiliy funcion bounded? 9. An exam quesion Consider a decenralized economy in coninuous ime. Facors of producion are capial and labour. The iniial capial sock is K 0, labour endowmen is L: Capial is he only asse, i.e. households can save only by buying capial. Capial can be accumulaed also a he aggregae level, _K () = I () K (). Households have a corresponding budge consrain and a sandard ineremporal uiliy funcion wih ime preference rae and in nie planning horizon. Firms produce under perfec compeiion. Describe such an economy in a formal way and derive is reduced form. Do his sep by sep: (a) Choose a ypical producion funcion. (b) Derive facor demand funcions by rms.

5.8. Furher reading and exercises 139 (c) Le he budge consrains of households be given by _a () = r () a () + w L () c () : Specify he maximizaion problem of a household and solve i. (d) Aggregae he opimal individual decision over all households and describe he evoluion of aggregae consumpion. (e) Formulae he goods marke equilibrium. (f) Show ha he budge consrain of he household is consisen wih he aggregae goods marke equilibrium. (g) Derive he reduced form _K () = Y () C () K () ; _C () C () = @Y () @K() by going hrough hese seps and explain he economics behind his reduced form.

140 Chaper 5. Finie and in nie horizon models

Chaper 6 In nie horizon again This chaper reanalyzes maximizaion problems in coninuous ime ha are known from he chaper on Hamilonians. I shows how o solve hem wih dynamic programming mehods. The sole objecive of his chaper is o presen he dynamic programming mehod in a well-known deerminisic seup such ha is use in a sochasic world in subsequen chapers becomes more accessible. 6.1 Ineremporal uiliy maximizaion We consider a maximizaion problem ha is very similar o he inroducory example for he Hamilonian in secion 5.1 or he in nie horizon case in secion 5.3. Compared o 5.1, he uiliy funcion here is more general and he planning horizon is in niy. None of his is imporan, however, for undersanding he di erences in he approach beween he Hamilonian and dynamic programming. 6.1.1 The seup Uiliy of he individual is given by U () = Z 1 e [ ] u (c ()) d: (6.1.1) Her budge consrained equaes wealh accumulaion wih savings, _a = ra + w pc: (6.1.2) The individual can choose he pah of consumpion fc ()g beween now and in niy and akes prices and facor rewards as given. 6.1.2 Solving by dynamic programming As in models of discree ime, he value V (a ()) of he opimal program is de ned by he maximum overall uiliy level ha can be reached by choosing he consumpion pah 141

142 Chaper 6. In nie horizon again opimally given he consrain. V (a ()) max fc()g U () subjec o (6.1.2). When households behave opimally beween oday and in niy by choosing he opimal consumpion pah fc ()g ; heir overall uiliy U () is given by V (a ()) : A prelude on he Bellman equaion The derivaion of he Bellman equaion under coninuous ime is no as obvious as under discree ime. Even hough we do have a oday,, we do no have a clear omorrow (like a + 1 in discree ime). We herefore need o consruc a omorrow by adding a small ime inerval o : Tomorrow would hen be + : Noe ha his derivaion is heurisic and more rigorous approaches exis. See e.g. Sennewald (2007) for furher references o he lieraure. Following Bellman s idea, we rewrie he objecive funcion as he sum of wo subperiods, U () = Z + e [ ] u (c ()) d + Z 1 e [ + ] u (c ()) d; where is a small ime inerval. As we did in discree ime, we exploi here he addiive separabiliy of he objecive funcion. This is he rs sep of simplifying he maximizaion problem, as discussed for he discree-ime case afer (3.3.4). When we approximae he rs inegral (hink of he area below he funcion u (c ()) ploed over ime ) by 1 u (c ()) and he discouning beween and + by and we assume ha as of 1+ + we behave opimally, we can rewrie he value funcion V (a ()) = max fc()g U () as 1 V (a ()) = max u (c ()) + V (a ( + )) : c() 1 + The assumpion of behaving opimally as of + can be seen formally in he fac ha V (a ( + )) now replaces U ( + ) : This is he second sep in he procedure o simplify he maximizaion problem. Afer hese wo seps, we are lef wih only one choice variable c () insead of he enire pah fc ()g :When we rs muliply his expression by 1 +, hen divide n by and nally move V (a()) o he righ hand side, we o ge V (a ()) = max c() u (c ()) [1 + ] + V (a(+)) V (a()) : Taking he limi lim!0 gives he Bellman equaion, V (a ()) = max c() u (c ()) + dv (a ()) : (6.1.3) d This equaion again shows Bellman s rick: A maximizaion problem, consising of he choice of a pah of a choice variable, was broken down o a maximizaion problem where only he level of he choice variable in has o be chosen. The srucure of his equaion can also be undersood from a more inuiive perspecive: The erm V (a ()) can bes be undersood when comparing i o rv; capial income a each insan of an individual who owns a capial sock of value v and he ineres rae is

6.1. Ineremporal uiliy maximizaion 143 r: A household ha behaves opimally owns he value V (a ()) from opimal behaviour and receives a uiliy sream of V (a ()) : This uiliy income a each insan is given by insananeous uiliy from consumpion plus he change in he value of opimal behaviour. Noe ha his srucure is idenical o he capial-marke no-arbirage condiion (4.4.7), r () v () = () + _v () - he capial income sream from holding wealh v () on a bank accoun is idenical o dividend paymens () plus he change _v () in he marke price when holding he same level of wealh in socks. While he derivaion jus shown is he sandard one, we will now presen an alernaive approach which illusraes he economic conen of he Bellman equaion and which is more sraighforward. Given he objecive funcion in (6.1.1), we can ask how overall uiliy U () changes over ime. To his end, we compue he derivaive du () =d and nd (using he Leibniz rule 4.3.3 from ch. 4.3.1) _U () = e [ ] u (c ()) + Z 1 d d e [ ] u (c ()) d = u (c ()) + U () : Overall uiliy U () reduces as ime goes by by he amoun u (c ()) a each insan (as he inegral becomes smaller when curren consumpion in is los and we sar an insan afer ) and increases by U () (as we gain because fuure uiliies come closer o oday when oday moves ino he fuure). Rearranging his equaion gives U () = u (c ())+ U _ () : When overall uiliy is replaced by he value funcion, we obain V (a ()) = u (c ())+ V _ (a ()) which corresponds in is srucure o he Bellman equaion (6.1.3). DP1: Bellman equaion and rs-order condiions We will now follow he hree-sep procedure o maximizaion when using he dynamic programming approach as we go o know i in discree ime seups is secion 3.3. When we compue he Bellman equaion for our case, we obain for he derivaive in (6.1.3) dv (a ()) =d = V 0 (a ()) _a which gives wih he budge consrain (6.1.2) The rs-order condiion reads V (a ()) = max fu (c ()) + V 0 (a ()) [ra + w pc]g : (6.1.4) c() u 0 (c ()) = pv 0 (a ()) (6.1.5) and makes consumpion a funcion of he sae variable, c () = c (a ()) : This rs-order condiion also capures pros and cons of more consumpion oday. The advanage is higher insananeous uiliy, he disadvanage is he reducion in wealh. In conras o discree ime models, here is no omorrow and he ineres rae and he ime preference rae presen, for example in (3.4.5), are absen here. The disadvanage is capured by he change in overall uiliy due o changes in a () ; i.e. he shadow price V 0 (a ()) ; imes he price of one uni of consumpion in unis of he capial good. Loosely speaking, when consumpion goes up oday by one uni, wealh goes down by p unis. Higher consumpion increases uiliy by u 0 (c ()) ; p unis less wealh reduces overall uiliy by pv 0 (a ()) :

144 Chaper 6. In nie horizon again DP2: Evoluion of he cosae variable In coninuous ime, he second sep of he dynamic programming approach o maximizaion can be subdivided ino wo subseps. (i) In he rs, we look a he maximized Bellman equaion, V (a) = u (c (a)) + V 0 (a) [ra + w pc (a)] : The rs-order condiion (6.1.5) ogeher wih he maximized Bellman equaion deermines he evoluion of he conrol variable c () and V (a ()). This sysem can be used as a basis for numerical soluion. Again, however, he maximized Bellman equaion does no provide very much insigh from an analyical perspecive. Compuing he derivaive wih respec o a () however (and using he envelope heorem) gives an expression for he shadow price of wealh ha will be more useful, V 0 (a) = V 00 (a) [ra + w pc] + V 0 (a) r, (6.1.6) ( r) V 0 (a) = V 00 (a) [ra + w pc] : (ii) In he second sep, we compue he derivaive of he cosae variable V 0 (a) wih respec o ime, giving dv 0 (a) d = V 00 (a) _a = ( r) V 0 (a) ; where he las equaliy used (6.1.6). Dividing by V 0 (a) and using he usual noaion _V 0 (a) dv 0 (a) =d, his can be wrien as _V 0 (a) V 0 (a) = r: (6.1.7) This equaion describes he evoluion of he cosae variable V 0 (a), he shadow price of wealh. DP3: Insering rs-order condiions The derivaive of he rs-order condiion wih respec o ime is given by (apply rs logs) u 00 (c) u 0 (c) _c = _p p + V _ 0 (a) V 0 (a) : Insering (6.1.7) gives u 00 (c) u 0 (c) _c = _p p + r () u00 (c) u 0 (c) _c = r This is he well-known Keynes Ramsey rule. _p p :

6.2. Comparing dynamic programming o Hamilonians 145 6.2 Comparing dynamic programming o Hamilonians When we compare opimaliy condiions under dynamic programming wih hose obained when employing he Hamilonian, we nd ha hey are idenical. Observing ha our cosae evolves according o _V 0 (a) V 0 (a) = r; as jus shown in (6.1.7), we obain he same equaion as we had in he Hamilonian approach for he evoluion of he cosae variable, see e.g. (5.1.5) or (5.6.13). Comparing rs-order condiions (5.1.4) or (5.2.9) wih (6.1.5), we see ha hey would be idenical if we had chosen exacly he same maximizaion problem. This is no surprising given our applied view of opimizaion: If here is one opimal pah ha maximizes some objecive funcion, his one pah should always be opimal, independenly of which maximizaion procedure is chosen. A comparison of opimaliy condiions is also useful for an alernaive purpose, however. As (6.1.7) and e.g. (5.1.5) or (5.6.13) are idenical, we can conclude ha he derivaive V 0 (a ()) of he value funcion wih respec o he sae variable, in our case a, is idenical o he cosae variable in he curren-value Hamilonian approach, V 0 (a) = : This is where he inerpreaion for he cosae variable in he Hamilonian approach in ch. 5.2.3 came from. There, we said ha he cosae variable sands for he increase in he value of he opimal program when an addiional uni of he sae variable becomes available; his is exacly wha V 0 (a) sands for. Hence, he inerpreaion of a cosae variable is similar o he inerpreaion of he Lagrange muliplier in saic maximizaion problems. 6.3 Dynamic programming wih wo sae variables As a nal example for maximizaion problems in coninuous ime ha can be solved wih dynamic programming, we look a a maximizaion problem wih wo sae variables. Think e.g. of an agen who can save by puing savings on a bank accoun or by accumulaing human capial. Or hink of a cenral planner who can increase oal facor produciviy or he capial sock. We look here a he rs case. Our agen has a sandard objecive funcion, U () = Z 1 e [ I is maximized subjec o wo consrains. variables wealh a and human capial h; ] u (c ()) d: They describe he evoluion of he sae _a = f (a; h; c) ; _ h = g (a; h; c) : (6.3.1)

146 Chaper 6. In nie horizon again We do no give explici expressions for he funcions f (:) or g (:) bu one can hink of a sandard resource consrain for f (:) as in (6.1.2) and a funcional form for g (:) ha capures a rade-o beween consumpion and human capial accumulaion: Human capial accumulaion is faser when a and h are large bu decreases in c: To be precise, we assume ha boh f (:) and g (:) increase in a and h bu decrease in c: DP1: Bellman equaion and rs-order condiions In his case, he Bellman equaion reads dv (a; h) V (a; h) = max u (c) + = max fu (c) + V a f (:) + V h g (:)g : d There are simply wo parial derivaives of he value funcion afer he u (c) erm imes he da and dh erm, respecively, insead of one as in (6.1.4), where here is only one sae variable. Given ha here is sill only one conrol variable, consumpion, here is only one rs-order condiion. This is clearly speci c o his example. One could hink of a ime consrain for human capial accumulaion (a rade-o beween leisure and learning - hink of he Lucas (1988) model) where agens choose he share of heir ime used for accumulaing human capial. In his case, here would be wo rs-order condiions. Here, however, we have jus he one for consumpion, given by u 0 (c) + V a @f (:) @c + V h @g (:) @c = 0 (6.3.2) When we compare his condiion wih he one-sae-variable case in (6.1.5), we see ha he rs wo erms u 0 @f(:) (c) + V a correspond exacly o (6.1.5): If we had speci ed @c f (:) as in he budge consrain (6.1.2), he rs wo erms would be idenical o (6.1.5). @g(:) The hird erm V h is new and sems from he second sae variable: Consumpion now @c no only a ecs he accumulaion of wealh bu also he accumulaion of human capial. More consumpion gives higher insananeous uiliy bu, a he same ime, decreases fuure wealh and - now new - he fuure human capial sock as well. DP2: Evoluion of he cosae variables As always, we need o undersand he evoluion of he cosae variable(s). In a seup wih wo sae variables, here are wo cosae variables, or, economically speaking, a shadow price of wealh a and a shadow price of human capial h: This is obained by parially di ereniaing he maximized Bellman equaion, rs wih respec o a; hen wih respec o h: Doing his, we ge (employing he envelope heorem righ away) V a = V aa f (:) + V a @f (:) @a + V hag (:) + V h @g (:) @a ; (6.3.3) V h = V ah f (:) + V a @f (:) @h + V hhg (:) + V h @g (:) @h : (6.3.4)

6.3. Dynamic programming wih wo sae variables 147 As in ch. 6.1, he second sep of DP2 consiss of compuing he ime derivaives of he cosae variables and in reinsering (6.3.3) and (6.3.4). We rs compue ime derivaives, insering (6.3.1) ino he las sep, dv a (a; h) d dv h (a; h) d = V aa _a + V ah _ h = Vaa f (:) + V ah g (:) ; = V ha _a + V hh _ h = Vha f (:) + V hh g (:) : Then insering (6.3.3) and (6.3.4), we nd dv a (a; h) d dv h (a; h) d @f (:) = V a V a @a @f (:) = V h V a @h V h @g (:) @a ; V h @g (:) @h : The nice feaure of his las sep is he fac ha he cross derivaives V ah and V ha disappear, i.e. can be subsiued ou again, using he fac ha f xy (x; y) = f yx (x; y) for any wice di ereniable funcion f (x; y). Wriing hese equaions as _V a = V a _V h = V h @f (:) @a V a @f (:) V h @h V h V a @g (:) @a ; @g (:) @h ; allows us o give an inerpreaion ha links hem o he sandard one-sae case in (6.1.7). The cosae variable V a evolves as above, only ha insead of he ineres rae, we nd here @f(:) + V h @g(:) @a V a : The rs derivaive @f (:) =@a capures he e ec a change in a has @a on he rs consrain; in fac, if f (:) represened a budge consrain as in (6.1.2), his would be idenical o he ineres rae. The second erm @g (:) =@a capures he e ec of a change in a on he second consrain; by how much would h increase if here was more a? This e ec is muliplied by V h =V a ; he relaive shadow price of h: An analogous inerpreaion is possible for V _ h =V h : DP3: Insering rs-order condiions The nal sep consiss of compuing he derivaive of he rs-order condiion (6.3.2) wih respec o ime and replacing he ime derivaives _ V a and _ V h by he expressions from he previous sep DP2. The principle of how o obain a soluion herefore remains unchanged when having wo sae variables insead of one. Unforunaely, however, i is generally no possible o eliminae he shadow prices from he resuling equaion which describes he evoluion of he conrol variable. Some economically suiable assumpions concerning f (:) or g (:) could, however, help.

148 Chaper 6. In nie horizon again 6.4 Nominal and real ineres raes and in aion We presen here a simple model which allows us o undersand he di erence beween he nominal and real ineres rae and he deerminans of in aion. This chaper is inended (i) o provide anoher example where dynamic programming can be used and (ii) o show again how o go from a general decenralized descripion of an economy o is reduced form and hereby obain economic insighs. 6.4.1 Firms, he cenral bank and he governmen Firms Firms use a sandard neoclassical echnology Y = Y (K; L) : Producing under perfec compeiion implies facor demand funcion of w K = p @Y @K ; wl = p @Y @L : (6.4.1) The cenral bank and he governmen Sudying he behaviour of cenral banks will ll a lo of books. We presen he behaviour of he cenral bank in a very simple way - so simple ha wha he cenral bank does here (buy bonds direcly from he governmen) is acually illegal in mos OECD counries. Despie his simple presenaion, he general resul we obain laer would hold in more realisic seups as well. The cenral bank issues money M () in exchange for governmen bonds B () : I receives ineres paymens ib from he governmen on he bonds. The balance of he cenral bank is herefore ib + M _ = B. _ This equaion says ha bond holdings by he cenral bank increase by B; _ eiher when he cenral bank issues money M _ or receives ineres paymens ib on bonds i holds. The governmen s budge consrain reads G + ib = T + B: _ General governmen expendiure G plus ineres paymens ib on governmen deb B is nanced by ax income T and de ci B: _ We assume ha only he cenral bank holds governmen bonds and no privae households. Combining he governmen wih he cenral bank budge herefore yields _M = G T: This equaion says ha an increase in moneary supply is eiher used o nance governmen expendiure minus ax income or, if G = 0 for simpliciy, any moneary increase is given o households in he form of negaive axes, T = _M:

6.4. Nominal and real ineres raes and in aion 149 6.4.2 Households Household preferences are described by a money-in-uiliy funcion, ] ln c () + ln m () d: p () Z 1 e [ In addiion o uiliy from consumpion c (), uiliy is derived from holding a cerain sock m () of money, given a price level p () : Given ha wealh of households consiss of capial goods plus money, a () = k () + m () ; and ha holding money pays no ineres, he household s budge consrain can be shown o read (see exercises) _a = i [a m] + w T=L pc: Tax paymens T =L per represenaive household are lump-sum. If axes are negaive, T=L represens ransfers from he governmen o households. The ineres rae i is de ned according o i wk + _v : v When households choose consumpion and he amoun of money held opimally, consumpion growh follows (see exercises) _c c = i _p p : Money demand is given by (see exercises) 6.4.3 Equilibrium The reduced form m = pc i : Equilibrium requires equaliy of supply and demand on he goods marke. This is obained if oal supply Y equals demand C + I: Leing capial accumulaion follow _K = I K; we ge _K = Y (K; L) C K: (6.4.2) This equaion deermines K: As capial and consumpion goods are raded on he same marke, his equaion implies v = p and he nominal ineres rae becomes wih (6.4.1) i = wk p + _p p = @Y @K + _p p : (6.4.3) The nominal ineres rae is given by marginal produciviy of capial w K =p = @Y=@K (he real ineres rae ) plus in aion _p=p.

150 Chaper 6. In nie horizon again Aggregaing over households yields aggregae consumpion growh of C=C _ = i _p=p : Insering (6.4.3) yields _C C = @Y : (6.4.4) @K Aggregae money demand is given by M = pc i : (6.4.5) Given an exogenous money supply rule and appropriae boundary condiions, hese four equaions deermine he pahs of K; C; i and he price level p: One sandard propery of models wih exible prices is he dichoomy beween real variables and nominal variables. The evoluion of consumpion and capial - he real side of he economy - is compleely independen of moneary in uences: Equaion (6.4.2) and (6.4.4) deermine he pahs of K and C jus as in he sandard opimal growh model wihou money - see (5.6.12) and (5.6.14) in ch. 5.6.3. Hence, when hinking abou equilibrium in his economy, we can hink abou he real side on he one hand - independenly of moneary issues - and abou he nominal side on he oher hand. Moneary variables have no real e ec bu real variables have an e ec on moneary variables like e.g. in aion. Needless o say ha he real world does no have perfecly exible prices such ha one should expec moneary variables o have an impac on he real economy. This model is herefore a saring poin o well undersanding srucures and no a fully developed model for analysing moneary quesions in a very realisic way. Price rigidiy would have o be included before doing his. A seady sae Assume he echnology Y (K; L) is such ha in he long-run K is consan. As a consequence, aggregae consumpion C is consan as well. Hence, wih respec o real variables (including, in addiion o K and C; he real ineres rae and oupu), we are in a seady sae as in ch. 5.6.3. Depending on exogenous money supply, equaions (6.4.3) and (6.4.5) deermine he price level and he nominal ineres rae. Subsiuing he nominal ineres rae ou, we obain _p p = pc @Y M @K : This is a di erenial equaion which is ploed in he nex gure. As his gure shows, provided ha M is consan, here is a price level p which implies ha here is zero in aion.

6.4. Nominal and real ineres raes and in aion 151 _p p N pc M @Y @K 0 p!! p N @Y @K Figure 6.4.1 The price level p in a moneary economy Now assume here is money growh, _M > 0: The gure hen shows ha he price level p increases as he slope of he line becomes aer. Money growh in an economy wih consan GDP implies in aion. By looking a (6.4.3) and (6.4.5) again and by focusing on equilibria wih consan in aion raes, we know from (6.4.3) ha a consan in aion rae implies a consan nominal ineres rae. Hence, by di ereniaing he equilibrium (6.4.5) on he money marke, we ge _p p = _ M M : (6.4.6) In an economy wih consan GDP and increasing money supply, he in aion rae is idenical o he growh rae of money supply. A growh equilibrium Now assume here is (exogenous) echnological progress a a rae g such ha in he long run Y _ =Y = C=C _ = K=K _ = g: Then by again assuming a consan in aion rae (implying a consan nominal ineres rae) and going hrough he same seps ha led o (6.4.6), we nd by di ereniaing (6.4.5) _p p = _ M M In aion is given by he di erence beween he growh rae of money supply and consumpion growh. _C C :

152 Chaper 6. In nie horizon again Exogenous nominal ineres raes The curren hinking abou cenral bank behaviour di ers from he view ha he cenral bank chooses money supply M as assumed so far. The cenral bank raher ses nominal ineres raes and money supply adjuss (where one should keep in mind ha money supply is more han jus cash used for exchange as modelled here). Can nominal ineres rae seing be analyzed in his seup? Equilibrium is described by equaions (6.4.2) o (6.4.5). They were undersood o deermine he pahs of K; C; i and he price level p; given a money supply choice M by he cenral bank. If one believes ha nominal ineres rae seing is more realisic, hese four equaions would simply deermine he pahs of K; C; M and he price level p; given a nominal ineres rae choice i by he cenral bank. Hence, simply by making an exogenous variable, M; endogenous and making a previously endogenous variable, i; exogenous, he same model can be used o undersand he e ecs of higher and lower nominal ineres raes on he economy. Due o perfec price exibiliy, real quaniies remain una eced by he nominal ineres rae. Consumpion, invesmen, GDP, he real ineres rae, real wages are all deermined, as before, by (6.4.2) and (6.4.4) - he dichoomy beween real and nominal quaniies coninues given price exibiliy. In (6.4.3), a change in he nominal ineres rae a ecs in aion: high (nominal) ineres raes imply high in aion, low nominal ineres raes imply low in aion. From he money marke equilibrium in (6.4.5) one can hen conclude wha his implies for money supply, again boh for a growing or a saionary economy. Much more needs o be said abou hese issues before policy implicaions can be discussed. Any analysis in a general equilibrium framework would however parly be driven by he relaionships presened here. 6.5 Furher reading and exercises An alernaive way, which is no based on dynamic programming, o reach he same conclusion abou he inerpreaion of he cosae variable for Hamilonian maximizaion as here in ch. 6.2 is provided by Inriligaor (1971, p. 352). An excellen overview and inroducion o Moneary economics is provided by Walsh (2003).

6.5. Furher reading and exercises 153 Exercises chaper 6 Applied Ineremporal Opimizaion Dynamic Programming in Coninuous Time 1. The envelope heorem once again Compue he derivaive (6.1.6) of he maximized Bellman equaion wihou using he envelope heorem. 2. A rm wih adjusmen coss Consider again, as in ch. 5.5.1, a rm wih adjusmen cos. The rm s objecive is Z 1 max fi();l()g e r[ ] () d: In conras o ch. 5.5.1, he rm now has an in nie planning horizon and employs wo facors of producion, capial and labour. Insananeous pro s are = pf (K; L) wl I I ; where invesmen I also comprises adjusmen coss for > 0: Capial, owned by he rm, accumulaes according o K _ = I K: All parameers ; ; are consan. (a) Solve his maximizaion problem by using he dynamic programming approach. You may choose appropriae (numerical or oher) values for parameers where his simpli es he soluion (and does no desroy he spiri of his exercise). (b) Show ha in he long-run wih adjusmen coss and a each poin in ime under he absence of adjusmen coss, capial is paid is value marginal produc. Why is labour being paid is value marginal produc a each poin in ime? 3. Money in he uiliy funcion Consider an individual wih he following uiliy funcion U () = Z 1 e [ ] ln c() + ln m() d: p() As always, is he ime preference rae and c () is consumpion. This uiliy funcion also capures demand for money by including a real moneary sock of

154 Chaper 6. In nie horizon again m () =p () in he uiliy funcion where m () is he amoun of cash and p () is he price level of he economy. Le he budge consrain of he individual be _a = i [a m] + w T=L pc: where a is he oal wealh consising of shares in rms plus money, a = k + m and i is he nominal ineres rae. (a) Derive he budge consrain by assuming ineres paymens of i on shares in rms and zero ineres raes on money. (b) Derive he opimal money demand. 4. Nominal and real ineres raes in general equilibrium Pu households from exercise 3 in general equilibrium wih capial accumulaion and a cenral bank which chooses money supply M: Compue he real and he nominal ineres rae in a long-run equilibrium.

6.6. Looking back 155 6.6 Looking back This is he end of par I and II. This is ofen also he end of a course. This is a good momen o look back a wha has been accomplished. Afer 14 or 15 lecures and he same number of exercise classes, he amoun of maerial covered is fairly impressive. In erms of maximizaion ools, his rs par has covered Solving by subsiuion Lagrange mehods in discree and coninuous ime Dynamic programming in discree ime and coninuous ime Hamilonian Wih respec o model building componens, we have learn how o build budge consrains how o srucure he presenaion of a model how o derive reduced forms From an economic perspecive, he rs par presened he wo-period OLG model he opimal saving cenral planner model in discree and coninuous ime he maching approach o unemploymen he decenralized opimal growh model and an opimal growh model wih money Mos imporanly, however, he ools presened here allow sudens o become independen. A very large par of he Economics lieraure (acknowledging ha game heoreic approaches have no been covered here a all) is now open and accessible and he basis for undersanding a paper in deail (and no jus he overall argumen) and for presening heir own argumens in a scieni c language are laid ou. Clearly, models wih uncerainy presen addiional challenges. They will be presened and overcome in par III and par IV.

156 Chaper 6. In nie horizon again

Par III Sochasic models in discree ime 157

In par III, he world becomes sochasic. Pars I and II provided many opimizaion mehods for deerminisic seups, boh in discree and coninuous ime. All economic quesions ha were analyzed were viewed as su cienly deerminisic. If here was any uncerainy in he seup of he problem, we simply ignored i or argued ha i is of no imporance for undersanding he basic properies and relaionships of he economic quesion. This is a good approach o many economic quesions. Generally speaking, however, real life has few cerain componens. Deah is cerain, bu when? Taxes are cerain, bu how high are hey? We know ha we all exis - bu don ask philosophers. Par III (and par IV laer) will ake uncerainy in life seriously and incorporae i explicily in he analysis of economic problems. We follow he same disincion as in par I and II - we rs analyse he e ecs of uncerainy on economic behaviour in discree ime seups in par III and hen move o coninuous ime seups in par IV. Chaper 7 and 8 are an exended version of chaper 2. As we are in a sochasic world, however, chaper 7 will rs spend some ime reviewing some basics of random variables, heir momens and disribuions. Chaper 7 also looks a di erence equaions. As hey are now sochasic, hey allow us o undersand how disribuions change over ime and how a disribuion converges - in he example we look a - o a limiing disribuion. The limiing disribuion is he sochasic equivalen o a x poin or seady sae in deerminisic seups. Chaper 8 looks a maximizaion problems in his sochasic framework and focuses on he simples case of wo-period models. A general equilibrium analysis wih an overlapping generaions seup will allow us o look a he new aspecs inroduced by uncerainy for an ineremporal consumpion and saving problem. We will also see how one can easily undersand dynamic behaviour of various variables and derive properies of longrun disribuions in general equilibrium by graphical analysis. One can for example easily obain he range of he long-run disribuion for capial, oupu and consumpion. This increases inuiive undersanding of he processes a hand remendously and helps a lo as a guide o numerical analysis. Furher examples include borrowing and lending beween risk-averse and risk-neural households, he pricing of asses in a sochasic world and a rs look a naural volailiy, a view of business cycles which sresses he link beween joinly endogenously deermined shor-run ucuaions and long-run growh. Chaper 9 is hen similar o chaper 3 and looks a muli-period, i.e. in nie horizon, problems. As in each chaper, we sar wih he classic ineremporal uiliy maximizaion problem. We hen move on o various imporan applicaions. The rs is a cenral planner sochasic growh model, he second is capial asse pricing in general equilibrium and how i relaes o uiliy maximizaion. We coninue wih endogenous labour supply and he maching model of unemploymen. The nex secion hen covers how many maximizaion problems can be solved wihou using dynamic programming or he Lagrangian. In fac, many problems can be solved simply by insering, despie uncerainy. This will be illusraed wih many furher applicaions. A nal secion on nie horizons concludes. 159

160

Chaper 7 Sochasic di erence equaions and momens Before we look a di erence equaions in secion 7.4, we will rs spend a few secions reviewing basic conceps relaed o uncerain environmens. These conceps will be useful a laer sages. 7.1 Basics on random variables Le us rs have a look a some basics of random variables. This follows Evans, Hasings and Peacock (2000). 7.1.1 Some conceps A probabilisic experimen is an occurrence where a complex naural background leads o a chance oucome. The se of possible oucomes of a probabilisic experimen is called he possibiliy space. A random variable (RV) X is a funcion which maps from he possibiliy space ino a se of numbers. The se of numbers his RV can ake is called he range of his variable X. The disribuion funcion F associaed wih he RV X is a funcion which maps from he range ino he probabiliy domain [0,1], F (x) = Prob (X x) : The probabiliy ha X has a realizaion of x or smaller is given by F (x) : We now need o make a disincion beween discree and coninuous RVs. When he RV X has a discree range hen f (x) gives nie probabiliies and is called he probabiliy funcion or probabiliy mass funcion. The probabiliy ha X has he realizaion of x is given by f (x) : 161

162 Chaper 7. Sochasic di erence equaions and momens When he RV X is coninuous, he rs derivaive of he disribuion funcion F f(x) = df (x) dx is called he probabiliy densiy funcion f. The probabiliy ha he realizaion of X lies beween, say, a and b > a is given by F (b) F (a) = R b f (x) dx: Hence he probabiliy a ha X equals a is zero. 7.1.2 An illusraion Discree random variable Consider he probabilisic experimen ossing a coin wice. The possibiliy space is given by fhh; HT; T H; T T g. De ne he RV Number of heads. The range of his variable is given by f0; 1; 2g : Assuming ha he coin falls on eiher side wih he same probabiliy, he probabiliy funcion of his RV is given by 8 < f (x) = : Coninuous random variable :25 :5 :25 9 = 8 < ; for x = : Think of nex weekend. You migh consider going o a pub o mee friends. Before you go here, you do no know how much ime you will spend here. If you mee a lo of friends, you will say longer; if you drink jus one beer, you will leave soon. Hence, going o a pub on a weekend is a probabilisic experimen wih a chance oucome. The se of possible oucomes wih respec o he amoun of ime spen in a pub is he possibiliy space. Our random variable T maps from his possibiliy space ino a se of numbers wih a range from 0 o, le s say, 4 hours (as he pub closes a 1 am and you never go here before 9 p.m.). As ime is coninuous, T 2 [0; 4] is a coninuous random variable. The disribuion funcion F () gives you he probabiliy ha you spend a period of lengh or shorer in he pub. The probabiliy ha you spend beween 1.5 and wo hours in he pub is given by R 2 f () d; where f () is he densiy funcion f () = df () =d. 1:5 7.2 Examples for random variables We now look a some examples of RVs ha are useful for laer applicaions. As an RV is compleely characerized by is range and is probabiliy or densiy funcion, we will describe RVs by providing his informaion. Many more random variables exis han hose presened here and he ineresed reader is referred o he furher reading secion a he end. 0 1 2 :

7.2. Examples for random variables 163 7.2.1 Discree random variables Discree uniform disribuion range x 2 fa; a + 1; :::; b 1; bg probabiliy funcion f (x) = 1= (b a + 1) An example for his RV is he die. Is range is 1 o 6, he probabiliy for any number (a leas for fair dice) is 1=6. The Poisson disribuion range probabiliy funcion x 2 f0; 1; 2; :::g f (x) = e x x! Here is some posiive parameer. When we alk abou sochasic processes in par IV, his will be called he arrival rae. An example for his RV is e.g. he number of falling sars visible on a warm summer nigh a a nice beach. 7.2.2 Coninuous random variables Normal disribuion range x 2 ] 1; +1[ densiy funcion f (x) = p 1 e 1 2( x 2 2 ) 2 The mean and he sandard deviaion of X are given by and : Sandard normal disribuion This is he normal disribuion wih mean and sandard deviaion given by = 0 and = 1: Exponenial disribuion range x 2 [0; 1[ densiy funcion f (x) = e x Again, is some posiive parameer. One sandard example for x is he duraion of unemploymen for an individual who jus los his or her job.

164 Chaper 7. Sochasic di erence equaions and momens 7.2.3 Higher-dimensional random variables So far we have sudied one-dimensional RVs. In wha follows, we will occasionally work wih muli-dimensional RVs as well. For our illusraion purposes here i will su ce o focus on a wo-dimensional normal disribuion. Consider wo random variables X 1 and X 2 : They are (joinly) normally disribued if he densiy funcion of X 1 and X 2 is given by 1 f (x 1 ; x 2 ) = p 2 1 e 1 2(1 2 ) (~x2 1 2~x 1 ~x 2 +~x 2 2) ; (7.2.1) 2 1 2 where ~x i = x i i ; = 12 : (7.2.2) i 1 2 The mean and sandard deviaion of he RVs are denoed by i and i : The parameer is called he correlaion coe cien beween X 1 and X 2 and is de ned as he covariance 12 divided by he sandard deviaions (see ch. 7.3.1). The nice aspecs abou his wo-dimensional normally disribued RV (he same holds for n-dimensional RVs) is ha he densiy funcion of each individual RV X i ; i.e. he marginal densiy, is given by he sandard expression which is independen of he correlaion coe cien, 1 f (x i ) = p e 1 2 ~x2 i 2 2 : (7.2.3) i This implies a very convenien way o go from independen o correlaed RVs in a mulidimensional seing: When we wan o assume independen normally disribued RVs, we assume ha (7.2.3) holds for each random variable X i and se he correlaion coe cien o zero. When we wan o work wih dependen RVs ha are individually normally disribued, (7.2.3) holds for each RV individually as well bu, in addiion, we x a nonzero coe cien of correlaion. 7.3 Expeced values, variances, covariances and all ha Here we provide various de niions, some properies and resuls on ransformaions of RVs. Only in some seleced cases do we provide proofs. A more in deph inroducion can be found in many exbooks on saisics. 7.3.1 De niions For de niions, we shall focus on coninuous random variables. For discree random variables, he inegral is replaced by a sum - in very loose noaion, R b g (x) dx is replaced a by b i=ag (x i ), where a and b are consans which can be minus or plus in niy and g (x) is some funcion. In he following de niions, R :dx means he inegral over he relevan

7.3. Expeced values, variances, covariances and all ha 165 range (i.e. from minus o plus in niy or from he lower o upper bound of he range of he RV under consideraion). Mean EX R xf (x) dx z = g (x; y) ) EZ = R R g (x; y) f (x; y) dx dy Variance varx R (x EX) 2 f (x) dx kh uncenered momen EX k R x k f (x) dx kh cenered momen E (X EX) k R (x EX) k f (x) dx covariance cov(x; Y ) R R (x EX) (y EY ) f (x; y) dx dy correlaion coe cien XY cov (X; Y ) = p varx vary independence p (X 2 A; Y 2 B) = P (X 2 A) P (Y 2 B) Table 7.3.1 Some basic de niions 7.3.2 Some properies of random variables Basic Here are some useful properies of random variables. They are lised here for laer reference. More background can be found in many saisics exbooks. E [a + bx] = a + bex E [bx + cy ] = bex + cey E (XY ) = EXEY + cov (X; Y ) Table 7.3.2 Some properies of expecaions varx = E (X EX) 2 = E X 2 2XEX + (EX) 2 = EX 2 2 (EX) 2 + (EX) 2 = EX 2 (EX) 2 (7.3.1) var (a + bx) = b 2 varx Table 7.3.3 Some properies of variances var (X + Y ) = varx + vary + 2cov (X; Y ) (7.3.2) cov (X; X) = varx cov (X; Y ) = E (XY ) EXEY (7.3.3) cov (a + bx; c + dy ) = bd cov (X; Y )

166 Chaper 7. Sochasic di erence equaions and momens Table 7.3.4 Some properies of covariances Advanced Here we presen a heorem which is very inuiive and highly useful for analyically sudying he pro- and counercyclical behaviour of endogenous variables in models of business cycles. The heorem says ha if wo variables depend in he same sense on some RV (i.e. hey boh increase or decrease in he RV), hen hese wo variables have a posiive covariance. If e.g. boh GDP and R&D expendiure increase in TFP and TFP is random, hen GDP and R&D expendiure are procyclical. Theorem 7.3.1 Le X be a random variable and f (X) and g (X) wo funcions such ha f 0 (X) g 0 (X) R 0 8 x 2 X: Then cov (f (X) ; g (X)) R 0: Proof. We only prove he > 0 par. We know from (7.3.3) ha cov(y; Z) = E (Y Z) EY EZ: Wih Y = f (X) and Z = g (X), we have cov (f (X) ; g (X)) = E (f (X) g (X)) Ef (X) Eg (X) = R R R f (x) g (x) p (x) dx f (x) p (x) dx g (x) p (x) dx; where p (x) is he densiy of X. Hence cov (f (X) ; g (X)) > 0, R f (x) g (x) p (X) dx > R f (x) p (x) dx R g (x) p (x) dx: The las inequaliy holds for f 0 (X) g 0 (X) > 0 as shown by µcebyšev and presened in Mirinović (1970, Theorem 10, sec. 2.5, p. 40). 7.3.3 Funcions on random variables We will occasionally encouner he siuaion where we need o compue densiy funcions of funcions of RVs. Here are some examples. Linearly ransforming a normally disribued RV Consider a normally disribued RV X N (; 2 ) : Wha is he disribuion of he Y = a + bx? We know from ch. 7.3.2 ha for any RV, E (a + bx) = a + bex and V ar (a + bx) = b 2 V ar (X). As i can be shown ha a linear ransformaion of a normally disribued RV gives a normally disribued RV again, Y is also normally disribued wih Y N (a + b; b 2 2 ) :

7.3. Expeced values, variances, covariances and all ha 167 An exponenial ransformaion Consider he RV Y = e X where X is again normally disribued. The RV Y is hen lognormally disribued. A variable is lognormally disribued if is logarihm is normally disribued, ln Y N (; 2 ). The mean and variance of his disribuion are given by y = e + 1 2 2 ; 2 y = e 2+2 e 2 1 : (7.3.4) Clearly, Y can only have non-negaive realizaions. Transformaions of lognormal disribuions Le here be wo lognormally disribued variables Y and Z. Any ransformaion of he ype Y ; where is a consan, or producs like Y Z are also lognormally disribued. To show his, remember ha we can express Y and Z as Y = e X 1 and Z = e X 2, wih he X i being (joinly) normally disribued. Hence, for he rs example, we can wrie Y = e X 1 : As X 1 is normally disribued, Y is lognormally disribued. For he second, we wrie Y Z = e X 1 e X 2 = e X 1+X 2 : As he X i are (joinly) normally disribued, heir sum is as well and Y Z is lognormally disribued. Toal facor produciviy is someimes argued o be lognormally disribued. Is logarihm is hen normally disribued. See e.g. ch. 8.1.6. The general case Consider now a general ransformaion of he ype y = y (x) where he RV X has a densiy f (x) : Wha is he densiy of Y? The answer comes from he following Theorem 7.3.2 Le X be a random variable wih densiy f (x) and range [a; b] which can be ] 1; +1[: Le Y be de ned by he monoonically increasing funcion y = y (x) : Then he densiy g (y) is given by g (y) = f (x (y)) dx on he range [y(a); y(b)]: dy This heorem can be easily proven as follows. The proof is illusraed in g. 7.3.1. The gure plos he RV X on he horizonal and he RV Y on he verical axis. A monoonically increasing funcion y (x) represens he ransformaion of realizaions x ino y: Y y(b) N y(x) y(ex) y(a) a ex b X Figure 7.3.1 Transforming a random variable N

168 Chaper 7. Sochasic di erence equaions and momens Proof. The ransformaion of he range is immediaely clear from he gure: When X is bounded beween a and b; Y mus be bounded beween y (a) and y (b) : The proof for he densiy of Y requires a few more seps: The probabiliy ha y is smaller han some y (~x) is idenical o he probabiliy ha X is smaller han his ~x: This follows from he monooniciy of he funcion y (x) : As a consequence, he disribuion funcion (cumulaive densiy funcion) of Y is given by G (y) = F (x) where y = y (x) or, equivalenly, x = x (y) : The derivaive hen gives he densiy funcion, g (y) d dy G (y) = d dy F (x (y)) = f (x (y)) dx dy : 7.4 Examples of sochasic di erence equaions We now reurn o he rs main objecive of his chaper, he descripion of sochasic processes hrough sochasic di erence equaions. 7.4.1 A rs example The di erence equaion Possibly he simples sochasic di erence equaion is he following x = ax 1 + " ; (7.4.1) where a is a posiive consan and he sochasic componen " is disribued according o some disribuion funcion over a range which implies a mean and a variance 2 ; " (; 2 ) : We do no make any speci c assumpion abou " a his poin. Noe ha he sochasic componens " are i.i.d. (idenically and independenly disribued) which implies ha he covariance beween any wo disinc " is zero, cov(" ; " s ) = 0 8 6= s. An alernaive represenaion of x wih idenical disribuional properies would be x = ax 1 + + v wih v (0; 2 ). Solving by subsiuion In complee analogy o deerminisic di erence equaions in ch. 2.5.3, equaion (7.4.1) can be solved for x as a funcion of ime and pas realizaions of " ; provided we have a boundary condiion x 0 for = 0. By repeaed reinsering, we obain and evenually x 1 = ax 0 + " 1 ; x 2 = a [ax 0 + " 1 ] + " 2 = a 2 x 0 + a" 1 + " 2 ; x 3 = a a 2 x 0 + a" 1 + " 2 + "3 = a 3 x 0 + a 2 " 1 + a" 2 + " 3 x = a x 0 + a 1 " 1 + a 2 " 2 + ::: + " = a x 0 + s=1a s " s : (7.4.2)

7.4. Examples of sochasic di erence equaions 169 The mean A sochasic di erence equaion does no predic how he sochasic variable x for > 0 acually evolves, i only predics he disribuion of x for all fuure poins in ime. The realizaion of x is random. Hence, we can only hope o undersand somehing abou he disribuion of x. To do so, we sar by analyzing he mean of x for fuure poins in ime. Denoe he condiional expeced value of x by ~x ; ~x E 0 x ; (7.4.3) i.e. ~x is he value expeced when we are in = 0 for x in > 0: The expecaions operaor E 0 is condiional, i.e. i uses our knowledge in 0 when we compue he expecaion for x. The 0 says ha we have all informaion for = 0 and know herefore x 0 and " 0 ; bu we do no know " 1 ; " 2 ; ec. Pu di erenly, looking a he condiional disribuion of x : Wha is he mean of x condiional on x 0? By applying he expecaions operaor E 0 o (7.4.1), we obain ~x = a~x 1 + : This is a deerminisic di erence equaion which describes he evoluion of he expeced value of x over ime. There is again a sandard soluion o his equaion which reads ~x = a x 0 + s=1a s = a x 0 + 1 a 1 a ; (7.4.4) where we used s=1a s = a 0 + a 1 + ::: + a 1 = 1 i=0 ai and, from ch. 2.5.1, n i=0a i = (1 a n+1 ) = (1 a), for a < 1: This equaion shows ha he expeced value of x changes over ime as changes. Noe ha ~x migh increase or decrease (see he exercises). The variance Le us now look a he variance of x : We obain an expression for he variance by saring from (7.4.2) and observing ha he erms in (7.4.2) are all independen from each oher: a x 0 is a consan and he disurbances " s are i.i.d. by assumpion. The variance of x is herefore given by he sum of variances (compare (7.3.2) for he general case including he covariance), V ar (x ) = 0 + s=1 a s 2 V ar ("s ) = 2 s=1 a 2 s = 2 1 i=0 a2 i = 2 1 a2 1 a : 2 (7.4.5) We see ha i is also a funcion of : The fac ha he variance becomes larger he higher appears inuiively clear. The furher we look ino he fuure, he more randomness here is: equaion (7.4.2) shows ha a higher means ha more random variables are

170 Chaper 7. Sochasic di erence equaions and momens added up. (One should keep in mind, however, ha i.i.d. variables are added up. If hey were negaively correlaed, he variance would no necessarily increase over ime.) The fac ha we used V ar (a x 0 ) = 0 here also shows ha we work wih a condiional disribuion of x. We base our compuaion of he variance a some fuure poin in ime on our knowledge of he RV X in = 0: If we waned o make his poin very clearly, we could wrie V ar 0 (x ) : Long-run behaviour When we considered deerminisic di erence equaions like (2.5.6), we found he longrun behaviour of x by compuing he soluion of his di erence equaion and by leing approach in niy. This would give us he xpoin of his di erence equaion as e.g. in (2.5.7). The concep ha corresponds o a xpoin/ seady sae in an uncerain environmen, e.g. when looking a sochasic di erence equaions like (7.4.1), is he limiing disribuion. As saed earlier, sochasic di erence equaions do no ell us anyhing abou he evoluion of x iself, hey only ell us somehing abou he disribuion of x : I would herefore make no sense o ask wha x is in he long run - i will always remain random. I makes a lo of sense, however, o ask wha he disribuion of x is for he long run, i.e. for! 1: All resuls so far were obained wihou a speci c disribuional assumpion for " apar from specifying a mean and a variance. Undersanding he long-run disribuion of x in (7.4.1) is easy if we assume ha he sochasic componen " is normally disribued for each ; " N (; 2 ). In his case, saring in = 0; he variable x 1 is from (7.4.1) normally disribued as well. As a weighed sum of wo random variables ha are (joinly) normally disribued gives again a random variable wih a normal disribuion (where he mean and variance can be compued as in ch. 7.3.2), we also know from (7.4.1) ha x 1 ; x 2... are all normally disribued wih a densiy (compare ch. 7.2.2) where he mean and variance are funcions of ime, f (x ) = 1 p e 1 x 2 2 2 2 : Hence, in order o undersand he evoluion of he disribuion of x, we only have o nd ou how he expeced value and variance of he variable x evolves. Neglecing he cases of a 1, he mean ~x of our long-run normal disribuion is given from (7.4.4) by he xpoin lim ~x ~x 1 = 1!1 1 a ; 0 < a < 1: The variance of he long-run disribuion is from (7.4.5) lim var (x ) = 2 1!1 1 a : 2 Hence, he long-run disribuion of x is a normal disribuion wih mean = (1 variance 2 = (1 a 2 ). a) and

118 19 7.4. Examples of sochasic di erence equaions 171 The evoluion of he disribuion of x Given our resuls on he evoluion of he mean and he variance of x and he fac ha we assumed " o be normally disribued, we know ha x is normally disribued a each poin in ime. We can herefore draw he evoluion of he disribuion of x as in he following gure. A simple sochasic difference equaion x = ax 1 + ε 0.4 iniial disribuion 0.35 0.3 0.25 0.2 limiing disribuion 0.15 0.1 0 1 2 3 4 0.05 5 6 7 8 9 0 10 11 12 13 14 15 16 17 x 3 5 7 9 11 13 15 17 19 ime Figure 7.4.1 Evoluion of a disribuion over ime Remember ha we were able o plo a disribuion for each only because of properies of he normal disribuion. If we had assumed ha " is lognormally or equally disribued, we would no have been able o say somehing abou he disribuion of x easily. The means and variances in (7.4.4) and (7.4.5) would sill have been valid bu he disribuion of x for fuure is generally unknown for disribuions of " oher han he normal disribuion. An inerpreaion for many agens Le us now make an excursion ino he world of heerogeneous agens. Imagine he variable x i represens he nancial wealh of an individual i. Equaion (7.4.1) can hen be inerpreed o describe he evoluion of wealh x i over ime, x i = ax i 1 + " i : Given informaion in period = 0, i predics he densiy funcion f(x i ) for wealh in all fuure periods, given some iniial value x i0. Shocks are idenically and independenly disribued (i.i.d.) over ime and across individuals.

172 Chaper 7. Sochasic di erence equaions and momens Assuming a large number of agens i, more precisely a coninuum of agens wih mass n, a law of large numbers can be applied: Le all agens sar wih he same wealh x i0 = x 0 in = 0. Then he densiy f(x i ) for he wealh of any individual i in equals he realized wealh disribuion in for he coninuum of agens wih mass n: Pu di erenly: when he probabiliy for an individual o hold wealh beween some lower and upper bound in is given by p, he share of individuals ha hold wealh beween hese wo bounds in is also given by p. Toal wealh in in such a pure idiosyncraic risk seup (i.e. no aggregae uncerainy) is hen deerminisic (as are all oher populaion shares) and is given by x = n Z 1 1 x i f(x i )dx i = n ; (7.4.6) where is he average wealh over individuals or he expeced wealh of any individual i: The same argumen can also be made wih a discree number of agens. The wealh x i of individual i a is a random variable. Le he probabiliy ha x i is smaller han x be given by P (x i x) = p: Now assume here are n agens and herefore, a each poin in ime ; here are n independen random variables x i : Denoe by n he number of random variables x i ha have a realizaion smaller han x: The share of random variables ha have a realizaion smaller han x is denoed by q n=n: I is hen easy o show ha E q = p for all n and, more imporanly, lim n!1 var (q) = 0: This equals in words he saemen based on Judd above: The share in he oal populaion n is equal o he individual probabiliy if he populaion is large. This share becomes deerminisic for large populaions. Here is now a formal proof: De ne Y i as he number of realizaions below x for one x i ; i.e. Y i I (x i x) where I is he indicaor funcion which is 1 if he condiion in parenheses holds and 0 if no. Clearly, he probabiliy ha Y i = 1 is given by p; i.e. Y i is Bernoulli(p) disribued, wih EY i = p and vary i = p(1 p). (The Y i are i.i.d. if he x i are i.i.d..) Then he share q is given by q = n i=1y i =n: Is momens are E q = p and varq = p(1 p)=n. Hence, he variance ends o zero for n going o in niy. (More echnically, q ends o p in quadraic mean and herefore in probabiliy. The laer means we have a weak law of large numbers.) 7.4.2 A more general case Le us now consider he more general di erence equaion x = x 1 + " ; where he coe cien is now also sochasic a; 2 a ; " ; 2 " : Analyzing he properies of his process is prey complex. Wha can be easily done, however, is o analyze he evoluion of momens. As Vervaa (1979) has shown, he

7.4. Examples of sochasic di erence equaions 173 limiing disribuion has he following momens Ex j = j k=0 j k E k " j k Ex k : Assuming we are ineresed in he expeced value, we would obain Ex 1 = 1 k=0 k 1 E k " 1 k Ex k = 0 1 E 0 " 1 Ex 0 + 1 1 E 1 " 0 Ex 1 = E" + EEx: Solving for Ex yields Ex = E" 1 E = 1 a :

174 Chaper 7. Sochasic di erence equaions and momens

Chaper 8 Two-period models 8.1 An overlapping generaions model Le us now reurn o maximizaion issues. We do so in he conex of he mos simple example of a dynamic sochasic general equilibrium model. I is a sraighforward exension of he deerminisic model analyzed in secion 2.4. The srucure of he maximizaion problem of individuals, he iming of when uncerainy reveals iself and wha is uncerain depends on he fundamenal and exogenous sources of uncerainy. As he fundamenal source of uncerainy resuls here from he echnology used by rms, echnologies will be presened rs. Once his is done, we can derive he properies of he maximizaion problem households or rms face. 8.1.1 Technology Le here be an aggregae echnology Y = A K L 1 (8.1.1) The oal facor produciviy level A is uncerain. We assume ha A is idenically and independenly disribued (i.i.d.) in each period. The random variable A is posiive, has a mean A and a variance 2 : No furher informaion is needed abou is disribuion funcion a his poin, A A; 2 ; A > 0: (8.1.2) Assuming ha oal facor produciviy is i.i.d. means, iner alia, ha here is no echnological progress. One can imagine many di eren disribuions for A. In principle, all disribuions presened in he las secion are viable candidaes. Hence, we can work wih discree disribuions or coninuous disribuions. 8.1.2 Timing The sequence of evens is as follows. A he beginning of period, he capial sock K is inheried from he las period, given decisions from he las period. Then, oal 175

176 Chaper 8. Two-period models facor produciviy is revealed. Wih his knowledge, rms choose facor employmen and households choose consumpion (and hereby savings). K A c w r Figure 8.1.1 Timing of evens Hence, only a he end of he day does one really know how much one has produced. This implies ha wages and ineres paymens are also only known wih cerainy a he end of he period. The sae of he economy in is compleely described by K and he realizaion of A : All variables of he model are coningen on he sae. 8.1.3 Firms As a consequence of his iming of evens, rms do no bear any risk and hey pay he marginal produc of labour and capial o workers and capial owners a he end of he period, @Y w = p ; (8.1.3) @L 1 @Y L r = p = p A : (8.1.4) @K K All risk is herefore born by households hrough labour and capial income. In wha follows, he price will be se o uniy, p 1. All oher prices will herefore be real prices in unis of he consumpion good. 8.1.4 Ineremporal uiliy maximizaion This is he rs ime in his book ha we encouner a maximizaion problem wih uncerainy. The presenaion will herefore be relaively deailed in order o sress crucial feaures which are new due o he uncerainy. General approach We consider an agen ha lives for wo periods and works and consumes in a world as jus described. Agens consume in boh periods and choose consumpion such ha hey maximize expeced uiliy. In all generaliy concerning he uncerainy, she maximizes max E fu (c ) + u (c +1 )g ; (8.1.5)

8.1. An overlapping generaions model 177 where is he subjecive discoun facor ha measures he agen s impaience o consume. The expecaions operaor has an index, similar o E 0 in (7.4.3), o indicae ha expecaions are based on he knowledge concerning random variables which is available in period. We will see in an insan wheher he expecaions operaor E is pu a a meaningful place in his objecive funcion. Placing i in fron of boh insananeous consumpion from c and from c +1 is he mos general way of handling i. We will also have o specify laer wha he conrol variable of he household is. Imagine he agen chooses consumpion for he rs and he second period. When consumpion is chosen and given wage income w ; savings adjus such ha he budge consrain w = c + s (8.1.6) for he rs period holds. Noe ha his consrain always holds, despie he uncerainy concerning he wage. I holds in realizaions, no in expeced erms. In he second period, he household receives ineres paymens on savings made in he rs period and uses savings plus ineress for consumpion, (1 + r +1 ) s = c +1 : (8.1.7) One way of solving his problem (for an alernaive, see he exercise) is o inser consumpion levels from hese wo consrains ino he objecive funcion (8.1.5). This gives max s E fu (w s ) + u ((1 + r +1 ) s )g : This nicely shows ha he household in uences consumpion in boh periods by choosing savings s in he rs period. In fac, he only conrol variable he household can choose is s : Le us now ake ino consideraion, as drawn in he above gure, ha consumpion akes place a he end of he period afer revelaion of produciviy A in ha period. Hence, he consumpion level in he rs period is deermined by savings only and is hereby cerain. Noe ha even if consumpion c (or savings) was chosen before revelaion of oal facor produciviy, households would wan o consume a di eren level of consumpion c afer A is known. The rs choice would herefore be irrelevan and we can herefore focus on consumpion choice afer revelaion of uncerainy righ away. The consumpion level in he second period is de niely uncerain, however, as he nex period ineres rae r +1 depends on he realizaion of A +1 which is unknown in when decisions abou savings s are made. The objecive funcion can herefore be rewrien as max s u (w s ) + E u ((1 + r +1 ) s ) : For illusraion purposes, le us now assume a discree random variable A wih a nie number n of possible realizaions. Then, his maximizaion problem can be wrien as max s u (w s ) + n i=1 i u ((1 + r i;+1 ) s )

178 Chaper 8. Two-period models where i is he probabiliy ha he ineres rae r i;+1 is in sae i in +1: This is he same probabiliy as he probabiliy ha he underlying source of uncerainy A is in sae i: The rs-order condiion hen reads u 0 (w s ) = n i=1 i u 0 ((1 + r i;+1 ) s ) (1 + r i;+1 ) = E u 0 ((1 + r +1 ) s ) (1 + r +1 ) (8.1.8) Marginal uiliy of consumpion oday mus equal expeced marginal uiliy of consumpion omorrow correced by ineres and ime preference rae. Opimal behaviour in an uncerain world herefore means ex ane opimal behaviour, i.e. before random evens are revealed. Ex pos, i.e. afer resoluion of uncerainy, behaviour is subopimal when compared o he case where he realizaion is known in advance: Marginal uiliy in will (wih high probabiliy) no equal marginal uiliy (correced by ineres and ime preference rae) in + 1. This re ecs a simple fac of life: If I had known before wha would happen, I would have behaved di erenly. Ex ane, behaviour is opimal, ex pos, probably no. Clearly, if here was only one realizaion for A, i.e. 1 = 1 and i = 0 8 i > 1; we would have he deerminisic rs-order condiion we had in exercise 1 in ch. 2. The rs-order condiion also shows ha closed-form soluions are possible if marginal uiliy is of a muliplicaive ype. As savings s are known, he only quaniy which is uncerain is he ineres rae r +1 : If he insananeous uiliy funcion u (:) allows us o separae he ineres rae from savings, i.e. he argumen (1 + r i;+1 ) s in u 0 ((1 + r i;+1 ) s ) in (8.1.8), an explici expression for s and hereby consumpion can be compued. This will be shown in he following example and in exercise 6. An example - Cobb-Douglas preferences Now assume he household maximizes a Cobb-Douglas uiliy funcion as in (2.2.1). In conras o he deerminisic seup in (2.2.1), however, expecaions abou consumpion levels need o be formed. Preferences are herefore capured by E f ln c + (1 ) ln c +1 g : (8.1.9) When we express consumpion by savings, we can express he maximizaion problem by max s E f ln (w s ) + (1 ) ln ((1 + r +1 ) s )g which is idenical o max s ln (w s ) + (1 ) [ln s + E ln (1 + r +1 )] : (8.1.10) The rs-order condiion wih respec o savings reads = 1 (8.1.11) w s s and he opimal consumpion and saving levels are given by he closed-form soluion c = w ; c +1 = (1 ) (1 + r +1 ) w ; s = (1 ) w ; (8.1.12)

8.1. An overlapping generaions model 179 jus as in he deerminisic case (2.4.7). Thus, despie he seup wih uncerainy, one can compue a closed-form soluion of he same srucure as in he deerminisic soluions (2.2.4) and (2.2.5). Wha is peculiar here abou he soluions and also abou he rs-order condiion is he fac ha he expecaions operaor is no longer visible. One could ge he impression ha households do no form expecaions when compuing opimal consumpion pahs. The expecaions operaor go los in he rs-order condiion only because of he logarihm, i.e. he Cobb-Douglas naure of preferences. Neverheless, here is sill uncerainy for an individual being in : consumpion in + 1 is unknown as i is a funcion of he ineres-rae in + 1. Exercise 6 shows ha closed-form soluions are possible also for he CRRA case beyond Cobb-Douglas. 8.1.5 Aggregaion and he reduced form for he CD case We now aggregae over all individuals. Le here be L newborns each period. Consumpion of all young individuals in period is given from (8.1.12), C y = Lw = (1 )Y : The second equaliy used compeiive wage seing from (8.1.3), he fac ha wih a Cobb- Douglas echnology (8.1.3) can be wrien as w L = p (1 ) Y ; he normalizaion of p o uniy and he ideniy beween number of workers and number of young, L = L. Noe ha his expression would hold idenically in a deerminisic model. Wih (8.1.4), consumpion by old individuals amouns o C o = L [1 ] [1 + r ] w 1 = (1 )(1 + r )(1 )Y 1 : Consumpion in depends on oupu in 1 as savings are based on income in 1: The capial sock in period + 1 is given by savings of he young. One could show his as we did in he deerminisic case in ch. 2.4.2. In fac, adding uncerainy would change nohing o he fundamenal relaionships. We can herefore direcly wrie K +1 = Ls = L [1 ] w = (1 ) (1 ) Y = (1 ) (1 ) A K L 1 ; where we used he expression for savings for he Cobb-Douglas uiliy case from (8.1.12). Again, we succeeded in reducing he presenaion of he model o one equaion in one unknown. This allows us o illusrae he economy by using he simples possible phase diagram.

180 Chaper 8. K = K +1 Two-period models N K +1 K 2 = (1 )(1 )A 1 K1 L 1 K 3 = (1 )(1 )A 2 K2 L 1 K 1 = (1 )(1 )A 0 K0 L 1 K 0 K 1 K 2 K 3 Figure 8.1.2 Convergence owards a sochasic seady sae K N In principle his phase diagram looks idenical o he deerminisic case. There is of course a fundamenal di erence. The K +1 loci are a a di eren poin in each period. In = 0, K 0 and A 0 are known. Hence, by looking a he K 1 loci, we can compue K 1 as in a deerminisic seup. Then, however, TFP changes and, in he example ploed above, A 1 is larger han A 0 : Once A 1 is revealed in = 1; he new capial sock for period 2 can be graphically derived. Wih A 2 revealed in = 2; K 3 can be derived and so on. I is clear ha his economy will never end up in one seady sae, as in he deerminisic case, as A is di eren for each : As illusraed before, however, he economy can converge o a saionary sable disribuion for K : 8.1.6 Some analyical resuls The basic di erence equaion In order o beer undersand he evoluion of his economy, le us now look a some analyical resuls. The logarihm of he capial sock evolves according o ln K +1 = ln ((1 ) (1 )) + ln A + ln K + (1 ) ln L : Assuming a consan populaion size L = L, we can rewrie his equaion as where we used +1 = m 0 + + ; N( ; 2 ); (8.1.13) ln K (8.1.14) m 0 ln [(1 ) (1 )] + (1 ) ln L (8.1.15) and ln A capures he uncerainy semming from he TFP level A : As TFP is i.i.d., so is is logarihm. Since we assumed he TFP o be lognormally disribued, is logarihm is normally disribued. When we remove he mean from he random variable by replacing according o = " + ; where " N(0; 2 ); we obain +1 = m 0 + + + " :

8.1. An overlapping generaions model 181 The expeced value and he variance We can now compue he expeced value of by following he same seps as ch. 7.4.1, noing ha he only addiional parameer is m 0. Saring in = 0 wih an iniial value 0 ; and solving his equaion recursively gives = 0 + (m 0 + ) 1 1 + s=0 1 1 s " s : (8.1.16) Compuing he expeced value from he perspecive of = 0 gives E 0 = 0 + (m 0 + ) 1 1 ; (8.1.17) where we used E 0 " s = 0 for all s > 0 (and we se " 0 = 0 or assumed ha expecaions in 0 are formed before he realizaion of " 0 is known). For going o in niy, we obain lim E 0 = m 0 +!1 1 : (8.1.18) Noe ha he soluion in (8.1.16) is neiher di erence- nor rend saionary. Only in he limi, we have a (pure) random walk. For a very large ; in (8.1.16) is zero and (8.1.16) implies a random walk, 1 = " 1 : The variance of can be compued wih (8.1.16), where again independence of all erms is used as in (7.4.5), V ar ( ) = 0 + 0 + V ar 1 s=0 1 s " s = 1 s=0 2( 1 s) 2 = 1 2 1 2 2 (8.1.19) In he limi, we obain lim V ar ( ) = 2!1 1 : (8.1.20) 2 A graphical illusraion of hese ndings would look similar o he one in g. 7.4.1. Relaion o fundamenal uncerainy For a more convincing economic inerpreaion of he mean and he variance, i is useful o express some of he equaions, no as a funcion of properies of he log of A ; i.e. properies of ; bu direcly of he level of A. As (7.3.4) implies, he mean and variances of hese wo variables are relaed in he following fashion! 2 2 = ln 1 + A A ; (8.1.21)

182 Chaper 8. Two-period models = ln A 1 2 2 = ln A 1 2 ln 1 + A A 2! : (8.1.22) We can inser hese expressions ino (8.1.17) and (8.1.19) and sudy he evoluion over ime or direcly focus on he long-run disribuion of ln K. Doing so by insering ino (8.1.18) and (8.1.20) gives lim E 0 =!1 m 0 + ln A 1 2 ln 1 + 1 2 A A ; lim!1 V ar ( ) = ln 1 + 2 A A 1 2 : These equaions ell us ha uncerainy in TFP as capured by A no only a ecs he spread of he long-run disribuion of he capial sock bu also is mean. More uncerainy leads o a lower long-run mean of he capial sock. This is ineresing given he sandard resul of precauionary saving and he porfolio e ec (in seups wih more han one asse). 8.1.7 CRRA and CARA uiliy funcions Before concluding his rs analysis of opimal behaviour in an uncerain world, i is useful o explicily inroduce CRRA (consan relaive risk aversion) and CARA (consan absolue risk aversion) uiliy funcions. Boh are widely used in various applicaions. The Arrow-Pra measure of absolue risk aversion is u 00 (c) =u 0 (c) and he measure of relaive risk aversion is cu 00 (c) =u 0 (c). An individual wih uncerain consumpion a a small risk would be willing o give up a cerain absolue amoun of consumpion which is proporional o u 00 (c) =u 0 (c) o obain cerain consumpion. The relaive amoun she would be willing o give up is proporional o he measure of relaive risk aversion. The CRRA uiliy funcion is he same funcion as he CES uiliy funcion which we know from deerminisic seups in (2.2.10) and (5.3.2). Insering a CES uiliy funcion (c 1 1) = (1 ) ino hese wo measures of risk aversion gives a measure of absolue c() 1 c() risk aversion of = =c () and a measure of relaive risk aversion of ; which is minus he inverse of he ineremporal elasiciy of subsiuion. This is why he CES uiliy funcion is also called CRRA uiliy funcion. Even hough his is no consisenly done in he lieraure, i seems more appropriae o use he erm CRRA (or CARA) in seups wih uncerainy only. In a cerain world wihou risk, risk-aversion plays no role. The fac ha he same parameer capures risk aversion and ineremporal elasiciy of subsiuion is no always desirable as wo di eren conceps should be capured by di eren parameers. The laer can be achieved by using a recursive uiliy funcion of he Epsein-Zin ype. The ypical example for a uiliy funcion wih consan absolue risk aversion is he exponenial uiliy funcion u (c ()) = e c where is he measure of absolue risk aversion. Given relaively consan risk-premia over ime, he CRRA uiliy funcion seems o be preferable for applicaions. See furher reading on references o a more in-deph analysis of hese issues.

8.2. Risk-averse and risk-neural households 183 8.2 Risk-averse and risk-neural households Previous analyses in his book have worked wih he represenaive-agen assumpion. This means ha we negleced all poenial di erences across individuals and assumed ha hey are all he same (especially idenical preferences, labour income and iniial wealh). We now exend he analysis of he las secion by (i) allowing individuals o give loans o each oher. These loans are given a a riskless endogenous ineres rae r: As before, here is a normal asse which pays an uncerain reurn r +1 : We also (ii) assume here are wo ypes of individuals, he risk-neural ones and he risk-averse, denoed by i = a; n. The second assumpion is crucial: if individuals were idenical, i.e. if hey had idenical preferences and experienced he same income sreams, no loans would be given in equilibrium. In his world wih heerogeneous agens, we wan o undersand who owns which asses. We keep he analysis simple by analyzing a parial equilibrium seup. Households The budge consrains (8.1.6) and (8.1.7) of all individuals i now read w = c i + s i ; (8.2.1) c i +1 = s i 1 + r i : (8.2.2) In he rs period, here is he classic consumpion-savings choice. In addiion, here is an invesmen problem as savings need o be allocaed o he wo ypes of asses. Consumpion in he second period is paid for enirely by capial income. Ineress paid on he porfolio amoun o r i = i r +1 + 1 i r: (8.2.3) Here and in subsequen chapers, i denoes he share of wealh held by individual i in he risky asse. We solve his problem by he subsiuion mehod which gives us an unconsrained maximizaion problem. A household i wih ime preference rae and herefore discoun facor = 1= (1 + ) maximizes U i = u w s i + E u 1 + r i s i! max s i ;i (8.2.4) by now choosing wo conrol variables: The amoun of resources no used in he rs period for consumpion, i.e. savings s i ; and he share i of savings held in he risky asse. Firs-order condiions for s i and i, respecively, are u 0 c i = E u 0 c i +1 1 + r i ; (8.2.5) Eu 0 c i +1 [r+1 r] = 0: (8.2.6)

184 Chaper 8. Two-period models Noe ha he rs-order condiion for consumpion (8.2.5) has he same inerpreaion, once slighly rewrien, as he inerpreaion in deerminisic wo-period or in nie horizon models (see (2.2.6) in ch. 2.2.2 and (3.1.6) in ch. 3.1.2). When rewriing i as ( u 0 c i +1 E u 0 (c i ) 1 1= (1 + r i ) ) = 1; (8.2.7) we see ha opimal behaviour again requires us o equae marginal uiliies oday and omorrow (he laer in is presen value) wih relaive prices oday and omorrow (he laer also in is presen value). Of course, in his sochasic environmen, we need o express everyhing in expeced erms. As he ineres raes and consumpion omorrow are joinly uncerain, we can no bring i exacly in he form as known from above in (2.2.6) and (3.1.6). However, his will be possible, furher below in (9.1.10) in ch. 9.1.3. The rs-order condiion for says ha expeced reurns from giving a loan and holding he risky asse mus be idenical. Reurns consis of he ineres rae imes marginal uiliy. This condiion can bes be undersood when rs hinking of a cerain environmen. In his case, (8.2.6) would read r +1 = r : agens would be indi eren when holding wo asses only if hey receive he same ineres rae on boh asses. Under uncerainy and wih risk-neuraliy of agens, i.e. u 0 = cons:, we ge E r +1 = r : Agens hold boh asses only if he expeced reurn from he risky asses equals he cerain reurn from he riskless asse. Under risk-aversion, we can wrie his condiion as E u 0 c+1 i r+1 = E u 0 c+1 i r (or, given ha r is non-sochasic, as E u 0 c+1 i r+1 = re u 0 c+1 i ). This says ha agens do no value he ineres rae per se bu raher he exra uiliy gained from holding an asse: An asse provides ineres of r +1 which increases uiliy by u 0 c+1 i r+1 : The share of wealh held in he risky asse hen depends on he increase in uiliy from realizaions of r +1 across various saes and he expecaions operaor compues a weighed sum of heses uiliy increases: E u 0 c+1 i r+1 n j=1u 0 cj+1 i rj+1 j ; where j is he sae, r j+1 he ineres rae in his sae and j he probabiliy for sae j o occur. An agen is hen indi eren beween holding wo asses when expeced uiliy-weighed reurns are idenical. Risk-neural and risk-averse behaviour For risk-neural individuals (i.e. he uiliy funcion is linear in consumpion), he rs-order condiions become 1 = E [1 + n r +1 + (1 n ) r] ; (8.2.8) Er +1 = r: (8.2.9) The rs-order condiion for how o inves implies, ogeher wih (8.2.8), ha he endogenous ineres rae for loans is pinned down by he ime preference rae, 1 = [1 + r], r = (8.2.10)

8.2. Risk-averse and risk-neural households 185 as he discoun facor is given by = (1 + ) 1 as saed before in (8.2.4). Reinsering his resul ino (8.2.9) shows ha we need o assume ha an inerior soluion for Er +1 = exiss. This is no obvious for an exogenously given disribuion for he ineres rae r +1 bu i is more plausible for a siuaion where r is sochasic bu endogenous as in he analysis of he OLG model in he previous chaper. For risk-averse individuals wih u (c a ) = ln c a ; he rs-order condiion for consumpion reads wih i in (8.2.3) being replaced by a 1 c a = E 1 [1 + a r c a +1 + (1 a ) r] = E 1 +1 s a = 1 s a (8.2.11) where we used (8.2.2) for he second equaliy and he fac ha s as a conrol variable is deerminisic for he hird. Hence, as in he las secion, we can derive explici expressions. Use (8.2.11) and (8.2.1) and nd s a = c a, w c a = c a, c a = 1 1 + w : This gives wih (8.2.1) again and wih (8.2.2) s a = 1 + w ; c a +1 = 1 + [1 + a r +1 + (1 a ) r] w : (8.2.12) This is our closed-form soluion for risk-averse individuals in our heerogeneous-agen economy. Le us now look a he invesmen problem of risk-averse households. The derivaive of heir objecive funcion is given by he lef-hand side of he rs-order condiion (8.2.6) imes. Expressed for logarihmic uiliy funcion, and insering he opimal consumpion resul (8.2.12) yields d d U a = E 1 r +1 r [r c a +1 r] = (1 + ) E +1 1 + a r +1 + (1 a ) r r +1 r = (1 + ) E 1 + a (r +1 r) + r (1 + ) E X + a X : (8.2.13) The las sep de ned X r +1 r as a RV and as a consan. Who owns wha? I can now easily be shown ha (8.2.13) implies ha risk-averse individuals will no allocae any of heir savings o he risky asse, i.e. a = 0. Firs, observe ha he derivaive of he expression EX= ( + a X) from (8.2.13) wih respec o a is negaive d d a E X + a X = E X 2 ( + a X) 2 < 0 8 a :

186 Chaper 8. Two-period models The sign of his derivaive can also easily be seen from (8.2.13) as an increase in a implies a larger denominaor. Hence, when ploed, he rs-order condiion is downward sloping in a : Second, by guessing, we nd ha, wih (8.2.9), a = 0 sais es he rs-order condiion for invesmen, a X = 0 ) E + a X = EX = 0: Hence, he rs-order condiion is zero for a = 0: Finally, as he rs-order condiion is monoonically decreasing, a = 0 is he only value for which i is zero, This is illusraed in he following gure. X E + a X = 0, a = 0: du a d = (1 + )E X + a X N 0 a N Figure 8.2.1 The rs-order condiion (8.2.13) for he share a of savings held in he risky asse The gure shows ha expeced uiliy of a risk-averse individual increases for negaive a and falls for posiive a. Risk-averse individuals will hence allocae all of heir savings o loans, i.e. a = 0. They give loans o risk-neural individuals who in urn pay a cerain ineres rae equal o he expeced ineres rae. All risk is born by risk-neural individuals.

8.3. Pricing of coningen claims and asses 187 8.3 Pricing of coningen claims and asses 8.3.1 The value of an asse The quesion is wha an individual is willing o pay in = 0 for an asse ha is worh p j (which is uncerain) in = 1. As he assumpion underlying he pricing of asses is ha individuals behave opimally, he answer is found by considering a maximizaion problem where individuals solve a saving and porfolio problem. The individual Le an individual s preferences be given by E fu (c 0 ) + u (c 1 )g : This individual can inves in a risky and in a riskless invesmen form. I earns labour income w in he rs period and does no have any oher source of income. This income w is used for consumpion and savings. Savings are allocaed o several risky asses j and a riskless bond. The rs period budge consrain herefore reads w = c 0 + n j=1m j p j0 + b where he number of shares j are denoed by m j ; heir price by p j0 and he amoun of wealh held in he riskless asse is b. The individual consumes all income in he second period which implies he budge consrain c 1 = n j=1m j p j1 + (1 + r) b; where r is he ineres rae on he riskless bond. We can summarize his maximizaion problem as max u (c 0 ) + Eu ((1 + r) (w c 0 j m j p j0 ) + j m j p j1 ) : c 0 ;m j This expression illusraes ha consumpion in period one is given by capial income from he riskless ineres rae r plus income from asses j. I also shows ha here is uncerainy only wih respec o period one consumpion. This is because consumpion in period zero is a conrol variable. The consumpion in period one is uncerain as prices of asses are uncerain. In addiion o consumpion c 0 ; he number m j of shares bough for each asse are also conrol variables. Firs-order condiions and asse pricing Compuing rs-order condiions gives u 0 (c 0 ) = (1 + r) Eu 0 (c 1 )

188 Chaper 8. Two-period models This is he sandard rs-order condiion for consumpion. The rs-order condiion for he number of asses o buy reads for any asse j E [u 0 (c 1 ) ((1 + r) ( p j0 ) + p j1 )] = 0, E [u 0 (c 1 ) p j1 ] = (1 + r) p j0 Eu 0 (c 1 ) : The las sep used he fac ha he ineres rae and p j0 are known in 0 and can herefore be pulled ou of he expecaions operaor on he righ-hand side. Reformulaing his condiion gives p j0 = 1 1 + r E u0 (c 1 ) Eu 0 (c 1 ) p j1 (8.3.1) which is an expression ha can be given an inerpreaion as a pricing equaion. The price of an asse j in period zero is given by he discouned expeced price in period one. The price in period one is weighed by marginal uiliies. 8.3.2 The value of a coningen claim Wha we are now ineresed in are prices of coningen claims. Generally speaking, coningen claims are claims ha can be made only if cerain oucomes occur. The simples example of he coningen claim is an opion. An opion gives he buyer he righ o buy or sell an asse a a se price on or before a given dae. We will consider curren coningen claims whose values can be expressed as a funcion of he price of he underlying asse. Denoe by g (p j1 ) he value of he claim as a funcion of he price p j1 of asse j in period one. The price of he claim in zero is denoed by v (p j0 ) : When we add an addiional asse, i.e. he coningen claim under consideraion, o he se of asses considered in he previous secion, hen opimal behaviour of households implies v (p j0 ) = 1 1 + r E u0 (c 1 ) Eu 0 (c 1 ) g (p j1) : (8.3.2) If all agens were risk neural, we would know ha he price of such a claim would be given by v (p j0 ) = 1 1 + r Eg (p j1) (8.3.3) Boh equaions direcly follow from he rs-order condiion for asses. Under risk neuraliy, a household s uiliy funcion is linear in consumpion and he rs derivaive is herefore a consan. Including he coningen claim in he se of asses available o households, exacly he same rs-order condiion for heir pricing would hold. 8.3.3 Risk-neural valuaion There is a srand in he nance lieraure, see e.g. Brennan (1979), ha asks under wha condiions a risk neural valuaion of coningen claims holds (risk neural valuaion relaionship, RNVR) even when households are risk averse. This is idenical o asking

8.4. Naural volailiy I 189 under which condiions marginal uiliies in (8.3.1) do no show up. We will now brie y illusrae his approach and drop he index j. Assume he disribuion of he price p 1 can be characerized by a densiy f (p 1 ; ) ; where Ep 1. Then, if a risk neural valuaion relaionship exiss, he price of he coningen claim in zero is given by v (p 0 ) = 1 1 + r Z g (p 1 ) f (p 1 ; ) dp 1 ; wih = (1 + r) p 0 : This is (8.3.3) wih he expecaions operaor being replaced by he inegral over realizaions g (p 1 ) imes he densiy f (p 1 ). Wih risk averse households, he pricing relaionship would read, under his disribuion, v (p 0 ) = 1 1 + r Z u 0 (c 1 ) Eu 0 (c 1 ) g (p 1) f (p 1 ; ) dp 1 : This is (8.3.2) expressed wihou he expecaions operaor. The expeced value is lef unspeci ed here as i is no a priori clear wheher his expeced value equals (1 + r) p 0 also under risk aversion. I is hen easy o see ha a RNVR holds if u 0 (c 1 ) =Eu 0 (c 1 ) = f (p 1 ) =f (p 1 ; ) : Similar condiions are derived in ha paper for oher disribuions and for longer ime horizons. 8.4 Naural volailiy I Naural volailiy is a view of why growing economies experience phases of high and phases of low growh. The cenral belief is ha boh long-run growh and shor-run ucuaions are joinly deermined by economic forces ha are inheren o any real world economy. Long-run growh and shor-run ucuaions are boh endogenous and wo sides of he same coin: They boh sem from he inroducion of new echnologies. I is imporan o noe ha no exogenous shocks occur according o his approach. In his sense, i di ers from real business cycle (RBC) and sunspo models and also from endogenous growh models wih exogenous disurbances. There are various models ha analyse his view in more deail and an overview is provided a hp://www.waelde.com/nv.hml. This secion will look a a simple model ha provides he basic inuiion. More on naural volailiy will follow in ch. 11.5. 8.4.1 The basic idea The basic mechanism of he naural volailiy lieraure (and his is probably a necessary propery of any model ha wans o explain boh shor-run ucuaions and long-run growh) is ha some measure of produciviy (his could be labour or oal facor produciviy) does no grow smoohly over ime as in mos models of exogenous or endogenous long-run growh bu ha produciviy follows a sep funcion.

190 Chaper 8. Two-period models N log of produciviy x h produciviy sep funcion x smooh produciviy growh x h x h x h x h h ime Figure 8.4.1 Smooh produciviy growh in balanced growh models (dashed line) and sep-wise produciviy growh in models of naural volailiy N Wih ime on he horizonal and he log of produciviy on he verical axis, his gure shows a smooh produciviy growh pah as he dashed line. This is he smooh growh pah ha induces balanced growh. In models of naural volailiy, he growh pah of produciviy has periods of no change a all and poins in ime of discree jumps. Afer a discree jump, reurns on invesmen go up and an upward jump in growh raes resuls. Growh raes gradually fall over ime as long as produciviy remains consan. Wih he nex jump, growh raes jump up again. While his sep funcion implies long-run growh as produciviy on average grows over ime, i also implies shor-run ucuaions. The precise economic reasons given for his sep funcion - which is no simply imposed bu always follows from some deeper mechanisms - di er from one approach o he oher. A crucial implicaion of his sep-funcion is he implici belief ha economically relevan echnological jumps ake place once every 4-5 years. Each cycle of an economy and herefore also long-run growh go back o relaively rare evens. Flucuaions in ime series ha are of higher frequency han hese 4-5 years eiher go back o exogenous shocks, o measuremen error or oher disurbances in he economy. The sep funcion someimes capures jumps in oal facor produciviy, someimes only in labour produciviy for he mos recen vinage of a echnology. This di erence is imporan for he economic plausibiliy of he models. Clearly, one should no build a heory on large aggregae shocks o TFP, as hose are no easily observable. Some papers in he lieraure show indeed how small changes in echnology can have large e ecs (see furher reading ). This secion presens he simple sochasic naural volailiy model which allows us o show he di erence from exogenous shock models in he business cycle mos easily. 8.4.2 A simple sochasic model This secion presens he simples possible model ha allows us o undersand he di erence beween he sochasic naural volailiy and he RBC approach.

8.4. Naural volailiy I 191 Technologies Le he echnology be described by a Cobb-Douglas speci caion, Y = A K L 1 ; where A represens oal facor produciviy, K is he capial sock and L are hours worked. As variaions in hours worked are no required for he main argumen here, we consider L consan. Capial can be accumulaed according o Toal facor produciviy follows where q = q 0 K +1 = (1 ) K + I A +1 = (1 + q ) A (8.4.1) wih probabiliy The probabiliy depends on resources R invesed ino R&D, p 1 p : (8.4.2) p = p (R ) : (8.4.3) Clearly, he funcion p (R ) in his discree ime seup mus be such ha 0 p (R ) 1: The speci caion of echnological progress in (8.4.2) is probably bes suied o poin ou he di erences o RBC ype approaches: The probabiliy ha a new echnology occurs is endogenous. This shows boh he new growh lieraure radiion and he di erences from endogenous growh ype RBC models. In he laer approach, he growh rae is endogenous bu shocks are sill exogenously imposed. Here, he source of growh and ucuaions all sem from one and he same source, he jumps in q in (8.4.2). Opimal behaviour by households The resource consrain he economy needs o obey in each period is given by C + I + R = Y ; where C ; I and R are aggregae consumpion, invesmen and R&D expendiure, respecively. Assume ha opimal behaviour by households implies consumpion and invesmen ino R&D amouning o C = s C Y ; R = s R Y ; where s C is he consumpion and s R he saving rae in R&D. Boh of hem are consan. This would be he oucome of a wo-period maximizaion problem or an in nie horizon maximizaion problem wih some parameer resricion. As he naural volailiy lieraure has various papers where he saving rae is no consan, i seems reasonable no o develop an opimal saving approach here fully as i is no cenral o he naural volailiy view. See furher reading, however, for references o papers which develop a full ineremporal approach.

192 Chaper 8. Two-period models 8.4.3 Equilibrium Equilibrium is deermined by K +1 = (1 ) K + Y R C = (1 ) K + (1 s R s C ) Y : plus he random realizaion of echnology jumps, where he probabiliy of a jump depends on invesmen in R&D, p = p (s R Y ) : Assume we sar wih a echnological level of A 0. Le here be no echnological progress for a while, i.e. q = 0 for a cerain number of periods. Then he capial sock converges o is emporary seady sae de ned by K +1 = K K; K = (1 s R s C ) A 0 K L 1, K 1 = 1 s R s C A 0 L 1 : (8.4.4) In a seady sae, all variables are consan over ime. Here, variables are consan only emporarily unil he nex echnology jump occurs. This is why he seady sae is said o be emporary. The convergence behaviour o he emporary seady sae is illusraed in he following gure. K +1 K +1 =(1 δ)k +(1 s R s C )A 12 K α L 1 α K +1 =(1 δ)k +(1 s R s C )A 0 K α L 1 α K 0 K Figure 8.4.2 The Sisyphus economy - convergence o a emporary seady sae Wih a new echnology coming in, say, period 12, he oal facor produciviy increases, according o (8.4.1), from A 0 o A 12 = (1 + q) A 0 : The K +1 line shifs upwards, as shown in g. 8.4.2. As a consequence, which can also be seen in (8.4.4), he seady

8.5. Furher reading and exercises 193 sae increases. Subsequenly, he economy approaches his new seady sae. As growh is higher immediaely afer a new echnology is inroduced (growh is high when he economy is far away from is seady sae), he growh rae afer he inroducion of a new echnology is high and hen gradually falls. Cyclical growh is herefore characerized by a Sisyphus-ype behaviour: K permanenly approaches he curren, emporary seady sae. Every now and hen, however, his seady sae jumps ouwards and capial sars approaching i again. 8.5 Furher reading and exercises More on basic conceps of random variables han in ch. 7.1 and 7.2 can be found in Evans, Hasings and Peacock (2000). A very useful reference is Spanos (1999) who also reas funcions of several random variables or Severini (2005). For a more advanced reamen, see Johnson, Koz and Balakrishnan (1995). The resuls on disribuions here are used and exended in Bossmann, Kleiber and Wälde (2007). There is a long discussion on he applicaion of laws of large numbers in economics. An early conribuion is by Judd (1985). See Kleiber and Koz (2003) for more on disribuions. The example for he value of an asse is inspired by Brennan (1979). The lieraure on naural volailiy can be found in ch. 11 on p. 293. See any good exbook on Micro or Public economics (e.g. Mas-Colell e al., 1995, Akinson and Sigliz, 1980) for a more deailed reamen of measures of risk aversion. There are many papers using a CARA uiliy funcion. Examples include Hassler e al. (2005), Acemoglu and Shimer (1999) and Shimer and Werning (2007, 2008). Epsein-Zin preferences have been developed by Epsein and Zin (1989) for discree ime seups and applied iner alia in Epsein and Zin (1991). For an applicaion in macroeconomics in discree ime building on he Kreps-Poreus approach, see Weil (1990). The coninuous ime represenaion was developed in Svensson (1989). See also Obsfeld (1994) or Epaulard and Pommere (2003).

194 Chaper 8. Two-period models Exercises Chaper 7 Applied Ineremporal Opimizaion Sochasic di erence equaions and applicaions 1. Properies of random variables Is he following funcion a densiy funcion? Draw his funcion. f(x) = ae jxj ; > 0: (8.5.1) Sae possible assumpions abou he range of x and values for a: 2. The exponenial disribuion Assume he ime beween wo rare evens (e.g. he ime beween wo earhquakes or wo bankrupcies of a rm) is exponenially disribued. (a) Wha is he disribuion funcion of an exponenial disribuion? (b) Wha is he probabiliy of no even beween 0 and x? (c) Wha is he probabiliy of a leas one even beween 0 and x? 3. The properies of uncerain echnological change Assume ha oal facor produciviy in (8.1.2) is given by (a) (b) he densiy funcion A = A wih p Ā wih 1-p ; f(a ); A 2 Ā,Ā ; (c) he probabiliy funcion g(a ); A 2 fa 1 ; A 2 ; :::; A n g ; (d) A = A i wih probabiliy p i and (e) ln A +1 = ln A + " +1 : Make assumpions for and " +1 and discuss hem. Wha is he expeced oal facor produciviy and wha is is variance? Wha is he expeced oupu level and wha is is variance, given a echnology as in (8.1.1)?

8.5. Furher reading and exercises 195 4. Sochasic di erence equaions Consider he following sochasic di erence equaion y +1 = by + m + v +1 ; v N(0; 2 v) (a) Describe he limiing disribuion of y. (b) Does he expeced value of y converge o is xpoin monoonically? How does he variance of y evolve over ime? 5. Saving under uncerainy Consider an individual s maximizaion problem max E fu (c ) + u (c +1 )g subjec o w = c + s ; (1 + r +1 ) s = c +1 (a) Solve his problem by replacing her second period consumpion by an expression ha depends on rs period consumpion. (b) Consider now he individual s decision problem given he uiliy funcion u (c ) = c : Should you assume a parameer resricion on? (c) Wha is your implici assumpion abou? Can i be negaive or larger han one? Can he ime preference rae ; where = (1 + ) 1 ; be negaive? 6. Closed-form soluion for a CRRA uiliy funcion Le households maximize a CRRA-uiliy funcion U = E c 1 + (1 ) c 1 +1 = c 1 + (1 ) E c 1 +1 subjec o budge consrains (8.1.6) and (8.1.7). (a) Show ha an opimal consumpion-saving decision given budge consrains (8.1.6) and (8.1.7) implies savings of s = n 1 + w (1 ) where " = 1= is he ineremporal elasiciy of subsiuion and E (1 + r+1 ) 1 is he expeced (ransformed) ineres rae. Show furher ha consumpion when old is w c +1 = (1 + r +1 ) n o " 1 + and ha consumpion of he young is n c = n 1 + o " (1 ) (1 ) o " ; (1 ) o " w : (8.5.2)

196 Chaper 8. Two-period models (b) Discuss he link beween he savings expression s and he one for he logarihmic case in (8.1.12). Poin ou why c +1 is uncerain from he perspecive of : Is c uncerain from he perspecive of - and of 1? 7. OLG in general equilibrium Build an OLG model in general equilibrium wih capial accumulaion and auoregressive oal facor produciviy, ln A +1 = ln A + " +1 wih " +1 N ("; 2 ) : Wha is he reduced form? Can a phase-diagram be drawn? 8. Asse pricing Under wha condiions is here a risk neural valuaion formula for asses? In oher words, under which condiions does he following equaion hold? p jo = 1 1 + r Ep j1

Chaper 9 Muli-period models Uncerainy in dynamic models is probably mos ofen used in discree ime models. Looking a ime wih a discree perspecive has he advanage ha iming issues are very inuiive: Somehing happens oday, somehing omorrow, somehing he day before. Time in he real world, however, is coninuous. Take any wo poins in ime, you will always nd a poin in ime in beween. Wha is more, working wih coninuous ime models under uncerainy has quie some analyical advanages which make cerain insighs - afer an iniial heavier invesmen ino echniques - much simpler. We will neverheless follow he previous srucure of his book and rs presen models in discree ime. 9.1 Ineremporal uiliy maximizaion Maximizaion problems in discree imes can be solved by using many mehods. One paricularly useful one is - again - dynamic programming. We herefore sar wih his mehod and consider he sochasic sibling o he deerminisic case in ch. 3.3. 9.1.1 The seup wih a general budge consrain Due o uncerainy, our objecive funcion is slighly changed. In conras o (3.1.1), i now reads U = E 1 = u (c ) ; (9.1.1) where he only di erence lies in he expecaions operaor E : As we are in an environmen wih uncerainy, we do no know all consumpion levels c wih cerainy. We herefore need o form expecaions abou he implied insananeous uiliy from consumpion, u (c ). The budge consrain is given by x +1 = f (x ; c ; " ) : (9.1.2) This consrain shows why we need o form expecaions abou fuure uiliy: The value of he sae variable x +1 depends on some random source, denoed by " : Think of his " as uncerain TFP or uncerain reurns on invesmen. 197

198 Chaper 9. Muli-period models The opimal program is de ned in analogy o (3.3.2) by V (x ; " ) max U subjec o x +1 = f (x ; c ; " ) : fc g The addiional erm in he value funcion is " : I is useful o rea " explicily as a sae variable for reasons we will soon see. Noe ha his issue is relaed o he discussion of wha is a sae variable? in ch. 3.4.2. 9.1.2 Solving by dynamic programming In order o solve he maximizaion problem, we again follow he hree sep scheme. DP1: Bellman equaion and rs-order condiions The Bellman equaion is V (x ; " ) = max c fu (c ) + E V (x +1 ; " +1 )g : I again explois he fac ha he objecive funcion (9.1.1) is addiively separable, despie he expecaions operaor, assumes opimal behaviour as of omorrow and shifs he expecaions operaor behind insananeous uiliy of oday as u (c ) is cerain given ha c is a conrol variable. The rs-order condiion is u 0 (c ) + E V x+1 (x +1 ; " +1 ) @x +1 = u 0 (c ) + E V x+1 (x +1 ; " +1 ) @c @f (:) = 0: @c (9.1.3) I corresponds o he rs-order condiion (3.3.5) in a deerminisic seup. The only di erence lies in he expecaions operaor E : Marginal uiliy from consumpion does no equal he loss in overall uiliy due o less wealh bu he expeced loss in overall uiliy. Equaion (9.1.3) provides again an (implici) funcional relaionship beween consumpion and he sae variable, c = c (x ; " ) : As we know all sae variables in, we know he opimal choice of our conrol variable. Be aware ha he expecaions operaor applies o all erms in he brackes. If no confusion arises, hese brackes will be omied in wha follows. DP2: Evoluion of he cosae variable The second sep under uncerainy also sars from he maximized Bellman equaion. The derivaive of he maximized Bellman equaion wih respec o he sae variable is (using he envelope heorem) V x (x ; " ) = E V x+1 (x +1 ; " +1 ) @x +1 @x :

9.1. Ineremporal uiliy maximizaion 199 Observe ha x +1 by he consrain (9.1.2) is given by f (x ; c ; " ) ; i.e. by quaniies ha are known in : Hence, he derivaive @x +1 =@x = f x is non-sochasic and we can wrie his expression as (noe he similariy o (3.3.6)) V x (x ; " ) = f x E V x+1 (x +1 ; " +1 ) : (9.1.4) This sep clearly shows why i is useful o include " as a sae variable ino he argumens of he value funcion. If we had no done so, one could ge he impression, for he same argumen ha x +1 is known in due o (9.1.2), ha he shadow price V x (x +1 ; " +1 ) is non-sochasic as well and he expecaions operaor would no be needed. Given ha he value of he opimal program depends on " ; however, i is clear ha he cosae variable V 0 (x +1 ; " +1 ) is random in indeed. (Some of he subsequen applicaions will rea " as an implici sae variable and will no always be as explici as here.) DP3: Insering rs-order condiions Given ha @f (:) =@c = f c is non-random in ; we can rewrie he rs-order condiion as u 0 (c ) + f c E V x+1 (x +1 ; " +1 ) = 0: Insering i ino (9.1.4) gives V x (x ; " ) = f x f c u 0 (c ) : Shifing his expression by one period yields V x+1 (x +1 ; " +1 ) = fx +1 f c+1 u 0 (c +1 ) : Insering V x (x ; " ) and V x+1 (x +1 ; " +1 ) ino he cosae equaion (9.1.4) again, we obain f c u 0 (c ) = E f x+1 u 0 (c +1 ) : f c+1 9.1.3 The seup wih a household budge consrain Le us now look a a rs example - a household ha maximizes uiliy. The objecive funcion is given by (9.1.1), U = E 1 = u (c ) : I is maximized subjec o he following budge consrain (of which we know from (2.5.13) or (3.6.6) ha i s well ino a general equilibrium seup), a +1 = (1 + r ) a + w p c : (9.1.5) Noe ha he budge consrain mus hold afer realizaion of random variables, no in expeced erms. From he perspecive of ; all prices (r, w, p ) in are known, prices in + 1 are uncerain.

200 Chaper 9. Muli-period models 9.1.4 Solving by dynamic programming DP1: Bellman equaion and rs-order condiions Having undersood in he previous general chaper ha uncerainy can be explicily reaed in he form of a sae variable, we limi our aenion o he endogenous sae variable a here. I will urn ou ha his keeps noaion simpler. The value of opimal behaviour is herefore expressed by V (a ) and he Bellman equaion can be wrien as The rs-order condiion for consumpion is V (a ) = max c fu (c ) + E V (a +1 )g : u 0 (c ) + E V 0 (a +1 ) @a +1 @c = u 0 (c ) E V 0 (a +1 ) p = 0; where he second sep compued he derivaive by using he budge consrain (9.1.5). Rewriing he rs-order condiion yields as he price p is known in : DP2: Evoluion of he cosae variable u 0 (c ) = p E V 0 (a +1 ) (9.1.6) Di ereniaing he maximized Bellman equaion gives (using he envelope heorem) V 0 (a ) = E V 0 (a +1 ) @a +1 =@a : Again using he budge consrain (9.1.5) for he parial derivaive, we nd V 0 (a ) = [1 + r ] E V 0 (a +1 ) : (9.1.7) Again, he erm [1 + r ] was pu in fron of he expecaions operaor as r is known in : This di erence equaion describes he evoluion of he shadow price of wealh in he case of opimal consumpion choices. DP3: Insering rs-order condiions Insering he rs-order condiion (9.1.6) gives V 0 (a ) = [1 + r ] u0 (c ) p : Insering his expression ino he di ereniaed maximized Bellman equaion (9.1.7) wice gives a nice Euler equaion, [1 + r ] u0 (c ) p = [1 + r ] E [1 + r +1 ] u0 (c +1 ) p +1, u0 (c ) p = E u 0 (c +1 ) (1 + r +1 ) 1 p +1 : (9.1.8) Rewriing i as we did before wih (8.2.7), we ge u 0 (c +1 ) p E u 0 (c ) (1 + r +1 ) 1 = 1 (9.1.9) p +1

9.2. A cenral planner 201 which allows us o give he same inerpreaion as he rs-order condiion in a wo-period model, boh deerminisic (as in eq. (2.2.6) in ch. 2.2.2) and sochasic (as in eq. (8.2.5) in ch. 8.2) and as in deerminisic in nie horizon models (as in eq. (3.1.6) in ch. 3.1.2): Relaive marginal uiliy mus be equal o relaive marginal prices - aking ino accoun ha marginal uiliy in + 1 is discouned a and he price in + 1 is discouned by using he ineres rae. Using wo furher assumpions, he expression in (9.1.8) can be rewrien such ha we come even closer o he deerminisic Euler equaions: Firs, le us choose he oupu good as numeraire and hereby se prices p = p +1 = p as consan. This allows us o remove p and p +1 from (9.1.8). Second, we assume ha he ineres rae is known (say we are in a small open economy wih inernaional capial ows). Hence, expecaions are formed only wih respec o consumpion in + 1: Taking hese wo aspecs ino accoun, we can wrie (9.1.8) as u 0 (c ) E u 0 (c +1 ) = 1 1= (1 + r +1 ) : (9.1.10) This is now as close o (2.2.6) and (3.1.6) as possible. The raio of (expeced discouned) marginal uiliies is idenical o he raio of relaive prices. 9.2 A cenral planner Le us now consider he classic cenral planner problem for a sochasic growh economy. We specify an opimal growh model where oal facor produciviy is uncerain. In addiion o his, we also allow for oil as an inpu good for producion. This allows us o undersand how some variables can easily be subsiued ou in a maximizaion problem even hough we are in an ineremporal sochasic world. Consider a echnology where oupu is produced wih oil O in addiion o he sandard facors of producion, Y = A K O L 1 Again, oal facor produciviy A is sochasic. Now, he price of oil, q ; is sochasic as well. Le capial evolve according o K +1 = (1 ) K + Y q O C (9.2.1) which is a rade balance and good marke clearing condiion all in one. planner maximizes max E 1 = u (C ) fc ;O g The cenral by choosing a pah of aggregae consumpion ows C and oil consumpion O : A, all variables indexed are known. The only uncerainy concerns TFP and he price of oil in fuure periods.

202 Chaper 9. Muli-period models DP1: Bellman equaion and rs-order condiions The Bellman equaion reads V (K ) = max C;O fu (C ) + E V (K +1 )g : The only sae variable included as argumen is he capial sock. Oher sae variables (like he price q of oil) could be included bu would no help in he derivaion of opimaliy condiions. See he discussion in ch. 3.4.2. The rs-order condiion for consumpion is u 0 (C ) + E dv (K +1 ) dk +1 [ 1] = 0, u 0 (C ) = E V 0 (K +1 ) For oil i reads E V 0 @ (K +1 ) [Y q O ] = 0, @Y = q : @O @O The las sep used he fac ha all variables in are known and he parial derivaive wih respec o O can herefore be moved in fron of he expecaions operaor - which hen cancels. We herefore have obained a sandard period-for-period opimaliy condiion as i is known from saic problems. This is he ypical resul for a conrol variable which has no ineremporal e ecs as, here, impors of oil a ec oupu Y and he coss q O in he resource consrain (9.2.1) conemporaneously. DP2: Evoluion of he cosae variable The derivaive of he Bellman equaion wih respec o he capial sock K gives (using he envelope heorem) V 0 @ (K ) = E V (K +1 ) = 1 + @Y E V 0 (K +1 ) (9.2.2) @K @K h jus as in he economy wihou oil. The erm 1 + @Y can be pulled ou of he expecaion operaor as A and hereby @Y @K is known a he momen of he savings decision. DP3: Insering rs-order condiions Following he same seps as above wihou oil, we again end up wih u 0 (C ) = E u 0 (C +1 ) 1 + @Y +1 @K +1 The crucial di erence is, ha now expecaions are formed wih respec o echnological uncerainy and uncerainy concerning he price of oil. @K i 9.3 Asse pricing in a one-asse economy This secion reurns o he quesion of asse pricing. Ch. 8.3.1 reaed his issue in a parial equilibrium seing. Here, we ake a general equilibrium approach and use a simple sochasic model wih one asse, physical capial. We hen derive an equaion ha expresses he price of capial in erms of income sreams from holding capial. In order o be as explici as possible abou he naure of his (real) capial price, we do no choose a numeraire good in his secion.

9.3. Asse pricing in a one-asse economy 203 9.3.1 The model Technologies The echnology used o produce a homogeneous oupu good is of he simple Cobb- Douglas form Y = A K L 1 ; (9.3.1) where TFP A is sochasic. Labour supply L is exogenous and x, he capial sock is denoed by K : Households Household preferences are sandard and given by U = E 1 = u (c ) : We sar from a budge consrain ha can be derived like he budge consrain (2.5.10) in a deerminisic world. Wealh is held in unis of capial K where he price of one uni is v : When we de ne he ineres rae as he budge consrain (2.5.10) reads Goods marke r wk v ; (9.3.2) k +1 = (1 + r ) k + w v p v c : (9.3.3) Invesmen and consumpion goods are raded on he same goods marke. Toal supply is given by Y ; demand is given by gross invesmen K +1 K + K and consumpion C. Expressed in a well-known way, goods marke equilibrium yields he resource consrain of he economy, K +1 = K + Y C K : (9.3.4) 9.3.2 Opimal behaviour Firms maximize insananeous pro s which implies rs-order condiions w = p @Y =@L; w K = p @Y =@K : (9.3.5) Facor rewards are given by heir value marginal producs. Given he households preferences and he consrain in (9.3.3), opimal behaviour by households is described by (his follows idenical seps as for example in ch. 9.1.3 and is reaed in ex. 2), u 0 (C ) p =v = E u 0 (C +1 ) (1 + r +1 ) 1 p +1 =v +1 : (9.3.6) This is he sandard Euler equaion exended for prices, given ha we have no chosen a numeraire. We replaced c by C o indicae ha his is he evoluion of aggregae (and no individual) consumpion.

204 Chaper 9. Muli-period models 9.3.3 The pricing relaionship Le us now urn o he main objecive of his secion and derive an expression for he real price of one uni of capial, i.e. he price of capial in unis of he consumpion good. Saring from he Euler equaion (9.3.6), we inser he ineres rae (9.3.2) in is general formulaion, i.e. including all prices, and rearrange o nd u 0 (C ) v = E u 0 (C +1 ) 1 + wk +1 p v +1 v+1 = E u 0 (C +1 ) (1 ) v +1 + @Y +1 p +1 p +1 @K +1 (9.3.7) Now de ne a discoun facor +1 u 0 (C +1 ) =u 0 (C ) and d as he ne dividend paymens, i.e. paymens o he owner of one uni of capial. Ne dividend paymens per uni of capial amoun o he marginal produc of capial @Y =@K minus he share of capial ha depreciaes - ha goes kapu - each period imes he real price v =p of one uni of capial, d @Y @K v p : Insering his yields v p = E +1 : d +1 + v +1 : (9.3.8) p +1 Noe ha all variables uncerain from he perspecive of oday in appear behind he expecaions operaor. Now assume for a second ha we are in a deerminisic world and he economy is in a seady sae. Equaion (9.3.8) could hen be wrien wih +1 = 1 and wihou h d +1 + v +1 he expecaions operaor as v p = forward, saring in v and insering repeaedly gives v p = d +1 + d +2 + p +1 i : Solving his linear di erenial equaion d +3 + d +4 + v +4 p +4 = d +1 + 2 d +2 + 3 d +3 + 4 d +4 + 4 v +4 p +4 : Coninuing o inser, one evenually and obviously ends up wih v p = T s=1 s d +s + T v +T p +T : The price v of a uni of capial is equal o he discouned sum of fuure dividend paymens plus is discouned price (once sold) in + T. In an in nie horizon perspecive, his becomes v = 1 p s=1 s v +T d +s + lim T!1 T : p +T In our sochasic seup, we can proceed according o he same principles as in he deerminisic world bu need o ake he expecaions operaor and he discoun facor ino accoun. We replace v +1 p +1 in (9.3.8) by E +1 +2 h d +2 + v +2 p +2 i and hen v +2 =p +2

9.3. Asse pricing in a one-asse economy 205 and so on o nd v p = E +1 d +1 + E +1 +2 d +2 + E +2 +3 d +3 + v +3 p +3 = E +1 d +1 + 2 +1 +2 d +2 + 3 +1 +2 +3 d +3 + 3 +1 +2 +3 v +3 p +3 = E 3 s=1 s d +s + E 3 v +3 p +3 ; (9.3.9) where we de ned he discoun facor o be s s s n=1 +n = s s u 0 (C +n ) n=1 u 0 (C +n 1 ) = u 0 (C +s ) s u 0 (C ) : The discoun facor adjuss discouning by he preference parameer ; by relaive marginal consumpion and by prices. Obviously, (9.3.9) implies for larger ime horizons = E T s=1 s d +s + E T v +T : Again, wih an in nie horizon, his reads v p v p = E 1 s=1 s d +s + lim T!1 E T v +T : (9.3.10) The real price v =p amouns o he discouned sum of fuure dividend paymens d +s : The discoun facor is s which conains marginal uiliies, relaive prices and he individual s discoun facor : The erm lim T!1 E T v +T is a bubble erm for he price of capial and can usually be se equal o zero. As he derivaion has shown, he expression for he price v =p is simply a rewrien version of he Euler equaion. 9.3.4 More real resuls The price of capial again The resul on he deerminans of he price of capial is useful for economic inuiions and received a lo of aenion in he lieraure. Bu can we say more abou he real price of capial? The answer is yes and i comes from he resource consrain (9.3.4). This consrain can be undersood as a goods marke clearing condiion. The supply of goods Y equals demand resuling from gross invesmen K +1 K + K and consumpion. The price of one uni of he capial good herefore equals he price of one uni of he consumpion and oupu good, provided ha invesmen akes place, i.e. I > 0: Hence, v = p : : (9.3.11) The real price of capial v =p is jus equal o one. No surprisingly, capial goods and consumpion goods are raded on he same marke.

206 Chaper 9. Muli-period models The evoluion of consumpion (and capial) When we wan o undersand wha his model ells us abou he evoluion of consumpion, we can look a a modi ed version of (9.3.6) by insering he ineres rae (9.3.2) wih he marginal produc of capial from (9.3.5) and he price expression (9.3.11), u 0 (C ) = E u 0 (C +1 ) 1 + @Y +1 : @K +1 This is he sandard Euler equaion (see e.g. (9.1.10)) ha predics how real consumpion evolves over ime, given he real ineres rae and he discoun facor : Togeher wih (9.3.4), we have a sysem in wo equaions ha deermine C and K (given appropriae boundary condiions). The price p and hereby he value v can no be deermined (which is of course a consequence of Walras law). The relaive price is rivially uniy from (9.3.11), v =p = 1: Hence, he predicions concerning real variables do no change when a numeraire good is no chosen. An endowmen economy Many papers work wih pure endowmen economies. We will look a such an economy here and see how his can be linked o our seup wih capial accumulaion. Consider an individual ha can save in one asse and whose budge consrain is given by (9.3.3). Le his household behave opimally such ha opimal consumpion follows (9.3.7). Now change he capial accumulaion equaion (9.3.4) such ha - for whaever reasons - K is consan and le also, for simpliciy, depreciaion be zero, = 0. Then, oupu is given according o (9.3.1) by Y = A K L 1 ; i.e. i follows some exogenous sochasic process, depending on he realizaion of A. This is he exogenous endowmen of he economy for each period : Furher, consumpion equals oupu in each period, C = Y : Insering oupu ino he Euler equaion (9.3.7) gives u 0 (Y ) v = E u 0 (Y +1 ) (1 ) v +1 + @Y +1 p p +1 @K +1 The equaion shows ha in an endowmen economy where consumpion is exogenously given a each poin in ime and households save by holding capial (which is consan on he aggregae level), he price v =p of he asse changes over ime such ha households wan o consume opimally he exogenously given amoun Y : This equaion provides a descripion of he evoluion of he price of he asse in an endowmen economy. These aspecs were analyzed for example by Lucas (1978) and many ohers.

9.4. Endogenous labour supply 207 9.4 Endogenous labour supply The maximizaion problem The analysis of business cycles is radiionally performed by including an endogenous labour supply decision in he consumpion and saving framework we know from ch. 9.1. We will now solve such an exended maximizaion problem. The objecive funcion (9.1.1) is exended o re ec ha households value leisure. We also allow for an increase in he size of he household. The objecive funcion now reads U = E 1 = u (c ; T l ) n ; where T is he oal endowmen of his individual wih ime and l is hours worked in period : Toal ime endowmen T is, say, 24 hours or, subracing ime for sleeping and oher regular non-work and non-leisure aciviies, 15 or 16 hours. Consumpion per member of he family is given by c and u (:) is insananeous uiliy of his member. The number of members in is given by n : The budge consrain of he household is given by ^a +1 = (1 + r ) ^a + n l w n c where ^a n a is household wealh in period : Leing w denoe he hourly wage, n l w sands for oal labour income of he family, i.e. he produc of individual income l w imes he number of family members. Family consumpion in is n c : Solving by he Lagrangian We now solve he maximizaion problem max U subjec o he consrain by choosing individual consumpion c and individual labour supply l : We solve his by using a Lagrange funcion. A soluion by dynamic programming would of course also work. For he Lagrangian we use a Lagrange muliplier for he consrain in : This makes i di eren o he Lagrange approach in ch. 3.1.2 where he consrain was an ineremporal budge consrain. I is similar o ch. 3.7 where an in nie number of mulipliers is also used. In ha chaper, uncerainy is, however, missing. The Lagrangian here reads L = E 1 = u (c ; T l ) n + [(1 + r ) ^a + n l w n c ^a +1 ] : We rs compue he rs-order condiions for consumpion and hours worked for one poin in ime s; L cs = E s @ u (c s ; T l s ) n s s n s = 0; (9.4.1) @c s L ls = E s @ @ (T l s ) u (c s; T l s ) n s + s n s w s = 0: (9.4.2)

208 Chaper 9. Muli-period models As discussed in 3.7, we also need o compue he derivaive wih respec o he sae variable. Consisen wih he approach in he deerminisic seup of ch. 3.8, we compue he derivaive wih respec o he rue sae variable of he family, i.e. wih respec o ^a : This derivaive is L^as = E f s 1 + s [1 + r s ]g = 0, E s 1 = E f(1 + r s ) s g : (9.4.3) This corresponds o (3.7.5) in he deerminisic case, only ha here we have an expecaions operaor. Opimal consumpion As he choice for he consumpion level c s in (9.4.1) will be made in s, we can assume ha we have all informaion in s a our disposal. When we apply expecaions E s, we see ha all expecaions are made wih respec o variables of s: Cancelling n s ; we herefore know ha in all periods s we have s @ @c s u (c s ; T l s ) = s : (9.4.4) We can now replace s and s 1 in (9.4.3) by he expressions we ge from his equaion where s is direcly available and s 1 is obained by shifing he opimaliy condiion back in ime, i.e. by replacing s by s 1: We hen nd E s 1 @ u (c s 1 ; T l s 1 ) = E (1 + r s ) s @ u (c s ; T l s ), @c s 1 @c s @ E u (c s 1 ; T l s 1 ) = E (1 + r s ) @ u (c s ; T l s ) : @c s 1 @c s Now imagine, we are in s 1: Then we form expecaions given all he informaion we have in s 1: As hours worked l s 1 and consumpion c s 1 are conrol variables, hey are known in s 1. Hence, using an expecaions operaor E s 1 ; we can wrie @ E s 1 u (c s 1 ; T l s 1 ) = @c s 1 @ @c s 1 u (c s 1 ; T l s 1 ) = E s 1 (1 + r s ) @ u (c s ; T l s ) : @c s Economically speaking, opimal consumpion-saving behaviour requires marginal uiliy from consumpion by each family member in s 1 o equal marginal uiliy in s; correced for impaience and he ineres rae. This condiion is similar o he one in (9.1.8), only ha here, we have uiliy from leisure as well.

9.5. Solving by subsiuion 209 Labour-leisure choice Le us now look a he second rs-order condiion (9.4.2) o undersand he condiion for opimal inra-emporal labour supply. When an individual makes he decision of how much o work in s; she acually nds herself in s: Expecaions are herefore formed in s and replacing by s and rearranging slighly, he condiion becomes, @ E s @ (T l s ) u (c s; T l s ) n s = E s f s n s w s g : As none of he variables in he curly brackes are random from he perspecive of s; afer removing he expecaions operaor and cancelling n s on boh sides, we obain, @ u (c @(T l s) s; T l s ) = s w s : We nally use (9.4.4) expressed for = s and wih n s cancelled @ o obain an expression for he shadow price s ; @c s u (c s ; T l s ) = s : Insering his yields @ @(T l s) s; T l s ) @ @c s u (c s ; T l s ) = w s : This is he sandard condiion known from saic models. I also holds here as he labour-leisure choice is a decision made in each period and has no ineremporal dimension. The rade-o is enirely inra-emporal. In opimum i requires ha he raio of marginal uiliy of leisure o marginal uiliy of consumpion is given by he raio of he price of leisure - he wage w s - o he price of one uni of he consumpion good - which is 1 here. Wheher an increase in he price of leisure implies an increase in labour supply depends on properies of he insananeous uiliy funcion u (:) : If he income e ec caused by higher w s dominaes he subsiuion e ec, a higher wage would imply fewer hours worked. More on his can be found in many inroducory exbooks o Microeconomics. 9.5 Solving by subsiuion This secion is on how o do wihou dynamic programming. We will ge o know a mehod ha allows us o solve sochasic ineremporal problems in discree ime wihou dynamic programming. Once his chaper is over, you will ask yourself why dynamic programming exiss a all... 9.5.1 Ineremporal uiliy maximizaion The objecive is again max fcg E 0 1 =0 u (c ) subjec o he consrain a +1 = (1 + r ) a + w c : The household s conrol variables are fc g ; he sae variable is a ; he ineres rae r and he wage w are exogenously given. Now rewrie he objecive funcion and

210 Chaper 9. Muli-period models inser he consrain wice, E 0 s 1 =0 u (c ) + s u (c s ) + s+1 u (c s+1 ) + 1 =s+2 u (c ) = E 0 s 1 =0 u (c ) + E 0 s u ((1 + r s )a s + w s a s+1 ) + E 0 s+1 u ((1 + r s+1 ) a s+1 + w s+1 a s+2 ) + E 0 1 =s+2 u (c ) : Noe ha he expecaions operaor always refers o knowledge available a he beginning of he planning horizon, i.e. o = 0: Now compue he rs-order condiion wih respec o a s+1 : This is unusual as we direcly choose he sae variable which is usually undersood o be only indirecly in uenced by he conrol variable. Clearly, however, his is jus a convenien rick: by choosing a s+1 ; which is a sae variable, we really choose c s. The derivaive wih respec o a s+1 yields E 0 s u 0 (c s ) = E 0 s+1 u 0 (c s+1 ) (1 + r s+1 ): This almos looks like he sandard opimal consumpion rule. The di erence lies in he expecaions operaor being presen on boh sides. This is no surprising as we opimally chose a s+1 (i.e. c s ), knowing only he sae of he sysem in = 0: If we now assume we are in s; our expecaions would be based on knowledge in s and we could replace E 0 by E s : We would hen obain E s s u 0 (c s ) = s u 0 (c s ) for he lef-hand side and our opimaliy rule reads u 0 (c s ) = E s u 0 (c s+1 ) (1 + r s+1 ): This is he rule we know from Bellman approaches, provided e.g. in (9.1.8). 9.5.2 Capial asse pricing Le us now ask how an asse ha pays an uncerain reurn in T periods would be priced. Consider an economy wih an asse ha pays a reurn r in each period and one longerm asse which can be sold only afer T periods a a price p T which is unknown oday. Assuming ha invesors behave raionally, i.e. hey maximize an ineremporal uiliy funcion subjec o consrains, he price of he long-erm asse can be found mos easily by using a Lagrange approach or by sraighforward insering. We assume ha an invesor maximizes her expeced uiliy E 0 T =0 u (c ) subjec o he consrains c 0 + m 0 p 0 + a 0 = w 0 ; c + a = (1 + r)a 1 + w ; 1 T 1; c T = m 0 p T + (1 + r) a T 1 + w T : In period zero, he individual uses labour income w 0 o pay for consumpion goods c 0 ; o buy m 0 unis of he long-erm asse and for normal asses a 0. In periods one o T 1; he individual uses her asses a 1 plus reurns r on asses and her wage income w o

9.5. Solving by subsiuion 211 nance consumpion and again buy asses a (or keep hose from he previous period). In he nal period T; he long-erm asse has a price of p T and is sold. Wealh from he previous period plus ineres plus labour income w T are furher sources of income o pay for consumpion c T. Hence, he individual s conrol variables are consumpion levels c and he number m 0 of long-erm asses. This maximizaion problem can be solved mos easily by insering consumpion levels for each period ino he objecive funcion. The objecive funcion hen reads u (w 0 m 0 p 0 a 0 ) + E 0 T 1 =1 u (w + (1 + r)a 1 a ) + E 0 T u(m 0 p T + (1 + r)a T 1 + w T ) : Wha could now be called conrol variables are he wealh holdings a in periods = 0; :::; T 1 and (as in he original seup) he number of asses m 0 bough in period zero. Le us now look a he rs-order condiions. The rs-order condiion for wealh in period zero is u 0 (c 0 ) = (1 + r) E 0 u 0 (c 1 ) : (9.5.1) Wealh holdings in any period > 0 are opimally chosen according o E 0 u 0 (c ) = E 0 +1 (1 + r)u 0 (c +1 ), E 0 u 0 (c ) = (1 + r) E 0 u 0 (c +1 ) : (9.5.2) We can inser (9.5.2) ino he rs-period condiion (9.5.1) su cienly ofen and nd u 0 (c 0 ) = (1 + r) 2 2 E 0 u 0 (c 2 ) = ::: = (1 + r) T T E 0 u 0 (c T ) (9.5.3) The rs-order condiion for he number of asses is p 0 u 0 (c 0 ) = T E 0 u 0 (c T )p T : (9.5.4) When we inser combined rs-order condiions (9.5.3) for wealh holdings ino he rs-order condiion (9.5.4) for asses, we obain p 0 (1 + r) T T E 0 u 0 (c T ) = T E 0 u 0 (c T )p T, p 0 = (1 + r) T E 0 u 0 (c T ) E 0 u 0 (c T ) p T : which is an equaion where we see he analogy o he wo period example in ch. 8.3.1 nicely. Insead of p j0 = 1 E u0 (c 1 ) p 1+r Eu 0 (c 1 ) j1 in (8.3.1) where we discoun by one period only and evaluae reurns a expeced marginal uiliy in period one, we discoun by T periods and evaluae reurns a marginal uiliy in period T: This equaion also o ers a lesson for life when we assume risk-neuraliy for simpliciy: if he payo p T from a long-erm asse is no high enough such ha he curren price is higher han he presen value of he payo, p 0 > (1 + r) T E 0 p T, hen he long-erm asse is simply dominaed by shor-erm invesmens ha pay a reurn of r per period. Opimal behaviour would imply no buying he long-erm asse and jus puing wealh ino normal asses. This should be kep in mind he nex ime you alk o your insurance agen who ries o sell you life-insurance or privae pension plans. Jus ask for he presen value of he payo s and compare hem o he presen value of wha you pay ino he savings plan.

212 Chaper 9. Muli-period models 9.5.3 Sicky prices The seup Sicky prices are a fac of life. In macroeconomic models, hey are eiher assumed righ away, or assumed following Calvo price seing or resul from some adjusmen-cos seup. Here is a simpli ed way o derive sluggish price adjusmen based on adjusmen cos. The rm s objecive funcion is o maximize is presen value de ned by he sum of discouned expeced pro s, 1 V = E 1 = : 1 + r Pro s a a poin in ime are given by = p x w l (p ; p 1 ) where (p ; p 1 ) are price adjusmen coss. These are similar in spiri o he adjusmen coss presened in ch. 5.5.1. We will laer use a speci caion given by (p ; p 1 ) = 2 p p 1 : (9.5.5) 2 This speci caion capures he essenial mechanism ha is required o make prices sicky, here are increasing coss in he di erence p p 1 : The fac ha he price change is squared is no essenial - in fac, as wih all adjusmen cos mechanisms, any power larger han 1 would do he job. More care abou economic implicaions needs o be aken when a reasonable model is o be speci ed. The rm uses a echnology x = A l : We assume ha here is a cerain demand elasiciy " for he rm s oupu. This can re ec a monopolisic compeiion seup. The rm can choose is oupu x a each poin in ime freely by hiring he corresponding amoun of labour l : Labour produciviy A or oher quaniies can be uncerain. Solving by subsiuion p 1 Insering everyhing ino he objecive funcion yields 1 V = E 1 w = p x x (p ; p 1 ) 1 + r A w w +1 = E p x x (p ; p 1 ) + E p +1 x +1 x +1 (p +1 ; p ) A A +1 1 + E 1 w =+2 p x x (p ; p 1 ) : 1 + r A

9.5. Solving by subsiuion 213 The second and hird line presen a rewrien mehod which allows us o see he ineremporal srucure of he maximizaion problem beer. We now maximize his objecive funcion by choosing oupu x for oday. (Oupu levels in he fuure are chosen a a laer sage.) The rs-order condiion is d [p x ] E dx d [p x ] dx w d (p ; p 1 ) A dx E d (p +1 ; p ) dx = 0, = w A + d (p ; p 1 ) dx + E d (p +1 ; p ) dx : I has cerain well-known componens and some new ones. If here were no adjusmen coss, i.e. (:) = 0; he ineremporal problem would become a saic one and he usual condiion would equae marginal revenue d [p x ] =dx wih marginal cos w =A. Wih adjusmen coss, however, a change in oupu oday no only a ecs adjusmen coss oday d(p;p 1) d(p dx bu also (expeced) adjusmen coss E +1 ;p ) dx omorrow. As all variables wih index are assumed o be known in ; expecaions are formed only wih respec o adjusmen coss omorrow in + 1: Specifying he adjusmen cos funcion as in (9.5.5) and compuing marginal revenue using he demand elasiciy " gives " d [p x ] = w + d # " 2 # 2 p p 1 d p+1 p + E, dx A dx 2 p 1 dx 2 p dp x + p = 1 + " 1 p = w p p 1 1 dp p+1 p d p +1 + + E dx A p 1 p 1 dx p dx p = w p p 1 p + " 1 p+1 p p+1 dp E, A p 1 x p 1 p p 2 dx 1 + " 1 p = w p p 1 p + " 1 p+1 p p+1 p E " A p 1 x p 1 p p 2 1, x 1 + " 1 p = w p p 1 p + " 1 1 " 1 p+1 p E p +1 A x p 1 p x p p 1 where " dx p dp x is he demand elasiciy for he rm s good. Again, = 0 would give he sandard saic opimaliy condiion 1 + " 1 p = w =A where he price is a markup over marginal cos. Wih adjusmen coss, prices change only slowly. 9.5.4 Opimal employmen wih adjusmen coss The seup Consider a rm ha maximizes pro s as in ch. 5.5.1, = E 1 1 = (1 + r) : (9.5.6)

214 Chaper 9. Muli-period models We are oday in ; ime exends unil in niy and he ime beween oday and in niy is denoed by : There is no paricular reason why he planning horizon is in niy in conras o ch. 5.5.1. Here we will sress ha he opimaliy condiion for employmen is idenical o a nie horizon problem. Insananeous pro s are given by he di erence beween revenue in ; which is idenical o oupu F (L ) wih an oupu price normalized o uniy, labour cos w L and adjusmen cos (L L 1 ) ; = F (L ) w L (L L 1 ) : (9.5.7) Coss induced by he adjusmen of he number of employees beween he previous period and oday are capured by (:). Usually, one assumes coss boh from hiring and from ring individuals, i.e. boh for an increase in he labour force, L L 1 > 0; and from a decrease, L L 1 < 0: A simple funcional form for (:) which capures his idea is a quadraic form, i.e. (L L 1 ) = 2 (L L 1 ) 2, where is a consan. Uncerainy for a rm can come from many sources: Uncerain demand, uncerainy concerning he producion process, uncerainy over labour coss or oher sources. As we express pro s in unis of he oupu good, we assume ha he real wage w, i.e. he amoun of oupu goods o be paid o labour, is uncerain. Adjusmen cos (L L 1 ) are cerain, i.e. he rm knows how many unis of oupu pro s reduce by when employmen changes by L L 1 : As in saic models of he rm, he conrol variable of he rm is employmen L : In conras o saic models, however, employmen decisions oday in no only a ecs employmen oday bu also employmen omorrow as he employmen decision in a ecs adjusmen coss in + 1: There is herefore an ineremporal link he rm needs o ake ino accoun which is no presen in he rm s saic models. Solving by subsiuion This maximizaion problem can be solved direcly by insering (9.5.7) ino he objecive funcion (9.5.6). One can hen choose opimal employmen for some poin in ime s < 1 afer having spli he objecive funcion ino several subperiods - as for example in he previous chaper 9.5.1. The soluion reads (o be shown in exercise 7) F 0 (L ) = w + 0 0 (L +1 L ) (L L 1 ) E : 1 + r When employmen L is chosen in ; here is only uncerainy concerning L +1 : The curren wage w (and all oher deerminisic quaniies as well) are known wih cerainy. L +1 is uncerain, however, from he perspecive of oday as he wage in + 1 is unknown and L +1 will be have o be adjused accordingly in + 1: Hence, expecaions apply only o he adjusmen-cos erm which refers o adjusmen coss which occur in period + 1: Economically speaking, given employmen L 1 in he previous period, employmen in is chosen such ha marginal produciviy of labour equals labour coss adjused for curren

9.6. An explici ime pah for a boundary condiion 215 and expeced fuure adjusmen coss. Expeced fuure adjusmen coss are discouned by he ineres rae r o obain is presen value. When we specify he adjusmen cos funcion as a quadraic funcion, (L L 1 ) = 2 [L L 1 ] 2 ; we obain F 0 [L +1 L ] (L ) = w + [L L 1 ] E : 1 + r If here were no adjusmen coss, i.e. = 0; we would have F 0 (L ) = w : Employmen would be chosen such ha marginal produciviy equals he real wage. This con rms he iniial saemen ha he ineremporal problem of he rm arises purely from he adjusmen coss. Wihou adjusmen coss, i.e. wih = 0; he rm has he sandard insananeous, period-speci c opimaliy condiion. 9.6 An explici ime pah for a boundary condiion Someimes, an explici ime pah for opimal behaviour is required. The ransversaliy condiion is hen usually no very useful. A more pragmaic approach ses asses a some fuure poin in ime a some exogenous level. This allows us o hen (a leas numerically) compue he opimal pah for all poins in ime before his nal poin easily. Le T be he nal period of life in our model, i.e. se a T +1 = 0 (or some oher level for example he deerminisic seady sae level). Then, from he budge consrain, we can deduce consumpion in T; a T +1 = (1 + r T ) a T + w T c T, c T = (1 + r T ) a T + w T : Opimal consumpion in T o (9.1.9), i.e. 1 sill needs o obey he Euler equaion, compare for example u 0 (c T 1 ) = E T 1 [1 + r T ] u 0 (c T ) : As he budge consrain requires a T = (1 + r T 1 ) a T 1 + w T 1 c T 1 ; opimal consumpion in T 1 is deermined by u 0 (c T 1 ) = E T 1 [1 + r T ] u 0 ((1 + r T ) [(1 + r T 1 ) a T 1 + w T 1 c T 1 ] + w T ) This is one equaion in one unknown, c T 1, where expecaions need o be formed abou r T and w T and w T 1 are unknown. When we assume a probabiliy disribuion for r T and w T, we can replace E T 1 by a summaion over saes and solve his expression numerically in a sraighforward way.

216 Chaper 9. Muli-period models 9.7 Furher reading and exercises A recen inroducion and deailed analysis of discree ime models wih uncerainy in he real business cycle radiion wih homogeneous and heerogenous agens is by Heer and Mausner (2005). Sokey and Lucas ake a more rigorous approach o he one aken here (1989). An almos comprehensive in-deph presenaion of macroeconomic aspecs under uncerainy is provided by Ljungqvis and Sargen (2004). On capial asse pricing in one-secor economies, references include Jermann (1998), Danhine, Donaldson and Mehra (1992), Abel (1990), Rouwenhors (1995), Sokey and Lucas (1989, ch. 16.2) and Lucas (1978). An overview is in ch. 13 of Ljungqvis and Sargen (2004). The example for sicky prices is inspired by Ireland (2004), going back o Roemberg (1982). The saemen ha he predicions concerning real variables do no change when a numeraire good is no chosen is no as obvious as i migh appear from remembering Walras law from undergraduae micro courses. There is a lieraure ha analyses he e ecs of he choice of numeraire for real oucomes for he economy when here is imperfec compeiion. See e.g. Gabszewicz and Vial (1972) or Dierker and Grodahl (1995).

9.7. Furher reading and exercises 217 Exercises Chaper 9 Applied Ineremporal Opimizaion Discree ime in nie horizon models under uncerainy 1. Cenral planner Consider an economy where oupu is produced by Y = A K L 1 Again, as in he OLG example in equaion (8.1.1), oal facor produciviy A is sochasic. Le capial evolve according o The cenral planner maximizes K +1 = (1 ) K + Y C max E 1 = u (C ) fc g by again choosing a pah of aggregae consumpion ows C : A, all variables indexed are known. The only uncerainy concerns A +1 : Wha are he opimaliy condiions? 2. A household maximizaion problem Consider he opimal saving problem of he household in ch. 9.3. Derive he Euler equaion (9.3.6). 3. Endogenous labour supply Solve he endogenous labour supply seup in ch. 9.4 by using dynamic programming. 4. Closed-form soluion Solve his model for he uiliy funcion u (C) = C1 1 and for = 1: Solve i for 1 a more general case (Benhabib and Rusichini, 1994). 5. Habi formaion Assume insananeous uiliy depends, no only on curren consumpion, bu also on habis (see for example Abel, 1990). Le he uiliy funcion herefore look like U = E 1 = u (c ; v ) ;

218 Chaper 9. Muli-period models where v sands for habis like e.g. pas consumpion, v = v (c 1 ; c 2 ; :::) : Le such an individual maximize uiliy subjec o he budge consrain a +1 = (1 + r ) a + w p c (a) Assume he individual lives in a deerminisic world and derive a rule for an opimal consumpion pah where he e ec of habis are explicily aken ino accoun. Specify habis by v = c 1 : (b) Le here be uncerainy wih respec o fuure prices. A a poin in ime, all variables indexed by are known. Wha is he opimal consumpion rule when habis are reaed in a parameric way? (c) Choose a plausible insananeous uiliy funcion and discuss he implicaions for opimal consumpion given habis v = c 1. 6. Risk-neural valuaion Under which condiions is here a risk neural valuaion relaionship for coningen claims in models wih many periods? 7. Labour demand under adjusmen cos Solve he maximizaion problem of he rm in ch. 9.5.4 by direcly insering pro s (9.5.7) ino he objecive funcion (9.5.6) and hen choosing L. 8. Solving by subsiuion Solve he problem from ch. 9.5 in a slighly exended version, i.e. wih prices p. Maximize E 0 1 =0 u (c ) by choosing a ime pah fc g for consumpion subjec o a +1 = (1 + r ) a + w p c : 9. Maching on labour markes Le employmen L in a rm follow L +1 = (1 s) L + V ; where s is a consan separaion rae, is a consan maching rae and V denoes he number of jobs a rm currenly o ers. The rm s pro s in period are given by he di erence beween revenue p Y (L ) and coss, where coss sem from wage paymens and coss for vacancies V capured by a parameer ; The rm s objecive funcion is given by = p Y (L ) w L V : = E 1 = ; where is a discoun facor and E is he expecaions operaor.

9.7. Furher reading and exercises 219 (a) Assume a deerminisic world. Le he rm choose he number of vacancies opimally. Use a Lagrangian o derive he opimaliy condiion. Assume ha here is an inerior soluion. Why is his an assumpion ha migh no always be sais ed from he perspecive of a single rm? (b) Le us now assume ha here is uncerain demand which ranslaes ino uncerain prices p which are exogenous o he rm. Solve he opimal choice of he rm by insering all equaions ino he objecive funcion. Maximize by choosing he sae variable and explain also in words wha you do. Give an inerpreaion of he opimaliy condiion. Wha does i imply for he opimal choice of V? 10. Opimal raining for a marahon Imagine you wan o paricipae in a marahon or any oher spors even. I will ake place in m days, i.e. in + m where is oday. You know ha aking par in his even requires raining e ; 2 [; + m] : Unforunaely, you dislike raining, i.e. your insananeous uiliy u (e ) decreases in e or, u 0 (e ) < 0. On he oher hand, raining allows you o be successful in he marahon: more e or increases your personal ness F. Assume ha ness follows F +1 = (1 ) F + e, wih 0 < < 1; and ness a + m is good for you yielding happiness of h (F +m ) : (a) Formally formulae an objecive funcion which capures he rade-o s in such a raining program. (b) Assume ha everyhing is deerminisic. How would your raining schedule look (he opimal pah of e )? (c) In he real world, normal nigh life reduces ness in a random way, i.e. is sochasic. How does your raining schedule look now?

220 Chaper 9. Muli-period models

Par IV Sochasic models in coninuous ime 221

Par IV is he nal par of his book and, logically, analyzes coninuous ime models under uncerainy. The choice beween working in discree or coninuous ime is parly driven by previous choices: If he lieraure is mainly in discree ime, sudens will nd i helpful o work in discree ime as well. The use of discree ime mehods seem o hold for macroeconomics, a leas when i comes o he analysis of business cycles. On he oher hand, when we alk abou economic growh, labour marke analyses and nance, coninuous ime mehods are very prominen. Whaever he radiion in he lieraure, coninuous ime models have he huge advanage ha hey are analyically generally more racable, once some iniial invesmen ino new mehods has been digesed. As an example, some papers in he lieraure have shown ha coninuous ime models wih uncerainy can be analyzed in simple phase diagrams as in deerminisic coninuous ime seups. See ch. 10.6 and ch. 11.6 on furher reading for references from many elds. To faciliae access o he magical world of coninuous ime uncerainy, par IV presens he ools required o work wih uncerainy in coninuous ime models. I is probably he mos innovaive par of his book as many resuls from recen research ow direcly ino i. This par also mos srongly incorporaes he cenral philosophy behind wriing his book: There will be hardly any discussion of formal mahemaical aspecs like probabiliy spaces, measurabiliy and he like. While some will argue ha one can no work wih coninuous ime uncerainy wihou having sudied mahemaics, his chaper and he many applicaions in he lieraure prove he opposie. The objecive here is o clearly make he ools for coninuous ime uncerainy available in a language ha is accessible for anyone wih an ineres in hese ools and some feeling for dynamic models and random variables. The chapers on furher reading will provide links o he more mahemaical lieraure. Maybe his is also a good poin for he auhor of his book o hank all he mahemaicians who helped him gain access o his magical world. I hope hey will forgive me for beraying heir secres o hose who, maybe in heir view, were no appropriaely iniiaed. Chaper 10 provides he background for opimizaion problems. As in par II where we rs looked a di erenial equaions before working wih Hamilonians, here we will rs look a sochasic di erenial equaions. Afer some basics, he mos ineresing aspec of working wih uncerainy in coninuous ime follows: Io s lemma and, more generally, change-of-variable formulas for compuing di erenials will be presened. As an applicaion of Io s lemma, we will ge o know one of he mos famous resuls in Economics - he Black-Scholes formula. This chaper also presens mehods for how o solve sochasic di erenial equaions or how o verify soluions and compue momens of random variables described by a sochasic process. Chaper 11 hen looks once more a maximizaion problems. We will ge o know he classic ineremporal uiliy maximizaion problem boh for Poisson uncerainy and for Brownian moion. The chaper also shows he link beween Poisson processes and maching models of he labour marke. This is very useful for working wih exensions of he simple maching model ha allows for savings. Capial asse pricing and naural 223

224 volailiy conclude he chaper.

Chaper 10 SDEs, di erenials and momens When working in coninuous ime, uncerainy eners he economy usually in he form of Brownian moion, Poisson processes or Levy processes. This uncerainy is represened in economic models by sochasic di erenial equaions (SDEs) which describe for example he evoluion of prices or echnology froniers. This secion will cover a wide range of di erenial equaions (and show how o work wih hem) ha appear in economics and nance. I will also show how o work wih funcions of sochasic variables, for example how oupu evolves given ha TFP is sochasic or how wealh of a household grows over ime, given ha he price of he asse held by he household is random. The enire reamen here, as before in his book, will be non-rigorous and will focus on how o compue hings. 10.1 Sochasic di erenial equaions (SDEs) 10.1.1 Sochasic processes We go o know random variables in ch. 7.1. A random variable relaes in some loose sense o a sochasic process of how (deerminisic) saic models relae o (deerminisic) dynamic models: Saic models describe one equilibrium, dynamic models describe a sequence of equilibria. A random variable has, when looked a once (e.g. when hrowing a die once), one realizaion. A sochasic process describes a sequence of random variables and herefore, when looked a once, describes a sequence of realizaions. More formally, we have he following: De niion 10.1.1 (Ross, 1996) A sochasic process is a parameerized collecion of random variables fx ()g 2[0 ;T ] : Le us look a an example for a sochasic process. We sar from he normal disribuion of ch. 7.2.2 whose mean and variance are given by and 2 and is densiy funcion is 225

226 Chaper 10. SDEs, di erenials and momens p 1 1 f (z) = 2 2 e 2( z ) 2 : Now de ne a normally disribued random variable Z () ha has a variance ha is a funcion of some : insead of 2, wrie 2 : Hence, he p 1 2 1 z random variables we jus de ned have as densiy funcion f (z) = 22 e 2 p : By having done so and by inerpreing as ime, Z () is in fac a sochasic process: we have a collecion of random variables, all normally disribued, hey are parameerized by ime : Sochasic processes can be saionary, weakly saionary or non-saionary. Saionariy is a more resricive concep han weak saionariy. De niion 10.1.2 (Ross, 1996, ch. 8.8): A process X () is saionary if X ( 1 ) ; :::; X ( n ) and X ( 1 + s) ; :::; X ( n + s) have he same join disribuion for all n and s: An implicaion of his de niion, which migh help o ge some feeling for his de niion, is ha a saionary process X () implies ha, being in = 0, X ( 1 ) and X ( 2 ) have he same disribuion for all 2 > 1 > 0: A weaker concep of saionariy only requires he rs wo momens of X ( 1 ) and X ( 2 ) (and a condiion on he covariance) o be sais ed. De niion 10.1.3 (Ross, 1996) A process X () is weakly saionary if he rs wo momens are he same for all and he covariance beween X ( 2 ) and X ( 1 ) depends only on 2 1 ; E 0 X () = ; V arx () = 2 ; Cov (X ( 2 ) ; X ( 1 )) = f ( 2 1 ) ; where and 2 are consans and f (:) is some funcion. De niion 10.1.4 A process which is neiher saionary nor weakly saionary is nonsaionary. Probably he bes-known sochasic process in coninuous ime is he Brownian moion. I is someimes called he Wiener process afer he mahemaician Wiener who provided he following de niion. De niion 10.1.5 (Ross, 1996) Brownian moion A sochasic process z () is a Brownian moion process if (i) z (0) = 0; (ii) he process has saionary independen incremens and (iii) for every > 0; z () is normally disribued wih mean 0 and variance 2 : The rs condiion z (0) = 0 is a normalizaion. Any z () ha sars a, say z 0, can be rede ned as z () z 0 : The second condiion says ha for 4 > 3 2 > 1 he incremen z ( 4 ) z ( 3 ) ; which is a random variable, is independen of previous incremens, say z ( 2 ) z ( 1 ). Independen incremens implies ha Brownian moion is a Markov process. Assuming ha we are in 3 oday, he disribuion of z ( 4 ) depends only on

10.1. Sochasic di erenial equaions (SDEs) 227 z ( 3 ) ; i.e. on he curren sae, and no on previous saes like z ( 1 ). Incremens are said o be saionary if, according o he above de niion of saionariy, he sochasic process X () z () z ( s) where s is a consan, has he same disribuion for any. Finally, he hird condiion is he hear of he de niion - z () is normally disribued. The variance increases linearly in ime; he Wiener process is herefore non-saionary. Le us now de ne a sochasic process which plays also a major role in economics. De niion 10.1.6 Poisson process (adaped following Ross 1993, p. 210) A sochasic process q () is a Poisson process wih arrival rae if (i) q (0) = 0; (ii) he process has independen incremens and (iii) he incremen q () q () in any inerval of lengh (he number of jumps ) is Poisson disribued wih mean [ ] ; i.e. q () q () Poisson( [ ]) : A Poisson process (and oher relaed processes) are also someimes called couning processes as q () couns how ofen a jump has occurred, i.e. how ofen somehing has happened. There is a close similariy in he rs wo poins of his de niion wih he de niion of Brownian moion. The hird poin here means more precisely ha he probabiliy ha he process increases n imes beween and > is given by P [q () q () = n] = e [ ] ( [ ])n ; n = 0; 1; ::: (10.1.1) n! We know his probabiliy from he de niion of he Poisson disribuion in ch. 7.2.1. This is probably where he Poisson process go is name from. Hence, one could hink of as many sochasic processes as here are disribuions, de ning each process by he disribuion of is incremens. The mos common way o presen Poisson processes is by looking a he disribuion of he incremen q () q () over a very small ime inerval [; ] : The incremen q () q () for very close o is usually expressed by dq () : A sochasic process q () is hen a Poisson process if is incremen dq () is driven by dq () = n 0 wih prob. 1 d 1 wih prob. d ; (10.1.2) where he parameer is again called he arrival rae. A high hen means ha he process jumps on average more ofen han wih a low. While his presenaion is inuiive and widely used, one should noe ha he probabiliies given in (10.1.2) are an approximaion of he ones in (10.1.1) for = d, i.e. for very small ime inervals. We will reurn o his below in ch. 10.5.2, see Poisson process II. These sochasic processes (and oher processes) can now be combined in various ways o consruc more complex processes. These more complex processes can be well represened by sochasic di erenial equaions (SDEs).

228 Chaper 10. SDEs, di erenials and momens 10.1.2 Sochasic di erenial equaions The mos frequenly used SDEs include Brownian moion as he source of uncerainy. These SDEs are used o model for example he evoluion of asse prices or budge consrains of households. Oher examples include SDEs wih Poisson uncerainy used explicily in he naural volailiy lieraure, in nance, labour markes, inernaional macro or in oher conexs menioned above. Finally and more recenly, Levy processes are used in nance as hey allow for a much wider choice of properies of disribuions of asse reurns han, le us say, Brownian moion. We will now ge o know examples for each ype. For all Brownian moions ha will follow, we will assume, unless explicily saed oherwise, ha incremens have a sandard normal disribuion, i.e. E [z () z ()] = 0 and var [z () z ()] = : We will call his sandard Brownian moion. I is herefore su cien, consisen wih mos papers in he lieraure and many mahemaical exbooks, o work wih a normalizaion of in de niion 10.1.5 of Brownian moion o 1: Brownian moion wih drif This is one of he simples SDEs. I reads dx () = ad + bdz () : (10.1.3) The consan a can be called drif rae, b 2 is someimes referred o as he variance rae of x () : In fac, ch. 10.5.4 shows ha he expeced increase of x () is deermined by a only (and no by b). In conras, he variance of x () for some fuure > is only deermined by b. The drif rae a is muliplied by d; a shor ime inerval, he variance parameer b is muliplied by dz () ; he incremen of he Brownian moion process z () over a small ime inerval. This SDE (and all he ohers following laer) herefore consis of a deerminisic par (he d-erm) and a sochasic par (he dz-erm). An inuiion for his di erenial equaion can be mos easily gained by underaking a comparison wih a deerminisic di erenial equaion. If we negleced he Wiener process for a momen (se b = 0), divide by d and rename he variable as y; we obain he simple ordinary di erenial equaion _y () = a (10.1.4) whose soluion is y () = y 0 + a: When we draw his soluion and also he above SDE for hree di eren realizaions of z (), we obain he following gure.

10.1. Sochasic di erenial equaions (SDEs) 229 Figure 10.1.1 The soluion of he deerminisic di erenial equaion (10.1.4) and hree realizaions of he relaed sochasic di erenial equaion (10.1.3) Hence, inuiively speaking, adding a sochasic componen o he di erenial equaion leads o ucuaions around he deerminisic pah. Clearly, how much he soluion of he SDE di ers from he deerminisic one is random, i.e. unknown. Furher below in ch. 10.5.4, we will undersand ha he soluion of he deerminisic di erenial equaion (10.1.4) is idenical o he evoluion of he expeced value of x () ; i.e. y () = E 0 x () for > 0: Generalized Brownian moion (Io processes) A more general way o describe sochasic processes is he following SDE dx () = a (x () ; z () ; ) d + b (x () ; z () ; ) dz () : (10.1.5) Here, one also refers o a (:) as he drif rae and o b 2 (:) as he insananeous variance rae. Noe ha hese funcions can be sochasic hemselves. In addiion o argumens x () and ime, Brownian moion z () can be included in hese argumens. Thinking of (10.1.5) as a budge consrain of a household, an example could be ha wage income or he ineres rae depend on he curren realizaion of he economy s fundamenal source of uncerainy, which is z () : Sochasic di erenial equaions wih Poisson processes Di erenial equaions ha are driven by a Poisson process can, of course, also be consruced. A very simple example is dx () = ad + bdq () : (10.1.6) A realizaion of his pah for x (0) = x 0 is in he following gure and can be undersood very easily. As long as no jump occurs, i.e. as long as dq = 0; he variable x () follows dx () = ad which means linear growh, x () = x 0 + a: This is ploed as he hin line.

230 Chaper 10. SDEs, di erenials and momens When q jumps, i.e. dq = 1; x () increases by b : wriing dx () = ~x () x () ; where ~x () is he level of x immediaely afer he jump, and leing he jump be very fas such ha d = 0 during he jump, we have ~x () x () = b 1; where he 1 sems from dq () = 1: Hence, ~x () = x () + b: (10.1.7) Clearly, he poins in ime when a jump occurs are random. A ilde (~) will always denoe in wha (and in various papers in he lieraure) follows he value of a quaniy immediaely afer a jump. Figure 10.1.2 An example of a Poisson process wih drif (hick line) and a deerminisic di erenial equaion (hin line) In conras o Brownian moion, a Poisson process conribues o he increase of he variable of ineres: wihou he dq () erm (i.e. for b = 0), x () would follow he hin line. Wih occasional jumps, x () jumps faser. In he Brownian moion case of he gure before, realizaions of x () remained close o he deerminisic soluion. This is simply due o he fac ha he expeced incremen of Brownian moion is zero while he expeced incremen of a Poisson process is posiive. Noe ha in he more formal lieraure, he ilde is no used bu a di erence is made beween x () and x ( ) where sands for he poin in ime an insan before. (This is probably easy o undersand on an inuiive level, hinking abou i for oo long migh no be a good idea as ime is coninuous...) The process x () is a so called cádlág process. The expression cádlág is an acronym from he french coninu a droie, limies a gauche. Tha is, he pahs of x () are coninuous from he righ wih lef limis. This is capured in he above gure by he black dos (coninuous from he righ) and he whie circles (limis from he lef). Wih his noaion, one would express he change in x due o a jump by x () = x ( ) + b as he value of x o which b is added is he value of x before he jump. As he ilde-noaion urned ou o be relaively inuiive, we will follow i in wha follows.

10.1. Sochasic di erenial equaions (SDEs) 231 A geomeric Poisson process An furher example would be he geomeric Poisson process dx () = a (q () ; ) x () d + b (q () ; ) x () dq () : (10.1.8) Processes are usually called geomeric when hey describe he rae of change of some RV x () ; i.e. dx () =x () is no a funcion of x () : In his example, he deerminisic par shows ha x () grows a he rae of a (:) in a deerminisic way and jumps by b (:) percen, when q () jumps. Noe ha in conras o a Brownian moion SDE, a (:) here is no he average growh rae of x () (see below on expecaions). Geomeric Poisson processes as here are someimes used o describe he evoluion of asse prices in a simple way. There is some deerminisic growh componen a (:) and some sochasic componen b (:) : When he laer is posiive, his could re ec new echnologies in he economy. When he laer is negaive, his equaion could be used o model negaive shocks like oil-price shocks or naural disasers. Aggregae uncerainy and random jumps An ineresing exension of a Poisson di erenial equaion consiss in making he ampliude of he jump random. Taking a simple di erenial equaion wih Poisson uncerainy as saring poin, da () = ba () dq () ; where b is a consan, we can now assume ha b () is governed by some disribuion, i.e. da () = b () A () dq () ; where b () ; 2 : (10.1.9) Assume ha A () is oal facor produciviy in an economy. Then, A () does no change as long as dq () = 0: When q () jumps, A () changes by b () ; i.e. da () ~ A () A () = b () A () ; which we can rewrie as ~A () = (1 + b ()) A () ; 8 where q () jumps. This equaion says ha whenever a jump occurs, A () increases by b () percen, i.e. by he realizaion of he random variable b () : Obviously, he realizaion of b () maers only for poins in ime where q () jumps. Noe ha (10.1.9) is he sochasic di erenial equaion represenaion of he evoluion of he saes of he economy in he Pissarides-ype maching model of Shimer (2005), where aggregae uncerainy, here A () follows from a Poisson process. The presenaion in Shimer s paper is, A shock his he economy according o a Poisson process wih arrival rae, a which poin a new pair (p 0 ; s 0 ) is drawn from a sae dependen disribuion. (p. 34). Noe also ha using (10.1.9) and assuming large families such ha here is no uncerainy from labour income lef on he household level would allow o analyze he e ecs of saving and hereby capial accumulaion over he business cycle in a closedeconomy model wih risk-averse households. The background for he saving decision would be ch. 11.1.

232 Chaper 10. SDEs, di erenials and momens 10.1.3 The inegral represenaion of sochasic di erenial equaions Sochasic di erenial equaions as presened here can also be represened by inegral versions. This is idenical o he inegral represenaions for deerminisic di erenial equaions in ch. 4.3.3. The inegral represenaion will be used frequenly when compuing momens of x (). As an example, hink of he expeced value of x for some fuure poin in ime ; when expecaions are formed oday in ; i.e. informaion unil is available, E x () : See ch. 10.5.4 or 11.1.6. Brownian moion Consider a di erenial equaion as (10.1.5). I can more rigorously be represened by is inegral version, x () x () = Z a(x; s)ds + Z b(x; s)dz (s) : (10.1.10) This version is obain by rs rewriing (10.1.5) as dx (s) = a (x; s) ds +b (x; s) dz (s) ; i.e. by simply changing he ime index from o s (and dropping z (s) and wriing x insead of x (s) o shoren noaion). Applying hen he inegral R on boh sides gives (10.1.10). This implies, iner alia, a di ereniaion rule Z Z d a(x; s)ds + b(x; s)dz (s) = d [x () x ()] = dx () Poisson processes = a(x; )d + b(x; )dz () : Now consider a generalized version of he SDE in (10.1.6), wih again replacing by s, dx (s) = a (x (s) ; q (s)) ds +b (x (s) ; q (s)) dq (s) : The inegral represenaion reads, afer applying R o boh sides, x () x () = Z a (x (s) ; q (s)) ds + Z b (x (s) ; q (s)) dq (s) : This can be checked by compuing he di erenial wih respec o ime : 10.2 Di erenials of sochasic processes Possibly he mos imporan aspec when working wih sochasic processes in coninuous ime is ha rules for compuing di erenials of funcions of sochasic processes are di eren from sandard rules. These rules are provided by various forms of Io s Lemma or change of variable formulas (CVF). Io s Lemma is a rule of how o compue differenials when he basic source of uncerainy is Brownian moion. The CVF provides corresponding rules when uncerainy sems from Poisson processes or Levy processes.

10.2. Di erenials of sochasic processes 233 10.2.1 Why all his? Compuing di erenials of funcions of sochasic processes sounds prey absrac. Le us sar wih an example from deerminisic coninuous ime seups which gives an idea abou wha he economic background for such di erenials are. Imagine he capial sock of an economy follows K _ () = I () K () ; an ordinary differenial equaion (ODE) known from ch. 4. Assume furher ha oal facor produciviy grows a an exogenous rae of g; A _ () =A () = g: Le oupu be given by Y (A () ; K () ; L) and le us ask how oupu grows over ime. The reply would be provided by looking a he derivaive of Y (:) wih respec o ime, d d Y (A () ; K () ; L) = Y da () dk () dl A + Y K + Y L d d d : Alernaively, wrien as a di erenial, we would have dy (A () ; K () ; L) = Y A da () + Y K dk () + Y L dl: We can now inser equaions describing he evoluion of TFP and capial, da () and dk () ; and ake ino accoun ha employmen L is consan. This gives dy (A () ; K () ; L) = Y A ga () d + Y K [I () K ()] d + 0: Dividing by d would give a di erenial equaion ha describes he growh of Y; i.e. _ Y () : The objecive of he subsequen secions is o provide rules on how o compue differenials, of which dy (A () ; K () ; L) is an example, in seups where K () or A () are described by sochasic DEs and no ordinary DEs as jus used in his example. 10.2.2 Compuing di erenials for Brownian moion We will now provide various versions of Io s Lemma. For formal reamens, see he references in furher reading a he end of his chaper. One sochasic process Lemma 10.2.1 Consider a funcion F (; x) of he di usion process x 2 R ha is a leas wice di ereniable in x and once in. The di erenial df reads where (dx) 2 is compued by using df = F d + F x dx + 1 2 F xx(dx) 2 (10.2.1) dd = ddz = dzd = 0; dzdz = d: (10.2.2)

234 Chaper 10. SDEs, di erenials and momens Le us look a an example. Assume ha x () is described by a generalized Brownian moion as in (10.1.5). The square of dx is hen given by (dx) 2 = a 2 (:) (d) 2 + 2a (:) b (:) ddz + b 2 (:) (dz) 2 = b 2 (:) d; where he las equaliy uses he rules from (10.2.2). These rules can inuiively be undersood by hinking abou he lengh of a graph of a funcion, more precisely speaking abou he oal variaion. For di ereniable funcions, he variaion is nie. For Brownian moion, which is coninuous bu no di ereniable, he variaion goes o in niy. As here is a nie variaion for di ereniable funcions, he quadraic (co)variaions involving d in (10.2.2) are zero. The quadraic variaion of Brownian moion, however, canno be negleced and is given by d: For deails, see he furher-reading chaper 10.6 on mahemaical background. The di erenial of F (; x) hen reads df = F d + F x a (:) d + F x b (:) dz + 1 2 F xxb 2 (:) d = F + F x a (:) + 1 2 F xxb 2 (:) d + F x b (:) dz: (10.2.3) When we compare his di erenial wih he normal one, we recognize familiar erms: The parial derivaives imes deerminisic changes, F + F x a (:) ; would appear also in circumsances where x follows a deerminisic evoluion. Pu di erenly, for b (:) = 0 in (10.1.5), he di erenial df reduces o ff + F x a (:)g d: Brownian moion herefore a ecs he di erenial df in wo ways: rs, he sochasic erm dz is added and second, maybe more surprisingly, he deerminisic par of df is also a eced hrough he quadraic erm conaining he second derivaive F xx : The lemma for many sochasic processes This was he simple case of one sochasic process. Now consider he case of many sochasic processes. Think of he price of many socks raded on he sock marke. We hen have he following Lemma 10.2.2 Consider he following se of sochasic di erenial equaions, In marix noaion, hey can be wrien as dx 1 = a 1 d + b 11 dz 1 + ::: + b 1m dz m ;. dx n = a n d + b n1 dz 1 + ::: + b nm dz m : dx = ad + bdz()

10.2. Di erenials of sochasic processes 235 where x = 0 B @ x 1. x n 1 C A ; a = 0 B @ a 1. a n 1 C A ; b = 0 B @ b 11 ::: b 1m.. b n1 ::: b nm 1 C A ; dz = 0 B @ dz 1. dz m Consider furher a funcion F (; x) from [0; 1[ R n o R wih ime and he n processes in x as argumens. Then 1 C A : df (; x) = F d + n i=1f xi dx i + 1 2 n i=1 n j=1f xi x j [dx i dx j ] (10.2.4) where, as an exension o (10.2.2), dd = ddz i = dz i d = 0 and dz i dz j = ij d: (10.2.5) When all z i are muually independen hen ij = 0 for i 6= j and ij = 1 for i = j: When wo Brownian moions z i and z j are correlaed, ij is he correlaion coe cien beween heir incremens dz i and dz j : An example wih wo sochasic processes Le us now consider an example for a funcion F (; x; y) of wo sochasic processes. As an example, assume ha x is described by a generalized Brownian moion similar o (10.1.5), dx = a (; x; y) d + b (; x; y) dz x and he sochasic process y is described by dy = c (; x; y) d + g (; x; y) dz y : Io s Lemma (10.2.4) gives he di erenial df as df = F d + F x dx + F y dy + 1 Fxx (dx) 2 + 2F xy dxdy + F yy (dy) 2 (10.2.6) 2 Given he rule in (10.2.5), he squares and he produc in (10.2.6) are (dx) 2 = b 2 (; x; y) d; (dy) 2 = g 2 (; x; y) d; dxdy = xy b (:) g (:) d; where xy is he correlaion coe cien of he wo processes. More precisely, i is he correlaion coe cien of he wo normally disribued random variables ha underlie he Wiener processes. The di erenial (10.2.6) herefore reads df = F d + a (:) F x d + b (:) F x dz x + c (:) F y d + g (:) F y dz y + 1 Fxx b 2 (:) d + 2 2 xy F xy b (:) g (:) d + F yy g 2 (:) d = F + a (:) F x + c (:) F y + 1 b 2 (:) F xx + 2 2 xy b (:) g (:) F xy + g 2 (:) F yy d + b (:) F x dz x + g (:) F y dz y (10.2.7) Noe ha his di erenial is almos simply he sum of he di erenials of each sochasic process independenly. The only erm ha is added is he erm ha conains he correlaion coe cien. In oher words, if he wo sochasic processes were independen, he di erenial of a funcion of several sochasic processes equals he sum of he di erenial of each sochasic process individually.

236 Chaper 10. SDEs, di erenials and momens An example wih one sochasic process and many Brownian moions A second example sipulaes a sochasic process x () governed by dx = u 1 d + m i=1v i dz i : This corresponds o n = 1 in he lemma above. When we compue he square of dx; we obain (dx) 2 = (u 1 d) 2 + 2u 1 d [ m i=1v i dz i ] + ( m i=1v i dz i ) 2 = 0 + 0 + ( m i=1v i dz i ) 2 ; where he second equaliy uses (10.2.2). The di erenial of F (; x) herefore reads from (10.2.4) df (; x) = F d + F x [u 1 d + m i=1v i dz i ] + 1 2 F xx [ m i=1v i dz i ] 2 = ff + F x u 1 g d + 1 2 F xx [ m i=1v i dz i ] 2 + F x m i=1v i dz i : Compuing he [ m i=1v i dz i ] 2 erm requires o ake poenial correlaions ino accoun. For any wo uncorrelaed incremens dz i and dz j ; dz i dz j would from (10.2.5) be zero. When hey are correlaed, dz i dz j = ij d which includes he case of dz i dz i = d: 10.2.3 Compuing di erenials for Poisson processes When we consider he di erenial of a funcion of he variable ha is driven by he Poisson process, we need o ake he following CVFs ino consideraion. One sochasic process Lemma 10.2.3 Le here be a sochasic process x () driven by Poisson uncerainy q () described by he following sochasic di erenial equaion dx () = a (:) d + b (:) dq () : Consider he funcion F (; x) : The di erenial of his funcion is df (; x) = F d + F x a (:) d + ff (; x + b (:)) F (; x)g dq: (10.2.8) Wha was sressed before for Brownian moion is valid here as well: he funcions a (:) and b (:) in he deerminisic and sochasic par of his SDE can have as argumens any combinaions of q () ; x () and or can be simple consans. The rule in (10.2.8) is very inuiive: he di erenial of a funcion is given by he normal erms and by a jump erm. The normal erms include he parial derivaives wih respec o ime and x imes changes per uni of ime (1 for he rs argumen and a (:) for x) imes d. Whenever he process q increases, x increases by he b (:) : The jump erm herefore capures ha he funcion F (:) jumps from F (; x) o F (; ~x) = F (; x + b (:)).

10.2. Di erenials of sochasic processes 237 Two sochasic processes Lemma 10.2.4 Le here be wo independen Poisson processes q x and q y sochasic processes x () and y () ; driving wo dx = a (:) d + b (:) dq x ; dy = c (:) d + g (:) dq y and consider he funcion F (x; y) : The di erenial of his funcion is df (x; y) = ff x a (:) + F y c (:)g d + ff (x + b (:) ; y) F (x; y)g dq x + ff (x; y + g (:)) F (x; y)g dq y : (10.2.9) Again, his di ereniaion rule consiss of he normal erms and he jump erms. As he funcion F (:) depends on wo argumens, he normal erm conains wo drif componens, F x a (:) and F y c (:) and he jump erm conains he e ec of jumps in q x and in q y : Noe ha he d erm does no conain he ime derivaive F (x; y) as in his example, F (x; y) is assumed no o be a funcion of ime and herefore F (x; y) = 0: In applicaions where F (:) is a funcion of ime, he F (:) would, of course, have o be aken ino consideraion. Basically, (10.2.9) is jus he sum of wo versions of (10.2.8). There is no addiional erm as he correlaion erm in he case of Brownian moion in (10.2.7). This is due o he fac ha any wo Poisson processes are, by consrucion, independen. Le us now consider a case ha is frequenly encounered in economic models when here is one economy-wide source of uncerainy, say new echnologies arrive or commodiy price shocks occur according o some Poisson process, and many variables in his economy (e.g. all relaive prices) are a eced simulaneously by his one shock. The CVF in siuaions of his ype reads Lemma 10.2.5 Le here be wo variables x and y following dx = a (:) d + b (:) dq; dy = c (:) d + g (:) dq; where uncerainy sems from he same q for boh variables. F (x; y) : The di erenial of his funcion is df (x; y) = ff x a (:) + F y c (:)g d + ff (x + b (:) ; y + g (:)) Consider he funcion F (x; y)g dq: One nice feaure abou di ereniaion rules for Poisson processes is heir very inuiive srucure. When here are wo independen Poisson processes as in (10.2.9), he change in F is given by eiher F (x + b (:) ; y) F (x; y) or by F (x; y + g (:)) F (x; y) ; depending on wheher one or he oher Poisson process jumps. When boh argumens x and y are a eced by he same Poisson process, he change in F is given by F (x + b (:) ; y + g (:)) F (x; y) ; i.e. he level of F afer a simulaneous change of boh x and y minus he pre-jump level F (x; y).

238 Chaper 10. SDEs, di erenials and momens Many sochasic processes We now presen he mos general case. Le here be n sochasic processes x i () and de ne he vecor x () = (x 1 () ; :::; x n ()) T : Le sochasic processes be described by n SDEs dx i () = i (:) d + i1 (:) dq 1 + ::: + im (:) dq m ; i = 1; : : : ; n; (10.2.10) where ij (:) sands for ij (; x ()) : Each sochasic process x i () is driven by he same m Poisson processes. The impac of Poisson process q j on x i () is capured by ij (:) : Noe he similariy o he seup for he Brownian moion case in (10.2.4). Proposiion 10.2.1 Le here be n sochasic processes described by (10.2.10). For a once coninuously di ereniable funcion F (; x), he process F (; x) obeys df (; x ()) = ff (:) + n i=1f xi (:) i (:)g d + m j=1 F ; x () + j (:) F (; x ()) dq j, (10.2.11) where F and F xi, i = 1; : : : ; n, denoe he parial derivaives of f wih respec o and x i, respecively, and j sands for he n-dimensional vecor funcion 1j ; : : : ; nj T. The inuiive undersanding is again simpli ed by focusing on normal coninuous erms and on jump erms. The coninuous erms are as before and simply describe he impac of he i (:) in (10.2.10) on F (:) : The jump erms show how F (:) changes from F (; x ()) o F ; x () + j (:) when Poisson process j jumps. The argumen x ()+ j (:) afer he jump of q j is obained by adding ij o componen x i in x; i.e. x () + j (:) = x 1 + 1j ; x 2 + 2j ; :::; x n + nj : 10.2.4 Brownian moion and a Poisson process There are much more general sochasic processes in he lieraure han jus Brownian moion or Poisson processes. This secion provides a CVF for a funcion of a variable which is driven by boh Brownian moion and a Poisson process. More general processes han jus addiive combinaions are so-called Levy processes, which will be analyzed in fuure ediions of hese noes. Lemma 10.2.6 Le here be a variable x which is described by dx = a (:) d + b (:) dz + g (:) dq (10.2.12) and where uncerainy sems from Brownian moion z and a Poisson process q. Consider he funcion F (; x) : The di erenial of his funcion is df (; x) = F + F x a (:) + 1 2 F xxb 2 (:) d + F x b (:) dz + ff (; x + g (:)) F (; x)g dq: (10.2.13) Noe ha his lemma is jus a combinaion of Io s Lemma (10.2.3) and he CVF for a Poisson process from (10.2.8). For an arrival rae of zero, i.e. for dq = 0 a all imes, (10.2.13) is idenical o (10.2.3). For b (:) = 0; (10.2.13) is idenical o (10.2.8).

10.3. Applicaions 239 10.3 Applicaions 10.3.1 Opion pricing One of he mos celebraed papers in economics is he paper by Black and Scholes (1973) in which hey derived a pricing formula for opions. This secion presens he rs seps owards obaining his pricing equaion. The subsequen chaper 10.4.1 will complee he analysis. This secion presens a simpli ed version (by neglecing jumps in he asse price) of he derivaion of Meron (1976). The basic quesion is: wha is he price of an opion on an asse if here is absence of arbirage on capial markes? The asse and opion price The saring poin is he price S of an asse which evolves according o a geomeric process ds = d + dz: (10.3.1) S Uncerainy is modelled by he incremen dz of Brownian moion. We assume ha he economic environmen is such (iner alia shor selling is possible, here are no ransacion coss) ha he price of he opion is given by a funcion F (:) having as argumens only he price of he asse and ime, F (; S ()): The di erenial of he price of he opion is hen given from (10.2.1) by df = F d + F S ds + 1 2 F SS [ds] 2 : (10.3.2) As by (10.2.2) he square of ds is given by (ds) 2 = 2 S 2 d; he di erenial reads df = F + SF S + 12 2 S 2 F SS d + SF S dz, df F F d + F dz (10.3.3) where he las sep de ned Absence of arbirage F = F + SF S + 1 2 2 S 2 F SS ; F = SF S F F : (10.3.4) Now comes he rick - he no-arbirage consideraion. Consider a porfolio ha consiss of N 1 unis of he asse iself, N 2 opions and N 3 unis of some riskless asses, say wealh in a savings accoun. The price of such a porfolio is hen given by P = N 1 S + N 2 F + N 3 M;

240 Chaper 10. SDEs, di erenials and momens where M is he price of one uni of he riskless asse. The proporional change of he price of his porfolio can be expressed as (holding he N i s consan, oherwise Io s Lemma would have o be used) dp = N 1 ds + N 2 df + N 3 dm, dp P = N 1S P ds S + N 2F df P F + N 3M dm P M : De ning shares of he porfolio held in hese hree asses by 1 N 1 S=P and 2 N 2 F=P; insering opion and sock price evoluions from (10.3.1) and (10.3.2) and leing he riskless asse M pay a consan reurn of r, we obain dp=p 1 d + 1 dz + 2 F d + 2 F dz + (1 1 2 ) rd = f 1 [ r] + 2 [ F r] + rg d + f 1 + 2 F g dz: (10.3.5) Now assume someone chooses weighs such ha he porfolio no longer bears any risk 1 + 2 F = 0: (10.3.6) The reurn of such a porfolio wih hese weighs mus hen of course be idenical o he reurn of he riskless ineres asse, i.e. idenical o r; dp=d P = 1 [ r] + 2 [ F r] + rj 1 = 2 F = r, r = F r : riskless F If he reurn of he riskless porfolio did no equal he reurn of he riskless ineres raes, here would be arbirage possibiliies. This approach is herefore called no-arbirage pricing. The Black-Scholes formula Finally, insering F and F from (10.3.4) yields he celebraed di erenial equaion ha deermines he evoluion of he price of he opion, r = F + SF S + 1 2 2 S 2 F SS rf SF S, 1 2 2 S 2 F SS + rsf S rf + F = 0: (10.3.7) Clearly, his equaion does no o say wha he price F of he opion acually is. I only says how i changes over ime and in reacion o S. Bu as we will see in ch. 10.4.1, his equaion can acually be solved explicily for he price of he opion. Noe also ha we did no make any assumpion so far abou wha ype of opion we are alking abou. 10.3.2 Deriving a budge consrain Mos maximizaion problems require a consrain. For a household, his is usually he budge consrain. I is shown here how he srucure of he budge consrain depends

10.3. Applicaions 241 on he economic environmen he household nds iself in and how he CVF needs o be applied here. Le wealh a ime be given by he number n () of socks a household owns imes heir price v (), a () = n () v (). Le he price follow a process ha is exogenous o he household (bu poenially endogenous in general equilibrium), dv () = v () d + v () dq () ; (10.3.8) where and are consans. For we require > 1 o avoid ha he price can become zero or negaive. Hence, he price grows wih he coninuous rae and a discree random imes i jumps by percen. The random imes are modeled by he jump imes of a Poisson process q () wih arrival rae, which is he probabiliy ha in he curren period a price jump occurs. The expeced (or average) growh rae is hen given by + (see ch. 10.5.4). Le he household earn dividend paymens, () per uni of asse i owns, and labour income, w (). Assume furhermore ha i spends p () c () on consumpion, where c () denoes he consumpion quaniy and p () he price of one uni of he consumpion good. When buying socks is he only way of saving, he number of socks held by he household changes in a deerminisic way according o dn () = n () () + w () v () p () c () d: When savings n () () + w () p () c () are posiive, he number of socks held by he household increases by savings divided by he price of one sock. When savings are negaive, he number of socks decreases. The change in he household s wealh, i.e. he household s budge consrain, is hen given by applying he CVF o a () = n () v (). The appropriae CVF comes from (10.2.9) where only one of he wo di erenial equaions shows he incremen of he Poisson process explicily. Wih F (x; y) = xy, we obain n () () + w () p () c () da () = v () + n () v () d v () where he ineres-rae is de ned as + fn () [v () + v ()] n () v ()g dq () = fr () a () + w () p () c ()g d + a () dq () ; (10.3.9) r () () v () + : This is a very inuiive budge consrain: As long as he asse price does no jump, i.e., dq () = 0, he household s wealh increases by curren savings, r () a ()+w () p () c (), where he ineres rae, r (), consiss of dividend paymens in erms of he asse price plus he deerminisic growh rae of he asse price. If a price jump occurs, i.e., dq () = 1, wealh jumps, as he price, by percen, which is he sochasic par of he overall ineres-rae. Alogeher, he average ineres rae amouns o r () + (see ch. 10.5.4).

242 Chaper 10. SDEs, di erenials and momens 10.4 Solving sochasic di erenial equaions Jus as here are heorems on uniqueness and exisence of soluions for ordinary di erenial equaions, here are heorems for SDEs on hese issues. There are also soluion mehods for SDEs. Here, we will consider some examples for soluions of SDEs. Jus as for ordinary deerminisic di erenial equaions in ch. 4.3.2, we will simply presen soluions and no show how hey can be derived. Soluions of sochasic di erenial equaions d (x ()) are, in analogy o he de niion for ODE, again ime pahs x () ha saisfy he di erenial equaion. Hence, by applying Io s Lemma or he CVF, one can verify wheher he soluions presened here are indeed soluions. 10.4.1 Some examples for Brownian moion This secion rs looks a SDEs wih Brownian moion which are similar o he ones ha were presened when inroducing SDEs in ch. 10.1.2: We sar wih Brownian moion wih drif as in (10.1.3) and hen look a an example for generalized Brownian moion in (10.1.5). In boh cases, we work wih SDEs which have an economic inerpreaion and are no jus SDEs. Finally, we complee he analysis of he Black-Scholes opion pricing approach. Brownian moion wih drif 1 As an example for Brownian moion wih drif, consider a represenaion of a producion echnology which could be called a di erenial-represenaion for he echnology. This ype of presening echnologies was dominan in early conribuions ha used coninuous ime mehods under uncerainy bu is someimes sill used oday. A simple example is dy () = AKd + Kdz () ; (10.4.1) where Y () is oupu in ; A is a (consan) measure of oal facor produciviy, K is capial, is some variance measure of oupu and z is Brownian moion. The change of oupu a each insan is hen given by dy (). See furher reading on references o he lieraure. Wha does such a represenaion of oupu imply? To see his, look a (10.4.1) as Brownian moion wih drif, i.e. consider A; K; and o be a consan. The soluion o his di erenial equaion saring in = 0 wih Y 0 and z (0) is Y () = Y 0 + AK + K [z () z (0)] : To simplify an economic inerpreaion se Y 0 = z (0) = 0: Oupu is hen given by Y () = (A + z ()) K: This says ha wih a consan facor inpu K; oupu in is deermined by a deerminisic and a sochasic par. The deerminisic par A implies linear (i.e. no exponenial as is usually assumed) growh, he sochasic par z () implies deviaions from he rend. As z () is Brownian moion, he sum of he deerminisic and sochasic par can become negaive. This is an undesirable propery of his approach.

10.4. Solving sochasic di erenial equaions 243 To see ha Y () is in fac a soluion of he above di erenial-represenaion, jus apply Io s Lemma and recover (10.4.1). Brownian moion wih drif 2 As a second example and in an aemp o beer undersand why oupu can become negaive, consider a sandard represenaion of a echnology Y () = A () K and le TFP A follow Brownian moion wih drif, da () = gd + dz () ; where g and are consans. Wha does his alernaive speci caion imply? Solving he SDE yields A () = A 0 +g+z () (which can again be checked by applying Io s lemma). Oupu is herefore given by and can again become negaive. Y () = (A 0 + g + z ()) K = A 0 K + gk + Kz () Geomeric Brownian moion Le us now assume ha TFP follows geomeric Brownian moion process, da () =A () = gd + dz () ; (10.4.2) where again g and are consans. Le oupu coninue o be given by Y () = A () K: The soluion for TFP, provided an iniial condiion A (0) = A 0, is given by A () = A 0 e (g 1 2 2 )+z() : (10.4.3) A any poin in ime ; he TFP level depends on ime and he curren level of he sochasic process z () : This shows ha TFP a each poin in ime is random and hereby unknown from he perspecive of = 0: Hence, a SDE and is soluion describe he deerminisic evoluion of a disribuion over ime. One could herefore plo a picure of A () which in principle would look like he evoluion of he disribuion in ch. 7.4.1. Ineresingly, and his is due o he geomeric speci caion in (10.4.2) and imporan for represening echnologies in general, TFP can no become negaive. While Brownian moion z () can ake any value beween minus and plus in niy, he erm e (g 1 2 2 )+z() is always posiive. Wih an AK speci caion for oupu, oupu is always posiive, Y () = A 0 e (g 1 2 2 )+z() K. In fac, i can be shown ha oupu and TFP are lognormally disribued. Hence, he speci caion of TFP wih geomeric Brownian moion provides an alernaive o he di erenial-represenaion in (10.4.1) which avoids he possibiliy of negaive oupu. The level of TFP a some fuure poin in ime is deermined by a deerminisic par, 1 g 2 ; and by a sochasic par, z () : Apparenly, he sochasic naure of TFP 2

244 Chaper 10. SDEs, di erenials and momens has an e ec on he deerminisic erm. The srucure (he facor 1=2 and he quadraic erm 2 ) reminds of he role he sochasic disurbance plays in Io s lemma. There as well (see e.g. (10.2.3)), he sochasic disurbance a ecs he deerminisic componen of he di erenial. As we will see laer, however, his does no a ec expeced growh. In fac, (10.5.2) will show ha expeced oupu grows a he rae g (and is hereby independen of he variance parameer ). One can verify ha (10.4.3) is a soluion o (10.4.2) by using Io s lemma. To do so, we need o bring (10.4.2) ino a form which allows us o apply he formulas which we go 1 o know in ch. 10.2.2. De ne x () g 2 + z () and A () F (x ()) = A 2 0 e x() : 1 As a consequence, he di erenial for x () is a nice SDE, dx () = g 2 d + dz () : 2 As his SDE is of he form as in (10.1.5), we can use Io s lemma from (10.2.3) and nd da () = df (x ()) = F x (x ()) g + 12 F xx (x ()) 2 d + F x (x ()) dz: 1 2 2 Insering he rs and second derivaives of F (x ()) yields da () = A 0 e x() 1 g 2 2 + 12 A 0e x() 2 d + A 0 e x() dz, 1 da () =A () = g 2 2 + 1 2 2 d + dz = gd + dz; where he i reinsered A () = A 0 e x() and divided by A () : As A () sais es he original SDE (10.4.2), A () is a soluion of (10.4.2). Opion pricing Le us come back o he Black-Scholes formula for opion pricing. The SDE derived above in (10.3.7) describes he evoluion of he price F (; S ()) ; where is ime and S () he price of he underlying asse a. We now look a a European call opion, i.e. an opion which gives he righ o buy an asse a some xed poin in ime T; he mauriy dae of he opion. The xed exercise or srike price of he opion, i.e. he price a which he asse can be bough is denoed by P: Clearly, a any poin in ime when he price of he asse is zero, he value of he opion is zero as well. This is he rs boundary condiion for our parial di erenial equaion (10.3.7). When he opion can be exercised a T and he price of he asse is S; he value of he opion is zero if he exercise price P exceeds he price of he asse and S P if no. This is he second boundary condiion. F (; 0) = 0; F (T; S) = max f0; S P g ; In he laer case where S P > 0, he owner of he opion would in fac buy he asse. Given hese wo boundary condiions, he parial di erenial equaion has he soluion F (; S ()) = S () (d 1 ) P e r[t ] (d 2 ) (10.4.4)

10.4. Solving sochasic di erenial equaions 245 where (y) = p 1 Z y e u2 2 du; 2 1 ln r S() + r2 (T ) P 2 d 1 = p ; d 2 = d 1 p T : T The expression F (; S ()) gives he price of an opion a a poin in ime T where he price of he asse is S () : I is a funcion of he cumulaive sandard normal disribuion (y) : For any pah of S () ; ime up o mauriy T a ecs he opion price hrough d 1 ; d 2 and direcly in he second erm of he above di erence. More inerpreaion is o ered by many nance exbooks. 10.4.2 A general soluion for Brownian moions Consider he linear sochasic di erenial equaion for x () ; dx () = fa () x () + b ()g d + m i=1 fc i () x () + g i ()g dz i () (10.4.5) where a () ; b () ; c i () and g i () are funcions of ime and z i () are Brownian moions. The correlaion coe ciens of is incremen wih he incremens of z j () are ij : Le here be a boundary condiion x (0) = x 0. The soluion o (10.4.5) is x () = e y() ' () (10.4.6) where y () = Z 0 ' () = x 0 + a (u) Z 0 1 2 (u) du + m i=1 Z e y(s) fb (s) (s)g ds + m i=1 (s) = m i=1 m j=1 ij c i (s) c j (s) ; (s) = m i=1 m j=1 ij c i (s) g i (s) : 0 c i (u) dz i (u) ; Z 0 e y(s) g i (s) dz i (s) ; (10.4.7a) (10.4.7b) (10.4.7c) (10.4.7d) To obain some inuiion for (10.4.6), we can rs consider he case of cerainy. For c i () = g i () = 0; (10.4.5) is a linear ODE and he soluion is x () = e R h 0 a(u)du x 0 + R i 0 a(u)du b (s) ds. This corresponds o he resuls we know from 0 e R s ch. 4.3.2, see (4.3.7). For he general case, we now prove ha (10.4.6) indeed sais es (10.4.5). In order o use Io s Lemma, wrie he claim (10.4.6) as x() = e y() '() f(y(); ' ()) (10.4.8)

246 Chaper 10. SDEs, di erenials and momens where dy() = a () 1 2 () d + m i=1c i () dz i () ; (10.4.9a) d' () = e y() fb () ()g d + m i=1e y() g i () dz i () : (10.4.9b) are he di erenials of (10.4.7a) and (10.4.7b). In order o compue he di erenial of x() in (10.4.8), we have o apply he mulidimensional Io-Formula (10.2.6) where ime is no an argumen of f(:): This gives dx() = e y() ' () dy() + e y() d' () + 1 2 e y() ' () [dy] 2 + 2e y() [dydz] + 0 [dz] 2 : (10.4.10) As [dy] 2 = [ m i=1c i () dz i ()] 2 by (10.2.5) - all erms muliplied by d equal zero - we obain Furher, again by (10.2.5), [dy] 2 = m i=1 m j=1 ij c i () c j () d: (10.4.11) dydz = ( m i=1c i () dz i ()) m i=1e y() g i () dz i () = e y() m i=1 m j=1 ij c i () g i () d: (10.4.12) Hence, reinsering (10.4.6), (10.4.9), (10.4.11) and (10.4.12) in (10.4.10) gives Rearranging gives dx() = x() a () 1 2 () d + m i=1c i () dz i () + fb () ()g d + m i=1g i () dz i () + 1 x() m 2 i=1 m j=1 ij c i () c j () d + 2 m i=1 m j=1 ij c i () g i () d dx() = fx()a () + b ()g d + m i=1 fx()c i () + g i ()g dz i () 1 x() () + () d 2 1 + 2 x()m i=1 m j=1 ij c i () c j () + m i=1 m j=1 ij c i () g i () d = fx()a () + b ()g d + m i=1 fx()c i () + g i ()g dz i () ; where he las equaliy sign used (10.4.7c) and (10.4.7d). This is he original SDE in (10.4.5) which shows ha he claim (10.4.6) is indeed a soluion of (10.4.5).

10.4. Solving sochasic di erenial equaions 247 10.4.3 Di erenial equaions wih Poisson processes The presenaion of soluions and heir veri caion for Poisson processes follows a similar srucure as for Brownian moion. We sar here wih a geomeric Poisson process and compare properies of he soluion wih he TFP Brownian moion case. We hen look a a more general process - he descripion of a budge consrain - which exends he geomeric Poisson process. One should keep in mind, as sressed already in ch. 4.3.3, ha soluions o di erenial equaions are di eren from he inegral represenaion of e.g. ch. 10.1.3. A geomeric Poisson process Imagine ha TFP follows a deerminisic rend and occasionally makes a discree jump. This is capured by a geomeric descripion as in (10.4.2), only ha Brownian moion is replaced by a Poisson process, Again, g and are consan wih > 1. The soluion o his SDE is given by da () =A () = gd + dq () : (10.4.13) A () = A 0 e g+[q() q(0)] ln(1+) : (10.4.14) Uncerainy does no a ec he deerminisic par here, in conras o he soluion (10.4.3) for he Brownian moion case. As before, TFP follows a deerminisic growh componen and a sochasic componen, [q () q (0)] ln (1 + ). The laer makes fuure TFP uncerain from he perspecive of oday. The claim ha A () is a soluion can be proven by applying he appropriae CVF. This will be done for he nex, more general, example. A budge consrain As a second example, we look a a dynamic budge consrain, De ning (s) w (s) a (0) = a 0 reads where y () is da () = fr () a () + w () c ()g d + a () dq: (10.4.15) y () = c (s) ; he backward soluion of (10.4.15) wih iniial condiion a () = e y() a 0 + Z 0 Z 0 e y(s) (s) ds (10.4.16) r (u) du + [q () q (0)] ln (1 + ) : (10.4.17) Noe ha he soluion in (10.4.16) has he same srucure as he soluion o a deerminisic version of he di erenial equaion (10.4.15) (which we would obain for = 0). In

248 Chaper 10. SDEs, di erenials and momens fac, he srucure of (10.4.16) is idenical o he srucure of (4.3.7). Pu di erenly, he sochasic componen in (10.4.15) only a ecs he discoun facor y () : This is no surprising - in a way - as uncerainy is proporional o a () and he facor can be seen as he sochasic componen of he ineres paymens on a () : We have jus saed ha (10.4.16) is a soluion. This should herefore be veri ed. To his end, de ne z() a 0 + Z 0 e y(s) (s)ds; (10.4.18) and wrie he soluion (10.4.16) as a() = e y() z() where from (10.4.17) and (10.4.18) and Leibniz rule (4.3.3), dy() = r()d + ln(1 + )dq(); dz() = e y() ()d: (10.4.19) We have hereby de ned a funcion a () = F (y () ; z ()) where he SDEs describing he evoluion of y () and z () are given in (10.4.19). This allows us o use he CVF (10.2.9) which hen says df = F y r () d + F z e y() () d + ff (y + ln (1 + ) ; z) F (y; z)g dq, da() = e y() z()r()d + e y() e y() ()d + e y()+ln(1+) z() e y() z() dq = fr () a () + ()g d + a () dq: This is he original di erenial equaion. Hence, a () in (10.4.16) is a soluion for (10.4.15). The ineremporal budge consrain In sochasic worlds, here is also a link beween dynamic and ineremporal budge consrains, jus as in deerminisic seups as in ch. 4.4.2. We can now use he soluion (10.4.16) o obain an ineremporal budge consrain. We rs presen here a budge consrain for a nie planning horizon and hen generalize he resul. For he nie horizon case, we can rewrie (10.4.16) as Z 0 e y(s) c (s) ds + e y() a () = a 0 + Z 0 e y(s) w (s) ds: (10.4.20a) This formulaion suggess a sandard economic inerpreaion. Toal expendiure over he planning horizon from 0 o on he lef-hand side mus equal oal wealh on he righhand side. Toal expendiure consiss of he presen value of he consumpion expendiure pah c (s) and he presen value of asses a () he household wans o hold a he end of he planning period, i.e. a : Toal wealh is given by iniial nancial wealh a 0 and he presen value of curren and fuure wage income w (s) : All discouning akes place a he realized sochasic ineres rae y (s) - no expecaions are formed. In order o obain an in nie horizon ineremporal budge consrain, he soluion (10.4.16) should be wrien more generally - afer replacing by and 0 by - as a () = e y() a + Z e y() y(s) (w (s) c (s)) ds (10.4.21)

10.5. Expecaion values 249 where y() is y () = Z Muliplying (10.4.21) by e y() and rearranging gives a () e y() + r (u) du + ln (1 + ) [q () q ()] : (10.4.22) Z e y(s) (c (s) w (s))ds = a(): Leing go o in niy, assuming a no-ponzi game condiion lim!1 a () e y() = 0 and rearranging again yields Z 1 e y(s) c (s) ds = a() + Z 1 e y(s) w (s) ds: The srucure here and in (10.4.20a) is close o he one in he deerminisic world. The presen value of consumpion expendiure needs o equal curren nancial wealh a () plus human wealh, i.e. he presen value of curren and fuure labour income. Here as well, discouning akes place a he risk-correced ineres rae as capured by y () in (10.4.22). Noe also ha his ineremporal budge consrain requires equaliy in realizaions, no in expeced erms. A swich process Here are wo funny processes which have an ineresing and simple soluion. Consider he iniial value problem, dx () = 2x () dq () ; x (0) = x 0 : The soluion is x () = ( 1) q() x 0 ; i.e. x () oscillaes beween x 0 and x 0 : Now consider he ransformaion y = y + x: I evolves according o dy = 2 (y y) dq; y 0 = y + x 0 : Is soluion is y () = y + ( 1) q() x 0 and y () oscillaes now beween y x 0 and y + x 0 : This could be nicely used for models wih wage uncerainy, where wages are someimes high and someimes low. An example would be a maching model where labour income is w (i.e. y + x 0 ) when employed and b (i.e. y x 0 ) when unemployed. The di erence beween labour income levels is hen 2x 0. The drawback is ha he probabiliy of nding a job is idenical o he probabiliy of losing i. 10.5 Expecaion values 10.5.1 The idea Wha is he idea behind expecaions of sochasic processes? When hinking of a sochasic process X (), eiher a simple one like Brownian moion or a Poisson process or

250 Chaper 10. SDEs, di erenials and momens more complex ones described by SDEs, i is useful o keep in mind ha one can hink of he value X () he sochasic process akes a some fuure poin in ime as a normal random variable. A each fuure poin in ime X () is characerized by some mean, some variance, by a range ec. This is illusraed in he following gure. Figure 10.5.1 The disribuion of X () a = :4 > = 0 The gure shows four realizaions of he sochasic process X () : The saring poin is oday in = 0 and he mean growh of his process is illusraed by he doed line. When we hink of possible realizaions of X () for = :4 from he perspecive of = 0; we can imagine a verical line ha cus hrough possible pahs a = :4: Wih su cienly many realizaions of hese pahs, we would be able o make inferences abou he disribuional properies of X (:4) : As he process used for his simulaion is a geomeric Brownian moion process as in (10.4.2), we would nd ha X (:4) is lognormally disribued as depiced above. Clearly, given ha we have a precise mahemaical descripion of our process, we do no need o esimae disribuional properies for X () ; as suggesed by his gure, bu we can explicily compue hem. Wha his secion herefore does is provide ools ha allow o deermine properies of he disribuion of a sochasic process for fuure poins in ime. Pu di erenly, undersanding a sochasic process means undersanding he evoluion of is disribuional properies. We will rs sar by looking a relaively sraighforward ways o compue means. Subsequenly, we provide some maringale resuls which allow us o hen undersand a more general approach o compuing means and also higher momens.

10.5. Expecaion values 251 10.5.2 Simple resuls Brownian moion I Consider a Brownian moion process Z () ; 0: Le Z (0) = 0. From he de niion of Brownian moion in ch. 10.1.1, we know ha Z () is normally disribued wih mean 0 and variance 2 : Le us assume we are in oday and Z () 6= 0. Wha is he mean and variance of Z () for >? We consruc a sochasic process Z () Z () which a = is equal o zero. Now imagining ha is equal o zero, Z () Z () is a Brownian moion process as de ned in ch. 10.1.1. Therefore, E [Z () Z ()] = 0 and var [Z () Z ()] = 2 [ ] : This says ha he incremens of Brownian moion are normally disribued wih mean zero and variance 2 [ ] : Hence, he reply o our quesion is E Z () = Z () ; var Z () = 2 [ ] : For our convenion 1 which we follow here as saed a he beginning of ch. 10.1.2, he variance would equal var Z () = : Brownian moion II Consider again he geomeric Brownian process da () =A () = gd+dz () describing he growh of TFP in (10.4.2). Given he soluion A () = A 0 e (g 1 2 2 )+z() from (10.4.3), we would now like o know wha he expeced TFP level A () in from he perspecive of = 0 is. To his end, apply he expecaions operaor E 0 and nd E 0 A () = A 0 E 0 e (g 1 2 2 )+z() = A 0 e (g 1 2 2 ) E 0 e z() ; (10.5.1) where he second equaliy exploied he fac ha e (g 1 2 2 ) is non-random. As z () is sandard normally disribued, z () N (0; ), z () is normally disribued wih N (0; 2 ) : As a consequence (compare ch. 7.3.3, especially eq. (7.3.4)), e z() is lognormally disribued wih mean e 1 2 2 : Hence, E 0 A () = A 0 e (g 1 2 2 ) e 1 2 2 = A 0 e g : (10.5.2) The expeced level of TFP growing according o a geomeric Brownian moion process grows a he drif rae g of he sochasic process. The variance parameer does no a ec expeced growh. Noe ha we can also deermine he variance of A () by simply applying he variance operaor o he soluion from (10.4.3), var 0 A () = var 0 A 0 e (g 1 2 2 ) e z() = A 2 0e 2(g 1 2 2 ) var 0 e z() :

252 Chaper 10. SDEs, di erenials and momens We now make similar argumens o before when we derived (10.5.2). As z () N (; 2 ) ; he erm e z() is lognormally disribued wih, see (7.3.4), variance e 2+2 e 2 1 : Hence, we nd var 0 A () = A 2 0e 2(g 1 2 2 ) e 2+2 = A 2 0e 2(+g) e 2 1 : e 2 1 = A 2 0e 2+(2(g 1 2 2 )+ 2 ) e 2 This clearly shows ha for any g > 0; he variance of A () increases over ime. Poisson processes I The expeced value and variance of a Poisson process q () wih arrival rae are E q () = q () + [ ] ; > (10.5.3) var q () = [ ] : As always, we compue momens from he perspecive of and > lies in he fuure. The expeced value E q () direcly follows from he de niion of a Poisson process in ch. 10.1.1, see (10.1.1). As he number of jumps is Poisson disribued wih mean [ ] and he variance of he Poisson disribuion is idenical o is mean, he variance also follows from he de niion of a Poisson process. Noe ha here are simple generalizaions of he Poisson process where he variance di ers from he mean (see ch. 10.5.4). Poisson processes II Poisson processes are widely used o undersand evens like nding a job, geing red, developing a new echnology, occurrence of an earhquake ec. One can model hese siuaions by a Poisson process q () wih arrival rae : In hese applicaions, he following quesions are ofen asked. Wha is he probabiliy ha q () will jump exacly once beween and? Given ha by def. 10.1.6, e [ ] ([ ]) n is he probabiliy ha q jumped n imes by ; i.e. q () = n; n! his probabiliy is given by P (q () = q () + 1) = e [ ] [ ] : Noe ha his is a funcion which is non-monoonic in ime : How can he expression P (q () = q () + 1) = e [ ] [ ] be reconciled wih he usual descripion of a Poisson process, as e.g. in (10.1.2), where i says ha he probabiliy of a jump is given by d? When we look a a small ime inerval d; we can neglec e d as i is very close o one and we ge P (q () q () = 1) = P (dq () = 1) = d: As over a very small insan d a Poisson process can eiher jump or no jump (i can no jump more han once in a small insan d), he probabiliy of no jump is herefore P (dq () = 0) = 1 d: 1

10.5. Expecaion values 253 Figure 10.5.2 The probabiliy of one jump by ime for a high arrival rae (peak a around 3) and a low arrival rae (peak a around 11) Wha is he lengh of ime beween wo jumps or evens? Clearly, we do no know as jumps are random. We can herefore only ask wha he disribuion of he lengh of ime beween wo jumps is. To undersand his, we sar from (10.1.1) which ells us ha he probabiliy ha here is no jump by is given by P [q () = q ()] = e [ ] : The probabiliy ha here is a leas one jump by is herefore P [q () > q ()] = 1 e [ ] : This erm 1 e [ ] is he cumulaive densiy funcion of an exponenial disribuion for a random variable (see ex. 2). Hence, he lengh beween wo jumps is exponenially disribued wih densiy e [ ] : Poisson process III Following he same srucure as wih Brownian moion, we now look a he geomeric Poisson process describing TFP growh (10.4.13). The soluion was given in (10.4.14) which reads, slighly generalized wih he perspecive of and iniial condiion A ; A () = A e g[ ]+[q() q()] ln(1+) : Wha is he expeced TFP level in? Applying he expecaion operaor gives E A () = A e g[ ] q() ln(1+) E e q() ln(1+) (10.5.4) where, as in (10.5.1), we spli he exponenial growh erm ino is deerminisic and sochasic par. To proceed, we need he following Lemma 10.5.1 (Posch and Wälde, 2006) Assume ha we are in and form expecaions abou fuure arrivals of he Poisson process. The expeced value of c kq(), condiional on where q () is known, is E (c kq() ) = c kq() e (ck 1)( ) ; > ; c; k = cons:

254 Chaper 10. SDEs, di erenials and momens Noe ha for ineger k, hese are he raw momens of c q() : Proof. We can rivially rewrie c kq() = c kq() c k[q() q()] : A ime ; we know he realizaion of q() and herefore E c kq() = c kq() E c k[q() q()] : Compuing his expecaion requires he probabiliy ha a Poisson process jumps n imes beween and : Formally, E c k[q() q()] = 1 n=0c kn e ( ) (( )) n = 1 e ( ) (c k ( )) n n=0 n! n! = e (ck 1)( ) 1 e ( ) (ck 1)( ) (c k ( )) n n=0 n! = e (ck 1)( ) 1 e ck ( ) (c k ( )) n n=0 = e (ck 1)( ) ; n! where e () n is he probabiliy of q () = n and 1 e ck ( ) (c k ( )) n n! n=0 = 1 denoes he n! summaion of he probabiliy funcion over he whole suppor of he Poisson disribuion which was used in he las sep. For a generalizaion of his lemma, see ex. 8. To apply his lemma o our case E e q() ln(1+) ; we se c = e and k = ln (1 + ) : Hence, E e q() ln(1+) = e ln(1+)q() e (eln(1+) 1)[ ] = e q() ln(1+) e [ ] q() ln(1+)+[ ] = e and, insered in (10.5.4), he expeced TFP level is E A () = A e g[ ] q() ln(1+) e q() ln(1+)+[ ] = A e (g+)[ ] : This is an inuiive resul: he growh rae of he expeced TFP level is driven boh by he deerminisic growh componen of he SDE (10.4.13) for TFP and by he sochasic par. The growh rae of expeced TFP is higher, he higher he deerminis par, he g; he more ofen he Poisson process jumps on average (a higher arrival rae ) and he higher he jump (he higher ). This con rms formally wha was already visible in and informally discussed afer g. 10.1.2 of ch. 10.1.2: A Poisson process as a source of uncerainy in a SDE implies ha average growh is no jus deermined by he deerminisic par of he SDE (as is he case when Brownian moion consiues he disurbance erm) bu also by he Poisson process iself. For a posiive ; average growh is higher, for a negaive ; i is lower. 10.5.3 Maringales Maringale is an impressive word for a simple concep. Here is a simpli ed de niion which is su cien for our purposes. More complee de niions (in a echnical sense), see furher reading. De niion 10.5.1 A sochasic process x () is a maringale if, being in oday, he expeced value of x a some fuure poin in ime equals he curren value of x; E x () = x () ; : (10.5.5)

10.5. Expecaion values 255 As he expeced value of x () is x () ; E x () = x () ; his can easily be rewrien as E [x () x ()] = 0: This is idenical o saying ha x () is a maringale if he expeced value of is incremens beween now and somewhere a in he fuure is zero. This de niion and he de niion of Brownian moion imply ha Brownian moion is a maringale. In wha follows, we will use he maringale properies of cerain processes relaively frequenly, for example when compuing momens. Here are now some fundamenal examples for maringales. Brownian moion Firs, look a Brownian moion, where we have a cenral resul useful for many applicaions where expecaions and oher momens are compued. I saes ha R f (z (s) ; s) dz (s), where f (:) is some funcion and z (s) is Brownian moion, is a maringale (Corollary 3.2.6, Øksendal, 1998, p. 33), Z E f (z (s) ; s) dz (s) = 0: (10.5.6) Poisson uncerainy A similar fundamenal resul for Poisson processes exiss. We will use in wha follows he maringale propery of various expressions conaining Poisson uncerainy. These expressions are idenical o or special cases of R f (q (s) ; s) dq (s) R f (q (s) ; s) ds; of which Garcia and Griego (1994, heorem 5.3) have shown ha i is a maringale indeed, Z Z E f (q (s) ; s) dq (s) f (q (s) ; s) ds = 0: (10.5.7) As always, is he (consan) arrival rae of q (s). 10.5.4 A more general approach o compuing momens When we wan o undersand momens of some sochasic process, we can proceed in wo ways. Eiher, a SDE is expressed in is inegral version, expecaions operaors are applied and he resuling deerminisic di erenial equaion is solved. Or, he SDE is solved direcly and hen expecaion operaors are applied. We already saw examples for he second approach in ch. 10.5.2. We will now follow he rs way as his is generally he more exible one. We rs sar wih examples for Brownian moion processes and hen look a cases wih Poisson uncerainy.

256 Chaper 10. SDEs, di erenials and momens The drif and variance raes for Brownian moion We will now reurn o our rs SDE in (10.1.3), dx () = ad + bdz () ; and wan o undersand why a is called he drif erm and b 2 he variance erm. To his end, we compue he mean and variance of x () for > : We sar by expressing (10.1.3) in is inegral represenaion as in ch. 10.1.3. This gives x () x() = Z ads + Z The expeced value E x () is hen simply bdz(s) = a [ ] + b [z() z ()] : (10.5.8) E x() = x () + a [ ] + be [z() z ()] = x () + a [ ] ; (10.5.9) where he second sep used he fac ha he expeced incremen of Brownian moion is zero. As he expeced value E x() is a deerminisic variable, we can compue he usual ime derivaive and nd how he expeced value of x (), being oday in, changes he furher he poin lies in he fuure, de x()=d = a: This makes clear why a is referred o as he drif rae of he random variable x (:) : Le us now analyse he variance of x () where we also sar from (10.5.8). The variance can from (7.3.1) be wrien as var x() = E x 2 () [E x()] 2 : In conras o he erm in (7.3.1) we need o condiion he variance on : If we compued he variance of x () from he perspecive of some earlier ; he variance would di er - as will become very clear from he expression we will see in a second. Applying he expression from (7.3.1) also shows ha we can look a any x () as a normal random variable: Wheher x () is described by a sochasic process or by some sandard descripion of a random variable, an x () for any x fuure poin in ime has some disribuion wih corresponding momens. This allows us o use sandard rules for compuing momens. Compuing rs E x 2 () gives by insering (10.5.8) E x 2 () = E [x() + a [ ] + b [z() z ()]] 2 = E [x() + a [ ]] 2 + 2 [x() + a ( )] b [z() z ()] + [b (z() z ())] 2 = [x() + a [ ]] 2 + 2 [x() + a ( )] be [z() z ()] + E [b (z() z ())] 2 = [x() + a [ ]] 2 + b 2 E [z() z ()] 2 ; (10.5.10) where he las equaliy used again ha he expeced incremen of Brownian moion is zero. As [E x()] 2 = [x () + a [ ]] 2 from (10.5.9), insering (10.5.10) ino he variance expression gives var x() = b 2 E [z() z ()] 2 : Compuing he mean of he second momen gives E [z() z ()] 2 = E z 2 () 2z()z () + z 2 () = E z 2 () z 2 () = var z () ;

10.5. Expecaion values 257 where we used ha z () is known in and herefore non-sochasic, ha E z() = z() and equaion (7.3.1) again. We herefore found ha var x() = b 2 var z () ; he variance of x () is b 2 imes he variance of z () : As he laer variance is given by var z () = ; given ha we focus on Brownian moion wih sandard normally disribued incremens, we found var x() = b 2 [ ] : This equaion shows why b is he variance parameer for x (:) and why b 2 is called is variance rae. The expression also makes clear why i is so imporan o sae he curren poin in ime, in our case. If we were furher in he pas or is furher in he fuure, he variance would be larger. Expeced reurns - Brownian moion Imagine an individual owns wealh a ha is allocaed o N asses such ha shares in wealh are given by i a i =a: The price of each asse follows a cerain pricing rule, say geomeric Brownian moion, and le s assume ha oal wealh of he household, neglecing labour income and consumpion expendiure, evolves according o da () = a () rd + N i=1 i i dz i () ; (10.5.11) where r N i=1 i r i : This is in fac he budge consrain (wih w c = 0) which will be derived and used in ch. 11.4 on capial asse pricing. Noe ha Brownian moions z i are correlaed, i.e. dz i dz j = ij d as in (10.2.5). Wha is he expeced reurn and he variance of holding such a porfolio, aking i and ineres raes and variance parameers as given? Using he same approach as in he previous example we nd ha he expeced reurn is simply given by r: This is due o he fac ha he mean of he BMs z i () are zero. The variance of a () is lef as an exercise. Expeced reurns - Poisson Le us now compue he expeced reurn of wealh when he evoluion of wealh is described by he budge consrain in (10.3.9), da () = fr () a () + w () p () c ()g d+ a () dq (). When we wan o do so, we rs need o be precise abou wha we mean by he expeced reurn. We de ne i as he growh rae of he mean of wealh when consumpion expendiure and labour income are idenical, i.e. when oal wealh changes only due o capial income. Using his de niion, we rs compue he expeced wealh level a some fuure poin in ime : Expressing his equaion in is inegral represenaion as in ch. 10.1.3 gives a () a () = Z fr (s) a (s) + w (s) p (s) c (s)g ds + Z a (s) dq (s) :

258 Chaper 10. SDEs, di erenials and momens Applying he expecaions operaor yields Z E a () a () = E fr (s) a (s) + w (s) = Z E r (s) E a (s) ds + Z p (s) c (s)g ds + E a (s) dq (s) Z E a (s) ds: (10.5.12) The second equaliy used an elemen of he de niion of he expeced reurn, i.e. w (s) = p (s) c (s) ; ha r (s) is independen of a (s) and he maringale resul of (10.5.7). When we de ne he mean of a (s) from he perspecive of by (s) E a (s) ; his equaion reads () a () = Z E r (s) (s) ds + Compuing he derivaive wih respec o ime gives _ () = E r () () + (), Z (s) ds: _ () () = E r () + : (10.5.13) Hence, he expeced reurn is given by E r () + : The assumed independence beween he ineres rae and wealh in (10.5.12) is useful here bu migh no hold in a general equilibrium seup if wealh is a share of and he ineres rae a funcion of he aggregae capial sock. Care should herefore be aken when using his resul more generally. Expeced growh rae Finally, we ask wha he expeced growh rae and he variance of he price v () is when i follows he geomeric Poisson process known from (10.3.8), dv () = v () d + v () dq () : v() v() The expeced growh rae can be de ned as E : Given ha v () is known in v() Ev() v() v() ; we can wrie his expeced growh rae as where expecaions are formed only wih respec o he fuure price v () : Given his expression, we can follow he usual approach. The inegral version of he SDE, applying he expecaions operaor, is E v () Z Z v () = E v (s) ds + E v (s) dq (s) : Pulling he expecaions operaor inside he inegral, using he maringale resul from (10.5.7) and de ning (s) E v (s) gives Z Z () v () = (s) ds + (s) ds: The ime- derivaive gives _ () = () + () which shows ha he growh rae of he mean of he price is given by + : This is, as jus discussed, also he expeced growh rae of he price v () :

10.5. Expecaion values 259 Disenangling he mean and variance of a Poisson process Consider he SDE dv () =v () = d + dq () where he arrival rae of q () is given by =: The mean of v () is E v () = v () exp (+)( ) and h hereby independen i of : The variance, however, is given by var v 1 () = [E v ()] 2 exp 2 [ ] 1. A mean preserving spread can hus be achieved by an increase of : This increases he randomly occurring jump and reduces he arrival rae = - he mean remains unchanged, he variance increases. Compuing he mean sep-by-sep for a Poisson example Le us now consider some sochasic processes X () described by a di erenial equaion and ask wha we know abou expeced values of X () ; where lies in he fuure, i.e. > : We ake as he rs example a sochasic process similar o (10.1.8). We ake as an example he speci caion for oal facor produciviy A, da () A () = d + 1dq 1 () 2 dq 2 () ; (10.5.14) where and i are posiive consans and he arrival raes of he processes are given by some consan i > 0. This equaion says ha TFP increases in a deerminisic way a he consan rae (noe ha he lef hand side of his di erenial equaion gives he growh rae of A ()) and jumps a random poins in ime. Jumps can be posiive when dq 1 = 1 and TFP increases by he facor 1 ; i.e. i increases by 1 %; or negaive when dq 2 = 1; i.e. TFP decreases by 2 %: The inegral version of (10.5.14) reads (see ch. 10.1.3) A () A () = = Z Z A (s) ds + Z When we form expecaions, we obain E A () 1 A (s) dq 1 (s) Z A (s) ds + 1 A (s) dq 1 (s) Z Z Z A () = E A (s) ds + 1 E A (s) dq 1 (s) Z Z = E A (s) ds + 1 1 E A (s) ds 2 A (s) dq 2 (s) Z 2 A (s) dq 2 (s) : (10.5.15) Z 2 E A (s) dq 2 (s) Z 2 2 E A (s) ds: (10.5.16) where he second equaliy used he maringale resul from ch. 10.5.3, i.e. he expression in (10.5.7). Pulling he expecaions operaor ino he inegral gives E A () A () = Z Z E A (s) ds + 1 1 E A (s) ds Z 2 2 E A (s) ds:

260 Chaper 10. SDEs, di erenials and momens When we nally de ne m 1 () E A () ; we obain a deerminisic di erenial equaion ha describes he evoluion of he expeced value of A () from he perspecive of : We rs have he inegral equaion m 1 () A () = Z Z m 1 (s) ds + 1 1 m 1 (s) ds Z 2 2 m 1 (s) ds which we can hen di ereniae wih respec o ime by applying he rule for di ereniaing inegrals from (4.3.3), _m 1 () = m 1 () + 1 1 m 1 () 2 2 m 1 () = ( + 1 1 2 2 ) m 1 () : (10.5.17) We now immediaely see ha TFP does no increase in expeced erms, more precisely E A () = A () ; if + 1 1 = 2 2 : Economically speaking, if he increase in TFP hrough he deerminisic componen and he sochasic componen 1 is desroyed on average by he second sochasic componen 2 ; TFP does no increase. 10.6 Furher reading and exercises Mahemaical background There are many exbooks on di erenial equaions, Io calculus, change of variable formulas and relaed aspecs in mahemaics. One ha is widely used is Øksendal (1998) and some of he above maerial is aken from here. A more echnical approach is presened in Proer (1995). Øksendal (1998, Theorem 4.1.2) covers proofs of some of he lemmas presened above. A much more general approach based on semi-maringales, and hereby covering all lemmas and CVFs presened here, is presened by Proer (1995). A classic mahemaical reference is Gihman and Skorohod (1972). See also Goodman (2002) on an inroducion o Brownian moion. A special focus wih a deailed formal analysis of SDEs wih Poisson processes can be found in Garcia and Griego (1994). They also provide soluions of SDEs and he background for compuing momens of sochasic processes. Furher soluions, applied o opion pricing, are provided by Das and Foresi (1996). The CVF for he combined Poisson-di usion seup in lemma 10.2.6 is a special case of he expression in Sennewald (2007) which in urn is based on Øksendal and Sulem (2005). Øksendal and Sulem presen CVFs for more general Levy processes of which he SDE (10.2.12) is a very simple special case. The claim for he soluion in (10.4.6) is an educaed guess. I builds on Arnold (1973, ch. 8.4) who provides a soluion for independen Brownian moions.

10.6. Furher reading and exercises 261 A less echnical background A very readable inroducion o sochasic processes is by Ross (1993, 1996). He wries in he inroducion o he rs ediion of his 1996 book ha his ex is a nonmeasure heoreic inroducion o sochasic processes. This makes his book highly accessible for economiss. An inroducion wih many examples from economics is Dixi and Pindyck (1994). See also Turnovsky (1997, 2000) for many applicaions. Brownian moion is reaed exensively in Chang (2004). The CVF for Poisson processes is mos easily accessible in Sennewald (2007) or Sennewald and Wälde (2006). Sennewald (2007) provides he mahemaical proofs, Sennewald and Wälde (2006) has a focus on applicaions. Proposiion 10.2.1 is aken from Sennewald (2007) and Sennewald and Wälde (2006). The echnical background for he noaion is he fac ha he process x () is a so called cádlág process ( coninu a droie, limies a gauche ), i.e. he pahs of x () are coninuous from he righ wih lef limis. The lef limi is denoed by x ( ) lim s" x (s). See Sennewald (2007) or Sennewald and Wälde (2006) for furher deails and references o he mahemaical lieraure. How o presen echnologies in coninuous ime There is a radiion in economics saring wih Eaon (1981) where oupu is represened by a sochasic di erenial equaion as presened in ch. 10.4.1. This and similar represenaions of echnologies are used by Epaular and Pommere (2003), Pommere and Smih (2005), Turnovsky and Smih (2006), Turnovsky (2000), Chaerjee, Giuliano and Turnovsky (2004), and many ohers. I is well known (see e.g. foonoe 4 in Grinols and Turnovsky (1998)) ha his implies he possibiliy of negaive Y. An alernaive where sandard Cobb-Douglas echnologies are used and TFP is described by a SDE o his approach is presened in Wälde (2005) or Wälde (2010). Applicaion of Poisson processes in economics The Poisson process is widely used in nance (early references are Meron, 1969, 1971) and labour economics (in maching and search models, see e.g. Pissarides, 2000, Burde and Morenson, 1998 or Moscarini, 2005). See also he lieraure on he real opions approach o invesmen (McDonald and Siegel, 1986, Dixi and Pindyck, 1994, Chen and Funke 2005 or Guo e al., 2005). I is also used in growh models (e.g. qualiy ladder models à la Aghion and Howi, 1992 or Grossman and Helpman, 1991), in analyses of business cycles in he naural volailiy radiion (e.g. Wälde, 2005), conrac heory (e.g. Guriev and Kvasov, 2005), in he search approach o moneary economics (e.g. Kiyoaki and Wrigh, 1993 and subsequen work) and many oher areas. Furher examples include Toche (2005), Seger (2005), Lainer and Solyarov (2004), Farzin e al. (1998), Hasse and Mecalf (1999), Thompson and Waldo (1994), Palokangas (2003, 2005) and Venegas-Marínez (2001).

262 Chaper 10. SDEs, di erenials and momens One Poisson shock a ecing many variables (as sressed in lemma 10.2.5) was used by Aghion and Howi (1992) in heir famous growh model. When deriving he budge consrain in Appendix 1 of Wälde (1999a), i is aken ino consideraion ha a jump in he echnology level a ecs boh he capial sock direcly as well as is price. Oher examples include he naural volailiy papers by Wälde (2002, 2005) and Posch and Wälde (2006). Disenangling he mean and variance of a Poisson process is aken from Sennewald and Wälde (2006). An alernaive is provided by Seger (2005) who uses wo symmeric Poisson processes insead of one here. He obains higher risk a an invarian mean by increasing he symmeric jump size. Oher Soluions o parial di erenial equaions as in he opion pricing example (10.4.4) are more frequenly used e.g. in physics (see Black and Scholes, 1973, p. 644). Parial di erenial equaions also appear in labour economics, however. See he Fokker-Planck equaions in Bayer and Wälde (2010a,b). De niions and applicaions of maringales are provided more sringenly in e.g. Øksendal (1998) or Ross (1996).

10.6. Furher reading and exercises 263 Exercises Chaper 10 Applied Ineremporal Opimizaion Sochasic di erenial equaions and rules for di ereniaing 1. Expecaions (a) Assume he price of bread follows a geomeric Brownian moion. Wha is he probabiliy ha he price will be 20% more expensive in he nex year? (b) Consider a Poisson process wih arrival rae : Wha is he probabiliy ha a jump occurs only afer 3 weeks? Wha is he probabiliy ha 5 jumps will have occurred over he nex 2 days? 2. Di erenials of funcions of sochasic processes I Assume x () and y () are wo correlaed Wiener processes. Compue d [x () y ()], d [x () =y ()] and d ln x () : 3. Di erenials of funcions of sochasic processes II (a) Show ha wih F = F (x; y) = xy and dx = f x (x; y) d + g x (x; y) dq x and dy = f y (x; y) d + g y (x; y) dq y, where q x and q y are wo independen Poisson processes, df = xdy + ydx. (b) Does his also hold for wo Wiener processes q x and q y? 4. Correlaed jump processes Le q i () for i = 1; 2; 3 be hree independen Poisson processes. De ne wo jump processes by q x () q 1 () + q 2 () and q y () q 1 () + q 3 () : Given ha q 1 () appears in boh de niions, q x () and q y () are correlaed jump processes. Le labour produciviy in secor X and Y be driven by da () = d + dq x () ; db () = d + dq y () : Le GDP in a small open economy (wih inernaionally given consan prices p x and p y ) be given by Y () p x A () L x + p y B () L y :

264 Chaper 10. SDEs, di erenials and momens (a) Given an economic inerpreaion o equaions da () and db (), based on q i () ; i = 1; 2; 3: (b) How does GDP evolve over ime in his economy if labour allocaion L x and L y is invarian? Express he di erenial dy () by using dq x and dq y if possible. 5. Deriving a budge consrain Consider a household who owns some wealh a = n 1 v 1 + n 2 v 2, where n i denoes he number of shares held by he household and v i is he value of he share. Assume ha he value of he share evolves according o dv i = i v i d + i v i dq i : Assume furher ha he number of shares held by he household changes according o dn i = i ( 1 n 1 + 2 n 2 + w e) d; v i where i is he share of savings used for buying sock i: (a) Give an inerpreaion (in words) of he las equaion. (b) Derive he household s budge consrain. 6. Opion pricing Assume he price of an asse follows ds=s = d + dz + dq (as in Meron, 1976), where z is Brownian moion and q is a Poisson process. This is a generalizaion of (10.3.1) where = 0: How does he di erenial equaion look like ha deermines he price of an opion on his asse? 7. Maringales (a) The weaher omorrow will be jus he same as oday. Is his a maringale? (b) Le z (s) be Brownian moion. Show ha Y () de ned by Y () exp Z 0 f (s) dz (s) 1 2 Z 0 f 2 (s) ds (10.6.1) is a maringale. (c) Show ha X () de ned by Z X () exp f (s) dz (s) 0 1 2 Z 0 f 2 (s) ds is also a maringale.

10.6. Furher reading and exercises 265 8. Expecaions - Poisson Assume ha you are in and form expecaions abou fuure arrivals of he Poisson process q (). Prove he following saemen by using lemma 10.5.1: The number of expeced arrivals in he ime inerval [; s] equals he number of expeced arrivals in a ime inerval of he lengh s for any s >, E (c k[q() q(s)] ) = E(c k[q() q(s)] ) = e (ck 1)( s) ; > s > ; c; k = cons: Hin: If you wan o chea, look a he appendix o Posch and Wälde (2006). I is available for example on www.waelde.com/publicaions. 9. Expeced values Show ha he growh rae of he mean of x () described by he geomeric Brownian moion dx () = ax () d + bx () dz () is given by a: (a) Do so by using he inegral version of his SDE and compue he increase of he expeced value of x (). (b) Do he same as in (a) bu solve rs he SDE and compue expecaions by using his soluion. (c) Compue he covariance of x () and x (s) for > s : (d) Compue he densiy funcion f(x()) for one speci c poin in ime > : Hin: Compue he variance of x() and he expeced value of x() as in (a) o (c). Wha ype of disribuion does x() come from? Compue he parameers and 2 of his disribuion for one speci c poin in ime = : 10. Expeced reurns Consider he budge consrain da () = fra () + w c ()g d + a () dz () : (a) Wha is he expeced reurn for wealh? Why does his expression di er from (10.5.13)? (b) Wha is he variance of wealh? 11. Di erenial-represenaion of a echnology (a) Consider he di erenial-represenaion of a echnology, dy () = AKd + Kdz () ; as presened in (10.4.1). Compue expeced oupu of Y () for > and he variance of Y () :

266 Chaper 10. SDEs, di erenials and momens (b) Consider a more sandard represenaion by Y () = A () K and le TFP follow da () =A () = gd + dz () as in (10.4.2). Wha is he expeced oupu level E Y () and wha is is variance? 12. Solving sochasic di erenial equaions Consider he di erenial equaion dx () = [a () x ()] d + c 1 () x () dz 1 () + g 2 () dz 2 () where z 1 and z 2 are wo correlaed Brownian moions. (a) Wha is he soluion of his di erenial equaion? (b) Use Io s Lemma o show ha your soluion is a soluion. 13. Dynamic and ineremporal budge consrains - Brownian moion Consider he dynamic budge consrain da () = (r () a () + w () p () c ()) d + a () dz () ; where z () is Brownian moion. (a) Show ha he ineremporal version of his budge consrain, using a no-ponzi game condiion, can be wrien as Z 1 e '() p () c () d = a + Z 1 where he discoun facor ' () is given by Z Z 1 ' () = r (s) 2 2 ds + dz () : e '() w () d (10.6.2) (b) Now assume we are willing o assume ha ineremporal budge consrains need o hold in expecaions only and no in realizaions: we require only ha agens balance a each insan expeced consumpion expendiure o curren nancial wealh plus expeced labour income. Show ha he ineremporal budge consrain hen simpli es o E Z 1 e R (r(s) 2 )ds p () c () d = a + E Z 1 e R (r(s) 2 )ds w () d i.e. all sochasic erms drop ou of he discoun facor bu he variance says here.

Chaper 11 In nie horizon models We now reurn o our main concern: How o solve maximizaion problems. We rs look a opimal behaviour under Poisson uncerainy where we analyse cases for uncerain asse prices and labour income. We hen swich o Brownian moion and look a capial asse pricing as an applicaion. 11.1 Ineremporal uiliy maximizaion under Poisson uncerainy 11.1.1 The seup Le us consider an individual ha ries o maximize his objecive funcion ha is given by Z 1 U () = E e [ ] u (c ()) d: (11.1.1) The srucure of his objecive funcion is idenical o he one we know from deerminisic coninuous ime models in e.g. (5.1.1) or (5.6.1): We are in oday, he ime preference rae is > 0; insananeous uiliy is given by u (c ()) : Given he uncerain environmen he individual lives in, we now need o form expecaions as consumpion in fuure poins in ime is unknown. When formulaing he budge consrain of a household, we have now seen a various occasions ha i is a good idea o derive i from he de niion of wealh of a household. We did so in discree ime models in ch. 2.5.5 and in coninuous ime models in ch. 4.4.2. Deriving he budge consrain in sochasic coninuous ime models is especially imporan as a budge consrain in an economy where he fundamenal source of uncerainy is Brownian moion looks very di eren from one where uncerainy sems from Poisson or Levy processes. For his rs example, we use he budge consrain (10.3.9) derived in ch. 10.3.2, da () = fr () a () + w () pc ()g d + a () dq () ; (11.1.2) 267

268 Chaper 11. In nie horizon models where he ineres rae r () + () =v () was de ned as he deerminisic rae of change of he price of he asse (compare he equaion for he evoluion of asses in (10.3.8)) plus dividend paymens () =v () : We rea he price p here as a consan (see he exercises for an exension). Following he ilde noaion from (10.1.7), we can express wealh ~a () afer a jump by ~a () = (1 + ) a () : (11.1.3) The budge consrain of his individual re ecs sandard economic ideas abou budge consrains under uncerainy. As visible in he derivaion in ch. 10.3.2, he uncerainy for his household sems from uncerainy abou he evoluion of he price (re eced in ) of he asse he saves in. No saemen was made abou he evoluion of he wage w () : Hence, we ake w () here as parameric, i.e. if here are sochasic changes, hey all come as a surprise and are herefore no anicipaed. The household does ake ino consideraion, however, he uncerainy resuling from he evoluion of he price v () of he asse. In addiion o he deerminisic growh rae of v () ; v () changes in a sochasic way by jumping occasionally by percen (again, see (10.3.8)). The reurns o wealh a () are herefore uncerain and are composed of he usual r () a () erm and he sochasic a () erm. 11.1.2 Solving by dynamic programming We will solve his problem as before by dynamic programming mehods. Noe, however, ha i is no obvious wheher he above problem can be solved by dynamic programming mehods. In principle, a proof is required ha dynamic programming indeed yields necessary and su cien condiions for an opimum. While proofs exis for bounded insananeous uiliy funcion u () ; such a proof did no exis unil recenly for unbounded uiliy funcions. Sennewald (2007) exends he sandard proofs and shows ha dynamic programming can also be used for unbounded uiliy funcions. We can herefore follow he usual hree-sep approach o dynamic programming here as well. DP1: Bellman equaion and rs-order condiions The rs ool we need o derive rules for opimal behavior is he Bellman equaion. De ning he opimal program as V (a) max fc()g U () subjec o a consrain, his equaion is given by (see Sennewald and Wälde, 2006, or Sennewald, 2007) V (a ()) = max c() u (c ()) + 1 d E dv (a ()) : (11.1.4) The Bellman equaion has his basic form for mos maximizaion problems in coninuous ime. I can herefore be aken as he saring poin for oher maximizaion problems as well, independenly, for example, of wheher uncerainy is driven by Poisson processes, Brownian moion or Levy processes. We will see examples of relaed problems laer (see ch. 11.1.4, ch. 11.2.2 or ch. 11.3.2) and discuss hen how o adjus cerain feaures of

11.1. Ineremporal uiliy maximizaion under Poisson uncerainy 269 his general Bellman equaion. In his equaion, he variable a () represens he sae variable, in our case wealh of he individual. See ch. 6.1 on dynamic programming in a deerminisic coninuous ime seup for a deailed inuiive discussion of he srucure of he Bellman equaion. Given he general form of he Bellman equaion in (11.1.4), we need o compue he di erenial dv (a ()) : Given he evoluion of a () in (11.1.2) and he CVF from (10.2.8), we nd dv (a ()) = V 0 (a) fra + w pcg d + fv (a + a) V (a)g dq: In conras o he CVF noaion in for example (10.2.8), we use here and in wha follows simple derivaive signs like V 0 (a) as ofen as possible in conras o for example V a (a) : This is possible as long as he funcions, like he value funcion V (a) here, have one argumen only. Forming expecaions abou dv (a ()) is easy and hey are given by E dv (a ()) = V 0 (a) fra + w pcg d + fv (~a) V (a)g E dq: The rs erm, he d-erm is known in : The curren sae a () and all prices are known and he shadow price V 0 (a) is herefore also known. As a consequence, expecaions need o be applied only o he dq-erm. The rs par of he dq-erm, he expression V ((1 + ) a) V (a) is also known in as again a (), parameers and he funcion V are all non-sochasic. We herefore only have o compue expecaions abou dq: From (10.5.3), we know ha E [q () q ()] = [ ] : Now replace q () q () by dq and by d and nd E dq = d: The Bellman equaion herefore reads V (a) = max fu (c ()) + V 0 (a) [ra + w pc] + [V ((1 + ) a) V (a)]g : (11.1.5) c() Noe ha forming expecaions he way jus used is, say, informal. Doing i in he more sringen way inroduced in ch. 10.5.4 would, however, lead o idenical resuls. The rs-order condiion is u 0 (c) = V 0 (a) p: (11.1.6) As always, (curren) uiliy from an addiional uni of consumpion u 0 (c) mus equal (fuure) uiliy from an addiional uni of wealh V 0 (a), muliplied by he price p of he consumpion good, i.e. by he number of unis of wealh for which one can buy one uni of he consumpion good. DP2: Evoluion of he cosae variable In order o undersand he evoluion of he marginal value V 0 (a) of he opimal program, i.e. he evoluion of he cosae variable, we need o (i) compue he parial derivaive of he maximized Bellman equaion wih respec o asses and (ii) compue he di erenial dv 0 (a) by using he CVF and inser he parial derivaive ino his expression. These wo seps correspond o he wo seps in DP2 in he deerminisic coninuous ime seup of ch. 6.1.

270 Chaper 11. In nie horizon models (i) In he rs sep, we sae he maximized Bellman equaion from (11.1.5) as he Bellman equaion where conrols are replaced by heir opimal values, V (a) = u (c (a)) + V 0 (a) [ra + w pc (a)] + [V ((1 + ) a) V (a)] : We hen compue again he derivaive wih respec o he sae - as in discree ime and deerminisic coninuous ime seups - as his gives us an expression for he shadow price V 0 (a). In conras o he previous emphasis on Io s Lemmas and CVFs, we can use for his sep sandard rules from algebra as we compue he derivaive for a given sae a - he sae variable is held consan and we wan o undersand he derivaive of he funcion V (a) wih respec o a: We do no compue he di erenial of V (a) and ask how he value funcion changes as a funcion of a change in a. Therefore, using he envelope heorem, V 0 (a) = V 0 (a) r + V 00 (a) [ra + w pc] + [V 0 (~a) [1 + ] V 0 (a)] : (11.1.7) We used here he de niion of ~a given in (11.1.3). (ii) In he second sep, he di erenial of he shadow price V 0 (a) is compued. Here, we do need a change of variable formula. Hence, given he evoluion of a () in (11.1.2), dv 0 (a) = V 00 (a) [ra + w pc] d + [V 0 (~a) V 0 (a)] dq: (11.1.8) Finally, replacing V 00 (a) [ra + w pc] in (11.1.8) by he same expression from (11.1.7) gives dv 0 (a) = f( r) V 0 (a) [V 0 (~a) [1 + ] V 0 (a)]g d + fv 0 (~a) V 0 (a)g dq: DP3: Insering rs-order condiions Finally, we can replace he marginal value by marginal uiliy from he rs-order condiion (11.1.6). In his sep, we employ ha p is consan and herefore dv 0 (a) = p 1 du 0 (c). Hence, he Keynes-Ramsey rule describing he evoluion of marginal uiliy reads du 0 (c) = f( r) u 0 (c) [u 0 (~c) [1 + ] u 0 (c)]g d + fu 0 (~c) u 0 (c)g dq: (11.1.9) Noe ha he consan price p dropped ou. This rule shows how marginal uiliy changes in a deerminisic and sochasic way. 11.1.3 The Keynes-Ramsey rule The dynamic programming approach provided us wih an expression in (11.1.9) which describes he evoluion of marginal uiliy from consumpion. While here is a one-o-one mapping from marginal uiliy o consumpion which would allow some inferences abou consumpion from (11.1.9), i would neverheless be more useful o have a Keynes-Ramsey rule for opimal consumpion iself.

11.1. Ineremporal uiliy maximizaion under Poisson uncerainy 271 The evoluion of consumpion If we wan o know more abou he evoluion of consumpion, we can use he CVF formula as follows. Le f (:) be he inverse funcion for u 0 ; i.e. f (u 0 (c)) = c; and apply he CVF o f (u 0 (c)) : This gives df (u 0 (c)) = f 0 (u 0 (c)) f( r) u 0 (c) [u 0 (~c) [1 + ] u 0 (c)]g d + ff (u 0 (~c)) f (u 0 (c))g dq: As f (u 0 (c)) = c; we know ha f (u 0 (~c)) = ~c and f 0 (u 0 (~c)) df(u0 (c)) Hence, dc = u 00 (c) u 0 (c) dc = = dc = 1 : du 0 (c) du 0 (c) u 00 (c) 1 u 00 (c) f( r) u0 (c) [u 0 (~c) [1 + ] u 0 (c)]g d + f~c cg dq, u 0 (~c) u 00 (c) r + [1 + ] 1 d f~c cg dq: (11.1.10) u 0 (c) u 0 (c) This is he Keynes-Ramsey rule ha describes he evoluion of consumpion under opimal behaviour for a household ha faces ineres rae uncerainy resuling from Poisson processes. This equaion is useful o undersand, for example, economic ucuaions in a naural volailiy seup. I corresponds o is deerminisic pendan in (5.6.4) in ch. 5.6.1: By seing = 0 here (implying dq = 0), noing ha we reaed he price as a consan and dividing by d, we obain (5.6.4). A speci c uiliy funcion Le us now assume ha he insananeous uiliy funcion is given by he widely used consan relaive risk aversion (CRRA) uiliy funcion, Then, he Keynes-Ramsey rule becomes dc c = n r u (c ()) = c ()1 1 ; > 0: (11.1.11) 1 h c + (1 + ) 1io d + ~c ~c c 1 dq: (11.1.12) The lef-hand side gives he proporional change of consumpion imes ; he inverse of he ineremporal elasiciy of subsiuion 1 : This corresponds o _c=c in he deerminisic Keynes-Ramsey rule in e.g. (5.6.5). Growh of consumpion depends on he righ-hand side in a deerminisic way on he usual di erence beween he ineres rae and ime preference rae plus he erm which capures he impac of uncerainy. When we wan o undersand he meaning of his erm, we need o nd ou wheher consumpion jumps up or down, following a jump of he Poisson process. When is posiive, he household holds more wealh and consumpion increases. Hence, he raio c=~c is smaller

272 Chaper 11. In nie horizon models han uniy and he sign of he bracke erm (1 + ) ~c c 1 is qualiaively unclear. If i is posiive, consumpion growh is faser in a world where wealh occasionally jumps by percen. The dq-erm gives discree changes in he case of a jump in q: I is, however, auological: When q jumps and dq = 1 and d = 0 for his small insan of he jump, (11.1.12) says ha dc=c on he lef-hand side is given by f~c=c 1g on he righ hand side. As he lef hand side is by de niion of dc given by [~c c] =c; boh sides are idenical. Hence, he level of ~c afer he jump needs o be deermined in an alernaive way. 11.1.4 Opimal consumpion and porfolio choice This secion analyses a more complex maximizaion problem han he one presened in ch. 11.1.1. In addiion o he consumpion-saving rade-o, i includes a porfolio choice problem. Ineresingly, he soluion is much simpler o work wih as ~c can explicily be compued and closed-form soluions can easily be found. The maximizaion problem Consider a household ha is endowed wih some iniial wealh a 0 > 0. A each insan, he household can inves is wealh a () in boh a risky and a safe asse. The share of wealh he household holds in he risky asse is denoed by (). The price v 1 () of one uni of he risky asse obeys he SDE dv 1 () = r 1 v 1 () d + v 1 () dq () ; (11.1.13) where r 1 2 R and > 0. Tha is, he price of he risky asse grows a each insan wih a xed rae r 1 and a random poins in ime i jumps by percen. The randomness comes from he well-known Poisson process q () wih arrival rae. The price v 2 () of one uni of he safe asse is assumed o follow dv 2 () = r 2 v 2 () d; (11.1.14) where r 2 0. Le he household receive a xed wage income w and spend c () 0 on consumpion. Then, in analogy o ch. 10.3.2, he household s budge consrain reads da () = f[ () r 1 + (1 ()) r 2 ] a () + w c ()g d + () a () dq () : (11.1.15) We allow wealh o become negaive bu we could assume ha deb is always covered by he household s lifeime labour income discouned wih he safe ineres rae r 2, i.e. a () > w=r 2. Le ineremporal preferences of households be idenical o he previous maximizaion problem - see (11.1.1). The insananeous uiliy funcion is again characerized by CRRA as in (11.1.11), u(c) = (c 1 1) = (1 ) : The conrol variables of he household are he

11.1. Ineremporal uiliy maximizaion under Poisson uncerainy 273 nonnegaive consumpion sream fc ()g and he share f ()g held in he risky asse. To avoid a rivial invesmen problem, we assume r 1 < r 2 < r 1 + : (11.1.16) Tha is, he guaraneed reurn of he risky asse, r 1, is lower han he reurn of he riskless asse, r 2, whereas, on he oher hand, he expeced reurn of he risky asse, r 1 +, shall be greaer han r 2. If r 1 was larger han r 2, he risky asse would dominae he riskless one and no one would wan o hold posiive amouns of he riskless asse. If r 2 exceeded r 1 + ; he riskless asse would dominae. DP1: Bellman equaion and rs-order condiions Again, he rs sep of he soluion of his maximizaion problem requires a Bellman equaion. De ne he value funcion V again as V (a ()) max fc();()g U () : The basic Bellman equaion is aken from (11.1.4). When compuing he di erenial dv (a ()) and aking ino accoun ha here are now wo conrol variables, he Bellman equaion reads V (a) = max fu(c) + [(r 1 + (1 ) r 2 ) a + w c] V 0 (a) + [V (~a) V (a)]g ; c();() (11.1.17) where ~a (1 + ) a denoes he pos-jump wealh if a wealh a a jump in he risky asse price occurs. The rs-order condiions which any opimal pah mus saisfy are given by u 0 (c) = V 0 (a) (11.1.18) and V 0 (a) (r 1 r 2 ) a + V 0 (~a) a = 0: (11.1.19) While he rs rs-order condiion equaes as always marginal uiliy wih he shadow price, he second rs-order condiion deermines opimal invesmen of wealh ino asses 1 and 2; i.e. he opimal share : The laer rs-order condiion conains a deerminisic and a sochasic erm and households hold heir opimal share if hese wo componens jus add up o zero. Assume, consisen wih (11.1.16), ha r 1 < r 2 : If we were in a deerminisic world, i.e. = 0; households would hen only hold asse 2 as is reurn is higher. In a sochasic world, he lower insananeous reurn on asse 1 is compensaed by he fac ha, as (11.1.13) shows, he price of his asse jumps up occasionally by he percenage : Lower insananeous reurns r 1 paid a each insan are herefore compensaed for by large occasional posiive jumps. As his rs-order condiion also shows, reurns and jumps per se do no maer: The di erence r 1 r 2 in reurns is muliplied by he shadow price V 0 (a) of capial and he e ec of he jump size imes is frequency, ; is muliplied by he shadow price V 0 (~a) of capial afer a jump. Wha maers for he household decision is herefore he impac of holding wealh in one or he oher asse on he overall value from behaving opimally, i.e. on he value funcion V (a). The channels hrough which asse reurns a ec he

274 Chaper 11. In nie horizon models value funcion is rs he impac on wealh and second he impac of wealh on he value funcion i.e. he shadow price of wealh. We can now immediaely see why his more complex maximizaion problem yields simpler soluions: Replacing in equaion (11.1.19) V 0 (a) wih u 0 (c) according o (11.1.18) yields for a 6= 0 u 0 (~c) u 0 (c) = r 2 r 1 ; (11.1.20) where ~c denoes he opimal consumpion choice for ~a. Hence, he raio for opimal consumpion afer and before a jump is consan. If we assume, for example, a CRRA uiliy funcion as in (11.1.11), his jump is given by 1= ~c c = : (11.1.21) r 2 r 1 No such resul on relaive consumpion before and afer he jump is available for he maximizaion problem wihou a choice beween a risky and a riskless asse. Since by assumpion (11.1.16) he erm on he righ-hand side is greaer han 1, his equaion shows ha consumpion jumps upwards if a jump in he risky asse price occurs. This resul is no surprising, as, if he risky asse price jumps upwards, so does he household s wealh. DP2: Evoluion of he cosae variable In he nex sep, we compue he evoluion of V 0 (a ()), he shadow price of wealh. Assume ha V is wice coninuously di ereniable. Then, due o budge consrain (11.1.15), he CVF from (10.2.8) yields dv 0 (a) = f[r 1 + (1 ) r 2 ] a + w cg V 00 (a) d + fv 0 (~a) V 0 (a)g dq () : (11.1.22) Di ereniaing he maximized Bellman equaion yields under applicaion of he envelope heorem Rearranging gives V 0 (a) = f[r 1 + (1 ) r 2 ] a + w cg V 00 (a) + fr 1 + [1 ] r 2 g V 0 (a) + fv 0 (~a) [1 + ] V 0 (a)g : f[r 1 + (1 ) r 2 ] a + w cg V 00 (a) = f [r 1 + (1 ) r 2 ]g V 0 (a) fv 0 (~a) [1 + ] V 0 (a)g : Insering his ino (11.1.22) yields f dv 0 [r1 + (1 ) r (a) = 2 ]g V 0 (a) f[1 + ] V 0 (~a) V 0 (a)g d + fv 0 (~a) V 0 (a)g dq () :

11.1. Ineremporal uiliy maximizaion under Poisson uncerainy 275 DP3: Insering rs-order condiions Replacing V 0 (a) by u 0 (c) following he rs-order condiion (11.1.18) for opimal consumpion, we obain f du 0 [r1 + (1 ) r (c) = 2 ]g u 0 (c) f[1 + ] u 0 (~c) u 0 d + fu 0 (~c) u 0 (c)g dq () : (c)g Now applying he CVF again o f (x) = (u 0 ) 1 (x) and using (11.1.20) leads o he Keynes- Ramsey rule for general uiliy funcions u, u 00 (c) u 0 (c) dc = r2 r 1 u 00 (c) r 1 + [1 ] r 2 + [1 + ] 1 d f~c cg dq () : u 0 (c) As ~c is also implicily deermined by (11.1.20), his Keynes-Ramsey rule describes he evoluion of consumpion under Poisson uncerainy wihou he ~c erm. This is he crucial modelling advanage of inroducing an addiional asse ino a sandard consumpionsaving problem. Apar from his simpli caion, he srucure of his Keynes-Ramsey rule is idenical o he one in (11.1.10) wihou a second asse. A speci c uiliy funcion For he CRRA uiliy funcion as in (11.1.11), he eliminaion of ~c becomes even simpler and we obain wih (11.1.21) dc () c () = 1 ( 1= r 2 r 1 r 2 1 d + 1) dq () : r 2 r 1 The opimal change in consumpion can hus be expressed in erms of well-known parameers. As long as he hprice ofhe risky asse does i no jump, opimal consumpion grows r consanly by he rae r 2 1 2 r 1 =. The higher he risk-free ineres rae, r 2, and he lower he guaraneed ineres rae of he risky asse, r 1, he discree growh rae,, he probabiliy of a price jump,, he ime preference rae,, and he risk aversion parameer,, he higher becomes he consumpion growh rae. If he risky asse price jumps, consumpion jumps as well o is new higher level c () = [() = (r 2 r 1 )] 1= c (). Here he growh rae depends posiively on,, and r 1, whereas r 2 and have a negaive in uence. 11.1.5 Oher ways o deermine ~c The quesion of how o deermine ~c wihou an addiional asse is in principle idenical o deermining he iniial level of consumpion, given a deerminisic Keynes-Ramsey rule as in for example (5.1.6). Whenever a jump in c following (11.1.12) occurs, he household faces he issue of how o choose he iniial level of consumpion afer he jump. In principle, he level ~c is herefore pinned down by some ransversaliy condiion. In pracice, he lieraure o ers wo ways, as o how ~c can be deermined.

276 Chaper 11. In nie horizon models One asse and idiosyncraic risk When households deermine opimal savings only, as in our seup where he only rs-order condiion is (11.1.6), ~c can be deermined (in principle) if we assume ha he value funcion is a funcion of wealh only - which would be he case in our household example if he ineres rae and wages did no depend on q. This would naurally be he case in idiosyncraic risk models where aggregae variables do no depend on individual uncerainy resuling from q: The rs-order condiion (11.1.6) hen reads u 0 (c) = V 0 (a) (wih p = 1). This is equivalen o saying ha consumpion is a funcion of he only sae variable, i.e. wealh a, c = c (a) : An example for c (a) is ploed in he following gure. N c dq = 1 c(a) ec s N c s a 0 a 1 a 2 a Figure 11.1.1 Consumpion c as a funcion of wealh a N As consumpion c does no depend on q direcly, we mus be a he same consumpion level c (a) ; no maer how we go here, i.e. no maer how many jumps ook place before. Hence, if we jump from some a 1 o a 2 because dq = 1; we are a he same consumpion level c (a 2 ) as if we had reached a 2 smoohly wihou jump. The consumpion level ~c afer a jump is herefore he consumpion level ha belongs o he asse level afer he jump according o he policy funcion c (a) ploed in he above gure, ~c = c (a 2 ). See Schlegel (2004) for a numerical implemenaion of his approach. Finding value funcions A very long radiion exiss in economics where value funcions are found by an educaed guess. Experience ells us - based on rs examples by Meron (1969, 1971) - wha value funcions generally look like. I is hen possible o nd, afer some aemps, he value funcion for some speci c problem. This hen implies explici - so called closed-form soluions - for consumpion (or any oher conrol variable). For he saving and invesmen problem in ch. 11.1.4, Sennewald and Wälde (2006, sec. 3.4) presened a value funcion and closed-form soluions for a 6= 0 of he form V (a) = h a + w r 2 i 1 1 1 ; c = " 1 a + wr2 ; = r 2 r 1 # a + w r 1 2 a :

11.1. Ineremporal uiliy maximizaion under Poisson uncerainy 277 Consumpion is a consan share ( is a collecion of parameers) ou of he oal wealh, i.e. nancial wealh a plus human wealh w=r 2 (he presen value of curren and fuure labour income). The opimal share depends on oal wealh as well, bu also on ineres raes, he degree of risk-aversion and he level of he jump of he risky price in (11.1.13). Hence, i is possible o work wih complex sochasic models ha allow o analyse many ineresing real-world feaures and neverheless end up wih explici closed-form soluions. Many furher examples exis - see ch. 11.6 on furher reading. Finding value funcions for special cases As we have jus seen, value funcions and closed-form soluions can be found for some models which have nice feaures. For a much larger class of models - which are hen sandard models - closed-form soluions canno be found for general parameer ses. Economiss hen eiher go for numerical soluions, which preserves a cerain generaliy as in principle he properies of he model can be analyzed for all parameer values, or hey resric he parameer se in a useful way. Useful means ha wih some parameer resricion, value funcions can be found again and closed-form soluions are again possible. 11.1.6 Expeced growh Le us now ry o undersand he impac of uncerainy on expeced growh. In order o compue expeced growh of consumpion from realized growh raes (11.1.12), we rewrie his equaion as dc () = r () c () + (1 + ) 1 c () d + f~c () ~c () Expressing i in is inegral version as in (10.5.15), we obain for > ; [c () c ()] = + Z Z r (s) c ()g dq: c (s) + (1 + ) 1 c (s) ds ~c (s) f~c (s) c (s)g dq (s) : Applying he expecaions operaor, given knowledge in ; yields E c () c () = 1 E Z r (s) Z + E f~c (s) c (s)g dq (s) : c (s) + (1 + ) 1 c (s) ds ~c (s)

278 Chaper 11. In nie horizon models Using again he maringale R resul from ch. 10.5.3 as already in (10.5.16), i.e. he expression R in (10.5.7), we replace E f~c (s) c (s)g dq (s) by E f~c (s) c (s)g ds; i.e. E c () c () = 1 Z c (s) E r (s) + (1 + ) 1 c (s) ds ~c (s) Z + E f~c (s) c (s)g ds: Di ereniaing wih respec o ime yields r () + (1 + ) de c () =d = 1 E c () ~c () 1 c () +E f~c () c ()g : Le us now pu ino he perspecive of ime ; i.e. le s move ime from o and le s ask wha expeced growh of consumpion is. This shif in ime means formally ha our expecaions operaor becomes E and we obain de c () =d c () = 1 E r () + (1 + ) c () ~c () 1 + E ~c () c () In his sep we used he fac ha due o his shif in ime, c () is now known and we can pull i ou of he expecaions operaor and divide by i. Combining brackes yields de c () =d = 1 c () ~c () c () E r () + (1 + ) 1 + 1 ~c () c () = 1 c () E ~c () r () + (1 + ) + 1 : ~c () c () 11.2 Maching on he labour marke: where value funcions come from Value funcions are widely used in maching models. Examples are unemploymen wih fricions models of he Morensen-Pissarides ype or shirking models of unemploymen a la Shapiro and Sigliz (1984). These value funcions can be undersood very easily on an inuiive level, bu hey really come from a maximizaion problem of households. In order o undersand when value funcions as he ones used in he jus-menioned examples can be used (e.g. under he assumpion of no saving, or being in a seady sae), we now derive value funcions in a general way and hen derive special cases used in he lieraure. 11.2.1 A household Le wealh a of a household evolve according o 1 : da = fra + z cg d:

11.2. Maching on he labour marke: where value funcions come from 279 Wealh increases per uni of ime d by he amoun da which depends on curren savings ra + z c: Labour income is denoed by z which includes income w when employed and unemploymen bene s b when unemployed, z = w; b: Labour income follows a sochasic Poisson di erenial equaion as here is job creaion and job desrucion. In addiion, we assume echnological progress ha implies a consan growh rae g of labour income. Hence, we can wrie dz = gzd dq w + dq b ; where w b: Job desrucion akes place a an exogenous, sae-dependen, arrival rae s (z) : The corresponding Poisson process couns how ofen our household moved from employmen ino unemploymen which is q w : Job creaion akes place a an exogenous rae (z) which is relaed o he maching funcion presened in (5.6.17). The Poisson process relaed o he maching process is denoed by q b : I couns how ofen a household leaves is b-saus, i.e. how ofen a job is found. As an individual canno lose his job when he does no have one and as nding a job makes (in his seup) no sense for someone who has a job, boh arrival raes are sae dependen. As an example, when an individual is employed, (w) = 0; when he is unemployed, s (b) = 0: Table 11.2.1 Sae dependen arrival raes z w b (z) 0 s (z) s 0 R 1 Le he individual maximize expeced uiliy E e [ ] u (c ()) d; where insananeous uiliy is of he CES ype, u (c) = c1 1 wih > 0: 1 11.2.2 The Bellman equaion and value funcions The sae space is described by a and z: The Bellman equaion has he same srucure as in (11.1.4). The adjusmens ha need o be made here follow from he fac ha we have wo sae variables insead of one. Hence, he basic srucure from (11.1.4) adoped o our problem reads V (a; z) = max fc()g u (c) + 1 E d dv (a; z) : The change of V (a; z) is, given he evoluion of a and z from above and he CVF from (10.2.11), dv (a; z) = fv a [ra + z c] + V z gzg d + fv (a; z ) V (a; z)g dq w + fv (a; z + ) V (a; z)g dq b : Forming expecaions, remembering ha E dq w = s (z) d and E dq b = (z) d; and dividing by d gives he Bellman equaion u (c) + V V (a; z) = max a [ra + z c] + V z gz : c +s (z) [V (a; z ) V (a; z)] + (z) [V (a; z + ) V (a; z)] (11.2.1)

280 Chaper 11. In nie horizon models The value funcions in he maching lieraure are all special cases of his general Bellman equaion. Denoe by U V (a; b) he expeced presen value of (opimal behaviour of a worker) being unemployed (as in Pissarides, 2000, ch. 1.3) and by W V (a; w), he expeced presen value of being employed. As he probabiliy of losing a job for an unemployed worker is zero, s (b) = 0; and (b) = ; he Bellman equaion (11.2.1) reads U = max fu (c) + U a [ra + b c] + U z gb + [W U]g ; c where we used ha W = V (a; b + ) : When we assume ha agens behave opimally, i.e. we replace conrol variables by heir opimal values, we obain he maximized Bellman equaion, U = u (c) + U a [ra + b c] + U z gb + [W U] : When we now assume ha households can no save, i.e. c = ra + b; and ha here is no echnological progress, g = 0; we obain U = u (ra + b) + [W U] : Assuming furher ha households are risk-neural, i.e. u (c) = c; and ha hey have no capial income, i.e. a = 0, consumpion is idenical o unemploymen bene s c = b: If he ineres rae equals he ime preference rae, we obain eq. (1.10) in Pissarides (2000), ru = b + [W U] : 11.3 Ineremporal uiliy maximizaion under Brownian moion 11.3.1 The seup Consider an individual whose budge consrain is given by da = fra + w pcg d + adz: The noaion is as always, uncerainy sems from Brownian R moion z: The individual 1 maximizes a uiliy funcion as given in (11.1.1), U () = E e [ ] u (c ()) d. The value funcion is de ned by V (a) = max fc()g U () subjec o he consrain. We again follow he hree sep scheme for dynamic programming.

11.3. Ineremporal uiliy maximizaion under Brownian moion 281 11.3.2 Solving by dynamic programming DP1: Bellman equaion and rs-order condiions The Bellman equaion is given for Brownian moion by (11.1.4) as well. When a maximizaion problem oher han one where (11.1.4) is suiable is o be formulaed and solved, ve adjusmens are, in principle, possible for he Bellman equaion. Firs, he discoun facor migh be given by some oher facor - for example he ineres rae r when he presen value of some rm is maximized. Second, he number of argumens of he value funcion needs o be adjused o he number of sae variables. Third, he number of conrol variables depends on he problem ha is o be solved and, fourh, he insananeous uiliy funcion is replaced by wha is found in he objecive funcion - which migh be, for example, insananeous pro s. Finally and obviously, he di erenial dv (:) needs o be compued according o he rules ha are appropriae for he sochasic processes which drive he sae variables. As he di erenial of he value funcion, following Io s Lemma in (10.2.3), is given by dv (a) = V 0 (a) [ra + w pc] + 12 V 00 (a) 2 a 2 d + V 0 (a) adz; forming expecaions E and dividing by d yields he Bellman equaion for our speci c problem V (a) = max u (c ()) + V 0 (a) [ra + w pc] + 12 V 00 (a) 2 a 2 c() and he rs-order condiion is DP2: Evoluion of he cosae variable u 0 (c ()) = V 0 (a) p () : (11.3.1) (i) The derivaive of he maximized Bellman equaion wih respec o he sae variable gives (using he envelope heorem) an equaion describing he evoluion of he cosae variable, V 0 = V 00 [ra + w pc] + V 0 r + 1 2 V 000 2 a 2 + V 00 2 a, ( r) V 0 = V 00 [ra + w pc] + 1 2 V 000 2 a 2 + V 00 2 a: (11.3.2) No surprisingly, given ha he Bellman equaion already conains he second derivaive of he value funcion, he derivaive of he maximized Bellman equaion conains is hird derivaive V 000. (ii) Compuing he di erenial of he shadow price of wealh V 0 (a) gives, using Io s Lemma (10.2.3), dv 0 = V 00 da + 1 2 V 000 2 a 2 d = V 00 [ra + w pc] d + 1 2 V 000 2 a 2 d + V 00 adz;

282 Chaper 11. In nie horizon models and insering ino he parial derivaive (11.3.2) of he maximized Bellman equaion yields dv 0 = ( r) V 0 d V 00 2 ad + V 00 adz = ( r) V 0 V 00 2 a d + V 00 adz: (11.3.3) As always a he end of DP2, we have a di erenial equaion (or di erence equaion in discree ime) which deermines he evoluion of V 0 (a) ; he shadow price of wealh. DP3: Insering rs-order condiions Assuming ha he evoluion of aggregae prices is independen of he evoluion of he marginal value of wealh, we can wrie he rs-order condiion for consumpion in (11.3.1) as du 0 (c) = pdv 0 + V 0 dp: This follows, for example, from Io s Lemma (10.2.6) wih pv 0 = 0: Using (11.3.3) o replace dv 0 ; we obain du 0 (c) = p ( r) V 0 V 00 2 a d + V 00 adz + V 0 dp = ( r) u 0 (c) u 00 (c) c 0 (a) 2 a d + u 00 (c) c 0 (a) adz + u 0 (c) dp=p; (11.3.4) where he second equaliy uses he rs-order condiion u 0 (c ()) = V 0 p () o replace V 0 and he parial derivaive of his rs-order condiion wih respec o asses, u 00 (c) c 0 (a) = V 00 p; o replace V 00 : When comparing his wih he expression in (11.1.9) where uncerainy sems from a Poisson process, we see wo common feaures: Firs, boh Keynes-Ramsey rules have a sochasic erm, he dz-erm here and he dq-erm in he Poisson case. Second, uncerainy a ecs he rend erm for consumpion in boh erms. Here, his erm conains he second derivaive of he insananeous uiliy funcion and c 0 (a) ; in he Poisson case, we have he ~c erms. The addiional dp-erm here sems from he assumpion ha prices are no consan. Such a erm would also be visible in he Poisson case wih exible prices. 11.3.3 The Keynes-Ramsey rule Jus as in he Poisson case, we wan a rule for he evoluion of consumpion here as well. We again de ne an inverse funcion and end up in he general case wih u 00 u dc 0 = r + u00 (c) u 0 (c) c a 2 a + 1 u 000 (c) 2 u 0 (c) [c aa] 2 d (11.3.5) u 00 (c) u 0 (c) c dp aadz p : Wih a CRRA uiliy funcion, we can replace he rs, second and hird derivaive of u (c) and nd he corresponding rule dc = r c a c c 2 a + 1 h 2 [ + 1] ca i 2 c a d (11.3.6) + c a c adz dp p :

11.4. Capial asse pricing 283 11.4 Capial asse pricing Le us again consider a ypical CAP problem. This follows and exends Meron (1990, ch. 15; 1973). The presenaion is in a simpli ed way. 11.4.1 The seup The basic srucure of he seup is idenical as before. There is an objecive funcion and a consrain. The objecive funcion capures he preferences of our agen and is described by a uiliy funcion as in (11.1.1). The consrain is given by a budge consrain which will now be derived, following he principles of ch. 10.3.2. Wealh a of households consis of a porfolio of asses i a = N i=1p i n i ; where he price of an asse is denoed by p i and he number of shares held, by n i. The oal number of asses is given by N. Le us assume ha he price of an asse follows geomeric Brownian moion, dp i p i = i d + i dz i ; (11.4.1) where each price is driven by is own drif parameer i and is own variance parameer i : Uncerainy resuls from Brownian moion z i which is also speci c for each asse. These parameers are exogenously given o he household bu would in a general equilibrium seing be deermined by properies of, for example, echnologies, preferences and oher parameers of he economy. Households can buy or sell asses by using a share i of heir savings, dn i = 1 p i i N i=1 i n i + w c d: (11.4.2) When savings are posiive and a share i is used for asse i; he number of socks held in i increases. When savings are negaive and i is posiive, he number of socks i decreases. The change in he households wealh is given by da = N i=1d (p i n i ) : The wealh held in one asse changes according o d (p i n i ) = p i dn i + n i dp i = i N i=1 i n i + w c d + n i dp i : The rs equaliy uses Io s Lemma from (10.2.6), aking ino accoun ha second derivaives of F (:) = p i n i are zero and ha dn i in (11.4.2) is deerminisic and herefore dp i dn i = 0: Using he pricing rule (11.4.1) and he fac ha shares add o uniy, N i=1 i = 1, he budge consrain of a household herefore reads da = N i=1 i n i + w c d + N i=1n i p i [ i d + i dz i ] = N i i=1 n i p i + n i p i i + w c d + N p i=1n i p i i dz i i = N i i=1a i + i + w c d + N p i=1a i i dz i : i

284 Chaper 11. In nie horizon models Now de ne i as always as he share of wealh held in asse i; i a i =a: Then, by de niion, a = N i=1 i a and shares add up o uniy, N i=1 i = 1: We rewrie his for laer purposes as N = 1 N 1 i=1 i (11.4.3) De ne furher he ineres rae for asse i and he ineres rae of he marke porfolio by This gives us he budge consrain, 11.4.2 Opimal behaviour r i i p i + i ; r N i=1 i r i : (11.4.4) da = a N i=1 i r i + w c d + a N i=1 i i dz i = fra + w cg d + a N i=1 i i dz i : (11.4.5) Le us now consider an agen who behaves opimally when choosing her porfolio and in making her consumpion-saving decision. We will no go hrough all he seps o derive a Keynes-Ramsey rule as asse pricing requires only he Bellman equaion and rs-order condiions. The Bellman equaion The Bellman equaion is given by (11.1.4), i.e. V (a) = max c();i () u (c ()) + 1 E d dv (a). Hence, we need again he expeced change of he value of one uni of wealh. Wih one sae variable, we simply apply Io s Lemma from (10.2.1) and nd 1 d E dv (a) = 1 d E V 0 (a) da + 12 V 00 (a) [da] 2 : (11.4.6) In a rs sep required o obain he explici version of he Bellman equaion, we compue he square of da: I is given, aking (10.2.2) ino accoun, by [da] 2 = a 2 N i=1 i i dz i 2 : When we compue he square of he sum, he expression for he produc of Brownian moions in (10.2.5) becomes imporan as correlaion coe ciens need o be aken ino consideraion. Denoing he covariances by ij i j ij, we ge [da] 2 = a 2 [ 1 1 dz 1 + 2 2 dz 2 + ::: + n n dz n ] 2 = a 2 2 1 2 1d + 1 1 2 2 12 d + ::: + 2 2 1 1 12 d + 2 2 2 2d + ::: + ::: = a 2 N j=1 N i=1 i j ij d: (11.4.7) Now rewrie he sum in (11.4.7) as follows N j=1 N i=1 j i ij = N 1 j=1 N i=1 j i ij + N i=1 i N in = N 1 j=1 N 1 i=1 j i ij + N i=1 i N in + N 1 j=1 N j Nj = N 1 j=1 N 1 i=1 j i ij + N 1 i=1 i N in + N 1 j=1 N j Nj + 2 N 2 N = N 1 j=1 N 1 i=1 j i ij + 2 N 1 i=1 i N in + 2 N 2 N

11.4. Capial asse pricing 285 As he second erm, using (11.4.3), can be wrien as N 1 i=1 i N in = 1 N 1 j=1 i N 1 i=1 i in ; our (da) 2 reads n (da) 2 = a 2 N 1 j=1 N 1 i=1 j i ij + 2 1 N 1 j=1 i N 1 i=1 i in + 1 N 1 j=1 o 2 i 2 N o 2 2 N = a 2 n N 1 j=1 N 1 i=1 j i ( ij 2 in ) + 2 N 1 i=1 i in + 1 N 1 i=1 i d d: (11.4.8) The second preliminary sep for obaining he Bellman equaion uses (11.4.3) again and expresses he ineres rae from (11.4.4) as a sum of he ineres rae of asse N (which could bu does no need o be a riskless asse) and weighed excess reurns r i r N ; r = N 1 i=1 ir i + 1 N 1 i=1 i rn = r N + N 1 i=1 i [r i r N ] : (11.4.9) The Bellman equaion wih (11.4.8) and (11.4.9) now nally reads V (a) = max u (c) + V 0 (a) (r N + N 1 i=1 i[r i r N ])a + w c + 1 c(); i () 2 V 00 (a) [da] 2 ; where (da) 2 should be hough of represening (11.4.8). Firs-order condiions The rs-order condiions are he rs-order condiion for consumpion, u 0 (c) = V 0 (a) ; and he rs-order condiion for asses. The rs-order condiion for consumpion has he well-known form. To compue rs-order condiions for shares i, we compue d f:g =d i for (11.4.8), d f:g = N 1 j=1 d j( ij 2 in ) + N 1 i=1 i( ij 2 in ) + 2 in 2 1 N 1 j=1 i 2 N i = 2 N 1 j=1 j( ij 2 in ) + in 1 N 1 j=1 i 2 N = 2 N 1 j=1 j( ij in ) + 1 N 1 j=1 i in 2 N = 2 N j=1 i [ ij in ] : (11.4.10) Hence, he derivaive of he Bellman equaion wih respec o i is wih (11.4.9) and (11.4.10) V 0 a[r i r N ] + 1 2 V 00 2a 2 N j=1 i [ ij in ] = 0 () r i r N = V 00 V 0 an j=1 i [ ij in ] : (11.4.11) The inerpreaion of his opimaliy rule should ake ino accoun ha we assumed ha an inerior soluion exiss. This condiion, herefore, says ha agens are indi eren beween he curren porfolio and a marginal increase in a share i if he di erence in insananeous reurns, r i r N, is compensaed by he covariances of of asses i and N: Remember ha from (11.4.4), insananeous reurns are cerain a each insan.

286 Chaper 11. In nie horizon models 11.4.3 Capial asse pricing Given opimal behaviour of agens, we now derive he well-known capial asse pricing equaion. Sar by assuming ha asse N is riskless, i.e. N = 0 in (11.4.1). This implies ha i has a variance of zero and herefore a covariance Nj wih any oher asse of zero as well, Nj = 0 8j: De ne furher a V 00 ; he covariance of asse i wih he marke V 0 porfolio as im N j=1 j ij ; he variance of he marke porfolio as 2 M N j=1 i im and he reurn of he marke porfolio as r N i=1 i r i as in (11.4.4). We are now only few seps away from he CAP equaion. Using he de niion of and im allows o rewrie he rs-order condiion for shares (11.4.11) as r i r N = im : (11.4.12) Muliplying his rs-order condiion by he share i gives i [r i r N ] = i im : Summing up all asses, i.e. applying N j=1 o boh sides, and using he above de niions yields r r N = 2 M: Dividing his expression by version (11.4.12) of he rs-order condiion yields he capial asse pricing equaion, r i r N = im (r r 2 N ) : M The raio im = 2 M is wha is usually called he -facor. 11.5 Naural volailiy II Before his book comes o an end, he discussion of naural volailiy models in ch. 8.4 is compleed in his secion. We will presen a simpli ed version of hose models ha appear in he lieraure which are presened in sochasic coninuous ime seups. The usefulness of Poisson processes will become clear here. Again, more background is available on hp://www.waelde.com/nv.hml. 11.5.1 An real business cycle model This secion presens he simples general equilibrium seup ha allows o sudy ucuaions semming from occasional jumps in echnology. The basic belief is ha economically relevan changes in echnologies are rare and occur every 5-8 years. Jumps in echnology means ha he echnological level, as capured by he TFP level, increases. Growh cycles herefore resul wihou any negaive TFP shocks.

11.5. Naural volailiy II 287 Technologies The economy produces a nal good by using a Cobb-Douglas echnology Y = K (AL) 1 : (11.5.1) Toal facor produciviy is modelled as labour augmening labour produciviy. While his is of no major economic imporance given he Cobb-Douglas srucure, i simpli es he noaion below. Labour produciviy follows a geomeric Poisson process wih drif da=a = gd + dq; (11.5.2) where g and are posiive consans and is he exogenous arrival rae of he Poisson process q. We know from (10.5.17) in ch. 10.5.4 ha he growh rae of he expeced value of A is given by g + : The nal good can be used for consumpion and invesmen, Y = C + I; which implies ha he prices of all hese goods are idenical. Choosing Y as he numeraire good, he price is one for all hese goods. Invesmen increases he sock of producion unis K if invesmen is larger han depreciaion, capured by a consan depreciaion rae ; dk = (Y C K) d: (11.5.3) There are rms who maximize insananeous pro s. They do no bear any risk and pay facors r and w; marginal produciviies of capial and labour. Households Households maximize heir objecive funcion U () = E Z 1 e [ ] u (c ()) d by choosing he consumpion pah fc ()g : Insananeous uiliy can be speci ed by u (c) = c1 1 1 : (11.5.4) Wealh of households consiss of shares in rms which are denoed by k: This wealh changes in a deerminisic way (we do no derive i here bu i could be done following he seps in ch. 10.3.2), despie he presence of TFP uncerainy. This is due o wo facs: Firs, wealh is measured in unis of physical capial, i.e. summing k over all households gives K: As he price of one uni of K equals he price of one uni of he oupu good and he laer was chosen as numeraire, he price of one uni of wealh is non-sochasic. This di ers from (10.3.8) where he price jumps when q jumps. Second, a jump in q does no a ec k direcly. This could be he case when new echnologies make par of he old capial sock obsolee. Hence, he consrain of households is a budge consrain which reads dk = (rk + w c) d: (11.5.5) The ineres rae is given by he di erence beween he marginal produc of capial and he depreciaion rae, r = @Y=@K :

288 Chaper 11. In nie horizon models Opimal behaviour When compuing opimal consumpion levels, households ake he capial sock k and he TFP level A as heir sae variables ino accoun. This seup is herefore similar o he deerminisic wo-sae maximizaion problem in ch. 6.3. Going hrough similar seps (concerning, for example, he subsiuing of cross derivaives in sep DP2) and aking he speci c aspecs of his sochasic framework ino accoun, yields following opimal consumpion (see exercise 8) dc c = n r h c ~c + 1io d + ~c c 1 dq: (11.5.6) Despie he deerminisic consrain (11.5.5) and due o TFP jumps in (11.5.2), consumpion jumps as well: a dq erm shows up in his expression and marginal uiliy levels before (c ) and afer (~c ) he jump, using he noaion from (10.1.7) appear as well. Marginal uiliies appear in he deerminisic par of his rule due o precauionary saving consideraions. The reason for he jump is sraighforward: whenever here is a discree increase in he TFP level, he ineres rae and wages jump. Hence, reurns for savings or households change and he household adjuss is consumpion level. This is in principle idenical o he behaviour in he deerminisic case as illusraed in g. 5.6.1 in ch. 5.6.1. (Underaking his here for his sochasic case would be very useful.) General equilibrium We are now in a posiion o sar hinking abou he evoluion of he economy as a whole. I is described by a sysem in hree equaions. Labour produciviy follows (11.5.2). The capial sock follows dk = K (AL) 1 C K d from (11.5.1) and (11.5.3). Aggregae consumpion follows dc C = ( 1 ) AL C + 1 d + K ~C ( ) ~C C 1 dq from aggregaing over households using (11.5.6). Individual consumpion c is replaced by aggregae consumpion C and he ineres rae is expressed by marginal produciviy of capial minus depreciaion rae. This sysem looks fairly similar o deerminisic models, he only subsanial di erence lies in he dq erm and he pos-jump consumpion levels ~C:

11.5. Naural volailiy II 289 Equilibrium properies for a cerain parameer se The simples way o ge an inuiion abou how he economy evolves consiss in looking a an example, i.e. by looking a a soluion of he above sysem ha holds for a cerain parameer se. We choose as example he parameer se for which he saving rae is consan and given by s = 1 1 : The parameer se for which C = (1 s) Y is opimal, is given by = + (1 + ) 1 1 = ( (1 ) g) : As we need > 1 for a meaningful saving rae, he ineremporal elasiciy of subsiuion 1 is smaller han one. For a derivaion of his resul, see he secion on Closed-form soluions wih parameer resricions in furher reading. The dynamics of capial and consumpion can hen be bes analyzed by looking a auxiliary variables, in his case capial per e ecive worker, ^K = K=A. This auxiliary variable is needed o remove he rend ou of capial K: The original capial sock grows wihou bound, he auxiliary variable has a nie range. Using auxiliary variables of his ype has a long radiion in growh models as derending is a common requiremen for an informaive analysis. The evoluion of his auxiliary variable is given by (applying he appropriae CVF) d ^K n o = ^K 1 L 1 2 ^K d 3 ^Kdq; (11.5.7) where i are funcions of preference and echnology parameers of he model. One can show ha 0 < 3 < 1: The evoluion of his capial sock per e ecive worker can be illusraed by using a gure which is similar o hose used o explain he Solow growh model. The following gure shows he evoluion of he capial sock per worker on he lef and he evoluion of GDP on he righ. Figure 11.5.1 Cyclical growh Assume he iniial capial sock ^K is given by ^K 0 : Assume also, for he ime being, ha here is no echnology jump, i.e. dq = 0: The capial sock ^K hen increases smoohly over ime and approaches a emporary seady sae ^K which follows from (11.5.7) wih dq = 0 and d ^K = 0: This emporary seady sae is given by ^K = ( 1 L 1 = 2 ) 1=(1 ) and has properies in perfec analogy o he seady sae in he Solow model. When a echnology jump occurs, he capial sock per e ecive worker diminishes as he denominaor in

290 Chaper 11. In nie horizon models ^K = K=A increases bu he capial sock in he numeraor in he insan of he jump does no change. The capial sock per e ecive worker is hrown back, as indicaed by he arrow in he lef panel above. Afer ha, he capial sock ^K again approaches he emporary seady sae ^K : The righ panel of he above gure shows wha hese echnology jumps do o GDP. As long as here is no echnology jump, GDP approaches an upper limi Yq which is speci c o he echnology level q: A echnology jump increases GDP Y q insananeously as TFP goes up. (This increase could be very small, depending on wha share of producion unis enjoy an increase in TFP.) The more imporan increase, however, resuls from he he shif in he upper limi from Yq o Yq+1: Capial accumulaion following he echnology jump increases TFP which now approaches he new upper limi. This process of endogenous growh cycles coninues ad in nium. 11.5.2 A naural volailiy model The above analysis can be exended o allow for endogenous echnology jumps. As in he discree ime version of his seup, he probabiliy ha a echnology jump occurs is made a funcion of resources R invesed ino R&D. In conras o (8.4.3), however, i is he arrival rae and no he probabiliy iself which is a funcion of R; = (R) : This is a requiremen of coninuous ime seups and is builds on a long radiion in coninuous ime models wih Poisson processes. The resource consrain of he economy is hen exended accordingly o dk = (Y C R K) d by including he resources R: A model of his ype can hen be solved as before. Eiher one considers special parameer values and obains closed-form soluions or one performs a numerical analysis (see below). The qualiaive properies of cyclical growh are idenical o he ones presened before in g. 11.5.1. The crucial economic di erence consiss in he fac ha he frequency of echnology jumps, i.e. he average lengh of a business cycle, depend on decisions of households. If households nd i pro able o shif a larger amoun of resources ino R&D, business cycles will be shorer. No only he long-run growh rae, bu also shor-run ucuaions are in uenced by fundamenal parameers of he model as also by governmen policy. All hese quesions are analyzed in deail in he naural volailiy lieraure. 11.5.3 A numerical approach I is helpful for a numerical soluion o have a model descripion in saionary variables. To his end, de ne auxiliary variables ^K = K=A and ^C = C=A. The one for capial is

11.6. Furher reading and exercises 291 he same as was used in (11.5.7), he one for consumpion is new as we will now no work wih a closed-form soluion. Le us look a a siuaion wih exogenous arrival raes, i.e. a he RBC model of above, o illusrae he basic approach for a numerical soluion. When compuing he dynamics of hese variables (see furher reading for references), we nd d ^C ( " 1 ^C = g + L^K ^C (1 + ) ~^C d ^K n o = ^Y ( + g) ^K ^C d! 1#) d + ( ) ~^C ^C 1 dq; (11.5.8) 1 + ^Kdq: (11.5.9) When we look a hese equaions, hey almos look like ordinary di erenial equaions. The only di culy is conained in he erm ~^C: To undersand he soluion procedure, hink of he saddle-pah rajecory in he opimal growh model - see g. 5.6.2 in ch. 5.6.3. Given he ransformaion underaken here, he soluion of his sysem is given by a policy funcion ^C ^K which is no a funcion of he echnological level A: The objecive of he soluion herefore consiss of nding his ^C ^K ; a saddle pah in analogy o he one in g. 5.6.2. The erm ~^C sands for he consumpion level afer he jump. As he funcional relaionship is independen of A and hereby he number of jumps, we can wrie ~^C = 1 ^C ^K ; i.e. i is he consumpion level a he capial sock 1 ^K afer a 1+ 1+ jump. As > 0; a jump implies a reducion in he auxiliary, i.e. echnology-ransformed, capial sock ^K: The rick in solving his sysem now consiss of providing no only wo iniial condiions bu one iniial condiion for capial an iniial pah for consumpion ^C ^K. This iniial pah is hen reaed as an exogenous variable in he denominaor of (11.5.8) and he above di erenial equaions have been ransformed ino a sysem of ordinary di erenial equaions! Leing he iniial pahs sar a he origin and making hem linear, he quesion hen simply consiss of nding he righ slope such ha he soluion of he above sysem ideni es a saddle pah and seady sae. For more deails and an implemenaion, see Numerical soluion in furher reading. 11.6 Furher reading and exercises Mahemaical background on dynamic programming There are many, and in mos cases much more echnical, presenaions of dynamic programming in coninuous ime under uncerainy. A classic mahemaical reference is Gihman and Skorohod (1972) and a widely-used mahemaical exbook is Øksendal (1998); see also Proer (1995). These books are probably useful only for hose wishing o work on he heory of opimizaion and no on applicaions of opimizaion mehods.

292 Chaper 11. In nie horizon models Kushner (1967) and Dempser (1991) have a special focus on Poisson processes. Opimizaion wih unbounded uiliy funcions by dynamic programming was sudied by Sennewald (2007). Wih SDEs we need boundary condiions as well. In he in nie horizon case, we would need a ransversaliy condiion (TVC). See Smih (1996) for a discussion of a TVC in a seup wih Epsein-Zinn preferences. Sennewald (2007) has TVCs for Poisson uncerainy. Applicaions Books in nance ha use dynamic programming mehods include Du e (1988, 2001) and Björk (2004). Sochasic opimizaion for Brownian moion is also covered nicely in Chang (2004). A maximizaion problem of he ype presened in ch. 11.1 was rs analyzed in Wälde (1999, 2008). This chaper combines hese wo papers. These wo papers were also joinly used in he Keynes-Ramsey rule appendix o Wälde (2005). I is also used in Posch and Wälde (2006), Sennewald and Wälde (2006) and elsewhere. Opimal conrol in sochasic coninuous ime seups is used in many applicaions. Examples include issues in inernaional macro (Obsfeld, 1994, Turnovsky, 1997, 2000), inernaional nance and deb crises (Sein, 2006) and also he analysis of he permanenincome hypohesis (Wang, 2006) or of he wealh disribuion hypohesis (Wang, 2007), and many ohers. A rm maximizaion problem wih risk-neuraliy where R&D increases qualiy of goods, modelled by a sochasic di erenial equaion wih Poisson uncerainy, is presened and solved by Dinopoulos and Thompson (1998, sec. 2.3). The Keynes-Ramsey rule in (11.3.4) was derived in a more or less complex framework by Breeden (1986) in a synhesis of his consumpion based capial asse pricing model, Cox, Ingersoll and Ross (1985) in heir coninuous ime capial asse pricing model, or Turnovsky (2000) in his exbook. Cf. also Obsfeld (1994). There are various recen papers which use coninuous ime mehods under uncerainy. For examples from nance and moneary economics, see DeMarzo and Urošević (2006), Gabaix e al. (2006), Maenhou (2006), Piazzesi (2005), examples from risk heory and learning include Kyle e al. (2006) and Keller and Rady (1999), for indusrial organizaion see, for example, Muro (2004). In spaial economics here is, for example, Gabaix (1999), he behaviour of households in he presence of durable consumpion goods is analyzed by Berola e al. (2005), R&D dynamics are invesigaed by Bloom (2007) and mismach and exi raes in labour economics are analyzed by Shimer (2007,2008). The e ec of echnological progress on unemploymen is analyzed by Pra (2007). The real opions approach o invesmen or hiring under uncerainy is anoher larger area. See, for example, Benolila and Berola (1990) or Guo e al. (2005). Furher references o papers ha use Poisson processes can be found in ch. 10.6. Closed form soluions Closed-form soluions and analyical expressions for value funcions have been derived by many auhors. This approach was pioneered by Meron (1969, 1971) for Brownian

11.6. Furher reading and exercises 293 moion. Chang (2004) devoes an enire chaper (ch. 5) o closed-form soluions for Brownian moion. For seups wih Poisson-uncerainy, Dinopoulos and Thompson (1998, sec. 2.3), Wälde (1999b) or Sennewald and Wälde (2006, ch. 3.4) derive closed-form soluions. Closed-form soluions for Levy processes are available, for example, from Aase (1984), Framsad, Øksendal and Sulem (2001) and in he exbook by Øksendal and Sulem (2005). Closed-form soluions wih parameer resricions Someimes, resricing he parameer se of he economy in some inelligen way allows o provide closed-form soluions for very general models. These soluions provide insighs which canno be obained ha easily by numerical analysis. Early examples are Long and Plosser (1983) and Benhabib and Rusichini (1994) who obain closed-form soluions for a discree-ime sochasic seup. In deerminisic, coninuous ime, Xie (1991, 1994) and Barro, Mankiw and Sala-i-Marin (1995) use his approach as well. Wälde (2005) and Wälde and Posch (2006) derive closed-form soluions for business cycle models wih Poisson uncerainy. Sennewald and Wälde (2006) sudy an invesmen and nance problem. The example in secion 11.5.1 is aken from Schlegel (2004). The mos deailed sep-by-sep presenaion of he soluion echnique is in he Referees appendix o Wälde (2005). Naural volailiy The expression naural volailiy represens a cerain view abou why almos any economic ime series exhibis ucuaions. Naural volailiy says ha ucuaions are naural, hey are inrinsic o any growing economy. An economy ha grows is also an economy ha ucuaes. Growh and ucuaions are wo sides of he same coin, hey have he same origin: new echnologies. Published papers in his lieraure are (in sequenial and alphabeical order) Fan (1995), Benal and Peled (1996), Freeman, Hong and Peled (1999), Masuyama (1999, 2001), Wälde (1999, 2002, 2005), Li (2001) Francois and Lloyd-Ellis (2003,2008), Maliar and Maliar (2004) and Phillips and Wrase (2006). Numerical soluion The numerical soluion was analyzed and implemened by Schlegel (2004). Maching and saving Ch. 11.2 shows where value funcions in he maching lieraure come from. This chaper uses a seup where uncerainy in labour income is combined wih saving. I hereby presens he ypical seup of he saving and maching lieraure. The seup used here was explored in more deail by Bayer and Wälde (2010a, b) and Bayer e al. (2010, ).

294 Chaper 11. In nie horizon models The maching and saving lieraure in general builds on incomplee marke models where households can insure agains income risk by saving (Hugge, 1993; Aiyagari, 1994; Hugge and Ospina, 2001; Marce e al., 2007). Firs analyses of maching and saving include Andolfao (1996) and Merz (1995) where individuals are fully insured agains labour income risk as labour income is pooled in large families. Papers which exploi he advanage of CARA uiliy funcions include Acemoglu and Shimer (1999), Hassler e al. (2005), Shimer and Werning, (2007, 2008) and Hassler and Rodriguez Mora, 1999, 2008). Their closed-form soluions for he consumpion-saving decision canno always rule ou negaive consumpion levels for poor households. Bils e al. (2007, 2009), Nakajima (2008) and Krusell e al. (2010) work wih a CRRA uiliy funcion in discree ime.

11.6. Furher reading and exercises 295 Exercises Chaper 11 Applied Ineremporal Opimizaion Dynamic Programming in coninuous ime under uncerainy 1. Opimal saving under Poisson uncerainy wih wo sae variables Consider he objecive funcion U () = E Z 1 e [ ] u (c ()) d and he budge consrain da () = fra () + w p () c ()g d + a () dq () ; where r and w are consan ineres and wage raes, q () is a Poisson process wih an exogenous arrival rae and is a consan as well. Leing g and denoe consans, assume ha he price p () of he consumpion good follows dp () = p () [gd + dq ()] : (a) Derive a rule which opimally describes he evoluion of consumpion. Deriving his rule in he form of marginal uiliy, i.e. du 0 (c ()) is su cien. (b) Derive a rule for consumpion, i.e. dc () = ::: (c) Derive a rule for opimal consumpion for = 0 or = 0: 2. Opimal saving under Brownian moion Derive he Keynes-Ramsey rules in (11.3.5) and (11.3.6), saring from he rule for marginal uiliy in (11.3.4). 3. Adjusmen cos R 1 Consider a rm ha maximizes is presen value de ned by = E e r[ ] () d: The rm s pro is given by he di erence beween revenues and coss, = px ci 2 ; where oupu is assumed o be a funcion of he curren capial sock, x = k : The rm s conrol variable is invesmen i ha deermines is capial sock, dk = (i k) d:

296 Chaper 11. In nie horizon models The rm operaes in an uncerain environmen. Oupu prices and coss for invesmen evolve according o dp=p = p d + p dz p ; dc=c = c d + c dz c ; where z p and z c are wo independen sochasic processes. (a) Assume z p and z c are wo independen Brownian moions. Se p = p = 0; such ha he price p is consan. Wha is he opimal invesmen behaviour of he rm? (b) Consider he alernaive case where coss c are consan bu prices p follow he above SDE. How much would he rm now inves? (c) Provide an answer o he quesion in a) when z c is a Poisson process. (d) Provide an answer o he quesion in b) when z p is a Poisson process. 4. Firm speci c echnological progress Consider a rm facing a demand funcion wih price elasiciy "; x = p " ; where p is he price and is a consan. Le he rm s echnology be given by x = a q l where a > 1: The rm can improve is echnology by invesing in R&D. R&D is modelled by he Poisson process q which jumps wih arrival rae (l q ) where l q is employmen in he research deparmen of he rm. The exogenous wage rae he rm faces amouns o w: (a) Wha is he opimal saic employmen level l of his rm for a given echnological level q? (b) Formulae an ineremporal objecive funcion given by he presen value of he rms pro ows over an in nie ime horizon. Coninue o assume ha q is consan and le he rm choose l opimally from his ineremporal perspecive. Does he resul change wih respec o (a)? (c) Using he same objecive funcion as in (b), le he rm now deermine boh l and l q opimally. Wha are he rs-order condiions? Give an inerpreaion in words. (d) Compue he expeced oupu level for >, given he opimal employmen levels l and l q: In oher words, compue E x () : Hin: Derive rs a sochasic di erenial equaion for oupu x () : 5. Budge consrains and opimal saving and nance decisions Imagine an economy wih wo asses, physical capial K () and governmen bonds B (). Le wealh a () of households be given by a () = v () k () + b () where v () is he price of one uni of capial, k () is he number of socks and b () is he

11.6. Furher reading and exercises 297 nominal value of governmen bonds held by he household. Assume he price of socks follows dv () = v () d + v () dq () ; where and are consans and q () is a Poisson process wih arrival rae : (a) Derive he budge consrain of he household. Use () v () k () =a () as he share of wealh held in socks. (b) Derive he budge consrain of he household by assuming ha q () is Brownian moion. (c) Now le he household live in a world wih hree asses (in addiion o he wo above, here are asses available on f oreign markes). Assume ha he budge consrain of he household is given by da () = fr () a () + w () p () c ()g d+ k k () a () dq ()+ f f () a () dq f () ; where r () = k () r k + f () r f + (1 k () f ()) r b is he ineres rae depending on weighs i () and consan insananeous ineres raes r k ; r f and r b : Le q () and q f () be wo Poisson processes. Given he usual objecive funcion U () = E Z 1 e [ ] u (c ()) d; wha is he opimal consumpion rule? Wha can be said abou opimal shares k and f? 6. Capial asse pricing in ch. 11.4.3 The covariance of asse i wih he marke porfolio is denoed by im ; he variance of he marke porfolio is 2 M and he reurn of he marke porfolio is r M i i r i : (a) Show ha he covariance of asse i wih he marke porfolio im is given by N j=1 j ij : (b) Show ha N j=1 i im is he variance of he marke porfolio 2 M.

298 Chaper 11. In nie horizon models 7. Sandard and non-sandard echnologies Le he social welfare funcion of a cenral planner be given by U () = E Z 1 e [ ] u (C ()) d: (a) Consider an economy where he capial sock follows dk = AK L 1 [d + dz] (K + C) d where dz is he incremen of Brownian moion and and are consans. Derive he Keynes-Ramsey rule for his economy. (b) Assume ha dy = [d + dz] and ha is consan. Wha is he expeced level of Y () for > ; i.e. E Y ()? (c) Consider an economy where he echnology is given by Y = AK L 1 wih da = Ad + Adz; where z is Brownian moion. Le he capial sock follow dk = (Y K C) d: Derive he Keynes-Ramsey rule for his economy as well. (d) Is here a parameer consellaion under which he Keynes-Ramsey rules are idenical? 8. Sandard and non-sandard echnologies II Provide answers o he same quesions as in "Sandard and non-sandard echnologies" bu assume ha z is a Poisson process wih arrival rae : Compare your resul o (11.5.6).

Chaper 12 Miscellanea, references and index The concep of ime Wha is ime? Wihou geing ino philosophical deails, i is useful o be precise here abou how ime is denoed. Time is always denoed by : Time can ake di eren values. The mos imporan (frequenly encounered) one is he value where sands for oday. This is rue boh for he discree and for he coninuous ime conex. In a discree ime conex, + 1 is hen obviously omorrow or he nex period. Anoher ypical value of ime is 0 ; which is usually he poin in ime for which iniial values of some process are given. Similarly, T denoes a fuure poin in ime where, for example, he planning horizon ends or for which some boundary values are given. In mos cases, ime refers o some fuure poin in ime in he sense of : Given hese de niions, how should one presen generic ransiion equaions, in mos examples above budge consrains? Should one use ime as argumen or ime? In wo-period models, i is mos naural o use for oday and + 1 for omorrow. This is he case in he wo-period models of ch. 2, for example, in he budge consrains (2.1.2) and (2.1.3). This choice becomes less obvious in muli-period models of ch. 3. When he rs ransiion equaion appears in (3.3.1) and he rs dynamic (in conras o ineremporal) budge consrain in (3.4.1), one can make argumens boh in favour of or as ime argumen. Using would be mos general: The ransiion equaion is valid for all periods in ime, hence one should use : On he oher hand, when he ransiion equaion is valid for as oday and as we know ha omorrow will be oday omorrow (or: oday urned ino yeserday omorrow - morgen is heue schon gesern ) and ha any fuure poin in ime will be oday a some poin, we could use as well. In mos cases, he choice of or as ime argumen is opporunisic: When a maximizaion mehod is used where he mos naural represenaion of budge consrains is in ; we will use : This is he case for he ransiion equaion (3.3.1) where he maximizaion problem is solved by using a Bellman equaion. Using is also mos naural when presening he seup of a model as in ch. 3.6. If by conras, he explici use of many ime periods is required in a maximizaion 299

300 Chaper 12. Miscellanea, references and index problem (like when using he Lagrangian wih an ineremporal budge consrain as in (3.1.3) or wih a sequence of dynamic budge consrains in (3.7.2)), budge consrains are expressed wih ime as argumen. Using as argumen in ransiion equaions is also more appropriae for Hamilonian problems where he curren value Hamilonian is used - as hroughou his book. See, for example, he budge consrain (5.1.2) in ch. 5.1. In he more formal ch. 4 on di erenial equaions, i is also mos naural (as his is he case in basically all exbooks on di erenial equaions) o represen all di erenial equaion wih as argumen. When we solve di erenial equaions, we need boundary condiions. When we use iniial condiions, hey will be given by 0 : If we use a erminal condiion as boundary condiion, we use T > o denoe some fuure poin in ime. (We could do wihou 0 ; use as argumen and solve for 0 T: While his would be more consisen wih he res of he book, i would be less comparable o more specialized books on di erenial equaions. As we believe ha he inconsisency is no oo srong, we sick o he more sandard mahemaical noaion in ch. 4). Here is now a summary of poins in ime and generic ime a poin in ime sanding for oday (discree and coninuous ime) generic ime (for di erenial equaions in ch. 4 and for model presenaions) + 1 omorrow (discree ime) some fuure poin in ime, (discree and coninuous ime) generic ime (for di erenial equaions in some maximizaion problems - e.g. when Hamilonians are used) 0 a poin in ime, usually in he pas (di erenial equaions) T a poin in ime, usually in he fuure (di erenial equaions and erminal poin for planning horizon) 0 oday when is normalized o zero (only in he examples of ch. 5.5.1) More on noaion The noaion is as homogeneous as possible. excepions are possible. Here are he general rules bu some Variables in capial, like capial K or consumpion C denoe aggregae quaniies, lower-case leers perain o he household level A funcion f (:) is always presened by using parenheses (:) ; where he parenheses give he argumens of funcions. Brackes [:] always denoe a muliplicaion operaion A variable x denoes he value of x in period in a discree ime seup. A variable x () denoes is value a a poin in ime in a coninuous ime seup. We will sick o his disincion in his exbook boh in deerminisic and sochasic seups. (Noe ha he mahemaical lieraure on sochasic coninuous ime processes, i.e. wha is reaed here in par IV, uses x o express he value of x a a poin in ime :)

301 A do indicaes a ime derivaive, _x () dx=d: A derivaive of a funcion f (x) where x is a scalar and no a vecor is abbreviaed by f 0 (x) df (x) =dx: Parial derivaives of a funcion f (x; y) are denoed by for example, f x (x; y) = f x (:) = f x @f (x; y) =@x: Variables Here is a lis of variables and abbreviaions which shows ha some variables have muli-asking abiliies, i.e. hey have muliple meanings Greek leers oupu elasiciy of capial = 1= (1 + ) discoun facor in discree ime preference parameer on rs period uiliy in wo-period models depreciaion rae share of wealh held in he risky asse i share of wealh held in asse i insananeous pro s, i.e. pro s in period or () a poin in ime ime preference rae, correlaion coe cien beween random variables ij correlaion coe cien beween wo random variables i and j see above on he concep of ime i he share of savings used for buying sock i " > misprin adjusmen cos funcion

302 Chaper 12. Miscellanea, references and index Lain leers fc g he ime pah of c from o in niy, i.e. for all, fc g fc ; c +1 ; :::g fc ()g he ime pah of c from o in niy, i.e. for all CVF change of variable formula e expendiure e = pc; e or, exponenial funcion f (x + y) a funcion f (:) wih argumen x + y f [x + y] some variable f imes x + y q () Poisson process r ineres rae RBC real business cycle RV random variable SDE sochasic di erenial equaion ; T; 0 see above on he concep of ime TVC ransversaliy condiion TFP oal facor produciviy u insananeous uiliy (see also ) w ; w L wage rae in period ; facor reward for labour. w L is used o sress di erence o w K w K facor reward for capial in period x xpoin of a di erence or di erenial equaion (sysem) z () Brownian moion, Wiener process

Bibliography Aase, K. K. (1984): Opimum Porfolio Diversi caion in a General Coninuous-Time Model, Sochasic Processes and heir Applicaions, 18, 81 98. Abel, A. B. (1990): Aggregae Asse Pricing; Asse Prices under Habi Formaion and Caching up he Joneses, American Economic Review; AEA Papers and Proceedings, 80(2), 38 42. Acemoglu, D., and R. Shimer (1999): E cien Unemploymen Insurance, Journal of Poliical Economy, 107, 893 928. Aghion, P., and P. Howi (1992): A Model of Growh Through Creaive Desrucion, Economerica, 60, 323 351. Aiyagari, S. R. (1994): Uninsured Idiosyncraic Risk and Aggregae Saving, Quarerly Journal of Economics, 109, 659 84. Andolfao, D. (1996): Business Cycles and Labor-Marke Search, American Economic Review, 86, 112 32. Araujo, A., and J. A. Scheinkman (1983): Maximum principle and ransversaliy condiion for concave in nie horizon economic models, Journal of Economic Theory, 30, 1 16. Arnold, L. (1973): Sochasische Di erenialgleichungen. Theorie und Anwendung. Oldenbourg Verlag München/ Wien. Arrow, K. J., and M. Kurz (1969): Opimal consumer allocaion over an in nie horizon, Journal of Economic Theory, 1, 68 91. Arrow, K. J., and M. Kurz (1970): Public invesmen, he rae of reurn and opimal scal policy. Johns Hopkins Universiy Press, Balimore. Akinson, A. B., and J. E. Sigliz (1980): Lecures on Public Economics. McGraw-Hill, New York. Azariadis, C. (1993): Ineremporal Macroeconomics. Basil Blackwell, Oxford. 303

304 BIBLIOGRAPHY Barro, R. J., N. G. Mankiw, and X. Sala-i Marin (1995): Capial Mobiliy in Neoclassical Models of Growh, American Economic Review, 85, 103 15. Bayer, C., and K. Wälde (2010a): Maching and Saving in Coninuous Time : Theory, CESifo Working Paper 3026. (2010b): Maching and Saving in Coninuous Time: Numerical Soluion, mimeo - www.waelde.com/pub. (2010c): Maching and Saving in Coninuous Time: Proofs, CESifo Working Paper 3026-A. Bellman, R. (1957): Dynamic Programming. Princeon Universiy Press, Princeon. Benhabib, J., and A. Rusichini (1994): A noe on a new class of soluions o dynamic programming problems arising in economic growh, Journal of Economic Dynamics and Conrol, 18, 807 813. Benal, B., and D. Peled (1996): The Accumulaion of Wealh and he Cyclical Generaion of New Technologies: A Search Theoreic Approach, Inernaional Economic Review, 37, 687 718. Benolila, S., and G. Berola (1990): Firing Coss and Labour Demand: How Bad Is Eurosclerosis?, Review of Economic Sudies, 57(3), 381 402. Berola, G., L. Guiso, and L. Pisaferri (2005): Uncerainy and Consumer Durables Adjusmen, Review of Economic Sudies, 72, 973 1007. Bils, M., Y. Chang, and S.-B. Kim (2007): Comparaive Advanage in Cyclical Unemploymen, NBER Working Paper 13231. (2009): Heerogeneiy and Cyclical Unemploymen, NBER Working Paper 15166. Björk, T. (2004): Arbirage Theory in Coninuous Time, 2nd ed. Oxford Universiy Press. Black, F., and M. Scholes (1973): The Pricing of Opions and Corporae Liabilies, Journal of Poliical Economy, 81, 637 659. Blackorby, C., and R. R. Russell (1989): Will he Real Elasiciy of Subsiuion Please Sand Up? (A Comparison of he Allen/Uzawa and Morishima Elasiciies), American Economic Review, 79(4), 882 88. Blanchard, O., and S. Fischer (1989): Lecures on Macroeconomics. MIT Press. Blanchard, O. J. (1985): Deb, De cis, and Finie Horizons, Journal of Poliical Economy, 93, 223 47.

BIBLIOGRAPHY 305 Bloom, N. (2007): Uncerainy and he Dynamics of R&D, American Economic Review, 97(2), 250 255. Bossmann, M., C. Kleiber, and K. Wälde (2007): Bequess, axaion and he disribuion of wealh in a general equilibrium model, Journal of Public Economics, 91, 1247 1271. Braun, M. (1975): Di erenial equaions and heir applicaions (shor version). Springer, Berlin. Brennan, M. J. (1979): The pricing of coningen claims in discree ime models, Journal of Finance, pp. 53 68. Brock, W. A., and A. G. Malliaris (1989): Di erenial Equaions, Sabiliy and Chaos in Dynamic Economics, Advanced Texbooks in Economics. Buier, W. H. (1981): Time Preference and Inernaional Lending and Borrowing in an Overlappig-Generaions Model, Journal of Poliical Economy, 89, 769 797. Buier, W. H., and A. C. Siber (2007): De aionary Bubbles, Macroeconomic Dynamics, 11(04), 431 454. Burde, K., and D. T. Morensen (1998): Wage Di erenials, Employer Size, and Unemploymen, Inernaional Economic Review, 39, 257 273. Chang, F.-R. (2004): Sochasic Opimizaion in Coninuous Time. Cambridge Universiy Press, Cambridge, UK. Chen, Y.-F., and M. Funke (2005): Non-Wage Labour Coss, Policy Uncerainy And Labour Demand - A Theoreical Assessmen, Scoish Journal of Poliical Economy, 52, 687 709. Chiang, A. C. (1984): Fundamenal Mehods of Mahemaical Economics. McGraw-Hill, Third Ediion, New York. (1992): Elemens of dynamic opimizaion. McGraw-Hill, New York. Chow, G. C. (1997): Dynamic Economics. Opimizaion by he Lagrange Mehod. Oxford Universiy Press, Oxford. Cox, J. C., J. E. Ingersoll, and S. A. Ross (1985): An Inermporal General Equilibrium Model of Asse Prices, Economerica, 53(2), 363 384. Danhine, Jean Pierre Donaldson, J. B., and R. Mehra (1992): he equiy premium and he allocaion of income risk, Journal of Economic Dynamics and Conrol, 16, 509 532. Das, S. R., and S. Foresi (1996): Exac Soluions for Bond and Opion Prices wih Sysemaic Jump Risk, Review of Derivaives Research, 1, 7 24.

306 BIBLIOGRAPHY de la Croix, D., and P. Michel (2002): A heory of economic growh: dynamics and policy in overlapping generaions. Cambridge Universiy Press. DeMarzo, P. M., and B. Urošević (2006): Ownership Dynamics and Asse Pricing wih a Large Shareholder, Journal of Poliical Economy, 114, 774 815. Dempser, M. A. H. (1991): Opimal Conrol of Piecewise Deerminisic Markov Processes, in Applied Sochasic Analysis, ed. by M. H. A. Davis, and R. J. Ellio, pp. 303 325. Gordon and Breach, New York. Dierker, E., and B. Grodahl (1995): Pro maximizaion miigaes compeiion, Economic Theory, 7, 139 160. Dinopoulos, E., and P. Thompson (1998): Schumpeerian Growh Wihou Scale Effecs, Journal of Economic Growh, 3, 313 335. Dixi, A. K. (1990, 2nd ed.): Opimizaion in economic heory. Oxford Universiy Press. Dixi, A. K., and R. S. Pindyck (1994): Invesmen Under Uncerainy. Princeon Universiy Press. Du e, D. (1988): Securiy Markes - Sochasic Models. Academic Press. (2001): Dynamic Asse Pricing Theory, 3rd ed. Princeon Universiy Press. Eaon, J. (1981): Fiscal Policy, In aion and he Accumulaion of Risky Capial, Review of Economic Sudies, 48(153), 435 445. Epaulard, A., and A. Pommere (2003): Recursive Uiliy, Endogenous Growh, and he Welfare Cos of Volailiy, Review of Economic Dynamics, 6(2), 672 684. Epsein, L. G., and S. E. Zin (1989): Subsiuion, Risk Aversion, and he Temporal Behavior of Consumpion and Asse Reurns: A Theoreical Framework, Economerica, 57(4), 937 69. (1991): Subsiuion, Risk Aversion, and he Temporal Behavior of Consumpion and Asse Reurns: An Empirical Analysis, Journal of Poliical Economy, 99(2), 263 86. Evans, Hasings, and Peacock (2000): Saisical Disribuions, 3rd ediion. Wiley. Fan, J. (1995): Endogenous Technical Progress, R and D Periods and Duraions of Business Cycles, Journal of Evoluionary Economics, 5, 341 368. Farzin, Y. H., K. Huisman, and P. Kor (1998): Opimal Timing of Technology Adopion, Journal of Economic Dynamics and Conrol, 22, 779 799.

BIBLIOGRAPHY 307 Feichinger, G., and R. F. Harl (1986): Opimale Konrolle Ökonomischer Prozesse: Anwendungen des Maximumprinzips in den Wirschafswissenschafen. de Gruyer, Berlin. Fichenholz (1997): Di erenial und Inegralrechnung, 14. Au age. Harri Deusch. Framsad, N. C., B. Øksendal, and A. Sulem (2001): Opimal Consumpion and Porfolio in a Jump Di usion Marke wih Proporional Transacion Coss, Journal of Mahemaical Economics, 35, 233 257. Francois, P., and H. Lloyd-Ellis (2003): Animal Spiris Trough Creaive Desrucion, American Economic Review, 93, 530 550. (2008): Implemenaion Cycles, Invesmen, And Growh, Inernaional Economic Review, 49(3), 901 942. Frederick, S., G. Loewensein, and T. O Donoghue (2002): Time Discouning and Time Preference: A Criical Review, Journal of Economic Lieraure, 40(2), 351 401. Freeman, S., D. P. Hong, and D. Peled (1999): Endogenous Cycles and Growh wih Indivisible Technological Developmens, Review of Economic Dynamics, 2, 403 432. Fudenberg, D., and J. Tirole (1991): Game Theory. MIT Press. Gabaix, X. (1999): Zipf s Law For Ciies: An Explanaion, Quarerly Journal of Economics, 114(3), 739 767. Gabaix, X., P. Gopikrishnan, V. Plerou, and H. E. Sanley (2006): Insiuional Invesors and Sock Marke Volailiy, Quarerly Journal of Economics, 121, 461 504. Gabszewicz, J., and J. P. Vial (1972): Oligopoly "à la Courno" in a general equilibrium analysis, Journal of Economic Theory, 4, 381 400. Galor, O., and O. Moav (2006): Das Human-Kapial: A Theory of he Demise of he Class Srucure, Review of Economic Sudies, 73(1), 85 117. Galor, O., and J. Zeira (1993): Income Disribuion and Macroeconomics, Review of Economic Sudies, 60, 35 52. Gandolfo, G. (1996): Economic Dynamics, 3rd ediion. Springer, Berlin. García, M. A., and R. J. Griego (1994): An Elemenary Theory of Sochasic Di erenial Equaions Driven by A Poisson Process, Communicaions in Saisics - Sochasic Models, 10(2), 335 363. Gihman, I. I., and A. V. Skorohod (1972): Sochasic Di erenial Equaions. Springer- Verlag, New York.

308 BIBLIOGRAPHY Gong, L., W. Smih, and H.-f. Zou (2007): Consumpion and Risk wih hyperbolic discouning, Economics Leers, 96(2), 153 160. Goodman, J. (2002): Sochasic Calculus Noes, Lecure 5, hp://www.mah.nyu.edu/faculy/goodman/, p. downloaded January 2010. Grinols, E. L., and S. J. Turnovsky (1998): Consequences of deb policy in a sochasically growing moneary economy, Inernaional Economic Review, 39(2), 495 521. Grossman, G. M., and E. Helpman (1991): Innovaion and Growh in he Global Economy. The MIT Press, Cambridge, Massachuses. Grossman, G. M., and C. Shapiro (1986): Opimal Dynamic R&D Programs, RAND Journal of Economics, 17, 581 593. Guo, X., J. Miao, and E. Morelle (2005): Irreversible invesmen wih regime shifs, Journal of Economic Theory, 122, 37 59. Guriev, S., and D. Kvasov (2005): Conracing on Time, American Economic Review, 95(5), 1369 1385. Hasse, K. A., and G. E. Mecalf (1999): Invesmen wih Uncerain Tax Policy: Does Random Tax Discourage Invesmen?, Economic Journal, 109, 372 393. Hassler, J., and J. V. R. Mora (1999): Employmen urnover and he public allocaion of unemploymen insurance, Journal of Public Economics, 73, 55 83. (2008): Unemploymen insurance design: Inducing moving and reraining, European Economic Review, 52, 757 791. Hassler, J., J. V. R. Mora, K. Soresleen, and F. Ziliboi (2005): A Posiive Theory of Geographic Mobiliy and Social Insurance, Inernaional Economic Review, 46(1), 263 303. Heer, B., and A. Maußner (2005): Dynamic General Equilibrium Modelling, Compuaional Mehods and Applicaions. Springer, Berlin. Inriligaor, M. D. (1971): Mahemaical opimizaion and economic heory. Englewood Cli s, New Jersey. Ireland, P. N. (2004): Technology Shocks in he New Keynesian Model, Nber working papers 10309, Naional Bureau of Economic Research. Jermann, U. J. (1998): Asse pricing in producion economies, Journal of Moneary Economics, 41, 257 275. Johnson, N. L., S. Koz, and N. Balakrishnan (1995): Coninuous Univariae Disribuions, Vol. 2, 2nd ediion. Wiley.

BIBLIOGRAPHY 309 Judd, K. L. (1985): The Law of Large Numbers wih a Coninuum of I.I.D. Random Variables, Journal of Economic Theory, 35, 19 25. Kamien, M. I., and N. L. Schwarz (1991): Dynamic opimizaion. The calculus of variaions and opimal conrol in economics and managemen. 2nd ediion. Amserdam, Norh-Holland. Kamihigashi, T. (2001): Necessiy of Transversaliy Condiions for In nie Horizon Problems, Economerica, 69, 995 1012. Keller, G., and S. Rady (1999): Opimal Experimenaion in a Changing Environmen, Review of Economic Sudies, 66, 475 507. Kiyoaki, N., and R. Wrigh (1993): A Search-Theoreic Approach o Moneary Economics, American Economic Review, 83, 63 77. Kleiber, C., and S. Koz (2003): Saisical Size Disribuions in Economics and Acuarial Sciences. Wiley. Krusell, P., T. Mukoyama, and A. Sahin (2010): Labor-Marke Maching wih Precauionary Savings and Aggregae Flucuaions, Review of Economic Sudies, forhcoming. Kushner, H. J. (1967): Sochasic Sabiliy and Conrol. Acad. Press, New York. Kyle, A. S., H. Ou-Yang, and W. Xiong (2006): Prospec heory and liquidaion decisions, Journal of Economic Theory, 129, 273½U288. Lainer, J., and D. Solyarov (2004): Aggregae reurns o scale and embodied echnical change: heory and measuremen using sock marke daa, Journal of Moneary Economics, 51, 191 233, pdf. Leonard, D., and N. V. Long (1992): Opimal Conrol Theory and Saic Opimizaion in Economics. Cambridge Universiy Press, New York. Li, C.-W. (2001): Science, Diminishing Reurns, and Long Waves, The Mancheser School, 69, 553 573. Ljungqvis, L., and T. J. Sargen (2004): Recursive Macroeconomic Theory 2nd ed. MIT Press. Long, J. B. Jr.., and C. I. Plosser (1983): Real Business Cycles, Journal of Poliical Economy, 91, 39 69. Lucas, R. E. Jr.. (1978): Asse Prices in an Exchange Economy, Economerica, 46, 1429 1445.

310 BIBLIOGRAPHY (1988): On he Mechanics of Economic Developmen, Journal of Moneary Economics, 22, 3 42. Maenhou, P. J. (2006): Robus porfolio rules and deecion-error probabiliies for a mean-revering risk premium, Journal of Economic Theory, 128, 136½U163. Maliar, L., and S. Maliar (2004): Endogenous Growh and Endogenous Business Cycles, Macroeconomic Dynamics, 8, 559 581. Mas-Colell, A., M. D. Whinson, and J. R. Green (1995): Microeconomic Theory. Oxford Universiy Press. Masuyama, K. (1999): Growing Through Cycles, Economerica, 67, 335 347. McDonald, R., and D. Siegel (1986): The Value of Waiing o Inves, Quarerly Journal of Economics, 101, 707 27. Meron, R. C. (1969): Lifeime Porfolio Selecion Under Uncerainy: The Coninuous- Time Case, Review of Economics and Saisics, 51, 247 257. (1971): Opimum Consumpion and Porfolio Rules in a Coninuous-Time Model, Journal of Economic Theory, 3, 373 413. (1976): Opion Pricing When Underlying Sock Reurns are Disconinuous, Journal of Financial Economics, 3, 125 144. Merz, M. (1995): Search in he labor marke and he real business cycle, Journal of Moneary Economics, 36, 269 300. Michel, P. (1982): On he Transversaliy Condiion in In nie Horizon Opimal Problems, Economerica, 50, 975 985. Mirinovic, D. (1970): Analyic Inequaliies. Springer, Berlin. Moscarini, G. (2005): Job Maching and he Wage Disribuion, Economerica, 73(2), 481 516. Muro, P. (2004): Exi in Duopoly under Uncerainy, RAND Journal of Economics, 35, 111 127. Nakajima, M. (2008): Business Cycle in he Equilibrium Model of Labor Marke Search and Self-Insurance, mimeo FED Philadelphia. Obsfeld, M. (1994): Risk-Taking, Global Diversi caion, and Growh, American Economic Review, 84, 1310 29. Øksendal, B. (1998): Sochasic Di erenial Equaions. Springer, Fifh Ediion, Berlin.

BIBLIOGRAPHY 311 Øksendal, B., and A. Sulem (2005): Springer-Verlag, Berlin. Applied Sochasic Conrol of Jump Di usions. Palokangas, T. (2003): Common Markes, Economic Growh, and Creaive Desrucion, Journal of Economics, 10 (Supplemen "Growh, Trade, and Economic Insiuions"), 57 76. (2005): Inernaional Labor Union policy and Growh wih Creaive Desrucion, Review of Inernaional Economics, 13, 90 105. Phillips, K. L., and J. Wrase (2006): Is Schumpeerian creaive desrucion a plausible source of endogenous real business cycle shocks?, Journal of Economic Dynamics and Conrol, 30(11), 1885 1913. Piazzesi, M. (2005): Bond Yields and he Federal Reserve, Journal of Poliical Economy, 113(2), 311 344. Pissarides, C. A. (1985): Shor-run Equilibrium Dynamics of Unemploymen Vacancies, and Real Wages, American Economic Review, 75, 676 90. Pissarides, C. A. (2000): Equilibrium Unemploymen Theory. MIT Press, Cambridge, Massachuses. Pommere, A., and W. T. Smih (2005): Feriliy, volailiy, and growh, Economics Leers, 87(3), 347 353. Pra, J. (2007): The impac of disembodied echnological progress on unemploymen, Review of Economic Dynamics, 10, 106 125. Proer, P. (1995): Sochasic Inegraion and Di erenial Equaions. Springer-Verlag, Berlin. Ramsey, F. (1928): A Mahemaical Theory of Saving, Economic Journal, 38, 543 559. Rebelo, S. (1991): Long-Run Policy Analysis and Long-Run Growh, Journal of Poliical Economy, 99(3), 500 521. Ross, S. M. (1993): Inroducion o Probabiliy Models, 5h ediion. Academic Press, San Diego. (1996): Sochasic processes, 2nd ediion. Academic Press, San Diego. Roemberg, J. J. (1982): Sicky Prices in he Unied Saes, Journal of Poliical Economy, 90(6), 1187 1211. Rouwenhors, K. G. (1995): Asse Pricing Implicaions fo Equilibrium Business Cycle Models. Princeon Universiy press, in Froniers of Business Cycles, ed. Thomas F. Cooley, 294-330, New Jersey.

312 BIBLIOGRAPHY Sananu Chaerjee, P. G., and S. J. Turnovsky (2004): Capial Income Taxes and Growh in a Sochasic Economy: A Numerical Analysis of he Role of Risk Aversion and Ineremporal Subsiuion, Journal of Public Economic Theory, 6(2), 277 310. Schlegel, C. (2004): Analyical and Numerical Soluion of a Poisson RBC model, Dresden Discussion Paper Series in Economics, 5/04. Sennewald, K. (2007): Conrolled Sochasic Di erenial Equaions under Poisson Uncerainy and wih Unbounded Uiliy, Journal of Economic Dynamics and Conrol, 31, 1106 1131. Sennewald, K., and K. Wälde (2005): "Io s Lemma" and he Bellman equaion for Poisson processes: An applied view, Journal of Economics, forhcoming. Severini, T. A. (2005): Elemens of Disribuion Theory. Cambridge Universiy Press. Shapiro, C., and J. E. Sigliz (1984): Equilibrium Unemploymen as a Worker Discipline Device, American Economic Review, 74, 433 44. Shell, K. (1969): Applicaions of Ponryagin s Maximum Principle o Economics. Springer, Berlin. Shimer, R. (2005): The Cyclical Behavior of Equilibrium Unemploymen and Vacancies, American Economic Review, 95, 25 49. (2007): Mismach, American Economic Review, 97(4), 1074 1101. (2008): The Probabiliy of Finding a Job, American Economic Review, 98(2), 268 73. Shimer, R., and I. Werning (2007): Reservaion Wages and Unemploymen Insurance, Quarerly Journal of Economics, 122, 1145 1185. (2008): Liquidiy and Insurance for he Unemployed, American Economic Review, 98, 1922 1942. Smih, W. T. (1996): Feasibiliy and Transversaliy Condiions for Models of Porfolio Choice wih Non-Expeced Uiliy in Coninuous Time, Economics Leers, 53, 123 31. Spanos, A. (1999): Probabiliy Theory and Saisical Inference - Economeric Modeling wih Observaional Daa. Cambridge Universiy Press. Seger, T. M. (2005): Sochasic growh under Wiener and Poisson uncerainy, Economics Leers, 86(3), 311 316. Sein, J. L. (2006): Sochasic Opimal Conrol, Inernaional Finance, and Deb Crises. Oxford Universiy Press.

BIBLIOGRAPHY 313 Sokey, N. L., R. E. Lucas Jr., and E. C. Presco (1989): Recursive Mehods in Economics Dynamics. Harvard Universiy Press, Cambridge, MA. Sroz, R. (1955/56): Myopia and Inconsisency in Dynamic Uiliy Maximizaion, Review of Economic Sudies, 23(3), 165 180. Svensson, L. E. O. (1989): Porfolio choice wih non-expeced uiliy in coninuous-ime, Economics Leers, 3, 313 317. Thompson, P., and D. Waldo (1994): Growh and rusi ed capialism, Journal of Moneary Economics, 34, 445 462. Toche, P. (2005): A racable model of precauionary saving in coninuous ime, Economics Leers, 87, 267 272. Turnovsky, S. J. (1997): Inernaional Macroeconomic Dynamics. MIT Press. (2000): Mehods of Macroeconomic Dynamics. MIT Press. Turnovsky, S. J., and W. T. Smih (2006): Equilibrium consumpion and precauionary savings in a sochasically growing economy, Journal of Economic Dynamics and Conrol, 30(2), 243 278. Varian, H. (1992): Microeconomic Analysis. Noron, Third Ediion. Venegaz-Marinez, F. (2001): Temporary Sabilizaion: A Sochasic Analysis, Journal of Economic Dynamics and Conrol, 25, 1429 1449. Vervaa, W. (1979): On a sochasic di erence equaion and a represenaion of nonnegaive in niely divisible random variables, Advances in Applied Probabiliy, 11, 750 783. Vissing-Jorgensen, A. (2002): Limied Asse Marke Paricipaion and he Elasiciy of Ineremporal Subsiuion, Journal of Poliical Economy, 110, 825 853. von Weizsäcker, C. C. (1965): Exisence of Opimal Programs of Accumulaion for an In nie Time Horizon, Review of Economic Sudies, 32(2), 85 104. Wälde, K. (2002): The Economic Deerminans of Technology Shocks in a Real Business Cycle Model, Journal of Economic Dynamics & Conrol, 27, 1 28. Walsh, C. E. (2003): Moneary Theory and Policy. MIT Press. Wang, N. (2006): Generalizing he permanen-income hypohesis: Revisiing Friedman s conjecure on consumpion, Journal of Moneary Economics, 53(4), 737 752. (2007): An equilibrium model of wealh disribuion, Journal of Moneary Economics, 54(7), 1882 1904.

314 BIBLIOGRAPHY Weil, P. (1990): Nonexpeced Uiliy in Macroeconomics, Quarerly Journal of Economics, 105(1), 29 42. Wälde, K. (1996): Proof of Global Sabiliy, Transiional Dynamics, and Inernaional Capial Flows in a Two-Counry Model of Innovaion and Growh, Journal of Economics, 64, 53 84. (1999a): A Model of Creaive Desrucion wih Undiversi able Risk and Opimising Households, Economic Journal, 109, C156 C171. (1999b): Opimal Saving under Poisson Uncerainy, Journal of Economic Theory, 87, 194 217. 894. (2005): Endogenous Growh Cycles, Inernaional Economic Review, 46, 867 (2008): Corrigendum o Opimal Saving under Poisson Uncerainy, Journal of Economic Theory, 140, e360 e364. (2010): Producion echnologies in sochasic coninuous ime models, forhcoming Journal of Economic Dynamics and Conrol. Xie, D. (1991): Increasing Reurns and Increasing Raes of Growh, Journal of Poliical Economy, 99, 429 435. (1994): Divergence in Economic Performance: Transiional Dynamics wih Muliple Equilibria, Journal of Economic Theory, 63, 97 112.

Index adjusmen coss deerminisic, coninuous ime, 117 sochasic, discree ime, 212, 213 backward soluion, see ordinary di erenial equaion Bellman equaion adjusing he basic form, 281 basic form, 268 deerminisic coninuous ime, 142 discree ime, 50, 53, 58 sochasic coninuous ime, 280, 281, 285 discree ime, 198, 200, 202 Bellman equaion, basic srucure sochasic coninuous ime, 268 Bellman equaion, maximized de niion, 51, 270 deerminisic discree ime, 58 sochasic coninuous ime, 144, 280 discree ime, 198, 200 boundary condiion iniial condiion, 30, 31, 33, 35, 36, 59, 65, 80, 82, 84, 95, 97, 114, 118, 128, 132, 171, 181, 243, 247, 253, 272, 289 erminal condiion, 59, 80, 96, 101, 114, 115 boundedness condiion, 99, 113, 123 Brownian moion, 225, 226, 228, 233, 242, 251, 255, 280, 302 de niion, 226 sandard Brownian moion, 228 budge consrain derivaion in coninuous ime, 98, 240 derivaion in discree ime, 37 dynamic, 57, 59, 72, 98, 105, 107, 122, 197, 199, 247, 266, 267, 280, 284 de niion, 59 from dynamic o ineremporal coninuous ime, 99 discree ime, 52, 60 from ineremporal o dynamic coninuous ime, 105 discree ime, 72 ineremporal, 12, 28, 45, 52, 60, 72, 99, 105, 121, 248, 266 de niion, 59 capial asse pricing (CAP), 188, 283 CARA uiliy, see insananeous uiliy funcion cenral planner, 25, 48, 72, 137, 201, 217 CES uiliy, see insananeous uiliy funcion closed-form soluion de niion, 15 deerminisic coninuous ime, 123 discree ime, 15 sochasic Cobb-Douglas uiliy, 178 coninuous ime, 272, 292 CRRA uiliy funcion, 195 discree ime, 178, 185, 217 closed-loop soluion, 15 315

316 INDEX Cobb-Douglas uiliy, see insananeous uiliy funcion condiional disribuion, 169, 170 consumpion level and ineres rae, 123 conrol variable, 110, 132, 144, 177, 209, 211, 280 de niion, 50 cosae variable de niion, 52 dynamic programming coninuous ime, 144, 146, 269, 274, 281 discree ime, 51, 54, 199 Hamilonian, 108, 111, 117, 132 inerpreaion, 145 inerpreaion for Hamilonian, 111 CRRA uiliy, see insananeous uiliy funcion CVF (change of variable formula), 232, 237, 241, 269, 271 densiy, 162, 163, 170, 189 of a funcion of a random variable, 166 depreciaion rae, 28 di erence equaion de niion of a soluion, 33 deerminisic, 32, 35, 49, 55, 59 non-linear, 30, 65 expecaional, 200 sochasic, 168, 172, 180 di erenial equaion (DE), see ordinary DE or sochasic DE elasiciy of subsiuion alernaive expression, 18 de niion, 18 envelope heorem, 47 applicaion, 49, 54, 144, 198, 202, 270, 274, 281 Euler equaion, 17, 55, 108, 200, 203, 206, 215 Euler heorem, 26, 27 expecaions operaor coninuous ime, 255, 259, 277, 278 discree ime, 169, 177, 179, 200, 210 expeced value, 252, 259 feliciy funcion, see insananeous uiliy funcion, 14, 126 rms, ineremporal opimizaion, 116, 212 rs-order condiion wih economic inerpreaion, 51, 54, 58, 130, 143, 178, 184, 198, 213, 269, 274, 285 xpoin, 36, 86 wo-dimensional, 84 uniquness, 84 forward soluion, see ordinary di erenial equaion goods marke equilibrium, 28 Hamilonian, 107, 111 curren-value, 108, 112, 117, 130, 145 presen-value, 108, 132, 134 heerogeneous agens, 171, 183 idenically and independenly disribued (iid), 168, 171, 175 iniial condiion, see boundary condiion insananeous uiliy funcion CARA, 182 CES, 53 Cobb-Douglas, 14, 178 CRRA, 182, 195 exponenial, 182 logarihmic as a special case of CES, 55 inegral represenaion of di erenial equaions, 97, 232, 256, 259, 277 inegraion by pars, 94, 105, 110, 133 ineres rae, de niion coninuous ime, 98, 100 discree ime, 29, 38 ineremporal elasiciy of subsiuion, 18 CES uiliy funcion, 19 coninuous ime, 122 de niion, 19 empirical esimaes, 124, 135

INDEX 317 logarihmic uiliy funcion, 19 ineremporal uiliy maximizaion, 11, 45, 52, 107, 141, 176, 197, 209, 267, 280 invesmen gross, 28 ne, 28 Io s Lemma, 232, 245, 281 Keynes-Ramsey rule, 108, 122, 127, 144, 271, 275 L Hôspial s rule, 55 Lagrange muliplier, 13, 21, 25, 66, 109, 121, 145, 207 de niion, 23 sign, 13, 26, 67 Lagrangian coninuous ime, 109, 121, 133 derivaion, 24, 68 in nie number of consrains, 66, 69, 207 in nie-horizon problem, 46 saic problem, 25 wo-period problem, 13, 14, 16 Leibniz rule, 93 level e ec dynamic, 59, 123 log uiliy, see insananeous uiliy funcion marginal rae of subsiuion and ime preference rae, 20 de niion, 17 maringale, 254, 255, 278 maching, 129 large rms, 129 value funcions, 278 maximisaion problem wihou soluion, 119 naural volailiy, 189, 271, 286 no-ponzi game condiion, 60, 61, 99, 115, 249 numeraire good, 14, 63, 202, 206, 287 numerical soluion, 215 ordinary di erenial equaion backward soluion, 95 de niion, 79 de niion of soluion, 94 forward soluion, 95 Poisson process, 225, 227, 236, 260, 263, 267 mean and variance, 252, 259, 262 Poisson processes dependen jump processes, 263 independence, 237 random variable funcions of and heir densiies, 166 random walk, 181 reduced form, de niion, 29 represenaive agen assumpion, 183 resource consrain, 25, 28, 48, 63, 66, 126, 191, 203, 290 risk aversion absolue, 182 relaive, 182 shadow price, 24 26, 52, 111, 130 soluion de niion, 15 vs Euler equaion, 17 vs necessary condiion, 17 solvency condiion, see also no-ponzi game condiion, 99 solving a maximizaion problem, 17 solving by insering principle, 15 sae variable, 55, 110, 114, 132, 145, 198, 209, 269, 281 de niion, 50, 56 seady sae, 119, 128, 150 and limiing disribuion, 170 coninuous ime, 86, 87 de niion, 65 discree ime, 30, 31, 67 emporary, 192, 289 sicky prices, 212

318 INDEX sochasic di erenial equaion, 228, 229, 234, 236, 242, 245 soluion, 242, 243, 245, 247 sochasic di erenial equaions de niion of soluion, 242 subsiuion, see elasiciy of subsiuion, see ineremporal elasiciy of subsiuion, see marginal rae of subsiuion swich process, 249 erminal condiion, see boundary condiion ime preference rae and discoun facor, 20 de niion, 20 ransversaliy condiion, 99, 102, 115, 292 variance de niion, 165 of Brownian moion, 226, 256 of capial sock, 181 of coninuous-ime process, 252 of discree-ime process, 169 of lognormal disribuion, 167 of Poisson process, 252, 259 properies, 165 Wiener process, 226, 302 zero-moion line, 85, 87, 128

.

op - download a repec see hp://logec.repec.org! Top Books This exbook provides all ools required o easily solve ineremporal opimizaion problems in economics, nance, business adminisraion and relaed disciplines. The focus of his exbook is on learning hrough examples : an example is worked ou rs and he mahemaical principles are explained laer. I gives a very quick access o all mehods required by a Bachelor, Maser or a PhD suden, and an experienced researcher who wans o explore new elds or con rm exising knowledge. Given ha discree and coninuous ime problems are given equal aenion, insighs gained in one area can be used o learn soluion mehods more quickly in oher conexs. This sep-by-sep approach is especially useful for he ransiion from deerminisic o sochasic worlds. When i comes o sochasic mehods in coninuous ime, he applied focus of his book is he mos useful. Formulaing and solving problems under coninuous ime uncerainy has never been explained in such a non-echnical and highly accessible way. The book is compleed by an exensive index which helps nding opics of ineres very quickly. ISBN 978-3-00-032428-4