Discrete Dynamic Optimization: Six Examples Dr. Tai-kuang Ho Associate Professor. Department of Quantitative Finance, National Tsing Hua University, No. 101, Section 2, Kuang-Fu Road, Hsinchu, Taiwan 30013, Tel: +886-3-571-5131, ext. 62136, Fax: +886-3-562-1823, E-mail: tkho@mx.nthu.edu.tw.
Example 1: the neoclassical growth model Uhlig (1999), section 4 Model principles Specify the environment explicitly: 1. Preferences 2. Technologies
3. Endowments 4. Information State the object of study: 1. The social planner s problem 2. The competition equilibrium 3. The game
The environment: Preferences: the representative agent experiences utility according to U = E t 2 1X 4 tc1 3 t 1 5 t=0 1 0 < < 1 > 0
Absolute risk aversion: u 00 (c) u 0 (c) Relative risk aversion: cu 00 (c) u 0 (c) is the coe cient of relative risk aversion Technologies: we assume a Cobb-Douglas production function Y t = Z t K t N 1 t
Budget constraint: K t+1 = I t + (1 ) K t C t + I t = Y t or equivalently C t + K t+1 = Z t Kt N t 1 + (1 ) K t
0 < < 1 0 < < 1 Z t, the total factor productivity, is exogenously evolving according to log Z t = (1 ) log Z + log Z t 1 + t t i:i:d:n 0; 2
0 < < 1 Endowment: N t = 1 K 0 Information: C t, N t, and K t+1 need to be chosen based on all information I t up to time t. The social planner s problem
The objective of the social planner is to maximize the utility of the representative agent subject to feasibility, max E (C t ;K t ) 1 t t=0 2 1X 4 tc1 3 t 1 5 t=0 1 s:t: K 0 ; Z 0 C t + K t+1 = Z t K t + (1 ) K t
log Z t = (1 ) log Z + log Z t 1 + t t i:i:d:n 0; 2 To solve it, form the Lagrangian: L = max (C t ;K t+1 ) 1 t=0e t 1 X t=0 2 4 tc1 t 1 1 3 + t t Zt Kt + (1 ) K t C t K t+1 5
The rst order conditions are: @L @ t : 0 = C t + K t+1 Z t K t (1 ) K t @L : 0 = C @C t t t @L @K t+1 : 0 = t + E t h t+1 Zt+1 K 1 t+1 + (1 )i
Notice that K t+1 appears in period t and period t + 1. The rst order conditions are often also called Euler equations. One also obtains the transversality condition. 0 = lim T!1 E 0 h T C T K i T +1 The transversality condition is a limiting Kuhn-Tucker condition.
It means that in net present value terms, the agent should not have capital left over at in nity. Rewrite the necessary conditions: C t = Z t K t + (1 ) K t K t+1 (1) R t = Z t K 1 t + (1 ) (2)
1 = E t " C t C t+1! R t+1# (3) log Z t = (1 ) log Z + log Z t 1 + t ; t i:i:d:n 0; 2 (4) Equation accounting. To solve for the steady state, dropping the time indices and yield
C = Z K + (1 ) K K R = Z K 1 + (1 ) 1 = R From the welfare theorems, the solution to the competitive equilibrium yields the same allocation as the solution to the social planner s problem.
The case for competitive equilibrium is provided in Uhlig (1999) section 4.4.
Example 2: Hansen s real business cycle model Uhlig (1999), section 4 Hansen s real business cycle model The model is an extension of the stochastic neoclassical growth model. The main di erence is to endogenize the labour supply. The social planner solves the problem of the representative agent:
X 1 max E t t t=1 0 @ C1 t 1 1 AN t 1 A s:t: C t + I t = Y t K t+1 = I t + (1 ) K t
Y t = Z t K t N 1 t log Z t = (1 ) log Z + log Z t 1 + t ; t i:i:d:n 0; 2 Hansen only consider the case = 1, so that the objective function is: E t 1 X t=1 t (log C t AN t )
Proof: lim!1 C 1 t 1 1 e (1 ) ln C t 1 = lim!1 1 e (1 ) ln C t ( ln C t ) = lim!1 1 @ h e (1 ) ln C t 1 i =@ = lim!1 @ [1 ] =@ = e0 ( ln C t ) 1 = ln C t To solve it, form the Lagrangian:
L = 2 X 1 max (C t ;N t ;K t+1 ) t=0e 1 t 6 4 t=0 t C 1! t 1 1 AN t + t t Zt Kt N t 1 + (1 ) K t C t K t+1 3 7 5 The rst order conditions are: @L = Ct t = 0 @C t @L @N t = A + t (1 ) Z t K t N t = 0
@L = t + E t t+1 Zt+1 K 1 @K t+1 N 1 t+1 + (1 ) = 0 t+1 After arrangement A = Ct (1 ) Y t (1) N t R t = Y t K t + 1 (2)
1 = E t " C t C t+1! R t+1# (3) The steady state for the real business cycle model above is obtained by dropping the time subscripts and stochastic shocks in the equations above. A = C (1 ) Y N 1 = R
R = Y K + 1 Exercise: Equation accounting.
Example 3: Brock-Mirman growth model I Chow (1997), chapter 2 Consider the Brock-Mirman growth model: max fc t g E t 1X t=0 t ln c t s:t:
k t+1 = k t z t c t The Lagrangian is: h t ln c t + t t (k t z t c t k t+1 ) i L = E t 1 X t=0 The rst order conditions are:
@L = t 1! t t @c t c t = 0 @L @k t+1 = t t + t+1 E t t+1 k 1 t+1 z t+1 = 0 The rst order conditions can be rearranged as: 1 c t = t
t = E t t+1 kt+1 1 z t+1 Solve the problem explicitly. k T +1 = 0 c T = k T z T c t = d k t z t (guess a solution)
! E t t+1 kt+1 1 z 1 t+1 = Et k 1 c t+1 z t+1 t+1 1 = E t d kt+1 z kt+1 1 z t+1 t+1! 1 1 = E t = 1 1 d dkt z t c t k t+1! t = E t t+1 kt+1 1 z 1 t+1, = 1 c t d k t z t 1 c t c t = d (k t z t c t )
d k t z t = d (k t z t d k t z t ) 1 = 1 (1 d) d = 1 ) c t = d k t z t = (1 ) k t z t
Example 4: Brock-Mirman growth model II Chow (1997), chapter 2 We shall use dynamic programming to solve the Brock-Mirman growth model. Consider the Brock-Mirman growth model: max fc t g E t 1X t=0 t ln c t
s:t: k t+1 = k t z t c t Control variable: c t State variables: k t ; z t The Bellman equation is:
V (k t ; z t ) = max c t [ln c t + E t V (k t+1 ; z t+1 )] We conjecture value function takes the form: V (k t ; z t ) = a + b ln k t + c ln z t a, b, c are coe cients to be determined. E t V (k t+1 ; z t+1 )
V (k t+1 ; z t+1 ) = a + b ln k t+1 + c ln z t+1 = a + b ln (k t z t c t ) + c ln z t+1 * E t ln z t+1 = 0 ) E t V (k t+1 ; z t+1 ) = a + b ln (k t z t c t ) It follows
V (k t ; z t ) = max [ln c c t + E t V (k t+1 ; z t+1 )] t = max fln c c t + [a + b ln (k t t z t c t )]g max fg c t First order condition. @ fg @c t = 1 c t b k t z t c t = 0
) c t = k t z t 1 + b Substitute this optimal c t = k t z t 1+b into max fg and equate the expression to the c t value function, we can solve the coe cients a, b, c, which after some algebraic manipulation are: a = (1 ) 1 h ln (1 ) + (1 ) 1 ln () i b = (1 ) 1
c = (1 ) 1
Example 5: the money-in-the-utility function (MIU) model Walsh (2003), chapter 2 Utility function, labour supply, and production function. u C 1 c t ; m t; l t = t 1 + l 1 t 1
C t h ac 1 t b + (1 a) mt 1 b i 1 1 b n t = 1 l t y t = e z t kt nt 1 = f (k t ; n t ; z t ) z t = z t 1 + e t
Two state variables. a t t + (1 + i t 1) b t 1 1 + t + m t 1 1 + t k t The objective function is:
X 1 E t t u c t ; m t; 1 t=0 n t s:t: f (k t ; n t ; z t ) + (1 ) k t + a t = c t + k t+1 + m t + b t The Bellman equation is:
V (a t ; k t ) = max h u c t ; m t; 1 n t + Et V (a t+1 ; k t+1 ) i Use the de nition of a t to substitute for a t+1. a t+1 t+1 + (1 + i t) b t 1 + t+1 + m t 1 + t+1 Use the budget constraint to eliminate k t+1.
k t+1 = f (k t ; n t ; z t ) + (1 ) k t + a t c t m t b t Rewrite the Bellman equation as: V (a t ; k t ) = max h u c t ; m t; 1 n t + Et V (a t+1 ; k t+1 ) i = max 2 6 4 +E t V = max c t ;b t ;m t ;n t fg 0 u c t ; m t; 1 n t @ t+1 + (1+i t)b t 1+ + m t t+1 1+ ; f (k t+1 t ; n t ; z t ) + (1 ) k t + a t c t m t b t 1 A 3 7 5
The rst order conditions are: @ fg : u c ct ; m t; 1 @c t n t E t V k (a t+1 ; k t+1 ) = 0 @ fg @b t : E t 1 + i t 1 + t+1! V a (a t+1 ; k t+1 ) E t V k (a t+1 ; k t+1 ) = 0! @ fg 1 : u m ct ; m t; 1 n t +Et V a (a t+1 ; k t+1 ) E t V k (a t+1 ; k t+1 ) = 0 @m t 1 + t+1 @ fg @n t : u n ct ; m t; 1 n t + Et V k (a t+1 ; k t+1 ) f n (k t ; n t ; z t ) = 0
@V (a t ; k t ) @a t = @ fg @a t : V a (a t ; k t ) = E t V k (a t+1 ; k t+1 ) @V (a t ; k t ) @k t = @ fg @k t : V k (a t ; k t ) = E t V k (a t+1 ; k t+1 ) [f k (k t ; n t ; z t ) + 1 ] Resources constrains. y t + (1 ) k t = c t + k t+1
We can eliminate the partial di erentiations of value function in the F.O.C. and then simplify the equilibrium conditions into 9 equations with 9 endogenous variables to be determined. Equilibrium conditions include F.O.C., de nitions, and resources constraints. y t ; c t ; k t+1 ; m t ; n t ; R t ; t ; z t ; u t u c ct ; m t; 1 n t = um ct ; m t; 1 n t + Et 2 4 u c ct+1 ; m t+1; 1 1 + t+1 3 n t+1 5 (1)
u l ct ; m t; 1 n t = uc ct ; m t; 1 n t fn (k t ; n t ; z t ) (2) u c ct ; m t; 1 n t = Et R t u c ct+1 ; m t+1; 1 n t+1 (3) R t = 1 + f k (k t ; n t ; z t ) (4) y t + (1 ) k t = c t + k t+1 (5)
y t = f (k t ; n t ; z t ) (6) m t = 1 + t 1 + t! m t 1 (7) z t = z t 1 + e t (8) u t = u t 1 + z t 1 + ' t (9)
Exercises: use Lagrangian to solve the MIU model. L = 1X t=0 1X + t u c t ; m t; 1 t=0 t+1 E t t+1 n t 2 4 f (k t; n t ; z t ) + (1 ) k t + t + (1+i t 1)b t 1 c t k t+1 m t b t 1+ + m t 1 t 1+ t 3 5 @L @c t ; @L @m t ; @L @n t ; @L @k ; @L t+1 @b t Show that combing the F.O.C. of the Lagrangian method gives us the same equations as by using the Bellman equation.
Example 6: the cash-in-advance (CIA) model Walsh (2003), chapter 3, a certainty case Utility function: 1X t=0 t u (c t ) CIA constraint:
P t c t M t 1 + T t CIA constraint in real terms: c t m t 1 t + t t P t P t 1 ; t T t P t ; I t = 1 + i t
The budget constraint in real terms: f (k t ) + (1 ) k t + t + m t 1 + I t 1 b t 1 t c t + k t+1 + m t + b t w t f (k t ) + (1 ) k t + t + m t 1 + I t 1 b t 1 t The budget constraint can be rewritten as: w t c t + k t+1 + m t + b t
Solve the problem Two state variables w t m t 1 The Bellman equation is:
V (w t ; m t 1 ) = max [u (c t ) + V (w t+1 ; m t )] s:t: w t c t + k t+1 + m t + b t c t m t 1 t + t
w t = f (k t ) + (1 ) k t + t + m t 1 + I t 1 b t 1 t To solve the problem, rst use the de nition of w t to eliminate w t+1 at the RHS of Bellman equation: V (w t ; m t 1 ) = max " u (c t ) + V f (k t+1 ) + (1 ) k t+1 + t+1 + m t + I t b t ; m t t+1!# Secondly, from the Lagrangian:
L = max " u (c t ) + V f (k t+1 ) + (1 ) k t+1 + t+1 + m t + I t b t ; m t + t (w t c t k t+1 m t b t ) + t m t 1 t + t c t! t+1!# The rst order conditions are: @L @c t : u c (c t ) t t = 0
@L @k t+1 : V w (w t+1 ; m t ) [f k (k t+1 ) + 1 ] t = 0 @L @b t : V w (w t+1 ; m t ) R t t = 0 R t I t t+1 @L @m t : V w (w t+1 ; m t ) 1 t+1 + V m (w t+1 ; m t ) t = 0
1 t+1 = R t i t t+1 Use envelope theorem to eliminate the partial di erentiations of value function in the rst order conditions. Envelope theorem: dv d = @L @ L
For details description of envelope theorem, see Dixit, Avinash K. (1990), Optimization in Economic Theory, Second Edition, Oxford University Press. V w (w t ; m t 1 ) = L w = t V m (w t ; m t 1 ) = L m = t t Rearrange the rst order conditions as:
u c (c t ) t t = 0 (1) t+1 [f k (k t+1 ) + 1 ] t = 0 (2) t+1 R t t = 0 (3) R t I t t+1 (4)
t+1 1 t+1 + t+1 t+1 t = 0 (5) 1 t+1 = R t i t t+1 (6) Exercises: use Lagrangian method to solve the CIA model.
L = 1X t=0 1X + + t u (c t ) t=0 1X t=0 t t 0 @ f (k t) + (1 ) k t + t + m t 1+I t 1 b t 1 t c t k t+1 m t b t t t m t 1 t + t c t! 1 A
References [1] Uhlig, Harald (1999), "A Toolkit for Analyzing Nonlinear Dynamic Stochastic Models Easily," in Ramon Marimon and Andrew Scott (eds.), Computational Methods for the Study of Dynamic Economies, Oxford University Press. [2] Chow, Gregory C. (1997), Dynamic Economics: Optimization by the Lagrange Method, Oxford University Press, Chapter 2. [3] Walsh, Carl (2003), Monetary Theory and Policy, Second Edition, The MIT Press, Chapters 2 & 3.