This is a simple guide to optimal control theory. In what follows, I borrow freely from Kamien and Shwartz (1981) and King (1986).

Transcription

1 ECON72: MACROECONOMIC THEORY I Martin Boileau A CHILD'S GUIDE TO OPTIMAL CONTROL THEORY 1. Introduction This is a simple guide to optimal control theory. In what follows, I borrow freely from Kamien and Shwartz (1981) and King (1986). 2. The Finite Horizon Problem Optimal control theory is useful to solve continuous time optimization problems of the following form: Z T max F (x(t);u(t);t) dt (P) _x i = Q i (x(t);u(t);t); i = 1;:::;n; (1) x i () = x i ; i = 1;:::;n; (2) x i (T) ; i = 1; :::;n; (3) u(t) 2 ; 2 IR m : (4) where, x(t) is a n-vector of state variables (x i (t)). These describes the state of the system at any point in time. Also, note that dx i =dt = _x i. u(t) is a m-vector of control variables. These are the choice variables in the optimization problem. F( ) is a twice continuously di erentiable objective function. Throughout, we assume that this function is time additive. Q i ( ) are twice continuously di erentiable transition functions for each state variable. 1

2 The di erent constraints are: i. Equation (1) de nes the transition equations for eachstate variables. It describes how the state variables evolve over time. ii. Equation (2) show the initial conditions for each state variable. In this, x i are constants. iii. Equation (3) show the terminal conditions for each state variable. iv. Equation (4) de nes the feasible set for control variables. We can now discuss the Pontryagin maximum principle. This principle says that we can solve the optimization problem P using a Hamiltonian function H over one period. A version of the Theorem is (see Kamien and Schwartz): Theorem In order that x (t), u (t) be optimal for the problem P, it is necessary that there exist a constant and continuous functions (t) = ( 1(t);:::; n(t)), where for all t T we have 6= and (t) 6= such that for every t T H (x (t);u; (t);t) H (x (t);u (t); (t);t); where the Hamiltonian function H is de ned by H (x;u; ;t) = F(x;u;t) + Except at points of discontinuity of u t, iq i (x;u; t): = _ i(t); i = 1;:::;n: Furthermore, = or = 1 and, nally, the following transversality conditions are satis ed: i(t) ; i(t)x i (T) = ; i = 1;:::;n: 3. The Present Value Hamiltonian The previous theorem suggests that one can solve problem P by simply setting up the following present value Hamiltonian: H (x;u; ; t) = F(x;u; t) + 2 iq i (x;u;t):

3 In the above, we call i(t) co-state variables. They are analogous to Lagrange multipliers. The necessary conditions for a maximum are: = ; k = 1;:::;m; = _ i(t); i = 1;:::;n; i(t) ; i(t)x i (T) = ; i = 1;:::;n: The su±cient conditions for a maximum are that the Hamiltonian function be concave in x and u. Finally, in some cases, our optimization problem will also include other constraints: Z T max F (x(t);u(t);t) dt (P1) _x i = Q i (x(t);u(t);t); G j (x(t);u(t);t) ; x i () = x i ; x i (T) ; i = 1;:::;n; i = 1; :::;n; u(t) 2 ; 2 IR m : In that case, we can write problem P1 as a Lagrangian: L(x;u; ; ;t) = H(x;u; ;t) + = F(x;u;t) + i = 1;:::;n; j = 1; :::;q; qx Á j G j (x;u;t) j=1 iq i (x;u;t) + qx Á j G j (x;u; t); where = (Á 1 ; :::;Á q ). In this case, the necessary conditions for a maximum are: j=1 = ; k = 1;:::;m; = _ i(t); i = 1;:::;n; i(t) ; i(t)x i (T) = ; i = 1; :::;n: Á j (t) ; Á j (t)q(x(t);u(t); t) = ; j = 1; :::;q: The su±cient conditions are as before. 3

4 4. An Economic Example Consider the following problem: The present value Hamiltonian is The rst-order conditions are Z T max e ½t u (c(t))dt _a(t) = ra(t) c(t); a() = a ; a(t) a T : H(a;c; ;t) = e ½t u(c) + [ra = =) e ½t u (c(t)) (t) = _ (t) =) (t)r = _ (t); (T) ; (T)a(T) = : A simple interpretation of these rst-order conditions is to relate the growth rate of consumption, g c (t), to the elasticity of intertemporal substitution, ¾(c(t)), and the market and subjective discount rates, r and ½. To do this, rst note that _ (t) = r (t) = re ½t u (c(t)): Then, totally di erentiating the rst rst-order condition yields ½e ½t u (c(t)) + e ½t u (c(t)) _c(t) _ (t) = : De ne the growth rate of consumption as g c = (1=c)dc=dt: g c (t) = _c(t) c(t) u (c(t)) = (r ½) c(t)u (c(t)) ; such that g c (t) = ¾(c(t))(r ½); where ¾(c(t)) = u (c(t))=[c(t)u (c(t))] >. Thus, consumption grows whenever the market discount rate is larger thanthe subjective discountrate. Inthat case, the consumer is less impatient than the market. 4

5 5. The In nite Horizon Problem In this section we consider an extension of the nite horizon problem to the case of in nite horizon. To simplify the exposition, we will focus our attention to stationary problems. The assumption of stationarity implies that the main functions of the problem are time invariant: F(x(t);u(t);t) = F(x(t);u(t)); G i (x(t);u(t);t) = G i (x(t);u(t)); Q j (x(t);u(t);t) = Q j (x(t);u(t)): In general, to obtain an interior solution to our optimization problem, we require the objective function to be bounded away from in nity. One way to achieve boundedness is to assume discounting. However, the existence of a discount factor is neither necessary nor su±cient to ensure boundedness. Thus, we de ne F(x(t);u(t)) = e ½t f(x(t);u(t)): Our general problem becomes: max Z 1 e ½t f (x(t);u(t)) dt (P2) _x i = Q i (x(t);u(t)); i = 1;:::;n; (1) G j (x(t);u(t)) ; The present value Hamiltonian is The Lagrangian is j = 1; :::;q; x i () = x i ; i = 1; :::;n; (2) H (x;u; ) = e ½t f(x;u) + L(x;u; ; ) = e ½t f(x;u) + The rst-order conditions are: = ; k = 1;:::;m; iq i (x;u): iq i (x;u) + qx Á j G j (x;u): j=1 _ i(t); i i = 1; :::;n; lim ; T!1 i(t) lim T!1 i(t)x i (T) = ; i = 1;:::;n: Á j (t) ; Á j (t)q(x(t);u(t)) = ; j = 1;:::;q: 5

6 The su±cient conditions are as before. 6. The Current Value Hamiltonian Any discounted optimal control problem can be solved using both a present value Hamiltonian or a current value Hamiltonian. We sometime prefer the current value Hamiltonian for its simplicity. We obtain the current value Hamiltonian as follows: H(x;u; ª) = e ½t H(x;u; ); where à i = e ½t i. The current value Hamiltonian is thus H(x;u; ª) = f(x;u) + à i Q i (x;u): The transformation from present value to current value a ects the rst-order conditions. To see this, let us restate our problem P2 using the current value Hamiltonian. The problem is Z 1 max e ½t f (x(t);u(t)) dt (P2) _x i = Q i (x(t);u(t)); i = 1;:::;n; (1) G j (x;u) ; j = 1; :::;q; x i () = x i ; i = 1; :::;n; (2) The current value Hamiltonian is H(x;u; ª) = f(x;u) + The Lagrangian is à i Q i (x;u): qx L(x;u;ª; ) = f(x;u) + à i Q i (x;u) + Á j G j (x;u): j=1 6

7 The rst-order conditions are: = ; k = 1;:::;m; h = à _ i (t) ½Ã i (t)i ; i = 1;:::;n; lim T!1 e ½T à i (T) ; lim T!1 e ½T à i (T)x i (T) = ; i = 1;:::;n: Á j (t) ; Á j (t)q(x(t);u(t)) = ; j = 1;:::;q: The su±cient conditions are as before. 7. The Economic Example in In nite Horizon Consider the following problem: Z 1 max e ½t u(c(t))dt _a(t) = ra(t) c(t); a() = a : Abstracting from the time subscript, the current value Hamiltonian is Its rst-order conditions are H(a;c; Ã) = u(c) + à = =) u (c) à = h i = à _ ½Ã =) Ãr = h à _ ½Ã ; lim T!1 e ½t Ã(T) ; lim T!1 e ½t Ã(T)a(T) = : Once again, the interpretation of these rst-order conditions relates consumption growth to the elasticity of intertemporal substitution, the market discount rate, and the subjective discount rate, r and ½. To show this, rst note that _à = (r ½)à = (r ½)u (c): Then, totally di erentiating the rst rst-order condition yields u (c)_c = _ à = (r ½)u (c); 7

8 such that or g c (t) = _c(t) c(t) u (c(t)) = (r ½) c(t)u (c(t)) g c (t) = ¾(c(t))(r ½); where ¾(c(t)) = u (c(t))=[c(t)u (c(t))] >. Thus, consumption grows whenever the market discount rate is larger than the subjective discount factor. In that case, the consumer is less impatient than the market. 8

9 8. Summary To summarize, here is the simple cookbook. Assume that you face the following problem: Z 1 max e ½t f (x(t);u(t)) dt _x i = Q i (x(t);u(t)); x i () = x i ; There are two ways to proceed: 1. Present Value Hamiltonian i = 1; :::;n: Step 1 Write the present value Hamiltonian: H (x;u; ) = e ½t f(x;u) + Step 2 Find the rst-order conditions: i = 1;:::;n; iq i (x;u): = ; k = 1; :::;m; = _ i(t); i = 1;:::;n; lim ; T!1 i(t) lim T!1 i(t)x i (T) = ; i = 1;:::;n: 2. Current Value Hamiltonian Step 1 Write the current value Hamiltonian: H(x;u;ª) = f(x;u) + Step 2 Find the rst-order conditions: à i Q i (x;u): = ; k = 1; :::;m; h = à _ i (t) ½Ã i (t)i ; i = 1;:::;n; lim T!1 e ½T à i (T) ; lim T!1 e ½T à i (T)x i (T) = ; i = 1; :::;n: 9

10 References Kamien, Morton I. and Nancy L. Schwartz, 1981, Dynamic Optimization: The Calculus ofvariations andoptimal Control ineconomics andmanagement, New York: North Holland. King, Ian P., 1986, A Cranker's Guide to Optimal Control Theory, mimeo Queen's University. 1