Vol. 18 No. Journal of Sysems Science and Complexiy Apr., 5 FORWARD-BACKWARD STOCHASTIC DIFFERENTIAL EQUATIONS, LINEAR QUADRATIC STOCHASTIC OPTIMAL CONTROL AND NONZERO SUM DIFFERENTIAL GAMES WU Zhen (School of Mahemaics and Sysems Science, Shandong Universiy, Jinan 51, China. Email: wuzhen@sdu.edu.cn) Absrac. In his paper, we use he soluions of forward-backward sochasic differenial equaions o ge he explici form of he opimal conrol for linear quadraic sochasic opimal conrol problem and he open-loop Nash equilibrium poin for nonzero sum differenial games problem. We also discuss he solvabiliy of he generalized Riccai equaion sysem and give he linear feedback regulaor for he opimal conrol problem using he soluion of his kind of Riccai equaion sysem. Key words. Sochasic differenial equaions, sochasic opimal conrol, Riccai equaion, nonzero sum sochasic differenial game. 1 Inroducion Le (Ω, F, P ) be a probabiliy space and le {B } be a d-dimensional Brownian moion in his space. We denoe he naural filraion of his Brownian moion by F. We consider he following fully coupled forward-backward sochasic differenial equaions (abbrev. o FBSDE) x = a + b(s, x s, y s, z s )ds + σ(s, x s, y s, z s )db s, y = Φ(x T ) + f(s, x s, y s, z s )ds z s db s, [, T ], where (x, y, z) akes values in R n R n R n d, b, σ and f are mappings wih appropriae dimensions and are, for each fixed (x, y, z), F -progressively measurable. We assume ha hey are Lipschiz wih respec o (x, y, z). T > is an arbirarily prescribed number and he ime inerval is called he ime duraion. To our knowledge, acually here exis wo main mehods in he sudy of FBSDE. Using parial differenial equaion mehod which is sandardized as four seps scheme, J. Ma, P. Proer and J. Yong [1] obained he exisence and uniqueness resul of FBSDE in an arbirarily prescribed ime duraion. Bu hey need he equaion o be non-degenerae and require ha he coefficiens can no be randomly disurbed which is ofen he case in he pracical Received November 11,. Revised Sepember 3, 4. *This work is suppored by he Naional Naural Science Foundaion (Gran No.137167), he Youh Teacher Foundaion of Fok Ying Tung Educaion Foundaion, he Excellen Young Teachers Program and he Docoral Program Foundaion of MOE and Shandong Province, China.
18 WU ZHEN Vol. 18 siuaion such as in financial markes. Using probabilisic mehod, Y. Hu and Peng [] go he exisence and uniqueness resuls under some monoone assumpions when x and y ake he same dimension. Then Peng and Wu [3] exended hem o differen dimensional FBSDE and weakened he monoone assumpions so ha he resuls can be widely used especially in Hamilonian sysem arising in sochasic opimal problem when we apply he maximum principle. J. Yong [4] le he above mehod be sysemaic and called i he coninuaion mehod. In Secion of his paper we give one exisence and uniqueness resul of he FBSDE which is useful in sudying linear quadraic nonzero sum sochasic differenial games problems. Sochasic linear quadraic opimal conrol problems have been firs sudied by Wonham [5] and hen by Bismu [6] also. In [6], Bismu proved he exisence of he opimal conrol and also used he dual mehod o give he feedback form for he opimal conrol. In Secion 3 we use he soluion of he FBSDE o give an explici form of opimal conrol for linear quadraic problem when he coefficiens are random. Using he classical mehod we can also prove ha he opimal conrol is unique. Using square complee echnique S. Chen, X. Li and X. Zhou [7] sudied his problem wih indefinie conrol weigh coss. They gave he opimal feedback conrol using he soluion of one kind of sochasic Riccai equaion. This kind of equaion is so complicaed ha hey canno prove he exisence and uniqueness in random case, hey only give he resul for one special deerminisic case. In our paper, for random case we can use he exisence and uniqueness resul of he FBSDE in [3] o give an explici opimal conrol form. For deerminisic case i is naural o sudy he associaed Riccai equaion. We inroduce one kind of generalized Riccai equaion sysem and give he linear feedback form of opimal conrol using he soluion of his kind of generalized marix Riccai equaion sysem. This kind of Riccai equaion is novel and differen from ha in [6] and [7]. However, he equaion form is suiable o he naure of FBSDE. A similar form can be seen in [8]. The mehod in our paper is more direc and easy o undersand because of he inroducion of FBSDE. In Secion 4, we discuss he solvabiliy of his kind of generalized Riccai equaion and also give one simple example of he generalized Riccai equaion sysem which has a unique soluion. In Secion 5 we sudy he nonzero sum sochasic differenial game problem. For deerminisic case his problem was considered by Friedman [9], Bensoussan [1] and Eisele [11]. For sochasic case Hamadene [1] showed one exisence resul of Nash equilibrium poin under some more assumpions. In his secion we improve his resul. Using he preliminary resul of he FBSDE in Secion we give an explici form of Nash equilibrium poin. The resuls of his paper are clear and easy o undersand. They can be applied in pracice direcly. The Preliminary Resuls of FBSDE In his secion le us give one exisence and uniqueness resul of he FBSDE which is useful in sudying sochasic differenial game problems. We consider he following special kind of FBSDE: dx = b(, x, y )d + σ(, x, y )db, dy = f(, x, y, z )d z db, (.1) x = a, y T = Φ(x T ). For noaional simplificaion, we assume d = 1; here (x, y, z) R n+n+n, b, f and σ are mappings wih appropriae dimensions and are, for each fixed (x, y, z), F -progressively measurable. We
No. FBSDE, OPTIMAL CONTROL AND DIFFERENTIAL GAMES 181 use he noaions u = x y z, A(, u) = f b (, u). σ We assume ha (i) A(, u) is uniformly Lipschiz wih respec o u; (ii) For each u, A(, u) is in M (, T ); (iii) Φ(x) is uniformly Lipschiz wih respec o x R n ; (iv) For each x, Φ(x) is in L (Ω, F T, P ) (H.1) wih he following monoone condiions: A(, u) A(, u), u u β 1 x β ŷ, Φ(x) Φ(x), x x µ 1 x, a.s. u = (x, y, z), u = (x, y, z), x = x x, ŷ = y y, ẑ = z z, (H.) where β 1, β and µ 1 are given nonnegaive consans wih β 1 + β >, µ 1 + β >. Then we have Theorem.1 We assume (H.1) and (H.). Then here exiss a unique riple u = (x, y, z ), [, T ], saisfying FBSDE (.1). Proof The uniqueness proof can be given by Theorem. in [3]. In order o prove he exisence, we consider he following family of FBSDE paramerized by α [, 1]: dx α = [ (1 α)β ( y α) + αb(, xα, yα ) + φ ] d, + [ ασ(, x α, yα ) + ψ ] db, dy α = [ (1 α)β 1 x α + αf(, uα ) + γ ] d z α db, x α = a, yα T = αφ(xα T ) + (1 α)x T + ξ, where φ, ψ and γ are given processes in M (, T ) wih values in R n. Clearly, when α = 1 he exisence of he soluion of equaion (.) implies ha of (.1). If α =, from Lemma.5 in [3], i is easy o obain he exisence and uniqueness resul. Then he following lemma gives a priori esimae for he exisence inerval of (.) wih respec o α [, 1]. Lemma. We assume (H.1) and (H.). Then here exiss a posiive consan δ such ha if, a priori, for an α [, 1) here exiss a soluion (x α, y α, z α ) of he equaion (.), hen for each δ [, δ ] here exiss a soluion (x α+δ, y α+δ, z α+δ ) of he equaion (.) for α = α + δ. The proof mehod is similar o ha for Lemma.4 and Lemma.7 in [3]. We omi i. When α =, he equaion (.) has a unique soluion. From Lemma., here exiss a posiive consan δ which only depends on he Lipschiz consans, β 1, β and T such ha for each δ [, δ ], equaion (.) for α = α + δ has a unique soluion. We can repea his process for N-imes wih 1 Nδ < 1 + δ. I hen follows ha, in paricular, equaion (.) for α = 1 wih φ s, γ s and ψ s has a unique soluion. The proof is compleed. (.)
18 WU ZHEN Vol. 18 3 Linear Quadraic Sochasic Opimal Conrol Problem We consider he following linear conrol sysem { dx = [A (ω)x + B (ω)v ]d + [C (ω)x + D (ω)v ]db, x = a, [, T ], where A (ω) and C (ω) are n n bounded progressively measurable marix-valued processes, v, [, T ], is an admissible conrol process, i.e. an F adaped square inegrable process aking values in a given subse U of R k, B (ω) and D (ω) are n k bounded progressively measurable marix-valued processes. We also assume ha here is no consrain imposed on he conrol processes: U = R k. A classical quadraic opimal conrol problem is o minimize he cos funcion J(v( )) = 1 [ ] E ( R (ω)x, x + N (ω)v, v )d + Q(ω)x T, x T (3.) over he se of admissible conrols, where Q(ω) is he F T measurable nonnegaive symmeric bounded marix and R (ω) is he n n nonnegaive symmeric bounded progressively measurable marix-valued process, N (ω) is an k k posiive symmeric bounded progressively measurable marix-valued process and he inverse N 1 (ω) is also bounded. In his secion we give an explici form of he opimal conrol using he soluion of he FBSDE and he following resul: Theorem 3.1 The funcion u = N 1 (ω)(b τ (ω)y + D τ(ω)z ), [, T ], is he unique opimal conrol for he linear quadraic conrol problem, where (x, y, z ) is he soluion of he following FBSDE: dx = [A (ω)x + B (ω)u ]d + [C (ω)x + D (ω)u ]db, dy = [A τ (ω)y + C τ (ω)z + R (ω)x ]d z db, (3.3) x = a, y T = Q(ω)x T. Proof We firs look a he FBSDE (3.3). From Theorem 3.1 in [3], he FBSDE (3.3) has a unique soluion (x, y, z ). Now we prove ha u is he opimal conrol. Denoing, v( ) R k, x v o be he corresponding rajecory of he sysem (3.1), hen J(v( )) J(u( )) = 1 [ E ( R (ω)x v, xv R (ω)x, x + N (ω)v, v N (ω)u, u )d ] + Q(ω)x v T, xv T Q(ω)x T, x T = 1 [ E ( R (ω)(x v x ), x v x + N (ω)(v u ), v u + R (ω)x, x v x + N (ω)u, v u )d ] + Q(ω)(x v T x T ), x v T x T + Q(ω)x T, x v T x T. From Qx T = y T, we use Iô s formula o x v T x T, y T and ge E x v T x T, y T = E (3.1) ( R (ω)x, x v x + B (ω)(v u ), y + D (ω)(v u ), z )d.
No. FBSDE, OPTIMAL CONTROL AND DIFFERENTIAL GAMES 183 And hen, because of R and Q being nonnegaive, N being posiive, we have So E = E =. J(v( )) J(u( )) ( N (ω)u, v u + B (ω)(v u ), y + D (ω)(v u ), z )d ( N (ω)n 1 (ω)(b τ (ω)y + D τ (ω)z ), v u + B τ (ω)y, v u + D τ (ω)z, v u )d u = N 1 (ω)(b τ (ω)y + D τ (ω)z ) is he opimal conrol. To prove he uniqueness of he opimal conrol, he mehod is classical. We give he proof for he reader s convenience. We assume ha u 1 ( ) and u ( ) are boh opimal conrols, and he corresponding rajecories are x 1 and x. I is easy o know he rajecories corresponding o u 1 ( ) + u ( ) and u 1 ( ) u ( ) are x 1 ( ) + x ( ) x 1 ( ) x ( ) and respecively. Because of N being posiive, R and Q being nonnegaive, we know ha and J(u 1 ( )) = J(u ( )) = α α = J(u 1 ( )) + J(u ( )) ( u 1 ( ) + u ) [ ( ) T ( x 1 x = J + E R, x1 x + Q x1 T x T, x1 T ] x T ( u 1 ( ) + u ) ( ) T u 1 J + E N u, u1 u d α + δ 4 E u 1 u d. Here δ >. So u 1 u + N E u 1 u d and u 1 ( ) = u ( )., u1 u ) d Here all coefficiens are sochasic processes. Now we le A, B, C, D, R, N be deerminisic funcions, Q be he deerminisic marix, and wan o give he feedback form of he opimal
184 WU ZHEN Vol. 18 conrol i.e. he opimal linear feedback regulaor. We inroduce he following generalized n n marix-valued Riccai equaion sysem of (K(), M()), [, T ]: K() = A τ K() + K()A K()B N 1 B τ K() K()B N 1 D τ M() + C τ M() + R, (3.4) M() = K()C K()D N 1 B τ K() K()D N 1 D τ M(), K(T ) = Q, [, T ]. This is a kind of generalized marix-valued Riccai equaion sysem formed by a marixvalued ordinary differenial equaion and an algebraic equaion which is differen from he ones in [6] and [7]. In hose papers hey gave he opimal conrol even when N is indefinie by using he soluion of a kind of Riccai equaion direcly. Bu hey canno ge he exisence and uniqueness for ha kind of Riccai equaion in random case. For deerminisic case hey only sudied he special case: C =. However if he Riccai equaion in heir papers has no soluion, hey canno obain he opimal conrol whaever in random or deerminisic siuaion. In he nex secion we will discuss he solvabiliy of our Riccai equaion sysem (3.4). Some relaed echniques o deal wih his kind of equaion sysem are derived from [8]. We firs have Theorem 3. Suppose here exiss marix (K(), M()), [, T ], saisfying he generalized marix Riccai equaion sysem (3.4). Then he opimal linear feedback regulaor for he linear quadraic opimal problem is and he opimal value funcion is u = N 1 [B τ K() + Dτ M()]x, [, T ], (3.5) J(u( )) = 1 K()a, a. (3.6) Proof We le (K(), M()) be he soluion of (3.4). I is easy o check ha he soluion (x, y, z ) of (3.3) saisfies y = K()x, z = M()x, so he opimal conrol u( ) saisfies he equaion (3.5). We apply Iô s formula o x T, y T in J(u( )); i is easy o ge (3.6). Using he above opimal conrol as he linear feedback we can also ge he opimal sae rajecory x( ) from equaion (3.3). 4 Solvabiliy of he Generalized Riccai Equaion Sysem In Secion 3, we can obain he opimal linear feedback regulaor for linear quadraic opimal problem by he soluion of he generalized Riccai equaion (3.4). We will discuss he solvabiliy for his kind of equaion in his secion. We firs look a he special case: D = o seek some illuminaion. This is a simple case; (3.4) is reduced o he following ordinary equaion: K() = A τ K() + K()A K()B N 1 B τ K() +C τ K()C + R, (4.1) K(T ) = Q, M() = K()C, [, T ]. From he convenional Riccai equaion heory which can be seen in [13], equaion (4.1) has he unique soluion K( ) C(, T ; S n +), here S n + represening he space of all n n nonnegaive definie symmeric marices, and M() = K()C.
No. FBSDE, OPTIMAL CONTROL AND DIFFERENTIAL GAMES 185 For he general case, we can sudy he following equaion: K() = A τ K() + K()A K()B N 1 B τ K() K(T ) = Q, +K()B N 1 K()B N 1 D τ (I n + K()D N 1 D τ ) 1 K()D N 1 B τ K() + R D τ (I n + K()D N 1 D τ ) 1 K()C +C T (I n + K()D N 1 D τ ) 1 K()C C T (I n + K()D N 1 D τ ) 1 K()D N 1 B T K(), I n + K()D N 1 D τ >, [, T ]. If we ge he soluion K() for (4.), hen we can le M() = (I n + K()D N 1 D τ ) 1 K()C (I n + K()D N 1 D τ ) 1 K()D N 1 B τ K() o ge he soluion of (3.4). Now we firs have he following uniqueness resul. Theorem 4.1 Riccai equaion (4.) has a mos one soluion K( ) C(, T ; S+ n ). Proof We suppose K is anoher soluion of (4.). Seing K = K K, hen K saisfies K() = A τ K() + K()A K()B N 1 B τ K() K()B N 1 B τ K() (4.) + K()B N 1 D τ (I n + K()D N 1 D τ ) 1 (I n + K()D N 1 D τ ) 1 K()D N 1 B τ K() +K()B N 1 (I n + K()D N 1 +K()B N 1 D τ (I n + K()D N 1 D τ ) 1 D τ) 1 K()D N 1 B τk() D τ (I n + K()D N 1 D τ ) 1 (I n + K()D N 1 D τ) 1 K()D N 1 B τ K() +I + II + III + IV, K(T ) =, [, T ]. Here I = K()B N 1 (I n + K()D N 1 K()B N 1 D τ (I n + K()D N 1 D τ ) 1 K()D N 1 (I n + K()D N 1 +K()B N 1 (I n + K()D N 1 +K()B N 1 (I n + K()D N 1 D τ) 1 K()D N 1 B τk() D τ D τ (I n + K()D N 1 D τ ) 1 K()D N 1 D τ) 1 K()D N 1 B τk() D τ(i n + K()D N 1 D τ ) 1 K()D N 1 D τ) 1 K()D N 1 B τk() D τ (I n + K()D N 1 D τ ) 1 K()D N 1 D τ ) 1 K()D N 1 B τ K(), D τ D τ D τ
186 WU ZHEN Vol. 18 II = K()B N 1 K()B N 1 K()B N 1 D τ (I n + K()D N 1 D τ ) 1 (I n + K()D N 1 D τ ) 1 K()C D τ (I n + K()D N 1 D τ ) 1 (I n + K()D N 1 D τ ) 1 K()C D τ (I n + K()D N 1 D τ ) 1 K()D N 1 (I n + K()D N 1 D τ ) 1 K()C +K()B N 1 D τ(i n + K()D N 1 D τ) 1 K()D N 1 (I n + K()D N 1 D τ ) 1 K()C K()B N 1 D τ(i n + K()D N 1 D τ) 1 K()D N 1 (I n + K()D N 1 D τ ) 1 K()C, III = C τ (I n + K()D N 1 C τ (I n + K()D N 1 + C τ (I n + K()D N 1 IV = C τ (I n + K()D N 1 C τ (I n + K()D N 1 +C τ (I n + K()D N 1 (I n + K()D N 1 C τ (I n + K()D N 1 (I n + K()D N 1 C τ (I n + K()D N 1 (I n + K()D N 1 D τ D τ D τ D τ ) 1 (I n + K()D N 1 D τ ) 1 K()C D τ) 1 K()D N 1 D τ (I n + K()D N 1 D τ ) 1 K()C D τ) 1 K()D N 1 D τ (I n + K()D N 1 D τ ) 1 K()C, D τ) 1 (I n + K()D N 1 D τ ) 1 K()D N 1 B τk() D τ) 1 (I n + K()D N 1 D τ ) 1 K()D N 1 B τ K() D τ ) 1 K()D N 1 D τ D τ) 1 K()D N 1 B τk() D τ ) 1 K()D N 1 D τ D τ) 1 K()D N 1 B τk() D τ ) 1 K()D N 1 D τ D τ) 1 K()D N 1 B τ K(), where I n +KD N 1 D τ > and I n +KD N 1 D τ >. Since (I n +KD N 1 D τ ) 1 and (I n + KD N 1 D τ ) 1 are uniformly bounded due o heir coninuiy, we can apply he Gronwall s inequaliy o ge K(). This proves he uniqueness. Now le us discuss he exisence of (4.) sep by sep. If we le Y () = F (K) = (I n + D τ ) 1 K, hen for any Y ( ) C(, T ; S+), n he following convenional Riccai equaion KD N 1 K() = A τ K() + K()A + C τ Y ()C + R K()[B N 1 B τ B N 1 D τ Y ()D N 1 B τ ]K() C τy ()D N 1 B τ K() K()B N 1 D τ Y ()C, K(T ) = Q, [, T ] (4.3) has a unique soluion K( ) C(, T ; S n + ) when [B N 1 B τ B N 1 D τ Y ()D N 1 B τ ] C(, T ; S+). n (4.4) We denoe by S n s he subspace of S n + formed by he symmeric marices saisfying (4.4), i.e. S n s = { Y ( ) : Y ( ) C(, T ; S n + ) and [B N 1 B τ B N 1 D τ Y ()D N 1 B τ ] C(, T ; Sn + )}.
No. FBSDE, OPTIMAL CONTROL AND DIFFERENTIAL GAMES 187 Obviously, Y Ss n, so his definiion is reasonable. Thus we can define a mapping Ψ : C(, T ; Ss n) C(, T ; Sn + ) when K = Ψ(Y ) by he equaion (4.3). We firs have following wo lemmas: Lemma 4. The operaor F (K) is monoonely increasing when K >. Proof We noice, when K >, ha F (K) = (I n + KD N 1 D τ ) 1 K = [K 1 (I n + KD N 1 D τ )] 1 = (K 1 + D N 1 D τ ) 1. So, if K 1 K, hen F (K 1 ) F (K ). Lemma 4.3 The operaor Ψ is monoonely increasing and coninuous. Proof Le K = Ψ(Y ), K = Ψ(Y ) and K = K K. We rewrie (4.3) as K() = A τ K() + K()A K()B N 1 B τk() + R +[C D N 1 B τ K()]τ Y ()[C D N 1 B τ K()], K(T ) = Q, [, T ]. From Lemma 4. and Lemma 8. in [8], we know if Y Y, hen K K; his proves he monooniciy. On he oher hand, by a similar mehod wih he uniqueness proof and Gronwall inequaliy, i is easily seen ha, if Y Y, K K = K. This yields he coninuiy; he proof is compleed. Looking back a equaion (4.3), i is easy o know ha Theorem 4.4 If here is Y C(, T ; Ss n ) such ha (4.5) Y = (I n + Ψ(Y )D N 1 D τ ) 1 Ψ(Y ), (4.6) hen he Riccai equaion (4.) has a unique soluion. The following ask is o find he suiable Y C(, T ; Ss n ) saisfying (4.6). We need he following resul. Lemma 4.5 If here exis Y +, Y C(, T ; Ss n ) such ha Y + (I n + Ψ(Y + )D N 1 D τ ) 1 Ψ(Y + ) (I n + Ψ(Y )D N 1 D τ ) 1 Ψ(Y ) Y, (4.7) hen equaion (4.3) admis a soluion. Proof Le Y +, Y be given saisfying (4.7). We define he sequences Y + i, Y i, K + i, K i as follows: From (4.7), we have Y + = Y + S n s, Y = Y S n s, K+ = Ψ(Y + ), K = Ψ(Y ), Y + i+1 = (I n + K + i D N 1 D τ ) 1 K + i, Y i+1 = (I n + K i D N 1 D τ ) 1 K i, K + i+1 = Ψ(Y + i+1 ), K i+1 = Ψ(Y i+1 ), i =, 1,,. Y + (I n + Ψ(Y + )D N 1 D τ ) 1 Ψ(Y + ) = (I n + K + D N 1 D τ ) 1 K + = Y + 1 Y 1 Y.
188 WU ZHEN Vol. 18 From Lemma 4.3, K + K+ 1 K 1 K. By inducion, we obain and also Y + i, Y i Ss n. So we have From Lemma 4.3, we have K + K+ i K + i+1 K i+1 K i K, Y + Y + i Y + i+1 Y i+1 Y i Y, lim Y i + = Y + Ss n, lim i i K+ i = K + S+ n. K + = lim i K + i = lim Ψ(Y + i i ) = Ψ( lim i Y + i ) = Ψ(Y + ). So K + is one soluion of Riccai equaion (4.3) corresponding o Y = Y + ; hen Y + = (I n + K + D N 1 D τ ) 1 K + ; by Theorem 4.4, K + is one soluion of (4.). By he same sep, we also can ge K = lim i K i and Y = lim Yi, i so K is also a soluion of (4.); by he uniqueness resul in Theorem 4.1, K + = K. We only need o find Y + and Y saisfying (4.7). The exisence of Y is obvious; we can le Y =. By he convenional Riccai equaion heory, i saisfies (4.7). Now we need one sufficien condiion below o find Y + Ss n saisfying (4.7) and ensure he exisence of a soluion o Riccai equaion (4.). We assume ha There exiss Y ( ) Ss n, such ha D τ Y D = N and (I n + KD N 1 D τ ) 1 K Y, here K being he unique soluion of he following equaion : (H4.1) K() = A τ K() + K()A + R + C τ Y C C τ Y D N 1 B τ K() K()B N 1 D τ Y C, K(T ) = Q. Hence, we have he following resul. Theorem 4.6 We assume (H4.1). The generalized Riccai equaion (4.) has unique soluion (K, M) C 1 (, T ; S+) n L (, T ; R n n ). We noice ha when n = k and D is inverible, i is easy o check ha (H4.1) is saisfied. We can also give one simple example of he generalized Riccai equaion which has a unique soluion. Example 4.1 We assume he dimensions of he sae and conrol in conrol sysem (3.1) are he same i.e. k = n, and assume D = I n. Now we can le Y = N and hen o check (H4.1). Y = N, so Y D + KDN 1 D τ Y D KD, where K is he soluion of he following equaion: K() = A τ K() + K()A + R + C τ N C K(T ) = Q, C τn DN 1 B τk() K()B N 1 D τ N C,
No. FBSDE, OPTIMAL CONTROL AND DIFFERENTIAL GAMES 189 and Y + KDN 1 D τ Y K, hen Y (I n + KDN 1 D τ ) 1 K. So from Theorem 4.6, he Riccai equaion (3.4), when k = n, D = I n, has a unique soluion. For his case, we can ge an explici form of he linear opimal feedback and he opimal value funcion for he linear quadraic opimal problem by Theorem 3., and also we can ge he opimal sae rajecory from he equaion (3.3). 5 Linear quadraic nonzero sum sochasic differenial games In his secion, we sudy linear quadraic nonzero sum sochasic differenial games problem. From he furher exisence and uniqueness resul of he FBSDE (Theorem.1), we improve a similar resul in Hamadene [1]. For noaional simplificaion, we only consider wo players; i is similar for n players. The conrol sysem is { dx v = [Ax v + B 1 v 1 + B v + α ]d + [Cx v + β ]db, (5.1) x v = a, where A and C are n n bounded marices, v 1 and v, [, T ], are wo admissible conrol processes, i.e. F -adaped square inegrable processes aking values in R k. B 1 and B are n k bounded marices. α and β are wo adaped square-inegrable processes. We denoe by J 1 (v( )) and J (v( )), v( ) = (v 1 ( ), v ( )), he cos funcions corresponding o he wo players 1 and : J 1 (v( )) = 1 [ ] E ( R 1 x v, x v + N 1 v 1, v 1 )d + Q 1 x v T, x v T, J (v( )) = 1 [ (5.) T E ( R x v, xv + N v, v )d + Q x v T, xv T ]. Here Q 1, Q, R 1 and R are n n nonnegaive symmeric bounded marices, N 1 and N are k k posiive symmeric bounded marices and he inverses (N 1 ) 1, (N ) 1 are also bounded. The problem is o look for (u 1 ( ), u ( )) R k R k which is called he Nash equilibrium poin for he game, such ha { J 1 (u 1 ( ), u ( )) J 1 (v 1 ( ), u ( )), v 1 ( ) R k ; (5.3) J (u 1 ( ), u ( )) J (u 1 ( ), v ( )), v ( ) R k. We need he following assumpions: { B i (N i ) 1 (B i ) τ A τ = A τ B i (N i ) 1 (B i ) τ, i = 1,, B i (N i ) 1 (B i ) τ C τ = C τ B i (N i ) 1 (B i ) τ, a.s. (H5.1) Now, using he soluion of FBSDE we can give an explici form of he Nash equilibrium poin for his problem. Theorem 5.1 The funcion (u 1, u ) = ( (N 1 ) 1 (B 1 ) τ y 1, (N ) 1 (B ) τ y ), [, T ],
19 WU ZHEN Vol. 18 is one Nash equilibrium poin for he above game problem, where (x, y 1, y, z 1, z ) is he soluion of he following differen dimensional FBSDE: dx = [Ax B 1 (N 1 ) 1 (B 1 ) τ y 1 B (N ) 1 (B ) τ y + α ]d +[Cx + β ]db, dy 1 = [Aτ y 1 + Cτ z 1 + R1 x ]d z 1dB, (5.4) dy = [Aτ y + Cτ z + R x ]d z db, x = a, yt 1 = Q1 x T, yt = Q x T. Proof We firs prove he exisence of he soluion of (5.4). We consider he following FBSDE: dx = [AX Y + α ]d + [CX + β ]db, dy = [A τ Y + (B 1 (N 1 ) 1 (B 1 ) τ R 1 + B (N ) 1 (B ) τ R )X (5.5) +C τ Z ]d Z db, X = a, Y T = [B 1 (N 1 ) 1 (B 1 ) τ Q 1 + B (N ) 1 (B ) τ Q ]X T. We noice ha if he (x, y 1, y, z 1, z ) is he soluion of (5.4), (X, Y, Z ) saisfies he FBSDE (5.5) from (H5.1). Here X = x, Y = B 1 (N 1 ) 1 (B 1 ) τ y 1 + B (N ) 1 (B ) τ y, Z = B 1 (N 1 ) 1 (B 1 ) τ z 1 + B (N ) 1 (B ) τ z. On he oher hand, we can firs use he FBSDE (5.5) o ge soluion X which is he forward soluion x of (5.4), hen obain (y 1, z 1 ) and (y, z ). I is easy o check ha he FBSDE (5.5) saisfies (H.1) and (H.); according o Theorem.1, here exiss a unique soluion (X, Y, Z ) of (5.5). Now, from Theorem 3.1 in [14], we can le (y 1, z 1 ) and (y, z ) be he soluions of he following backward sochasic differenial equaions (abbrev. o BSDE): dy 1 = [A τ y 1 + C τ z 1 + R 1 X ]d z 1 db, dy = [Aτ y + Cτ z + R X ]d z db, yt 1 = Q1 X T, yt = Q X T. We le Y = B 1 (N 1 ) 1 (B 1 ) τ y 1 + B (N ) 1 (B ) τ y, hen we ge Z = B 1 (N 1 ) 1 (B 1 ) τ z 1 + B (N ) 1 (B ) τ z ; dy = [A τ Y + (B 1 (N 1 ) 1 (B 1 ) τ R 1 +B (N ) 1 (B ) τ R )X + C τ Z ]d Z db, Y T = [B 1 N 1 ) 1 (B 1 ) τ Q 1 + B (N ) 1 (B ) τ Q ]X T. For fixed {X }, because of he exisence and uniqueness of he BSDE, we have Y = Y = B 1 (N 1 ) 1 (B 1 ) τ y 1 + B (N ) 1 (B ) τ y, Z = Z = B 1 (N 1 ) 1 (B 1 ) τ z 1 + B (N ) 1 (B ) τ z.
No. FBSDE, OPTIMAL CONTROL AND DIFFERENTIAL GAMES 191 Then (X, Y, Z ) saisfies he FBSDE (5.5) and is he unique soluion. So (X, y 1, y, z 1, z ) is one soluion of FBSDE (5.4). Now we ry o prove (u 1 ( ), u ( )) is one Nash equilibrium poin for our nonzero sum game problem. We only prove J 1 (u 1 ( ), u ( )) J 1 (v 1 ( ), u ( )), v 1 ( ) R k. I is similar o ge anoher inequaliy of (5.3). We denoe by x v1 he soluion of sysem: { dx v 1 x = a, = [Ax v1 + B 1 v 1 + B u + α ]d + [Cx v1 + β ]db, (5.6) J 1 (v 1 ( ), u ( )) J 1 (u 1 ( ), u ( )) = 1 E [ = 1 E [ ( R 1 x v1, xv1 + Q 1 x v1 T, xv1 T Q1 x T, x T ( R 1 (x v1 + Q 1 (x v1 T + R 1 x, x v1 R1 x, x + N 1 v 1, v1 N 1 u 1, u1 ) d ] x ), x v1 x + N 1 (v 1 u1 ), v1 u1 x T ), x v1 T x + N 1 u 1, v1 u1 ) d ] x T + Q 1 x T, x v1 T x T. From Q 1 x T = yt 1, we use Iô s formula o xv1 T x T, yt 1 and ge E x v1 T x T, y 1 T = E ( R 1 x, x v1 x + B 1 (v 1 u1 ), y1 ) d. Because of R 1 and Q 1 being nonnegaive, N 1 being posiive, we have So E = E J 1 (v 1 ( ), u ( )) J 1 (u 1 ( ), u ( )) ( N 1 u 1, v1 u1 + B1 (v 1 u1 ), y1 ) d ( N 1 (N 1 ) 1 (B 1 ) τ y 1, v 1 u 1 + (B 1 ) τ y 1, v 1 u 1 ) d =. (u 1, u ) = ( (N 1 ) 1 (B 1 ) τ y 1, (N ) 1 (B ) τ y ) is one Nash equilibrium poin for our nonzero sum game problem. References [1] J. Ma, P. Proer and J. Yong, Solving forward-backward sochasic differenial equaions explicily a four sep scheme, Proba. Theory and Relaed Fields, 1994, 98: 339 359. [] Y. Hu, and S. Peng, Soluion of forward-backward sochasic differenial equaions, Proba. Theory and Relaed Fields, 1995, 13: 73 83. [3] S. Peng and Z. Wu, Fully coupled forward-backward sochasic differenial equaions and applicaions o opimal conrol, SIAM J. Conrol Opim., 1999, 37: 85 843.
19 WU ZHEN Vol. 18 [4] J. Yong, Finding adaped soluion of forward backward sochasic differenial equaions-mehod of coninuaion, Proba Theory and Relaed Fields, 1997, 17: 537 57. [5] W. M. Wonham, On he separaion heorem of sochasic conrol, SIAM J. Conrol Opim., 1968, 6: 31 36. [6] J. M. Bismu, Conrole des sysemes lineaires quadraiques: applicaions de l inegrale sochasique, Lecure Noes in Mahemaics, vol.649, Seminaire de Probabilies XII, Proceedings, Srasbourg, 1976 1977, Edie par C. Dellacherie, P. A. Meyer e M. Weil, Springer-Verlag, 1978, 18 64. [7] S. Chen, X. Li and X. Zhou, Sochasic linear quadraic regulaors wih indefinie conrol weigh coss, SIAM J. Conrol Opim. 1998, 36: 1685 17. [8] S. Peng, Problem of eigenvalues of sochasic Hamilonian sysems wih boundary condiions, Sochasic Processes and Their Applicaions,, 88: 59 9. [9] A. Friedman, Differenial Games, Wiley-Inerscience, New York, 1971. [1] A. Bensoussan, Poin de Nash dans de cas de foncionnelles quadraiques e jeux différeniels à N personnes, SIAM J. Conrol, 1974, 1(3). [11] T. Eisele, Nonexisence and nonuniqueness of open-loop equilibria in linear-quadraic differenial games, J. Mah. Anal. Appl., 198, 37: 443 468. [1] S. Hamadène, Nonzero sum linear-quadraic sochasic differenial games and backward-forward equaions, Sochasic Anal. Appl. 1999, 14(). [13] W. T. Ried, Riccai Differenial Equaions, Mahemaics in Science and Engineering, Academic Press, New York, 197, 86. [14] E. Pardoux and S. Peng, Adaped soluion of a backward sochasic differenial equaion, Sysem and Conrol Leers, 199, 14: 55 61.