Mean Field Games. Math 581 Project

Transcription

1 Mean Field Games Tiago Miguel Saldanha Salvador Mah 58 Projec April 23

2 Conens Inroducion 2 2 Analysis of second order MFG 3 2. On he Fokker-Plank equaion Exisence of soluions o a 2 nd MFG Uniqueness of soluions of a 2 nd order MFG Analysis of firs order MFG 3. On he Hamilon-Jacobi equaion On he coninuiy equaion Exisence of soluions o a s order MFG A Sochasic Calculus 25 A. Brownian Moion and filraion A.2 Sochasic inegral and Iô s formula A.3 Sochasic differenial equaions B Auxiliary resuls 27 Inroducion Mean Field Games (MFG) is a class of sysems of parial differenial equaions ha are used o undersand he behaviour of muliples agens each individually rying o opimize heir posiion in space and ime, bu wih heir preferences being parly deermined by he choices of all oher agens, in he asympoic limi when he number of agens goes o infiniy. This heory has been recenly developed by J. M. Lasry and P. L. Lions in a series of papers [6, 7, 8, 9] and presened hrough several lecures of P. L. Lions a he Collège de France. The ypical model for MFG is he following: u ν u + H(x, m, D x u) = F (x, m) in R d [, T ], m ν m div(d p H(x, m, D x u)m) = in R d [, T ], (MFG) m() = m, u(x, T ) = G(x, m(t )) in R d. where ν is a non-negaive parameer. The firs equaion is an Hamilon-Jacobi equaion evolving backward in ime whose soluion is he value funcion of each agen. Indeed, he inerpreaion is he following: an average agen moves accordingly o he sochasic differenial equaion dx = α d + 2νdW where W = {W : R + } is a sandard Brownian moion and α is he conrol o be chosen by he agen. He hen wishes o minimize [ ] T E L(X s, m(s), α(s)) + F (X s, m(s))ds + G(X T, m(t ))

3 where L is he Legendre ransform of H wih respec o he las variable. The second equaion is a Fokker-Planck ype equaion evolving forward in ime ha governs he evoluion of he densiy funcion m of he agens. In his repor we will focus on sudying he exisence and uniqueness of soluions of MFG. In Secion 2 we consider (MFG) wih ν = and he Hamilonian H(p) = 2 p 2, proving he exisence and uniqueness of classical soluions. In Secion 3 we consider he same Hamilonian bu wih ν = and prove exisence and uniqueness of (weak) soluions. For boh secions we follow closely [3], rying o provide more deail in he proofs where i fel needed. Finally in he Appendix we review some basic definiions and resuls of sochasic calculus, as well as some resuls from measure heory ha are used. 2 Analysis of second order MFG Our goal in his Secion is o prove he exisence of classical soluions for he following MFG: u u + 2 D xu 2 = F (x, m) in R d (, T ) m m div(md x u) = in R d (, T ) () m() = m, u(x, T ) = G(x, m(t )) in R d Here D x u denoes he parial gradien wih respec o x. We need o inroduce some definiions. Definiion 2.. A pair (u, m) is a classical soluion o () if u, m C 2, (R d (, T )) C(R d [, T ]) and (u, m) saisfies () in he classical sense. Definiion 2.2. P is he se of Borel probabiliy measures m on R d wih finie firs order momen, i.e., R d x dm(x) <. We endow P wih he following (Kanorovich-Rubisein) disance d(µ, ν) = inf x y dγ(x, y) γ Π(µ,ν) R 2d where Π(µ, ν) is he se of Borel probabiliy measures on R 2d such ha for any Borel se A R d. γ(a R d ) = µ(a) and γ(r d A) = ν(a) We can now sae he main heorem of his Secion: Theorem 2.3. Suppose here is some consan C such ha (Bounds on F and G) F and G are uniformly bounded by C over R d P, (Lipschiz coninuiy of F and G) For all (x, m ), (x 2, m 2 ) R d P, we have F (x, m ) F (x 2, m 2 ) C ( x x 2 + d(m, m 2 )) and G(x, m ) G(x 2, m 2 ) C ( x x 2 + d(m, m 2 )), 3

4 The probabiliy measure m is absoluely coninuous wih respec o he Lebesgue measure, denoed by L d and has a Hölder coninuous densiy, sill denoe by m, which saisfies R d x 2 m (x)dx C. Then here is a leas one classical soluion o (). We will firs rea wo PDE s in () separaely: we obain some esimaes on he Fokker-Planck equaion and recall some known facs of he hea equaion. 2. On he Fokker-Plank equaion In his Secion we will derive some resuls on he following Fokker-Planck equaion m m div(mb) = in R d (, T ) (2) m() = m where b : R d [, T ] R is a given vecor field. We can look a i as an evoluion equaion on he space of probabiliy measures. We will assume ha he vecor field b is coninuous, uniformly Lipschiz in space and bounded. The reason for his is ha in he proof of Theorem 2.3 we will ake b = D x u. Definiion 2.4. We say ha m is a weak soluion o (2) if m L ([, T ], P) is such ha for any es funcion ϕ D(R d [, T )) we have R d ϕ(x, )dm (x) + T R d ( ϕ(x, ) + ϕ(x, ) D x ϕ(x, ) b(x, )) dm()(x). Consider he following sochasic differenial equaion (SDE) dx = b(x, )d + 2dW [, T ] (3) X = Z where W is a sandard d-dimensional Brownian moion and he iniial condiion Z L is random variable independen of W. Under he assumpion on b by Theorem A. here is a unique soluion o (3). The nex Lemma shows ha he soluion of (3) is closely relaed o he soluion of (2). Lemma 2.5. If L(Z ) = m, hen m() := L(X ) is a weak soluion of (2), where X is he soluion of (3). Here L(X) denoes he law (densiy funcion) of he random variable X. Proof. This is a sraighforward consequence of Iô s formula. Indeed, le ϕ C 2, (R d [, T ]). Then by Iô s formula (Theorem A.9) T T ϕ(x, ) = ϕ(z, ) + ( ϕ(x s, s) D x ϕ(x s, s) b(x s, s) + ϕ(x s, s)) ds + D x ϕ(x s, s) dw s. We know ha [ ] T E D x ϕ(x s, s) dw s =. 4

5 Hence aking he expecaion on he above equaliy leads o [ E [ϕ(x, )] = E ϕ(z, ) + So by he definiion of m we have ϕ(x, )dm()(x) = ϕ(x, )dm (x)+ R d R d Therefore for ϕ D ( R d [, T ) ) and aking = T we have R d ϕ(x, )dm (x) + T i.e., m is a weak soluion of (2). ] ( ϕ(x s, s) D x ϕ(x s, s) b(x s, s) + ϕ(x s, s)) ds. R d ( ϕ(x, s) Dϕ(x, s) b(x, s) + ϕ(x, s)) dm(s)(x)ds. R d ( ϕ(x, ) Dϕ(x, ) b(x, ) + ϕ(x, )) dm()(x)ds =, The above inerpreaion of m as he probabiliy densiy of he soluion of (3) allows us o show ha he map m() is Hölder coninuous. Lemma 2.6. Le m() := L(X ) where X is he soluion of (3). Then here is a consan c = c (T ) (i.e., depending only on T ), such ha for all s, [, T ] d(m(), m(s)) c ( + b ) s /2. Proof. We sar by observing ha he probabiliy measure γ of he pair (X, X s ) belongs o Π(m(), m(s)). Therefore d(m(), m(s)) x y dγ(x, y) = E [ X X s ]. R 2d Wihou loss of generaliy suppose s <. Then [ E [ X X s ] = E b(x τ, τ)dτ + ] 2(W W s ) s [ E b(x τ, τ) dτ + ] 2 W W s s b ( s) + 2 s π ( ) 2 s b T + π { } T 2 s ( b + ) max, π So by aking c = max { T, 2 π } we are done. We can also obain easily an esimae on he second order momen of m Lemma 2.7. Le m() := L(X ) where X is he soluion of (3). Then here is a consan c = c (T ) such ha for all [, T ] ( ) x 2 dm()(x) c R d x 2 dm (x) + + b 2 R d. 5

6 Proof. By definiion of m we have Hence R d x 2 dm()(x) = E [ X 2] [ ] 2 x 2 dm()(x) 3E X 2 + b(x s, s)ds + 2 W 2 R d ( ) 3 x 2 dm (x) + b R d ) c x (R 2 dm (x) + b 2 + d where c = max{3, 3T 2, 6T }. 2.2 Exisence of soluions o a 2 nd MFG In his Secion we prove Theorem 2.3. In order o do ha we need firs o recall some exisence and uniqueness resuls for he following hea equaion w w + a(x, ) Dw + b(x, )w = f(x, ) in R d [, T ] (4) w(x, ) = w (x) in R d where a, b, f : R d [, T ] R and w : R d R. For his we inroduce some noaion. Definiion 2.8. Le s be an ineger and α (, ). We denoe by C s,α (R d [, T ]) he se of funcions f : R d [, T ] R such ha for any pair (k, l) wih 2k + l s, k D l xf exiss and such ha hese derivaives are bounded, α-hölder coninous in space and α/2-hölder coninous in ime. We hen have he following heorem whose proof can be found in [5]: Theorem 2.9. Suppose ha a, b, f C,α (R d [, T ]) and ha w C,α (R d ) (he classical Hölder space). Then (4) has a unique weak soluion u C 2,α (R d [, T ]). We also have he following inerior esimae: Theorem 2.. Suppose a b and ha f C(R d [, T ]) is bounded. Then any classical bounded soluion w of (4) saisfies, for any compac se K R d (, T ) D x w(x, ) D x w(y, s) sup C f (x,),(y,s) K x y β + s β/2 where β (, ) depends only on he dimension d and C = C(K, w, d). The idea of he proof is o consruc a map Ψ such ha a fixed poin of Ψ is a soluion of he sysem (). Then we use he Schauder fixed poin heorem o prove he exisence of he fixed poin. Theorem 2. (Schauder fixed poin). Le K be a convex, closed and compac subspace of a opological vecor space V and Ψ : K K a coninuous map. Then Ψ has a fixed poin. 6

7 Proof of Theorem 2.3. Le C be a large consan o be fixed laer and le M be he se of maps µ C([, T ], P) such ha and o d(µ(), µ(s)) sup C s, [,T ] s /2 s sup x 2 dµ()(x) C. [,T ] R d To any µ M we associae an m = Ψ(µ) M in he following way: le u be he unique soluion u u + 2 D xu 2 = F (x, µ()) in R d [, T ] (5) u(x, T ) = G(x, µ(t ) in R d Then we define m = ψ(µ) M as he unique soluion of he Fokker-Plank equaion m m div(md x u) = in R d [, T ] (6) m() = m (x) in R d In order o apply he Schauder fixed poin heorem, we need o show ha: M is a convex closed and compac subse of C([, T ], P), Ψ is well defined and Ψ is coninuous. ) M is a convex closed and compac subse of C([, T ], P). Le λ [, ], µ, µ 2 M, γ Π(µ (), µ (s)) and γ 2 Π(µ 2 (), µ 2 (s)). We have ha and herefore λγ + ( λ)γ 2 Π(λµ () + ( λ)µ 2 (), λµ (s) + ( λ)µ 2 (s)) d(λµ () + ( λ)µ 2 (), λµ (s) + ( λ)µ 2 (s)) x y d(λγ (x, y) + ( λ)γ 2 (x, y)) R 2d = λ x y dγ (x, y) + ( λ) x y dγ 2 (x, y)). R 2d R 2d Then aking he infimum over γ Π(µ (), µ (s)) and γ 2 Π(µ 2 (), µ 2 (s)) shows ha d(λµ () + ( λ)µ 2 (), λµ (s) + ( λ)µ 2 (s)) λd(µ (), µ (s)) + ( λ)d(µ 2 (), µ 2 (s)). We also have x 2 d(λµ + ( λ)µ 2 )()(x) = λ R d x 2 dµ ()(x) + ( λ) R d x 2 dµ 2 ()(x). R d From he las wo equaliies i s now easy o see ha, indeed, λµ + ( λ)µ 2 M and so M is convex. Now le µ n M such ha µ n µ in C([, T ], P). To prove ha M is closed we need o show ha µ M. 7

8 I s easy o show ha and from his i follows easily ha d(µ(), µ(s)) d(µ() µ n (), µ(s) µ n (s)) + d(µ n (), µ n (s)) d(µ(), µ(s)) sup C. s, [,T ] s /2 s As for he second order momen esimae we noe ha x 2 dµ()(x) = R d x 2 d(µ() µ n ())(x) + R d x 2 dµ n ()(x) R d Taking he supremum for [, T ] we ge sup x 2 dµ()(x) sup x 2 d(µ() µ n ())(x) + sup [,T ] R d [,T ] R d sup x 2 d(µ() µ n ())(x) + C [,T ] R d Now since µ n µ in C([, T ], P), by aking n we ge as desired. sup [,T ] R d x 2 dµ()(x) C [,T ] For he proof ha M is compac we refer he reader o Lemma 5.7 of [3]. 2) ψ is well-defined. Firs we need o see ha a soluion of (5) exiss and is unique. R d x 2 dµ n ()(x) Consider hen he Hopf-Cole ransformaion given by w = e u/2. Then i is easy o check ha u is a soluion of (5) if and only if w is a soluion of The map (x, ) F (x, m()) belongs o C,/2 bounded over R d P and w w = wf (x, µ()) in R d [, T ] (7) w(x, T ) = e G(x,µ(T ))/2 in R d since F is Lipschiz in boh variables, uniformly d(µ(), µ(s)) sup C. s, [,T ] s /2 s because µ M. The map x e G(x,µ(T ))/2 is in C,/2 (R d ) since G is Lipschiz in x and uniformly bounded over R d P. Then appealing o Theorem 2.9 here is a unique soluion in C 2,/2 o (7) which implies ha here is a unique soluion in C 2,/2 o (5). Recall ha, by assumpion, he maps (x, ) F (x, m()) and x G(x, µ(t ) are bounded by C. Hence a sraighforward applicaion of he comparison principle implies ha u is bounded by ( + T )C. Similarly he maps x F (x, m()) and x G(x, µ(t ) are C -Lipschiz coninuous (again by our assumpions on F and G) and so u is also C -Lipschiz coninuous. Hence D x u is bounded by C. 8

9 Now we look a he Fokker-Planck equaion (6). By expanding he divergence erm, we can wrie i ino he form m m D x m D x u(x, ) m u(x, ) = in R d (, T ) m() = m Since u C 2,/2, he maps (x, ) D x u(x, ) and (x, ) u(x, ) belong o C,/2. Also by assumpion m C,α (R d ). Hence by Theorem (2.9) here is a unique soluion m C 2,/2 o (6). Moreover, by Lemma 2.6, for all s, [, T ] and by Lemma 2.7 for all [, T ] d(m(), m(s)) c ( + C ) s 2 R d x 2 dm()(x) c (C + + C 2 ) where c depends only on T. So if we choose C = max{c ( + C ), c (C + + C 2 )}, m M and Ψ is hen well-defined. 3) Ψ is coninuous. Le µ n M converge o some µ. Le (u n, m n ) and (u, m) be he corresponding soluions. Noe ha (x, ) F (x, µ n ()) and x G(x, µ n (T )) converge locally uniformly o (x, ) F (x, µ()) and x G(x, µ(t ) respecively. Hence we can conclude ha (u n ) converges locally uniformly o u by a sandard argumen wih viscosiy soluions. Since he (D x u n ) are uniformly bounded (by C ), he (u n ) solve an equaion of he form u n u n = f n where f n = 2 D xu n 2 F (x, m n ) is uniformly bounded in x and n. Then by Theorem 2. (D x u n ) is locally uniform Hölder coninuous and herefore converge locally uniform o D x u. This implies ha any converging subsequence of he relaively compac sequence (m n ) is a weak soluion of (6). Bu m is he unique soluion of (6), which proves ha (m n ) converges o m. Hence Ψ is coninuous. Finally, by he Schauder fixed poin heorem, he coninuous map µ m = Ψ(µ) has a fixed poin in M. To his fixed poin m M corresponds a pair (u, m) ha is a classical soluion o () and so we are done. 2.3 Uniqueness of soluions of a 2 nd order MFG In his Secion we prove a uniqueness resul o he sysem (). Theorem 2.2. Besides he assumpions of Theorem 2.3, assume ha For all m, m 2 P wih m m 2 we have R d (F (x, m ) F (x, m 2 ))d(m m 2 )(x) >, 9

10 For all m, m 2 P Then here is a unique soluion o (). R d (G(x, m ) G(x, m 2 ))d(m m 2 )(x). Proof. Le (u, m ) and (u 2, m 2 ) be wo classical soluions of (). We se u = u u 2 and m = m m 2. Then u u + 2 ( D xu 2 D x u 2 2 ) (F (x, m ) F (x, m 2 )) = m m div(m D x u m 2 D x u 2 ) = Since u C 2, (R d (, T )), we can muliply he second equaion by u, inegrae over R d [, T ], followed by par inegraion o ge m(t )u(x, T )dx + R d m (x)u(x, )dx + R d T (8) R d ( u + u)m Du (m D x u m 2 D x u 2 )dxd. Muliplying now he firs equaion by m, inegraing over R d [, T ] and adding o he previous equaliy, leads o m(t )(G(x, m (T )) G(x, m 2 (T ))dx R d T ( + m ) 2 D xu D x u 2 2 m(f (x, m ) F (x, m 2 ))) dxd = R d where we used he fac ha m() = and ha m 2 ( D xu 2 Du 2 2 ) D x u (m D x u m 2 D x u 2 ) = m 2 D xu D x u 2 2. By assumpion m(t )(G(x, m (T )) G(x, m 2 (T ))dx R d and herefore T m(f (x, m ) F (x, m 2 ))dxd. R d Hence, by our assumpion on F, his implies ha m = and herefore u = since u and u 2 (now) solve he same equaion. We finish his Secion by menioning ha he exisence of soluions for second order MFG hold under more general assumpions. Indeed, in [7, 8] he auhors consider equaions of he form u(x, ) u + H(x, Du) = F (x, m)) in Q (, T ) m(x, ) m div(m H p (x, D xu)) = in Q (, T ) m() = m, u(x, T ) = G(x, m(t )) in Q where Q = [, ] d (wih periodic boundary condiions), H : R d R d is Lipschiz coninuous wih respec o x and uniformly bounded in p, convex and of class C wih respec o p. The condiions on F and G are one of he following: F and G are regularizing, i.e., saisfy he same condiions as in Theorem 2.3. F (x, m) = f(x, m(x)) and G(x, m) = g(x, m(x)), where f = f(x, λ) and g = g(x, λ) saisfy suiable growh condiions wih respec o λ and H is sufficienly sricly convex.

11 3 Analysis of firs order MFG In his Secion we will prove he exisence of soluions o he following firs order MFG: u(x, ) + 2 Du(x, ) 2 = F (x, m()) in R d (, T ) m(x, ) div(du(x, )m(x, ) = in R d (, T ) (9) m() = m, u(x, T ) = G(x, m(t )) in R d We consider he following definiion of weak soluions. Definiion 3.. We call he pair (u, m) a weak soluion of (9) if u W, loc (Rd [, T ]), m L (R d (, T )) such ha he firs equaion of (9) is saisfied in he viscosiy sense and he second in saisfied in he sense of disribuions. Noe ha here we don look any more for classical soluions mainly because we no longer have he smoohing erms u and m. Our goal is hen o prove he following. Theorem 3.2. Suppose ha. F and G are coninuous over R d P, 2. There is a consan C such ha for any m P, F (, m), G(, m) C 2 (R d ) and F (, m) C2 (R d ) C G(, m) C2 (R d ) C where for f C 2 (R d ) we denoe C 2 (R d ) by f C 2 (R d ) = sup x R d { f(x) + Df(x) + D 2 f(x) }, 3. m is absoluely coninuous wih respec o he Lebesgue measure and has a densiy, sill denoed by m, which is bounded and has a compac suppor. Then here is a leas one weak soluion of (9). Remark 3.3. Under he assumpions of Theorem 2.2 we can show ha he soluion is unique. The proof is he same wih he only difference being ha now we use he Lipschiz coninuous map u as a es funcion because he densiy m is bounded and has compac suppor. As in Secion 2, we will sudy he wo equaions separaely. 3. On he Hamilon-Jacobi equaion In his Secion we sudy he Hamilon-Jacobi equaion u + 2 D xu 2 = f(x, ) in R d (, T ) () u(x, T ) = g(x) in R d We will sar by recalling some basic facs abou he noion of semi-concaviy which will play a role here. The proofs for hese resuls can be found in [2].

12 Definiion 3.4. A map w : R d R is semi-concave if here is some consan C > such ha one of he following equivalen condiions is saisfied:. he map x w(x) C 2 x 2 is concave in R d. 2. w(λx + ( λ)y) λw(x) + ( λ)w(y) Cλ( λ) x y 2 for any x, y R d and λ [, ]. 3. D 2 w CI d in he sense of disribuions. 4. (p q) (x y) C x y 2 for any x, y R d, [, T ], p D + x w(x) and q D + x w(y), where D + x w denoes he super-differenial of w wih respec o he x variable, namely D + x w(x) = {p R d : lim sup y x w(y) w(x) p (y x) y x }. Lemma 3.5. Le w : R d R be semi-concave. Then w is locally Lipschiz coninuous in R d. Moreover D + x w(x) is he closed convex hull of he se D xw(x) of reachable gradiens defined by D xw(x) = {p R d : (x n ) wih x n x such ha D x w(x n ) exiss and converges o p} In paricular, D + x w(x) is compac, convex and non empy subse of R d for any x R d. Finally w is differeniable a x if and only if D + w(x) is a singleon. Lemma 3.6. Le (w n ) be a sequence of uniformly semi-concave maps on R d which converge poinwiseo a map w : R d R. Then he convergence is locally uniform and w is semi-concave. Moreover, for any x n x and any p n D + w n (x n ), he se of accumulaion poins of (p n ) is conained in D + w(x). Finally, Dw n (x) converges o Dw(x) for a.e. x R d. Definiion 3.7. Le (x, ) R d [, T ]. We denoe by A(x, ) he nonempy se of opimal conrols o u(x, ), i.e., α L 2 ([, T ], R d ) such ha T u(x, ) = 2 α(s) 2 + f(x(s), s) ds + g(x(t )) where x(s) = x + s α(τ)dτ. We call x( ) he associaed rajecory o he conrol α. Lemma 3.8. If (x n, n ) (x, ) wih α n A(x n, n ), hen, up o a subsequence, (α n ) weakly converges in L 2 o some α A(x, ). We can now sudy equaion (). Lemma 3.9. Le f : R d [, T ] R and g : R d R be coninuous funcions. For any [, T ], f(, ), g C 2 (R d ) wih f(, ) C 2 C, g C 2 C () for some consan C. Then equaion () has a unique bounded uniformly coninuous viscosiy soluion which is given by he represenaion formula u(x, ) = T inf α L 2 ([,T ],R d ) 2 α(s) 2 + f(x(s), s) ds + g(x(t )), 2

13 where x(s) = x + α(τ)dτ. Moreover u is Lipschiz coninuous and saisfies s D x, u C, D 2 xxu C I d where he las inequaliy holds in he sense of disribuions. Proof. From he heory of Hamilon-Jacobi equaions we already know ha () has a unique bounded uniformly coninuous viscosiy soluion given by u(x, ) = α L 2 ([,T ],R d ) T 2 α(s) 2 + f(x(s), s) ds + g(x(t )). Hence we only need o check ha u is Lipschiz coninuous wih D x, u C and D 2 xxu C I d in he sense of disribuions for some consan C = C (T ). ) u is Lipschiz coninuous wih respec o x. Le x, x 2 R d, [, T ] and α A(x, ). We hen have u(x 2, ) T T 2 α(s) 2 + f(x(s) + x 2 x, s) ds + g(x(t ) + x 2 x ) 2 α(s) 2 + f(x(s), s) + C x 2 x ds + g(x(t )) + C x 2 x u(x, ) + C(T + ) x 2 x Thus u is Lipschiz coninuous wih respec o x wih Lipschiz consan C(T + ). 2) u is Lipschiz coninuous wih respec o. Fix x R d and [, T ]. From he dynamic programming principle we have for any < s T u(x, ) = s 2 α(τ) 2 + f(x(τ), τ)dτ + u(x(s), s) where α A(x, ) and x( ) is is associaed rajecory. We have u(x, ) u(x, s) u(x, ) u(x(s), s) + u(x(s), s) u(x, s) s 2 α(τ) 2 + f(x(τ), τ) dτ + C(T + ) x(s) x (s ) 2 α 2 + f + C(T + ) where in he second inequaliy we used he fac u is C(T + )-Lipschiz coninuous wih respec o x. In Lemma 3., we show ha α is bounded by a consan C 2 = C 2 (T ). Hence he inequaliy above proves ha u is Lipschiz coninuous wih respec o. 3) D x, u C. I follows easily from ) and 2). 3

14 4) D 2 xxu C I d in he sense of disribuions. Le x, y R d, [, T ], λ [, ] and se x λ = λx + ( λ)y. By Definiion 3.4 i s enough o show ha λu(x, ) + ( λ)u(y, ) u(x λ, ) + Cλ( λ) x y 2 where C = C(T ) is a consan. Le α A(x λ, ) and x λ ( ) is associaed rajecory. Then [ T ] λu(x, ) + ( λ)u(y, ) λ 2 α(s) 2 + f(x λ (s) + x x λ, s) ds + g(x λ (T ) + x x λ ) [ T ] + ( λ) 2 α(s) 2 + f(x λ (s) + y x λ, s) ds + g(x λ (T ) + y x λ ) T 2 α(s) 2 + f(x λ (s), s) ds + g(x λ (T ) + C(T + )λ( λ) x y 2 Hence u is semi-concave. = u(x λ, ) + α (T + )λ( λ) x y 2. Lemma 3. (Euler-Lagrange opimaliy condiion). If α A(x, ), hen α is of class C ([, T ]) wih α (s) = Df(x(s), s) in [, T ] α(t ) = Dg(x(T )) In paricular, here is a consan C = C (C) such ha for (x, ) R d [, T ) and any α A(x, ) we have α C, where C saisfies (). Lemma 3. (Regulariy of u along opimal soluion). Le (x, ) R d [, T ], α A(x, ) and le us se x(s) = x + α(τ)dτ. Then s. (Uniqueness of he opimal conrol along opimal rajecories) for any s (, T ], he resricion of α o [s, T ] is he unique elemen of A(x(s), s). 2. (Uniqueness of he opimal rajecories) D x u(x, ) exiss if and only if A(x, ) is a reduced o singleon. In his case, D x u(x, ) = α() where A(x, ) = {α}. Remark 3.2. In paricular, if we combine he above saemens, we see ha u(, s) is always differeniable a x(s) for s (, T ) wih D x u(x(s), s) = α(s). Proof. Le α A(x(s), s) and le x ( ) be is associaed rajecory. For any h > sufficienly small we define α h L 2 ([, T ], R d ) in he following way α(τ) if τ [, s h) x α h (τ) = (s+h) x(s h) 2h if τ [s h, s + h) α (τ) if τ [s + h, T ] 4

15 Then one easily checks ha x(τ) if τ [, s h) x h (τ) = x(s h) + (τ (s h)) x(s+h) x(s h) 2h if τ [s h, s + h) x (τ) if τ [s + h, T ] Since boh α [s,t ] and α are opimal for u(x(s), s), α, which is nohing bu he concaenaion of α [,s] and α, is also opimal for u(x, ). Also observe ha x (τ) = x + τ α (σ)dσ is given by x(τ) on [, s] and x (τ) on [s, T ]. Hence and u(x, ) = s T 2 α(τ) 2 + f(x(τ), τ)dτ + s 2 α (τ) 2 + f(x (τ), τ) dτ + g(x (T )) u(x, ) T s 2 α h(τ) 2 + f(x h (τ), τ) dτ + g(x h (T )). Using he definiions of α h and x h we can wrie he above inequaliy as s s h 2 α(τ) 2 + f(x(τ), τ) dτ + s+h s s+h s h Now dividing h and aking h + shows ha 2 α (τ) 2 + f(x (τ), τ) dτ ( ) 2 x (s + h) x(s h) 2 2h + f(x h (τ), τ) dτ 2 α(s) α (s) 2 4 α(s) + α (s) 2 since lim h x h (s) = x(s) = x (s). Therefore α(s) α (s) 2, i.e., α(s) = α (s). In paricular x( ) and x ( ) saisfy he same second order differenial equaion y (τ) = D x f(y(τ), τ) y (s) = x (s) = α(s) = α (s) = x (s) y(s) = x(s) = x (s) Hence x( ) = x ( ) and α = α on [s, T ]. This means ha he opimal soluion for u(x(s), s) is unique, hus proving poin. We now show ha if D x u(x, ) exiss, hen A(x, ) is reduced o singleon and D x u(x, ) = α() where A(x, ) = {α}. Indeed, le α A(x, ) and x( ) be he associaed rajecory. Then for any v R d u(x + v, ) T 2 α(s) 2 ds + f(x(s) + v, s)ds + g(x(t ) + v). Since equaliy holds for v = and since boh sides of he inequaliy are differeniable wih respec o v a v = we ge D x u(x, ) = T D x f(x(s), s)ds + D x g(x(t )). 5

16 Then by Lemma 3. we have D x u(x, ) = α(). Therefore x( ) has o be he unique soluion of he second order differenial equaion x (s) = D x f(x(s), s) x () = D x u(x, ) x() = x which in urn implies ha α = x is unique. Conversely, suppose ha A(x, ) is a singleon. We wan o show ha u(, ) is differeniable a x. For his we noe ha, if p belongs o Dxu(x, ) (he se of reachable gradiens of he map u(, )), hen he soluion x (s) = D x f(x(s), s) x () = p x() = x is opimal. Indeed, by definiion of p here is a sequence x n x such ha u(, ) is differeniable a x n and D x u(x n, ) p. Now since u(, ) is differeniable a x n, we know ha he unique soluion x n ( ) of x n(s) = D x f(x n (s), s) x n () = x x n() = Du(x n, ) is opimal. Passing o he limi as n implies by Lemma 3.8 ha x( ), which is he uniform limi of he x n ( ), is also opimal. Bu from our assumpions, here is a unique opimal soluion in A(x, ). Hence Dxu(x, ) has o be reduced o a singleon and since u(, ) is semi-concave by Lemma 3.9, we have ha u(, ) is differeniable a x by Lemma 3.5. Le us consider again (x, ) R d [, T ), α A(x, ) and x( ). Then we have jus proved ha u(, s) is differeniable a x(s) for any s (, T ) wih x (s) = α(s) = D x u(x(s), s). So given α opimal, is associaed rajecory x( ) is a soluion of he differenial equaion x (s) = D x u(x(s), s) on [, T ] x() = x The following Lemma, saes ha he reverse also holds. This is an opimal synhesis resul since i says he opimal conrol can be obained a each posiion y and a each ime s as by he synhesis α (y, s) = D x u(y, s). 6

17 Lemma 3.3 (Opimal synhesis). Le (x, ) R d [, T ) and x( ) be an absoluely coninuous soluion o he differenial equaion x (s) = D x u(x(s), s), a.e. in [, T ] x() = x Then he conrol α := x is opimal for u(x, ), i.e., α A(x, ). In paricular, if u(, ) is differeniable a x, hen equaion (2) has a unique soluion, corresponding o he opimal rajecory. Proof. We sar by observing ha x( ) is Lipschiz coninuous because u is. Le s (, T ) be such ha equaion (2) holds. Hence u is differeniable wih respec o x a (x(s), s) and he Lipschiz coninuous map s u(x(s), s) has a derivaive a s. Since u is Lipschiz coninuous, Lebourg s mean value heorem ([4], Th. (2) 2.3.7), saes ha, for any h > small enough here is some (y h, s h ) [(x(s), s), (x(s + h), s + h)] and some (ξ h x, ξ h ) CoD xu(y h, s h ) wih u(x(s + h), s + h) u(x(s), s) = ξ h x (x(s + h) x(s)) + ξ h h, (3) where CoDx,u(y, s) denoes he closure of he convex hull of he se of reachable gradiens Dx,u(y, s). Now from Carahéodory Theorem, here are (λ h,i, ξx h,i, ξ h,i ) i=,...,d+2 such ha λ h,i, d+2 i= d+2 λ h,i =, (ξx h,i, ξ h,i ) Dxu(y h, s h ) and (ξx, h ξ h ) = i= λ h,i (ξx h,i, ξ h,i ). For each i =,..., d + 2, he ξx h,i converges o D x u(x(s), s) as h because, from Lemma 3.6, any accumulaion poin of (ξx h,i ) h mus belong o D x + u(x(s), s) which is reduced D x u(x(s), s) since u(, s) is differeniable a x(s). Therefore ξ x,h = i λ h,i ξ h,i x D x u(x(s), s) as h. Since u is a viscosiy soluion o () and (ξ h,i x Therefore d+2 ξ h = i= λ h,i ξ h,i = d+2 2 ξ h,i + 2 ξh,i x 2 = f(y h, s h ). as h. Then dividing (3) by h and leing h we ge i=, ξ h,i ) Dx,u(x(s), s) we have λ h,i ξ h,i x 2 f(y h, s h ) 2 D xu(x(s), s) 2 f(x(s), s) d ds u(x(s), s) = D xu(x(s), s) x (s) + 2 D xu(x(s), s) 2 f(x(s), s). and, since x (s) = D x u(x(s), s), we have d ds u(x(s), s) = 2 x (s) 2 f(x(s), s) a.e. in (, T ). Inegraing he above inequaliy over [, T ] we finally obain u(x, ) = T 2 x (s) 2 + f(x(s), s)ds + g(x(t )) where we used he fac ha u(y, T ) = g(y) for y R d. Therefore α := x is opimal. The las saemen of he Lemma is a jus direc consequence of poin 2. of Lemma 3.. 7

18 From he sabiliy of opimal soluions, he graph map (x, ) A(x, ) is closed when he se L 2 ([, T ], R d ) is endowed wih he weak opology. This implies ha he map (x, ) A(x, ) is measurable wih nonempy closed values, so ha i has a Borel measurable selecion ᾱ: namely ᾱ(x, ) A(x, ) for any (x, ) (see []). Fix (x, ) R d (, T ). We define he flow for all s [, T ]. equaion. Φ(x,, s) = x + s ᾱ(x, )(τ)dτ We will use i in he nex Secion o consruc a soluion o he Fokker-Planck Lemma 3.4. The flow Φ has he semi-group propery Φ(x,, s ) = Φ(Φ(x,, s), s, s ) (4) for all s s T. Moreover for any x R d and s, s (, T ) s Φ(x,, s) = D x u(φ(x,, s), s) and Φ(x,, s ) Φ(x,, s) D x u s s. Proof. For any s (, T ) we know from Lemma 3. ha A(Φ(x,, s), s) = {ᾱ(x, ) [s,t ] } and so (4) holds. Moreover, Lemma 3. also saes ha u(, s) is differeniable a Φ(x,, s) wih D x u(φ(x,, s), s) = ᾱ(x, )(s). Bu by definiion s Φ(x,, s) = ᾱ(x, )(s) and so s Φ(x,, s) = D x u(φ(x,, s), s). Finally his las equaliy also implies he D x u -Lipschiz coninuiy of Φ(x,, ) on (, T ). We finish his Secion wih he following conracion propery of he flow Φ. Lemma 3.5. If C saisfies (), hen here is some consan C 2 = C 2 (C) such ha, if u is a soluion of (), hen for all s T and x, y R d x y C 2 Φ(x,, s) Φ(y,, s). In paricular, he map x Φ(x,, s) has a Lipshiz coninuous inverse on he se Φ(R d,, s). Proof. Le u be he soluion of (). Then by Lemma 3.9 D 2 xxu C I d on R d (, T ) in he sense of disribuions. Le x(τ) = Φ(x,, s τ) and y(τ) = Φ(y,, s τ) for τ [, s ]. Then from Lemma 3.4, x( ) and y( ) saisfy respecively x (τ) = D x u(x(τ, s τ) τ [, s ) x() = Φ(x,, s) and y (τ) = D x u(y(τ, s τ) τ [, s ) y() = Φ(y,, s) We observe ha for almos all τ [, s ] we have d (x y)(τ) 2 = ((x y )(τ)) ((x y)(τ)) C (x y)(τ) 2 dτ 2 8 (5)

19 where he las inequaliy comes from (5) and he fac ha Dxxu 2 C I d (see Definiion 3.4). Hence by Grownwall s inequaliy (x y)(τ) e C/2τ x() y() for all τ [, s ]. In paricular for τ = s we ge x y e C/2τT Φ(x,, s) Φ(y,, s) hus proving he claim. 3.2 On he coninuiy equaion Our aim is now o show ha, given a soluion () and under assumpion (), he coninuiy equaion µ(x, s) div(d x u(x, s)µ(x, s)) = in R d (, T ) (6) µ(x, ) = m (x) in R d has a unique soluion which is he densiy of he measure µ(s) = Φ(,, s) m for s [, T ], where Φ(,, s) m denoes he push-forward of he measure m by he map Φ(,, s), i.e., he measure defined by Φ(,, s) m (A) = m (Φ(,, s) (A)) for any Borel se A R d. We sar by observing ha he measure Φ(,, s) m is absoluely coninuous wih respec o he Lebesgue measure. Lemma 3.6. Le C be a consan such ha () holds and such ha m is absoluely coninuous, has a compac suppor conained in he ball B(, C) and saisfies m L C. Le us se µ(s) := Φ(,, s) m for s [, T ]. Then here is a consan C 3 = C 3 (C) such ha, for any s [, T ], µ(s) is absoluely coninuous, has a compac suppor conained in he ball B(, C 3 ) and saisfies µ(s) L C 3. Moreover d(µ(s ), µ(s)) D x u s s for all s s T. Proof. By definiion µ saisfies d(µ(s ), µ(s)) Φ(x,, s ) Φ(x,, s) dm (x) D x u (s s). R d Recall ha Φ is given by Φ(x,, s) = x + s ᾱ(x, )(τ)dτ where ᾱ(x, )(τ) = D x u(φ(x,, τ), τ). Also since u is a soluion of (), D x u C. Addiionally, m has compac suppor conained in B(, C). Hence he (µ(s)) have a compac suppor conained in B(, R) where R = C + T C. Le us now fix [, T ]. From Lemma 3.5, we know ha here is some C 2 = C 2 (T ) such ha he map x Φ(x,, ) has a C 2 -Lipschiz coninuous inverse on he se Φ(R d,, ). Le us denoe his inverse by Ψ. Then, if A is a Borel subse of R d we have µ(s)(a) = m (Φ (,, )(A)) = m (Ψ(A)) m L d (Ψ(A)) m C 2 L d (A). 9