Differentiability of convex functions, subdifferential

Transcription

1 Differeniabiliy of convex funcions, subdifferenial Basic definiions and general facs. Le us recall a few basic facs abou differeniabiliy of real-valued funcions. Le X be a normed space, A X an open se, f : A R a funcion, and a A a poin. For a direcion v X (no necessarily of norm one), we shall consider he righ direcional derivaive f +(a, v), he lef direcional derivaive f (a, v), and he (bilaeral) direcional derivaive f (a, v), which are defined by: f +(a, f(a + v) f(a) v) = lim, 0+ f (a, f(a + v) f(a) v) = lim, 0 f f(a + v) f(a) (a, v) = lim. 0 I is easy o see ha f (a, v) = f +(a, v). Thus f (a, v) exiss if and only if boh f +(a, ±v) exis and f +(a, v) = f +(a, v). Anoher easy fac is ha f +(a, 0) = 0 and f +(a, λv) = λf +(a, v) for each λ > 0. Definiion 0.. Le X, A, f, a be as above. We shall say ha f is: Gâeaux differeniable a a if here exiss x X such ha f (a, v) = x (v) for each v X (ha is, f (a, ) is everywhere defined, real-valued, linear and coninuous); Fréche differeniable a a if here exiss x X such ha f(a + h) f(a) x (h) lim = 0. h 0 h The funcional x is called he Gâeaux/Fréche differenial (or derivaive) of f a a, and i is denoed by f (a). I is easy o see ha he noions of Gâeaux and Fréche differeniabiliy are local noions (i.e., hey depend only on he values of f a a neighborhood of he poin) and hey do no change if we pass o an equivalen norm on X. Observaion 0.2. The following asserions are equivalen: (i) f is Gâeaux differeiable a a; (ii) f +(a, ) X ; (iii) here exiss x X such ha for each v X one has f(a + v) = f(a) + x (v) + o() as 0. Observaion 0.3. The following asserions are equivalen: (i) f is Fréche differeiable a a; (ii) here exiss x X such ha f(a + h) = f(a) + x (h) + o( h ) as h 0;

2 2 (iii) f is Gâeaux differeniable a a and he limi f(a + v) f(a) lim 0 is uniform for v =. = f (a)(v) I is well known ha we have always he implicaions f is Fréche diff. a a = f is Gâeaux diff. a a f is coninuous a a f (a, v) exiss finie for each v and no oher implicaion holds. (Of course, he Gâeaux and Fréche differenial of f a a coincide whenever boh of hem exis.) In finie-dimensional spaces we have he following imporan consequence of compacness of he uni sphere. Lemma 0.4. Le X, A, f, a be as above. Assume ha X = R d and f is Lipschiz on some neighborhood of a. Then f is Fréche differeniable a a if and only if f is Gâeaux differeniable a a. Proof. We can suppose ha f is Lipschiz on A wih a Lipschiz consan L. Suppose ha f is Gâeaux differeniable a a wih f (a) = x bu no Fréche differeniable a a. There exiss a sequence {h n } X \ {0} such ha h n 0 and () D n := f(a + h n) f(a) x (h n ) h n 0. We can wrie h n = n v n where n = h n and v n = hn h n. By compacness of he uni sphere S R d, we can assume ha v n v 0 S R d. Then we obain D n = f(a + n v n ) f(a) x (v n ) n f(a + n v 0 ) f(a) x (v 0 ) n + f(a + nv n ) f(a + n v 0 ) + x (v 0 ) x (v n ) n f(a + n v 0 ) f(a) x (v 0 ) n + ( L + x ) v n v 0 0, a conradicion wih (). Derivaives and differenials of convex funcions. In wha follows, A will be an open convex subse of a normed space X, f : A R a convex funcion, a A a poin. The following proposiion collecs main imporan properies of righ direcional derivaives of a convex funcion a a poin. Proposiion 0.5. Le X, A, f, a be as above. (a) f +(a, v) exiss finie for each v X, and he funcional p = f +(a, ) is sublinear on X. (b) p( v) p(v) for each v X.

3 (c) The se V = {v X : f (a, v) exiss} = {v X : p( v) = p(v)} is linear (ha is, V is a subspace of X) and he resricion p V is linear. (d) If f is coninuous, hen p is Lipschiz (in paricular, p V V ). Moreover, he properies (b),(c) hold for any sublinear funcional p: X R. Proof. (a) Exisence of f +(a, v) follows immediaely from he well known facs abou exisence of one-sided derivaives of a convex funcion of one real variable (indeed, f +(a, v) = ϕ +(0) where ϕ() = f(a + v)). We already know ha p is posiively homogeneous. To show subaddiiviy, consider u, v X and compue p(u + v) = 2 p ( ) u+v f( ) f(a) 2 = 2 2 (a+u)+ 2 (a+v) lim lim f(a+u)+ 2 f(a+v) f(a) 0+ = p(u) + p(v). (b) is easy: 0 = p(0) = p(v + ( v)) p(v) + p( v). (c) Now, le u, v V and λ R. By posiive homogeneiy of p, if λ 0 hen λv V and p(λv) = λp(v). By definiion of V, we have v V and p( v) = p(v). For λ < 0, we have λ > 0 and hence p( λv) = λp(v) = λp( v) = p(λv), which shows ha λv V and p(λv) = λp(v). Finally, he inequaliies p(u + v) p(u) + p(v) = [ p( u) + p( v) ] p( u v) (b) p(u + v) imply ha u + v V and p(u + v) = p(u) + p(v). (d) If f is coninuous, i is Lipschiz on a neighborhood of 0 wih some consan L. This easily implies ha p(v) L v for each v X. For u, v X, we have by subaddiiviy p(u) p(v) = p(v + (u v)) p(v) p(u v) L u v. By inerchanging he role of u and v, we obain ha p is L-Lipschiz on X. Corollary 0.6. Le X, A, f, a be as above. If f (a, v) exiss for each v X, hen f is Gâeaux differeniable a a. Proof. In he noaion of Proposiion 0.5, V = X. Thus f (a, ) X. Corollary 0.7. Le A R d be an open convex se, f : A R a convex funcion, a A. If all parial derivaives f x i (a) (i =,..., d) exis, hen f is Fréche differeniable a a. Proof. Noice ha, in he noaion of Proposiion 0.5, V = R d since V conains he canonical orhonormal basis of R d. Moreover, f is coninuous and hence locally Lipschiz on A. By Corollary 0.6, f is Gâeaux differeniable a a. Apply Lemma

4 4 Example 0.8. Consider he coninuous convex funcion f : l R, f(x) = x = + x(i). This funcion f is Gâeaux differeniable exacly a he poins having all coordinaes nonzero, and f is nowhere Fréche differeniable. Proof. Le e k denoe he k-h canonical uni vecor in l. If f is Gâeaux differeniable a a poin a l hen he funcions ϕ k () := f(a+e k ) = a(k)+ + i k a(i) are differeniable a 0, and hence a(k) 0 for each k. Now, le us prove ha f is Gâeaux differeniable a a whenever a(k) 0 for each k. Since f(x) remains he same if we change signs of some of he coordinaes of x, we can suppose ha a(k) > 0 for each k. Fix an arbirary v l wih v =. For such v and any > 0, we have f(a + v) f(a) = + a(i) + v(i) a(i) For n N and 0 < < min{a(),..., a(n)}, we have v(i) < a(i) whenever i n, and hence a(i) + v(i) > 0 for such i. Thus we can wrie f(a + v) f(a) a(i)+v(i) a(i) = n v(i) + ε n () where ε n () = i>n. Noice ha ε n () i>n v(i). Passing o limi as 0+, we obain i>n f +(a, v) = n v(i) + λ n. a(i)+v(i) a i = where λ n = lim 0+ ε n (). Of course, λ n 0 as n +, since λ n i>n v(i). Consequenly, f +(a, v) = + v(i). By posiive homogeneiy, we conclude ha he same formula holds for each v l. Since f +(a, ) is linear and coninuous on l, f is Gâeaux differeniable a a. I remains o show ha f is no Fréche differeniable a any such a. Consider he vecors h n = 2a(n)e n (n N) and observe ha h n = 2a(n) 0. If f were Fréche differeniable a a, we would have 0 = lim n f(a + h n ) f(a) f +(a, h n ) h n a conradicion. = = lim n a(n) a(n) + 2a(n) 2a(n) =

5 Subdifferenial. We shall need several imes he following corollary of he Hahn- Banach heorem (see he las corollary in he chaper Hahn-Banach heorems ). Corollary 0.9. Le X be a normed space, f : X (, + ] a convex funcion, H X an affine se, and h: H R a coninuous affine funcion such ha h f on H. Assume ha in(dom(f)) H, and f is coninuous on in(dom(f)). Then here exiss a coninuous affine exension ĥ: X R of h, such ha ĥ f. In wha follows, unless oherwise specified, A is an open convex se in a normed space X, (2) f : A R is a coninuous convex funcion, a A. If necessary (e.g., for applicaion of he above corollary), we consider f exended by + ouside of A. Definiion 0.0. The subdifferenial of f a a is he se f(a) = {x X : f(x) f(a) + x (x a) for each x A }. The elemens of f(a) are called subgradiens of f a a. The mulivalued mapping f : A 2 X (where 2 X is he se of all subses of X ), x f(x), is called he subdifferenial map or simply he subdifferenial of f. The image f(e) of a se E A is he se f(e) = x A f(x). Geomerically, he subgradiens of are hose elemens of X ha define affine funcions h: X R such ha h(a) = f(a) and h f on A. We say ha such h suppors f a a. Proposiion 0.. Assume (2). Then f(a). Proof. By Corollary 0.9, applied o he affine se H = {a} and he funcion h(a) = f(a) on H, here exiss a coninuous affine exension ĥ: X R of H such ha ĥ f on A. Since ĥ(a) = f(a), ĥ mus be of he form ĥ(x) = f(a) + x (x a) for some x X. Then x f(a). Lemma 0.2. Assume ha (2) holds and L 0 is given. Then f is L-lipschiz on A if and only if f(a) L B X. Proof. Le f be L-Lipschiz on A. Given x A, le r > 0 be such ha he closed ball B(x, r) is conained in A. For each x f(x), we have x = r sup v =r x (v) r sup [ ] v =r f(a + v) f(a) r sup v =r L v = L. Now, le f(a) L B X. For x, y A and x f(x), we have f(x) f(y) x (x y) x x y L x y. By inerchanging he roles of x and y, we conclude ha f is L-Lipschiz. 5

6 6 Corollary 0.3. Assume (2). Then f is locally bounded on A. (Indeed, f is locally Lipschiz on A.) Proposiion 0.4. Assume (2). Then f(a) is convex, bounded and w -closed (and hence w -compac). Proof. We already know ha f(a) is bounded (by Corollary 0.3). To see ha i is also convex and w -closed, i suffices o noice ha f(a) = x A{x X : x (x a) f(x) f(a)} is inersecion of a family of w -closed halfspaces in X. Lemma 0.5. Assume (2). Then he subdifferenial map f is monoone on A, ha is, (x y )(x y) 0 whenever x, y A, x f(x) and y f(y). Proof. Given x, y A, x f(x) and y f(y), we have x (y x) f(y) f(x) y (x y) f(x) f(y). Summing he wo inequaliies, one easily ges (x y )(x y) 0. Proposiion 0.6. Assume (2). For x X he following asserions are equivalen: (i) x f(a); (ii) x (v) f +(a, v) for each v X; (iii) f +(a, v) x (v) f +(a, v) for each v X. Proof. (ii) and (iii) are equivalen since x (v) = x ( v) f +(a, v). (i) (ii). Le x f(a) and v X. There exiss τ > 0 such ha a + τv A. Then, for each (0, τ), x (v) = f(a + v) f(a) x (v). Leing 0 +, we obain he inequaliy in (ii). (ii) (i). Le (ii) hold, and x A. Then we have which shows (i). x (x a) f +(a, f(a + (x a)) f(a) x a) = lim 0+ = inf (0,] f(a + (x a)) f(a) = f(x) f(a), As a corollary of Proposiion 0.6, we obain ha f(a) is fully deermined by he values of f in any neighborhood of a. While Proposiion 0.6 deermines he subdifferenial in erms of he direcional derivaives, he following proposiion shows ha i is possible o do also he opposie: deermine he direcional derivaives in erms of he subdifferenial.

7 Proposiion 0.7. Assume (2). Then for each v X f +(a, v) = max v( f(a)) ( = max x f(a) x (v) ). Proof. By Proposiion 0.6, i suffices o show ha here exiss x f(a) such ha x (v) = f +(a, v). Consider he (one-dimensional) affine se H = a + Rv and he affine funcion h: H R, h(a + v) = f(a) + f +(a, v). Well-known properies of convex funcions of one real variable imply ha h f on H A. By Corollary 0.9, here exiss a coninuous affine exension ĥ: X R of h such ha ĥ f on A. Since ĥ(a) = h(a) = f(a), we mus have ĥ(x) = f(a) + x (x a) where x X. Then x (v) = ĥ(a + v) f(a) = h(a + v) f(a) = f +(a, v). Theorem 0.8. Assume (2). Then f is Gâeaux differeniable a a if and only if f(a) is a singleon. In his case, f(a) = {f (a)}. Proof. If f is Gâeaux differeniable a a, hen f +(a, v) = f +(a, v) = f (a)(v) for each v X. For any x f(a), Proposiion 0.6 implies ha x (v) = f (a)(v) for each v, ha is x = f (a). Now, assume ha f is no Gâeaux differeniable a a. By Corollary 0.6, here exiss v X such ha f +(a, v) f +(a, v). By Proposiion 0.7, here exis x, x 2 f(a) such ha x (v) = f +(a, v) and x 2 ( v) = f +(a, v). Then x (v) x 2 (v) = f +(a, v) + f +(a, v) 0, and hence f(a) is no a singleon. Le us show ha he subdifferenial of a Gâeaux differeniable funcion saisfies a kind of a coninuiy in he w -opology. Theorem 0.9. Assume (2). If f is Gâeaux differeniable a a, hen he following propery holds: x n a 0, x n A, x n f(x n ) = x w n f (a). Proof. Assume ha x n do no converge in he w -opology o f (a). This means ha here exiss v X such ha x n(v) f (a)(v). Passing o a subsequence if necessary, we can (and do) suppose ha, for some ε > 0, (3) x n (v) f (a)(v) ε for each n. By Corollary 0.3, he sequence {x n} is bounded, and hence (by Alaoglu s heorem) i has a w -accumulaion poin x 0. This means ha each w -neighborhood of x 0 conains infiniely many x n s. Fix an arbirary y A. Since he ses W n = { z X : x n(y a) x 0(y a) < n, } x n(v) x 0(v) < n (n N) are w -neighborhoods of x 0, i is easy o see ha we can suppose (by passing o a furher subsequence) ha x n(y a) x 0(y a) and x n(v) x 0(v). By definiion of subdifferenial, f(y) f(x n ) x n(y x n ) = x n(y a) + x n(a x n ). Noice ha x n(a x n ) x n a x n 0 since {x n} is bounded. Thus we can pass o limis o obain f(y) f(a) x 0(y a). Since y A was arbirary, we have x 0 f(a), ha is, x 0 = f (a). Consequenly, x n(v) f (a)(v). Bu his conradics (3). 7

8 8 Now, we are going o characerize Fréche differeniabiliy in erms of he subdifferenial. We shall need he following definiion. Definiion Le X, Y be a normed spaces, A X a se, Φ: A 2 Y mulivalued mapping, and a A. (a) The image by Φ of a se E A is defined as he se Φ(E) = x E Φ(x). (b) We say ha Φ is single-valued and coninuous a a if Φ(a) = {y 0 } and [ ε > 0 δ > 0 : x A, x a < δ Φ(x) B 0 (y 0, ε) ]. Exercise 0.2. In he noaion of he above definiion, he following asserions are equivalen. (i) Φ is single-valued and coninuous a a. (ii) Φ(a) = {y 0 } and [ A x n a, y n Φ(x n ) y n y 0 ]. (iii) osc(φ, a) := lim δ 0+ diam [ Φ(B(a, δ)) ] = 0. (osc(φ, a) is called he oscilaion of Φ a a.) Theorem Assume (2). Then f is Fréche differeniable a a if and only if f is single-valued and coninuous a a. Proof. If par. Le f be single-valued and coninuous a a. For x = a + h A, le x h f(a + h). (Thus f(a) = {x 0 }.) Then we have 0 f(a + h) f(a) x 0 (h) h x h (h) x 0 (h) h as h 0, which means ha f is Fréche differeniable a a. x h x 0 0 Only if par. Le f be Fréche differeniable a a wih f (a) = x 0, ha is, ε(v) := f(a + v) f(a) x 0 (v) v 0 as v 0. Le ϱ > 0 be such ha B(a, ϱ) A and f is Lipschiz (wih a consan L 0) on B(a, ϱ). Consider sequences {h n } B(0, ϱ) \ {0} and {x n} X such ha h n 0 and x n f(a + h n ) for each n. We wan o show ha x n f (a). Observe ha (x n x 0 )(v) = x 0 (v) + x n((a + v) (a + h n )) + x n(h n ) [ v ε(v) f(a + v) + f(a) ] + [ f(a + v) f(a + h n ) ] + L h n = v ε(v) + f(a) f(a + h n ) + L h n. Consequenly, for each r (0, ϱ) we have Hence x n x 0 = sup (x n x 0)(v/r) sup ε(v) + f(a) f(a + h n) + L h n. v =r v =r r lim sup x n x 0 sup ε(v) n v =r (r (0, ϱ)). Since sup v =r ε(v) 0 as r 0 +, we conclude ha x n x 0 0. Corollary Assume (2). If f is Fréche differeniable a each poin of A, hen f C (A). a

9 Corollary Le f be a convex funcion on an open convex se A R d. If f has all parial derivaives a each poin of A, hen f C (A). (Indeed, by Corollary 0.7, f is Fréche differeniable on A.) 9