Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for all x R. We say that ϕ is cocave (or cocave dow) if ϕ (x) 0 for all x R. For example, a quadratic fuctio ϕ(x) = ax 2 + bx + c is covex if a 0, ad is cocave if a 0. Ufortuately, the defiitios above are ot sufficietly geeral, sice they require ϕ to be twice differetiable. Istead, we will use the followig defiitios: Defiitio: Covex ad Cocave Fuctios Let a < b, ad let ϕ: (a, b) R be a fuctio. 1. We say that ϕ is covex if for all x, y (a, b) ad λ [0, 1]. 2. We say that ϕ is cocave if for all x, y (a, b) ad λ [0, 1]. ϕ ( (1 λ)x + λy ) (1 λ)ϕ(x) + λϕ(y) ϕ ( (1 λ)x + λy ) (1 λ)ϕ(x) + λϕ(y)
Covexity, Iequalities, ad Norms 2 y φ x y, φ y x, φ x Figure 1: For a covex fuctio, every chord lies above the graph. Geometrically, the fuctio λ ( (1 λ)x + λy, (1 λ)ϕ(x) + λϕ(y) ), 0 λ 1 is a parametrizatio of a lie segmet i R 2. This lie segmet has edpoits ( x, ϕ(x) ) ad ( y, ϕ(y) ), ad is therefore a chord of the graph of ϕ (see figure 1). Thus our defiitios of cocave ad covex ca be iterpreted as follows: A fuctio ϕ is covex if every chord lies above the graph of ϕ. A fuctio ϕ is cocave if every chord lies below the graph of ϕ. Aother fudametal geometric property of covex fuctios is that each taget lie lies etirely below the graph of the fuctio. This statemet ca be made precise eve for fuctios that are ot differetiable: Theorem 1 Taget Lies for Covex Fuctios Let ϕ: (a, b) R be a covex fuctio. The for every poit c (a, b), there exists a lie L i R 2 with the followig properties: 1. L passes through the poit ( c, ϕ(c) ). 2. The graph of ϕ lies etirely above L. PROOF See exercise 1.
Covexity, Iequalities, ad Norms 3 Figure 2: A taget lie to y = x at the poit (0, 0). We will refer to ay lie satisfyig the coclusios of the above theorem as a taget lie for ϕ at c. If ϕ is ot differetiable, the the slope of a taget lie may ot be uiquely determied. For example, if ϕ(x) = x, the a taget lie for ϕ at 0 may have ay slope betwee 1 ad 1 (see figure 2). We shall use the existece of taget lies to provide a geometric proof of the cotiuity of covex fuctios: Theorem 2 Cotiuity of Covex Fuctios Every covex fuctio is cotiuous. PROOF Let ϕ: (a, b) R be a covex fuctio, ad let c (a, b). Let L be a liear fuctio whose graph is a taget lie for ϕ at c, ad let P be a piecewiseliear fuctio cosistig of two chords to the graph of ϕ meetig at c (see figure 3). The L ϕ P i a eighborhood of c, ad L(c) = ϕ(c) = P (c). Sice L ad P are cotiuous at c, it follows from the Squeeze Theorem that ϕ is also cotiuous at c. We ow come to oe of the most importat iequalities i aalysis:
Covexity, Iequalities, ad Norms 4 P x φ x L x c Figure 3: Usig the Squeeze Theorem to prove that ϕ(x) is cotiuous at c Theorem 3 Jese s Iequality (Fiite Versio) Let ϕ: (a, b) R be a covex fuctio, where a < b, ad let x 1,..., x (a, b). The ϕ(λ 1 x 1 + + λ x ) λ 1 ϕ(x 1 ) + + λ ϕ(x ) for ay λ 1,..., λ [0, 1] satisfyig λ 1 + + λ = 1. PROOF Let c = λ 1 x 1 + + λ x, ad let L be a liear fuctio whose graph is a taget lie for ϕ at c. Sice λ 1 + + λ = 1, we kow that L(λ 1 x 1 + + λ x ) = λ 1 L(x 1 ) + + λ L(x ). Sice L ϕ ad L(c) = ϕ(c), we coclude that ϕ(c) = L(c) = L(λ 1 x 1 + + λ x ) = λ 1 L(x 1 ) + + λ L(x ) λ 1 ϕ(x 1 ) + + λ ϕ(x ). This statemet ca be geeralized from fiite sums to itegrals. Specifically, we ca replace the poits x 1,..., x by a fuctio f : R, ad we ca replace the weights λ 1,..., λ by a measure µ o for which µ() = 1.
Covexity, Iequalities, ad Norms 5 Theorem 4 Jese s Iequality (Itegral Versio) Let (, µ) be a measure space with µ() = 1. Let ϕ: (a, b) R be a covex fuctio, where a < b, ad let f : (a, b) be a L 1 fuctio. The ( ) ϕ f dµ (ϕ f) dµ PROOF Let c = f dµ, ad let L be a liear fuctio whose graph is a taget lie for ϕ at c. Sice µ() = 1, we kow that L( f dµ) = (L f) dµ. Sice L(c) = ϕ(c) ad L ϕ, this gives ( ) ϕ(c) = L(c) = L f dµ = (L f) dµ (ϕ f) dµ. Meas You are probably aware of the arithmetic mea ad geometric mea of positive umbers: x 1 + + x ad x 1 x. More geerally, we ca defie weighted versios of these meas. Give positive weights λ 1,..., λ satisfyig λ 1 + + λ = 1, the correspodig weighted arithmetic ad geometric meas are λ 1 x 1 + + λ x ad x λ 1 1 x λ. These reduce to the uweighted meas i the case where λ 1 = = λ = 1/. Arithmetic ad geometric meas satisfy a famous iequality, amely that the geometric mea is always less tha or equal to the arithmetic mea. This turs out to be a simple applicatio of Jese s iequality: Theorem 5 AM GM Iequality Let x 1,..., x > 0, ad let λ 1,..., λ [0, 1] so that λ 1 + + λ = 1. The x λ 1 1 x λ λ 1 x 1 + + λ x.
Covexity, Iequalities, ad Norms 6 b a b 2 y e x ab a log a 1 log a log b log b 2 Figure 4: A visual proof that ab < (a + b)/2. PROOF This theorem is equivalet to the covexity of the expoetial fuctio (see figure 4). Specifically, we kow that e λ 1t 1 + λ t λ 1 e t 1 + λ e t for all t 1,..., t R. Substitutig x i = e t i gives the desired result. The followig theorem geeralizes this iequality to arbitrary measure spaces. The proof is essetially the same as the proof of the previous theorem. Theorem 6 Itegral AM GM Iequality Let (, µ) be a measure space with µ() = 1, ad let f : (0, ) be a measurable fuctio. The ( ) exp log f dµ f dµ PROOF Sice the expoetial fuctio is covex, Jese s iequality gives ( ) exp log f dµ exp(log f) dµ = f dµ. By the way, we ca rescale to get a versio of this iequality that applies wheever
Covexity, Iequalities, ad Norms 7 µ() is fiite ad ozero: ( 1 exp µ() ) log f dµ 1 f dµ. µ() Note that the quatity o the right is simply the average value of f o. The quatity o the left ca be thought of as the (cotiuous) geometric mea of f. p-meas There are may importat meas i mathematics ad sciece, beyod just the arithmetic ad geometric meas. For example, the harmoic mea of positive umbers x 1,..., x is 1/x 1 + + 1/x. This mea is used, for example, i calculatig the average resistace of resistors i parallel. For aother example, the Euclidea mea of x 1,..., x is x 2 1 + + x 2. This mea is used to average measuremets take for the stadard deviatio of a radom variable. The AM GM iequality ca be exteded to cover both of these meas. I particular, the iequality 1/x 1 + + 1/x x 1 x x 1 + + x holds for all x 1,..., x (0, ). Both of these meas are examples of p-meas: x 2 1 + + x 2. Defiitio: p-meas Let x 1,..., x > 0. If p R {0}, the p-mea of x 1,..., x is ( x p ) 1 + + x p 1/p. For example: The 2-mea is the same as the Euclidea mea. The 1-mea is the same as the arithmetic mea.
Covexity, Iequalities, ad Norms 8 The ( 1)-mea is the same as the harmoic mea. Though it may ot be obvious, the geometric mea also fits ito the family of p-meas. I particular, it is possible to show that ( x p ) 1 + + x p 1/p lim = p 0 x 1 x for ay x 1,..., x (0, ). Thus, we may thik of the geometric mea as the 0-mea. It is also possible to use limits to defie meas for ad. It turs out the the -mea of x 1,..., x is simply max(x 1,..., x ), while the ( )-mea is mi(x 1,..., x ). As with the arithmetic ad geometric meas, we ca also defie weighted versios of p-meas. Give positive weights λ 1,..., λ satisfyig λ 1 + + λ = 1, the correspodig weighted p-mea is ( ) λ1 x p 1/p. 1 + + λ x p As you may have guessed, the p-meas satisfy a geeralizatio of the AM-GM iequality: Theorem 7 Geeralized Mea Iequality Let x 1,..., x > 0, ad let λ 1,..., λ [0, 1] so that λ 1 + + λ = 1. The p q ( ) λ1 x p 1 + + λ x p 1/p ( λ1 x q 1 + + λ x) q 1/q for all p, q R {0}. PROOF If p = 1 ad q > 1, this iequality takes the form ( λ1 x 1 + + λ x ) q λ1 x q 1 + + λ x q which follows immediately from the covexity of the fuctio ϕ(x) = x q. The case where 0 < p < q follows from this. Specifically, sice q/p > 1, we have ( λ1 (x p 1) + + λ (x p ) ) q/p λ1 (x p 1) q/p + + λ (x p ) q/p ad the desired iequality follows. Cases ivolvig egative values of p or q are left as a exercise to the reader.
Covexity, Iequalities, ad Norms 9 Applyig the same reasoig usig the itegral versio of Jese s iequality gives p q ( f p dµ) 1/p ( ) 1/q f q dµ for ay L 1 fuctio f : (0, ), where (, µ) is a measure space with a total measure of oe. Norms A orm is a fuctio that measures the legths of vectors i a vector space. The most familiar orm is the Euclidea orm o R, which is defied by the formula (x 1,..., x ) = x 2 1 + + x 2. Defiitio: Norm o a Vector Space Let V be a vector space over R. A orm o V is a fuctio : V R, deoted v v, with the followig properties: 1. v 0 for all v V, ad v = 0 if ad oly if v = 0. 2. λv = λ v for all λ R ad v V. 3. v + w v + w for all v, w V. For those familiar with topology, ay orm o a vector space gives a metric d o the vector space defied by the formula d(v, w) = v w. Thus ay vector space with a orm ca be thought of as a topological space. The Euclidea orm o R ca be geeralized to the family of p-orms. ay p 1, the p-orm o R is defied by the formula For (x 1,..., x ) p = ( x 1 p + + x p ) 1/p The usual Euclidea orm correspods to the case where p = 2. It is easy to see that the defiitio of the p-orm satisfies axioms (1) ad (2) for a orm, but third axiom (which is kow as the triagle iequality) is far from clear. The followig theorem establishes the p-orm is i fact a orm.
Covexity, Iequalities, ad Norms 10 Theorem 8 Mikowski s Iequality If u, v R ad p [1, ), the u + v p u p + v p PROOF Sice p 1, the fuctio x x p is covex. It follows that (1 λ)u + λv p p = (1 λ)u i + λv i p i=1 (1 λ) u i p + λ v i p = (1 λ) u p p + λ v p p i=1 for all u ad v ad λ [0, 1]. I particular, this proves that (1 λ)u + λv p 1 wheever u p = v p = 1. From this Mikowski s iequality follows. I particular, we may assume that u ad v are ozero. The u/ u p ad v/ v p are uit vectors, so u + v p u p + v p = u p u p + v p u u p + v p u p + v p v v p 1. p If is a orm o a vector space V, the uit ball i V is the set B V (1) = { v V v 1 }. For example, the uit ball i R 2 with respect to the Euclidea orm is a roud disk of radius 2 cetered at the origi. Figure 5 shows the uit ball i R 2 with respect to various p-orms. Their shapes are all fairly similar, with the the uit ball beig a diagoal square whe p = 1, ad the uit ball approachig a horizotal square as p. All of these uit balls have a certai importat geometric property. Defiitio: Covex Set Let V is a vector space over R. If v ad w are poits i V, the lie segmet from v to w is the set L(v, w) = {λv + (1 λ)w 0 λ 1}. A subset S V is covex if L(v, w) S for all v, w S. The followig theorem gives a geometric iterpretatio of Mikowski s iequality.
Covexity, Iequalities, ad Norms 11 p = 1 p = 1.5 p = 2 p = 3 p = 10 Figure 5: The shape of the uit ball i R 2 for various p-orms. Theorem 9 Shapes of Uit Balls Let V be a vector space over R, ad let : V R be a fuctio satisfyig coditios (1) ad (2) for a orm. The satisfies the triagle iequality if ad oly if the uit ball { v V v 1 } is covex. PROOF Suppose first that satisfies the triagle iequality. Let v ad w be poits i the uit ball, ad let p = λv + (1 λ)w be ay poit o the lie segmet from v to w. The p = λv + (1 λ)w λv + (1 λ)w = λ v + (1 λ) w λ(1) + (1 λ)(1) = 1 so p lies i the uit ball as well. For the coverse, suppose that the uit ball is covex, ad let v, w V. If v or w is 0, the clearly v + w v + w, so suppose they are both ozero. Let ˆv = v/ v ad ŵ = w/ w. The ˆv = 1 v v = 1 v v = 1 ad similarly ŵ = 1. Thus ˆv ad ŵ both lie i the uit ball. Sice the poit v v + w + w v + w = 1, v + w v + w = v v + w ˆv + w v + w ŵ must lie i the uit ball as well, ad it follows that v + w v + w.
Covexity, Iequalities, ad Norms 12 Fially, we should metio that there is a itegral versio of Mikowski s iequality. This ivolves the otio of a p-orm o a measure space. Defiitio: p-norm Let (, µ) be a measure space, let f be a measurable fuctio o, ad let p [1, ). The p-orm of f is the quatity f p = ( ) 1/p f p dµ Note that f p is always defied, sice it ivolves the itegral of a o-egative fuctio, but it may be ifiite. We ca ow state Mikowski s iequality for arbitrary measure spaces. Theorem 10 Mikowski s Iequality (Itegral Versio) Let (, µ) be a measure space, let f, g : R be measurable fuctios, ad let p [1, ). The f + g p f p + g p. PROOF Sice p 1, the fuctio x x p is covex. It follows that (1 λ)f + λg p p = (1 λ)f + λg p dµ ( (1 λ) f p + λ g p) dµ = (1 λ) f p p + λ g p p for ay measurable fuctios f ad g ad ay λ [0, 1]. I particular, this proves that (1 λ)f + λg p 1 wheever f p = g p = 1. From this Mikowski s iequality follows. First, observe that if f p = 0, the f = 0 almost everywhere, so Mikowski s iequality follows i this case. A similar argumet holds if g p = 0, so suppose that f p > 0 ad g p > 0. Let ˆf = f/ f p ad ĝ = g/ g p, ad ote that ˆf p = ĝ p = 1. The f + g p f p + g p = f p g p ˆf + ĝ f p + g p f p + g p 1. p
Covexity, Iequalities, ad Norms 13 Hölder s Iequality Recall that the Euclidea orm o R satisfies the Cauchy-Schwarz Iequality u v u 2 v 2 for all u, v R. Our ext task is to prove a geeralizatio of this kow as Hölder s iequality. Lemma 11 Youg s Iequality If x, y [0, ) ad p, q (1, ) so that 1/p + 1/q = 1, the xy xp p + yq q PROOF This ca be writte (x p ) 1/p (y q ) 1/q 1 p xp + 1 q yq which is a istace of the weighted AM GM iequality. Theorem 12 Hölder s Iequality Let u, v R, ad let p, q (1, ) so that 1/p + 1/q = 1. The u v u p v q. PROOF By Youg s iequality, u v u 1 v 1 + + u v u 1 p + + u p p + v 1 q + + v q q u p p p + v q q q I particular, if u p = v q = 1, the u v 1/p + 1/q = 1, which proves Hölder s iequality i this case. For the geeral case, we may assume that u ad v are ozero. The u/ u p ad v/ v q are uit vectors for their respective orms, ad therefore u v u p v q 1.
Covexity, Iequalities, ad Norms 14 Multiplyig through by u p v q gives the desired result. This iequality is ot hard to geeralize to itegrals. We begi with the followig defiitio. Defiitio: Ier Product of Fuctios Let (, µ) be a measure space, ad let f ad g be measurable fuctios o. The L 2 ier product of f ad g is defied as follows: f, g = fg dµ. Note that f, g may be udefied if fg is ot Lebesgue itegrable o. Note also that f, g = fg 1 if f ad g are oegative, but that i geeral f, g fg 1. Theorem 13 Hölder s Iequality (Itegral Versio) Let (, µ) be a measure space, let f ad g be measurable fuctios o, ad let p, q (1, ) so that 1/p + 1/q = 1. If f p < ad g p <, the f, g is defied, ad f, g f p g q PROOF Suppose first that f p = g q = 1. By Youg s iequality, fg 1 = fg dµ ( f p p ) + g q dµ f p p + g q q q p q = 1. Sice fg is L 1, it follows that f, g is defied. The f, g fg 1 1, which proves Hölder s iequality i this case. For the geeral case, ote first that if either f p = 0 or g q = 0, the either f = 0 almost everywhere or g = 0 almost everywhere, so Hölder s iequality holds i that case. Otherwise, let ˆf = f/ f p ad ĝ = g/ g q. The ˆf p = ĝ q = 1, so f, g = f p g p ˆf, ĝ f p g q
Covexity, Iequalities, ad Norms 15 I the case where p = q = 2, this gives the followig. Corollary 14 Cauchy-Schwarz Iequality (Itegral Versio) Let (, µ) be a measure space, ad let f ad g be measurable fuctios o. If f 2 < ad g 2 <, the f, g is defied, ad f, g f 2 g 2. Exercises 1. a) Prove that a fuctio ϕ: R R is covex if ad oly if ϕ(y) ϕ(x) y x for all x, y, z R with x < y < z. ϕ(z) ϕ(x) z x ϕ(z) ϕ(y) z y b) Use this characterizatio of covex fuctios to prove Theorem 1 o the existece of taget lies. 2. Let ϕ: (a, b) R be a differetiable fuctio. Prove that ϕ is covex if ad oly if ϕ is o-decreasig. 3. Let ϕ: R R be a covex fuctio. Prove that ϕ ( (1 λ)x + λy ) (1 λ)ϕ(x) + λϕ(y) for λ R [0, 1]. Use this to provide a alterative proof that ϕ is cotiuous. 4. If x, y 0, prove that ( x p + y p lim p 0 What is 2 lim p ) 1/p = xy ad lim p ( ) x p + y p 1/p? 2 ( ) x p + y p 1/p = max(x, y). 5. a) If x 1,..., x, y 1,..., y (0, ), prove that x1 y 1 + x y x1 + + x y1 + + y. That is, the arithmetic mea of geometric meas is less tha or equal to the correspodig geometric mea of arithmetic meas. 2
Covexity, Iequalities, ad Norms 16 b) If λ, µ [0, 1] ad λ + µ = 1, prove that x λ 1y µ 1 + x λ y µ ( ) λ ( ) µ x1 + + x y1 + + y. 6. Prove the Geeralized Mea Iequality (Theorem 6) i the case where p or q is egative. 7. Let f : [0, 1] R be a bouded measurable fuctio, ad defie ϕ: [1, ) R by ϕ(p) = f p dµ. Prove that log ϕ is covex o [1, ). [0,1] 8. Prove that ( 1 + x 2 y + x 4 y 2) 3 ( 1 + x 3 + x 6) 2( 1 + y 3 + y 6) for all x, y (0, ). 9. Let (, µ) be a measure space, let f, g : [0, ) be measurable fuctios, ad let p, q, r (1, ) so that 1/p + 1/q = 1/r. Prove that fg r f p g q.