When v = e j is a standard basis vector, we write F

Transcription

1 1. Differeniabiliy Recall he definiion of derivaive from one variable calculus Definiion 1.1. We say ha f : R R is differeniable a a poin a R if he quaniy f f(a + h f(a (a := h 0 h exiss. We hen call f (a he derivaive of f a a. One way o ransfer his definiion o higher dimensions is via direcional derivaives. Definiion 1.2. The direcional derivaive of a funcion F : R n R m a a poin a R n in he direcion v R n is he quaniy (if i exiss F (a + v F (a D v F (a := 0 When v = e j is a sandard basis vecor, we wrie F x j (a := D ej F (a and call his quaniy he parial derivaive of F wih respec o x j. Anoher way of saing his definiion is ha D v F (a = h (0 where h : R R m is he composie funcion h( := F (a + v obained by resricing F o he line hrough a in he direcion v. This way of formulaing direcional derivaives is quie useful when you acually have o compue one! A shorcoming of direcional derivaives is ha hey don always do a very good job of conrolling he behavior of F near a given poin a (see Secion 3.1 e.g. 2 in Shifrin for a good illusraion of his. One needs a lile bi more resricive noion of derivaive in order o guaranee his sor of conrol. Definiion 1.3. We say ha a funcion F : R n R m is differeniable a a poin a R n if here exiss a linear ransformaion T : R n R m such ha F (a + h F (a T h (1 h 0 If such a T exiss, hen we call i he derivaive of F a a wrie DF (a := T. So under his definiion, he derivaive DF (a of F a a is no a number bu raher a linear ransformaion. This is no so srange if you remember any linear ransformaion T : R n R m has a sandard marix A M m n, so you can hink of he derivaive of F a a more concreely as a marix, i.e. as a collecion of mn numbers ha describe he way all he differen componens of F = (F 1,..., F m are changing in all he differen direcions one can approach a. I m sor of doing ha already when I suppress parenheses in T (h and wrie T h insead. In paricular, if f : R R is jus a scalar funcion of a single variable, hen he number f (a above is jus he lone enry in he 1 1 marix for he linear ransformaion T : R R given by T (h = f (ah. Noe ha Equaion (1 can be wrien in several slighly differen bu equivalen ways. For insance, one could ake he magniude of he numeraor and wrie insead (I ll use DF (a in place of T now. F (a + h F (a DF (ah h 0 1

2 2 Or one could se x := a + h and rewrie he i as x a F (x F (a DF (a(x a. x a Anoher very useful way o resae (1 is o say ha (2 F (a + h = F (a + DF (ah + E(h, where he error erm E(h saisfies h 0 E(h The firs indicaion ha our definiion of differeniabiliy will give us sufficien conrol of F a nearby values of a is he following. Theorem 1.4. If F : R n R m is differeniable a a, hen F is coninuous a a. Proof. From Equaion (2 and coninuiy of linear ransformaions, I find ha F (x = F (a + DF (a(x a + E(x a = F (a + DF (a0 + E(x a. x a x a x a Moreover, since F is differeniable a a, I can dismiss he las i as follows. E(x a E(x a = x a x a x a x a = E(x a x a x a x a x a = 0 0. Noe ha in he las equaliy, I use coninuiy of he magniude funcion x x. A any rae, I conclude ha F (x = F (a, x a i.e. F is coninuous a a. The nex fac abou our new noion of derivaive Df(a is ha i s no ha far from parial and direcional derivaives. Theorem 1.5. Suppose ha F : R n R m is differeniable a a poin a R n. Then he direcional derivaive of F a a in direcion v R n exiss and is given by (3 D v F (a = DF (av. In paricular, he sandard marix for he linear ransformaion DF (a : R n R m is given column-wise by [ (4 F F (a... x n (a ] Among oher hings, his heorem ells us ha here is only one candidae for Df(a and gives us a pracical means for finding ou wha i is (by aking parial derivaives. I does no, however, ell us how o deermine wheher our candidae is a winner, i.e. wheher F is acually differeniable a a. For mos purposes, he following condiion suffices for ha purpose. Proof. The main hing here is o jusify he formula (3 for he direcional derivaive. This formula implies in paricular ha F x j (a = D ej F (a = DF (ae j. So he expression (4 for he sandard marix of DF (a proceeds immediaely from his and he fac ha he jh column of he sandard marix of a linear ransformaion is obained by applying he ransformaion o he sandard basis vecor e j.

3 To prove (3, I mus show ha F (a + v F (a = DF (av 0 If v = 0, he wo sides are clearly equal. Oherwise, I can use equaion (2 o rewrie he difference quoien on he lef side as follows F (a + v F (a = DF (av + E(v = DF (av + E(v. E(v So from here i suffices o show ha 0 To his end, le ɛ > 0 be given. Differeniabiliy of F a a guaranees ha here exiss δ > 0 such ha < δ implies ha E(h < ɛ v. I herefore choose δ = δ. If < δ, hen v < δ. Hence v E(v = v E(v v < v ɛ v = ɛ. Hence 0 E(v I conclude ha D v F (a = DF (av. Definiion 1.6. A funcion F : R n R m is said o be coninuously differeniable a a R n, if all parial dervaives F x j exis near and a a, and each is coninuous a a. If F is coninuously differeniable a each poin in is domain, hen we say simply ha F is coninuously differeniable (or C 1 for shor. Theorem 1.7. If F : R n R m is C 1, hen F is differeniable a every poin a in is domain. The following preinary resul reduces he proof of he heorem o he special case where F = f : R n R is scalar-valued. Lemma 1.8. Le F : R n R m be a vecor-valued funcion wih componen funcions F 1,..., F m : R n R. Then F is differeniable a a R n if and only if each componen funcion F j is differeniable a a. In his case, he sandard marix for DF (a has jh row equal o he sandard marix for DF j (a (noe ha his is a 1 n marix i.e. a row vecor. Proof. Exercise: follows from he definiion of differeniable and he fac (Proposiion 6.7 in my glossary finding he i of a vecor-valued funcion reduces o finding he is of each of is componen funcions. To resae he lemma a bi less formally, F is differeniable a exacly hose poins where all is componens are differeniable, and a each of hese poins he componens of he derivaive of F are equal o he derivaives of he componens of F. I will also need o use he following signal fac from one variable calculus Theorem 1.9 (Mean Value Theorem. If (a, b R is open and f : (a, b R is differeniable on (a, b hen for any wo disinc poins x, y (a, b, here exiss a poin c beween x and y such ha f(x f(y = f (c. x y Now back o he program: 3

4 4 Proof of Theorem 1.7. I will give he proof in he special case where F = f : R 2 R is scalar-valued and depends on only wo variables. The proof for scalar-valued funcions of n > 2 variables is similar and lef as an exercise. [ ] Theorem 1.5 ells me ha here is only one candidae (a (a for he sandard marix for Df(a. So assuming ha f is C 1 a some poin a = (a 1, a 2, I mus show ha ( f(a + h f(a (ah 1 + (ah 2 (5 h 0 Being C 1 a a means ha f is a leas defined near and a a (why?. Tha is, here exiss r > 0 such ha f(x is defined for all x B r (a. Moreover, given a poin a = (a 1, a 2 R 2, and a displacemen h = (h 1, h 2 R 2 wih < r, I may rewrie he expression inside he i in equaion (5 as follows. f(a 1 + h 1, a 2 + h 2 f(a 1, a 2 (a 1, a 2 h 1 (a 1, a 2 h 2 = f(a 1 + h 1, a 2 + h 2 f(a 1, a 2 + h 2 (a 1, a 2 h 1 + f(a 1, a 2 + h 2 f(a 1, a 2 (a 1, a 2 h 2. So o esablish (5 i suffices o show ha each of he las wo expressions have i 0 as h 0. I will show his for he firs (i.e. second las expression only, he argumen for he oher expression being similar. Given ɛ > 0, coninuiy of parial derivaives ells me ha here exiss δ > 0 such ha x a < δ implies ha (x (a < ɛ. Moreover, if I hink of f as a funcion of only he firs variable x 1, hen he one variable mean value heorem ells me ha ( f(a 1 + h 1, a 2 + h 2 f(a 1, a 2 + h 2 (a 1, a 2 h 1 (a 1 + h 1, a 2 + h 2 (a 1, a 2 = for some number h 1 beween 0 and h 1. In paricular ( h 1, h 2. So if < δ, I can esimae as follows f(a 1 + h 1, a 2 + h 2 f(a 1, a 2 + h 2 (a 1, a 2 h 1 = (a 1 + x h 1, a 2 + h 2 (a 1, a 2 h 1 1 < ɛ 1 = ɛ. This proves ha he lef side converges o 0 as h 0, which is wha I inended o show. h 1

5 2. Differeniaing composie funcions Theorem 2.1 (Chain Rule. Suppose ha G : R n R m is differeniable a a R n and F : R m R l is differeniable a G(a. Then he composiion F G : R n R l is differeniable a a and D(F G(a = DF (G(a DG(a. Proof. The composiion DF (G(a DG(a of wo linear maps is linear, so i suffices for me o show ha F (G(a + h F (G(a DF (G(aDG(ah h 0 Differeniabiliy of F a G(a implies ha F (G(a + h F (G(a = DF (G(a(G(a + h G(a + E F (G(a + h G(a where v 0 E F (v v Hence he i above can be rewrien DF (G(a(G(a + h G(a DG(ah h 0 = DF (G(a ( h 0 G(a + h G(a DG(ah = DF (G(a0 + h 0 =. h 0 + h 0 + h 0 The firs equaliy holds because DF (G(a is linear and herefore coninuous. The second equaliy follows from he definiion of differeniabiliy. For he remaining i, I use he fac ha where h 0 E G (h E G h G(a + h G(a = DG(ah + E G (h. In paricular, here exiss δ 1 > 0 such ha 0 < < δ 1 implies < 1. Hence when < δ 1, I can employ he riangle and Cauchy-Schwarz inequaliy o esimae G(a + h G(a DG(a + E G (h ( DG(a + 1. Given ɛ > 0, I can hen choose δ 2 > 0 such ha 0 < k < δ 2 implies ha E F (k ɛ. So if 0 < < δ := min{δ δ ( DG(a +1 1, 2 }, hen ( DG(a +1 and herefore < In shor, 0 < < δ implies ha G(a + h G(a ( DG(a + 1 < δ 2, ɛ ɛ( DG(a + 1 G(a + h G(a ( DG(a + 1 ( DG(a + 1 < ɛ. 5 k < = ɛ.

6 6 Hence h 0 which is he hing i remained for me o show. = 0, Firs a cauionary ale. 3. Equaliy of mixed parial derivaives Example 3.1. Le f(x 1, x 2 = x2 1 x2 2 x 2 1 +x2 2. Observe ha However, f(x x 2 1 1, x 2 = x 1 0 x 2 0 x1 0 x 2 1 f(x x 2 2 1, x 2 = x 2 0 x 1 0 x2 0 x 2 2 = 1. = 1. The moral? One canno generally swich he order in which one akes is and expec o ge he same answer. Definiion 3.2. A funcion F : R n R m is said o be C 2 (or wice coninuously differeniable if all firs and second parial derivaives of f exis and are coninuous a every poin a R n ha belongs o he domain of F. Now ha I ve defined C 1 and C 2, you can probably imagine hen wha C k means when k > 2. The following heorem ells us ha order is irrelevan when we ake second (and higher order parial derivaives of a decen funcion of several variables. Theorem 3.3. Suppose ha f : R n R is C 2. Then for any 1 i, j n and any a R n in he domain of f, one has x i x j (a = x j x i (a. My proof is quie similar o Shifrin s, bu (in my humble opinion mine ends a lile more honesly. In any case, he main hing is o show ha one can reverse he order of he wo is involved in aking a second parial derivaive. Proof. To sar wih, noe ha since we are only considering derivaives of f wih respec o x i and x j, we migh as well assume ha hese are he only variables on which f depends. Tha is, i suffices o assume ha n = 2 in he saemen of he heorem, fix a poin a = (a 1, a 2 in he domain of f and show ha (a = (a.

7 To his end, I go back o he definiion of derivaive, applying i o boh parial derivaives: (a = h1 0 h2 0 = h1 0 = h1 0 h 2 0 (a 1 + h 1, a 2 (a 1, a 2 h 1 ( f(a1 +h 1,a 2 +h 2 f(a 1 +h 1,a 2 h 2 h2 0 h 1 ( f(a1,a 2 +h 2 f(a 1,a 2 h 2 f(a 1 + h 1, a 2 + h 2 f(a 1 + h 1, a 2 f(a 1, a 2 + h 2 + f(a 1, a 2 h 1 h 2 Le me (for breviy s sake call he quaniy inside he las i Q(h 1, h 2. Unnecessary moivaional digression: Similarly, when he parial derivaives are reversed, one finds: (a = x Q(h 1 h2 1, h 2 0 h 1 0 Tha is, we ge he same hing as before, excep ha he order of he is is reversed. If we could swich he is, we d be home-free. Bu wihou jusificaion, we can. Insead we ake a less direc bu more jusifiable approach ha relies on he mean value heorem. Lemma 3.4. For each h = (h 1, h 2 R 2, here exiss h = ( h 1, h 2 inside he recangle deermined by h and 0 such ha Q(h = (a + h Proof. Noe (i.e. really check i! ha we can rewrie Q(h 1, h 2 = 1 h 2 g(a 1 + h 1 g(a 1 h 1 where g : R R is given by g( := f(, a 2 + h 2 f(, a 2. In paricular g is a differeniable funcion of one variable wih derivaive given by g ( = (, a 2 + h 2 (, a 2. So I can apply he mean value heorem, obaining a number h 1 beween 0 and h 1 such ha Q(h 1, h 2 = 1 ( g(a1 + h 1 g(a 1 = 1 g (a 1 + h 1 = 1 ( (a 1 + h 2 h 1 h 2 h 2 x h 1, a 2 + h 2 (a x h 1, a 2. 1 Applying he Mean Value Theorem a second ime, o his las expression, gives me a number h 2 beween 0 and h 2 such ha Q(h 1, h 2 = (a 1 + h 1, a 2 + h 2 To finish he proof of he heorem, I will use he convenien noaion A ɛ B o mean ha A, B R saisfy A B < ɛ. Noe ha (by he riangle inequaliy we have approximae ransiiviy i.e. A ɛ1 B and B ɛ2 C implies A ɛ1 +ɛ 2 C. I will suffice o show ha (a ɛ (a 7

8 8 for every ɛ > 0. So le ɛ > 0 be given. By coninuiy of second parial derivaives, here exiss δ > 0 such ha < δ implies ha (a + h (a < 1 3 ɛ. Using he definiion of i wice and hen he above lemma, I herefore obain ha when h 1 and hen h 2 are small enough, (a = x Q(h 2 h1 1, h 2 ɛ/3 0 h 2 0 In shor, which is wha I sough o show. Q(h h2 1, h 2 ɛ/3 Q(h 1, h 2 = 0 (a ɛ (a, (a+ h ɛ/3 (a.