Chapter 4: Matrix Norms

Transcription

1 EE448/58 Vesion.0 John Stensby Chate 4: Matix Noms The analysis of matix-based algoithms often equies use of matix noms. These algoithms need a way to quantify the "size" of a matix o the "distance" between two matices. o examle, suose an algoithm only wos well with full-an, n n matices, and it oduces inaccuate esults when sulied with a nealy an deficit matix. Obviously, the concet of ε- an (also nown as numeical an), defined by an(, ε) = min an( B) B ε (4-) is of inteest hee. ll matices B that ae "within" ε of ae examined when comuting the ε- an of. We define a matix nom in tems of a given vecto nom; in ou wo, we use only the - vecto nom, denoted as X. Let be an m n matix, and define = su X 0 X X, (4-) whee "su" stands fo suemum, also nown as least ue bound. Note that we use the same notation fo both vecto and matix noms. Howeve, the meaning should be clea fom context. Since the matix nom is defined in tems of the vecto nom, we say that the matix nom is subodinate to the vecto nom. lso, we say that the matix nom is induced by the vecto nom. Now, since X is a scala, we have X = X su = X X X su /. (4-3) 0 X 0 CH4.DOC Page 4-

2 EE448/58 Vesion.0 John Stensby n (4-3), note that X / X has unit length; fo this eason, we can exess the nom of in tems of a suemum ove all vectos of unit length, = su X = X. (4-4) That is, is the suemum of X on the unit ball X =. Caeful consideation of (4-) eveals that X X (4-5) fo all X. Howeve, X is a continuous function of X, and the unit ball X = is closed and bounded (eal analysis boos, it is said to be comact). Now, on a closed and bounded set, a continuous function always achieves its maximum and minimum values. Hence, in the definition of the matix nom, we can elace the "su" with "max" and wite X = max = max X 0 X X = X. (4-6) When comuting the nom of, the definition is used as a stating oint. The ocess has two stes. ) ind a "candidate" fo the nom, call it K fo now, that satisfies X K X fo all X. ) ind at least one nonzeo X 0 fo which X0 = K X0 Then, you have you nom: set = K. MatLab's Matix Nom unctions om an alication standoint, the -nom, -nom and the -nom ae amoung the most CH4.DOC Page 4-

3 EE448/58 Vesion.0 John Stensby imotant; MatLab comutes these matix noms. n MatLab, the -nom, -nom and -nom ae invoed by the statements nom(,), nom(,), and nom(,inf), esectively. The -nom is the default in MatLab. The statement nom() is inteeted as nom(,) by MatLab. Since the -nom used in the majoity of alications, we will adot it as ou default. n what follows, an "un-designated" nom is to be inteeted as the -nom. The Matix -Nom Recall that the vecto -nom is given by X n = x i. (4-7) i= Subodinate to the vecto -nom is the matix -nom max aij. (4-8) j i = H G KJ That is, the matix -nom is the maximum of the column sums. To see this, let m n matix be eesented in the column fomat = L n. (4-9) Then we can wite n X = L n X = x, (4-0) = CH4.DOC Page 4-3

4 EE448/58 Vesion.0 John Stensby whee x, n, ae the comonents of abitay vecto X. The tiangle inequality and standad analysis alied to the nom of (4-0) yields n x X = x x = = max j j n x = = max j j X. KJ (4-) With the develoment of (4-), we have comleted ste # fo comuting the matix -nom. That is, we have found a constant K = max j j such that X K X fo all X. Ste equies us to find at least one vecto fo which we have equality in (4-). But this is easy; to maximize X, it is natual to select the X 0 that uts "all of the allowable weight" on the comonent that will "ull-out" the maximum column sum. That is, the "otimum" vecto is X 0 = 0 0 L 0 0 L 0 whee j is the index that satisfies j = max j th -osition CH4.DOC Page 4-4

5 EE448/58 Vesion.0 John Stensby (the sum of the magnitudes in the j th column is equal to, o lage than, the sum of the magnitudes in any column). When X 0 is used, we have equality in (4-), and we have comleted ste #, so (4-8) is the matix -nom. The Matix -Nom X Recall that the vecto -nom is given by = max x, (4-) the vecto's lagest comonent. Subodinate to the vecto -nom is the matix -nom KJ = max aij. (4-3) i j That is, the matix -nom is the maximum of the ow sums. To see this, let be an abitay m n matix and comute a x ax X = = max aix max aix M i i amx max i ai KJ max x KJ (4-4) CH4.DOC Page 4-5

6 EE448/58 Vesion.0 John Stensby = max i ai KJ X We have comleted the fist ste. We have found a constant K = max i ai KJ fo which X K X (4-5) fo all X. Ste # equies that we find a non-zeo X 0 fo which equality holds in (4-4) and (4-5). close examination of these fomulas leads to the conclusion that equality evails if X 0 is defined to have the comonents x = ai ai, ai 0, n, =, ai = 0, (4-6) (the oveba denotes comlex conjugate) whee i is the index fo the maximum ow sum. That is, in (4-6), use index i fo which ai a j, i j. (4-7) Hence, (4-3) is the matix -nom as claimed. The Matix -Nom Recall that the vecto -nom is given by CH4.DOC Page 4-6

7 EE448/58 Vesion.0 John Stensby n X = xi = X, X. (4-8) i= Subodinate to the vecto -nom is the matix -nom = lagest eigenvalue of. (4-9) Due to this connection with eigenvalues, the matix -nom is called the sectal nom. To see (4-9) fo an abitay m n matix, note that * is n n and Hemitian. By Theoem 4.. (see endix 4.), the eigenvalues of * ae eal-valued. lso, * is at least ositive semi-definite since X *(*)X = (X )*(X ) 0 fo all X. Hence, the eigenvalues of * ae both eal-valued and non-negative; denote them as σ σ σ3 L σ n 0. (4-0) Note that these eigenvalues ae aanged accoding to size with σ being the lagest. These eigenvalues ae nown as the singula values of matix. Coesonding to these eigenvalues ae n othonomal (hence, they ae indeendent) eigenvectos U, U,..., U n with (*)U = (σ )U, n. (4-) The n eigenvectos fom the columns of a unitay n n matix U that diagonalizes matix * unde similaity (matix U*(*)U is diagonal with eigenvalues (4-0) on the diagonal). Since the n eigenvectos U, U,..., U n ae indeendent, they can be used as a basis, and vecto X can be exesssed as CH4.DOC Page 4-7

8 EE448/58 Vesion.0 John Stensby n X = c U, (4-) = whee the c ae X -deendent constants. Multily X, in the fom of (4-), by * by to obtain n n X = cu = c σ U, (4-3) = = which leads to n n n X X X X X cu cj j U ( ) ( ) jj = c σ σ = j= = = = = H G K J H G K n σ c = KJ = σ X (4-4) fo abitay X. Hence, we have comleted ste#: we found a constant K = σ such that X K X fo all X. Ste# equies us to find at least one vecto X 0 fo which equality holds; that is, we must find an X 0 with the oety that X0 = K X 0. But, it is obvious that X 0 = U, the unit-length eigenvecto associated with eigenvalue σ, will wo. Hence, the matix -nom is given by = σ, the squae oot of the lagest eigenvalue of *. The -nom is the default in MatLab. lso, it is the default hee. om now on, unless secified othewise, the -nom is assumed: means. Examle = L N M O Q P CH4.DOC Page 4-8

9 EE448/58 Vesion.0 John Stensby ind, and find the unit-length vecto X 0 that maximizes X. ist, comute the oduct L = T = N M 5 3O 3 Q P. The eigenvalues of this matix ae σ = and σ =.459; note that T is ositive definite symmetic. The -nom of is = σ =. 68, the squae oot of the lagest eigenvalue of T. The coesonding eigenvectos ae U = [ ] and U = [ ]. They ae columns in an othogonal matix U = [U U ]; note that U T U = o U T = U -. uthemoe, matix U diagonalizes T T T U ( ) U = L N M M σ 0 0 σ O QP L = N M O Q P The unit-length vecto that maximizes X is X 0 = U, and max. X X = σ = 6854 = -Nom of Matix Poduct o m n matix we now that X X (4-5) fo all X. Now, conside m n matix and n q matix B. The oduct B is m q. o q- dimensional X, we have CH4.DOC Page 4-9

10 EE448/58 Vesion.0 John Stensby BX = ( BX) BX B X. (4-6) by alying (4-5) twice in a ow. Hence, we have BX X B (4-7) fo all non-zeo X. s a esult, it follows that B B, (4-8) a useful, simle esult. -Nom Bound Let be an m n matix with elements a ij, i m, j n. Let max i, j aij denote the magnitude of the lagest element in. Let (i 0, j 0 ) be the indices of the lagest element so that ai j 0 0 aij (4-9) fo all i, j. CH4.DOC Page 4-0

11 EE448/58 Vesion.0 John Stensby Theoem 4. (-nom bound): max aij mn max i, j i, j a ij (4-30) Poof: Note that X = a x a x L amx (4-3) = max X. X = Define X 0 as the vecto with in the j 0 osition and zeo elsewhee (see (4-9) fo definition of i 0 and j 0 ). Since X0 max X = (4-3) X = we have a i j a j X 0 0 max = 0 X = (4-33) Hence, we have max i, j aij (4-34) as claimed. Now, we show the emainde of the -nom bound. Obseve that CH4.DOC Page 4-

12 EE448/58 Vesion.0 John Stensby = max ax + ax + L + amx X = max a x + ax + L + amx X = (4-35) a + a + L + am mn max i,j aij. When combined with (4-34), we have max aij mn max a ij. i, j i, j Tiangle nequality fo the -Nom Recall the Tiangle inequality fo eal numbes: α = β α + β. simila esult is valid fo the matix -nom. Theoem 4. (Tiangle nequality fo the matix -nom) Let and B be m n matices. Then + B + B (4-36) lication of Matix Nom: nvese of Matices Close to a Non-singula Matix. Let be an n n non-singula matix. Can we state conditions on the size (i.e., nom) of n n matix E which guaantees that +E is nonsingula? Befoe develoing these conditions, we deive a closely elated esult, the geometic seies fo matices. Recall the geometic seies = x, x x < (4-37) = fo eal vaiables. We ae inteested in the matix vesion of this esult. CH4.DOC Page 4-

13 EE448/58 Vesion.0 John Stensby Theoem 4.3 (Geometic Seies fo Matices) Let be any n n matix with <. Let denote the n n identity matix. Then, the diffeence - is nonsingula and ( ) = = 0 with (4-38) ( ). (4-39) Poof: ist, we show that - is nonsingula. Suose that this is not tue; suose that - is singula. Then thee exists at least one non-zeo X such that ( - )X = 0 so that X = X. But since X X, we must have X X = X which equies that, a contadiction. Hence, the n n matix - must be nonsingula. To obtain a seies fo ( - ) -, conside the obvious identity N N+ ( ) = (4-40) = 0 KJ Howeve, limit N, and since <, we have 0 as. s a esult of this, N ( ) = (4-4) = 0 KJ so that ( ) = = 0 as claimed. inally CH4.DOC Page 4-3

14 EE448/58 Vesion.0 John Stensby ( ) = = 0 = 0 = 0 (this is an "odinay" scala geometic seies). (4-4) = as claimed. n many alications, Theoem 4.3 is used in the fom ( ε) = ε fo ε < /, (4-43) = 0 whee ε is consideed to be a "small" aamete. Next, we genealize Theoem 4.3 and obtain a esult that will be used in ou study of how small eos in and b influence the solution of the linea algebaic oblem X = b. Theoem 4-4 ssume that is n n and nonsingula. Let E be an abitay n n matix. f ρ = E <, (4-44) then + E is nonsingula and ( + E) E ρ (4-45) Poof: Since is nonsingula, we have CH4.DOC Page 4-4

15 EE448/58 Vesion.0 John Stensby + E = ( ), whee - - E. (4-46) Since ρ = = E <, it follows fom Theoem 4.3 that - is nonsingula and ( ) = ρ. (4-47) om (4-46) we have ( + E) - = ( - ) - - ; with the aid of (4-47), this can be used to wite ( + E) ρ. (4-48) Now, multily both sides of ( + E) = - - E(+E) - (4-49) by + E to see this matix identity. inally, tae the nom of (4-49) to obtain ( + E) = E( + E) E ( + E) (4-50) Now use (4-48) in this last esult to obtain ( + E) E as claimed. E = ρ ρ (4-5) CH4.DOC Page 4-5

16 EE448/58 Vesion.0 John Stensby We will use Theoem 4.4 when studying the sensitivity of the linea equation X = b. That is, we want to elate changes in solution X to small changes (eos) in both and b. CH4.DOC Page 4-6