ROTATION MATRICES. cos θ sin θ. = cos θ

Transcription

1 ROTATION MATRICES A = [ cos θ sin θ sin θ cos θ Note that [ 1 A 0 = [ cos θ sin θ, A [ 0 1 = [ sin θ cos θ This shows the vectors e (1) and e (2) are rotated counterclockwise thru an angle of θ radians. In particular, [ Ae (2) sin θ = = cos ( θ + π ) 2 cos θ sin ( θ + π ) 2 Thus in general, the transformation x Ax corresponds to a rotation of x counter-clockwise thru an angle of θ radians. A 1 should correspond to a clockwise rotation thru θ radians; or replacing θ by θ in the original formula for A, wehave [ [ A 1 cos ( θ) sin ( θ) cos θ sin θ = = sin ( θ) cos( θ) sin θ cos θ

2 We generalize this in a very simple way to R n. Let 1 k<l n, and define the matrix R k,l to the following matrix: cosθ 0 0 sin θ sinθ 0 0 cosθ In it, we have modified the identity matrix I, changing it in the four elements in positions (k, k), (k, l), (l, k), and (l, l). The matrix R k,l will rotate the (k, l)-plane thru an angle of θ, while leaving unchanged the remainder of R n, that part perpendicular to (k, l)-plane.

3 HOUSEHOLDER MATRICES Let w C n be a vector of Euclidean length 1, w w =1 ( w T w =1forw R n) Define the matrix H = I 2ww ( I 2ww T for w R n ) Then H is a Hermitian unitary matrix (orthogonal if w R n ). First, Also, H = (I 2ww ) H H = H 2 = I 2(ww ) = I 2(w ) w = H = (I 2ww ) 2 = I 4ww +4(ww )(ww )

4 Note that Thus (ww )(ww )=w(w w) w = ww } {{ } =1 H H = I 4ww +4(ww )(ww )=I showing H is unitary. What does H do in a geometric sense? that First, note Hw =(I 2ww ) w = w 2ww w = w 2w = w Also, let v be any vector orthogonal to w. Then (ww ) v = w (w v)=w (0) = 0 Hv =(I 2ww ) v = v 2(ww ) v = v Thus H is a reflection of space thru the (n 1)- dimensional hyperplane perpendicular to w. The rotation matrices R k,l and the Householder matrices H are the most commonly used orthogonal (or unitary) matrices used in numerical analysis.

5 EXAMPLES For the n =3case,withw R 3, H = 1 2w 2 1 2w 1w 2 2w 1 w 3 2w 1 w 2 1 2w 2 2 2w 2w 3 2w 1 w 3 2w 2 w 3 1 2w 2 3 Then H T = H and H 2 = I. For w = [ 1 3, 2 3, 2 3 T, H = For w = [ 0, 3 5, 4 5 T, H =

6 REDUCTION OF A VECTOR Given a vector d R m,wewanttofindv R m with v 2 =1and ( I 2vv T ) d = α 0. 0 (1) Since I 2vv T is orthogonal, the length of d is preserved in the tranformation of d: α = d 2 S and α = ±S = ± sqrt ( d d2 m ) with the sign to be determined later.

7 Introduce p = v T d From (1), d 2pv =[α, 0,..., 0 T (2) Multiply on the left by v T to get p 2pv T v = v 1 α p = αv 1 Substitute this back into (2). Then look at the individual components, obtaining d 1 +2αv1 2 = α d i 2pv i = 0, i =2,..., m From the first equation, (3) v1 2 = α d 1 2α = 1 1 d 1 (4) 2 α Recall that we have not yet chosen the sign of α. Now choose the sign of α according to ( sign (α) = sign (d 1 ) )

8 The subtraction in (4) is now actually an addition, so as to avoid a loss of significance error. We can take the square root in (4) to find v 1,andthereisno obvious choice of the sign here, although most people would choose v 1 > 0. Return to p = αv 1 to obtain p. Then return to (3) to find v 2,..., v m : v i = d i 2p, i =2,..., m This completes the construction of v and therefore the Householder matrix I 2vv T based upon it. Again, we now have ( I 2vv T ) d = α 0. 0

9 EXAMPLE Let Then d =[2, 2, 1 T α = d 2 = 3, v 1 =sqrt ( 5 6 ), p =sqrt ( 152 ) Then v 2 = 2 sqrt (30), v 3 = H = sqrt (30) In practice, one does not need H explicitly.

10 QR FACTORIZATION Let A be a matrix that is n n. We want to factor it into the form A = QR with Q orthogonal and R upper triangular. We do this by working on each of the columns in succession. For the first step, let P 1 = I 2w (1) w (1)T, w (1) 2 =1 Let A = [ A,1,..., A,n.Then We choose P 1 so that P 1 A = [ P 1 A,1,..., P 1 A,n P 1 A,1 =[α, 0,..., 0 T We can of course do this by the construction already described above, with d = A,1.

11 With this choice of P 1,thematrixP 1 A will have zeroes in the first column below the diagonal position. Next, we wish to do the same to the second column, but without changing the first column. Define P 2 = I 2w (2) w (2)T, w (2) 2 =1 But now require [ w (2) 0 = v Then Calculate P 2 P 1 A: P 2 =, v T v =1, v R n 1 [ I 2vv T P 2 P 1 A = [ P 2 P 1 A,1,..., P 2 P 1 A,n

12 We know that and therefore P 2 P 1 A,1 = P 1 A,1 =[α, 0,..., 0 T [ I 2vv T [ α 0 = [ α 0 Thus the first column of P 2 P 1 A retains its zero structure below the diagonal. Now choose v so as to force the second column of P 2 P 1 A to have zeroes below the diagonal position. Writing [ c P 1 A,2 =, d R d n 1 we choose v by forcing ( I 2vv T ) d =[α 2, 0,..., 0 T for a suitable value of α 2.

13 Continue in this manner, defining P i = I 2w (i) w (i)t, w (i) 2 =1 with w (i) = [ 0 v, v T v =1, v R n i+1 and choosing v to force P i P 1 A to have zeros below the diagonal in column i. After n 1 steps, the matrix P n 1 P 2 P 1 A R is upper triangular. Note that the matrix P n 1 P 2 P 1 is orthogonal, (P n 1 P 2 P 1 ) T (P n 1 P 2 P 1 ) We define Then = P T 1 P T n 1 P n 1 P 2 P 1 = I Q T = P n 1 P 2 P 1 Q T A = R A = QR

14 EXAMPLE An example for the matrix A = is given on page 613 in the text. The result is A = QR with R = Q = P 1 P 2 = This factorization can also be accomplished using the Matlab instruction qr: [Q R=qr(A)

15 HOUSEHOLDER MULTIPLICATIONS Consider multiplying a Householder matrix H times another matrix: HA = ( I 2ww T ) A = A ( 2ww T ) A = A 2w ( w T A ) The quantity w T A is a vector, and it can be produced with approximately 2n 2 operations if w has all nonzero components. Then calculating 2w ( w T A ) will cost a further n 2 multiplications, approximately, and A 2w ( w T A ) will cost n 2 subtractions. Thus the total operations cost to produce HA will be around 4n 2 operations, rather than the 2n 3 one would usually expect with multiplying two n n matrices. This also shows why we generally do not produce Q = P 1 P n 1 explicitly, as it is cheaper to carry out matrix multiplications when Q is in factored form.

16 REDUCTION OF SYMMETRIC MATRICES Let A be a symmetric matrix. We want to use a similarity transformation to reduce it to symmetric tridiagonal form. Assume A is n n. We will perform a sequence of similarity transformations, using Householder matrices, to reduce A to tridiagonal form. We begin by using a similarity transformation to put zeros into the first column of the matrix, in the positions below the (2,1) position. P T 1 AP 1 = A 2 = We use the Householder matrix with P 1 = I 2w (2) w (2)T, a 1,1 â 2,1 0 0 â 2,1 â 2,2 â n, â n,2 â n,n w (2) 2 =1 w (2) =[0,v 1,..., v m T, m = n 1

17 Then P 1 = v1 2 2v 1 v m v 1 v m 1 2v 2 m Let A = [ A,1,..., A,n.Then P 1 A = [ P 1 A,1,..., P 1 A,n The first column looks like a 1,1 a 1, v 2 1 2v 1 v m a 2, = â 2,1 0 2v 1 v m 1 2vm 2 a n,1 0. With reference to our earlier work on reducing a vector d to a simpler form with only a single nonzero component, we use that algorithm with d = [ a 2,1,..., a n,1 T With that algorithm, we can find v R m with ( I 2vv T ) d = [ â 2,1, 0,..., 0 T

18 In fact, â 2,1 = sign ( a 2,1 ) sqrt ( a 2 2,1 + + a 2 n,1 ) Now look at P 1 A and P 1 AP 1 : P 1 A = P 1 AP 1 is given by a 1,1 â 2, a 1,1 â 2, v 2 1 2v 1 v m v 1 v m 1 2v 2 m = a 1,1 â 2,

19 In addition, the matrix P 1 AP 1 is symmetric: (P 1 AP 1 ) T = P T 1 AT P T 1 = P 1AP 1 Therefore, P 1 AP 1 must look like A 2 = P 1 AP 1 = a 1,1 â 2,1 0 0 â 2, We continue this process by now reducing the second column by a similar operation. We use the Householder matrix P 2 = I 2w (3) w (3)T, w (3) 2 =1 Then w (3) =[0, 0,v 1,..., v m T, m = n 2 P 2 = v v 1 v m v 1 v m 1 2vm 2

20 We choose v R n 2 such that ( I 2vv T ) d = [ â 3,2, 0,..., 0 T with d the elements in positions 3 thru n of column 2ofthematrixA 2. Note that the form of P 2 is such that P 2 A 2 P 2 will have the same first column and row as in A 2. For example, consider calculating first P 2 A 2 : v v 1 v m v 1 v m 1 2vm 2 We continue in this way, obtaining finally a 1,1 â 2,1 0 0 â 2, P n 2 P 1 AP 1 P n 2 = T with T a symmetric tridiagonal matrix. Define Q = P 1 P n 2 It is orthogonal, and Q T AQ = T

21 Therefore the eigenvalues of A and T are the same. For the eigenvectors, let Then Tx = λx, x 0 Q T AQx = λx A (Qx) = λ (Qx) Thus Qx is the eigenvector of A corresponding to the eigenvector x for T. EXAMPLE: (From page 617) A = , w (2) = P 1 = I 2w (2) w (2)T = T = A 2 = 5 [ 0, 2 sqrt(5), 1 sqrt(5)

22 When the rounding errors inherent in producing T from A are taken into account, how do the eigenvalues of T and A compare. This is given in the text as Theorem 9.4 (page 617). It is assume that the arithmetic being used is t-digit binary arithmetic with rounding; and moreover, it is assumed that all inner products m j=1 a j b j are accumulated in a higher precision and then rounded back to t digits at the completion of the summation process. With this, we obtain for the eigenvalues { λ j } and { τ j } of A and T respectively, that n j=1 ( τj λ j ) 2 n λ 2 j j=1 1 2 c n 2 t c n =25(n 1) [ 1+(12.36) 2 t 2n 4. =25(n 1)