LINEAR TRANSFORMATIONS AND THEIR REPRESENTING MATRICES

LINEAR TRANSFORMATIONS AND THEIR REPRESENTING MATRICES DAVID WEBB CONTENTS Liner trnsformtions 2 The representing mtrix of liner trnsformtion 3 3 An ppliction: reflections in the plne 6 4 The lgebr of liner trnsformtions 8 LINEAR TRANSFORMATIONS DEFINITION A function T : R n R m is liner if it stisfies two properties: () For ny vectors v nd w in R n, (2) T (v + w) T (v) + T (w) ( comptibility with ddition ); (2) For ny vector v R n nd ny λ R, (3) T (λv) λt (v) ( comptibility with sclr multipliction ) Often liner mpping T : R n R m is clled liner trnsformtion or liner mp Some uthors reserve the word function for rel-vlued function, nd cll vector-vlued mpping trnsformtion or mpping In the specil cse when n m bove, so the domin nd trget spces re the sme, we cll liner mp T : R n R n liner opertor From the two properties () nd (2) defining linerity bove, number of fmilir fcts follow For instnce: 4 PROPOSITION A liner trnsformtion crries the origin to the origin Tht is, if T : R n R m is liner nd k R k denotes the zero vector in R k, then T ( n ) m Proof Consider the eqution n + n n in R n Apply T to both sides: then in R m we hve the eqution T ( n + n ) T ( n ) By the comptibility of T with ddition (the first property bove), this becomes T ( n ) + T ( n ) T ( n ) Now dd T ( n ) to both sides to obtin T ( n ) m 5 EXAMPLES We consider some exmples nd some non-exmples of liner trnsformtions () The identity mp Id : R n R n defined by Id(v) v is liner (check this) (2) Define R : R 3 R 3 by R x y x y, z z Dte: September 7, 2

2 DAVID WEBB reflection through the xy-plne Then R is liner Indeed, R v v 2 + w w 2 R v + w v 2 + w 2 v + w v 2 + w 2 v 3 w 3 v 3 + w 3 (v 3 + w 3 ) v v 2 + w w 2 R v v 2 + R w w 2, v 3 w 3 v 3 w 3 so (2) holds, nd one checks (3) similrly ([ [ x x (3) The mpping T : R 2 R 2 defined by T is liner (check this) y]) x] ([ ([ x x (4) The mpping T : R 2 R 2 given by T is not liner: indeed, T y]) y ]) but we know by Proposition (4) tht if([ T were liner, then we would hve T () x x (5) The mpping T : R 2 R 2 given by T y]) x 2 is not liner (why not?), (6) Any liner mpping T : R R is given by T (x) mx for some m R; ie, its grph in R 2 is line through the origin To see this, let m T () Then by (3), T (x) T (x ) xt () xm mx Wrning: Note tht liner mp f : R R in the sense of Definition () is not wht you probbly clled liner function in single-vrible clculus; the ltter ws function of the form f(x) mx + b whose grph need not pss through the origin The proper term for function like f(x) mx + b is ffine (Some uthors cll such function ffine liner, but the terminology is misleding it incorrectly suggests tht n ffine liner mp is specil kind of liner mp, when in fct s noted bove, n ffine mp is usully not liner t ll! We will use the term ffine in order to void this potentil confusion More generlly, n ffine mp A : R n R m is mpping of the form A(x) M(x) + b for x R n, where M : R n R m is liner mp nd b R m is some fixed vector in the trget spce) (7) (Very importnt exmple) Let A be ny m n mtrix Then A defines liner mp L A : R n R m ( left-multipliction by A ) defined by (6) L A (v) Av for v R n Note tht this mkes sense: the mtrix product of A (n n m mtrix) nd v ( column vector in R n, ie, n n mtrix) is n m mtrix, ie, column vector in R m Tht L A is liner is cler from some bsic properties of mtrix multipliction For exmple, to check tht (2) holds, note tht for v, w R n, L A (v + w) A(v + w) Av + Aw L A (v) + L A (w), nd (3) is checked similrly As specil cse of this lst exmple, let A Then () [ x x L A y ] y [ x x], so in this cse L A is just the liner function T of the third exmple bove

LINEAR MAPS, REPRESENTING MATRICES 3 2 THE REPRESENTING MATRIX OF A LINEAR TRANSFORMATION The remrkble fct tht mtrix clculus so useful is tht every lienr trnsformtion T : R n R m is of the form L A for some suitble m n mtrix A; this mtrix A is clled the representing mtrix of T, nd we my denote it by [T ] (2) Thus A [T ] is just nother wy of writing T L A At the risk of belboring the obvious, note tht sying tht mtrix A is the representing mtrix of liner trnsformtion T just mounts to sying tht for ny vector v in the domin of T, T (v) Av (the product on the right side is mtrix multipliction), since T (v) L A (v) Av by definition of L A Proof To see tht every liner trnsformtion T : R n R m hs the form L A for some m n mtrix A, let s consider the effect of T on n rbitrry vector in the domin R n Let x x v 2 x n e, e 2,, e n be the stndrd coordinte unit vectors (in R 3, these vectors re trditionlly denoted i, j, k) Then x x v 2 x + x 2 + + x n x e + x 2 e 2 + + x n e n, x n so T (v) T (x e + x 2 e 2 + + x n e n ) T (x e ) + T (x 2 e 2 ) + + T (x n e n ) (22) x T (e ) + x 2 T (e 2 ) + + x n T (e n ) Thus, to know wht T (v) is for ny v, ll we need to know is the vectors T (e ), T (e 2 ),, T (e n ) Ech of these is vector in R m Let s write them s 2 n T (e ) 2, T (e 2) 22,, T (e n) 2n ; m m2 mn

4 DAVID WEBB thus ij is the ith component of the vector T (e j ) R m By (22), where T (v) x T (e ) + x 2 T (e 2 ) + + T (e n ) x 2 + x 22 2 + + x 2n n m m2 mn x + 2 x 2 + + n x n 2 n x 2 x + 22 x 2 + + 2n x n 2 22 2n x 2 Av L A(v), m x + m2 x 2 + + mn x n m m2 mn x n 2 n A 2 22 2n m m2 mn Thus we hve shown tht T is just L A, where A is the m n mtrix whose columns re the vectors T (e ), T (e 2 ),, T (e n ); equivlently: (23) The representing mtrix [T ] is the mtrix whose columns re the vectors T (e ), T (e 2 ),, T (e n ) 2 n We might write this observtion schemticlly s [T ] T (e ) T (e 2 ) T (e n ) 24 EXAMPLE Let us determine the representing mtrix [Id] of the identity mp Id : R n R n By (23), the columns of [Id] re Id(e ), Id(e 2 ),, Id(e n ), ie, e, e 2,, e n But the mtrix whose columns re e, e 2,, e n is just, the n n identity mtrix I Thus [Id] I Note tht two liner trnsformtions T : R n R m nd U : R n R m tht hve the sme representing mtrix must in fct be the sme trnsformtion: indeed, if [T ] A [U], then by (2), T L A U Finlly, there is wonderfully convenient fct tht helps to orgnize mny seemingly complicted computtions very simply; for exmple, s we will see lter, it permits very simple nd esily remembered sttement of the multivrible Chin Rule Suppose tht T : R n R m nd U : R m R p re liner trnsformtions Then there is composite liner trnsformtion U T : R n R p given by (U T )(v) U(T (v)) for v R n

LINEAR MAPS, REPRESENTING MATRICES 5 R n T R m U R p U T 25 THEOREM [U T ] [U] [T ] Tht is, the representing mtrix of the composite of two liner trnsformtions is the product of their representing mtrices (Note tht this mkes sense: [T ] is n m n mtrix while U is p m mtrix, so the mtrix product [U][T ] is p n mtrix, s [U T ] should be if it represents liner trnsformtion R n R p ) Proof To see why this is true, let A [T ] nd B [U] By (2), this is just nother wy of sying tht T L A nd U L B Then for ny vector v R n, (U T )(v) U(T (v)) L B (L A (v)) B(Av) (BA)v L BA (v), where in the fourth equlity we hve used the ssocitivity of mtrix multipliction Thus U T L BA, which by (2) is just nother wy of sying tht [U T ] BA, ie, tht [U T ] [U] [T ] In fct, this theorem is the reson tht mtrix multipliction is defined the wy it is! 26 EXAMPLE Consider the liner trnsformtion R θ tht rottes the plne R 2 counterclockwise by n ngle θ Clerly, () cos θ R θ (e ) R θ, sin θ by definition of the sine nd cosine [ functions ] Now e 2 is perpendiculr to e, so R θ (e 2 ) should be unit cos θ vector perpendiculr to R θ (e ), ie, R sin θ θ (e 2 ) is one of the vectors sin θ sin θ, cos θ cos θ From the picture it is cler tht R θ (e 2 ) sin θ, cos θ since the other possibility would lso involve reflection By (23), the representing mtrix [R θ ] of R θ is the mtrix whose columns re R θ (e ) nd R θ (e 2 ); thus we hve cos θ sin θ (27) [R θ ] sin θ cos θ Consider nother rottion R ϕ by n ngle ϕ; its representing mtrix is given by cos ϕ sin ϕ [R ϕ ] sin ϕ cos ϕ Similrly, the representing mtrix of the rottion R ϕ+θ through the ngle ϕ + θ is given by cos(ϕ + θ) sin(ϕ + θ) [R ϕ+θ ] sin(ϕ + θ) cos(ϕ + θ) Now the effect of rotting by n ngle θ nd then by n ngle ϕ should be simply rottion by the ngle ϕ+θ, ie, (28) R ϕ+θ R ϕ R θ

6 DAVID WEBB Thus By Theorem (25), this becomes ie, cos(ϕ + θ) sin(ϕ + θ) sin(ϕ + θ) cos(ϕ + θ) [R ϕ+θ ] [R ϕ R θ ] [R ϕ+θ ] [R ϕ ] [R θ ], [ cos ϕ sin ϕ cos θ sin θ sin ϕ cos ϕ sin θ cos θ After multiplying out the right hnd side, this becomes cos(ϕ + θ) sin(ϕ + θ) cos ϕ cos θ sin ϕ sin θ cos ϕ sin θ sin ϕ cos θ sin(ϕ + θ) cos(ϕ + θ) sin ϕ cos θ + cos ϕ sin θ sin ϕ sin θ + cos ϕ sin θ Compring the entries in the first column of ech mtrix, we obtin (29) (2) cos(ϕ + θ) cos ϕ cos θ sin ϕ sin θ, sin(ϕ + θ) sin ϕ cos θ + cos ϕ sin θ Thus from (28) long with Theorem (25) we recover the trigonometric sum identities 3 AN APPLICATION: REFLECTIONS IN THE PLANE Let l θ be line through the origin in the plne; sy its eqution is y mx, where m tn θ; thus θ is the ngle of tht the line l θ mkes with the x-xis Consider the liner trnsformtion F θ : R 2 R 2 clled reflection through l θ defined geometriclly s follows: intuitively, we view the line l θ s mirror; then for ny vector v R 2 (viewed s the position vector of point P in the plne), F θ (v) is the position vector of the mirror imge point of P on the other side of the line l θ More precisely, write v s sum of two vectors: (3) v v + v, where v is vector long the line l θ nd v is perpendiculr to l θ Then F θ (v) v v We wish to determine n explicit formul for F θ This cn be done using elementry plne geometry nd fmilir fcts bout the dot product, but we seek here to outline method tht works in high dimensions s well, when it is not so esy to visulize the mp geometriclly To this end, note tht we lredy know formul for F θ when θ : indeed, in tht cse, the line l θ is just the x-xis, nd the reflection F is given by () [ x x F y y] For our purposes, though, more useful (but equivlent) wy of sying this is vi the representing mtrix [F ], which we now compute The representing mtrix [F ] is the mtrix whose columns re F (e ) nd F (e 2 ) It is obvious tht F (e ) e nd F (e 2 ) e 2, but let s check it nywy It we tke v e in eqution (3), we see tht v e (v e is on the line l, the x-xis), so v, the zero vector; thus F (e ) v v e e Similrly, if we tke v e 2 in eqution (3), we find v while v e 2 (v e 2 is itself perpendiculr to the x-xis), so F (e 2 ) v v e 2 e 2 Thus [F ] is the mtrix whose columns re e nd e 2, ie, (32) [F ] ]

LINEAR MAPS, REPRESENTING MATRICES 7 To use this seemingly trivil observtion to compute F θ, we note tht F θ cn be written s composition of three liner trnsformtions: (33) F θ R θ F R θ, where R θ is the rottion by θ opertor whose representing mtrix we lredy determined in eqution (27) Tht is, in order to reflect through the line l θ, we cn proceed s follows: first, rotte the whole plne through n ngle θ; this opertion moves the mirror line l θ to the x-xis We now reflect the plne through the x-xis, which we know how to do by eqution (32) Finlly, we rotte the whole plne bck through n ngle of θ in order to undo the effect of our originl rottion through n ngle of θ; this opertion restores the line in which we re interested (the mirror ) to its originl position, mking n ngle θ with the x-xis By Theorem (25), we cn now compute [F θ ] esily: [F θ ] [R θ F R θ ] [R θ ][F ][R θ ], ie, cos θ sin θ cos( θ) sin( θ) [F θ ] sin θ cos θ sin( θ) cos( θ) multiplying out the mtrices yields [F θ ] [ cos θ sin θ sin θ cos θ cos 2 θ sin 2 θ 2 cos θ sin θ 2 cos θ sin θ sin 2 θ cos 2 θ Using the identities (29) with ϕ θ, we finlly conclude tht cos(2θ) sin(2θ) (34) [F θ ] sin(2θ) cos(2θ) ] [ ] cos θ sin θ ; sin θ cos θ To see tht this mkes some sense geometriclly, note tht the first column of the mtrix [F θ ] is the sme s the first column of [R 2θ ], the representing mtrix of the rottion through n ngle 2θ But for ny liner mp T : R 2 R 2, the first column of [T ] is just the vector T (e ) Thus the equlity of the first columns of [F θ ] nd [R 2θ ] just mens tht F θ (e ) R 2θ (e ), which mkes geometric sense: if you reflect the horizontl vector e through the line l θ, the resulting vector should surely be unit vector mking n ngle θ with the line l θ, hence mking n ngle 2θ with the x-xis Although we will not mke serious use of the striking power of the technique illustrted bove, we conclude by showing how the lgebr of liner trnsformtions, once codified vi their representing mtrices, cn led to illuminting geometric insights We consider very simple exmple to illustrte Let us rewrite eqution (34): cos(2θ) sin(2θ) cos(2θ) sin(2θ) [F θ ] [F θ ], sin(2θ) cos(2θ) sin(2θ) cos(2θ) the right hnd side of which we immeditely recognize s [R 2θ ][F ] But by Theorem (25), the ltter is just [R 2θ F ] Thus [F θ ] [R 2θ F ] Since two liner trnsformtions with the sme representing mtrix re the sme, it follows tht F θ R 2θ F Now compose on the right by F nd use ssocitivity of function composition: (35) F θ F (R 2θ F ) F R 2θ (F F )

8 DAVID WEBB Finlly, observe tht F F Id, the identity trnsformtion (This is obvious geometriclly, but we cn prove it crefully by using the representing mtrix: by Theorem (25) nd eqution (32), [F F ] [F ][F ] I [Id], so F F Id) Then eqution (35) becomes (36) F θ F R 2θ Thus we hve proved: 37 THEOREM The effect of reflection of the plne through the x-xis followed by reflection of the plne through the line l θ mking n ngle θ with the x-xis is just tht of rottion of the plne through the ngle 2θ You should drw pictures nd convince yourself of this fct In fct, more is true: for ny line λ in the plne, let F λ : R 2 R 2 denote reflection through the line λ 38 THEOREM For ny two lines α nd β in the plne, the composite F β F α is the rottion through double the ngle between the lines α nd β The proof is immedite: we merely set up our coordinte xes so tht the x-xis is the line α; then we hve reduced to the cse of Theorem (37) 4 THE ALGEBRA OF LINEAR TRANSFORMATIONS We will denote the collection of ll liner mps T : R n R m by L(R n, R m ) We cn dd two elements of L(R n, R m ) in n obvious wy: 4 DEFINITION If T, U L(R n, R m ), then T + U is the mpping given by (T + U)(v) T (v) + U(v) for ll vectors v R n You should check tht T + U is indeed liner 42 THEOREM Let T, U L(R n, R m ) Then [T + U] [T ] + [U] In words, the representing mtrix of the sum of two liner trnsformtions is the sum of their representing mtrices The proof is simple exercise using the definition of the representing mtrix nd the definition of mtrix ddition 43 DEFINITION Given liner mp T : R n R m nd sclr c R, we define liner mp ct from R n to R m (clled sclr multipliction of T by c) by (ct )(v) c T (v) for ll v R n As bove, you should check tht ct is liner, nd prove the following theorem: 44 THEOREM Let T L(R n, R m ), nd let c R Then [ct ] c[t ] Note tht the two definitions bove endow the set L(R n, R m ) with the structure of vector spce The definition of mtrix ddition nd sclr multipliction endow the spce M m n (R) of m by n mtrices (with rel entries) with the structure of vector spce There is mpping : L(R n, R m ) M m n (R) tht sends liner trnsformtion T to its representing mtrix [T ] The two simple theorems bove merely sy tht this mpping is itself liner mp, using the vector spce structures on its domin nd trget tht we

LINEAR MAPS, REPRESENTING MATRICES 9 defined bove! But is even better thn just liner mp: it hs n inverse L ( ) : M m n (R) L(R n, R m ) tht sends mtrix A M m n (R) to the liner trnsformtion L A : R n R m given by L A (v) Av The fct tht nd L ( ) re inverses of ech other is just the observtion (2) Since : L(R n, R m ) M m n is liner, ny liner question bout liner mps from R n to R m (ie, ny question involving ddition of such liner mps or multipliction by sclr) cn be trnslted vi to question bout mtrices, which re much more concrete nd better suited to computtion thn bstrct entities like liner trnsformtions; the fct tht hs liner inverse L ( ) mens tht nothing is lost in the trnsltion The mpping is clssic exmple of n isomorphism Formlly, n isomorphism of two vector spces V nd W is just liner mp V W with liner inverse W V The ide is tht V nd W re to ll intents nd purposes the sme, t lest s fr s liner lgebr is concerned An isomorphism of n bstrct vector spce V with more concrete nd redily understood vector spce W mkes V just s esy to understnd s W : we simply use the isomorphism to trnslte question bout V to n equivlent question bout W, solve our problem in W (where is is esy), then use the inverse of the isomorphism to trnslte our solution bck to V While we will not mke serious use of the ide of isomorphism (t lest not explicitly), there is one importnt exmple tht will use, nd in ny cse the ide is so importnt throughout mthemtics tht is is worth mking it explicit t this point We will show tht the spce of liner mps L(R, R m ) of liner mps from the rel line to R m is the sme s the spce R m itself In fct, ny liner mp T : R R n hs representing mtrix [T ] M m in the spce of m by mtrices; but n m by mtrix (m rows, one column) is just column vector of length m, ie, n element of R m Wht does this representing mtrix [T ] look like? Recll tht its columns re just the vlues tht the liner mp T tkes on the stndrd coordinte vectors But in R, there is only one stndrd coordinte vector: e, the rel number Thus the first (nd only!) column of [T ] is just the vector T () R m We cn summrize s follows: (45) We cn view ny liner mp T : R R m s column vector in R m ; we do so by ssociting to the liner mp T the vector T () R m which, when viewed s n m by mtrix, is just the representing mtrix [T ] Another wy of rriving t the sme conclusion is the following If T : R R m is ny liner mp from the rel line to R m, note tht for ny x R, T (x) T (x ) xt (), by (3); equivlently, T (x) T ()x, which just sys tht T L T (), ie (by (2)), [T ] T ()