LINEAR ALGEBRA GABRIEL NAGY

Transcription

1 LINEAR ALGEBRA GABRIEL NAGY Contents Systems of liner equtions Row picture Column picture 5 3 Guss elimintion method 9 4 The spn of set of vectors 3 5 Mtri-vector product 4 6 Homogeneous liner systems nd Spns 5 7 Liner dependence nd independence 8 Appendi A Nottion nd conventions 9 Systems of liner equtions Liner Algebr is the brnch of mthemtics concerned with the study of systems of liner equtions, vectors nd vector spces, nd liner trnsformtions The equtions re clled system when there is more thn one eqution, nd they re clled liner when the unknown ppers s multiplictive fctor with power zero or one Systems of liner equtions is the min subject of this Section, nd n emple is given by Eqs (3)-(4) An emple of vector is n oriented segment, which my belong to the rel line R, or to the plne R, or to spce R 3 These three sets, together with preferred point tht is clled the origin, re emples of vector spces (see Fig ) Elements in these spces re oriented segments with origin t the origin point nd hed t ny point in these spces The origin of the word spce in the term vector spce origintes precisely in these first emples, which were ssocited with the physicl spce Two opertions re defined on oriented segments: An oriented segment cn be stretched or compressed, nd two oriented segments with the sme origin point cn be dded using the prllelogrm lw An ddition of severl stretched or compressed vectors is clled liner combintion Liner trnsformtions re prticulr type of functions on vectors tht preserve the opertion of liner combintion This is the essentil structure clled vector spce These notes re ment to be n elementry introduction into this subject Row picture A centrl problem in liner lgebr is to find solutions of system of liner equtions A liner system is system of two liner Dte: September 7, 007

2 GABRIEL NAGY O O O Figure Emple of vectors in the line, plne nd spce, respectively equtions in two unknowns, tht is, given the rel numbers,,,, b, nd b, find the rel numbers nd y solutions of () () + y = b, + y = b These equtions re clled system becuse there is more thn one eqution, nd they re clled liner becuse the unknown ppers s multiplictive fctor with power zero or one An emple of liner system is the following: Find the numbers nd y solutions of (3) (4) y = 0, + y = 3 The row picture consists of finding the solutions to the system s the intersection of ll solutions to every single eqution of the system The individul equtions re clled row equtions, or simply row of the system The solution is found geometriclly in Fig Anlyticlly, the solution cn be found by substitution: y second row y = +3 (,) first row y = 3 Figure The solution of liner system in the row picture is the intersection of the two lines, which re the solutions of ech row eqution

3 LINEAR ALGEBRA 3 y y y Figure 3 An emple of the cses given in (i)-(iii), in Theorem y = 0 y = + 4 = 3 { =, y = A consequence of the row picture in liner systems is the following result Theorem Every liner system stisfies only one of the following sttements: (i) There eists unique solution; (ii) There eist infinity mny solutions (iii) There eists no solution; Proof of Theorem : The solutions of ech eqution in liner system represents line in R Two lines in R cn intersect t point, or cn be coincident, or cn be prllel but not coincident These re the cses given in (i)-(iii), respectively, nd they re represented geometriclly in Fig 3 An m n liner system is defined s system of m liner equtions in n unknowns, tht is, given the rel numbers ij nd b i, with i =,, m nd j =, n, find the rel numbers j solutions of + + n n = b m + + mn n = b m An m n liner system is clled consistent iff it hs solutions, nd it is clled inconsistent iff it hs no solutions Emples of 3 nd 3 3 liner systems re given, respectively, s follows, (5) = = = + = + 3 = The row picture is pproprite to solve smll systems of liner equtions However it becomes difficult to crry out in 3 3 nd bigger systems For emple, find the numbers,, 3 solutions of the 3 3 liner system bove Substitute the second eqution into the first, = + 3 = = = 4 5 ; then, substitute the second eqution nd 3 = 4 5 into the third eqution, ( + ) + (4 5 ) = =,

4 4 GABRIEL NAGY nd then, substituting bckwrds, = nd 3 =, so the solution is single point in spce (,, ) Grphiclly, the solution of ech seprte eqution represents plne in R 3 A solution to the system is point tht below to the three plnes In the emple bove there is unique solution, the point (,, ), which mens tht the three plnes intersect t single point In the generl cse, 3 3 system cn hve unique solution, infinitely mny solutions or no solutions t ll, depending on how the three plnes in spce intersect mong them The cse with unique solution ws represented in Fig 4, while two possible situtions corresponding to no solution re given in Fig 5 Finlly, two cses of 3 3 liner system hving infinitely mny solutions re pictured in Fig 6, where in the first cse the solutions form line, nd in the second cse the solution form plne becuse the three plnes coincide Figure 4 Plnes representing the solutions of ech row eqution in 3 3 liner system hving unique solution Figure 5 Two cses of plnes representing the solutions of ech row eqution in 3 3 liner systems hving no solutions The solutions to bigger thn 3 3 liner system cn not be represented grphiclly, nd the substitution method becomes more involved to solve, hence lterntive ides re needed to solve such systems In the net section we introduce

5 LINEAR ALGEBRA 5 Figure 6 Two cses of plnes representing the solutions of ech row eqution in 3 3 liner systems hving infinity mny solutions the column picture, which together with Guss elimintion opertions prove to be pproprite to solve efficiently lrge systems of liner equtions Column picture Consider gin the liner system in Eqs (3)-(4) nd introduce chnge in the nmes of the unknowns, clling them nd insted of nd y The problem is to find the numbers, nd solutions of (6) (7) = 0, + = 3 We know tht the nswer is =, = The row picture consisted in solving ech row seprtely The min ide in the column picture is to interpret the liner system s n ddition of new objects, in the following wy, (8) + = 0 3 We cll these new objects column vectors, nd we use boldfce letters to denote them, tht is, 0 =, =, b = 3 We cn represent these vectors in the plne, s it is shown in Fig 7 This column 3 b Figure 7 Grphicl representtion of column vectors in the plne

6 6 GABRIEL NAGY vector interprettion of liner system determines the ddition lw of vectors nd the multipliction lw of vector by number In the emple bove, we know tht the solution is given by = nd =, therefore in the column picture interprettion the following eqution must hold + = 0 3 This emple nd the study of other emples determines the multipliction lw of vector by numbers nd the ddition lw of two vectors, ccording the following equtions, = ( ), () + = The study of severl emples of liner systems in the column picture determines the following rule Given ny -vectors u = u u nd v = v v, nd rel numbers nd b, introduce the liner combintion of u nd v s follows, u v u + bv + b = u v u + bv A liner combintion includes the prticulr cses of ddition ( = b = ), nd multipliction of vector by number (b = 0), respectively given by, u v u + v + = u u, = u v u + v u u The ddition lw in terms of components is represented grphiclly by the prllelogrm lw, s it cn be seen in Fig 8 The multipliction of vector by number ffects the length nd direction of the vector The product u stretches the vector u when > nd it compresses u when 0 < < If < 0 then it reverses the direction of u nd it stretches when < nd compresses when < < 0 Fig 8 represents some of these possibilities Notice tht the difference of two vectors is prticulr cse of the prllelogrm lw, s it cn be seen in Fig 9 v w V W V+W (v+w) V V > V 0 < < V = v (v+w) w Figure 8 The ddition of vectors cn be computed with the prllelogrm lw The multipliction of vector by number stretches or compresses the vector, nd chnges it direction in the cse tht the number is negtive

7 LINEAR ALGEBRA 7 V V W V + W V+( W) V W W W Figure 9 The difference of two vectors is prticulr cse of the prllelogrm lw of ddition of vectors The column picture interprettion of generl liner system given in Eqs ()-() is the following: Introduce the coefficient nd source column vectors (9) = b, =, b =, b nd then find the coefficients nd tht chnge the length of the coefficient column vectors nd such tht they dd up to the source column vector b, tht is, + = b For emple, the column picture of the liner system in Eqs (6)-(7) is given in Eq (8) The solution of this system re the numbers = nd =, nd this solution is represented in Fig b Figure 0 Representtion of the solution of liner system in the column picture The eistence nd uniqueness of solutions in the cse of systems cn be studied geometriclly in the column picture s it ws done in the row picture In this ltter cse we hve seen tht ll possible systems fll into one of these three cses, unique solution, infinitely mny solutions nd no solutions t ll In Fig we present these three cses in both row nd column pictures

8 8 GABRIEL NAGY y y y b b b Figure Emples of solutions of generl liner systems hving unique, infinite mny, nd no solution, represented in the row picture nd in the column picture The ides in the column picture cn be generlized to m n liner equtions, which gives rise to the generliztion to m-vectors of the definitions of liner combintion presented bove Given m-vectors u = u u m v = nd rel numbers nd b, introduce the liner combintion of the vectors u nd v s follows u v u + bv + b = u m v m u m + bv m This definition cn be generlized to n rbitrry number of vectors The m-vector b is liner combintion of the m-vectors,, n iff there eist rel numbers,, n such tht the following eqution holds, v v m + + n n = b For emple, recll the 3 3 system given s the second system in Eqs(5) This system in the column picture is the following: Find numbers, nd 3 such tht (0) = These re the min ides in the column picture We will see lter tht liner lgebr emerges from the column picture The net section we give method, due to Guss, to solve in n efficient wy m n liner systems for lrge m nd n,

9 LINEAR ALGEBRA 9 3 Guss elimintion method The Guss elimintion opertions (GEO) is method to find solutions to m n liner systems in n efficient wy Efficient mens performing s few s possible lgebric steps to find the solution or to show tht the solutions does not eist Before introducing this method, we need severl definitions Consider n m n liner system + + n n = b m + + mn n = b m Introduce the mtri of coefficients nd the ugmented mtri of liner system, given respectively by the following epressions, A := n columns {}} { n m rows, m mn n b A b := m mn b m We cll A n m n mtri, nd so the ugmented mtri of n m n liner system is given by the coefficients nd the source vector together, so it is n m (n + ) mtri The symbol := denote definition For emple, in the liner system = 0, + = 3, the mtri of coefficients is nd the ugmented mtri is 3, given respectively by 0, 3 We lso use the lterntive nottion A = ij, b = b i Given mtri A = ij, the elements ii re clled digonl elements Emples of digonl elements in 3 3, 3 nd 3 mtrices re given by the following mtrices, where mens non-digonl element, 33,, The Guss elimintion opertions refers to the following three opertions performed on the ugmented mtri: (i) Adding to one row multiple of the nother; (ii) Interchnging two rows; (iii) Multiplying row by non-zero number These opertions re respectively represented by the symbols given in Fig Figure A representtion of the Guss elimintion opertions

10 0 GABRIEL NAGY The Guss elimintion opertions chnge the coefficients of the ugmented mtri of system but do not chnge its solution Two systems of liner equtions hving the sme solutions re clled equivlent It cn be shown tht there is n lgorithm using these opertions such tht given ny m n liner system there eists n equivlent system whose ugmented mtri is simple in the sense tht the solution cn be found by inspection For emple, consider the liner system in Eq (8), construct its ugmented mtri, nd perform the following Guss elimintion opertions, , 0 nd in the lst ugmented mtri the solution, =, = is esy to red A precise wy to define the notion of esy to red is cptured in the notion is in echelon form An m n mtri is in echelon form iff every element below the digonl vnishes Mtrices with this property re lso clled upper tringulr A mtri is in reduced echelon form iff it is in echelon form nd the first nonzero element in every row stisfies both tht it is equl to nd it is the only nonzero element in tht column As n emple, the following mtrices re in echelon form, 3, 0 3, And the following mtrices re not only in echelon form but lso in reduced echelon form, 0 0 4,, Summrizing, the Guss elimintion opertions cn trnsform ny mtri into reduce echelon form Once the ugmented mtri of liner system is written in reduced echelon form, it is not difficult to decide whether the system hs solutions or not For emple, suppose tht the ugmented mtri of 3 3 liner system hs the following reduced echelon form, 0 = 3, 0 3 = 3 3, : free vrible A vrible of n m n liner system is clled free vrible iff for every vlue of tht vrible there eists solution to the liner system The following result chrcterizes n n liner systems hving free vribles Lemm An n n liner system hs k free vribles iff the reduced echelon form of its ugmented mtri contins k rows of the form 0,, 0 0 We left the proof s n eercise We remrk tht the Lemm bove is concerned only with liner systems with squre mtri of coefficients The Lemm is not true for m n liner systems when m n, s the following emples show:

11 LINEAR ALGEBRA (i) The 3 liner system below hs one free vrible but its ssocited ugmented mtri hs no line of the form 0, 0, 0 0; =, = (ii) The 3 liner system below hs one line of the form 0, 0 0 but it hs no free vribles; + 3 =, + = 0, 3 + = As n emple of the result presented in Lemm, consider the liner system () = () + 4 = 4 It is not difficult to check tht Guss elimintion opertions cn trnsform the system ugmented mtri s follow, 4 4 so the system bove hs free vrible, nd therefore, infinitely mny solutions, Lemm presented condition to determine whether squre liner system hs infinity mny solutions We now present condition on n rbitrry liner system tht determines whether the system hs no solutions We first present n emple Consider liner system with the sme mtri coefficients s the one in Eq ()-() but with different source vector: (3) = 0 (4) + 4 = 4 Multiplying the second eqution by 4 one obtins the eqution =, whose solutions form prllel line to the line given in Eq (3) Therefore, the system in Eqs (3)-(4) hs no solution Using Guss elimintion opertions it is not difficult to check tht the system ugmented mtri cn be trnsformed s follows, The lst row hs the form 0, 0, which is contrdiction, therefore the system in Eqs (3)-(4) hs no solutions These emples re prticulr cses of the following result Lemm An m n liner system is inconsistent iff the reduced echelon form of its ugmented mtri contins row of the form 0,, 0 Furthermore, consistent system contins: (i) A unique solution iff hs no free vribles; (ii) Infinitely mny solutions iff it contins t lest one free vrible

12 GABRIEL NAGY Proof of Lemm : The ide of the proof is to study ll possible forms reduced echelon form cn hve, one concludes tht there re three min cses, no solutions, unique solutions, or infinitely mny solutions, ccording to the form of the reduced echelon for of the system ugmented mtri We only give here the proof in the cse of 3 3 liner systems The reduced echelon form of n inconsistent 3 3 liner system cn hve only one of the following forms , the reduced echelon form of 3 3 liner system hving the unique solution cse hs the form ; 0 0 while the reduced echelon form of 3 3 liner system hving infinitely mny solutions cn hve only one of the following the form 0 0, , which correspond to one free vrible nd two free vribles, respectively Eercise: Find ll numbers h nd k such tht the system below hs only one, mny, or no solutions, + h = + = k Solution: Strt finding the ssocited ugmented mtri nd reducing it into echelon form, h h k 0 h k Suppose h, for emple set h =, then 0 k, 0 k 0 k so the system hs unique solution for ll vlues of k (The sme conclusion holds if one sets h to ny number different of ) Suppose now tht h =, then, 0 0 k If k = then { =, so there re infinitely mny solutions If k, then 0 0 k 0 ; : free vrible nd the system is inconsistent Summrizing, for h the system hs unique solution for every k If h = nd k = the system hs infinitely mny solutions, nd if h = nd k the system hs no solution

13 LINEAR ALGEBRA 3 4 The spn of set of vectors Recll tht in Sec we hve sid tht vector u is liner combintion of the vectors v,, v n iff there eist rel numbers c,, c n such tht u = c v + + c n v n An importnt concept in liner lgebr is tht of Spn of set of vectors The Spn of the set of m-vectors {v,, v n } is the set in R m of ll possible liner combintions of the vectors v,, v n We use the nottion Spn{v,, v n } R m For emple, ll possible liner combintions of single vector v re vectors of the form cv, nd these vectors belong to line tngent to v All liner combintions of two vectors v, w belong to plne contining both vectors Emples of these situtions cn be seen in Fig 3 R 3 R 3 Spn{V,W} Spn{V} V W V Figure 3 Emples of the spn of set of single vector, nd the spn of set of two vectors The concept of spn enters in the column picture interprettion of liner system The m n liner system + + n n = b hs solution iff the source vector b belongs to the Spn{,, n } In Fig 4 we recll the emples of liner systems of the form + = b hving unique solution, infinitely mny solutions, nd no solution, respectively In the first two cses the source vector b belongs to Spn{, }, while in the third cse b / Spn{, } b b b Figure 4 Emples of solutions of generl liner systems hving unique, infinite mny, nd no solution, represented in the column picture Therefore, in the column picture there re two equivlent interprettions of n m n liner system The first one is: Given the m-vectors,, n nd the m-vector b,

14 4 GABRIEL NAGY find the rel numbers,, n solutions of the eqution + + n n = b The second interprettion is then the following: Given the m-vectors,, n nd the m-vector b, find the rel numbers,, n such tht b Spn{,, n } 5 Mtri-vector product Mtrices hve been introduced in Sec 3 s wy to simplify the clcultions needed to find solutions of liner systems For emple, the ugmented mtri of liner system is n object contining ll the eqution coefficients nd the equtions sources which re used in the Guss elimintion opertions in order to obtin the solution to tht liner system In this section we epress liner system in yet nother wy, introducing n opertion between mtrices nd vectors Consider n m n liner system given by + + n n = b, nd introduce the mtri of coefficients A =,, n The new ide is to consider the numbers,, n s n n-vector R n given by := The mtri-vector product of the m n mtri A nd the n-vector given bove is defined s follows, A :=,, n n n := + + n n Therefore, n m n liner system cn be presented in the following wy: Given n m n mtri A nd n m-vector b, find n n-vector solution of the mtri-vector product eqution A = b As n emple, consider the liner system given in Eq (8) Introduce then the vector = Then the liner system bove cn be written s = b b Let us consider second emple, given by the 3 liner system in the unknowns nd given by = 0, + =, + = 0 The column picture interprettion is 0 + = 0

15 LINEAR ALGEBRA 5 while the mtri-vector product interprettion is 0 = 0 Consider third emple, the 3 liner system The column picture interprettion is = 0, + 3 = 3 + while the mtri-vector product interprettion is = 0 3 = The mtri-vector product introduced bove stisfies severl properties, nd one of those properties we re most interested in is tht the mtri-vector product preserves liner combintion This property is summrized in the following result Theorem If A is n m n mtri, u, v re rbitrry n-vectors, nd, b re rbitrry rel numbers, then the mtri-vector product stisfies the following eqution A(u + bv) = Au + bav This Theorem sys tht the mtri-vector product of liner combintion of vectors is the liner combintion of the mtri-vector products Proof of Theorem : This property follows directly from the definition of the mtri-vector product: u + bv A(u + bv) =,, n u n + bv n = (u + bv ) + + n (u n + bv n ) = ( u + + n u n ) + b ( v + + n v n ) = Au + bav As usul, the epression bove contins the prticulr cses = b = nd b = 0, which re given, respectively, by A(u + v) = Au + Av, A(u) = Au 6 Homogeneous liner systems nd Spns An m n liner system A = b is clled homogeneous iff the source vector b = 0 This type of liner systems possess severl interesting properties, the simplest one is tht they lwys hve t lest one solution, = 0, clled the trivil solution An eqully importnt property

16 6 GABRIEL NAGY is tht homogeneous liner systems cn lso hve non-zero solutions Consider the following emple: Find the solutions = of the system 4 0 (5) = 0 The solution of this system cn be found using Guss elimintion opertions, s follows { 4 4 =, free vrible Therefore, the set of ll solutions of the liner system bove is given by = = =, R Spn { } The epression bove mens tht ll solutions to the liner system given in Eq (5) re elements in the spn of the vector Therefore, the set of ll solutions of this liner system cn be identified with the set of points tht belong to the line shown in Fig 5 Spn{ } Figure 5 Solutions of the homogeneous liner system given in Eq (5) From the emple bove we find out tht there eists reltion between nontrivil solutions to homogeneous liner systems nd free vribles of this system This reltion is summrized in the following result Proposition An m n homogeneous liner system hs non-trivil solutions iff the system hs free vribles Proof of Proposition : Suppose tht the vector is the solution to n homogeneous liner system, nd suppose tht the component of the vector is free vrible Then, the component cn tke ny vlue nd the vector is solution of the liner system, so then tke =, which then implies tht there eists non-zero vector solution of the liner system The non-trivil solutions of homogeneous m n liner systems A = 0, where the coefficient mtri A =,, n is formed by column vectors i R m, i =, m, cn be epressed in terms of spns of vectors in R n As n emple, consider the following 3 liner system, (6) 4 3 = 3 0 0

17 LINEAR ALGEBRA 7 The coefficient mtri hs column vectors tht belong to R Let us now compute the solutions to the homogeneous liner system bove We strt with Guss elimintion opertions = 3, = 0, free vrible Therefore, the set of ll solutions of the liner system bove is given by = = 3 0 = 0 3, 3 R Spn { 0 } 3 3 In Fig 6 we emphsize tht the vectors, which re solutions of the homogeneous liner system bove, belong to the spce R 3, while the column vectors of the coefficient mtri of this sme system belong to the spce R Figure 6 The picture on the left represents the solutions of the homogeneous liner system given in Eq (6), which re elements in the spce R 3 The picture on the right represents the column vectors of the coefficient mtri in this system given in Eq (6), which re vectors in the spce R Knowing the solutions of n homogeneous liner system gives informtion bout the solutions of n inhomogeneous liner system with the sme coefficient mtri The net result estblishes this reltion in precise wy Theorem 3 If the m n liner system A = b is consistent, nd the vector p is one prticulr solution of this liner system, then ll solutions to this liner system re vectors of the form = p + h, where the vector h is ny solution of the homogeneous liner system A h = 0 Proof of Theorem 3: Let the vector be ny solution of the liner system bove, tht is, A = b Given the prticulr solution p of this liner system, introduce the vector ˆ := p Then, this vector ˆ stisfies the eqution Aˆ = A( p ) = A A p = b b = 0 Therefore, the vector ˆ is solution of the homogeneous liner system Aˆ = 0 This estblishes the Theorem We sy tht the solutions of liner system re epressed in prmetric form iff the solution vector is written s is described in Theorem 3, tht is, = p + h, where the vector h is solution of the homogeneous liner system, nd the vector p is ny solution of the inhomogeneous liner system

18 8 GABRIEL NAGY Eercise: Find ll solutions of the liner system below, nd write them in prmetric form, where the liner system is given by 4 6 (7) = 3 Solution: Find the solutions of this inhomogeneous liner system using Guss elimintion opertions, { = + 3, free vrible Therefore, the set of ll solutions of the liner system bove is given by = = + 3 = The solution in the eercise bove cn lso be epressed in the form 3 = + 0 h, h Spn{ } In Fig 7 we represent these solutions on the plne 3 3 Figure 7 The picture on the left represents four solutions of the inhomogeneous liner system given in Eq (7), which re vectors ending in the line drwn in tht picture The picture on the right represents the line ssocited with the spce Spn{ }, which pss through the origin, nd the line ssocited with the solutions of the inhomogeneous system given in Eq (7) 7 Liner dependence nd independence A non-empty set of m-vectors {v,, v n } is linerly independent (li) iff the m n homogeneous liner system V = 0 hs only the trivil solution = 0, where the mtri V is given by V = v,, v n nd the the vector R n The set of vectors bove is clled linerly dependent (Ld) iff there eists non-trivil solution, 0, of the liner system V = 0

19 LINEAR ALGEBRA 9 Appendi A Nottion nd conventions Vectors will be denoted by boldfce letters, like nd b Mtrices will be denoted by cpitl letters like A nd B We list below severl mthemticl symbols used in these notes: := Definition, = Implies, For ll, eists, Proof Begining of proof, End of proof