# Basic Linear Algebra. 2.1 Matrices and Vectors. Matrices. For example,, 1 2 3

Save this PDF as:

Size: px
Start display at page:

Download "Basic Linear Algebra. 2.1 Matrices and Vectors. Matrices. For example,, 1 2 3"

## Transcription

1 Basic Linear Algebra In this chapter, we study the topics in linear algebra that will be needed in the rest of the book. We begin by discussing the building blocks of linear algebra: matrices and vectors. Then we use our knowledge of matrices and vectors to develop a systematic procedure (the Gauss Jordan method) for solving linear equations, which we then use to invert matrices. We close the chapter with an introduction to determinants. The material covered in this chapter will be used in our study of linear and nonlinear programming.. Matrices and Vectors Matrices DEFINITION A matrix is any rectangular array of numbers. For example,,,, [ ] are all matrices. If a matrix A has m rows and n columns, we call A an m n matrix. We refer to m n as the order of the matrix. A typical m n matrix A may be written as A a a a m 6 a a a m a n a n a mn DEFINITION The number in the ith row and jth column of A is called the ijth element of A and is written a ij. For example, if A 7 then a, a 6, and a

2 Sometimes we will use the notation A [a ij ] to indicate that A is the matrix whose ijth element is a ij. DEFINITION Two matrices A [a ij ] and B [b ij ] are equal if and only if A and B are of the same order and for all i and j, a ij b ij. For example, if x y A and B then A B if and only if x, y, w, and z. w z Vectors Any matrix with only one column (that is, any m matrix) may be thought of as a column vector. The number of rows in a column vector is the dimension of the column vector. Thus, may be thought of as a matrix or a two-dimensional column vector. R m will denote the set of all m-dimensional column vectors. In analogous fashion, we can think of any vector with only one row (a n matrix as a row vector. The dimension of a row vector is the number of columns in the vector. Thus, [9 ] may be viewed as a matrix or a three-dimensional row vector. In this book, vectors appear in boldface type: for instance, vector v. An m-dimensional vector (either row or column) in which all elements equal zero is called a zero vector (written ). Thus, [ ] and are two-dimensional zero vectors. Any m-dimensional vector corresponds to a directed line segment in the m-dimensional plane. For example, in the two-dimensional plane, the vector u corresponds to the line segment joining the point to the point The directed line segments corresponding to are drawn in Figure. u, v, w CHAPTER Basic Linear Algebra

3 x u = v = w = u (, ) x w v (, ) FIGURE Vectors Are Directed Line Segments (, ) The Scalar Product of Two Vectors An important result of multiplying two vectors is the scalar product. To define the scalar product of two vectors, suppose we have a row vector u = [u u u n ] and a column vector v v v n of the same dimension. The scalar product of u and v (written u v) is the number u v u v u n v n. For the scalar product of two vectors to be defined, the first vector must be a row vector and the second vector must be a column vector. For example, if u [ ] and v then u v () () (). By these rules for computing a scalar product, if then u v is not defined. Also, if u and v [ ] u [ ] and v then u v is not defined because the vectors are of two different dimensions. Note that two vectors are perpendicular if and only if their scalar product equals. Thus, the vectors [ ] and [ ] are perpendicular. We note that u v u v cos u, where u is the length of the vector u and u is the angle between the vectors u and v. v. Matrices and Vectors

4 Matrix Operations We now describe the arithmetic operations on matrices that are used later in this book. The Scalar Multiple of a Matrix Given any matrix A and any number c (a number is sometimes referred to as a scalar), the matrix ca is obtained from the matrix A by multiplying each element of A by c. For example, 6 if A, then A For c, scalar multiplication of the matrix A is sometimes written as A. Addition of Two Matrices Let A [a ij ] and B [b ij ] be two matrices with the same order (say, m n). Then the matrix C A B is defined to be the m n matrix whose ijth element is a ij b ij. Thus, to obtain the sum of two matrices A and B, we add the corresponding elements of A and B. For example, if then A and B A B. This rule for matrix addition may be used to add vectors of the same dimension. For example, if u [ ] and v [ ], then u v [ ] [ ]. Vectors may be added geometrically by the parallelogram law (see Figure ). We can use scalar multiplication and the addition of matrices to define the concept of a line segment. A glance at Figure should convince you that any point u in the m-dimensional plane corresponds to the m-dimensional vector u formed by joining the origin to the point u. For any two points u and v in the m-dimensional plane, the line segment joining u and v (called the line segment uv) is the set of all points in the m-dimensional plane that correspond to the vectors cu ( c)v, where c (Figure ). For example, if u (, ) and v (, ), then the line segment uv consists x v = u = u + v = (, ) (, ) u u + v (, ) FIGURE Addition of Vectors (, ) v x CHAPTER Basic Linear Algebra

5 x u c = c = FIGURE Line Segment Joining u (, ) and v (, ) c = v x of the points corresponding to the vectors c[ ] ( c)[ ] [ c c], where c. For c and c, we obtain the endpoints of the line segment uv; for c, we obtain the midpoint (.u.v) of the line segment uv. Using the parallelogram law, the line segment uv may also be viewed as the points corresponding to the vectors u c(v u), where c (Figure ). Observe that for c, we obtain the vector u (corresponding to point u), and for c, we obtain the vector v (corresponding to point v). The Transpose of a Matrix Given any m n matrix A the transpose of A (written A T ) is the n m matrix a a a m a a a m a n a n a mn A T a a a n a a a n a m a m a mn x u v u c = c = c = v x FIGURE Representation of Line Segment uv u v u. Matrices and Vectors

6 Thus, A T is obtained from A by letting row of A be column of A T, letting row of A be column of A T, and so on. For example, if A, then A T 6 6 Observe that (A T ) T A. Let B [ ]; then B T and (B T ) T [ ] B As indicated by these two examples, for any matrix A, (A T ) T A. Matrix Multiplication Given two matrices A and B, the matrix product of A and B (written AB) is defined if and only if Number of columns in A number of rows in B () For the moment, assume that for some positive integer r, A has r columns and B has r rows. Then for some m and n, A is an m r matrix and B is an r n matrix. DEFINITION The matrix product C AB of A and B is the m n matrix C whose ijth element is determined as follows: ijth element of C scalar product of row i of A column j of B () If Equation () is satisfied, then each row of A and each column of B will have the same number of elements. Also, if () is satisfied, then the scalar product in Equation () will be defined. The product matrix C AB will have the same number of rows as A and the same number of columns as B. EXAMPLE Solution Matrix Multiplication Compute C AB for A and B Because A is a matrix and B is a matrix, AB is defined, and C will be a matrix. From Equation (), c [ ] c [ ] c [ ] () () () () () () 8 () () () 7 6 CHAPTER Basic Linear Algebra

7 c [ ] C AB 7 8 () () () EXAMPLE Column Vector Times Row Vector Find AB for A and B [ ] Solution Because A has one column and B has one row, C AB will exist. From Equation (), we know that C is a matrix with c () c () c () 6 c () 8 Thus, 6 C 8 EXAMPLE Row Vector Times Column Vector Compute D BA for the A and B of Example. Solution In this case, D will be a matrix (or a scalar). From Equation (), d [ ] () () Thus, D []. In this example, matrix multiplication is equivalent to scalar multiplication of a row and column vector. Recall that if you multiply two real numbers a and b, then ab ba. This is called the commutative property of multiplication. Examples and show that for matrix multiplication, it may be that AB BA. Matrix multiplication is not necessarily commutative. (In some cases, however, AB BA will hold.) EXAMPLE Solution Undefined Matrix Product Show that AB is undefined if A and B This follows because A has two columns and B has three rows. Thus, Equation () is not satisfied.. Matrices and Vectors 7

9 This answer agrees with Example. [ ] [7 ] Column j of AB A(column j of B). Thus, for A and B as given, the first column of AB is 7 Properties and are helpful when you need to compute only part of the matrix AB. Matrix multiplication is associative. That is, A(BC) (AB)C. To illustrate, let A [ ], B, C Then AB [ ] and (AB)C () () []. On the other hand, 7 BC so A(BC) (7) () []. In this case, A(BC) (AB)C does hold. Matrix multiplication is distributive. That is, A(B C) AB AC and (B C)D BD CD. Matrix Multiplication with Excel Mmult.xls Using the Excel MMULT function, it is easy to multiply matrices. To illustrate, let s use Excel to find the matrix product AB that we found in Example (see Figure and file Mmult.xls). We proceed as follows: Step Step Enter A and B in D:F and D:E7, respectively. Select the range (D9:E) in which the product AB will be computed. Step In the upper left-hand corner (D9) of the selected range, type the formula MMULT(D:F,D:E7) Then hit Control Shift Enter (not just Enter), and the desired matrix product will be computed. Note that MMULT is an array function and not an ordinary spreadsheet function. This explains why we must preselect the range for AB and use Control Shift Enter. FIGURE A B C D E F MatrixMultiplication A B 8 C 7. Matrices and Vectors 9

10 PROBLEMS Group A For A 6 and B, find: a A b A c A B d A T e B T f AB g BA Only three brands of beer (beer, beer, and beer ) are available for sale in Metropolis. From time to time, people try one or another of these brands. Suppose that at the beginning of each month, people change the beer they are drinking according to the following rules: % of the people who prefer beer switch to beer. % of the people who prefer beer switch to beer. % of the people who prefer beer switch to beer. % of the people who prefer beer switch to beer. % of the people who prefer beer switch to beer. For i,,, let x i be the number who prefer beer i at the beginning of this month and y i be the number who prefer beer i at the beginning of next month. Use matrix multiplication to relate the following: y y y x x x Group B Prove that matrix multiplication is associative. Show that for any two matrices A and B, (AB) T B T A T. An n n matrix A is symmetric if A A T. a Show that for any n n matrix, AA T is a symmetric matrix. b Show that for any n n matrix A, (A A T ) is a symmetric matrix. 6 Suppose that A and B are both n n matrices. Show that computing the matrix product AB requires n multiplications and n n additions. 7 The trace of a matrix is the sum of its diagonal elements. a For any two matrices A and B, show that trace (A B) trace A trace B. b For any two matrices A and B for which the products AB and BA are defined, show that trace AB trace BA.. Matrices and Systems of Linear Equations Consider a system of linear equations given by a x a x a n x n b a x a x a n x n b () a m x a m x a mn x n b m In Equation (), x, x,..., x n are referred to as variables, or unknowns, and the a ij s and b i s are constants. A set of equations such as () is called a linear system of m equations in n variables. DEFINITION A solution to a linear system of m equations in n unknowns is a set of values for the unknowns that satisfies each of the system s m equations. To understand linear programming, we need to know a great deal about the properties of solutions to linear equation systems. With this in mind, we will devote much effort to studying such systems. We denote a possible solution to Equation () by an n-dimensional column vector x, in which the ith element of x is the value of x i. The following example illustrates the concept of a solution to a linear system. CHAPTER Basic Linear Algebra

11 EXAMPLE Solution Solution to Linear System Show that is a solution to the linear system and that is not a solution to linear system (). To show that x x x x x x x is a solution to Equation (), we substitute x and x in both equations and check that they are satisfied: () and (). The vector x is not a solution to (), because x and x fail to satisfy x x. () Using matrices can greatly simplify the statement and solution of a system of linear equations. To show how matrices can be used to compactly represent Equation (), let A a a a m a a a m a n a n a mn, x, x x b x n b b b m Then () may be written as Ax b () Observe that both sides of Equation () will be m matrices (or m column vectors). For the matrix Ax to equal the matrix b (or for the vector Ax to equal the vector b), their corresponding elements must be equal. The first element of Ax is the scalar product of row of A with x. This may be written as [a a a n ] x x a x a x a n x n x n This must equal the first element of b (which is b ). Thus, () implies that a x a x a n x n b. This is the first equation of (). Similarly, () implies that the scalar. Matrices and Systems of Linear Equations

12 product of row i of A with x must equal b i, and this is just the ith equation of (). Our discussion shows that () and () are two different ways of writing the same linear system. We call () the matrix representation of (). For example, the matrix representation of () is x Sometimes we abbreviate () by writing Ab (6) If A is an m n matrix, it is assumed that the variables in (6) are x, x,..., x n. Then (6) is still another representation of (). For instance, the matrix represents the system of equations x x x x x x x x x PROBLEM Group A Use matrices to represent the following system of equations in two different ways: x x x x 6 x x 8. The Gauss Jordan Method for Solving Systems of Linear Equations We develop in this section an efficient method (the Gauss Jordan method) for solving a system of linear equations. Using the Gauss Jordan method, we show that any system of linear equations must satisfy one of the following three cases: Case Case The system has no solution. The system has a unique solution. Case The system has an infinite number of solutions. The Gauss Jordan method is also important because many of the manipulations used in this method are used when solving linear programming problems by the simplex algorithm (see Chapter ). Elementary Row Operations Before studying the Gauss Jordan method, we need to define the concept of an elementary row operation (ERO). An ERO transforms a given matrix A into a new matrix A via one of the following operations. CHAPTER Basic Linear Algebra

13 Type ERO A is obtained by multiplying any row of A by a nonzero scalar. For example, if A then a Type ERO that multiplies row of A by would yield A Type ERO Begin by multiplying any row of A (say, row i) by a nonzero scalar c. For some j i, let row j of A c(row i of A) row j of A, and let the other rows of A be the same as the rows of A. For example, we might multiply row of A by and replace row of A by (row of A) row of A. Then row of A becomes [ 6] [ ] [ 7] and A Type ERO Interchange any two rows of A. For instance, if we interchange rows and of A, we obtain A 9 Type and Type EROs formalize the operations used to solve a linear equation system. To solve the system of equations x x (7) x x 7 we might proceed as follows. First replace the second equation in (7) by (first equation in (7)) second equation in (7). This yields the following linear system: x x (7.) x Then multiply the second equation in (7.) by, yielding the system x x (7.) x Finally, replace the first equation in (7.) by [second equation in (7.)] first equation in (7.). This yields the system The Gauss Jordan Method for Solving Systems of Linear Equations

14 x (7.) x System (7.) has the unique solution x and x. The systems (7), (7.), (7.), and (7.) are equivalent in that they have the same set of solutions. This means that x and x is also the unique solution to the original system, (7). If we view (7) in the augmented matrix form (Ab), we see that the steps used to solve (7) may be seen as Type and Type EROs applied to Ab. Begin with the augmented matrix version of (7): (7) Now perform a Type ERO by replacing row of (7) by (row of (7)) row of (7). The result is (7.) which corresponds to (7.). Next, we multiply row of (7.) by (a Type ERO), resulting in (7.) which corresponds to (7.). Finally, perform a Type ERO by replacing row of (7.) by (row of (7.)) row of (7.). The result is (7.) which corresponds to (7.). Translating (7.) back into a linear system, we obtain the system x and x, which is identical to (7.). 7 Finding a Solution by the Gauss Jordan Method The discussion in the previous section indicates that if the matrix Ab is obtained from Ab via an ERO, the systems Ax b and Ax b are equivalent. Thus, any sequence of EROs performed on the augmented matrix Ab corresponding to the system Ax b will yield an equivalent linear system. The Gauss Jordan method solves a linear equation system by utilizing EROs in a systematic fashion. We illustrate the method by finding the solution to the following linear system: x x x 9 x x x 6 (8) x x x The augmented matrix representation is Ab 6 (8) Suppose that by performing a sequence of EROs on (8) we could transform (8) into 9 CHAPTER Basic Linear Algebra

15 (9) We note that the result obtained by performing an ERO on a system of equations can also be obtained by multiplying both sides of the matrix representation of the system of equations by a particular matrix. This explains why EROs do not change the set of solutions to a system of equations. Matrix (9) corresponds to the following linear system: x x (9) x System (9) has the unique solution x, x, x. Because (9) was obtained from (8) by a sequence of EROs, we know that (8) and (9) are equivalent linear systems. Thus, x, x, x must also be the unique solution to (8). We now show how we can use EROs to transform a relatively complicated system such as (8) into a relatively simple system like (9). This is the essence of the Gauss Jordan method. We begin by using EROs to transform the first column of (8) into Then we use EROs to transform the second column of the resulting matrix into Finally, we use EROs to transform the third column of the resulting matrix into As a final result, we will have obtained (9). We now use the Gauss Jordan method to solve (8). We begin by using a Type ERO to change the element of (8) in the first row and first column into a. Then we add multiples of row to row and then to row (these are Type EROs). The purpose of these Type EROs is to put zeros in the rest of the first column. The following sequence of EROs will accomplish these goals. Step Multiply row of (8) by. This Type ERO yields A b 9 6 Step Replace row of A b by (row of A b ) row of A b. The result of this Type ERO is A b 9. The Gauss Jordan Method for Solving Systems of Linear Equations

16 Step Replace row of A b by (row of A b row of A b. The result of this Type ERO is 9 A b The first column of (8) has now been transformed into By our procedure, we have made sure that the variable x occurs in only a single equation and in that equation has a coefficient of. We now transform the second column of A b into We begin by using a Type ERO to create a in row and column of A b. Then we use the resulting row to perform the Type EROs that are needed to put zeros in the rest of column. Steps 6 accomplish these goals. Step Multiply row of A b by.the result of this Type ERO is 9 A b Step Replace row of A b by (row of A b ) row of A b. The result of this Type ERO is A b 7 Step 6 Replace row of A b by (row of A b ) row of A b. The result of this Type ERO is 7 6 A 6 b 6 6 Column has now been transformed into Observe that our transformation of column did not change column. To complete the Gauss Jordan procedure, we must transform the third column of A 6 b 6 into 6 6 CHAPTER Basic Linear Algebra

17 We first use a Type ERO to create a in the third row and third column of A 6 b 6. Then we use Type EROs to put zeros in the rest of column. Steps 7 9 accomplish these goals. Step 7 Multiply row of A 6 b 6 by 6. The result of this Type ERO is A 7 b 7 7 Step 8 Replace row of A 7 b 7 by (row 6 of A 7b 7 ) row of A 7 b 7. The result of this Type ERO is A 8 b 8 Step 9 Replace row of A 8 b 8 by (row of A 8b 8 ) row of A 8 b 8. The result of this Type ERO is A 9 b 9 A 9 b 9 represents the system of equations x x x x x x (9) x x x Thus, (9) has the unique solution x, x, x. Because (9) was obtained from (8) via EROs, the unique solution to (8) must also be x, x, x. The reader might be wondering why we defined Type EROs (interchanging of rows). To see why a Type ERO might be useful, suppose you want to solve x x 6 x x x () x x x To solve () by the Gauss Jordan method, first form the augmented matrix Ab The in row and column means that a Type ERO cannot be used to create a in row and column. If, however, we interchange rows and (a Type ERO), we obtain 6 () Now we may proceed as usual with the Gauss Jordan method The Gauss Jordan Method for Solving Systems of Linear Equations 7

18 Special Cases: No Solution or an Infinite Number of Solutions Some linear systems have no solution, and some have an infinite number of solutions. The following two examples illustrate how the Gauss Jordan method can be used to recognize these cases. EXAMPLE 6 Solution Linear System with No Solution Find all solutions to the following linear system: x x x x We apply the Gauss Jordan method to the matrix Ab We begin by replacing row of Ab by (row of Ab) row of Ab. The result of this Type ERO is () We would now like to transform the second column of () into but this is not possible. System () is equivalent to the following system of equations: x x () x x Whatever values we give to x and x, the second equation in () can never be satisfied. Thus, () has no solution. Because () was obtained from () by use of EROs, () also has no solution. () Example 6 illustrates the following idea: If you apply the Gauss Jordan method to a linear system and obtain a row of the form [ c] (c ), then the original linear system has no solution. EXAMPLE 7 Linear System with Infinite Number of Solutions Solution Apply the Gauss Jordan method to the following linear system: x x x x () The augmented matrix form of () is 8 CHAPTER Basic Linear Algebra x x x Ab

19 We begin by replacing row (because the row, column value is already ) of Ab by (row of Ab) row of Ab. The result of this Type ERO is A b () Next we replace row of A b by (row of A b ) row of A b. The result of this Type ERO is A b Now we replace row of A b by (row of A b ) row of A b. The result of this Type ERO is A b We would now like to transform the third column of A b into but this is not possible. The linear system corresponding to A b is x x x (.) x x x (.) x x x (.) Suppose we assign an arbitrary value k to x. Then (.) will be satisfied if x k, or x k. Similarly, (.) will be satisfied if x k, or x k. Of course, (.) will be satisfied for any values of x, x, and x. Thus, for any number k, x k, x k, x k is a solution to (). Thus, () has an infinite number of solutions (one for each number k). Because () was obtained from () via EROs, () also has an infinite number of solutions. A more formal characterization of linear systems that have an infinite number of solutions will be given after the following summary of the Gauss Jordan method. Summary of the Gauss Jordan Method Step To solve Ax b, write down the augmented matrix Ab. Step At any stage, define a current row, current column, and current entry (the entry in the current row and column). Begin with row as the current row, column as the current column, and a as the current entry. (a) If a (the current entry) is nonzero, then use EROs to transform column (the current column) to. The Gauss Jordan Method for Solving Systems of Linear Equations 9

20 Then obtain the new current row, column, and entry by moving down one row and one column to the right, and go to step. (b) If a (the current entry) equals, then do a Type ERO involving the current row and any row that contains a nonzero number in the current column. Use EROs to transform column to Then obtain the new current row, column, and entry by moving down one row and one column to the right. Go to step. (c) If there are no nonzero numbers in the first column, then obtain a new current column and entry by moving one column to the right. Then go to step. Step (a) If the new current entry is nonzero, then use EROs to transform it to and the rest of the current column s entries to. When finished, obtain the new current row, column, and entry. If this is impossible, then stop. Otherwise, repeat step. (b) If the current entry is, then do a Type ERO with the current row and any row that contains a nonzero number in the current column. Then use EROs to transform that current entry to and the rest of the current column s entries to. When finished, obtain the new current row, column, and entry. If this is impossible, then stop. Otherwise, repeat step. (c) If the current column has no nonzero numbers below the current row, then obtain the new current column and entry, and repeat step. If it is impossible, then stop. This procedure may require passing over one or more columns without transforming them (see Problem 8). Step Write down the system of equations Ax b that corresponds to the matrix Ab obtained when step is completed. Then Ax b will have the same set of solutions as Ax b. Basic Variables and Solutions to Linear Equation Systems To describe the set of solutions to Ax b (and Ax b), we need to define the concepts of basic and nonbasic variables. DEFINITION After the Gauss Jordan method has been applied to any linear system, a variable that appears with a coefficient of in a single equation and a coefficient of in all other equations is called a basic variable (BV). Any variable that is not a basic variable is called a nonbasic variable (NBV). Let BV be the set of basic variables for Ax b and NBV be the set of nonbasic variables for Ax b. The character of the solutions to Ax b depends on which of the following cases occurs. Case Ax b has at least one row of form [ c] (c ). Then Ax b has no solution (recall Example 6). As an example of Case, suppose that when the Gauss Jordan method is applied to the system Ax b, the following matrix is obtained: CHAPTER Basic Linear Algebra

21 Ab In this case, Ax b (and Ax b) has no solution. Case Suppose that Case does not apply and NBV, the set of nonbasic variables, is empty. Then Ax b (and Ax b) will have a unique solution. To illustrate this, we recall that in solving x x x 9 x x x 6 x x x the Gauss Jordan method yielded Ab In this case, BV {x, x, x } and NBV is empty. Then the unique solution to Ax b (and Ax b) is x, x, x. Case Suppose that Case does not apply and NBV is nonempty. Then Ax b (and Ax b) will have an infinite number of solutions. To obtain these, first assign each nonbasic variable an arbitrary value. Then solve for the value of each basic variable in terms of the nonbasic variables. For example, suppose Ab () Because Case does not apply, and BV {x, x, x } and NBV {x, x }, we have an example of Case : Ax b (and Ax b) will have an infinite number of solutions. To see what these solutions look like, write down Ax b: x x x x x (.) x x x x x (.) x x x x x (.) x x x x x (.) Now assign the nonbasic variables (x and x ) arbitrary values c and k, with x c and x k. From (.), we find that x c k. From (.), we find that x c. From (.), we find that x k. Because (.) holds for all values of the variables, x c k, x c, x k, x c, and x k will, for any values of c and k, be a solution to Ax b (and Ax b). Our discussion of the Gauss Jordan method is summarized in Figure 6. We have devoted so much time to the Gauss Jordan method because, in our study of linear programming, examples of Case (linear systems with an infinite number of solutions) will occur repeatedly. Because the end result of the Gauss Jordan method must always be one of Cases, we have shown that any linear system will have no solution, a unique solution, or an infinite number of solutions.. The Gauss Jordan Method for Solving Systems of Linear Equations

22 Does A b have a row [ c] (c )? YES NO Ax = b has no solution Find BV and NBV FIGURE 6 Description of Gauss Jordan Method for Solving Linear Equations YES Ax = b has a unique solution Is NBV empty? NO Ax = b has an infinite number of solutions PROBLEMS Group A Use the Gauss Jordan method to determine whether each of the following linear systems has no solution, a unique solution, or an infinite number of solutions. Indicate the solutions (if any exist). x x x x x x x x x x x x 8 x x x x x x 6 x x x x x x x x x x 6 x x x x x x x x x x x x x x x.x x x x 6 x x x x x x xx x x 7 x x x x x x x x x 8 x x x x x x x x x x x x Group B 9 Suppose that a linear system Ax b has more variables than equations. Show that Ax b cannot have a unique solution.. Linear Independence and Linear Dependence In this section, we discuss the concepts of a linearly independent set of vectors, a linearly dependent set of vectors, and the rank of a matrix. These concepts will be useful in our study of matrix inverses. Before defining a linearly independent set of vectors, we need to define a linear combination of a set of vectors. Let V {v, v,..., v k } be a set of row vectors all of which have the same dimension. This section covers topics that may be omitted with no loss of continuity. CHAPTER Basic Linear Algebra

23 DEFINITION A linear combination of the vectors in V is any vector of the form c v c v c k v k, where c, c,..., c k are arbitrary scalars. For example, if V {[ ], [ ]}, then v v ([ ]) [ ] [ ] v v [ ] ([ ]) [7 ] v v [ ] ([ ]) [6 ] are linear combinations of vectors in V. The foregoing definition may also be applied to a set of column vectors. Suppose we are given a set V {v, v,..., v k } of m-dimensional row vectors. Let [ ] be the m-dimensional vector. To determine whether V is a linearly independent set of vectors, we try to find a linear combination of the vectors in V that adds up to. Clearly, v v v k is a linear combination of vectors in V that adds up to. We call the linear combination of vectors in V for which c c c k the trivial linear combination of vectors in V. We may now define linearly independent and linearly dependent sets of vectors. DEFINITION A set V of m-dimensional vectors is linearly independent if the only linear combination of vectors in V that equals is the trivial linear combination. A set V of m-dimensional vectors is linearly dependent if there is a nontrivial linear combination of the vectors in V that adds up to. The following examples should clarify these definitions. EXAMPLE 8 Vector Makes Set LD Show that any set of vectors containing the vector is a linearly dependent set. Solution To illustrate, we show that if V {[ ], [ ], [ ]}, then V is linearly dependent, because if, say, c, then c ([ ]) ([ ]) ([ ]) [ ]. Thus, there is a nontrivial linear combination of vectors in V that adds up to. EXAMPLE 9 LI Set of Vectors Show that the set of vectors V {[ ], [ ]} is a linearly independent set of vectors. Solution EXAMPLE We try to find a nontrivial linear combination of the vectors in V that yields. This requires that we find scalars c and c (at least one of which is nonzero) satisfying c ([ ]) c ([ ]) [ ]. Thus, c and c must satisfy [c c ] [ ]. This implies c c. The only linear combination of vectors in V that yields is the trivial linear combination. Therefore, V is a linearly independent set of vectors. LD Set of Vectors Show that V {[ ], [ ]} is a linearly dependent set of vectors. Solution Because ([ ]) ([ ]) [ ], there is a nontrivial linear combination with c and c that yields. Thus, V is a linearly dependent set of vectors. Intuitively, what does it mean for a set of vectors to be linearly dependent? To understand the concept of linear dependence, observe that a set of vectors V is linearly dependent (as. Linear Independence and Linear Dependence

24 x x v = v = = AB = AC v = v = C FIGURE 7 (a) Two Linearly Dependent Vectors (b) Two Linearly Independent Vectors a A v B v x b v v x long as is not in V ) if and only if some vector in V can be written as a nontrivial linear combination of other vectors in V (see Problem 9 at the end of this section). For instance, in Example, [ ] ([ ]). Thus, if a set of vectors V is linearly dependent, the vectors in V are, in some way, not all different vectors. By different we mean that the direction specified by any vector in V cannot be expressed by adding together multiples of other vectors in V. For example, in two dimensions it can be shown that two vectors are linearly dependent if and only if they lie on the same line (see Figure 7). The Rank of a Matrix The Gauss Jordan method can be used to determine whether a set of vectors is linearly independent or linearly dependent. Before describing how this is done, we define the concept of the rank of a matrix. Let A be any m n matrix, and denote the rows of A by r, r,..., r m. Also define R {r, r,..., r m }. DEFINITION The rank of A is the number of vectors in the largest linearly independent subset of R. The following three examples illustrate the concept of rank. EXAMPLE Matrix with Rank Show that rank A for the following matrix: A Solution For the set of vectors R {[ ], [, ]}, it is impossible to choose a subset of R that is linearly independent (recall Example 8). EXAMPLE Matrix with Rank of Show that rank A for the following matrix: A CHAPTER Basic Linear Algebra

25 Solution Here R {[ ], [ ]}. The set {[ ]} is a linearly independent subset of R, so rank A must be at least. If we try to find two linearly independent vectors in R, we fail because ([ ]) [ ] [ ]. This means that rank A cannot be. Thus, rank A must equal. EXAMPLE Matrix with Rank of Show that rank A for the following matrix: A Solution Here R {[ ], [ ]}. From Example 9, we know that R is a linearly independent set of vectors. Thus, rank A. To find the rank of a given matrix A, simply apply the Gauss Jordan method to the matrix A. Let the final result be the matrix A. It can be shown that performing a sequence of EROs on a matrix does not change the rank of the matrix. This implies that rank A rank A. It is also apparent that the rank of A will be the number of nonzero rows in A. Combining these facts, we find that rank A rank A number of nonzero rows in A. EXAMPLE Solution Using Gauss Jordan Method to Find Rank of Matrix Find The Gauss Jordan method yields the following sequence of matrices: A A Thus, rank A rank A. rank A How to Tell Whether a Set of Vectors Is Linearly Independent We now describe a method for determining whether a set of vectors V {v, v,..., v m } is linearly independent. Form the matrix A whose ith row is v i. A will have m rows. If rank A m, then V is a linearly independent set of vectors, whereas if rank A m, then V is a linearly dependent set of vectors. EXAMPLE A Linearly Dependent Set of Vectors Determine whether V {[ ], [ ], [ ]} is a linearly independent set of vectors.. Linear Independence and Linear Dependence

26 Solution The Gauss Jordan method yields the following sequence of matrices: A Thus, rank A rank A. This shows that V is a linearly dependent set of vectors. In fact, the EROs used to transform A to A can be used to show that [ ] [ ] [ ]. This equation also shows that V is a linearly dependent set of vectors. A PROBLEMS Group A Determine if each of the following sets of vectors is linearly independent or linearly dependent. V {[ ], [ ], [ ]} V {[ ], [ ], [ ]} V {[ ], [ ]} V {[ ], [ ]} V,, 7 6 V,, 6 9 Group B 7 Show that the linear system Ax b has a solution if and only if b can be written as a linear combination of the columns of A. 8 Suppose there is a collection of three or more twodimensional vectors. Provide an argument showing that the collection must be linearly dependent. 9 Show that a set of vectors V (not containing the vector) is linearly dependent if and only if there exists some vector in V that can be written as a nontrivial linear combination of other vectors in V.. The Inverse of a Matrix To solve a single linear equation such as x, we simply multiply both sides of the equation by the multiplicative inverse of, which is, or. This yields (x) ( ), or x. (Of course, this method fails to work for the equation x, because zero has no multiplicative inverse.) In this section, we develop a generalization of this technique that can be used to solve square (number of equations number of unknowns) linear systems. We begin with some preliminary definitions. DEFINITION A square matrix is any matrix that has an equal number of rows and columns. The diagonal elements of a square matrix are those elements a ij such that i j. A square matrix for which all diagonal elements are equal to and all nondiagonal elements are equal to is called an identity matrix. The m m identity matrix will be written as I m. Thus, I, I, 6 CHAPTER Basic Linear Algebra

27 If the multiplications I m A and AI m are defined, it is easy to show that I m A AI m A. Thus, just as the number serves as the unit element for multiplication of real numbers, I m serves as the unit element for multiplication of matrices. Recall that is the multiplicative inverse of. This is because ( ) ( ). This motivates the following definition of the inverse of a matrix. DEFINITION For a given m m matrix A, the m m matrix B is the inverse of A if BA AB I m (6) (It can be shown that if BA I m or AB I m, then the other quantity will also equal I m.) Some square matrices do not have inverses. If there does exist an m m matrix B that satisfies Equation (6), then we write B A. For example, if the reader can verify that and Thus, A 7 7 A To see why we are interested in the concept of a matrix inverse, suppose we want to solve a linear system Ax b that has m equations and m unknowns. Suppose that A exists. Multiplying both sides of Ax b by A, we see that any solution of Ax b must also satisfy A (Ax) A b. Using the associative law and the definition of a matrix inverse, we obtain (A A)x A b or I m x A b or I m x A b This shows that knowing A enables us to find the unique solution to a square linear system. This is the analog of solving x by multiplying both sides of the equation by. The Gauss Jordan method may be used to find A (or to show that A does not exist). To illustrate how we can use the Gauss Jordan method to invert a matrix, suppose we want to find A for A 7. The Inverse of a Matrix 7

28 This requires that we find a matrix that satisfies A a b (7) From Equation (7), we obtain the following pair of simultaneous equations that must be satisfied by a, b, c, and d: Thus, to find a ; b a (the first column of A ), we can apply the Gauss Jordan method to the augmented matrix Once EROs have transformed to I, will have been transformed into the first column of A. To determine b (the second column of A ), we apply EROs to the augmented matrix When has been transformed into I, c a c c b d will have been transformed into the second column of A. Thus, to find each column of A, we must perform a sequence of EROs that transform d c d d 8 CHAPTER Basic Linear Algebra

29 into I. This suggests that we can find A by applying EROs to the matrix When has been transformed to I, AI will have been transformed into the first column of A, and will have been transformed into the second column of A. Thus, as A is transformed into I, I is transformed into A. The computations to determine A follow. Step Step Step Step Multiply row of AI by. This yields AI Replace row of AI by (row of AI ) row of AI. This yields AI Multiply row of AI by. This yields AI Replace row of AI by (row of AI ) row of AI. This yields Because A has been transformed into I, I will have been transformed into A. Hence, A The reader should verify that AA A A I. A Matrix May Not Have an Inverse Some matrices do not have inverses. To illustrate, let A and A (8) e g f h. The Inverse of a Matrix 9

30 To find A we must solve the following pair of simultaneous equations: e (8.) f (8.) When we try to solve (8.) by the Gauss Jordan method, we find that is transformed into g h This indicates that (8.) has no solution, and A cannot exist. Observe that (8.) fails to have a solution, because the Gauss Jordan method transforms A into a matrix with a row of zeros on the bottom. This can only happen if rank A. If m m matrix A has rank A m, then A will not exist. The Gauss Jordan Method for Inverting an m m Matrix A Step Write down the m m matrix AI m. Step Use EROs to transform AI m into I m B. This will be possible only if rank A m. In this case, B A. If rank A m, then A has no inverse. Using Matrix Inverses to Solve Linear Systems As previously stated, matrix inverses can be used to solve a linear system Ax b in which the number of variables and equations are equal. Simply multiply both sides of Ax b by A to obtain the solution x A b. For example, to solve x x 7 (9) x x write the matrix representation of (9): x Let A We found in the previous illustration that A x 7 () CHAPTER Basic Linear Algebra

31 FIGURE A B C D E F G H Inverting a Matrix - A - A Multiplying both sides of () by A, we obtain x 7 x x Thus, x, x is the unique solution to system (9). x Inverting Matrices with Excel Minverse.xls The Excel MINVERSE command makes it easy to invert a matrix. See Figure 8 and file Minverse.xls. Suppose we want to invert the matrix A Simply enter the matrix in E:G and select the range (we chose E7:G9) where you want A to be computed. In the upper left-hand corner of the range E7:G9 (cell E7), we enter the formula MINVERSE(E:G) and select Control Shift Enter. This enters an array function that computes A in the range E7:G9. You cannot edit part of an array function, so if you want to delete A, you must delete the entire range where A is present. PROBLEMS Group A Find A (if it exists) for the following matrices: Use the answer to Problem to solve the following linear system: x x x x 7 6 Use the answer to Problem to solve the following linear system: x x x x x x x x x Group B 7 Show that a square matrix has an inverse if and only if its rows form a linearly independent set of vectors. 8 Consider a square matrix B whose inverse is given by B. a In terms of B, what is the inverse of the matrix B?. The Inverse of a Matrix

32 b Let B be the matrix obtained from B by doubling every entry in row of B. Explain how we could obtain the inverse of B from B. c Let B be the matrix obtained from B by doubling every entry in column of B. Explain how we could obtain the inverse of B from B. 9 Suppose that A and B both have inverses. Find the inverse of the matrix AB. Suppose A has an inverse. Show that (A T ) (A ) T. (Hint: Use the fact that AA I, and take the transpose of both sides.) A square matrix A is orthogonal if AA T I. What properties must be possessed by the columns of an orthogonal matrix?.6 Determinants Associated with any square matrix A is a number called the determinant of A (often abbreviated as det A or A). Knowing how to compute the determinant of a square matrix will be useful in our study of nonlinear programming. For a matrix A [a ], For a matrix det A a () A () det A a a a a For example, det () () Before we learn how to compute det A for larger square matrices, we need to define the concept of the minor of a matrix. DEFINITION If A is an m m matrix, then for any values of i and j, the ijth minor of A (written A ij ) is the (m ) (m ) submatrix of A obtained by deleting row i and column j of A. For example, if A, then A 6 6 and A 7 8 Let A be any m m matrix. We may write A as 9 A a a a m a a a a a m a n To compute det A, pick any value of i (i,,..., m) and compute det A: det A () i a i (det A i ) () i a i (det A i ) () im a im (det A im ) () a a 7 9 a n a mn 6 CHAPTER Basic Linear Algebra

33 Formula () is called the expansion of det A by the cofactors of row i. The virtue of () is that it reduces the computation of det A for an m m matrix to computations involving only (m ) (m ) matrices. Apply () until det A can be expressed in terms of matrices. Then use Equation () to find the determinants of the relevant matrices. To illustrate the use of (), we find det A for A 7 We expand det A by using row cofactors. Notice that a, a, and a. Also 6 A 8 9 so by (), det A (9) 8(6) ; 6 A 7 9 so by (), det A (9) 7(6) 6; and A 7 8 so by (), det A (8) 7(). Then by (), det A () a (det A ) () a (det A ) () a (det A ) ()()() ()()(6) ()()() 9 The interested reader may verify that expansion of det A by either row or row cofactors also yields det A. We close our discussion of determinants by noting that they can be used to invert square matrices and to solve linear equation systems. Because we already have learned to use the Gauss Jordan method to invert matrices and to solve linear equation systems, we will not discuss these uses of determinants PROBLEMS Group A Verify that det row and row cofactors. Find det by using expansions by A matrix is said to be upper triangular if for i j, a ij. Show that the determinant of any upper triangular matrix is equal to the product of the matrix s diagonal elements. (This result is true for any upper triangular matrix.) Group B a Show that for any and matrix, det A det A. b Show that for any and matrix, det A det A. c Generalize the results of parts (a) and (b)..6 Determinants

34 SUMMARY Matrices A matrix is any rectangular array of numbers. For the matrix A, we let a ij represent the element of A in row i and column j. A matrix with only one row or one column may be thought of as a vector. Vectors appear in boldface type (v). Given a row vector u [u u u n ] and a column v v v n of the same dimension, the scalar product of u and v (written u v) is the number u v u v u n v n. Given two matrices A and B, the matrix product of A and B (written AB) is defined if and only if the number of columns in A the number of rows in B. Suppose this is the case and A has m rows and B has n columns. Then the matrix product C AB of A and B is the m n matrix C whose ijth element is determined as follows: The ijth element of C the scalar product of row i of A with column j of B. v Matrices and Linear Equations The linear equation system a x a x a n x n b may be written as Ax b or Ab, where A a a a m a x a x a n x n b a m x a m x a mn x n b m a a a m a n a n a mn, x, x x b x n b b b m The Gauss Jordan Method Using elementary row operations (EROs), we may solve any linear equation system. From a matrix A, an ERO yields a new matrix A via one of three procedures. Type ERO Obtain A by multiplying any row of A by a nonzero scalar. Type ERO Multiply any row of A (say, row i) by a nonzero scalar c. For some j i, let row j of A c(row i of A) row j of A, and let the other rows of A be the same as the rows of A. CHAPTER Basic Linear Algebra

35 Type ERO Interchange any two rows of A. The Gauss Jordan method uses EROs to solve linear equation systems, as shown in the following steps. Step To solve Ax b, write down the augmented matrix Ab. Step Begin with row as the current row, column as the current column, and a as the current entry. (a) If a (the current entry) is nonzero, then use EROs to transform column (the current column) to Then obtain the new current row, column, and entry by moving down one row and one column to the right, and go to step. (b) If a (the current entry) equals, then do a Type ERO switch with any row with a nonzero value in the same column. Use EROs to transform column to and proceed to step after moving into a new current row, column, and entry. (c) If there are no nonzero numbers in the first column, then proceed to a new current column and entry. Then go to step. Step (a) If the current entry is nonzero, use EROs to transform it to and the rest of the current column s entries to. Obtain the new current row, column, and entry. If this is impossible, then stop. Otherwise, repeat step. (b) If the current entry is, then do a Type ERO switch with any row with a nonzero value in the same column. Transform the column using EROs and move to the next current entry. If this is impossible, then stop. Otherwise, repeat step. (c) If the current column has no nonzero numbers below the current row, then obtain the new current column and entry, and repeat step. If it is impossible, then stop. This procedure may require passing over one or more columns without transforming them. Step Write down the system of equations Ax b that corresponds to the matrix Ab obtained when step is completed. Then Ax b will have the same set of solutions as Ax b. To describe the set of solutions to Ax b (and Ax b), we define the concepts of basic and nonbasic variables. After the Gauss Jordan method has been applied to any linear system, a variable that appears with a coefficient of in a single equation and a coefficient of in all other equations is called a basic variable. Any variable that is not a basic variable is called a nonbasic variable.. Summary

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### Matrix Algebra and Applications

Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2 - Matrices and Matrix Algebra Reading 1 Chapters

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### Helpsheet. Giblin Eunson Library MATRIX ALGEBRA. library.unimelb.edu.au/libraries/bee. Use this sheet to help you:

Helpsheet Giblin Eunson Library ATRIX ALGEBRA Use this sheet to help you: Understand the basic concepts and definitions of matrix algebra Express a set of linear equations in matrix notation Evaluate determinants

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### Introduction to Matrix Algebra I

Appendix A Introduction to Matrix Algebra I Today we will begin the course with a discussion of matrix algebra. Why are we studying this? We will use matrix algebra to derive the linear regression model

### The Inverse of a Matrix

The Inverse of a Matrix 7.4 Introduction In number arithmetic every number a ( 0) has a reciprocal b written as a or such that a ba = ab =. Some, but not all, square matrices have inverses. If a square

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89. by Joseph Collison

SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89 by Joseph Collison Copyright 2000 by Joseph Collison All rights reserved Reproduction or translation of any part of this work beyond that permitted by Sections

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Matrices, Determinants and Linear Systems

September 21, 2014 Matrices A matrix A m n is an array of numbers in rows and columns a 11 a 12 a 1n r 1 a 21 a 22 a 2n r 2....... a m1 a m2 a mn r m c 1 c 2 c n We say that the dimension of A is m n (we

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Solving Systems of Linear Equations. Substitution

Solving Systems of Linear Equations There are two basic methods we will use to solve systems of linear equations: Substitution Elimination We will describe each for a system of two equations in two unknowns,

### 4.5 Linear Dependence and Linear Independence

4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then

### 13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.

3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### Matrix Algebra. Some Basic Matrix Laws. Before reading the text or the following notes glance at the following list of basic matrix algebra laws.

Matrix Algebra A. Doerr Before reading the text or the following notes glance at the following list of basic matrix algebra laws. Some Basic Matrix Laws Assume the orders of the matrices are such that

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

### 2.5 Elementary Row Operations and the Determinant

2.5 Elementary Row Operations and the Determinant Recall: Let A be a 2 2 matrtix : A = a b. The determinant of A, denoted by det(a) c d or A, is the number ad bc. So for example if A = 2 4, det(a) = 2(5)

### 2.1: MATRIX OPERATIONS

.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and

### EC9A0: Pre-sessional Advanced Mathematics Course

University of Warwick, EC9A0: Pre-sessional Advanced Mathematics Course Peter J. Hammond & Pablo F. Beker 1 of 55 EC9A0: Pre-sessional Advanced Mathematics Course Slides 1: Matrix Algebra Peter J. Hammond

### Typical Linear Equation Set and Corresponding Matrices

EWE: Engineering With Excel Larsen Page 1 4. Matrix Operations in Excel. Matrix Manipulations: Vectors, Matrices, and Arrays. How Excel Handles Matrix Math. Basic Matrix Operations. Solving Systems of

### Introduction to Matrices for Engineers

Introduction to Matrices for Engineers C.T.J. Dodson, School of Mathematics, Manchester Universit 1 What is a Matrix? A matrix is a rectangular arra of elements, usuall numbers, e.g. 1 0-8 4 0-1 1 0 11

### B such that AB = I and BA = I. (We say B is an inverse of A.) Definition A square matrix A is invertible (or nonsingular) if matrix

Matrix inverses Recall... Definition A square matrix A is invertible (or nonsingular) if matrix B such that AB = and BA =. (We say B is an inverse of A.) Remark Not all square matrices are invertible.

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### We know a formula for and some properties of the determinant. Now we see how the determinant can be used.

Cramer s rule, inverse matrix, and volume We know a formula for and some properties of the determinant. Now we see how the determinant can be used. Formula for A We know: a b d b =. c d ad bc c a Can we

### 5.3 The Cross Product in R 3

53 The Cross Product in R 3 Definition 531 Let u = [u 1, u 2, u 3 ] and v = [v 1, v 2, v 3 ] Then the vector given by [u 2 v 3 u 3 v 2, u 3 v 1 u 1 v 3, u 1 v 2 u 2 v 1 ] is called the cross product (or

### MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.

MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Solutions to Math 51 First Exam January 29, 2015

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

### Name: Section Registered In:

Name: Section Registered In: Math 125 Exam 3 Version 1 April 24, 2006 60 total points possible 1. (5pts) Use Cramer s Rule to solve 3x + 4y = 30 x 2y = 8. Be sure to show enough detail that shows you are

### 1 Sets and Set Notation.

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

### L1-2. Special Matrix Operations: Permutations, Transpose, Inverse, Augmentation 12 Aug 2014

L1-2. Special Matrix Operations: Permutations, Transpose, Inverse, Augmentation 12 Aug 2014 Unfortunately, no one can be told what the Matrix is. You have to see it for yourself. -- Morpheus Primary concepts:

### INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL

SOLUTIONS OF THEORETICAL EXERCISES selected from INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL Eighth Edition, Prentice Hall, 2005. Dr. Grigore CĂLUGĂREANU Department of Mathematics

### MATH2210 Notebook 1 Fall Semester 2016/2017. 1 MATH2210 Notebook 1 3. 1.1 Solving Systems of Linear Equations... 3

MATH0 Notebook Fall Semester 06/07 prepared by Professor Jenny Baglivo c Copyright 009 07 by Jenny A. Baglivo. All Rights Reserved. Contents MATH0 Notebook 3. Solving Systems of Linear Equations........................

### Lecture L3 - Vectors, Matrices and Coordinate Transformations

S. Widnall 16.07 Dynamics Fall 2009 Lecture notes based on J. Peraire Version 2.0 Lecture L3 - Vectors, Matrices and Coordinate Transformations By using vectors and defining appropriate operations between

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### 4. Matrix inverses. left and right inverse. linear independence. nonsingular matrices. matrices with linearly independent columns

L. Vandenberghe EE133A (Spring 2016) 4. Matrix inverses left and right inverse linear independence nonsingular matrices matrices with linearly independent columns matrices with linearly independent rows

### Linear Equations ! 25 30 35\$ & " 350 150% & " 11,750 12,750 13,750% MATHEMATICS LEARNING SERVICE Centre for Learning and Professional Development

MathsTrack (NOTE Feb 2013: This is the old version of MathsTrack. New books will be created during 2013 and 2014) Topic 4 Module 9 Introduction Systems of to Matrices Linear Equations Income = Tickets!

### Solving a System of Equations

11 Solving a System of Equations 11-1 Introduction The previous chapter has shown how to solve an algebraic equation with one variable. However, sometimes there is more than one unknown that must be determined

### Matrix Representations of Linear Transformations and Changes of Coordinates

Matrix Representations of Linear Transformations and Changes of Coordinates 01 Subspaces and Bases 011 Definitions A subspace V of R n is a subset of R n that contains the zero element and is closed under

### University of Lille I PC first year list of exercises n 7. Review

University of Lille I PC first year list of exercises n 7 Review Exercise Solve the following systems in 4 different ways (by substitution, by the Gauss method, by inverting the matrix of coefficients

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### Section 5.3. Section 5.3. u m ] l jj. = l jj u j + + l mj u m. v j = [ u 1 u j. l mj

Section 5. l j v j = [ u u j u m ] l jj = l jj u j + + l mj u m. l mj Section 5. 5.. Not orthogonal, the column vectors fail to be perpendicular to each other. 5..2 his matrix is orthogonal. Check that

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### SECTION 8.3: THE INVERSE OF A SQUARE MATRIX

(Section 8.3: The Inverse of a Square Matrix) 8.47 SECTION 8.3: THE INVERSE OF A SQUARE MATRIX PART A: (REVIEW) THE INVERSE OF A REAL NUMBER If a is a nonzero real number, then aa 1 = a 1 a = 1. a 1, or

### Images and Kernels in Linear Algebra By Kristi Hoshibata Mathematics 232

Images and Kernels in Linear Algebra By Kristi Hoshibata Mathematics 232 In mathematics, there are many different fields of study, including calculus, geometry, algebra and others. Mathematics has been

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### SYSTEMS OF EQUATIONS

SYSTEMS OF EQUATIONS 1. Examples of systems of equations Here are some examples of systems of equations. Each system has a number of equations and a number (not necessarily the same) of variables for which

### The Determinant: a Means to Calculate Volume

The Determinant: a Means to Calculate Volume Bo Peng August 20, 2007 Abstract This paper gives a definition of the determinant and lists many of its well-known properties Volumes of parallelepipeds are

### Excel supplement: Chapter 7 Matrix and vector algebra

Excel supplement: Chapter 7 atrix and vector algebra any models in economics lead to large systems of linear equations. These problems are particularly suited for computers. The main purpose of this chapter

### Using row reduction to calculate the inverse and the determinant of a square matrix

Using row reduction to calculate the inverse and the determinant of a square matrix Notes for MATH 0290 Honors by Prof. Anna Vainchtein 1 Inverse of a square matrix An n n square matrix A is called invertible

### SECTION 8-1 Systems of Linear Equations and Augmented Matrices

86 8 Systems of Equations and Inequalities In this chapter we move from the standard methods of solving two linear equations with two variables to a method that can be used to solve linear systems with

### MAT 200, Midterm Exam Solution. a. (5 points) Compute the determinant of the matrix A =

MAT 200, Midterm Exam Solution. (0 points total) a. (5 points) Compute the determinant of the matrix 2 2 0 A = 0 3 0 3 0 Answer: det A = 3. The most efficient way is to develop the determinant along the

### 1.2 Solving a System of Linear Equations

1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables

### Chapter 7. Matrices. Definition. An m n matrix is an array of numbers set out in m rows and n columns. Examples. ( 1 1 5 2 0 6

Chapter 7 Matrices Definition An m n matrix is an array of numbers set out in m rows and n columns Examples (i ( 1 1 5 2 0 6 has 2 rows and 3 columns and so it is a 2 3 matrix (ii 1 0 7 1 2 3 3 1 is a

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### 8 Square matrices continued: Determinants

8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You

### Solving simultaneous equations using the inverse matrix

Solving simultaneous equations using the inverse matrix 8.2 Introduction The power of matrix algebra is seen in the representation of a system of simultaneous linear equations as a matrix equation. Matrix

### Arithmetic and Algebra of Matrices

Arithmetic and Algebra of Matrices Math 572: Algebra for Middle School Teachers The University of Montana 1 The Real Numbers 2 Classroom Connection: Systems of Linear Equations 3 Rational Numbers 4 Irrational

### MATH10212 Linear Algebra B Homework 7

MATH22 Linear Algebra B Homework 7 Students are strongly advised to acquire a copy of the Textbook: D C Lay, Linear Algebra and its Applications Pearson, 26 (or other editions) Normally, homework assignments

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### Lecture 2 Matrix Operations

Lecture 2 Matrix Operations transpose, sum & difference, scalar multiplication matrix multiplication, matrix-vector product matrix inverse 2 1 Matrix transpose transpose of m n matrix A, denoted A T or

### Basic Terminology for Systems of Equations in a Nutshell. E. L. Lady. 3x 1 7x 2 +4x 3 =0 5x 1 +8x 2 12x 3 =0.

Basic Terminology for Systems of Equations in a Nutshell E L Lady A system of linear equations is something like the following: x 7x +4x =0 5x +8x x = Note that the number of equations is not required

### Lecture Notes 1: Matrix Algebra Part B: Determinants and Inverses

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 1 of 57 Lecture Notes 1: Matrix Algebra Part B: Determinants and Inverses Peter J. Hammond email: p.j.hammond@warwick.ac.uk Autumn 2012,

### Rotation Matrices and Homogeneous Transformations

Rotation Matrices and Homogeneous Transformations A coordinate frame in an n-dimensional space is defined by n mutually orthogonal unit vectors. In particular, for a two-dimensional (2D) space, i.e., n

### Linear Algebra: Vectors

A Linear Algebra: Vectors A Appendix A: LINEAR ALGEBRA: VECTORS TABLE OF CONTENTS Page A Motivation A 3 A2 Vectors A 3 A2 Notational Conventions A 4 A22 Visualization A 5 A23 Special Vectors A 5 A3 Vector

### 1 Eigenvalues and Eigenvectors

Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

### Diagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions

Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential

### Solving Systems of Linear Equations; Row Reduction

Harvey Mudd College Math Tutorial: Solving Systems of Linear Equations; Row Reduction Systems of linear equations arise in all sorts of applications in many different fields of study The method reviewed

### Lecture Notes: Matrix Inverse. 1 Inverse Definition

Lecture Notes: Matrix Inverse Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Inverse Definition We use I to represent identity matrices,

### 4.6 Null Space, Column Space, Row Space

NULL SPACE, COLUMN SPACE, ROW SPACE Null Space, Column Space, Row Space In applications of linear algebra, subspaces of R n typically arise in one of two situations: ) as the set of solutions of a linear

### Chapter 17. Orthogonal Matrices and Symmetries of Space

Chapter 17. Orthogonal Matrices and Symmetries of Space Take a random matrix, say 1 3 A = 4 5 6, 7 8 9 and compare the lengths of e 1 and Ae 1. The vector e 1 has length 1, while Ae 1 = (1, 4, 7) has length

### 9 MATRICES AND TRANSFORMATIONS

9 MATRICES AND TRANSFORMATIONS Chapter 9 Matrices and Transformations Objectives After studying this chapter you should be able to handle matrix (and vector) algebra with confidence, and understand the

### Linear Algebra Notes

Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note

### MA106 Linear Algebra lecture notes

MA106 Linear Algebra lecture notes Lecturers: Martin Bright and Daan Krammer Warwick, January 2011 Contents 1 Number systems and fields 3 1.1 Axioms for number systems......................... 3 2 Vector

### SECTION 9-1 Matrices: Basic Operations

9 Matrices and Determinants In this chapter we discuss matrices in more detail. In the first three sections we define and study some algebraic operations on matrices, including addition, multiplication,

### Lecture Notes 2: Matrices as Systems of Linear Equations

2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably

### Matrices Worksheet. Adding the results together, using the matrices, gives

Matrices Worksheet This worksheet is designed to help you increase your confidence in handling MATRICES. This worksheet contains both theory and exercises which cover. Introduction. Order, Addition and

### Matrix Differentiation

1 Introduction Matrix Differentiation ( and some other stuff ) Randal J. Barnes Department of Civil Engineering, University of Minnesota Minneapolis, Minnesota, USA Throughout this presentation I have

### 1 0 5 3 3 A = 0 0 0 1 3 0 0 0 0 0 0 0 0 0 0

Solutions: Assignment 4.. Find the redundant column vectors of the given matrix A by inspection. Then find a basis of the image of A and a basis of the kernel of A. 5 A The second and third columns are

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### Linear Algebra Review. Vectors

Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

### Situation 23: Simultaneous Equations Prepared at the University of Georgia EMAT 6500 class Date last revised: July 22 nd, 2013 Nicolina Scarpelli

Situation 23: Simultaneous Equations Prepared at the University of Georgia EMAT 6500 class Date last revised: July 22 nd, 2013 Nicolina Scarpelli Prompt: A mentor teacher and student teacher are discussing