# Matrix Calculations: Kernels & Images, Matrix Multiplication

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Matrix Calculations: Kernels & Images, Matrix Multiplication A. Kissinger (and H. Geuvers) Institute for Computing and Information Sciences Intelligent Systems Version: spring 2016 A. Kissinger Version: spring 2016 Matrix Calculations 1 / 43

2 Outline Matrix multiplication Matrix multiplication A. Kissinger Version: spring 2016 Matrix Calculations 2 / 43

3 From last time Matrix multiplication Vector spaces V, W,... are special kinds of sets whose elements are called vectors. Vectors can be added together, or multiplied by a real number, For v, w V, a R: v + w V a v V The simplest examples are: R n := {(a 1,..., a n ) a i R} Linear maps are special kinds of functions which satisfy two properties: f (v + w) = f (v) + f (w) f (a v) = a f (v) A. Kissinger Version: spring 2016 Matrix Calculations 3 / 43

4 From last time Matrix multiplication Whereas there exist LOTS of functions between the sets V and W......there actually aren t that many linear maps: Theorem For every linear map f : R n R m, there exists an m n matrix A where: f (v) = A v (where is the matrix multiplication of A and a vector v) More generally, every linear map f : V W is representable as a matrix, but you have to fix a basis for V and W first: {v 1,..., v m } V {w 1,..., w n } W...whereas in R n there is an obvious choice: {(1, 0,..., 0), (0, 1,..., 0),..., (0,..., 0, 1)} R n A. Kissinger Version: spring 2016 Matrix Calculations 4 / 43

5 Matrix-vector multiplication For a matrix A and a vector v, w := A v is the vector whose i-th row is the dot product of the i-th row of A with v: a 11 a 1n.. a m1 a mn v 1.. v n = a 11 v a 1n v n. a m1 v a mn v n i.e. w i := a 11 v a 1n v n = n a ij v j. j=1 A. Kissinger Version: spring 2016 Matrix Calculations 5 / 43

6 Example: systems of equations a 11 x a 1n x n = b 1.. a m1 x a mn x n = b m A x = b a 11 a 1n x 1.. a m1 a mn x n = b 1. b n a 11 x a 1n x n = 0... a m1 x a mn x n = 0 A x = 0 a 11 a 1n... a m1 a mn x 1. x n = 0. 0 A. Kissinger Version: spring 2016 Matrix Calculations 6 / 43

7 Matrix multiplication Consider linear maps g, f represented by matrices A, B: g(v) = A v f (w) = B w Can we find a matrix C that represents their composition? Let s try: g(f (v)) = C v g(f (v)) = g(b v) = A (B v) ( ) = (A B) v (where step ( ) is currently wishful thinking ) Great! Let C := A B. But we don t know what means for two matrices yet... A. Kissinger Version: spring 2016 Matrix Calculations 8 / 43

8 Matrix multiplication Solution: generalise from A v A vector is a matrix with one column: The number in the i-th row and the first column of A v is the dot product of the i-th row of A with the first column of v. So for matrices A, B: The number in the i-th row and the j-th column of A B is the dot product of the i-th row of A with the j-th column of B. A. Kissinger Version: spring 2016 Matrix Calculations 9 / 43

9 Matrix multiplication For A an m n matrix, B an n p matrix: is an m p matrix... a i1 A B = C.. b j1.... a in.... = c ij.. b jn n c ij = a ik b kj k=1 A. Kissinger Version: spring 2016 Matrix Calculations 10 / 43

10 Special case: vectors Matrix multiplication For A an m n matrix, B an n 1 matrix: A b = c is an m 1 matrix... a i1.. a in.. b 11.. b n1 =. c i1. n c i1 = a ik b k1 k=1 A. Kissinger Version: spring 2016 Matrix Calculations 11 / 43

11 Matrix composition Matrix multiplication Theorem Matrix composition is associative: (A B) C = A (B C) Proof. Let X := (A B) C. This is a matrix with entries: x ip = k a ik b kp Then, the matrix entries of X C are: x ip c pj = ( ) a ik b kp c pk = p p kp k a ik b kp c pk (because sums can always be pulled outside, and combined) A. Kissinger Version: spring 2016 Matrix Calculations 12 / 43

12 Associativity of matrix composition Proof (cont d). Now, let Y := B C. This has matrix entries: y kj = p b kp c pj Then, the matrix entries of A Y are: a ik y kj = ( a ik k k p ) b kp c pj = a ik b kp c pk kp...which is the same as before! So: (A B) C = X C = A Y = A (B C) So we can drop those pesky parentheses: A B C := (A B) C = A (B C) A. Kissinger Version: spring 2016 Matrix Calculations 13 / 43

13 Matrix product and composition Corollary The composition of linear maps is given by matrix product. Proof. Let g(w) = A w and f (v) = B v. Then: g(f (v)) = g(b v) = A B v No wishful thinking necessary! A. Kissinger Version: spring 2016 Matrix Calculations 14 / 43

14 Example 1 Matrix multiplication Consider the following two linear maps, and their associated matrices: R 3 f R 2 R 2 g R 2 f (x 1, x 2, x 3 ) = (x 1 x 2, x 2 + x 3 ) g(y 1, y 2 ) = (2y 1 y 2, 3y 2 ) ( ) ( ) M f = M g = 0 3 We can compute the composition directly: (g f )(x 1, x 2, x 3 ) = g ( f (x 1, x 2, x 3 ) ) So: = g(x 1 x 2, x 2 + x 3 ) = ( 2(x 1 x 2 ) (x 2 + x 3 ), 3(x 2 + x 3 ) ) = ( 2x 1 3x 2 x 3, 3x 2 + 3x 3 ) ( ) M g f = which is just the product of the matrices: M g f = M g M f A. Kissinger Version: spring 2016 Matrix Calculations 15 / 43

15 Note: matrix composition is not commutative In general, A B B A ( ) 1 0 For instance: Take A = and B = 0 1 ( ) ( ) A B = ( ) = B A = = ( ) ( ) ( ) ( ) 0 1. Then: 1 0 = = ( ) ( 0 ) A. Kissinger Version: spring 2016 Matrix Calculations 16 / 43

16 But it is... Matrix multiplication...associative, as we ve already seen: A B C := (A B) C = A (B C) It also has a unit given by the identity matrix I : A I = I A = A where: I := A. Kissinger Version: spring 2016 Matrix Calculations 17 / 43

17 Example: political swingers, part I We take an extremely crude view on politics and distinguish only left and right wing political supporters We study changes in political views, per year Suppose we observe, for each year: 80% of lefties remain lefties and 20% become righties 90% of righties remain righties, and 10% become lefties Questions... start with a population L = 100, R = 150, and compute the number of lefties and righties after one year; similarly, after 2 years, and 3 years,... Find a convenient way to represent these computations. A. Kissinger Version: spring 2016 Matrix Calculations 18 / 43

18 Political swingers, part II So if we start with a population L = 100, R = 150, then after one year we have: lefties: = = 95 righties: = = 155 Two observations: this looks like a matrix-vector multiplication long-term developments can be calculated via iterated matrices A. Kissinger Version: spring 2016 Matrix Calculations 19 / 43

19 Political swingers, part III We can write the political transition ( matrix ) as P = ( ( ) L 100 If =, then after one year we have: R) 150 ( ) ( ) ( ) P = 150 ( ) = = After two ( years ) we have: ( ) P = = ( ) ( ) = ( ) ( ) A. Kissinger Version: spring 2016 Matrix Calculations 20 / 43

20 Political swingers, part IV The situation after two years is obtained as: ( ) ( ) ( ) ( ) L L P P = R R }{{} ( do this multiplication first ) ( ) L = R ( ) ( ) L = R The situation after n years is described by the n-fold iterated matrix: P n = P } P {{ P} n times A. Kissinger Version: spring 2016 Matrix Calculations 21 / 43

21 Political swingers, part V Interpret the following iterations: ( ) P 2 = P P = ( ) ( ) P 3 = P P P = ( ) = ( ) ( ) P 4 = P P P P = ( ) = Etc. Does this stabilise? We ll talk about fixed points later on... A. Kissinger Version: spring 2016 Matrix Calculations 22 / 43

22 Solving equations the old fashioned way... We now know that systems of equations look like this: A x = b The goal is to solve for x, in terms of A and b. Here comes some more wishful thinking: x = 1 A b Well, we can t really divide by a matrix, but if we are lucky, we can find another matrix called A 1 which acts like 1 A. A. Kissinger Version: spring 2016 Matrix Calculations 24 / 43

23 Inverse Matrix multiplication Definition The inverse of a matrix A is another matrix A 1 such that: A 1 A = A A 1 = I Not all matrices have inverses, but when they do, we are happy, because: A x = b = A 1 A x = A 1 b = x = A 1 b So, how do we compute the inverse of a matrix? A. Kissinger Version: spring 2016 Matrix Calculations 25 / 43

24 Remember me? Matrix multiplication A. Kissinger Version: spring 2016 Matrix Calculations 26 / 43

25 Gaussian elimination as matrix multiplication Each step of Gaussian elimination can be represented by a matrix multiplication: A A A := G A For instance, multiplying the i-th row by c is given by: G (Ri :=cr i ) A where G (Ri :=cr i ) is just like the identity matrix, but g ii = c. Exercise. What are the other Gaussian elimination matrices? G (Ri R j ) G (Ri :=R i +cr j ) A. Kissinger Version: spring 2016 Matrix Calculations 27 / 43

26 Reduction to Echelon form The idea: treat A as a coefficient matrix, and compute its reduced Echelon form If the Echelon form of A has n pivots, then its reduced Echelon form is the identity matrix: A A 1 A 2 A p = I Now, we can use our Gauss matrices to remember what we did: A 1 := G 1 A A 2 := G 2 G 1 A A p := G p G 1 A = I A. Kissinger Version: spring 2016 Matrix Calculations 28 / 43

27 Computing the inverse A ha! G p G 1 A = I = A 1 = G p G 1 So all we have to do is construct p different matrices and multiply them all together! Since I already have plans for this afternoon, lets take a shortcut: Theorem For C a matrix and (A B) an augmented matrix: C (A B) = (C A C B) A. Kissinger Version: spring 2016 Matrix Calculations 29 / 43

28 Computing the inverse Since Gaussian elimination is just multiplying by a certain matrix on the left... A G A...doing Gaussian elimination (for A) on an augmented matrix applies G to both parts: So, if G = A 1 : (A B) (G A G B) (A B) (A 1 A A 1 B) = (I A 1 B) A. Kissinger Version: spring 2016 Matrix Calculations 30 / 43

29 Computing the inverse We already (secretly) used this trick to solve: A x = b = x = A 1 b Here, we are only interested in the vector A 1 b Which is exactly what Gaussian elimination on the augmented matrix gives us: (A b) (I A 1 b) To get the entire matrix, we just need to choose something clever to the right of the line Like this: (A I ) (I A 1 I ) = (I A 1 ) A. Kissinger Version: spring 2016 Matrix Calculations 31 / 43

30 Computing the inverse: example For example, we compute the inverse of: ( ) 1 1 A := 1 2 as follows: ( ) ( ) ( ) So: A 1 := ( 2 ) A. Kissinger Version: spring 2016 Matrix Calculations 32 / 43

31 Computing the inverse: non-example Unlike transpose, not every matrix has an inverse. For example, if we try to compute the inverse for: ( ) 1 1 B := 1 1 we have: ( ) ( ) We don t have enough pivots to continue reducing. So B does not have an inverse. A. Kissinger Version: spring 2016 Matrix Calculations 33 / 43

32 Subspace definition Matrix multiplication Definition A subset S V of a vector space V is called a (linear) subspace if S is closed under addition and scalar multiplication: 0 S v, v S implies v + v S v S and a R implies a v S. Note A subspace S V is a vector space itself, and thus also has a basis. Also S has its own dimension, where dim(s) dim(v ). A. Kissinger Version: spring 2016 Matrix Calculations 35 / 43

33 Subspace examples Matrix multiplication 1 Earlier we saw that the subset of solutions of a system of equations is closed under addition and (scalar) multiplication, and thus is a linear subspace. 2 The diagonal D = {(x, x) x R} R 2 is a linear subspace: if (x 1, x 1 ), (x 2, x 2 ) D, then also (x 1, x 1 ) + (x 2, x 2 ) = (x 1 + x 2, x 1 + x 2 ) D if (x, x) D and a R, also a (x, x) = (a x, a x) D Also: D has a single vector as basis, for example (1, 1) thus, D has dimension 1 A. Kissinger Version: spring 2016 Matrix Calculations 36 / 43

34 Basis for subspaces Matrix multiplication Let the space V have dimension n, and a subspace S V dimension p, where p n. Then: any set of > p vectors in S is linearly dependent any set of < p vectors in S does not span S any set of p independent vectors in S is a basis for S any set of p vectors that spans S is a basis for S A. Kissinger Version: spring 2016 Matrix Calculations 37 / 43

35 : definitions Definition Let f : V W be a linear map the kernel of f is the subset of V given by: ker(f ) = {v V f (v) = 0} the image of f is the subset of W given by: Example im(f ) = {f (v) v V } Consider the function f : R 2 R 2 given by f (x, y) = (x, 0) the kernel is {(x, y) R 2 f (x, y) = (0, 0)}, which is {(0, y) y R}, i.e. the y-axis. the image is the x-axis {(x, 0) x R} A. Kissinger Version: spring 2016 Matrix Calculations 38 / 43

36 Kernels and images are subspaces Theorem For a linear map f : V W, ker(f ) = {v f (v) = 0} V is a linear subspace im(f ) = {f (v) v V } W is a linear subspace. Proof: We check two cases (do the others yourself!) Closure of ker(f ) under addition: if v, v ker(f ), then f (v) = 0 and f (v ) = 0. By linearity of f, f (v + v ) = f (v) + f (v ) = = 0, so v + v ker(f ). Closure of im(f ) under scalar multiplication: Assume w im(f ), say w = f (v), and a R. Again by linearity: a w = a f (v) = f (a v), so a w im(f ). A. Kissinger Version: spring 2016 Matrix Calculations 39 / 43

37 Injectivity and surjectivity A linear map f : V W is surjective: w v.f (v) = w if and only if im(f ) = W. A linear map f : V W is injective: f (v) = f (w) = v = w if and only if ker(f ) = 0. A. Kissinger Version: spring 2016 Matrix Calculations 40 / 43

38 The kernel as solution space With this kernel (and image) terminology we can connect some previous concepts. Theorem Suppose a linear map f : V W has matrix A. Then: v ker(f ) f (v) = 0 A v = 0 v solves a system of homogeneous equations Moreover, the dimension of the kernel dim(ker(f )) is the same as the number of basic solutions of A, that is the number of columns without pivots in the echelon form of A. A. Kissinger Version: spring 2016 Matrix Calculations 41 / 43

39 We can learn a lot about a matrix......by looking at its columns. Suppose a linear map f is represented by a matrix A with columns {v 1,..., v n }: f (w) = v 1 v n w Then, dim(im(f )) is the dimension of the space spanned by {v 1,..., v n }...which is the same as the number of pivots in A A. Kissinger Version: spring 2016 Matrix Calculations 42 / 43

40 Kernel-image-dimension theorem (aka. rank-nullity) Theorem For a linear map f : V W one has: dim(ker(f )) + dim(im(f )) = dim(v ) Proof: Let A be a matrix that represents f. It has dim(v ) columns. dim(im(f )) of those are pivots, and the rest correspond to basic solutions to A x = 0, which give a basis for ker(f ). A. Kissinger Version: spring 2016 Matrix Calculations 43 / 43

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### 1.5 Elementary Matrices and a Method for Finding the Inverse

.5 Elementary Matrices and a Method for Finding the Inverse Definition A n n matrix is called an elementary matrix if it can be obtained from I n by performing a single elementary row operation Reminder:

### Practice Math 110 Final. Instructions: Work all of problems 1 through 5, and work any 5 of problems 10 through 16.

Practice Math 110 Final Instructions: Work all of problems 1 through 5, and work any 5 of problems 10 through 16. 1. Let A = 3 1 1 3 3 2. 6 6 5 a. Use Gauss elimination to reduce A to an upper triangular

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Linear Maps. Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (February 5, 2007)

MAT067 University of California, Davis Winter 2007 Linear Maps Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (February 5, 2007) As we have discussed in the lecture on What is Linear Algebra? one of

### 2.1: MATRIX OPERATIONS

.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### T ( a i x i ) = a i T (x i ).

Chapter 2 Defn 1. (p. 65) Let V and W be vector spaces (over F ). We call a function T : V W a linear transformation form V to W if, for all x, y V and c F, we have (a) T (x + y) = T (x) + T (y) and (b)

### MATH 240 Fall, Chapter 1: Linear Equations and Matrices

MATH 240 Fall, 2007 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 9th Ed. written by Prof. J. Beachy Sections 1.1 1.5, 2.1 2.3, 4.2 4.9, 3.1 3.5, 5.3 5.5, 6.1 6.3, 6.5, 7.1 7.3 DEFINITIONS

### Matrix Calculations: Inverse and Basis Transformation

Matrix Calculations: Inverse and asis Transformation A. Kissinger (and H. Geuvers) Institute for Computing and Information ciences Intelligent ystems Version: spring 25 A. Kissinger (and H. Geuvers) Version:

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Lecture Notes 2: Matrices as Systems of Linear Equations

2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably

### Chapter 4: Binary Operations and Relations

c Dr Oksana Shatalov, Fall 2014 1 Chapter 4: Binary Operations and Relations 4.1: Binary Operations DEFINITION 1. A binary operation on a nonempty set A is a function from A A to A. Addition, subtraction,

### 1 Gaussian Elimination

Contents 1 Gaussian Elimination 1.1 Elementary Row Operations 1.2 Some matrices whose associated system of equations are easy to solve 1.3 Gaussian Elimination 1.4 Gauss-Jordan reduction and the Reduced

### 1 Eigenvalues and Eigenvectors

Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

### ( % . This matrix consists of \$ 4 5 " 5' the coefficients of the variables as they appear in the original system. The augmented 3 " 2 2 # 2 " 3 4&

Matrices define matrix We will use matrices to help us solve systems of equations. A matrix is a rectangular array of numbers enclosed in parentheses or brackets. In linear algebra, matrices are important

### 1 0 5 3 3 A = 0 0 0 1 3 0 0 0 0 0 0 0 0 0 0

Solutions: Assignment 4.. Find the redundant column vectors of the given matrix A by inspection. Then find a basis of the image of A and a basis of the kernel of A. 5 A The second and third columns are

### Lecture Note on Linear Algebra 15. Dimension and Rank

Lecture Note on Linear Algebra 15. Dimension and Rank Wei-Shi Zheng, wszheng@ieee.org, 211 November 1, 211 1 What Do You Learn from This Note We still observe the unit vectors we have introduced in Chapter

### Lecture 2 Matrix Operations

Lecture 2 Matrix Operations transpose, sum & difference, scalar multiplication matrix multiplication, matrix-vector product matrix inverse 2 1 Matrix transpose transpose of m n matrix A, denoted A T or

### ( ) which must be a vector

MATH 37 Linear Transformations from Rn to Rm Dr. Neal, WKU Let T : R n R m be a function which maps vectors from R n to R m. Then T is called a linear transformation if the following two properties are

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

### Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M 1 such that

0. Inverse Matrix Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M such that M M = I = M M. Inverse of a 2 2 Matrix Let M and N be the matrices: a b d b M =, N = c

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### Linear Transformations

a Calculus III Summer 2013, Session II Tuesday, July 23, 2013 Agenda a 1. Linear transformations 2. 3. a linear transformation linear transformations a In the m n linear system Ax = 0, Motivation we can

### MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix.

MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix. Matrices Definition. An m-by-n matrix is a rectangular array of numbers that has m rows and n columns: a 11

### Math 113 Homework 3. October 16, 2013

Math 113 Homework 3 October 16, 2013 This homework is due Thursday October 17th at the start of class. Remember to write clearly, and justify your solutions. Please make sure to put your name on the first

### Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Name: Section Registered In:

Name: Section Registered In: Math 125 Exam 3 Version 1 April 24, 2006 60 total points possible 1. (5pts) Use Cramer s Rule to solve 3x + 4y = 30 x 2y = 8. Be sure to show enough detail that shows you are

### Sec 4.1 Vector Spaces and Subspaces

Sec 4. Vector Spaces and Subspaces Motivation Let S be the set of all solutions to the differential equation y + y =. Let T be the set of all 2 3 matrices with real entries. These two sets share many common

### A general element of R 3 3 has the form A = d e f and the condition A T = A becomes. b e f, so that. b = d, c = g, and f = h. Thus, A = a b c.

Let V = {f(t) P 3 f() = f()} Show that V is a subspace of P 3 and find a basis of V The neutral element of P 3 is the function n(t) = In particular, n() = = n(), so n(t) V Let f, g V Then f() = f() and

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if

### MA 242 LINEAR ALGEBRA C1, Solutions to Second Midterm Exam

MA 4 LINEAR ALGEBRA C, Solutions to Second Midterm Exam Prof. Nikola Popovic, November 9, 6, 9:3am - :5am Problem (5 points). Let the matrix A be given by 5 6 5 4 5 (a) Find the inverse A of A, if it exists.

### Orthogonal Diagonalization of Symmetric Matrices

MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

### Lecture 10: Invertible matrices. Finding the inverse of a matrix

Lecture 10: Invertible matrices. Finding the inverse of a matrix Danny W. Crytser April 11, 2014 Today s lecture Today we will Today s lecture Today we will 1 Single out a class of especially nice matrices

### 4. Matrix inverses. left and right inverse. linear independence. nonsingular matrices. matrices with linearly independent columns

L. Vandenberghe EE133A (Spring 2016) 4. Matrix inverses left and right inverse linear independence nonsingular matrices matrices with linearly independent columns matrices with linearly independent rows

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### MA106 Linear Algebra lecture notes

MA106 Linear Algebra lecture notes Lecturers: Martin Bright and Daan Krammer Warwick, January 2011 Contents 1 Number systems and fields 3 1.1 Axioms for number systems......................... 3 2 Vector

### Lecture 11. Shuanglin Shao. October 2nd and 7th, 2013

Lecture 11 Shuanglin Shao October 2nd and 7th, 2013 Matrix determinants: addition. Determinants: multiplication. Adjoint of a matrix. Cramer s rule to solve a linear system. Recall that from the previous

### Basic Terminology for Systems of Equations in a Nutshell. E. L. Lady. 3x 1 7x 2 +4x 3 =0 5x 1 +8x 2 12x 3 =0.

Basic Terminology for Systems of Equations in a Nutshell E L Lady A system of linear equations is something like the following: x 7x +4x =0 5x +8x x = Note that the number of equations is not required

### Images and Kernels in Linear Algebra By Kristi Hoshibata Mathematics 232

Images and Kernels in Linear Algebra By Kristi Hoshibata Mathematics 232 In mathematics, there are many different fields of study, including calculus, geometry, algebra and others. Mathematics has been

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### Math 312 Homework 1 Solutions

Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

### Methods for Finding Bases

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,

### 4.2. Linear Combinations and Linear Independence that a subspace contains the vectors

4.2. Linear Combinations and Linear Independence If we know that a subspace contains the vectors v 1 = 2 3 and v 2 = 1 1, it must contain other 1 2 vectors as well. For instance, the subspace also contains

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible:

Cramer s Rule and the Adjugate Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible: Theorem [Cramer s Rule] If A is an invertible

### University of Ottawa

University of Ottawa Department of Mathematics and Statistics MAT 1302A: Mathematical Methods II Instructor: Alistair Savage Final Exam April 2013 Surname First Name Student # Seat # Instructions: (a)

### The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

### Matrix Representations of Linear Transformations and Changes of Coordinates

Matrix Representations of Linear Transformations and Changes of Coordinates 01 Subspaces and Bases 011 Definitions A subspace V of R n is a subset of R n that contains the zero element and is closed under

### Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Pearson Education, Inc.

2 Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Theorem 8: Let A be a square matrix. Then the following statements are equivalent. That is, for a given A, the statements are either all true

### Algebra and Linear Algebra

Vectors Coordinate frames 2D implicit curves 2D parametric curves 3D surfaces Algebra and Linear Algebra Algebra: numbers and operations on numbers 2 + 3 = 5 3 7 = 21 Linear algebra: tuples, triples,...

### 4.6 Null Space, Column Space, Row Space

NULL SPACE, COLUMN SPACE, ROW SPACE Null Space, Column Space, Row Space In applications of linear algebra, subspaces of R n typically arise in one of two situations: ) as the set of solutions of a linear

### Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

### Interpolating Polynomials Handout March 7, 2012

Interpolating Polynomials Handout March 7, 212 Again we work over our favorite field F (such as R, Q, C or F p ) We wish to find a polynomial y = f(x) passing through n specified data points (x 1,y 1 ),

### Math 215 Exam #1 Practice Problem Solutions

Math 5 Exam # Practice Problem Solutions For each of the following statements, say whether it is true or false If the statement is true, prove it If false, give a counterexample (a) If A is a matrix such

### 5.3 ORTHOGONAL TRANSFORMATIONS AND ORTHOGONAL MATRICES

5.3 ORTHOGONAL TRANSFORMATIONS AND ORTHOGONAL MATRICES Definition 5.3. Orthogonal transformations and orthogonal matrices A linear transformation T from R n to R n is called orthogonal if it preserves

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### 1 Sets and Set Notation.

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

### Linear Algebra Notes

Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note

### Matrices provide a compact notation for expressing systems of equations or variables. For instance, a linear function might be written as: b k

MATRIX ALGEBRA FOR STATISTICS: PART 1 Matrices provide a compact notation for expressing systems of equations or variables. For instance, a linear function might be written as: y = x 1 b 1 + x 2 + x 3

### Matrix Inverse and Determinants

DM554 Linear and Integer Programming Lecture 5 and Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1 2 3 4 and Cramer s rule 2 Outline 1 2 3 4 and

### Review: Vector space

Math 2F Linear Algebra Lecture 13 1 Basis and dimensions Slide 1 Review: Subspace of a vector space. (Sec. 4.1) Linear combinations, l.d., l.i. vectors. (Sec. 4.3) Dimension and Base of a vector space.

### Direct Methods for Solving Linear Systems. Linear Systems of Equations

Direct Methods for Solving Linear Systems Linear Systems of Equations Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University

### Section 6.1 - Inner Products and Norms

Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,

### Math 240: Linear Systems and Rank of a Matrix

Math 240: Linear Systems and Rank of a Matrix Ryan Blair University of Pennsylvania Thursday January 20, 2011 Ryan Blair (U Penn) Math 240: Linear Systems and Rank of a Matrix Thursday January 20, 2011

### University of Lille I PC first year list of exercises n 7. Review

University of Lille I PC first year list of exercises n 7 Review Exercise Solve the following systems in 4 different ways (by substitution, by the Gauss method, by inverting the matrix of coefficients

### MATH 2030: SYSTEMS OF LINEAR EQUATIONS. ax + by + cz = d. )z = e. while these equations are not linear: xy z = 2, x x = 0,

MATH 23: SYSTEMS OF LINEAR EQUATIONS Systems of Linear Equations In the plane R 2 the general form of the equation of a line is ax + by = c and that the general equation of a plane in R 3 will be we call

### Math 54. Selected Solutions for Week Is u in the plane in R 3 spanned by the columns

Math 5. Selected Solutions for Week 2 Section. (Page 2). Let u = and A = 5 2 6. Is u in the plane in R spanned by the columns of A? (See the figure omitted].) Why or why not? First of all, the plane in

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### MATH1231 Algebra, 2015 Chapter 7: Linear maps

MATH1231 Algebra, 2015 Chapter 7: Linear maps A/Prof. Daniel Chan School of Mathematics and Statistics University of New South Wales danielc@unsw.edu.au Daniel Chan (UNSW) MATH1231 Algebra 1 / 43 Chapter

### Homogeneous and inhomogeneous solutions, and what they look like. Subspaces. The null space and column space of a matrix.

Math 34 (Elementary Linear Algebra I, Fall 24 Practice Midterm 2 Solutions Answer all questions in the space provided. Approximate List of Topics (a long list, but most of these things are interrelated:

### Topic 1: Matrices and Systems of Linear Equations.

Topic 1: Matrices and Systems of Linear Equations Let us start with a review of some linear algebra concepts we have already learned, such as matrices, determinants, etc Also, we shall review the method

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Lecture 5 - Triangular Factorizations & Operation Counts

LU Factorization Lecture 5 - Triangular Factorizations & Operation Counts We have seen that the process of GE essentially factors a matrix A into LU Now we want to see how this factorization allows us

### Markov Chains, part I

Markov Chains, part I December 8, 2010 1 Introduction A Markov Chain is a sequence of random variables X 0, X 1,, where each X i S, such that P(X i+1 = s i+1 X i = s i, X i 1 = s i 1,, X 0 = s 0 ) = P(X

### Revision of ring theory

CHAPTER 1 Revision of ring theory 1.1. Basic definitions and examples In this chapter we will revise and extend some of the results on rings that you have studied on previous courses. A ring is an algebraic

### MAT188H1S Lec0101 Burbulla

Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u

### 5.5. Solving linear systems by the elimination method

55 Solving linear systems by the elimination method Equivalent systems The major technique of solving systems of equations is changing the original problem into another one which is of an easier to solve

### v 1. v n R n we have for each 1 j n that v j v n max 1 j n v j. i=1

1. Limits and Continuity It is often the case that a non-linear function of n-variables x = (x 1,..., x n ) is not really defined on all of R n. For instance f(x 1, x 2 ) = x 1x 2 is not defined when x

### Chapter 7. Matrices. Definition. An m n matrix is an array of numbers set out in m rows and n columns. Examples. ( 1 1 5 2 0 6

Chapter 7 Matrices Definition An m n matrix is an array of numbers set out in m rows and n columns Examples (i ( 1 1 5 2 0 6 has 2 rows and 3 columns and so it is a 2 3 matrix (ii 1 0 7 1 2 3 3 1 is a

### Inverses and powers: Rules of Matrix Arithmetic

Contents 1 Inverses and powers: Rules of Matrix Arithmetic 1.1 What about division of matrices? 1.2 Properties of the Inverse of a Matrix 1.2.1 Theorem (Uniqueness of Inverse) 1.2.2 Inverse Test 1.2.3

### 1 Determinants. Definition 1

Determinants The determinant of a square matrix is a value in R assigned to the matrix, it characterizes matrices which are invertible (det 0) and is related to the volume of a parallelpiped described

### MATH 110 Spring 2015 Homework 6 Solutions

MATH 110 Spring 2015 Homework 6 Solutions Section 2.6 2.6.4 Let α denote the standard basis for V = R 3. Let α = {e 1, e 2, e 3 } denote the dual basis of α for V. We would first like to show that β =

### Solutions to Math 51 First Exam January 29, 2015

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

### MAT Solving Linear Systems Using Matrices and Row Operations

MAT 171 8.5 Solving Linear Systems Using Matrices and Row Operations A. Introduction to Matrices Identifying the Size and Entries of a Matrix B. The Augmented Matrix of a System of Equations Forming Augmented

### Row and column operations

Row and column operations It is often very useful to apply row and column operations to a matrix. Let us list what operations we re going to be using. 3 We ll illustrate these using the example matrix