# Lecture No. # 02 Prologue-Part 2

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Advanced Matrix Theory and Linear Algebra for Engineers Prof. R.Vittal Rao Center for Electronics Design and Technology Indian Institute of Science, Bangalore Lecture No. # 02 Prologue-Part 2 In the last lecture, we discussed two important problems in Linear Algebra, and we shall review this since, these forms a basic foundation for many of the things that we are going to study. (Refer Slide Time: 00:32) The first problem we studied was linear systems of equations. We have an m by n matrix and we have given an m by 1 matrix, and we want to find an x an n by 1 matrix, such that A x equal to b. This is the first fundamental problem that we looked at. The first basic question that we raised was, what is the criterion b should satisfy, so that the system one has a solution? That was the first fundamental question.

3 criterion for the existence of a solution. Then the conclusion we can make is no solution exists. Then we found last time that under these circumstances, we can find, what is meant by least square solutions? That is the solution that minimizes the error between b and A x. Then once we have assured that the d square solution exists, we again ask, how many least square solutions are there? How many least square solutions are there? (Refer Slide Time: 04:51) Again, we encounter the same situation as before, unique and infinitely many. Again, when is it unique? When is it infinitely many? Then, one shows the answers to these questions. We want to find, the unique least square solution and then, it is infinitely many and we know when it is, we want find all find all the least square solutions. Again as before, we want a criterion to choose criterion to choose unique representative. Then finally, find the representative. So, these are the fundamental questions that arise, when we are dealing with system of equations? Our aim will be to find the answers to all these questions.

4 (Refer Slide Time: 06:21) The other problem that we discussed was basically, about diagonalization. The question was given, an m by n matrix p given an m by n matrix A, find invertible. Now, let us look at a simple situation first; let us take a square matrix n by n square matrix and then, find an invertible matrix P, which is also n by n such that, P inverse A P is a diagonal matrix. (Refer Slide Time: 07:30)

5 We found, that this is not always pass it well and therefore, we ask a question, what is the criterion A should satisfy? So, that such a P exists. Let us, called this criterion as C so, we will use the symbol C for the criterion C. (Refer Slide Time: 08:04) Then, the given A we ask, does A satisfy C? As before, they are lead to two possible answers yes and no. Again, when it is yes? We ask find the P such that P inverse A P is diagonal, when it is no? We know, there is no such P, so no such P exists. Now in this situation, there are two types of analysis we can do; the one is find a P such that, P inverse A P is nearly diagonal, can we do such a thing? And an answer to this, would lead as to what is known as the Jordon canonical form? On the other hand, there is another option available for as, is instead of finding one such P, find Q and P such that, Q inverse A P is diagonal.

6 (Refer Slide Time: 09:55) We shall see that, this leads to an answer to this leads to, what is known as the singular value decomposition of the matrix A? Now, the advantage of this singular value decomposition is, it allows for generalization to rectangular matrices as well. It allows for generalization to rectangular matrices. So, these are two basic problems which we discussed in the last lecture. (Refer Slide Time: 10:47) Now, we shall look at a third important problem. It must mention that, the other two problems are interrelated. You may recall the second problem a rows out of reducing the

7 system to a simpler system. Let us now, look at another important problem which is as follows. Suppose, we start with a vector x are what we call, a column matrix and another column matrix say x is x 1 x 2 x n and y is y 1 y 2 y n and let us say all are real. So, let us say, we have two vectors or two column matrices each having n entries. (Refer Slide Time: 11:42) Then we define, what is known as the inner product of x and y? We denote it by x comma y both are bracket and we define, it as y transpose x which is simply summation i equal to 1 to n x i y i. This is a real number thus, the inner product of these two is a number; the inner product of two such column matrices is always a number.

8 (Refer Slide Time: 12:43) Let us look at a very simple example; let us take x equal to and y equal to minus then, the inner product is simply the sum of the product of the corresponding components which is minus 1 plus 0 plus 2, which is one which is a real number. Now, we do something different. (Refer Slide Time: 13:13) We take another type of product between two vectors and this is called the tensor product. Let us once again, start with a vector x or a column matrix x and another column matrix. Once again, let us take all are real. So, we take two column matrices x 1

9 x 2 x n y 1 y 2 y n both of them, having exactly n entries. Then we define, the tensor product of x with y as, we denote it by this symbol x with this into symbol enclose by a circle; that is the symbol for the dot tensor product and we define, it to be x y transpose. Now, since x is an n by 1 matrix and y is a n by 1 matrix, y transpose will be a 1 by n matrix. So, the product will be an n by n matrix. So now, previously when we took the inner product, the inner product up to column matrices was a number. Now, we go higher in the hierarchy, when we take the inner product or the tensor product of two such matrices, we get an m by n matrix column matrices now expand and we get, n by m matrix. Thus, the tensor product gives as a matrix. Now, supposing I can I can instead of taking an n by m matrix and an m by n matrix or taking two column vectors of the same size, we take two column vectors of different size. (Refer Slide Time: 15:35) So, that is the generalization we will do, what we do now is? We take a column matrix, which is x 1 x 2 up to x m and another column matrix, which is y 1 y 2 y n again, all are real. So, one of them has m, the other one has n and now, we defined the tensor product of x with y; the tensor product of two such matrices or two such column vectors. We define, it to be x y transpose as before. Now, what do we get this term? Again x is m by 1, y is n by 1 so y transpose is 1 by n. So, the product is therefore, an m by n matrix.

10 (Refer Slide Time: 16:49) So, what is our conclusion? What are the entries of this matrix? This is a matrix, whose i j th entry is x i y j this is a matrix, whose entry in the i th row and the j th column is precisely the product of x i and y j. (Refer Slide Time: 17:17) Thus, the conclusion is the tensor product of an m by 1 matrix x with an n by 1 matrix y is an m by n matrix. Note, the tensor product of x with y is not necessarily equal to the tensor product of y with x is non commutative product between matrices and which generates a higher dimensional matrix.

11 (Refer Slide Time: 18:22) Let us look at a simple example, let s take, x to be equal to 1 minus 1 2 and y to be 2 3 then, what is the tensile product of x with y? You must take x y transpose which is 1 minus and when we take the product, we get 2 3 minus which is now, a 3 by 2 matrix. Notice that, all the rows of the tensor product x y are multiples of y transpose. For example, this row this row here give the just 1 multiple of 2 3; this row here is the minus 1 multiple of 2 3; this row here is the double twice multiple of this 2 3. (Refer Slide Time: 20:00)

12 So, thus is the x x the tensor product x y has a simple structure. Namely, all his rows are multiples of just y transpose; this is one simple way of generating an m by n matrix. Taking tensor product x y, where x is m by 1 and y is n by 1 is a simple way of generating an m by n matrix and at the matrix that, we are generated as a very simple structure as we absorb, all the rows are just multiples of the row the row vector y transpose or the row matrix y transpose. (Refer Slide Time: 21:28) Now, let us push this construction little further. So, further generalization, instead of taking 1 x vector of m by 1 size, a y vector of n by 1 size. Let us, take a number of them and put them together, what do we mean by this? So, consider the positive integer k, look at k m by 1 matrices or column vectors u 1 u 2 u k and k n by 1 matrices v 1 v 2 v k. Now, we will form the tensor product of u with v 1 u 2 with v 2 u k with v k.

13 (Refer Slide Time: 22:34) So, for each i 1 less than equal to i less than equal to k, look at the tensor product of x i with y i. Now since, x i is m by 1 y i is n by 1, the tensor product will be an m by n matrix, this will be an m by n matrix. Now, for i equal to 1, we get 1 m by n matrix for equal to 2; we get 1 m by n matrix and so on. Finally, for i equal to k, we get 1 m by n matrix. If you add all this, we still get an m by n matrix, i am sorry this should be u i tensor product. Now, for each i we get a m by n matrix denoted by this tensor product, now let us add all these we get, summation i equal to 1 to k u i tensor product with V i. (Refer Slide Time: 24:02)

14 Since, each term is an m by n matrix; this is an m by n matrix. Now, we see that, we are able to generate m by n matrices by such sums of product; we can get, we can generate lots of m by n matrices by varying the number k, that is whether you would like to add k such tensor product; 1 such tensor product and so on. So, you can choose the number of tensor products that you use, that can be changed and we can change the u 1, u 2, u k; we can vary the v 1, v 2, v k. So, we have lots of option of changing these combinations and each time, we may end up generating more and more different m by n matrices. So, we can generate a lot of m by n matrices by this process of tensor product sum. The fundamental question is, does this construction exhaust all m by n matrices does this construction exhaust all m by n matrices? What does this mean? This means, given any m by n matrix A. (Refer Slide Time: 25:55) Can we find one, a positive integer k; two column matrices u 1, u 2, u k, m by 1 matrices, that is m by 1 column matrices; v 1, v 2, v k n by 1 column matrices such that; A can be now expressed as, the sum of all this tensor products u i tensor product v i. This is the question. Now, if we can do it? We can see that, A has been split into a number of small simple matrices because each one of these tensor products has a simple structure namely, every row is a multiple of a fixed row; namely, the i th row is a multiple of v i transpose and therefore, that would break the matrix into simpler parts.

15 (Refer Slide Time: 27:32) The answer we will see that, the answer is yes; this raises leads to the following question. Now that, we are assure that the matrix A, an m by n matrix A can be broken into such small pieces. Each one of which is simple, you would like to do it efficient way. So, what is an efficient way of doing this? That is, what is an efficient way of expressing a given matrix A at the sum of tensor product? First of all, we would like to have as few terms as possible. (Refer Slide Time: 28:41)

16 So that is, the first question is what is the minimum value of k required? That is, what is a minimum number of sums that you have to take in order to construct the matrix A? Of course, the answer will dependent on, what is the matrix? The properties of that matrix so what is this number k will depend on A? The second question is having fixed that number k, say minimum is row. Usual eventually, it is see that, row is the rank of the matrix. Now, having fix that minimum number, can we choose u 1, u 2, u rho, these are the m by 1 matrices v 1, v 2, v rho n by 1 matrix that, will appear in the tensor product. Such that, the matrix A equal to i equal to 1 to k u i v i is now, easy to analyze. What do you mean by easy to analyze? For example, can solve A x equal to b using this decomposition, expressing A as the sum of the tensor product, we can solve a equal to b easily. (Refer Slide Time: 30:31) Now, an answer to this question will again, leads to what is known as the singular value decomposition? We have already seen, a singular value decomposition earlier, there we were trying to express the matrix as the product of matrices. Here, we are trying to express the matrix A as the sum of matrix, both have closely related to each other. We will when we get to these details; we will see all these are totally related to each other. This is another important problem in particular, the way we choose the u i and the v i would make not only solve in the system ax to be easy to solve, it would even help us in compressing that the data A stores. Before we begin, over formal study of Linear

17 Algebra, we would look at one more important problem connected with matrices or one more important problem of linear algebra. (Refer Slide Time: 31:39) This will call, at the fourth problem. This is again, connected directly to the systems of equations and will later, will find its connection to the singular value decomposition etcetera. Let us recall that, when we have a system A x equal to b; we had situations like does b satisfy the condition C for existence of a solution; we had two situations yes and no and the yes led to two situations, the unique solution and the unique representative solution. That is in the case, when there are infinite number of solution we had to pick one of them by using some criterion. So, that our final answer is no vector x. In the case of no, we had least square solution and this again gave us two situations, unique least square solution, the unique least square solution and the unique representative unique representative least square solution. By we see that, finally, in all the situation, we are looking at the unique solution. In this case, the first case, the unique representative solution, the unique least square solution, the unique representative least square solution therefore, in all this finally, we want the answer to be one unique vector.

18 (Refer Slide Time: 31:51) So, we have this situation. The situation is we are given b, we want to find in each one of this case the solution. When we say, the solution; we mean, the unique solution in the first case; the unique representative solution in the second case; the unique the least square solution the third case and the unique representative solution in the fourth case. You will in general, denote this by x naught (b) because it will depend on b and it is a vector or a solution matrix m by 1 matrix m by 1 matrix and will denote it by x naught (b). So, our answer is finally, to find x naught b. This x naught b again, a reprove will represent the unique solution whenever, the unique solution exists; the unique representative solution whenever, there are in infinite number solutions; the unique least square solution when the least square solution exists only an unique or the unique representative least square solution then, there are an infinite number of least square solution and we pick, the unique representative. So, our final job is we are given the vector b produce b the vector x 0 (b), which is the ultimate answer to have question about solving the system of equation. We can view this problem in the following way.

19 (Refer Slide Time: 35:38) Namely, let us look at the matrix as, the transfer matrix or the transfer function of a linear system. What does it do? When we have an input x, which is an n by 1 vector, so the permissible inputs for the system are all n by 1 matrices, then the input and m by 1 matrix; the output is going to be A x, which is an input. If you put input n by 1, the output is going to be m by 1. What we are asking is, when we solve A x equal to b means, we are asking for a particular output b, and we would like to know what the input x is. So, we are looking for the input x, which will produce the output b. Our answer should be that x naught (b). What do you mean by x naught (b)? Whenever there is only one input that, produced the desired output b, x naught (b) will be that; whenever there are several inputs which produce the same output b, then the x naught (b) will be the representative input, which produces b; whenever there is no input x, which produces the output b, then we will be looking for inputs, which will take you as close to the required output b, that is that minimizes the error. So, whenever there is a unique minimize or the unique least square solution, that input will be the x naught (b). Whenever there are several inputs which give the same minimal error then, x naught (b) will be the unique representative input. So, x naught (b) is our final answer; if you want the output b, x naught (b) is the best that can produce it. It can probably go to b and nobody else can give the better answer than that, that is what the final answer should be so. We are given b from that we are trying to construct the x

20 naught (b). So, we are given the required output b, you would like to construct the required input x naught (b). So, in other words we are trying to control the output b. (Refer Slide Time: 38:43) So, therefore, what you would like is, we are only given b; we had to do some calculations on this d to produce the x naught (b). So, this boils down to, we are constructing another system for which, that another transfer function. We do not know what it is? We denote it by A dagger for which, we are going to put the input b and there is going to be an output because of this which is A dagger b. We want to construct A dagger such that, this A dagger b is our ultimate answer to our linear system. Naturally, we are expecting x 0 to be n by 1, b is m by 1. So, therefore, A dagger has to be constructed as an n by m matrix. So, the fundamental problem therefore here, the construction about the system of equations is the construction of this matrix A.

21 (Refer Slide Time: 40:08) So, main problem is given, an m by n matrix A. Construct an n by m matrix, A dagger such that, x naught (b) is equal to A dagger b. I repeat again, x naught (b) as different commutations and different situation. When there is unique solution for A x equal to b, x naught (b) represents the unique solution; when there are infinite number solutions for A x equal to b x naught (b) represents the unique representative solution; when there are more solutions for A x equal to b and if there is only a unique least square solution x naught (b) represents the unique least square solution and when the only least square solution, but infinite number of them then x naught (b) represent the unique representative least square solution. So, the fundamental problem therefore is, in the system; in the given the matrix A which is m by n, construct the matrix A dagger which is n by m. So, that x naught (b) is equal to A dagger. So, the moment pray you are b, I will send it through a dagger and produce my answer x 0 b. Which is the best answer possible for the system A x equal to b, this A dagger is reaches to several things one, inverse of A whenever A is a square matrix.

22 (Refer Slide Time: 40:08) And has an inverse and in the general case, whether it is an inverse; whether it is a square; whether it is rectangle etcetera, it is called the pseudo inverse. So, the pseudo inverse into same as the inverse, whenever quit makes sense. So, this leads us to the notion of the pseudo inverse of a matrix. (Refer Slide Time: 42:53) Thus, we have look at four problems one, the system A x equal to b the four problem we have studied are to generally discussed. We shall going to form, the back bone of the various studies that we have going to do. Then, we looked at the question of

23 diagonalization and near diagonalization; then, we look at the question of decomposition of a matrix as sum of tensor products and finally, the notion of the pseudo inverse. Now, we should not loose site of the fact that, the second third and fourth problems are confidential the question that, we ask in the first problem for the linear systems of equation. So, there are all highly inter related problems and go hand in hand, one helping the other or one leading to the other. in the (Refer Slide Time: 44:37) Therefore, in answering 2 3 and 4, the major role or the decisive role in 2 3 and 4 will be played by the notion of Eigen values and Eigen vectors.

24 (Refer Slide Time: 45:23) We look at the problem 3, in which, we are decomposing a matrix as sum of tensor products. Two important situations in these are the following one, when A is a square real symmetric matrix; two, A what is known as, we look at later the definition A square normal matrix. In case of complex, we should look at hermition complex matrix. This decomposition, what is known as the spectral decomposition for these matrices? As absorbed above, the Eigen values and Eigen vectors pay a crucial role in all this analysis. (Refer Slide Time: 46:52)

25 Our goal, for the course shall be the following; one, develop the mathematical frame work for analyzing these problems; two, find the answers to the various question raised in the above problems. (Refer Slide Time: 48:02) Three, look for generalizations and abstractions of these ideas. The main discussions in the course will be driven by these three goals. All our attention will always be too eventually to attain these three goals. Now ready to became a formal course on Linear Algebra, we have what we have seen is only a normal view of the important problems that lead as to study all these aspects of Linear Algebra that reserve the convicts on this course. So, before we begin our course, we shall give some standard notations that we will be using in this course.

26 (Refer Slide Time: 49:12) So, the first thing that we will study will be the linear systems. The notations that we will use are the following. Most of the time, we shall be dealing with matrices whose entries are real or complex numbers. So, what we will do is? We will use the general notation F, which may denote the real numbers R or the complex numbers C. R denotes the set of all real numbers, C denotes the set of all complex numbers. Whenever necessary, you will mention very specifically, whether we need R or C. If you do not mention then, it could be; it will work with F as R or it will also work with F as C. Then by F k, where k is the positive integer, we shall denote all column matrices with k entries that is, k by 1 column matrices where, all this entries are from F. So, when we say R k you mean all the entries are real numbers; when we say C k there is all the entries could be complex numbers.

27 (Refer Slide Time: 50:44) In particular, by theta k you will denote the vector or the matrix k by 1 matrix whose, entries are all 0. Then by F m by n, we shall denote all matrices which are m by n matrices and all the entries are from F. (Refer Slide Time: 51:21) And in particular by 0 m by n, we meant zero m by n matrix; that is m by n matrix, all of whose entries are 0. And in particular when m equal to n, we get F n square the set of all n by m matrices such that, a i j epsilon F; and in this situation, we shall denote by I n the

28 n by n identity matrix, 0 n to be n by n zero matrix. With this notation, we shall begin our next lecture, the study of linear system and equations.

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Introduction to Matrix Algebra I

Appendix A Introduction to Matrix Algebra I Today we will begin the course with a discussion of matrix algebra. Why are we studying this? We will use matrix algebra to derive the linear regression model

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### [1] Diagonal factorization

8.03 LA.6: Diagonalization and Orthogonal Matrices [ Diagonal factorization [2 Solving systems of first order differential equations [3 Symmetric and Orthonormal Matrices [ Diagonal factorization Recall:

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### Linear Algebra Review. Vectors

Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### Diagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions

Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential

### MATH 551 - APPLIED MATRIX THEORY

MATH 55 - APPLIED MATRIX THEORY FINAL TEST: SAMPLE with SOLUTIONS (25 points NAME: PROBLEM (3 points A web of 5 pages is described by a directed graph whose matrix is given by A Do the following ( points

### Matrix Algebra and Applications

Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2 - Matrices and Matrix Algebra Reading 1 Chapters

### 13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.

3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms

### Similar matrices and Jordan form

Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

### Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras

Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture No. # 31 Recursive Sets, Recursively Innumerable Sets, Encoding

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### Lecture 5: Singular Value Decomposition SVD (1)

EEM3L1: Numerical and Analytical Techniques Lecture 5: Singular Value Decomposition SVD (1) EE3L1, slide 1, Version 4: 25-Sep-02 Motivation for SVD (1) SVD = Singular Value Decomposition Consider the system

### A Primer on Index Notation

A Primer on John Crimaldi August 28, 2006 1. Index versus Index notation (a.k.a. Cartesian notation) is a powerful tool for manipulating multidimensional equations. However, there are times when the more

### Helpsheet. Giblin Eunson Library MATRIX ALGEBRA. library.unimelb.edu.au/libraries/bee. Use this sheet to help you:

Helpsheet Giblin Eunson Library ATRIX ALGEBRA Use this sheet to help you: Understand the basic concepts and definitions of matrix algebra Express a set of linear equations in matrix notation Evaluate determinants

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### Notes on Orthogonal and Symmetric Matrices MENU, Winter 2013

Notes on Orthogonal and Symmetric Matrices MENU, Winter 201 These notes summarize the main properties and uses of orthogonal and symmetric matrices. We covered quite a bit of material regarding these topics,

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### Lecture L3 - Vectors, Matrices and Coordinate Transformations

S. Widnall 16.07 Dynamics Fall 2009 Lecture notes based on J. Peraire Version 2.0 Lecture L3 - Vectors, Matrices and Coordinate Transformations By using vectors and defining appropriate operations between

### L1-2. Special Matrix Operations: Permutations, Transpose, Inverse, Augmentation 12 Aug 2014

L1-2. Special Matrix Operations: Permutations, Transpose, Inverse, Augmentation 12 Aug 2014 Unfortunately, no one can be told what the Matrix is. You have to see it for yourself. -- Morpheus Primary concepts:

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Lecture 2 Matrix Operations

Lecture 2 Matrix Operations transpose, sum & difference, scalar multiplication matrix multiplication, matrix-vector product matrix inverse 2 1 Matrix transpose transpose of m n matrix A, denoted A T or

### 2.1: MATRIX OPERATIONS

.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and

### Computer Graphics Prof. Sukhendu Das Dept. of Computer Science and Engineering Indian Institute of Technology, Madras Lecture 7 Transformations in 2-D

Computer Graphics Prof. Sukhendu Das Dept. of Computer Science and Engineering Indian Institute of Technology, Madras Lecture 7 Transformations in 2-D Welcome everybody. We continue the discussion on 2D

### MATH10212 Linear Algebra B Homework 7

MATH22 Linear Algebra B Homework 7 Students are strongly advised to acquire a copy of the Textbook: D C Lay, Linear Algebra and its Applications Pearson, 26 (or other editions) Normally, homework assignments

### SECTION 8.3: THE INVERSE OF A SQUARE MATRIX

(Section 8.3: The Inverse of a Square Matrix) 8.47 SECTION 8.3: THE INVERSE OF A SQUARE MATRIX PART A: (REVIEW) THE INVERSE OF A REAL NUMBER If a is a nonzero real number, then aa 1 = a 1 a = 1. a 1, or

### B such that AB = I and BA = I. (We say B is an inverse of A.) Definition A square matrix A is invertible (or nonsingular) if matrix

Matrix inverses Recall... Definition A square matrix A is invertible (or nonsingular) if matrix B such that AB = and BA =. (We say B is an inverse of A.) Remark Not all square matrices are invertible.

### Cryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur

Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 12 Block Cipher Standards

### Vector algebra Christian Miller CS Fall 2011

Vector algebra Christian Miller CS 354 - Fall 2011 Vector algebra A system commonly used to describe space Vectors, linear operators, tensors, etc. Used to build classical physics and the vast majority

### 4. Matrix inverses. left and right inverse. linear independence. nonsingular matrices. matrices with linearly independent columns

L. Vandenberghe EE133A (Spring 2016) 4. Matrix inverses left and right inverse linear independence nonsingular matrices matrices with linearly independent columns matrices with linearly independent rows

### 4.6 Null Space, Column Space, Row Space

NULL SPACE, COLUMN SPACE, ROW SPACE Null Space, Column Space, Row Space In applications of linear algebra, subspaces of R n typically arise in one of two situations: ) as the set of solutions of a linear

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### Using row reduction to calculate the inverse and the determinant of a square matrix

Using row reduction to calculate the inverse and the determinant of a square matrix Notes for MATH 0290 Honors by Prof. Anna Vainchtein 1 Inverse of a square matrix An n n square matrix A is called invertible

### ALGEBRA. sequence, term, nth term, consecutive, rule, relationship, generate, predict, continue increase, decrease finite, infinite

ALGEBRA Pupils should be taught to: Generate and describe sequences As outcomes, Year 7 pupils should, for example: Use, read and write, spelling correctly: sequence, term, nth term, consecutive, rule,

### CURVE FITTING LEAST SQUARES APPROXIMATION

CURVE FITTING LEAST SQUARES APPROXIMATION Data analysis and curve fitting: Imagine that we are studying a physical system involving two quantities: x and y Also suppose that we expect a linear relationship

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Orthogonal Projections

Orthogonal Projections and Reflections (with exercises) by D. Klain Version.. Corrections and comments are welcome! Orthogonal Projections Let X,..., X k be a family of linearly independent (column) vectors

### Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

### Solving simultaneous equations using the inverse matrix

Solving simultaneous equations using the inverse matrix 8.2 Introduction The power of matrix algebra is seen in the representation of a system of simultaneous linear equations as a matrix equation. Matrix

### Solving a System of Equations

11 Solving a System of Equations 11-1 Introduction The previous chapter has shown how to solve an algebraic equation with one variable. However, sometimes there is more than one unknown that must be determined

### Cofactor Expansion: Cramer s Rule

Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating

### Inner product. Definition of inner product

Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product

### Mathematics of Cryptography Modular Arithmetic, Congruence, and Matrices. A Biswas, IT, BESU SHIBPUR

Mathematics of Cryptography Modular Arithmetic, Congruence, and Matrices A Biswas, IT, BESU SHIBPUR McGraw-Hill The McGraw-Hill Companies, Inc., 2000 Set of Integers The set of integers, denoted by Z,

### Linear Algebra Notes

Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note

### Linear Algebra and TI 89

Linear Algebra and TI 89 Abdul Hassen and Jay Schiffman This short manual is a quick guide to the use of TI89 for Linear Algebra. We do this in two sections. In the first section, we will go over the editing

### Row Echelon Form and Reduced Row Echelon Form

These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation

### The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

### Tensors on a vector space

APPENDIX B Tensors on a vector space In this Appendix, we gather mathematical definitions and results pertaining to tensors. The purpose is mostly to introduce the modern, geometrical view on tensors,

### Unified Lecture # 4 Vectors

Fall 2005 Unified Lecture # 4 Vectors These notes were written by J. Peraire as a review of vectors for Dynamics 16.07. They have been adapted for Unified Engineering by R. Radovitzky. References [1] Feynmann,

### Torgerson s Classical MDS derivation: 1: Determining Coordinates from Euclidean Distances

Torgerson s Classical MDS derivation: 1: Determining Coordinates from Euclidean Distances It is possible to construct a matrix X of Cartesian coordinates of points in Euclidean space when we know the Euclidean

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Images and Kernels in Linear Algebra By Kristi Hoshibata Mathematics 232

Images and Kernels in Linear Algebra By Kristi Hoshibata Mathematics 232 In mathematics, there are many different fields of study, including calculus, geometry, algebra and others. Mathematics has been

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### Introduction to Matrices for Engineers

Introduction to Matrices for Engineers C.T.J. Dodson, School of Mathematics, Manchester Universit 1 What is a Matrix? A matrix is a rectangular arra of elements, usuall numbers, e.g. 1 0-8 4 0-1 1 0 11

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### 2D Geometric Transformations. COMP 770 Fall 2011

2D Geometric Transformations COMP 770 Fall 2011 1 A little quick math background Notation for sets, functions, mappings Linear transformations Matrices Matrix-vector multiplication Matrix-matrix multiplication

### Lecture Notes 2: Matrices as Systems of Linear Equations

2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably

### SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89. by Joseph Collison

SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89 by Joseph Collison Copyright 2000 by Joseph Collison All rights reserved Reproduction or translation of any part of this work beyond that permitted by Sections

### The Inverse of a Square Matrix

These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### 1 Introduction. 2 Matrices: Definition. Matrix Algebra. Hervé Abdi Lynne J. Williams

In Neil Salkind (Ed.), Encyclopedia of Research Design. Thousand Oaks, CA: Sage. 00 Matrix Algebra Hervé Abdi Lynne J. Williams Introduction Sylvester developed the modern concept of matrices in the 9th

### Clock Arithmetic and Modular Systems Clock Arithmetic The introduction to Chapter 4 described a mathematical system

CHAPTER Number Theory FIGURE FIGURE FIGURE Plus hours Plus hours Plus hours + = + = + = FIGURE. Clock Arithmetic and Modular Systems Clock Arithmetic The introduction to Chapter described a mathematical

The Hadamard Product Elizabeth Million April 12, 2007 1 Introduction and Basic Results As inexperienced mathematicians we may have once thought that the natural definition for matrix multiplication would

### Solutions to Math 51 First Exam January 29, 2015

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

### Matrices, Determinants and Linear Systems

September 21, 2014 Matrices A matrix A m n is an array of numbers in rows and columns a 11 a 12 a 1n r 1 a 21 a 22 a 2n r 2....... a m1 a m2 a mn r m c 1 c 2 c n We say that the dimension of A is m n (we

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### 5. Orthogonal matrices

L Vandenberghe EE133A (Spring 2016) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal

### Review Jeopardy. Blue vs. Orange. Review Jeopardy

Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round \$200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?

### 1 2 3 1 1 2 x = + x 2 + x 4 1 0 1

(d) If the vector b is the sum of the four columns of A, write down the complete solution to Ax = b. 1 2 3 1 1 2 x = + x 2 + x 4 1 0 0 1 0 1 2. (11 points) This problem finds the curve y = C + D 2 t which

### Basic Quantum Mechanics Prof. Ajoy Ghatak Department of Physics. Indian Institute of Technology, Delhi

Basic Quantum Mechanics Prof. Ajoy Ghatak Department of Physics. Indian Institute of Technology, Delhi Module No. # 02 Simple Solutions of the 1 Dimensional Schrodinger Equation Lecture No. # 7. The Free

### Linear Algebra: Determinants, Inverses, Rank

D Linear Algebra: Determinants, Inverses, Rank D 1 Appendix D: LINEAR ALGEBRA: DETERMINANTS, INVERSES, RANK TABLE OF CONTENTS Page D.1. Introduction D 3 D.2. Determinants D 3 D.2.1. Some Properties of

### Solution. Area(OABC) = Area(OAB) + Area(OBC) = 1 2 det( [ 5 2 1 2. Question 2. Let A = (a) Calculate the nullspace of the matrix A.

Solutions to Math 30 Take-home prelim Question. Find the area of the quadrilateral OABC on the figure below, coordinates given in brackets. [See pp. 60 63 of the book.] y C(, 4) B(, ) A(5, ) O x Area(OABC)

### Lecture Notes: Matrix Inverse. 1 Inverse Definition

Lecture Notes: Matrix Inverse Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Inverse Definition We use I to represent identity matrices,

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Solutions to Linear Algebra Practice Problems 1. form (because the leading 1 in the third row is not to the right of the

Solutions to Linear Algebra Practice Problems. Determine which of the following augmented matrices are in row echelon from, row reduced echelon form or neither. Also determine which variables are free

### 8 Square matrices continued: Determinants

8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### LS.6 Solution Matrices

LS.6 Solution Matrices In the literature, solutions to linear systems often are expressed using square matrices rather than vectors. You need to get used to the terminology. As before, we state the definitions

### A linear combination is a sum of scalars times quantities. Such expressions arise quite frequently and have the form

Section 1.3 Matrix Products A linear combination is a sum of scalars times quantities. Such expressions arise quite frequently and have the form (scalar #1)(quantity #1) + (scalar #2)(quantity #2) +...

### Situation 23: Simultaneous Equations Prepared at the University of Georgia EMAT 6500 class Date last revised: July 22 nd, 2013 Nicolina Scarpelli

Situation 23: Simultaneous Equations Prepared at the University of Georgia EMAT 6500 class Date last revised: July 22 nd, 2013 Nicolina Scarpelli Prompt: A mentor teacher and student teacher are discussing

### 1 Scalars, Vectors and Tensors

DEPARTMENT OF PHYSICS INDIAN INSTITUTE OF TECHNOLOGY, MADRAS PH350 Classical Physics Handout 1 8.8.2009 1 Scalars, Vectors and Tensors In physics, we are interested in obtaining laws (in the form of mathematical

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### Lecture 1: Schur s Unitary Triangularization Theorem

Lecture 1: Schur s Unitary Triangularization Theorem This lecture introduces the notion of unitary equivalence and presents Schur s theorem and some of its consequences It roughly corresponds to Sections

### Linear Algebra Review Part 2: Ax=b

Linear Algebra Review Part 2: Ax=b Edwin Olson University of Michigan The Three-Day Plan Geometry of Linear Algebra Vectors, matrices, basic operations, lines, planes, homogeneous coordinates, transformations

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### MATHEMATICS FOR ENGINEERS BASIC MATRIX THEORY TUTORIAL 2

MATHEMATICS FO ENGINEES BASIC MATIX THEOY TUTOIAL This is the second of two tutorials on matrix theory. On completion you should be able to do the following. Explain the general method for solving simultaneous

### We know a formula for and some properties of the determinant. Now we see how the determinant can be used.

Cramer s rule, inverse matrix, and volume We know a formula for and some properties of the determinant. Now we see how the determinant can be used. Formula for A We know: a b d b =. c d ad bc c a Can we

### Mehtap Ergüven Abstract of Ph.D. Dissertation for the degree of PhD of Engineering in Informatics

INTERNATIONAL BLACK SEA UNIVERSITY COMPUTER TECHNOLOGIES AND ENGINEERING FACULTY ELABORATION OF AN ALGORITHM OF DETECTING TESTS DIMENSIONALITY Mehtap Ergüven Abstract of Ph.D. Dissertation for the degree

### Generalized Inverse Computation Based on an Orthogonal Decomposition Methodology.

International Conference on Mathematical and Statistical Modeling in Honor of Enrique Castillo. June 28-30, 2006 Generalized Inverse Computation Based on an Orthogonal Decomposition Methodology. Patricia