# SOLVING LINEAR SYSTEMS

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis of some other problem As examples of the latter, we have the numerical solution of systems of nonlinear equations, ordinary and partial differential equations, integral equations, and the solution of optimization problems There are many ways of classifying linear systems Size: Small, moderate and large This of course varies with the machine you are using Most PCs are now being sold with a memory of 64 to 18 megabytes (MB), my own HP workstation has 768 MB, and the Weeg computer center s SGI shared memory parallel computer has a memory of gigabytes (GB)

2 For a matrix A of order n n, it will take 8n bytes to store it in double precision Thus a matrix of order 4000 will need around 18 MB of storage The latter would be too large for most present day PCs, if the matrix was to be stored in the computer s memory Sparse vs Dense Many linear systems have a matrix A in which almost all the elements are zero These matrices are said to be sparse For example, it is quite common to work with tridiagonal matrices A = a 1 c b a c 0 0 b 3 a 3 c 3 0 b n a n in which the order is 10 4 or much more For such matrices, it does not make sense to store the zero elements; and the sparsity should be taken into account when solving the linear system Ax = b Also, the sparsity need not be as regular as in this example

3 Special properties: Often the matrix will have some other special properties For example, A might be symmetric, tridiagonal, banded, or positive definite, a i,j =0 for i j >p (Ax, x) > 0 for all x C n,x = 0 All of these properties can be used to develop more efficient and accurate methods for solving Ax = b

4 METHODS OF SOLUTION There are two general categories of numerical methods for solving Ax = b Direct Methods: These are methods with a finite number of steps; and they end with the exact solution x, provided that all arithmetic operations are exact The most used of these methods is Gaussian elimination, which we will begin with There are other direct methods, and we will study them later in connection with solving the matrix eigenvalue problem Iteration Methods: These are used in solving all types of linear systems, but they are most commonly used with large sparse systems, especially those produced by discretizing partial differential equations This is an extremely active area of research

5 SOLVING LINEAR SYSTEMS We want to solve the linear system Ax = b: a 1,1 a 1,n a n,1 a n,n x 1 x n = b 1 b n This will be done by the method used in beginning algebra, by successively eliminating unknowns from equations, until eventually we have only one equation in one unknown This process is known as Gaussian elimination To help us keep track of the steps of this process, we will denote the initial system by A (1) x = b (1) : a (1) 1,1 a(1) 1,n a (1) n,1 a(1) n,n x 1 x n = b (1) 1 b (1) n Initially we will make the assumption that every pivot element will be nonzero; and later we remove this assumption

6 Step 1 We will eliminate x 1 from equations thru n Begin by defining the multipliers m i,1 = a(1) i,1 a (1) 1,1, i =,, n Here we are assuming the pivot element a (1) 1,1 0 Then in succession, multiply m i,1 times row 1 (called the pivot row) and subtract the result from row i This yields new matrix elements a () i,j = a(1) i,j m i,1a (1) 1,j, j =,, n b () i = b (1) i m i,1 b (1) 1 for i =,, n Note that the index j does not include j =1 The reason is that with the definition of the multiplier m i,1, it is automatic that a () i,1 = a(1) i,1 m i,1a (1) 1,1 =0, i =,, n

7 The linear system is now denoted by A () x = b () : a (1) 1,1 a(1) 1, a(1) 1,n 0 a (), a (),n 0 a () n, a() n,n x 1 x x n = b (1) 1 b () b () n Step k: Assume that for i =1,, k 1 the unknown x i has been eliminated from equations i +1 thru n We have the system A (k) x = b (k) a (1) 1,1 a(1) 1, a (1) 1,n 0 a (), a (),n 0 a (k) k,k 0 0 a (k) n,k a(k) k,n a(k) n,n x 1 x x n = b (1) 1 b () b (k) ḳ b (k) n We want to eliminate unknown x k from equations k +1thrun

8 Begin by defining the multipliers m i,k = a(k) i,k a (k) k,k, i = k +1,, n The pivot element is a (k) k,k, and we assume it is nonzero Using these, we eliminate x k from equations k +1 thru n Multiply m i,k times row k (the pivot row) and subtract from row i, fori = k +1 thrun a (k+1) i,j = a (k) i,j m i,ka (k) k,j, b (k+1) i for i = k +1,, n A (k) x = b (k) : = b (k) i j = k +1,, n m i,k b (k) k This yields the linear system

9 a (1) 1,1 a (1) 1,n 0 a (k) k,k a (k) k,k+1 a (k) k,n 0 a (k+1) k+1,k+1 a (k+1) k+1,n 0 0 a (k+1) n,k+1 a (k+1) n,n = x 1 x k x k+1 x n b (1) 1 b (k) k b (k+1) k+1 b (k+1) n Doing this for k =1,,, n leads to an upper triangular system of linear equations A (n) x = b (n) : a (1) 1,1 a(1) 1,n a (n) n,n x 1 x n = b (1) 1 b (n) n

10 At each step, we have assumed for the pivot element that a (k) k,k 0 Later we remove this assumption For now, we consider the solution of the upper triangular system A (n) x = b (n) To avoid the superscripts, and to emphasize we are now solving an upper triangular system, denote this system by Ux = g u 1,1 u 1,n u n,n x 1 x n = g 1 g n This is the linear system u 1,1 x 1 + u 1, x + + u 1,n 1 x n 1 + u 1,n x n = g 1 u n 1,n 1 x n 1 + u n 1,n x n = g n 1 u n,n x n = g n

11 u 1,1 x 1 + u 1, x + + u 1,n 1 x n 1 + u 1,n x n = g 1 u n 1,n 1 x n 1 + u n 1,n x n = g n 1 u n,n x n = g n We solve for x n, then x n 1, and backwards to x 1 This process is called back substitution x n = g n u n,n u k = g k { } u k,k+1 x k u k,n x n u k,k for k = n 1,, 1 Examples of the process of producing A (n) x = b (n) and of solving Ux = g are given in the text They are simply a more carefully defined and methodical version of what you have done in high school algebra

12 QUESTIONS How do we remove the assumption on the pivot elements? How many operations are involved in this procedure? How much error is there in the computed solution due to rounding errors in the calculations? Later (if there is time), how does the machine architecture affect the implementation of this algorithm Before doing this, we consider a result on factoring matrices that is associated with Gaussian elimination

13 THE LU FACTORIZATION Using the multipliers from Gaussian elimination, define the lower triangular matrix L = m,1 1 0 m 3,1 m 3, 1 m n,1 m n, m n,n 1 1 Recall the upper triangular matrix U = A (n) Then A = LU This is called the LU-factorization of the matrix A

14 Proof The proof is obtained by looking at the (i, j) element of LU, (LU) i,j = [ m i,1,, m i,i 1, 1, 0,, 0 ] u 1,j u j,j 0 We consider separately the cases of i j and i>j With the first case, we have more nonzero elements in the column U,j than in the row L i, Thus (LU) i,j = m i,1 u 1,j + + m i,i 1 u i 1,j + u i,j The elements u i,j are given by u i,j = a (i) i,j, j = i,, n, i =1,, n 0

15 Recall that a () i,j = a(1) i,j m i,1a (1) 1,j, j =,, n for i =,, n Then Similarly, m i,1 u 1,j = m i,1 a (1) 1,j = a(1) i,j a() i,j m i, u,j = m i, a (),j = a () m i,i 1 u i 1,j = m i,i 1 a (i 1) i 1,j = a (i 1) i,j Combining these results, i,j a(3) i,j a (i) i,j (LU) i,j = m i,1 u 1,j + + m i,i 1 u i 1,j + u i,j = ( a (1) ) i,j a() i,j + + = a (1) i,j = a i,j + ( a (i 1) i,j ( a () ) i,j a(3) i,j ) a (i) i,j + a (i) i,j The second case, in which i>j, is given in the text

16 EXAMPLE In the text, we consider the linear system Gaussian elimination leads to x 1 x x 3 = 0 3 with multipliers x 1 x x 3 = m,1 =, m 3,1 = 1, m 3, = 1 Then =

17 In Matlab, you can find the LU factorization of a matrix using the function lu For example, [L U P]=lu(A) yields a lower triangular matrix L, an upper triangular matrix U, andapermuation matrix P,forwhich LU = PA The reason for this arises in a modification of Gaussian elimination, one which eliminates our assumption regarding the pivot elements being nonzero and simultaneously improves the error behaviour of Gaussian elimination

18 AN OPERATIONS COUNT How many operations are expended in solving Ax = b using Gaussian elimination? To calculate this, we must return to the formal definitions, of both A (k) x = b (k) and the solution of Ux = g Step 1 Multipliers: n 1 divisions A (1) A () :(n 1) additions and multiplications [Note that we consider additions and subtractions as the same type of operation] b (1) b () : n 1 additions and multiplications Step Multipliers: n divisions A () A (3) :(n ) additions and multiplications b () b (3) : n additions and multiplications

19 Step n 1 Multipliers: 1 division A (n) A (n 1) : 1 addition and multiplication b (n 1) b (n) : 1 addition and multiplication Thus we have the following totals for the transformation of A to U = A (n) Divisions (n 1) + (n ) + +1 Additions (n 1) +(n ) + +1 Multiplications (n 1) +(n ) + +1 To add these up, we need the following general formulas p j=1 j = p(p +1) p j=1 j = p(p + 1)(p +1) 6 These can proven using mathematical induction

20 With these, the number of operations for A U are as follows n(n 1) Divisions n(n 1)(n 1) Additions 6 n(n 1)(n 1) Multiplications 6 Similarly, we can calulate the cost of b (1) b (n) as Additions Multiplications The cost of solving Ux = g is n(n 1) n(n 1) Divisions Additions Multiplications n n(n 1) n(n 1)

21 On some machines, the cost of a division is much more than that of a multiplication; whereas on others there is not any important difference For this last case, we usually combine the additions and subtractions to obtain the following costs MD(A U) = n ( n 1 ) MD(b (1) g) = 3 n(n 1) MD(Find x) = n(n +1) For additions and subtractions, we have AS(A U) = AS(b (1) g) = AS(Find x) = n(n 1)(n 1) 6 n(n 1) n(n 1)

22 Summarizing, the cost of factoring the matrix A into L and U involves approximately 1 3 n3 multiplications and divisions, and approximately 1 3 n3 additions and subtractions The total number of arithmetic operationsisapproximately 3 n3 Thus as the size of a linear system is doubled, the cost of factoring the matrix increases by a factor of about 8 Once the matrix has been factored, the cost of modifying the right side b to g, and of then solving for x, is MD(b x) = n AS(b x) = n(n 1) = n Thus the principal cost is the solution of a linear system Ax = b is in the factorization of the matrix A into LU; and the subsequent solving for x is much smaller if n is a moderately large number

23 We often wish to solve linear systems Ax = b with varying vectors b, and A unchanged For such cases, it is important to keep the information used in producing L and U, as this this is the costly portion of the solution process In the machine, store the elements u i,j and m i,j into the portions of A with the same subscripts This can be done, as those positions are no longer being changed; and for the portion below the diagonal, those elements are being set to zero in A To solve m systems Ax (l) = b (l), l =1,, m for m different right hand vectors b (l), the operations cost will be MD = n ( n 1 ) + mn = n 3 + mn n(n 1)(n 1) AS = + mn(n 1) = 1 3 n3 + mn

24 CALCULATING AN INVERSE To find the inverse of a matrix A amounts to solving the equation In partioned form, AX = I A[X,1,, X,n ]= [ e (1),, e (n)] which yields the n equivalent linear systems AX,k = e (k), k =1,, n Thus the cost of calculating the inverse X = A 1 is approximately 3 n3 +n 3 = 8 3 n3 This is only around 4 times the cost of solving a single system Ax = b

### LINEAR SYSTEMS. Consider the following example of a linear system:

LINEAR SYSTEMS Consider the following example of a linear system: Its unique solution is x +2x 2 +3x 3 = 5 x + x 3 = 3 3x + x 2 +3x 3 = 3 x =, x 2 =0, x 3 = 2 In general we want to solve n equations in

### Solution of Linear Systems

Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start

### Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### Direct Methods for Solving Linear Systems. Linear Systems of Equations

Direct Methods for Solving Linear Systems Linear Systems of Equations Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University

### Practical Numerical Training UKNum

Practical Numerical Training UKNum 7: Systems of linear equations C. Mordasini Max Planck Institute for Astronomy, Heidelberg Program: 1) Introduction 2) Gauss Elimination 3) Gauss with Pivoting 4) Determinants

### Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix multiplication).

MAT 2 (Badger, Spring 202) LU Factorization Selected Notes September 2, 202 Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix

### Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold:

Linear Systems Example: Find x, x, x such that the following three equations hold: x + x + x = 4x + x + x = x + x + x = 6 We can write this using matrix-vector notation as 4 {{ A x x x {{ x = 6 {{ b General

### 6. Cholesky factorization

6. Cholesky factorization EE103 (Fall 2011-12) triangular matrices forward and backward substitution the Cholesky factorization solving Ax = b with A positive definite inverse of a positive definite matrix

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

### Operation Count; Numerical Linear Algebra

10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floating-point

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### 7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix

7. LU factorization EE103 (Fall 2011-12) factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization algorithm effect of rounding error sparse

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### SCHOOL OF MATHEMATICS MATHEMATICS FOR PART I ENGINEERING. Self Study Course

SCHOOL OF MATHEMATICS MATHEMATICS FOR PART I ENGINEERING Self Study Course MODULE 17 MATRICES II Module Topics 1. Inverse of matrix using cofactors 2. Sets of linear equations 3. Solution of sets of linear

### (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular.

Theorem.7.: (Properties of Triangular Matrices) (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. (b) The product

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Solving Sets of Equations. 150 B.C.E., 九章算術 Carl Friedrich Gauss,

Solving Sets of Equations 5 B.C.E., 九章算術 Carl Friedrich Gauss, 777-855 Gaussian-Jordan Elimination In Gauss-Jordan elimination, matrix is reduced to diagonal rather than triangular form Row combinations

### 9. Numerical linear algebra background

Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

### 6 Gaussian Elimination

G1BINM Introduction to Numerical Methods 6 1 6 Gaussian Elimination 61 Simultaneous linear equations Consider the system of linear equations a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 +

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Cofactor Expansion: Cramer s Rule

Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating

### e-issn: p-issn: X Page 67

International Journal of Computer & Communication Engineering Research (IJCCER) Volume 2 - Issue 2 March 2014 Performance Comparison of Gauss and Gauss-Jordan Yadanar Mon 1, Lai Lai Win Kyi 2 1,2 Information

### Lecture 5 - Triangular Factorizations & Operation Counts

LU Factorization Lecture 5 - Triangular Factorizations & Operation Counts We have seen that the process of GE essentially factors a matrix A into LU Now we want to see how this factorization allows us

### 1.5 Elementary Matrices and a Method for Finding the Inverse

.5 Elementary Matrices and a Method for Finding the Inverse Definition A n n matrix is called an elementary matrix if it can be obtained from I n by performing a single elementary row operation Reminder:

### Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants

Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants Hartmut Führ fuehr@matha.rwth-aachen.de Lehrstuhl A für Mathematik, RWTH Aachen October 30, 2008 Overview

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### 10.2 ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS. The Jacobi Method

578 CHAPTER 1 NUMERICAL METHODS 1. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS As a numerical technique, Gaussian elimination is rather unusual because it is direct. That is, a solution is obtained after

### General Framework for an Iterative Solution of Ax b. Jacobi s Method

2.6 Iterative Solutions of Linear Systems 143 2.6 Iterative Solutions of Linear Systems Consistent linear systems in real life are solved in one of two ways: by direct calculation (using a matrix factorization,

### Matrix Inverse and Determinants

DM554 Linear and Integer Programming Lecture 5 and Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1 2 3 4 and Cramer s rule 2 Outline 1 2 3 4 and

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M 1 such that

0. Inverse Matrix Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M such that M M = I = M M. Inverse of a 2 2 Matrix Let M and N be the matrices: a b d b M =, N = c

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 4. Gaussian Elimination In this part, our focus will be on the most basic method for solving linear algebraic systems, known as Gaussian Elimination in honor

### Further Maths Matrix Summary

Further Maths Matrix Summary A matrix is a rectangular array of numbers arranged in rows and columns. The numbers in a matrix are called the elements of the matrix. The order of a matrix is the number

### Linear Systems COS 323

Linear Systems COS 323 Last time: Constrained Optimization Linear constrained optimization Linear programming (LP) Simplex method for LP General optimization With equality constraints: Lagrange multipliers

### Jones and Bartlett Publishers, LLC. NOT FOR SALE OR DISTRIBUTION

8498_CH08_WilliamsA.qxd 11/13/09 10:35 AM Page 347 Jones and Bartlett Publishers, LLC. NOT FOR SALE OR DISTRIBUTION C H A P T E R Numerical Methods 8 I n this chapter we look at numerical techniques for

### Using the Singular Value Decomposition

Using the Singular Value Decomposition Emmett J. Ientilucci Chester F. Carlson Center for Imaging Science Rochester Institute of Technology emmett@cis.rit.edu May 9, 003 Abstract This report introduces

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### 2.5 Elementary Row Operations and the Determinant

2.5 Elementary Row Operations and the Determinant Recall: Let A be a 2 2 matrtix : A = a b. The determinant of A, denoted by det(a) c d or A, is the number ad bc. So for example if A = 2 4, det(a) = 2(5)

### Doolittle Decomposition of a Matrix

Doolittle Decomposition of a Matrix It is always possible to factor a square matrix into a lower triangular matrix and an upper triangular matrix. That is, [A] = [L][U] EXAMPLE: = 2 2 2 4 Having the LU

### 1.2 Solving a System of Linear Equations

1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables

### Solving Linear Systems of Equations. Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.

Solving Linear Systems of Equations Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.edu These slides are a supplement to the book Numerical Methods with Matlab:

Facts About Eigenvalues By Dr David Butler Definitions Suppose A is an n n matrix An eigenvalue of A is a number λ such that Av = λv for some nonzero vector v An eigenvector of A is a nonzero vector v

### Determinants. Dr. Doreen De Leon Math 152, Fall 2015

Determinants Dr. Doreen De Leon Math 52, Fall 205 Determinant of a Matrix Elementary Matrices We will first discuss matrices that can be used to produce an elementary row operation on a given matrix A.

### (57) (58) (59) (60) (61)

Module 4 : Solving Linear Algebraic Equations Section 5 : Iterative Solution Techniques 5 Iterative Solution Techniques By this approach, we start with some initial guess solution, say for solution and

### Math 315: Linear Algebra Solutions to Midterm Exam I

Math 35: Linear Algebra s to Midterm Exam I # Consider the following two systems of linear equations (I) ax + by = k cx + dy = l (II) ax + by = 0 cx + dy = 0 (a) Prove: If x = x, y = y and x = x 2, y =

### ( % . This matrix consists of \$ 4 5 " 5' the coefficients of the variables as they appear in the original system. The augmented 3 " 2 2 # 2 " 3 4&

Matrices define matrix We will use matrices to help us solve systems of equations. A matrix is a rectangular array of numbers enclosed in parentheses or brackets. In linear algebra, matrices are important

### MATH 2030: SYSTEMS OF LINEAR EQUATIONS. ax + by + cz = d. )z = e. while these equations are not linear: xy z = 2, x x = 0,

MATH 23: SYSTEMS OF LINEAR EQUATIONS Systems of Linear Equations In the plane R 2 the general form of the equation of a line is ax + by = c and that the general equation of a plane in R 3 will be we call

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### 1 Gaussian Elimination

Contents 1 Gaussian Elimination 1.1 Elementary Row Operations 1.2 Some matrices whose associated system of equations are easy to solve 1.3 Gaussian Elimination 1.4 Gauss-Jordan reduction and the Reduced

### Eigenvalues and Eigenvectors

Chapter 6 Eigenvalues and Eigenvectors 6. Introduction to Eigenvalues Linear equations Ax D b come from steady state problems. Eigenvalues have their greatest importance in dynamic problems. The solution

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix.

MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix. Matrices Definition. An m-by-n matrix is a rectangular array of numbers that has m rows and n columns: a 11

### Solving Systems of Linear Equations. Substitution

Solving Systems of Linear Equations There are two basic methods we will use to solve systems of linear equations: Substitution Elimination We will describe each for a system of two equations in two unknowns,

### Using row reduction to calculate the inverse and the determinant of a square matrix

Using row reduction to calculate the inverse and the determinant of a square matrix Notes for MATH 0290 Honors by Prof. Anna Vainchtein 1 Inverse of a square matrix An n n square matrix A is called invertible

### 10.3 POWER METHOD FOR APPROXIMATING EIGENVALUES

55 CHAPTER NUMERICAL METHODS. POWER METHOD FOR APPROXIMATING EIGENVALUES In Chapter 7 we saw that the eigenvalues of an n n matrix A are obtained by solving its characteristic equation n c n n c n n...

### 2.1: Determinants by Cofactor Expansion. Math 214 Chapter 2 Notes and Homework. Evaluate a Determinant by Expanding by Cofactors

2.1: Determinants by Cofactor Expansion Math 214 Chapter 2 Notes and Homework Determinants The minor M ij of the entry a ij is the determinant of the submatrix obtained from deleting the i th row and the

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### Matrix Differentiation

1 Introduction Matrix Differentiation ( and some other stuff ) Randal J. Barnes Department of Civil Engineering, University of Minnesota Minneapolis, Minnesota, USA Throughout this presentation I have

### We seek a factorization of a square matrix A into the product of two matrices which yields an

LU Decompositions We seek a factorization of a square matrix A into the product of two matrices which yields an efficient method for solving the system where A is the coefficient matrix, x is our variable

### 5.4 Gaussian Elimination and Its Tri-Diagonal Version

140 A Theoretical Introduction to Numerical Analysis 2 Prove that for a square matrix A and its transpose A T, the following equalities hold: μ (A)=μ 1 (A T ), μ 1 (A)=μ (A T ) 3 Show that the condition

### Text. 7. Summary. A. Gilat, MATLAB: An Introduction with Applications, 4th ed., Wiley Wikipedia: Matrix, SLE, etc.

2. Linear algebra 1. Matrix operations 2. Determinant and matrix inverse 3. Advanced matrix manipulations in the MATLAB 4. Systems of linear equations (SLEs). Solution of a SLE by the Gauss elimination

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if

### Torgerson s Classical MDS derivation: 1: Determining Coordinates from Euclidean Distances

Torgerson s Classical MDS derivation: 1: Determining Coordinates from Euclidean Distances It is possible to construct a matrix X of Cartesian coordinates of points in Euclidean space when we know the Euclidean

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### Numerical Methods for Civil Engineers. Linear Equations

Numerical Methods for Civil Engineers Lecture Notes CE 311K Daene C. McKinney Introduction to Computer Methods Department of Civil, Architectural and Environmental Engineering The University of Texas at

### Introduction to Matrix Algebra I

Appendix A Introduction to Matrix Algebra I Today we will begin the course with a discussion of matrix algebra. Why are we studying this? We will use matrix algebra to derive the linear regression model

### Partitioned Matrices and the Schur Complement

P Partitioned Matrices and the Schur Complement P 1 Appendix P: PARTITIONED MATRICES AND THE SCHUR COMPLEMENT TABLE OF CONTENTS Page P1 Partitioned Matrix P 3 P2 Schur Complements P 3 P3 Block Diagonalization

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Matrix Algebra and Applications

Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2 - Matrices and Matrix Algebra Reading 1 Chapters

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Numerical Methods Lecture 2 Simultaneous Equations

Numerical Methods Lecture 2 Simultaneous Equations Topics: matrix operations solving systems of equations Matrix operations: Mathcad is designed to be a tool for quick and easy manipulation of matrix forms

### 1 Determinants. Definition 1

Determinants The determinant of a square matrix is a value in R assigned to the matrix, it characterizes matrices which are invertible (det 0) and is related to the volume of a parallelpiped described

### Lecture 1: Systems of Linear Equations

MTH Elementary Matrix Algebra Professor Chao Huang Department of Mathematics and Statistics Wright State University Lecture 1 Systems of Linear Equations ² Systems of two linear equations with two variables

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### Solving linear systems. Solving linear systems p. 1

Solving linear systems Solving linear systems p. 1 Overview Chapter 12 from Michael J. Quinn, Parallel Programming in C with MPI and OpenMP We want to find vector x = (x 0,x 1,...,x n 1 ) as solution of

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### Inverses and powers: Rules of Matrix Arithmetic

Contents 1 Inverses and powers: Rules of Matrix Arithmetic 1.1 What about division of matrices? 1.2 Properties of the Inverse of a Matrix 1.2.1 Theorem (Uniqueness of Inverse) 1.2.2 Inverse Test 1.2.3

### MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

### MATH 105: Finite Mathematics 2-6: The Inverse of a Matrix

MATH 05: Finite Mathematics 2-6: The Inverse of a Matrix Prof. Jonathan Duncan Walla Walla College Winter Quarter, 2006 Outline Solving a Matrix Equation 2 The Inverse of a Matrix 3 Solving Systems of

### Methods for Finding Bases

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,

### 2. Perform elementary row operations to get zeros below the diagonal.

Gaussian Elimination We list the basic steps of Gaussian Elimination, a method to solve a system of linear equations. Except for certain special cases, Gaussian Elimination is still state of the art. After

### Numerical Methods for Solving Systems of Nonlinear Equations

Numerical Methods for Solving Systems of Nonlinear Equations by Courtney Remani A project submitted to the Department of Mathematical Sciences in conformity with the requirements for Math 4301 Honour s

### Linear Algebraic and Equations

Next: Optimization Up: Numerical Analysis for Chemical Previous: Roots of Equations Subsections Gauss Elimination Solving Small Numbers of Equations Naive Gauss Elimination Pitfalls of Elimination Methods

### Solving Systems of Linear Equations; Row Reduction

Harvey Mudd College Math Tutorial: Solving Systems of Linear Equations; Row Reduction Systems of linear equations arise in all sorts of applications in many different fields of study The method reviewed

### Elementary Matrices and The LU Factorization

lementary Matrices and The LU Factorization Definition: ny matrix obtained by performing a single elementary row operation (RO) on the identity (unit) matrix is called an elementary matrix. There are three

### MAT188H1S Lec0101 Burbulla

Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u

### We know a formula for and some properties of the determinant. Now we see how the determinant can be used.

Cramer s rule, inverse matrix, and volume We know a formula for and some properties of the determinant. Now we see how the determinant can be used. Formula for A We know: a b d b =. c d ad bc c a Can we

### Topic 1: Matrices and Systems of Linear Equations.

Topic 1: Matrices and Systems of Linear Equations Let us start with a review of some linear algebra concepts we have already learned, such as matrices, determinants, etc Also, we shall review the method