Practical Numerical Training UKNum


 Crystal Leonard
 2 years ago
 Views:
Transcription
1 Practical Numerical Training UKNum 7: Systems of linear equations C. Mordasini Max Planck Institute for Astronomy, Heidelberg Program: 1) Introduction 2) Gauss Elimination 3) Gauss with Pivoting 4) Determinants 5) LU Decomposition 6) Inverse of a matrix with LU
2 1 Introduction
3 Task One of the most important numerical tasks: Solve a system of linear equations. a 11 x 1 + a 12 x 2 + a 13 x a 1N x N = b 1 a 21 x 1 + a 22 x 2 + a 23 x a 2N x N = b 2 a 31 x 1 + a 32 x 2 + a 33 x a 3N x N = b 3 a M1 x 1 + a M2 x 2 + a M3 x a MN x N = b M N unknowns: x j, j =1, 2,...,N M equations, =12 with the known coefficients = a ij with i =1, 2,...,M and j =1, 2,...,N =12 And known right hand side b i, i =1, 2,...,M.
4 (Non)Singular systems If N=M: same number of equations and unknowns. Chance to find a unique solution. But not always: One or several of the M equations is a linear combination of the other ones (row degeneracy). All equations contain certain variables only in exactly the same linear combination (column degeneracy). For square matrixes, one implies the other. Such matrixes are called singular. This is an analytical problem. Additional conditions arise from the numerical treatment of the task.
5 Numerical issues While not exact linear combinations of each other, some of the equations may be so close to linearly dependent that roundoff errors in the machine render them linearly dependent at some stage in the solution process. In this case your numerical procedure will fail, even if there would be an analytical solution. Accumulated roundoff errors in the solution process can swamp the true solution. This problem particularly emerges if N is too large. The numerical procedure does not fail algorithmically. However, it returns a set of x s that are wrong, as can be discovered by direct substitution back into the original equations. The closer a set of equations is to being singular, the more likely this is to happen, since increasingly close cancellations will occur during the solution. In fact, the preceding item can be viewed as the special case where the loss of significance is unfortunately total. Press et al.
6 Matrix form Our master equation is usually written in the form of a matrix equation: A x = b A is the matrix of coefficients, and b is the righthand side written as a column vector A = a 11 a a 1N a 21 a a 2N a M1 a M2... a MN b = b 1 b 2 b M By convention, the first index on an element aij denotes its row, the second index its column.
7 Special matrixes In this context, several special types of matrixes are important: 1) Unit matrix A diagonal matrix with all diagonal elements equal to one is called an identity/unit matrix. 2)Upper triangular matrix All the elements below the diagonal entries are zero.
8 Special matrixes II 3)Lower triangular matrix All the elements above the diagonal entries are zero. 4)Tridiagonal matrix A tridiagonal matrix is a square matrix in which all elements not on the following are zero  the major diagonal, the diagonal above the major diagonal, and the diagonal below the major diagonal.
9 Other special matrixes In many situations (e.g. solutions of partial differential equations in two or more dimensions), one is dealing with so called sparse linear systems, i.e. matrixes where only a small fraction of all matrix elements are nonzero. For such matrixes, one should use specialized solvers, as they can be much more rapid than general purpose solvers. zeros zeros zeros (a) (b) (c) (d) (e) (f ) For large systems (N~ ), one wants to consider prewritten specialized libraries, like LAPACK which is freely available. (g) (h) (i) ( j) (k) Press et al.
10 Methods to solve In this lecture, we consider the following methods: Naive Gaussian Elimination Gaussian Elimination with Partial Pivoting LU Decomposition (Gauss Seidel Iteration) For large systems (N~ ), one wants to consider prewritten specialized libraries, like LAPACK which is freely available. They have also been parallelized and ported onto GPUs.
11 2 Gaussian Elimination
12 Naive Gaussian elimination Two step procedure: Forward elimination: In this step, an unknown is eliminated in each equation starting with the first equation. This way, the equations are reduced to one unknown in each equation. Back Substitution: In this step, starting from the last equation, each of the unknowns is found.
13 Forward elimination goal The goal of forward elimination is to transform the coefficient matrix into an upper triangular matrix:
14 Initial equations A set of n equations and n unknowns To bring this to the form of an upper triangular matrix, (n1) steps of forward elimination are needed.
15 Forward elimination step 1 a In the first step of forward elimination, the first unknown, x1 is eliminated from all rows below the first row. The first equation is selected as the pivot equation to eliminate x1. So, to eliminate x1 in the second equation, one divides the first equation by a11 (hence called the pivot element) and then multiplies it by a21. This is the same as multiplying the first equation by a21/a11 to give
16 Forward elimination step 1 b Now, this equation can be subtracted from the second equation to give Note that we have eliminated in the new second equation x1.
17 Forward elimination step 1 c We can write this new second equation also as where a " = a a 21 a a a " = a a 21 2n 2n a 11 a 1n
18 Forward elimination step 1 d Repeat this procedure of eliminating x1 for the remaining equations (3 to n) to reduce the set of equations as This is the end of step 1.
19 Forward elimination step 2 For the second step of forward elimination, we start with the second equation as the pivot equation and a 22 as the pivot element. To eliminate x2 in the third equation, one thus divides the second equation by a 22 (the pivot element) and then multiply it by a 32. Then we subtract this from the third equation. This makes the coefficient of x2 zero in the third equation. The same procedure is now repeated for the fourth equation until the n th equation to give......
20 Forward elimination further steps The next steps of forward elimination are conducted by using the third equation as a pivot equation and so on. Thus, there will be a total of n1 steps of forward elimination. At the end of n1 steps of forward elimination, we get a set of equations that look like......
21 Back substitution Example of a system of 3 equations: # & % ( % ( $ % '( # % % $ % x 1 x 2 x 3 & ( ( '( = # & % ( % 96.21( $ % '( Solve each equation starting (obviously) from the last equation as it has only one unknown: We have now determined a first unknown quantity.
22 Back substitution II Then the second last equation, that is the (n1)th equation, has two unknowns: xn and xn1, but xn is already known from just before. This reduces the (n1)th equation also to one unknown. We now iteratively work up to the first equation. Back substitution hence can be represented for all equations by the formula x i = b ( i 1 ) ( i 1 i a ) ( i 1 i,i+1 x i+1 a ) ( i 1 i,i+2 x i+2... a ) i,n x n ( i 1) for i = n 1,...,1 a ii x i = ( ) i 1 a ij b i i 1 n j =i+1 i 1 a ii ( ) x j ( ) for i = n 1,...,1
23 Example I The upward velocity of a rocket is given at three different times The velocity data is approximated by a polynomial as: Find the velocity at t=6 seconds.
24 Example II We need to determine the unknown coefficients ai. This results in a matrix of the form (Vandermonde matrix) Using data from the table, the matrix becomes:
25 Example III We can also write this as: 1) Forward elimination Number of steps of forward elimination is (n 1)=(3 1)=2
26 Example IV Forward elimination step 1 Divide Equation 1 by 25 and multiply it by 64,. [ ] 2.56 = [ ] Subtract the result from Equation 2 Substitute new equation for Eq. 2
27 Example V Forward elimination step 1 continued Divide Equation 1 by 25 and multiply it by 144,. [ ] 5.76 = [ ] Subtract the result from Equation 3 Substitute new equation for Equation 3
28 Example VI Forward elimination step 2 Divide Equation 2 by 4.8 and multiply it by 16.8,! [ ] 3.5 = [ ] Subtract the result from Equation 3 Substitute new equation for Equation 3
29 Example VII Back substitution # & % ( % ( $ % '( # % % $ % a 1 a 2 a 3 & ( ( '( = # & % ( % ( $ % 0.76 '( Solving for a 3 Solving for a 2
30 Example VIII Back substitution continued # & % ( % ( $ % '( # % % $ % a 1 a 2 a 3 & ( ( '( = # & % ( % ( $ % 0.76 '( Solving for a 1
31 Example IX Solution The solution vector is The polynomial that passes through the three data points is then:
32 Possible issues There are two pitfalls of the Naive Gauss elimination method. 1) Division by zero Here, we cannot even start the algorithm as we would have to devise by zero. This problem can also occur later on:
33 Possible issues II Here we can start the algorithm. However, at the first step of forward elimination, we get the following equations in matrix form Now at the beginning of the 2nd step of forward elimination, the coefficient of x2 in Equation 2 would be used as the pivot element. That element is zero and hence would create the division by zero problem. Division by zero is a possibility at any step of forward elimination.
34 Possible issues III 2) Large roundoff errors The Naive Gauss elimination method is prone to roundoff errors. This is true when there are large numbers of equations as errors propagate. Also, if there is subtraction of numbers from each other, it may create large errors. Example: Exact Solution
35 Possible issues III Solve it on a computer using 6 significant digits with chopping. One finds Solve it on a computer using 5 significant digits with chopping Is there a way to reduce the round off error? Obviously, increase the number of significant digits (always use double precision). This Decreases roundoff error Does not avoid division by zero
36 3 Gaussian Elimination with Partial Pivoting
37 Partial pivoting Gaussian Elimination with Partial Pivoting Avoids division by zero Reduces round off error This method is an improvement of the algorithm we just saw before. What is Different About Partial Pivoting? At the beginning of the k th step of forward elimination, find the maximum of If the maximum of the values is in the p th row, then switch rows p and k. Otherwise, exactly the same algorithm as naive Gauss elimination except that we switch rows before each of the (n1) steps of forward elimination.
38 Switching rows Matrix Form at Beginning of 2 nd Step of Forward Elimination Switched Rows
39 Example We solve the following set of equations by Gaussian elimination with partial pivoting Number of steps of forward elimination is again (n 1)=(3 1)=2.
40 Example II Forward Elimination: Step 1 Examine absolute values of first column, first row and below. Largest absolute value is 144 and exists in row 3. Switch row 1 and row 3. Begin forward elimination Divide Equation 1 by 144 and multiply it by 64,.
41 Example III Subtract the result from Equation 2 [ ] = [ ] Substitute new equation for Equation 2 Equation 3 Divide Equation 1 by 144 and multiply it by 25,. [ ] = [ ]
42 Example IV Subtract the result from Equation 3 Substitute new equation for Equation 3 Forward Elimination: Step 2 Examine absolute values of second column, second row and below Largest absolute value is and exists in row 3. Switch row 2 and row 3.
43 Example V Divide Equation 2 by and multiply it by 2.667, [ ] = [ ] Subtract the. result from Equation 3 Substitute new equation for Equation 3
44 Example VI Back Substitution Solve exactly as before for a 3 then for a 2 and finally for a 1
45 Round off errors Comparison with/without pivoting We studied earlier the system Exact Solution We found for Gaussian Elimination without pivoting, and 5 significant digits With pivoting, and 5 digits, one finds in contrast the exact solution. The fact that the round off error is fully removed here is by coincidence only, the general trend of a smaller error is however not a coincidence.
46 4 Determinants of a Square Matrix
47 Determinants Using naive Gauss Eliminations to find the determinant of a square matrix One of the more efficient ways to find the determinant of a square matrix is by taking advantage of the following two theorems on a determinant of matrices coupled with Naive Gauss elimination. Theorem 1 Theorem 2 If a multiple of one row of [A] nxn is added or subtracted to another row of [A] nxn to result in [B] nxn then det(a)=det(b). The same is true for column operations also. The determinant of an upper triangular, lower triangular or diagonal matrix [A] nxn is given by
48 Determinants II This implies that if we apply the forward elimination steps of the Naive Gauss elimination method, the determinant of the matrix stays the same according to Theorem 1. Then since at the end of the forward elimination steps, the resulting matrix is upper triangular, the determinant will be given by Theorem 2.
49 Example Find the determinant of (rocket example) For a 3 x 3 Matrix we of course know directly that But we can also use (in particular for higher dimensions) that with Naive Gaussian Elimination, we get the upper triangular matrix. In this example, we found earlier
50 Example II Therefore, according to theorem 1 and 2, the determinant is = 25 ( 4.8) 0.7
51 Determinants and pivoting What if one cannot find the determinant of the matrix using the Naive Gauss elimination method, for example, because one gets division by zero problems during the Naïve Gauss elimination method? In this case, one can apply Gaussian elimination with partial pivoting. However, the determinant of the resulting upper triangular matrix may differ by a sign. The following theorem applies in addition to the previous two to find the determinant of a square matrix. Theorem 3 Let [A] nxn be a n x n matrix. Then, if [B] nxn is a matrix that results from switching one row with another row, then det(a)=det(b).
52 5 LU Decomposition
53 LU Decomposition LU Decomposition is another method to solve a set of simultaneous linear equations Which is better, Gauss Elimination or LU Decomposition? To answer this, a closer look at LU decomposition is needed.
54 LU Decomposition Method For most nonsingular matrix [A] that one could conduct Naive Gauss Elimination forward elimination steps, one can write it as where [L] = lower triangular matrix [U] = upper triangular matrix [A] = [L][U]
55 LU Decomposition idea Note: [Z] is a n x 1 matrix, i.e. a vector
56 LU Decomposition algorithm How can this be used? Given [A][X] = [C] 1. Decompose [A] into [L] and [U] 2. Solve [L][Z] = [C] for [Z] 3. Solve [U][X] = [Z] for [X] As we will see below, both solving equation 2 and 3 is very simple. Eq. 2 is solved using forward substitution and then we use Eq. 3 to calculate the solution vector [X] by back substitution.
57 Computational time To solve [A][X] = [B] Gaussian Elimination LU Decomposition T=clock cycle time n = size of the matrix So both methods are equally efficient. However, for calculating the inverse of a matrix, LU can be much faster (cf below)
58 Method: Decompose [A] to [L] and [U] [ A] = [ L] [ U] = " 1 0 0% " u u u $ ' $ $ ' $ 0 u u # $ &' # $ 0 0 u 33 % ' ' &' [U] is the same as the coefficient matrix at the end of the forward elimination step during Naive Gauss elimination. [L] is obtained using the multipliers that were used in the forward elimination process
59 Example: Find the [U] matrix I Using the Forward Elimination Procedure like for Gauss Elimination Step 1:
60 Example: Find the [U] matrix II Matrix after Step 1: Step 2:
61 Example: Find the [L] matrix I Using the multipliers used during the Forward Elimination Procedure From the first step of forward elimination
62 Example: Find the [L] matrix II From the second step of forward elimination The [L] matrix is thus obtained for free if we do Naive Gaussian Elimination.
63 Does [L][U] = [A]? [ L] [ U] = =? This can always be checked by simple matrix multiplication.
64 LU Decomposition to solve SLEs I Solve the following set of linear equations using LU Decomposition Step 1 Use the algorithm we just saw for finding the [L] and [U] matrices [ A] = [ L] [ U] =
65 LU Decomposition to solve SLEs II Step 2 Set [L][Z] = [C] and then solve for [Z]. Due to the particular shape of [L], this is very simple by forward substitution starting from the first equation: z 1 = z 1 + z 2 = z z 2 + z 3 = 279.2
66 LU Decomposition to solve SLEs III Step 2 continued Complete the forward substitution to solve for [Z]
67 LU Decomposition to solve SLEs IV Step 3 Set [U][X] = [Z] Solve for [X] The 3 equations become Again, due to the particular shape of [U], this is very simple by backward substitution starting from the last equation.
68 LU Decomposition to solve SLEs V Step 3 continued From the 3 rd equation Substituting in a 3 and using the second equation Substituting in a 3 and a 2 in the first equation Hence the Solution Vector [X] is:
69 6 Inverse of a Matrix with LU Decomposition
70 Finding the inverse of a square matrix The inverse [B] of a square matrix [A] is defined as [A][B] = [I] = [B][A] How can LU Decomposition be used to find the inverse? Assume the first column of [B] to be [b 11 b 12 b n1 ] T. Use this and the definition of matrix multiplication First column of [B] Second column of [B] The remaining columns in [B] can be found in the same manner. Thus, we now determine [B] in one column after another, where such a column is taking the role of the [X] vector in the last examples.
71 Example: Inverse of a Matrix I Find the inverse of a square matrix [A] Step 1: Using the decomposition procedure, the [L] and [U] matrices are found to be [ A] = [ L] [ U] = These two matrixes will now be used for all columns of [B].
72 Example: Inverse of a Matrix II As before solving for the each column of [B] requires two furhter steps 2) Solve [L] [Z] = [C] for [Z] 3) Solve [U] [X] = [Z] for [X] Step 2: [ L] [ Z] = [ C] This generates the equations: z 1 z 2 z 3 = 1 0 0
73 Example: Inverse of a Matrix III Step 2: Solving [U][X] = [Z] for [X] This generates the equations: So the first column of the inverse of [A] is:
74 Example: Inverse of a Matrix IV Repeating for the second and third columns of the inverse Second Column Third Column Final result: The inverse of [A] is To check: [A][A] 1 = [I] = [A] 1 [A]
75 Inverse of a Matrix with LU For calculations of columns of the inverse of the matrix, the LU decomposition needs to be done only once, then the forward substitution n times, and the back substitution n times. In comparison, if Gaussian elimination method were used to find the inverse of a matrix, the forward elimination as well as the back substitution will have to be done n times. Time taken by Gaussian Elimination Time taken by LU Decomposition For large n, the term n 4 will become dominant. Comparing computational times of finding inverse of a matrix using LU decomposition and Gaussian elimination. n CT inverse GE / CT inverse LU
76 References This script is based on by Autar Kaw, Jai Paul and Numerical Recipes (2nd/3rd Edition) by Press et al., Cambridge University Press
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More information7. LU factorization. factorsolve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix
7. LU factorization EE103 (Fall 201112) factorsolve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization algorithm effect of rounding error sparse
More informationSOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis
More informationAbstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix multiplication).
MAT 2 (Badger, Spring 202) LU Factorization Selected Notes September 2, 202 Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix
More information7 Gaussian Elimination and LU Factorization
7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method
More informationDirect Methods for Solving Linear Systems. Matrix Factorization
Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011
More informationLecture 6. Inverse of Matrix
Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that
More informationOperation Count; Numerical Linear Algebra
10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floatingpoint
More informationSolution of Linear Systems
Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start
More information6. Cholesky factorization
6. Cholesky factorization EE103 (Fall 201112) triangular matrices forward and backward substitution the Cholesky factorization solving Ax = b with A positive definite inverse of a positive definite matrix
More informationLinear Dependence Tests
Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks
More informationWe seek a factorization of a square matrix A into the product of two matrices which yields an
LU Decompositions We seek a factorization of a square matrix A into the product of two matrices which yields an efficient method for solving the system where A is the coefficient matrix, x is our variable
More informationby the matrix A results in a vector which is a reflection of the given
Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the yaxis We observe that
More information2.1: MATRIX OPERATIONS
.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and
More informationSolving Systems of Linear Equations
LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how
More informationLINEAR SYSTEMS. Consider the following example of a linear system:
LINEAR SYSTEMS Consider the following example of a linear system: Its unique solution is x +2x 2 +3x 3 = 5 x + x 3 = 3 3x + x 2 +3x 3 = 3 x =, x 2 =0, x 3 = 2 In general we want to solve n equations in
More informationDETERMINANTS. b 2. x 2
DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in
More information10.2 ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS. The Jacobi Method
578 CHAPTER 1 NUMERICAL METHODS 1. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS As a numerical technique, Gaussian elimination is rather unusual because it is direct. That is, a solution is obtained after
More informationSolving Linear Systems, Continued and The Inverse of a Matrix
, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing
More informationThe Inverse of a Matrix
The Inverse of a Matrix 7.4 Introduction In number arithmetic every number a ( 0) has a reciprocal b written as a or such that a ba = ab =. Some, but not all, square matrices have inverses. If a square
More informationElementary Matrices and The LU Factorization
lementary Matrices and The LU Factorization Definition: ny matrix obtained by performing a single elementary row operation (RO) on the identity (unit) matrix is called an elementary matrix. There are three
More informationSolving Systems of Linear Equations
LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how
More informationMatrices Worksheet. Adding the results together, using the matrices, gives
Matrices Worksheet This worksheet is designed to help you increase your confidence in handling MATRICES. This worksheet contains both theory and exercises which cover. Introduction. Order, Addition and
More information4. Matrix inverses. left and right inverse. linear independence. nonsingular matrices. matrices with linearly independent columns
L. Vandenberghe EE133A (Spring 2016) 4. Matrix inverses left and right inverse linear independence nonsingular matrices matrices with linearly independent columns matrices with linearly independent rows
More informationGeneral Framework for an Iterative Solution of Ax b. Jacobi s Method
2.6 Iterative Solutions of Linear Systems 143 2.6 Iterative Solutions of Linear Systems Consistent linear systems in real life are solved in one of two ways: by direct calculation (using a matrix factorization,
More informationIntroduction to Matrix Algebra I
Appendix A Introduction to Matrix Algebra I Today we will begin the course with a discussion of matrix algebra. Why are we studying this? We will use matrix algebra to derive the linear regression model
More informationUniversity of Lille I PC first year list of exercises n 7. Review
University of Lille I PC first year list of exercises n 7 Review Exercise Solve the following systems in 4 different ways (by substitution, by the Gauss method, by inverting the matrix of coefficients
More informationLecture Notes: Matrix Inverse. 1 Inverse Definition
Lecture Notes: Matrix Inverse Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Inverse Definition We use I to represent identity matrices,
More informationSYSTEMS OF EQUATIONS AND MATRICES WITH THE TI89. by Joseph Collison
SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI89 by Joseph Collison Copyright 2000 by Joseph Collison All rights reserved Reproduction or translation of any part of this work beyond that permitted by Sections
More information9. Numerical linear algebra background
Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization
More information1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form
1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and
More informationFactorization Theorems
Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization
More informationDiagonal, Symmetric and Triangular Matrices
Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by
More informationVector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a nonempty
More informationUsing row reduction to calculate the inverse and the determinant of a square matrix
Using row reduction to calculate the inverse and the determinant of a square matrix Notes for MATH 0290 Honors by Prof. Anna Vainchtein 1 Inverse of a square matrix An n n square matrix A is called invertible
More informationLINEAR ALGEBRA. September 23, 2010
LINEAR ALGEBRA September 3, 00 Contents 0. LUdecomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................
More information1 Introduction to Matrices
1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns
More informationSolving Linear Systems of Equations. Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.
Solving Linear Systems of Equations Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.edu These slides are a supplement to the book Numerical Methods with Matlab:
More informationCS3220 Lecture Notes: QR factorization and orthogonal transformations
CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss
More information1 Determinants and the Solvability of Linear Systems
1 Determinants and the Solvability of Linear Systems In the last section we learned how to use Gaussian elimination to solve linear systems of n equations in n unknowns The section completely sidestepped
More information1 Eigenvalues and Eigenvectors
Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x
More informationApplied Linear Algebra I Review page 1
Applied Linear Algebra Review 1 I. Determinants A. Definition of a determinant 1. Using sum a. Permutations i. Sign of a permutation ii. Cycle 2. Uniqueness of the determinant function in terms of properties
More informationUNIT  I LESSON  1 The Solution of Numerical Algebraic and Transcendental Equations
UNIT  I LESSON  1 The Solution of Numerical Algebraic and Transcendental Equations Contents: 1.0 Aims and Objectives 1.1 Introduction 1.2 Bisection Method 1.2.1 Definition 1.2.2 Computation of real root
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 3 Linear Least Squares Prof. Michael T. Heath Department of Computer Science University of Illinois at UrbanaChampaign Copyright c 2002. Reproduction
More informationMATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).
MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0
More informationHelpsheet. Giblin Eunson Library MATRIX ALGEBRA. library.unimelb.edu.au/libraries/bee. Use this sheet to help you:
Helpsheet Giblin Eunson Library ATRIX ALGEBRA Use this sheet to help you: Understand the basic concepts and definitions of matrix algebra Express a set of linear equations in matrix notation Evaluate determinants
More information1 VECTOR SPACES AND SUBSPACES
1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such
More informationDecember 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS
December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in twodimensional space (1) 2x y = 3 describes a line in twodimensional space The coefficients of x and y in the equation
More informationTypical Linear Equation Set and Corresponding Matrices
EWE: Engineering With Excel Larsen Page 1 4. Matrix Operations in Excel. Matrix Manipulations: Vectors, Matrices, and Arrays. How Excel Handles Matrix Math. Basic Matrix Operations. Solving Systems of
More informationIntroduction to Matrix Algebra
Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra  1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary
More informationSECTION 8.3: THE INVERSE OF A SQUARE MATRIX
(Section 8.3: The Inverse of a Square Matrix) 8.47 SECTION 8.3: THE INVERSE OF A SQUARE MATRIX PART A: (REVIEW) THE INVERSE OF A REAL NUMBER If a is a nonzero real number, then aa 1 = a 1 a = 1. a 1, or
More informationSystems of Linear Equations
Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and
More informationLU Decomposition. The original equation is to solve. Ax b =0ff At the end of the Gauss elimination, the resulting equations were
LU Decomposition LU decomposition is a better way to implement Gauss elimination, especially for repeated solving a number of equations with the same lefthand side That is, for solving the equation Ax
More information10.3 POWER METHOD FOR APPROXIMATING EIGENVALUES
58 CHAPTER NUMERICAL METHODS. POWER METHOD FOR APPROXIMATING EIGENVALUES In Chapter 7 you saw that the eigenvalues of an n n matrix A are obtained by solving its characteristic equation n c nn c nn...
More informationSolving Systems of Linear Equations. Substitution
Solving Systems of Linear Equations There are two basic methods we will use to solve systems of linear equations: Substitution Elimination We will describe each for a system of two equations in two unknowns,
More information13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.
3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in threespace, we write a vector in terms
More informationB such that AB = I and BA = I. (We say B is an inverse of A.) Definition A square matrix A is invertible (or nonsingular) if matrix
Matrix inverses Recall... Definition A square matrix A is invertible (or nonsingular) if matrix B such that AB = and BA =. (We say B is an inverse of A.) Remark Not all square matrices are invertible.
More informationLecture 2 Matrix Operations
Lecture 2 Matrix Operations transpose, sum & difference, scalar multiplication matrix multiplication, matrixvector product matrix inverse 2 1 Matrix transpose transpose of m n matrix A, denoted A T or
More informationQuestion 2: How do you solve a matrix equation using the matrix inverse?
Question : How do you solve a matrix equation using the matrix inverse? In the previous question, we wrote systems of equations as a matrix equation AX B. In this format, the matrix A contains the coefficients
More information8 Square matrices continued: Determinants
8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You
More informationLecture 3: Finding integer solutions to systems of linear equations
Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture
More informationa 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given
More informationLinearly Independent Sets and Linearly Dependent Sets
These notes closely follow the presentation of the material given in David C. Lay s textbook Linear Algebra and its Applications (3rd edition). These notes are intended primarily for inclass presentation
More information4. MATRICES Matrices
4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:
More informationIterative Methods for Linear Systems
Iterative Methods for Linear Systems We present the basic concepts for three of the most fundamental and well known iterative techniques for solving a system of linear equations of the form Ax = b. Iterative
More informationMath 313 Lecture #10 2.2: The Inverse of a Matrix
Math 1 Lecture #10 2.2: The Inverse of a Matrix Matrix algebra provides tools for creating many useful formulas just like real number algebra does. For example, a real number a is invertible if there is
More informationLinear Equations ! 25 30 35$ & " 350 150% & " 11,750 12,750 13,750% MATHEMATICS LEARNING SERVICE Centre for Learning and Professional Development
MathsTrack (NOTE Feb 2013: This is the old version of MathsTrack. New books will be created during 2013 and 2014) Topic 4 Module 9 Introduction Systems of to Matrices Linear Equations Income = Tickets!
More informationMATH10212 Linear Algebra. Systems of Linear Equations. Definition. An ndimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0534405967. Systems of Linear Equations Definition. An ndimensional vector is a row or a column
More informationNotes on Determinant
ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 918/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without
More informationMatrices, Determinants and Linear Systems
September 21, 2014 Matrices A matrix A m n is an array of numbers in rows and columns a 11 a 12 a 1n r 1 a 21 a 22 a 2n r 2....... a m1 a m2 a mn r m c 1 c 2 c n We say that the dimension of A is m n (we
More informationSection 2.1. Section 2.2. Exercise 6: We have to compute the product AB in two ways, where , B =. 2 1 3 5 A =
Section 2.1 Exercise 6: We have to compute the product AB in two ways, where 4 2 A = 3 0 1 3, B =. 2 1 3 5 Solution 1. Let b 1 = (1, 2) and b 2 = (3, 1) be the columns of B. Then Ab 1 = (0, 3, 13) and
More informationSituation 23: Simultaneous Equations Prepared at the University of Georgia EMAT 6500 class Date last revised: July 22 nd, 2013 Nicolina Scarpelli
Situation 23: Simultaneous Equations Prepared at the University of Georgia EMAT 6500 class Date last revised: July 22 nd, 2013 Nicolina Scarpelli Prompt: A mentor teacher and student teacher are discussing
More informationRow Echelon Form and Reduced Row Echelon Form
These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for inclass presentation
More informationMATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.
MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all ndimensional column
More informationMatrices Summary. To add or subtract matrices they must be the same dimensions. Just add or subtract the corresponding numbers.
Matrices Summary To transpose a matrix write the rows as columns. Academic Skills Advice For example: 2 1 A = [ 1 2 1 0 0 9] A T = 4 2 2 1 2 1 1 0 4 0 9 2 To add or subtract matrices they must be the same
More informationThe Characteristic Polynomial
Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem
More informationMatrix Algebra and Applications
Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2  Matrices and Matrix Algebra Reading 1 Chapters
More informationLinear Algebra: Determinants, Inverses, Rank
D Linear Algebra: Determinants, Inverses, Rank D 1 Appendix D: LINEAR ALGEBRA: DETERMINANTS, INVERSES, RANK TABLE OF CONTENTS Page D.1. Introduction D 3 D.2. Determinants D 3 D.2.1. Some Properties of
More informationMatrices 2. Solving Square Systems of Linear Equations; Inverse Matrices
Matrices 2. Solving Square Systems of Linear Equations; Inverse Matrices Solving square systems of linear equations; inverse matrices. Linear algebra is essentially about solving systems of linear equations,
More informationNumerical Methods I Solving Linear Systems: Sparse Matrices, Iterative Methods and NonSquare Systems
Numerical Methods I Solving Linear Systems: Sparse Matrices, Iterative Methods and NonSquare Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420001,
More information2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system
1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables
More informationAu = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.
Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry
More informationMethods for Finding Bases
Methods for Finding Bases Bases for the subspaces of a matrix Rowreduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,
More informationK80TTQ1EP??,VO.L,XU0H5BY,_71ZVPKOE678_X,N2Y8HI4VS,,6Z28DDW5N7ADY013
Hill Cipher Project K80TTQ1EP??,VO.L,XU0H5BY,_71ZVPKOE678_X,N2Y8HI4VS,,6Z28DDW5N7ADY013 Directions: Answer all numbered questions completely. Show nontrivial work in the space provided. Noncomputational
More information1 Lecture: Integration of rational functions by decomposition
Lecture: Integration of rational functions by decomposition into partial fractions Recognize and integrate basic rational functions, except when the denominator is a power of an irreducible quadratic.
More informationGeneralized Inverse Computation Based on an Orthogonal Decomposition Methodology.
International Conference on Mathematical and Statistical Modeling in Honor of Enrique Castillo. June 2830, 2006 Generalized Inverse Computation Based on an Orthogonal Decomposition Methodology. Patricia
More informationBasic Terminology for Systems of Equations in a Nutshell. E. L. Lady. 3x 1 7x 2 +4x 3 =0 5x 1 +8x 2 12x 3 =0.
Basic Terminology for Systems of Equations in a Nutshell E L Lady A system of linear equations is something like the following: x 7x +4x =0 5x +8x x = Note that the number of equations is not required
More informationWe shall turn our attention to solving linear systems of equations. Ax = b
59 Linear Algebra We shall turn our attention to solving linear systems of equations Ax = b where A R m n, x R n, and b R m. We already saw examples of methods that required the solution of a linear system
More informationLecture 1: Systems of Linear Equations
MTH Elementary Matrix Algebra Professor Chao Huang Department of Mathematics and Statistics Wright State University Lecture 1 Systems of Linear Equations ² Systems of two linear equations with two variables
More informationLecture 5: Singular Value Decomposition SVD (1)
EEM3L1: Numerical and Analytical Techniques Lecture 5: Singular Value Decomposition SVD (1) EE3L1, slide 1, Version 4: 25Sep02 Motivation for SVD (1) SVD = Singular Value Decomposition Consider the system
More informationSolution to Homework 2
Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if
More informationLinear Algebra Notes
Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note
More informationCofactor Expansion: Cramer s Rule
Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating
More information2.5 Elementary Row Operations and the Determinant
2.5 Elementary Row Operations and the Determinant Recall: Let A be a 2 2 matrtix : A = a b. The determinant of A, denoted by det(a) c d or A, is the number ad bc. So for example if A = 2 4, det(a) = 2(5)
More informationPresentation 3: Eigenvalues and Eigenvectors of a Matrix
Colleen Kirksey, Beth Van Schoyck, Dennis Bowers MATH 280: Problem Solving November 18, 2011 Presentation 3: Eigenvalues and Eigenvectors of a Matrix Order of Presentation: 1. Definitions of Eigenvalues
More informationA Direct Numerical Method for Observability Analysis
IEEE TRANSACTIONS ON POWER SYSTEMS, VOL 15, NO 2, MAY 2000 625 A Direct Numerical Method for Observability Analysis Bei Gou and Ali Abur, Senior Member, IEEE Abstract This paper presents an algebraic method
More information1 Review of Least Squares Solutions to Overdetermined Systems
cs4: introduction to numerical analysis /9/0 Lecture 7: Rectangular Systems and Numerical Integration Instructor: Professor Amos Ron Scribes: Mark Cowlishaw, Nathanael Fillmore Review of Least Squares
More informationLS.6 Solution Matrices
LS.6 Solution Matrices In the literature, solutions to linear systems often are expressed using square matrices rather than vectors. You need to get used to the terminology. As before, we state the definitions
More information1.2 Solving a System of Linear Equations
1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems  Basic De nitions As noticed above, the general form of a linear system of m equations in n variables
More information