# Eigenvalues and Singular Values

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Chapter 10 Eigenvalues and Singular Values This chapter is about eigenvalues and singular values of matrices. Computational algorithms and sensitivity to perturbations are both discussed Eigenvalue and Singular Value Decompositions An eigenvalue and eigenvector of a square matrix A are a scalar λ and a nonzero vector x so that Ax = λx. A singular value and pair of singular vectors of a square or rectangular matrix A are a nonnegative scalar σ and two nonzero vectors u and v so that Av = σu, A H u = σv. The superscript on A H stands for Hermitian transpose and denotes the complex conjugate transpose of a complex matrix. If the matrix is real, then A T denotes the same matrix. In Matlab, these transposed matrices are denoted by A. The term eigenvalue is a partial translation of the German eigenwert. A complete translation would be something like own value or characteristic value, but these are rarely used. The term singular value relates to the distance between a matrix and the set of singular matrices. Eigenvalues play an important role in situations where the matrix is a transformation from one vector space onto itself. Systems of linear ordinary differential equations are the primary examples. The values of λ can correspond to frequencies of vibration, or critical values of stability parameters, or energy levels of atoms. Singular values play an important role where the matrix is a transformation from one vector space to a different vector space, possibly with a different dimension. Systems of over- or underdetermined algebraic equations are the primary examples. September 16,

2 2 Chapter 10. Eigenvalues and Singular Values The definitions of eigenvectors and singular vectors do not specify their normalization. An eigenvector x, or a pair of singular vectors u and v, can be scaled by any nonzero factor without changing any other important properties. Eigenvectors of symmetric matrices are usually normalized to have Euclidean length equal to one, x 2 = 1. On the other hand, the eigenvectors of nonsymmetric matrices often have different normalizations in different contexts. Singular vectors are almost always normalized to have Euclidean length equal to one, u 2 = v 2 = 1. You can still multiply eigenvectors, or pairs of singular vectors, by 1 without changing their lengths. The eigenvalue-eigenvector equation for a square matrix can be written (A λi)x = 0, x 0. This implies that A λi is singular and hence that det(a λi) = 0. This definition of an eigenvalue, which does not directly involve the corresponding eigenvector, is the characteristic equation or characteristic polynomial of A. The degree of the polynomial is the order of the matrix. This implies that an n-by-n matrix has n eigenvalues, counting multiplicities. Like the determinant itself, the characteristic polynomial is useful in theoretical considerations and hand calculations, but does not provide a sound basis for robust numerical software. Let λ 1, λ 2,..., λ n be the eigenvalues of a matrix A, let x 1, x 2,..., x n be a set of corresponding eigenvectors, let Λ denote the n-by-n diagonal matrix with the λ j on the diagonal, and let X denote the n-by-n matrix whose jth column is x j. Then AX = XΛ. It is necessary to put Λ on the right in the second expression so that each column of X is multiplied by its corresponding eigenvalue. Now make a key assumption that is not true for all matrices assume that the eigenvectors are linearly independent. Then X 1 exists and A = XΛX 1, with nonsingular X. This is known as the eigenvalue decomposition of the matrix A. If it exists, it allows us to investigate the properties of A by analyzing the diagonal matrix Λ. For example, repeated matrix powers can be expressed in terms of powers of scalars: A p = XΛ p X 1. If the eigenvectors of A are not linearly independent, then such a diagonal decomposition does not exist and the powers of A exhibit a more complicated behavior. If T is any nonsingular matrix, then A = T BT 1 is known as a similarity transformation and A and B are said to be similar. If Ax = λx and x = T y, then By = λy. In other words, a similarity transformation preserves eigenvalues. The eigenvalue decomposition is an attempt to find a similarity transformation to diagonal form.

3 10.1. Eigenvalue and Singular Value Decompositions 3 are Written in matrix form, the defining equations for singular values and vectors AV = UΣ, A H U = V Σ H. Here Σ is a matrix the same size as A that is zero except possibly on its main diagonal. It turns out that singular vectors can always be chosen to be perpendicular to each other, so the matrices U and V, whose columns are the normalized singular vectors, satisfy U H U = I and V H V = I. In other words, U and V are orthogonal if they are real, or unitary if they are complex. Consequently, A = UΣV H, with diagonal Σ and orthogonal or unitary U and V. This is known as the singular value decomposition, or SVD, of the matrix A. In abstract linear algebra terms, eigenvalues are relevant if a square, n-by-n matrix A is thought of as mapping n-dimensional space onto itself. We try to find a basis for the space so that the matrix becomes diagonal. This basis might be complex even if A is real. In fact, if the eigenvectors are not linearly independent, such a basis does not even exist. The SVD is relevant if a possibly rectangular, m-by-n matrix A is thought of as mapping n-space onto m-space. We try to find one change of basis in the domain and a usually different change of basis in the range so that the matrix becomes diagonal. Such bases always exist and are always real if A is real. In fact, the transforming matrices are orthogonal or unitary, so they preserve lengths and angles and do not magnify errors. If A is m by n with m larger than n, then in the full SVD, U is a large, square m-by-m matrix. The last m n columns of U are extra ; they are not needed A = U Σ V A = U Σ V Figure Full and economy SVDs.

4 4 Chapter 10. Eigenvalues and Singular Values to reconstruct A. A second version of the SVD that saves computer memory if A is rectangular is known as the economy-sized SVD. In the economy version, only the first n columns of U and first n rows of Σ are computed. The matrix V is the same n-by-n matrix in both decompositions. Figure 10.1 shows the shapes of the various matrices in the two versions of the SVD. Both decompositions can be written A = UΣV H, even though the U and Σ in the economy decomposition are submatrices of the ones in the full decomposition A Small Example An example of the eigenvalue and singular value decompositions of a small, square matrix is provided by one of the test matrices from the Matlab gallery. A = gallery(3) The matrix is A = This matrix was constructed in such a way that the characteristic polynomial factors nicely: det(a λi) = λ 3 6λ λ 6 = (λ 1)(λ 2)(λ 3). Consequently, the three eigenvalues are λ 1 = 1, λ 2 = 2, and λ 3 = 3, and Λ = The matrix of eigenvectors can be normalized so that its elements are all integers: X = It turns out that the inverse of X also has integer entries: X 1 = These matrices provide the eigenvalue decomposition of our example: A = XΛX 1. The SVD of this matrix cannot be expressed so neatly with small integers. The singular values are the positive roots of the equation σ σ σ 2 36 = 0, but this equation does not factor nicely. The Symbolic Toolbox statement

5 10.3. eigshow 5 svd(sym(a)) returns exact formulas for the singular values, but the overall length of the result is 922 characters. So we compute the SVD numerically. [U,S,V] = svd(a) produces U = S = V = The expression U*S*V generates the original matrix to within roundoff error. For gallery(3), notice the big difference between the eigenvalues, 1, 2, and 3, and the singular values, 817, 2.47, and This is related, in a way that we will make more precise later, to the fact that this example is very far from being a symmetric matrix eigshow The function eigshow is available in the Matlab demos directory. The input to eigshow is a real, 2-by-2 matrix A, or you can choose an A from a pull-down list in the title. The default A is ( ) 1/4 3/4 A =. 1 1/2 Initially, eigshow plots the unit vector x = [1, 0], as well as the vector Ax, which starts out as the first column of A. You can then use your mouse to move x, shown in green, around the unit circle. As you move x, the resulting Ax, shown in blue, also moves. The first four subplots in Figure 10.2 show intermediate steps as x traces out a green unit circle. What is the shape of the resulting orbit of Ax? An important, and nontrivial, theorem from linear algebra tells us that the blue curve is an ellipse. eigshow provides a proof by GUI of this theorem. The caption for eigshow says Make Ax parallel to x. For such a direction x, the operator A is simply a stretching or magnification by a factor λ. In other words, x is an eigenvector and the length of Ax is the corresponding eigenvalue.

6 6 Chapter 10. Eigenvalues and Singular Values A*x x A*x x x A*x A*x x x A*x A*x x Figure eigshow. The last two subplots in Figure 10.2 show the eigenvalues and eigenvectors of our 2-by-2 example. The first eigenvalue is positive, so Ax lies on top of the eigenvector x. The length of Ax is the corresponding eigenvalue; it happens to be 5/4 in this example. The second eigenvalue is negative, so Ax is parallel to x, but points in the opposite direction. The length of Ax is 1/2, and the corresponding eigenvalue is actually 1/2. You might have noticed that the two eigenvectors are not the major and minor axes of the ellipse. They would be if the matrix were symmetric. The default eigshow matrix is close to, but not exactly equal to, a symmetric matrix. For other matrices, it may not be possible to find a real x so that Ax is parallel to x. These examples, which we pursue in the exercises, demonstrate that 2-by-2 matrices can have fewer than two real eigenvectors. The axes of the ellipse do play a key role in the SVD. The results produced by the svd mode of eigshow are shown in Figure Again, the mouse moves x around the unit circle, but now a second unit vector, y, follows x, staying perpendicular to it. The resulting Ax and Ay traverse the ellipse, but are not usually perpendicular to each other. The goal is to make them perpendicular. If they are,

7 10.4. Characteristic Polynomial 7 y A*x x A*y Figure eigshow(svd). they form the axes of the ellipse. The vectors x and y are the columns of U in the SVD, the vectors Ax and Ay are multiples of the columns of V, and the lengths of the axes are the singular values Characteristic Polynomial Let A be the 20-by-20 diagonal matrix with 1, 2,..., 20 on the diagonal. Clearly, the eigenvalues of A are its diagonal elements. However, the characteristic polynomial det(a λi) turns out to be λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ The coefficient of λ 19 is 210, which is the sum of the eigenvalues. The coefficient of λ 0, the constant term, is 20!, which is the product of the eigenvalues. The other coefficients are various sums of products of the eigenvalues. We have displayed all the coefficients to emphasize that doing any floatingpoint computation with them is likely to introduce large roundoff errors. Merely representing the coefficients as IEEE floating-point numbers changes five of them. For example, the last 3 digits of the coefficient of λ 4 change from 776 to 392. To 16 significant digits, the exact roots of the polynomial obtained by representing the coefficients in floating point are as follows.

8 8 Chapter 10. Eigenvalues and Singular Values We see that just storing the coefficients in the characteristic polynomial as doubleprecision floating-point numbers changes the computed values of some of the eigenvalues in the fifth significant digit. This particular polynomial was introduced by J. H. Wilkinson around His perturbation of the polynomial was different than ours, but his point was the same, namely that representing a polynomial in its power form is an unsatisfactory way to characterize either the roots of the polynomial or the eigenvalues of the corresponding matrix Symmetric and Hermitian Matrices A real matrix is symmetric if it is equal to its transpose, A = A T. A complex matrix is Hermitian if it is equal to its complex conjugate transpose, A = A H. The eigenvalues and eigenvectors of a real symmetric matrix are real. Moreover, the matrix of eigenvectors can be chosen to be orthogonal. Consequently, if A is real and A = A T, then its eigenvalue decomposition is A = XΛX T, with X T X = I = XX T. The eigenvalues of a complex Hermitian matrix turn out to be real, although the eigenvectors must be complex. Moreover, the matrix of eigenvectors can be chosen to be unitary. Consequently, if A is complex and A = A H, then its eigenvalue decomposition is with Λ real and X H X = I = XX H. A = XΛX H,

9 10.6. Eigenvalue Sensitivity and Accuracy 9 For symmetric and Hermitian matrices, the eigenvalues and singular values are obviously closely related. A nonnegative eigenvalue, λ 0, is also a singular value, σ = λ. The corresponding vectors are equal to each other, u = v = x. A negative eigenvalue, λ < 0, must reverse its sign to become a singular value, σ = λ. One of the corresponding singular vectors is the negative of the other, u = v = x Eigenvalue Sensitivity and Accuracy The eigenvalues of some matrices are sensitive to perturbations. Small changes in the matrix elements can lead to large changes in the eigenvalues. Roundoff errors introduced during the computation of eigenvalues with floating-point arithmetic have the same effect as perturbations in the original matrix. Consequently, these roundoff errors are magnified in the computed values of sensitive eigenvalues. To get a rough idea of this sensitivity, assume that A has a full set of linearly independent eigenvectors and use the eigenvalue decomposition Rewrite this as A = XΛX 1. Λ = X 1 AX. Now let δa denote some change in A, caused by roundoff error or any other kind of perturbation. Then Λ + δλ = X 1 (A + δa)x. Hence Taking matrix norms, δλ = X 1 δax. δλ X 1 X δa = κ(x ) δa, where κ(x ) is the matrix condition number introduced in Chapter 2, Linear Equations. Note that the key factor is the condition of X, the matrix of eigenvectors, not the condition of A itself. This simple analysis tells us that, in terms of matrix norms, a perturbation δa can be magnified by a factor as large as κ(x ) in δλ. However, since δλ is usually not a diagonal matrix, this analysis does not immediately say how much the eigenvalues themselves may be affected. Nevertheless, it leads to the correct overall conclusion: The sensitivity of the eigenvalues is estimated by the condition number of the matrix of eigenvectors. You can use the function condest to estimate the condition number of the eigenvector matrix. For example, A = gallery(3) [X,lambda] = eig(a); condest(x)

10 10 Chapter 10. Eigenvalues and Singular Values yields e+003 A perturbation in gallery(3) could result in perturbations in its eigenvalues that are times as large. This says that the eigenvalues of gallery(3) are slightly badly conditioned. A more detailed analysis involves the left eigenvectors, which are row vectors y H that satisfy y H A = λy H. In order to investigate the sensitivity of an individual eigenvalue, assume that A varies with a perturbation parameter and let Ȧ denote the derivative with respect to that parameter. Differentiate both sides of the equation to get Multiply through by the left eigenvector: Ax = λx Ax + Aẋ = λx + λẋ. y H Ax + y H Aẋ = y H λx + y H λẋ. The second terms on each side of this equation are equal, so Taking norms, λ = yh Ax y H x. λ y x y H x A. Define the eigenvalue condition number to be Then κ(λ, A) = y x y H x. λ κ(λ, A) Ȧ. In other words, κ(λ, A) is the magnification factor relating a perturbation in the matrix A to the resulting perturbation in an eigenvalue λ. Notice that κ(λ, A) is independent of the normalization of the left and right eigenvectors, y and x, and that κ(λ, A) 1. If you have already computed the matrix X whose columns are the right eigenvectors, one way to compute the left eigenvectors is to let Y H = X 1.

11 10.6. Eigenvalue Sensitivity and Accuracy 11 Then, since Y H A = ΛY H, the rows of Y H are the left eigenvectors. In this case, the left eigenvectors are normalized so that Y H X = I, so the denominator in κ(λ, A) is y H x = 1 and κ(λ, A) = y x. Since x X and y X 1, we have κ(λ, A) κ(x ). The condition number of the eigenvector matrix is an upper bound for the individual eigenvalue condition numbers. The Matlab function condeig computes eigenvalue condition numbers. Continuing with the gallery(3) example, yields A = gallery(3) lambda = eig(a) kappa = condeig(a) lambda = kappa = This indicates that λ 1 = 1 is slightly more sensitive than λ 2 = 2 or λ 3 = 3. A perturbation in gallery(3) may result in perturbations in its eigenvalues that are 200 to 600 times as large. This is consistent with the cruder estimate of obtained from condest(x). To test this analysis, let s make a small random perturbation in A = gallery(3) and see what happens to its eigenvalues. format long delta = 1.e-6; lambda = eig(a + delta*randn(3,3)) lambda =

12 12 Chapter 10. Eigenvalues and Singular Values The perturbation in the eigenvalues is lambda - (1:3) ans = 1.0e-003 * This is smaller than, but roughly the same size as, the estimates provided by condeig and the perturbation analysis. delta*condeig(a) ans = 1.0e-003 * If A is real and symmetric, or complex and Hermitian, then its right and left eigenvectors are the same. In this case, y H x = y x, so, for symmetric and Hermitian matrices, κ(λ, A) = 1. The eigenvalues of symmetric and Hermitian matrices are perfectly well conditioned. Perturbations in the matrix lead to perturbations in the eigenvalues that are roughly the same size. This is true even for multiple eigenvalues. At the other extreme, if λ k is a multiple eigenvalue that does not have a corresponding full set of linearly independent eigenvectors, then the previous analysis does not apply. In this case, the characteristic polynomial for an n-by-n matrix can be written p(λ) = det(a λi) = (λ λ k ) m q(λ), where m is the multiplicity of λ k and q(λ) is a polynomial of degree n m that does not vanish at λ k. A perturbation in the matrix of size δ results in a change in the characteristic polynomial from p(λ) = 0 to something like In other words, The roots of this equation are p(λ) = O(δ). (λ λ k ) m = O(δ)/q(λ). λ = λ k + O(δ 1/m ).

13 10.6. Eigenvalue Sensitivity and Accuracy 13 This mth root behavior says that multiple eigenvalues without a full set of eigenvectors are extremely sensitive to perturbation. As an artificial, but illustrative, example, consider the 16-by-16 matrix with 2 s on the main diagonal, 1 s on the superdiagonal, δ in the lower left-hand corner, and 0 s elsewhere: A = δ 2 The characteristic equation is (λ 2) 16 = δ. If δ = 0, this matrix has an eigenvalue of multiplicity 16 at λ = 2, but there is only 1 eigenvector to go along with this multiple eigenvalue. If δ is on the order of floating-point roundoff error, that is, δ 10 16, then the eigenvalues are on a circle in the complex plane with center at 2 and radius (10 16 ) 1/16 = 0.1. A perturbation on the size of roundoff error changes the eigenvalue from to 16 different values, including , , and i. A tiny change in the matrix elements causes a much larger change in the eigenvalues. Essentially the same phenomenon, but in a less obvious form, explains the behavior of another Matlab gallery example, A = gallery(5) The matrix is A = The computed eigenvalues, obtained from lambda = eig(a), are lambda = i i i i How accurate are these computed eigenvalues? The gallery(5) matrix was constructed in such a way that its characteristic equation is λ 5 = 0.

14 14 Chapter 10. Eigenvalues and Singular Values Figure plot(eig(gallery(5))). You can confirm this by noting that A 5, which is computed without any roundoff error, is the zero matrix. The characteristic equation can easily be solved by hand. All five eigenvalues are actually equal to zero. The computed eigenvalues give little indication that the correct eigenvalues are all zero. We certainly have to admit that the computed eigenvalues are not very accurate. The Matlab eig function is doing as well as can be expected on this problem. The inaccuracy of the computed eigenvalues is caused by their sensitivity, not by anything wrong with eig. The following experiment demonstrates this fact. Start with A = gallery(5) e = eig(a) plot(real(e),imag(e), r*,0,0, ko ) axis(.1*[ ]) axis square Figure 10.4 shows that the computed eigenvalues are the vertices of a regular pentagon in the complex plane, centered at the origin. The radius is about Now repeat the experiment with a matrix where each element is perturbed by a single roundoff error. The elements of gallery(5) vary over four orders of magnitude, so the correct scaling of the perturbation is obtained with e = eig(a + eps*randn(5,5).*a) Put this statement, along with the plot and axis commands, on a single line and use the up arrow to repeat the computation several times. You will see that the pentagon flips orientation and that its radius varies between 0.03 and 0.07, but that the computed eigenvalues of the perturbed problems behave pretty much like the computed eigenvalues of the original matrix. The experiment provides evidence for the fact that the computed eigenvalues are the exact eigenvalues of a matrix A + E, where the elements of E are on the order of roundoff error compared to the elements of A. This is the best we can expect to achieve with floating-point computation.

15 10.7. Singular Value Sensitivity and Accuracy Singular Value Sensitivity and Accuracy The sensitivity of singular values is much easier to characterize than the sensitivity of eigenvalues. The singular value problem is always perfectly well conditioned. A perturbation analysis would involve an equation like Σ + δσ = U H (A + δa)v. But, since U and V are orthogonal or unitary, they preserve norms. Consequently, δσ = δa. Perturbations of any size in any matrix cause perturbations of roughly the same size in its singular values. There is no need to define condition numbers for singular values because they would always be equal to one. The Matlab function svd always computes singular values to full floating-point accuracy. We have to be careful about what we mean by same size and full accuracy. Perturbations and accuracy are measured relative the norm of the matrix or, equivalently, the largest singular value: A 2 = σ 1. The accuracy of the smaller singular values is measured relative to the largest one. If, as is often the case, the singular values vary over several orders of magnitude, the smaller ones might not have full accuracy relative to themselves. In particular, if the matrix is singular, then some of the σ i must be zero. The computed values of these σ i will usually be on the order of ϵ A, where ϵ is eps, the floating-point accuracy parameter. This can be illustrated with the singular values of gallery(5). The statements A = gallery(5) format long e svd(a) produce e e e e e-14 The largest element of A is 93365, and we see that the largest singular value is a little larger, about There are three singular values near Recall that all the eigenvalues of this matrix are zero, so the matrix is singular and the smallest singular value should theoretically be zero. The computed value is somewhere between ϵ and ϵ A.

16 16 Chapter 10. Eigenvalues and Singular Values Now let s perturb the matrix. Let this infinite loop run for a while. while 1 clc svd(a+eps*randn(5,5).*a) pause(.25) end This produces varying output like this e ****e ****e ****e+000 *.****************-0** The asterisks show the digits that change as we make the random perturbations. The 15-digit format does not show any changes in σ 1. The changes in σ 2, σ 3, and σ 4 are smaller than ϵ A, which is roughly The computed value of σ 5 is all roundoff error, less than The gallery(5) matrix was constructed to have very special properties for the eigenvalue problem. For the singular value problem, its behavior is typical of any singular matrix Jordan and Schur Forms The eigenvalue decomposition attempts to find a diagonal matrix Λ and a nonsingular matrix X so that A = XΛX 1. There are two difficulties with the eigenvalue decomposition. A theoretical difficulty is that the decomposition does not always exist. A numerical difficulty is that, even if the decomposition exists, it might not provide a basis for robust computation. The solution to the nonexistence difficulty is to get as close to diagonal as possible. This leads to the Jordan canonical form (JCF). The solution to the robustness difficulty is to replace diagonal with triangular and to use orthogonal and unitary transformations. This leads to the Schur form. A defective matrix is a matrix with at least one multiple eigenvalue that does not have a full set of linearly independent eigenvectors. For example, gallery(5) is defective; zero is an eigenvalue of multiplicity five that has only one eigenvector. The JCF is the decomposition A = XJX 1. If A is not defective, then the JCF is the same as the eigenvalue decomposition. The columns of X are the eigenvectors and J = Λ is diagonal. But if A is defective, then X consists of eigenvectors and generalized eigenvectors. The matrix J has the

17 10.9. The QR Algorithm 17 eigenvalues on the diagonal and ones on the superdiagonal in positions corresponding to the columns of X that are not ordinary eigenvectors. The rest of the elements of J are zero. The function jordan in the Matlab Symbolic Toolbox uses unlimited-precision rational arithmetic to try to compute the JCF of small matrices whose entries are small integers or ratios of small integers. If the characteristic polynomial does not have rational roots, the Symbolic Toolbox regards all the eigenvalues as distinct and produces a diagonal JCF. The JCF is a discontinuous function of the matrix. Almost any perturbation of a defective matrix can cause a multiple eigenvalue to separate into distinct values and eliminate the ones on the superdiagonal of the JCF. Matrices that are nearly defective have badly conditioned sets of eigenvectors, and the resulting similarity transformations cannot be used for reliable numerical computation. A numerically satisfactory alternative to the JCF is provided by the Schur form. Any matrix can be transformed to upper triangular form by a unitary similarity transformation: B = T H AT. The eigenvalues of A are on the diagonal of its Schur form B. Since unitary transformations are perfectly well conditioned, they do not magnify any errors. For example, A = gallery(3) [T,B] = schur(a) produces A = T = B = The diagonal elements of B are the eigenvalues of A. If A were symmetric, B would be diagonal. In this case, the large off-diagonal elements of B measure the lack of symmetry in A The QR Algorithm The QR algorithm is one of the most important, widely used, and successful tools we have in technical computation. Several variants of it are in the mathematical

18 18 Chapter 10. Eigenvalues and Singular Values core of Matlab. They compute the eigenvalues of real symmetric matrices, real nonsymmetric matrices, and pairs of complex matrices, and the singular values of general matrices. These functions are used, in turn, to find zeros of polynomials, to solve special linear systems, and to assess stability, and for many other tasks in various toolboxes. Dozens of people have contributed to the development of the various QR algorithms. The first complete implementation and an important convergence analysis are due to J. H. Wilkinson. Wilkinson s book, The Algebraic Eigenvalue Problem [56], as well as two fundamental papers, was published in The QR algorithm is based on repeated use of the QR factorization that we described in Chapter 5, Least Squares. The letter Q denotes orthogonal and unitary matrices and the letter R denotes right, or upper, triangular matrices. The qr function in Matlab factors any matrix, real or complex, square or rectangular, into the product of a matrix Q with orthonormal columns and matrix R that is nonzero only its upper, or right, triangle. Using the qr function, a simple variant of the QR algorithm, known as the single-shift algorithm, can be expressed as a Matlab one-liner. Let A be any square matrix. Start with n = size(a,1) I = eye(n,n) Then one step of the single-shift QR iteration is given by s = A(n,n); [Q,R] = qr(a - s*i); A = R*Q + s*i If you enter this on one line, you can use the up arrow key to iterate. The quantity s is the shift; it accelerates convergence. The QR factorization makes the matrix triangular: A si = QR. Then the reverse-order multiplication RQ restores the eigenvalues because RQ + si = Q T (A si)q + si = Q T AQ, so the new A is orthogonally similar to the original A. Each iteration effectively transfers some mass from the lower to the upper triangle while preserving the eigenvalues. As the iterations are repeated, the matrix often approaches an upper triangular matrix with the eigenvalues conveniently displayed on the diagonal. For example, start with A = gallery(3) The first iterate,

### Using the Singular Value Decomposition

Using the Singular Value Decomposition Emmett J. Ientilucci Chester F. Carlson Center for Imaging Science Rochester Institute of Technology emmett@cis.rit.edu May 9, 003 Abstract This report introduces

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression

The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression The SVD is the most generally applicable of the orthogonal-diagonal-orthogonal type matrix decompositions Every

### Numerical Methods I Eigenvalue Problems

Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001, Fall 2010 September 30th, 2010 A. Donev (Courant Institute)

### Section 1.1. Introduction to R n

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### Eigenvalues and Eigenvectors

Chapter 6 Eigenvalues and Eigenvectors 6. Introduction to Eigenvalues Linear equations Ax D b come from steady state problems. Eigenvalues have their greatest importance in dynamic problems. The solution

### 3 Orthogonal Vectors and Matrices

3 Orthogonal Vectors and Matrices The linear algebra portion of this course focuses on three matrix factorizations: QR factorization, singular valued decomposition (SVD), and LU factorization The first

### Iterative Methods for Computing Eigenvalues and Eigenvectors

The Waterloo Mathematics Review 9 Iterative Methods for Computing Eigenvalues and Eigenvectors Maysum Panju University of Waterloo mhpanju@math.uwaterloo.ca Abstract: We examine some numerical iterative

### Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

### State of Stress at Point

State of Stress at Point Einstein Notation The basic idea of Einstein notation is that a covector and a vector can form a scalar: This is typically written as an explicit sum: According to this convention,

### 13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.

3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms

### Solution of Linear Systems

Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start

### NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS

NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS DO Q LEE Abstract. Computing the solution to Least Squares Problems is of great importance in a wide range of fields ranging from numerical

### Lecture 5: Singular Value Decomposition SVD (1)

EEM3L1: Numerical and Analytical Techniques Lecture 5: Singular Value Decomposition SVD (1) EE3L1, slide 1, Version 4: 25-Sep-02 Motivation for SVD (1) SVD = Singular Value Decomposition Consider the system

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold:

Linear Systems Example: Find x, x, x such that the following three equations hold: x + x + x = 4x + x + x = x + x + x = 6 We can write this using matrix-vector notation as 4 {{ A x x x {{ x = 6 {{ b General

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### Summary of week 8 (Lectures 22, 23 and 24)

WEEK 8 Summary of week 8 (Lectures 22, 23 and 24) This week we completed our discussion of Chapter 5 of [VST] Recall that if V and W are inner product spaces then a linear map T : V W is called an isometry

### Operation Count; Numerical Linear Algebra

10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floating-point

### ALGEBRAIC EIGENVALUE PROBLEM

ALGEBRAIC EIGENVALUE PROBLEM BY J. H. WILKINSON, M.A. (Cantab.), Sc.D. Technische Universes! Dsrmstedt FACHBEREICH (NFORMATiK BIBL1OTHEK Sachgebieto:. Standort: CLARENDON PRESS OXFORD 1965 Contents 1.

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 3 Linear Least Squares Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Quick Reference Guide to Linear Algebra in Quantum Mechanics

Quick Reference Guide to Linear Algebra in Quantum Mechanics Scott N. Walck September 2, 2014 Contents 1 Complex Numbers 2 1.1 Introduction............................ 2 1.2 Real Numbers...........................

### Nonlinear Iterative Partial Least Squares Method

Numerical Methods for Determining Principal Component Analysis Abstract Factors Béchu, S., Richard-Plouet, M., Fernandez, V., Walton, J., and Fairley, N. (2016) Developments in numerical treatments for

### Linear Algebra Review. Vectors

Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

### The Image Deblurring Problem

page 1 Chapter 1 The Image Deblurring Problem You cannot depend on your eyes when your imagination is out of focus. Mark Twain When we use a camera, we want the recorded image to be a faithful representation

### 9.3 Advanced Topics in Linear Algebra

548 93 Advanced Topics in Linear Algebra Diagonalization and Jordan s Theorem A system of differential equations x = Ax can be transformed to an uncoupled system y = diag(λ,, λ n y by a change of variables

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### CONTROLLABILITY. Chapter 2. 2.1 Reachable Set and Controllability. Suppose we have a linear system described by the state equation

Chapter 2 CONTROLLABILITY 2 Reachable Set and Controllability Suppose we have a linear system described by the state equation ẋ Ax + Bu (2) x() x Consider the following problem For a given vector x in

### Orthogonal Bases and the QR Algorithm

Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries

### 1 Spherical Kinematics

ME 115(a): Notes on Rotations 1 Spherical Kinematics Motions of a 3-dimensional rigid body where one point of the body remains fixed are termed spherical motions. A spherical displacement is a rigid body

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Chapter 7. Lyapunov Exponents. 7.1 Maps

Chapter 7 Lyapunov Exponents Lyapunov exponents tell us the rate of divergence of nearby trajectories a key component of chaotic dynamics. For one dimensional maps the exponent is simply the average

### Similar matrices and Jordan form

Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

### The Second Undergraduate Level Course in Linear Algebra

The Second Undergraduate Level Course in Linear Algebra University of Massachusetts Dartmouth Joint Mathematics Meetings New Orleans, January 7, 11 Outline of Talk Linear Algebra Curriculum Study Group

### Eigenvalues and eigenvectors of a matrix

Eigenvalues and eigenvectors of a matrix Definition: If A is an n n matrix and there exists a real number λ and a non-zero column vector V such that AV = λv then λ is called an eigenvalue of A and V is

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 6. Eigenvalues and Singular Values In this section, we collect together the basic facts about eigenvalues and eigenvectors. From a geometrical viewpoint,

### x1 x 2 x 3 y 1 y 2 y 3 x 1 y 2 x 2 y 1 0.

Cross product 1 Chapter 7 Cross product We are getting ready to study integration in several variables. Until now we have been doing only differential calculus. One outcome of this study will be our ability

### CORRELATED TO THE SOUTH CAROLINA COLLEGE AND CAREER-READY FOUNDATIONS IN ALGEBRA

We Can Early Learning Curriculum PreK Grades 8 12 INSIDE ALGEBRA, GRADES 8 12 CORRELATED TO THE SOUTH CAROLINA COLLEGE AND CAREER-READY FOUNDATIONS IN ALGEBRA April 2016 www.voyagersopris.com Mathematical

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks

Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks Welcome to Thinkwell s Homeschool Precalculus! We re thrilled that you ve decided to make us part of your homeschool curriculum. This lesson

### Linear Least Squares

Linear Least Squares Suppose we are given a set of data points {(x i,f i )}, i = 1,...,n. These could be measurements from an experiment or obtained simply by evaluating a function at some points. One

### ASEN 3112 - Structures. MDOF Dynamic Systems. ASEN 3112 Lecture 1 Slide 1

19 MDOF Dynamic Systems ASEN 3112 Lecture 1 Slide 1 A Two-DOF Mass-Spring-Dashpot Dynamic System Consider the lumped-parameter, mass-spring-dashpot dynamic system shown in the Figure. It has two point

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

### South Carolina College- and Career-Ready (SCCCR) Pre-Calculus

South Carolina College- and Career-Ready (SCCCR) Pre-Calculus Key Concepts Arithmetic with Polynomials and Rational Expressions PC.AAPR.2 PC.AAPR.3 PC.AAPR.4 PC.AAPR.5 PC.AAPR.6 PC.AAPR.7 Standards Know

### AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 19: SVD revisited; Software for Linear Algebra Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 9 Outline 1 Computing

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

### On the general equation of the second degree

On the general equation of the second degree S Kesavan The Institute of Mathematical Sciences, CIT Campus, Taramani, Chennai - 600 113 e-mail:kesh@imscresin Abstract We give a unified treatment of the

### Factoring Patterns in the Gaussian Plane

Factoring Patterns in the Gaussian Plane Steve Phelps Introduction This paper describes discoveries made at the Park City Mathematics Institute, 00, as well as some proofs. Before the summer I understood

### pp. 4 8: Examples 1 6 Quick Check 1 6 Exercises 1, 2, 20, 42, 43, 64

Semester 1 Text: Chapter 1: Tools of Algebra Lesson 1-1: Properties of Real Numbers Day 1 Part 1: Graphing and Ordering Real Numbers Part 1: Graphing and Ordering Real Numbers Lesson 1-2: Algebraic Expressions

### The Fourth International DERIVE-TI92/89 Conference Liverpool, U.K., 12-15 July 2000. Derive 5: The Easiest... Just Got Better!

The Fourth International DERIVE-TI9/89 Conference Liverpool, U.K., -5 July 000 Derive 5: The Easiest... Just Got Better! Michel Beaudin École de technologie supérieure 00, rue Notre-Dame Ouest Montréal

### MATH36001 Background Material 2015

MATH3600 Background Material 205 Matrix Algebra Matrices and Vectors An ordered array of mn elements a ij (i =,, m; j =,, n) written in the form a a 2 a n A = a 2 a 22 a 2n a m a m2 a mn is said to be

### MATH 240 Fall, Chapter 1: Linear Equations and Matrices

MATH 240 Fall, 2007 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 9th Ed. written by Prof. J. Beachy Sections 1.1 1.5, 2.1 2.3, 4.2 4.9, 3.1 3.5, 5.3 5.5, 6.1 6.3, 6.5, 7.1 7.3 DEFINITIONS

### Solving Sets of Equations. 150 B.C.E., 九章算術 Carl Friedrich Gauss,

Solving Sets of Equations 5 B.C.E., 九章算術 Carl Friedrich Gauss, 777-855 Gaussian-Jordan Elimination In Gauss-Jordan elimination, matrix is reduced to diagonal rather than triangular form Row combinations

### Applied Linear Algebra I Review page 1

Applied Linear Algebra Review 1 I. Determinants A. Definition of a determinant 1. Using sum a. Permutations i. Sign of a permutation ii. Cycle 2. Uniqueness of the determinant function in terms of properties

### 1 Introduction. 2 Matrices: Definition. Matrix Algebra. Hervé Abdi Lynne J. Williams

In Neil Salkind (Ed.), Encyclopedia of Research Design. Thousand Oaks, CA: Sage. 00 Matrix Algebra Hervé Abdi Lynne J. Williams Introduction Sylvester developed the modern concept of matrices in the 9th

### Chapter 17. Orthogonal Matrices and Symmetries of Space

Chapter 17. Orthogonal Matrices and Symmetries of Space Take a random matrix, say 1 3 A = 4 5 6, 7 8 9 and compare the lengths of e 1 and Ae 1. The vector e 1 has length 1, while Ae 1 = (1, 4, 7) has length

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### Figure 1.1 Vector A and Vector F

CHAPTER I VECTOR QUANTITIES Quantities are anything which can be measured, and stated with number. Quantities in physics are divided into two types; scalar and vector quantities. Scalar quantities have

### Bindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8

Spaces and bases Week 3: Wednesday, Feb 8 I have two favorite vector spaces 1 : R n and the space P d of polynomials of degree at most d. For R n, we have a canonical basis: R n = span{e 1, e 2,..., e

### Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors

1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number

### Vector algebra Christian Miller CS Fall 2011

Vector algebra Christian Miller CS 354 - Fall 2011 Vector algebra A system commonly used to describe space Vectors, linear operators, tensors, etc. Used to build classical physics and the vast majority

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### Interactive Math Glossary Terms and Definitions

Terms and Definitions Absolute Value the magnitude of a number, or the distance from 0 on a real number line Additive Property of Area the process of finding an the area of a shape by totaling the areas

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### Chapter 1 - Matrices & Determinants

Chapter 1 - Matrices & Determinants Arthur Cayley (August 16, 1821 - January 26, 1895) was a British Mathematician and Founder of the Modern British School of Pure Mathematics. As a child, Cayley enjoyed

### MATH 2030: SYSTEMS OF LINEAR EQUATIONS. ax + by + cz = d. )z = e. while these equations are not linear: xy z = 2, x x = 0,

MATH 23: SYSTEMS OF LINEAR EQUATIONS Systems of Linear Equations In the plane R 2 the general form of the equation of a line is ax + by = c and that the general equation of a plane in R 3 will be we call

### 1 Eigenvalues and Eigenvectors

Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### Derive 5: The Easiest... Just Got Better!

Liverpool John Moores University, 1-15 July 000 Derive 5: The Easiest... Just Got Better! Michel Beaudin École de Technologie Supérieure, Canada Email; mbeaudin@seg.etsmtl.ca 1. Introduction Engineering

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

### Solution based on matrix technique Rewrite. ) = 8x 2 1 4x 1x 2 + 5x x1 2x 2 2x 1 + 5x 2

8.2 Quadratic Forms Example 1 Consider the function q(x 1, x 2 ) = 8x 2 1 4x 1x 2 + 5x 2 2 Determine whether q(0, 0) is the global minimum. Solution based on matrix technique Rewrite q( x1 x 2 = x1 ) =

### Orthogonal Diagonalization of Symmetric Matrices

MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

### Equations Involving Lines and Planes Standard equations for lines in space

Equations Involving Lines and Planes In this section we will collect various important formulas regarding equations of lines and planes in three dimensional space Reminder regarding notation: any quantity

### Lecture 1: Schur s Unitary Triangularization Theorem

Lecture 1: Schur s Unitary Triangularization Theorem This lecture introduces the notion of unitary equivalence and presents Schur s theorem and some of its consequences It roughly corresponds to Sections

### Lecture Topic: Low-Rank Approximations

Lecture Topic: Low-Rank Approximations Low-Rank Approximations We have seen principal component analysis. The extraction of the first principle eigenvalue could be seen as an approximation of the original

### Practical Numerical Training UKNum

Practical Numerical Training UKNum 7: Systems of linear equations C. Mordasini Max Planck Institute for Astronomy, Heidelberg Program: 1) Introduction 2) Gauss Elimination 3) Gauss with Pivoting 4) Determinants

### Bindel, Fall 2012 Matrix Computations (CS 6210) Week 8: Friday, Oct 12

Why eigenvalues? Week 8: Friday, Oct 12 I spend a lot of time thinking about eigenvalue problems. In part, this is because I look for problems that can be solved via eigenvalues. But I might have fewer

### Chapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis

Chapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis 2. Polar coordinates A point P in a polar coordinate system is represented by an ordered pair of numbers (r, θ). If r >

### Presentation 3: Eigenvalues and Eigenvectors of a Matrix

Colleen Kirksey, Beth Van Schoyck, Dennis Bowers MATH 280: Problem Solving November 18, 2011 Presentation 3: Eigenvalues and Eigenvectors of a Matrix Order of Presentation: 1. Definitions of Eigenvalues

### Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.

This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra

### Manifold Learning Examples PCA, LLE and ISOMAP

Manifold Learning Examples PCA, LLE and ISOMAP Dan Ventura October 14, 28 Abstract We try to give a helpful concrete example that demonstrates how to use PCA, LLE and Isomap, attempts to provide some intuition

### Numerical Linear Algebra Chap. 4: Perturbation and Regularisation

Numerical Linear Algebra Chap. 4: Perturbation and Regularisation Heinrich Voss voss@tu-harburg.de Hamburg University of Technology Institute of Numerical Simulation TUHH Heinrich Voss Numerical Linear

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### 9 MATRICES AND TRANSFORMATIONS

9 MATRICES AND TRANSFORMATIONS Chapter 9 Matrices and Transformations Objectives After studying this chapter you should be able to handle matrix (and vector) algebra with confidence, and understand the

### = [a ij ] 2 3. Square matrix A square matrix is one that has equal number of rows and columns, that is n = m. Some examples of square matrices are

This document deals with the fundamentals of matrix algebra and is adapted from B.C. Kuo, Linear Networks and Systems, McGraw Hill, 1967. It is presented here for educational purposes. 1 Introduction In

### Mathematics Notes for Class 12 chapter 3. Matrices

1 P a g e Mathematics Notes for Class 12 chapter 3. Matrices A matrix is a rectangular arrangement of numbers (real or complex) which may be represented as matrix is enclosed by [ ] or ( ) or Compact form

### Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain

Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain 1. Orthogonal matrices and orthonormal sets An n n real-valued matrix A is said to be an orthogonal

### Algebra 2 Chapter 1 Vocabulary. identity - A statement that equates two equivalent expressions.

Chapter 1 Vocabulary identity - A statement that equates two equivalent expressions. verbal model- A word equation that represents a real-life problem. algebraic expression - An expression with variables.