# Inner Product Spaces

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and which satisfies: (1) u, u 0 with equality if and only if u = 0 (2) u, v = v, u and (3) αu + v, w = α u, w + v, w Combining (2) and (3), we also have u, αv + w = α u, v + u, w. Condition (1) is called positive definite, condition (2) is called symmetric and condition (3) with the note above is called bilinear. Thus an inner product is an example of a positive definite, symmetric bilinear function or form on the vector space V. Definition Let V be an inner product space and u and v be vectors in V. definitions: We make the following (1) The length or norm of the vector u is: u = u, u (2) The distance between u and v is: u v (3) The angle between u and v is: ( ) u, v θ = cos 1 u v (4) We say that u and v are orthogonal if u, v = 0 (5) The orthogonal projection of u onto the space spanned by v is: Note that, a priori, we do not know that 1 p = ( ) u, v v v, v u,v u v 1 so θ may not be defined. Later we will show that these bounds are valid and so our definition of θ is also valid. When referring to (5), we will usually say the projection of u onto v. Now we give some preliminary results: Theorem (Pythagorean Theorem). Suppose that u and v are orthogonal vectors. Then u ± v 2 = u 2 + v 2 1

2 Proof: We ll just do the case u + v 2 as the argument for the other case is similar. Note, as u and v are orthogonal, u, v = 0. Now: u + v 2 = u + v, u + v = u, u + u, v + v, u + v, v = u, u + v, v = u 2 + v 2 Theorem Suppose that p is the orthogonal projection of u onto the space spanned by v. Then p and u p are orthogonal. Proof: Recall that the projection of u onto v is given by: For notational convenience, we set β = u,v v,v. Then Therefore they are orthogonal. p = u, v v, v v p, u p = p, u p, p = βv, u βv, βv = β v, u β 2 v, v = = = 0 u, v u, v 2 v, u v, v v, v v, v 2 u, v 2 v, v u, v 2 v, v Theorem (Cauchy-Schwarz Inequality). Suppose that v and v are vectors in an inner product space. Then u, v u v Proof: Let p be the orthogonal projection of u onto v. From the previous result, p and u p are orthogonal. We may then apply the Pythagorean Theorem to get p 2 + u p 2 = (p) + (u p) 2 = u 2 Subtracting u p 2 from both sides and noting 0 u p 2 gives: In the proof of the previous theorem, we found that p 2 = u 2 u p 2 u 2 p 2 u, v 2 = p, p = v, v 2 = u, v 2 v 2

3 So we have u, v 2 v 2 u 2 u, v 2 u 2 v 2 which finally leads to u, v u v 2. Examples of Inner Product Spaces 2.1. Example: R n. Just as R n is our template for a real vector space, it serves in the same way as the archetypical inner product space. The usual inner product on R n is called the dot product or scalar product on R n. It is defined by: x, y = x T y where the right-hand side is just matrix multiplication. In particular, if then x = x 1 x 2. x n x, y = x T y = (x 1,..., x n ) and y = y 1 y 2. y n y 1 y 2. y n = x 1 y x n y n ****PROOF OF DOT PRODUCT BEING INNER PRODUCT GOES HERE**** ****GEOMETRIC PROOF OF ORTHOGONAL PROJECTIONS GOES HERE**** ****SPECIFIC EXAMPLE GOES HERE**** p = ( x T ) y y T y y x T y = x y cos(θ) An alternate inner product can be defined on R n by: x, y = x T A y where the right-hand side is just matrix multiplication. The n n matrix A must be a type of matrix known as a symmetric, positive definite matrix in order for this to satisfy the conditions of an inner product. If we choose A to be a symmetric matrix in which all of its entries are non-negative and has only positive entries on the main diagonal, then it will be such a matrix. (More generally a symmetric, positive definite matrix is a symmetric matrix with only positive eigenvalues.) 3

4 2.2. Example: R m n. Suppose A = (a ij ) and B = (b ij ) are matrices in R m n. The usual inner product on R m n is given by: m A, B = a ij b ij j=1 Note that this is the sum of the point-wise products of the elements of A and B. ****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE**** ****SPECIFIC EXAMPLE GOES HERE**** 2.3. Example: P n. Here we will describe a type of inner product on P n which we will term a discrete inner product on P n. Let {x 1,..., x n } be distinct real numbers. If p(x) is a polynomial in P n, then it has degree at most (n 1), and therefore has at most (n 1) roots. This means that p(x i ) 0 for at least one i. We know define an inner product by: p(x), q(x) = p(x i )q(x i ) ****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE**** ****SPECIFIC EXAMPLE GOES HERE**** Since every polynomial is continuous at every real number, we can use the next example of an inner product as an inner product on P n. Each of these are a continuous inner product on P n Example: C[a, b]. An inner product on C[a, b] is given by: f(x), g(x) = b a f(x)g(x)w(x) dx where w(x) is some continuous, positive real-valued function on [a, b]. The standard inner product on C[a, b] is where w(x) = 1 in the above definition. ****PROOF OF THIS PRODUCT BEING INNER PRODUCT GOES HERE**** ****SPECIFIC EXAMPLE GOES HERE**** 3. Subspaces, Orthogonal Complements and Projections Definition Let U and V be subspaces of a vector space W such that U V = {0}. The direct sum of U and V is the set U V = {u + v u U and v V }. Definition Let S be a subspace of the inner product space V. The the orthogonal complement of S is the set S = {v V v, s = 0 for all s S}. Theorem (1) If U and V are subspaces of a vector space W with U V = {0}, then U V is also a subspace of W. (2) If S is a subspace of the inner product space V, then S is also a subspace of V. Proof: (1.) Note that 0+0 = 0 is in U V. Now suppose w 1, w 2 U V, then w 1 = u 1 +v 1 and w 2 = u 2 +v 2 with u i U and v i V and w 1 + w 2 = (u 1 + v 1 ) + (u 2 + v 2 ) = (u 1 + u 2 ) + (v 1 + v 2 ). Since U and V are subspaces, it follows that w 1 +w 2 U V. Suppose now that α is a scalar, then αw 1 = α(u 1 +v 1 ) = αu 1 +αv 1. As above, it then follows that αw 1 U V. Thus U V is a subspace for W. 4

5 For (2.), first note that 0 S. Now suppose that v 1 and v 2 S. Then v 1, s = v 2, s = 0 for all s S. So v 1 + v 2, s = v 1, s + v 2, s = = 0 for all s S. Thus v 1 + v 2 S. Similarly, if α is a scalar, then αv 1, s = α v 1, s = α 0 = 0 for all s S. Thus S is a subspace of V. Theorem If U and V are subspaces of W with U V = {0} and w U V, then w = u + v for unique u U and v V. Proof: Write w = u 1 + v 1 and w = u 2 + v 2. Then u 1 + v 1 = u 2 + v 2 u 1 u 2 = v 2 v 1 u 1 u 2 = 0 = v 2 v 1 u 1 = u 2 and v 2 = v 1. Recall that one of the axioms of an inner product is that x, x 0 with equality if and only if x = 0. An immediate consequence of this is that S S = 0. Definition Let S be a subspace of the inner product space V and let {x 1,..., x n } be a basis for S such that x i, x j = 0 if i j, then this basis is called an orthogonal basis. Furthermore, if x i, x i = 1 then this basis is called an orthonormal basis. Definition Let S be a finite dimensional subspace of the inner product space V and let {x 1,..., x n } be an orthogonal basis for S. If v is any vector in V then the orthogonal projection of v onto S is the vector: v, x i p = x i, x i x i Note that if {x 1,..., x n } is an orthonormal basis, then we have the simpler expression: p = v, x i x i Also in the special case where S is spanned be the single vector x 1, then p is just the usual orthogonal projection of v onto S, which is the line spanned by x 1. Now we can prove the main theorem of this section: Theorem Let S be a finite dimensional subspace of the inner product space V and v be some vector in V. Moreover let {x 1,..., x n } be an orthogonal basis for S and p be the orthogonal projection of v onto S. Then (1) v p S. (2) V = S S. (3) If y is any vector in S with y p, then v p < v y Note that part (3.) says that p is the vector in S which is closest to v. Moreover, an immediate consequence of (2.) is that the orthogonal projection p of v onto S is independent of the choice of orthogonal basis for S. Proof: (1.) We need to show that p and v p are orthogonal. So consider p, v p = p, v p, p. Note that x i, x j = 0 when i j, so that p, v = c i x i, v with c i = v, x i n x i, x i and p, p = c i x i, c i x i p, v = v, x i x i, x i x i, v and p, p = 5 v, x i 2 x i, x i 2 x i, x i

6 Thus p, v = v, x i 2 n x i, x i and p, p = v, x i 2 x i, x i and the result follows for this part. Now let v be any vector in V, then v = p + (v p). Note that p S and from (1.), v p S, and S S = {0}. Therefore we must have V = S S, proving (2.). For part (3.), let y be some vector in S with y p. Then v p = v p + p y. Since p y S and v p S by (1.), we have By the Pythagorean Theorem. So v p 2 + p y 2 = v y 2 v p 2 = v y 2 p y 2 Since y p, p y = 0. Therefore v p 2 < v y 2 and v y < v y. Note that by (3.) of the above theorem, if v is actually in S, then p = v. Definition Let S be a subspace of the inner product space V, v be a vector in V and p be the orthogonal projection of v onto S. Then p is called the least squares approximation of v (in S) and the vector r = v p is called the residual vector of v. 4. Least squares in R n In this section we consider the following situation: Suppose that A is an m n real matrix with m > n. If b is a vector in R m then the matrix equation Ax = b corresponds to an overdetermined linear system. Generally such a system does not have a solution, however we would like to find an ˆx such that Aˆx is as close to b as possible. In this case Aˆx is the least squares approximation to b and we refer to ˆx as the least squares solution to this system. Recall that if r = b Ax, then r is the residual of this system. Moreover, our goal is then to find a x which minimizes r. Before we continue, we mention a result without proof: Theorem Suppose that A is a real matrix. Then Col(A) = N(A T ) and Col(A T ) = N(A). We will use the results of the previous section to find ˆx, or more precisely Aˆx. Given b there is a unique vector p in Col(A) such that b p is minimal by theorem Moreover, by the same theorem, b p N(A T ). Thus: A T (b p) = 0 A T b A T p = 0 A T p = A T b However, p = Aˆx for some vector ˆx (note: ˆx is not necessarily unique, but Aˆx is). So A T p = A T b A T Aˆx = A T b Thus to find ˆx we simply solve for ˆx in the equation: A T Aˆx = A T b which is necessarily consistent. Note that in this case we did not need to know an orthogonal basis for Col(A). This is because we never explicitly calculate p. 6

7 Another general fact about A in this case is that the rank of A is generally n. That is, the columns of A will usually be linearly independent. We have the following theorem which gives us an additional way to solve for ˆx in this situation: Theorem If A is an m n matrix and the rank of A is n then A T A is invertible. Proof: Clearly, N(A) is a subset of N(A T A). We now wish to show that N(A T A) is a subset of N(A). This would establish that N(A) = N(A T A). Let x N(A T A), then (A T A)x = 0 A T (Ax) = 0 Ax N(A T ). Note also that Ax Col(A) so that Ax N(A T ) Col(A). Since Col(A) = N(A T ) Ax = 0, thus x N(A) and N(A T A) = N(A). By the rank-nullity theorem we see that the rank of A T A is the same as the rank of A which is assumed to be n. As A T A is an n n matrix, it must be invertible. Thus, when A has rank n, A T A is invertible, and ˆx = (A T A) 1 A T b Now we proceed with some examples: Example 1: Consider the linear system: x 1 + x 2 = 10 2x 1 + x 2 = 5 x 1 2x 2 = 20 This system is overdetermined and inconsistent. We would like to find the least squares approximation to b and the least squares solution ˆx to this system. We can rewrite this linear system as a matrix system Ax = b where: A = 2 1 and b = It is easy to check that A has rank 2, hence A T A is invertible. Therefore: ˆx = (A T A) 1 A T b = , Aˆx = 1.71 and r = b Aˆx = Example 2: Suppose some system is modeled by a quadratic function f(x), so that f(x) = ax 2 + bx + c. Experimental data is recorded in the form (x, f(x)) with the following results: (1, 1), (2, 10), (3, 9), (4, 16) We would like to find the best approximation for f(x). Using these data points, we see that: a(1) + b(1) + c = 1 a(4) + b(2) + c = 10 a(9) + b(3) + c = 9 a(16) + b(4) + c = 16 7

8 This corresponds to the matrix equation Ax = b where: A = and b = 9 16 As in the previous example, the matrix A has full rank, hence A T A is invertible. Therefore the least squares solution to this system is: 0.5 ˆx = (A T A) 1 A T b = Therefore f(x) is approximately 0.5x x 4.5 Example 3: The orbit of a comet around the sun is either elliptical, parabolic, or hyperbolic. In particular, the orbit can be expressed by the polar equation: r = β e(r cos θ) where β is some positive constant and e is the eccentricity. Note that the orbit is elliptical if 0 e < 1, parabolic if e = 1 and hyperbolic if e > 1. A certain comet is observed over time and the following set of data (r, θ) was recorded: (1.2, 0.3), (2.1, 1.2), (4.1, 2.6), (6.3, 3.8) Using this data we would like to find the approximate orbital equation of this comet. Plugging these data points in the equation above gives us the linear system: β e(1.146) = 1.2 β e(0.761) = 2.1 β e( 3.513) = 4.1 β e( 4.983) = 6.3 This system corresponds to the matrix equation Ax = b where A = and b = Once again, the matrix A has full rank so that the least squares solution is given by: ˆx = (A T A) 1 A T b = Therefore the orbital equation is approximately r = (r cos θ). This example is similar to one of the first applications of least squares. Gauss is credited with developing the method of least squares and applying it to predicting the path of the asteroid Ceres. He did this to a remarkable degree of accuracy in

9 5. Least squares in C[a, b] Recall that an inner product in C[a, b] is given by f(x), g(x) = b a f(x)g(x)w(x) dx where w(x) is some continuous, positive function on [a, b]. Consider that we have a collection of functions {f 1 (x),..., f n (x)} which are mutually orthogonal. Moreover, assume that they form an orthogonal basis for S. Then, given any function f(x) in C[a, b], we can approximate f(x) by a linear combination of the f i. The best such approximation (in terms of least squares) will be given by the orthogonal projection p(x) of f(x) onto S. The most common application of such an approximation is in Fourier Series which will be covered in the next section. There is an analytical consideration which has to be made, and that is how good can we make this approximation. In particular, can we enlarge S is a regular way so that the limit of this process is f(x). This is a question which is beyond our scope, but the answers is yes in some cases and no in others. 6. Fourier Series In this section we consider the function space C[, π] and we wish to know if given a function f(x) in C[, π] how can we approximate this function using functions of the form cos mx and sin mx. This is obviously useful for periodic functions. Our setup is as follows: and f(x), g(x) = 1 π f(x)g(x) dx S n = Span( 1 2, sin x,..., sin nx, cos x,..., cos nx) We can check that the following are true: 1, cos αx = 1 π cos αx dx = 0 1, sin βx = 1 sin βx dx = 0 π cos αx, cos βx = 1 1 if α = β cos αx cos βx dx = π 0 if α β sin αx, sin βx = 1 1 if α = β sin αx sin βx dx = π 0 if α β Therefore is an orthonormal basis for S n. cos αx, sin βx = 1 π cos αx sin βx dx = 0 { 1 2, sin x,..., sin nx, cos x,..., cos nx} 9

10 Given a function f(x) in C[, π], the least squares approximation of f(x) in S n will be where a 0 n 2 + (a k cos kx + b k sin kx) a 0 = 1 π a k = 1 π b k = 1 π f(x) dx f(x) cos kx dx f(x) sin kx dx Note that a k and b k are just the inner products of f(x) with the basis vectors of S n and a 0 n 2 + (a k cos kx + b k sin kx) is just the orthogonal projection of f(x) onto S n. The series a0 2 + n (a k cos kx+b k sin kx) is called the nth order or nth degree Fourier series approximation of f(x) and a k, b k are called the Fourier coefficients. If we consider the approximations as partial sums, then as n we get the usual Fourier series a0 2 + (a k cos kx + b k sin kx). There still remains the question of how good an approximation this is and if this series actually converges to f(x). Now lets try see what the projection of an easy function onto S n is: Example 4: Let f(x) = x. Then clearly f(x) is a vector in C[, π]. The Fourier coefficients are: a 0 = 1 x dx = 0 π a k = 1 π x sin kx x cos kx dx = π kπ 1 sin kx dx = 0 kπ b k = 1 π x cos kx x sin kx dx = π kπ 1 cos kx dx = ( 1) k+1 2 kπ k Therfore the closest vector in S n to f(x) is p n (x) = b k sin kx = ( 1) k+1 2 sin kx k It is beyond our scope, but as n, these approximations do converge to f(x) on the interval (, π). In particular, on that interval, and If we take x = π/2, then we see that x = f(x) = ( 1) k+1 2 sin kx k x 2 = ( 1) n+1 1 sin kx k π 4 = ( 1) k+1 1 k sin k π 2 =

### MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.

MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α

### Inner product. Definition of inner product

Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product

### 5. Orthogonal matrices

L Vandenberghe EE133A (Spring 2016) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### Section 4.4 Inner Product Spaces

Section 4.4 Inner Product Spaces In our discussion of vector spaces the specific nature of F as a field, other than the fact that it is a field, has played virtually no role. In this section we no longer

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### MAT 242 Test 3 SOLUTIONS, FORM A

MAT Test SOLUTIONS, FORM A. Let v =, v =, and v =. Note that B = { v, v, v } is an orthogonal set. Also, let W be the subspace spanned by { v, v, v }. A = 8 a. [5 points] Find the orthogonal projection

### Orthogonal Diagonalization of Symmetric Matrices

MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

### Sec 4.1 Vector Spaces and Subspaces

Sec 4. Vector Spaces and Subspaces Motivation Let S be the set of all solutions to the differential equation y + y =. Let T be the set of all 2 3 matrices with real entries. These two sets share many common

### α = u v. In other words, Orthogonal Projection

Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### Math 215 HW #6 Solutions

Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T

### 12. Inner Product Spaces

1. Inner roduct Spaces 1.1. Vector spaces A real vector space is a set of objects that you can do to things ith: you can add to of them together to get another such object, and you can multiply one of

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Diagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions

Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential

### Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

### Section 6.1 - Inner Products and Norms

Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,

### Summary of week 8 (Lectures 22, 23 and 24)

WEEK 8 Summary of week 8 (Lectures 22, 23 and 24) This week we completed our discussion of Chapter 5 of [VST] Recall that if V and W are inner product spaces then a linear map T : V W is called an isometry

### 1 Orthogonal projections and the approximation

Math 1512 Fall 2010 Notes on least squares approximation Given n data points (x 1, y 1 ),..., (x n, y n ), we would like to find the line L, with an equation of the form y = mx + b, which is the best fit

### Linear Least Squares

Linear Least Squares Suppose we are given a set of data points {(x i,f i )}, i = 1,...,n. These could be measurements from an experiment or obtained simply by evaluating a function at some points. One

### MA106 Linear Algebra lecture notes

MA106 Linear Algebra lecture notes Lecturers: Martin Bright and Daan Krammer Warwick, January 2011 Contents 1 Number systems and fields 3 1.1 Axioms for number systems......................... 3 2 Vector

### Chapter 17. Orthogonal Matrices and Symmetries of Space

Chapter 17. Orthogonal Matrices and Symmetries of Space Take a random matrix, say 1 3 A = 4 5 6, 7 8 9 and compare the lengths of e 1 and Ae 1. The vector e 1 has length 1, while Ae 1 = (1, 4, 7) has length

### Fourier Series Chapter 3 of Coleman

Fourier Series Chapter 3 of Coleman Dr. Doreen De eon Math 18, Spring 14 1 Introduction Section 3.1 of Coleman The Fourier series takes its name from Joseph Fourier (1768-183), who made important contributions

### Bindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8

Spaces and bases Week 3: Wednesday, Feb 8 I have two favorite vector spaces 1 : R n and the space P d of polynomials of degree at most d. For R n, we have a canonical basis: R n = span{e 1, e 2,..., e

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### MATH 551 - APPLIED MATRIX THEORY

MATH 55 - APPLIED MATRIX THEORY FINAL TEST: SAMPLE with SOLUTIONS (25 points NAME: PROBLEM (3 points A web of 5 pages is described by a directed graph whose matrix is given by A Do the following ( points

### 1 Inner Products and Norms on Real Vector Spaces

Math 373: Principles Techniques of Applied Mathematics Spring 29 The 2 Inner Product 1 Inner Products Norms on Real Vector Spaces Recall that an inner product on a real vector space V is a function from

### Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors

1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number

### Linear Algebra Review. Vectors

Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### Elementary Linear Algebra

Elementary Linear Algebra Kuttler January, Saylor URL: http://wwwsaylororg/courses/ma/ Saylor URL: http://wwwsaylororg/courses/ma/ Contents Some Prerequisite Topics Sets And Set Notation Functions Graphs

### Notes on Orthogonal and Symmetric Matrices MENU, Winter 2013

Notes on Orthogonal and Symmetric Matrices MENU, Winter 201 These notes summarize the main properties and uses of orthogonal and symmetric matrices. We covered quite a bit of material regarding these topics,

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### Math 4310 Handout - Quotient Vector Spaces

Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Recall that two vectors in are perpendicular or orthogonal provided that their dot

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

### 17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function

17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function, : V V R, which is symmetric, that is u, v = v, u. bilinear, that is linear (in both factors):

### Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain

Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain 1. Orthogonal matrices and orthonormal sets An n n real-valued matrix A is said to be an orthogonal

### Solutions to Linear Algebra Practice Problems

Solutions to Linear Algebra Practice Problems. Find all solutions to the following systems of linear equations. (a) x x + x 5 x x x + x + x 5 (b) x + x + x x + x + x x + x + 8x Answer: (a) We create the

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008 This chapter is available free to all individuals, on understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

### 1 Eigenvalues and Eigenvectors

Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

### Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

### Linear Algebra I. Ronald van Luijk, 2012

Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.

### Vectors, Gradient, Divergence and Curl.

Vectors, Gradient, Divergence and Curl. 1 Introduction A vector is determined by its length and direction. They are usually denoted with letters with arrows on the top a or in bold letter a. We will use

### 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each)

Math 33 AH : Solution to the Final Exam Honors Linear Algebra and Applications 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each) (1) If A is an invertible

### Solutions to Math 51 First Exam January 29, 2015

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

### T ( a i x i ) = a i T (x i ).

Chapter 2 Defn 1. (p. 65) Let V and W be vector spaces (over F ). We call a function T : V W a linear transformation form V to W if, for all x, y V and c F, we have (a) T (x + y) = T (x) + T (y) and (b)

### Chapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis

Chapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis 2. Polar coordinates A point P in a polar coordinate system is represented by an ordered pair of numbers (r, θ). If r >

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### MATH 240 Fall, Chapter 1: Linear Equations and Matrices

MATH 240 Fall, 2007 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 9th Ed. written by Prof. J. Beachy Sections 1.1 1.5, 2.1 2.3, 4.2 4.9, 3.1 3.5, 5.3 5.5, 6.1 6.3, 6.5, 7.1 7.3 DEFINITIONS

### Vector Math Computer Graphics Scott D. Anderson

Vector Math Computer Graphics Scott D. Anderson 1 Dot Product The notation v w means the dot product or scalar product or inner product of two vectors, v and w. In abstract mathematics, we can talk about

### A matrix over a field F is a rectangular array of elements from F. The symbol

Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F) denotes the collection of all m n matrices over F Matrices will usually be denoted

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### Notes on Symmetric Matrices

CPSC 536N: Randomized Algorithms 2011-12 Term 2 Notes on Symmetric Matrices Prof. Nick Harvey University of British Columbia 1 Symmetric Matrices We review some basic results concerning symmetric matrices.

### Polynomials can be added or subtracted simply by adding or subtracting the corresponding terms, e.g., if

1. Polynomials 1.1. Definitions A polynomial in x is an expression obtained by taking powers of x, multiplying them by constants, and adding them. It can be written in the form c 0 x n + c 1 x n 1 + c

### Vector Spaces. 2 3x + 4x 2 7x 3) + ( 8 2x + 11x 2 + 9x 3)

Vector Spaces The idea of vectors dates back to the middle 1800 s, but our current understanding of the concept waited until Peano s work in 1888. Even then it took many years to understand the importance

### MATH 289 PROBLEM SET 1: INDUCTION. 1. The induction Principle The following property of the natural numbers is intuitively clear:

MATH 89 PROBLEM SET : INDUCTION The induction Principle The following property of the natural numbers is intuitively clear: Axiom Every nonempty subset of the set of nonnegative integers Z 0 = {0,,, 3,

### Some Notes on Taylor Polynomials and Taylor Series

Some Notes on Taylor Polynomials and Taylor Series Mark MacLean October 3, 27 UBC s courses MATH /8 and MATH introduce students to the ideas of Taylor polynomials and Taylor series in a fairly limited

### some algebra prelim solutions

some algebra prelim solutions David Morawski August 19, 2012 Problem (Spring 2008, #5). Show that f(x) = x p x + a is irreducible over F p whenever a F p is not zero. Proof. First, note that f(x) has no

### Section 1.1. Introduction to R n

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

### Homework One Solutions. Keith Fratus

Homework One Solutions Keith Fratus June 8, 011 1 Problem One 1.1 Part a In this problem, we ll assume the fact that the sum of two complex numbers is another complex number, and also that the product

### Matrix Representations of Linear Transformations and Changes of Coordinates

Matrix Representations of Linear Transformations and Changes of Coordinates 01 Subspaces and Bases 011 Definitions A subspace V of R n is a subset of R n that contains the zero element and is closed under

### Math 333 - Practice Exam 2 with Some Solutions

Math 333 - Practice Exam 2 with Some Solutions (Note that the exam will NOT be this long) Definitions (0 points) Let T : V W be a transformation Let A be a square matrix (a) Define T is linear (b) Define

### Applied Linear Algebra I Review page 1

Applied Linear Algebra Review 1 I. Determinants A. Definition of a determinant 1. Using sum a. Permutations i. Sign of a permutation ii. Cycle 2. Uniqueness of the determinant function in terms of properties

### [1] Diagonal factorization

8.03 LA.6: Diagonalization and Orthogonal Matrices [ Diagonal factorization [2 Solving systems of first order differential equations [3 Symmetric and Orthonormal Matrices [ Diagonal factorization Recall:

MA 134 Lecture Notes August 20, 2012 Introduction The purpose of this lecture is to... Introduction The purpose of this lecture is to... Learn about different types of equations Introduction The purpose

### October 3rd, 2012. Linear Algebra & Properties of the Covariance Matrix

Linear Algebra & Properties of the Covariance Matrix October 3rd, 2012 Estimation of r and C Let rn 1, rn, t..., rn T be the historical return rates on the n th asset. rn 1 rṇ 2 r n =. r T n n = 1, 2,...,

### Vector Spaces II: Finite Dimensional Linear Algebra 1

John Nachbar September 2, 2014 Vector Spaces II: Finite Dimensional Linear Algebra 1 1 Definitions and Basic Theorems. For basic properties and notation for R N, see the notes Vector Spaces I. Definition

### BANACH AND HILBERT SPACE REVIEW

BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but

### Fourier series. Jan Philip Solovej. English summary of notes for Analysis 1. May 8, 2012

Fourier series Jan Philip Solovej English summary of notes for Analysis 1 May 8, 2012 1 JPS, Fourier series 2 Contents 1 Introduction 2 2 Fourier series 3 2.1 Periodic functions, trigonometric polynomials

### 4.5 Linear Dependence and Linear Independence

4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then

### MATH31011/MATH41011/MATH61011: FOURIER ANALYSIS AND LEBESGUE INTEGRATION. Chapter 4: Fourier Series and L 2 ([ π, π], µ) ( 1 π

MATH31011/MATH41011/MATH61011: FOURIER ANALYSIS AND LEBESGUE INTEGRATION Chapter 4: Fourier Series and L ([, π], µ) Square Integrable Functions Definition. Let f : [, π] R be measurable. We say that f

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

### 9 Multiplication of Vectors: The Scalar or Dot Product

Arkansas Tech University MATH 934: Calculus III Dr. Marcel B Finan 9 Multiplication of Vectors: The Scalar or Dot Product Up to this point we have defined what vectors are and discussed basic notation

### WHICH LINEAR-FRACTIONAL TRANSFORMATIONS INDUCE ROTATIONS OF THE SPHERE?

WHICH LINEAR-FRACTIONAL TRANSFORMATIONS INDUCE ROTATIONS OF THE SPHERE? JOEL H. SHAPIRO Abstract. These notes supplement the discussion of linear fractional mappings presented in a beginning graduate course

### We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

### Problem Set 5 Due: In class Thursday, Oct. 18 Late papers will be accepted until 1:00 PM Friday.

Math 312, Fall 2012 Jerry L. Kazdan Problem Set 5 Due: In class Thursday, Oct. 18 Late papers will be accepted until 1:00 PM Friday. In addition to the problems below, you should also know how to solve

### 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is:

CONVERGENCE OF FOURIER SERIES 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is: with coefficients given by: a n = 1 π f(x) a 0 2 + a n cos(nx) + b n sin(nx), n 1 f(x) cos(nx)dx

### CITY UNIVERSITY LONDON. BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION

No: CITY UNIVERSITY LONDON BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION ENGINEERING MATHEMATICS 2 (resit) EX2005 Date: August

### Chapter 20. Vector Spaces and Bases

Chapter 20. Vector Spaces and Bases In this course, we have proceeded step-by-step through low-dimensional Linear Algebra. We have looked at lines, planes, hyperplanes, and have seen that there is no limit

### Solution based on matrix technique Rewrite. ) = 8x 2 1 4x 1x 2 + 5x x1 2x 2 2x 1 + 5x 2

8.2 Quadratic Forms Example 1 Consider the function q(x 1, x 2 ) = 8x 2 1 4x 1x 2 + 5x 2 2 Determine whether q(0, 0) is the global minimum. Solution based on matrix technique Rewrite q( x1 x 2 = x1 ) =

### minimal polyonomial Example

Minimal Polynomials Definition Let α be an element in GF(p e ). We call the monic polynomial of smallest degree which has coefficients in GF(p) and α as a root, the minimal polyonomial of α. Example: We

### 1 Norms and Vector Spaces

008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### Section 1.2. Angles and the Dot Product. The Calculus of Functions of Several Variables

The Calculus of Functions of Several Variables Section 1.2 Angles and the Dot Product Suppose x = (x 1, x 2 ) and y = (y 1, y 2 ) are two vectors in R 2, neither of which is the zero vector 0. Let α and

### Course 221: Analysis Academic year , First Semester

Course 221: Analysis Academic year 2007-08, First Semester David R. Wilkins Copyright c David R. Wilkins 1989 2007 Contents 1 Basic Theorems of Real Analysis 1 1.1 The Least Upper Bound Principle................

### Recall the basic property of the transpose (for any A): v A t Aw = v w, v, w R n.

ORTHOGONAL MATRICES Informally, an orthogonal n n matrix is the n-dimensional analogue of the rotation matrices R θ in R 2. When does a linear transformation of R 3 (or R n ) deserve to be called a rotation?

### The determinant of a skew-symmetric matrix is a square. This can be seen in small cases by direct calculation: 0 a. 12 a. a 13 a 24 a 14 a 23 a 14

4 Symplectic groups In this and the next two sections, we begin the study of the groups preserving reflexive sesquilinear forms or quadratic forms. We begin with the symplectic groups, associated with

### MATH1231 Algebra, 2015 Chapter 7: Linear maps

MATH1231 Algebra, 2015 Chapter 7: Linear maps A/Prof. Daniel Chan School of Mathematics and Statistics University of New South Wales danielc@unsw.edu.au Daniel Chan (UNSW) MATH1231 Algebra 1 / 43 Chapter

### Høgskolen i Narvik Sivilingeniørutdanningen STE6237 ELEMENTMETODER. Oppgaver

Høgskolen i Narvik Sivilingeniørutdanningen STE637 ELEMENTMETODER Oppgaver Klasse: 4.ID, 4.IT Ekstern Professor: Gregory A. Chechkin e-mail: chechkin@mech.math.msu.su Narvik 6 PART I Task. Consider two-point

### 4. Matrix inverses. left and right inverse. linear independence. nonsingular matrices. matrices with linearly independent columns

L. Vandenberghe EE133A (Spring 2016) 4. Matrix inverses left and right inverse linear independence nonsingular matrices matrices with linearly independent columns matrices with linearly independent rows

### Linear Algebra: Vectors

A Linear Algebra: Vectors A Appendix A: LINEAR ALGEBRA: VECTORS TABLE OF CONTENTS Page A Motivation A 3 A2 Vectors A 3 A2 Notational Conventions A 4 A22 Visualization A 5 A23 Special Vectors A 5 A3 Vector

### PROVING STATEMENTS IN LINEAR ALGEBRA

Mathematics V2010y Linear Algebra Spring 2007 PROVING STATEMENTS IN LINEAR ALGEBRA Linear algebra is different from calculus: you cannot understand it properly without some simple proofs. Knowing statements

### Methods for Finding Bases

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,