# Numerical Analysis Lecture Notes

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number of other important norms that are used in numerical analysis. In this section, we review the basic properties of inner products and norms Inner Products. Some, but not all, norms are based on inner products. The most basic example is the familiar dot product v;w = v w = v 1 w 1 + v 2 w v n w n = n v i w i, (5.1) between (column) vectors v = ( v 1, v 2,..., v n ) T, w = ( w 1, w 2,..., w n ) T, lying in the Euclidean space R n. A key observation is that the dot product (5.1) is equal to the matrix product v w = v T w = ( v 1 v 2... v n ) w 1 w 2.. w n (5.2) between the row vector v T and the column vector w. The key fact is that the dot product of a vector with itself, v v = v v v2 n, is the sum of the squares of its entries, and hence, by the classical Pythagorean Theorem, equals the square of its length; see Figure 5.1. Consequently, the Euclidean norm or length of a vector is found by taking the square root: v = v v = v v v2 n. (5.3) Note that every nonzero vector v 0 has positive Euclidean norm, v > 0, while only the zero vector has zero norm: v = 0 if and only if v = 0. The elementary properties of dot product and Euclidean norm serve to inspire the abstract definition of more general inner products. 5/18/08 77 c 2008 Peter J. Olver

2 v v v 3 v 2 v 1 v 1 v 2 Figure 5.1. The Euclidean Norm in R 2 and R 3. Definition 5.1. An inner product on the vector space R n is a pairing that takes two vectors v,w R n and produces a real number v;w R. The inner product is required to satisfy the following three axioms for all u,v,w V, and scalars c, d R. (i) Bilinearity: cu + dv;w = c u;w + d v;w, (5.4) u;cv + dw = c u;v + d u;w. (ii) Symmetry: v;w = w;v. (5.5) (iii) Positivity: v;v > 0 whenever v 0, while 0;0 = 0. (5.6) Given an inner product, the associated norm of a vector v V is defined as the positive square root of the inner product of the vector with itself: v = v;v. (5.7) The positivity axiom implies that v 0 is real and non-negative, and equals 0 if and only if v = 0 is the zero vector. Example 5.2. While certainly the most common inner product on R 2, the dot product v w = v 1 w 1 + v 2 w 2 is by no means the only possibility. A simple example is provided by the weighted inner product ( ) ( ) v1 w1 v;w = 2v 1 w 1 + 5v 2 w 2, v =, w =. (5.8) v 2 w 2 Let us verify that this formula does indeed define an inner product. The symmetry axiom (5.5) is immediate. Moreover, cu + dv;w = 2(cu 1 + dv 1 )w 1 + 5(cu 2 + dv 2 )w 2 = c(2u 1 w 1 + 5u 2 w 2 ) + d(2v 1 w 1 + 5v 2 w 2 ) = c u;w + d v;w, 5/18/08 78 c 2008 Peter J. Olver

3 which verifies the first bilinearity condition; the second follows by a very similar computation. Moreover, 0;0 = 0, while v;v = 2v v2 2 > 0 whenever v 0, since at least one of the summands is strictly positive. This establishes (5.8) as a legitimate inner product on R 2. The associated weighted norm v = 2v v2 2 defines an alternative, non-pythagorean notion of length of vectors and distance between points in the plane. A less evident example of an inner product on R 2 is provided by the expression v;w = v 1 w 1 v 1 w 2 v 2 w 1 + 4v 2 w 2. (5.9) Bilinearity is verified in the same manner as before, and symmetry is immediate. Positivity is ensured by noticing that v;v = v 2 1 2v 1 v 2 + 4v2 2 = (v 1 v 2 )2 + 3v is always non-negative, and, moreover, is equal to zero if and only if v 1 v 2 = 0, v 2 = 0, i.e., only when v = 0. We conclude that (5.9) defines yet another inner product on R 2, with associated norm v = v;v = v 2 1 2v 1 v 2 + 4v2 2. The second example (5.8) is a particular case of a general class of inner products. Example 5.3. Let c 1,..., c n > 0 be a set of positive numbers. The corresponding weighted inner product and weighted norm on R n are defined by n v;w = c i v i w i, v = v;v = n c i vi 2. (5.10) The numbers c i are the weights. Observe that the larger the weight c i, the more the i th coordinate of v contributes to the norm. We can rewrite the weighted inner product in the useful vector form c c v;w = v T Cw, where C = 0 0 c c n (5.11) is the diagonal weight matrix. Weighted norms are particularly relevant in statistics and data fitting, [12], where one wants to emphasize certain quantities and de-emphasize others; this is done by assigning appropriate weights to the different components of the data vector v. 5/18/08 79 c 2008 Peter J. Olver

4 w θ v Figure 5.2. Angle Between Two Vectors Inequalities. There are two absolutely fundamental inequalities that are valid for any inner product on any vector space. The first is inspired by the geometric interpretation of the dot product on Euclidean space in terms of the angle between vectors. It is named after two of the founders of modern analysis, Augustin Cauchy and Herman Schwarz, who established it in the case of the L 2 inner product on function space. The more familiar triangle inequality, that the length of any side of a triangle is bounded by the sum of the lengths of the other two sides is, in fact, an immediate consequence of the Cauchy Schwarz inequality, and hence also valid for any norm based on an inner product. The Cauchy Schwarz Inequality In Euclidean geometry, the dot product between two vectors can be geometrically characterized by the equation v w = v w cos θ, (5.12) where θ measures the angle between the vectors v and w, as drawn in Figure 5.2. Since cosθ 1, the absolute value of the dot product is bounded by the product of the lengths of the vectors: v w v w. This is the simplest form of the general Cauchy Schwarz inequality. We present a simple, algebraic proof that does not rely on the geometrical notions of length and angle and thus demonstrates its universal validity for any inner product. Theorem 5.4. Every inner product satisfies the Cauchy Schwarz inequality v;w v w, for all v,w V. (5.13) Here, v is the associated norm, while denotes absolute value of real numbers. Equality holds if and only if v and w are parallel vectors. Russians also give credit for its discovery to their compatriot Viktor Bunyakovskii, and, indeed, some authors append his name to the inequality. 5/18/08 80 c 2008 Peter J. Olver

5 Proof : The case when w = 0 is trivial, since both sides of (5.13) are equal to 0. Thus, we may suppose w 0. Let t R be an arbitrary scalar. Using the three inner product axioms, we have 0 v + tw 2 = v + tw;v + tw = v 2 + 2t v;w + t 2 w 2, (5.14) with equality holding if and only if v = tw which requires v and w to be parallel vectors. We fix v and w, and consider the right hand side of (5.14) as a quadratic function, 0 p(t) = at 2 + 2bt + c, where a = w 2, b = v;w, c = v 2, of the scalar variable t. To get the maximum mileage out of the fact that p(t) 0, let us look at where it assumes its minimum, which occurs when its derivative is zero: p (t) = 2at + 2b = 0, and so t = b a = v;w w 2. Substituting this particular value of t into (5.14), we find 0 v 2 2 v;w 2 w 2 + v;w 2 w 2 Rearranging this last inequality, we conclude that = v 2 v;w 2 w 2. v;w 2 w 2 v 2, or v;w 2 v 2 w 2. Also, as noted above, equality holds if and only if v and w are parallel. Taking the (positive) square root of both sides of the final inequality completes the proof of the Cauchy Schwarz inequality (5.13). Q.E.D. Given any inner product, we can use the quotient cos θ = v;w v w (5.15) to define the angle between the vector space elements v,w V. The Cauchy Schwarz inequality tells us that the ratio lies between 1 and +1, and hence the angle θ is well defined, and, in fact, unique if we restrict it to lie in the range 0 θ π. For example, the vectors v = ( 1, 0, 1 ) T, w = (0, 1, 1 ) T have dot product v w = 1 and norms v = w = 2. Hence the Euclidean angle between them is given by cos θ = = 1 2, and so θ = 1 3 π = On the other hand, if we adopt the weighted inner product v;w = v 1 w 1 + 2v 2 w 2 + 3v 3 w 3, then v w = 3, v = 2, w = 5, and hence their weighted angle becomes cos θ = = , with θ = /18/08 81 c 2008 Peter J. Olver

6 v + w w v Figure 5.3. Triangle Inequality. Thus, the measurement of angle (and length) is dependent upon the choice of an underlying inner product. In Euclidean geometry, perpendicular vectors meet at a right angle, θ = 1 2 π or 3 2 π, with cos θ = 0. The angle formula (5.12) implies that the vectors v,w are perpendicular if and only if their dot product vanishes: v w = 0. Perpendicularity is of interest in general inner product spaces, but, for historical reasons, has been given a more suggestive name. Definition 5.5. Two elements v,w V of an inner product space V are called orthogonal if their inner product vanishes: v ; w = 0. In particular, the zero element is orthogonal to everything: 0 ; v = 0 for all v V. Orthogonality is a remarkably powerful tool in all applications of linear algebra, and often serves to dramatically simplify many computations. The Triangle Inequality The familiar triangle inequality states that the length of one side of a triangle is at most equal to the sum of the lengths of the other two sides. Referring to Figure 5.3, if the first two sides are represented by vectors v and w, then the third corresponds to their sum v + w. The triangle inequality turns out to be an elementary consequence of the Cauchy Schwarz inequality, and hence is valid in any inner product space. Theorem 5.6. The norm associated with an inner product satisfies the triangle inequality v + w v + w for all v,w V. (5.16) Equality holds if and only if v and w are parallel vectors. Proof : We compute v + w 2 = v + w ;v + w = v v;w + w 2 v v;w + w 2 v v w + w 2 = ( v + w ) 2, where the middle inequality follows from Cauchy Schwarz. Taking square roots of both sides and using positivity completes the proof. Q.E.D. 5/18/08 82 c 2008 Peter J. Olver

7 Example 5.7. The vectors v = and w = sum to v + w = Their Euclidean norms are v = 6 and w = 13, while v + w = 17. The triangle inequality (5.16) in this case says , which is valid Norms. Every inner product gives rise to a norm that can be used to measure the magnitude or length of the elements of the underlying vector space. However, not every norm that is used in analysis and applications arises from an inner product. To define a general norm, we will extract those properties that do not directly rely on the inner product structure. Definition 5.8. A norm on the vector space R n assigns a real number v to each vector v V, subject to the following axioms for every v,w V, and c R. (i) Positivity: v 0, with v = 0 if and only if v = 0. (ii) Homogeneity: cv = c v. (iii) Triangle inequality: v + w v + w. As we now know, every inner product gives rise to a norm. Indeed, positivity of the norm is one of the inner product axioms. The homogeneity property follows since cv = cv;cv = c 2 v;v = c v;v = c v. Finally, the triangle inequality for an inner product norm was established in Theorem 5.6. Let us introduce some of the principal examples of norms that do not come from inner products. The 1 norm of a vector v = ( v 1, v 2,..., v n ) T R n is defined as the sum of the absolute values of its entries: v 1 = v 1 + v v n. (5.17) The max or norm is equal to its maximal entry (in absolute value): v = max { v 1, v 2,..., v n }. (5.18) Verification of the positivity and homogeneity properties for these two norms is straightforward; the triangle inequality is a direct consequence of the elementary inequality a + b a + b, a, b R, for absolute values. The Euclidean norm, 1 norm, and norm on R n are just three representatives of the general p norm v p = p n v i p. (5.19) 5/18/08 83 c 2008 Peter J. Olver

8 v v w w Figure 5.4. Distance Between Vectors. This quantity defines a norm for any 1 p <. The norm is a limiting case of (5.19) as p. Note that the Euclidean norm (5.3) is the 2 norm, and is often designated as such; it is the only p norm which comes from an inner product. The positivity and homogeneity properties of the p norm are not hard to establish. The triangle inequality, however, is not trivial; in detail, it reads p n v i + w i p p n v i p + p n w i p, (5.20) and is known as Minkowski s inequality. A complete proof can be found in [31]. Every norm defines a distance between vector space elements, namely d(v,w) = v w. (5.21) For the standard dot product norm, we recover the usual notion of distance between points in Euclidean space. Other types of norms produce alternative (and sometimes quite useful) notions of distance that are, nevertheless, subject to all the familiar properties: (a) Symmetry: d(v, w) = d(w, v); (b) Positivity: d(v, w) = 0 if and only if v = w; (c) Triangle Inequality: d(v, w) d(v, z) + d(z, w). Equivalence of Norms While there are many different types of norms on R n, in a certain sense, they are all more or less equivalent. Equivalence does not mean that they assume the same value, but rather that they are always close to one another, and so, for many analytical purposes, may be used interchangeably. As a consequence, we may be able to simplify the analysis of a problem by choosing a suitably adapted norm. This statement remains valid in any finite-dimensional vector space, but is not correct in infinite-dimensional function spaces. 5/18/08 84 c 2008 Peter J. Olver

9 Theorem 5.9. Let 1 and 2 be any two norms on R n. Then there exist positive constants c, C > 0 such that c v 1 v 2 C v 1 for every v R n. (5.22) Proof : We just sketch the basic idea, leaving the details to a more rigorous real analysis course, cf. [11; 7.6]. We begin by noting that a norm defines a continuous real-valued function f(v) = v on R n. (Continuity is, in fact, a consequence of the triangle inequality.) Let S 1 = { u 1 = 1 } denote the unit sphere of the first norm. Any continuous function defined on a compact set achieves both a maximum and a minimum value. Thus, restricting the second norm function to the unit sphere S 1 of the first norm, we can set c = min { u 2 u S 1 }, C = max { u 2 u S 1 }. (5.23) Moreover, 0 < c C <, with equality holding if and only if the norms are the same. The minimum and maximum (5.23) will serve as the constants in the desired inequalities (5.22). Indeed, by definition, c u 2 C when u 1 = 1, (5.24) which proves that (5.22) is valid for all unit vectors v = u S 1. To prove the inequalities in general, assume v 0. (The case v = 0 is trivial.) The homogeneity property of the norm implies that u = v/ v 1 S 1 is a unit vector in the first norm: u 1 = v / v 1 = 1. Moreover, u 2 = v 2 / v 1. Substituting into (5.24) and clearing denominators completes the proof of (5.22). Q.E.D. Example For example, consider the Euclidean norm 2 and the max norm on R n. The bounding constants are found by minimizing and maximizing u = max{ u 1,..., u n } over all unit vectors u 2 = 1 on the (round) unit sphere. The maximal value is achieved ( at the poles ±e k, with ±e k = C = 1 The minimal value is attained at the points ± 1 n,..., ± 1 n ), whereby c = 1 n. Therefore, 1 n v 2 v v 2. (5.25) We can interpret these inequalities as follows. Suppose v is a vector lying on the unit sphere in the Euclidean norm, so v 2 = 1. Then (5.25) tells us that its norm is bounded from above and below by 1 n v 1. Therefore, the Euclidean unit sphere 1 sits inside the norm unit sphere and outside the norm sphere of radius n. 5/18/08 85 c 2008 Peter J. Olver

### Section 1.1. Introduction to R n

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

### α = u v. In other words, Orthogonal Projection

Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### Recall that two vectors in are perpendicular or orthogonal provided that their dot

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Lecture 14: Section 3.3

Lecture 14: Section 3.3 Shuanglin Shao October 23, 2013 Definition. Two nonzero vectors u and v in R n are said to be orthogonal (or perpendicular) if u v = 0. We will also agree that the zero vector in

### 1.3. DOT PRODUCT 19. 6. If θ is the angle (between 0 and π) between two non-zero vectors u and v,

1.3. DOT PRODUCT 19 1.3 Dot Product 1.3.1 Definitions and Properties The dot product is the first way to multiply two vectors. The definition we will give below may appear arbitrary. But it is not. It

### THREE DIMENSIONAL GEOMETRY

Chapter 8 THREE DIMENSIONAL GEOMETRY 8.1 Introduction In this chapter we present a vector algebra approach to three dimensional geometry. The aim is to present standard properties of lines and planes,

### 9 Multiplication of Vectors: The Scalar or Dot Product

Arkansas Tech University MATH 934: Calculus III Dr. Marcel B Finan 9 Multiplication of Vectors: The Scalar or Dot Product Up to this point we have defined what vectors are and discussed basic notation

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

### Section 4.4 Inner Product Spaces

Section 4.4 Inner Product Spaces In our discussion of vector spaces the specific nature of F as a field, other than the fact that it is a field, has played virtually no role. In this section we no longer

### Geometry of Vectors. 1 Cartesian Coordinates. Carlo Tomasi

Geometry of Vectors Carlo Tomasi This note explores the geometric meaning of norm, inner product, orthogonality, and projection for vectors. For vectors in three-dimensional space, we also examine the

### Cross product and determinants (Sect. 12.4) Two main ways to introduce the cross product

Cross product and determinants (Sect. 12.4) Two main ways to introduce the cross product Geometrical definition Properties Expression in components. Definition in components Properties Geometrical expression.

### 5 =5. Since 5 > 0 Since 4 7 < 0 Since 0 0

a p p e n d i x e ABSOLUTE VALUE ABSOLUTE VALUE E.1 definition. The absolute value or magnitude of a real number a is denoted by a and is defined by { a if a 0 a = a if a

### Geometric description of the cross product of the vectors u and v. The cross product of two vectors is a vector! u x v is perpendicular to u and v

12.4 Cross Product Geometric description of the cross product of the vectors u and v The cross product of two vectors is a vector! u x v is perpendicular to u and v The length of u x v is uv u v sin The

### Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors

1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1982, 2008. This chapter originates from material used by author at Imperial College, University of London, between 1981 and 1990. It is available free to all individuals,

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### i=(1,0), j=(0,1) in R 2 i=(1,0,0), j=(0,1,0), k=(0,0,1) in R 3 e 1 =(1,0,..,0), e 2 =(0,1,,0),,e n =(0,0,,1) in R n.

Length, norm, magnitude of a vector v=(v 1,,v n ) is v = (v 12 +v 22 + +v n2 ) 1/2. Examples v=(1,1,,1) v =n 1/2. Unit vectors u=v/ v corresponds to directions. Standard unit vectors i=(1,0), j=(0,1) in

### Dot product and vector projections (Sect. 12.3) There are two main ways to introduce the dot product

Dot product and vector projections (Sect. 12.3) Two definitions for the dot product. Geometric definition of dot product. Orthogonal vectors. Dot product and orthogonal projections. Properties of the dot

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### A vector is a directed line segment used to represent a vector quantity.

Chapters and 6 Introduction to Vectors A vector quantity has direction and magnitude. There are many examples of vector quantities in the natural world, such as force, velocity, and acceleration. A vector

### 5.3 The Cross Product in R 3

53 The Cross Product in R 3 Definition 531 Let u = [u 1, u 2, u 3 ] and v = [v 1, v 2, v 3 ] Then the vector given by [u 2 v 3 u 3 v 2, u 3 v 1 u 1 v 3, u 1 v 2 u 2 v 1 ] is called the cross product (or

### ISOMETRIES OF R n KEITH CONRAD

ISOMETRIES OF R n KEITH CONRAD 1. Introduction An isometry of R n is a function h: R n R n that preserves the distance between vectors: h(v) h(w) = v w for all v and w in R n, where (x 1,..., x n ) = x

### MATH 304 Linear Algebra Lecture 24: Scalar product.

MATH 304 Linear Algebra Lecture 24: Scalar product. Vectors: geometric approach B A B A A vector is represented by a directed segment. Directed segment is drawn as an arrow. Different arrows represent

### MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.

MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α

### Math 241, Exam 1 Information.

Math 241, Exam 1 Information. 9/24/12, LC 310, 11:15-12:05. Exam 1 will be based on: Sections 12.1-12.5, 14.1-14.3. The corresponding assigned homework problems (see http://www.math.sc.edu/ boylan/sccourses/241fa12/241.html)

### Section 1.4. Lines, Planes, and Hyperplanes. The Calculus of Functions of Several Variables

The Calculus of Functions of Several Variables Section 1.4 Lines, Planes, Hyperplanes In this section we will add to our basic geometric understing of R n by studying lines planes. If we do this carefully,

### Inner Product Spaces

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

### Problem Set 5 Due: In class Thursday, Oct. 18 Late papers will be accepted until 1:00 PM Friday.

Math 312, Fall 2012 Jerry L. Kazdan Problem Set 5 Due: In class Thursday, Oct. 18 Late papers will be accepted until 1:00 PM Friday. In addition to the problems below, you should also know how to solve

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### 28 CHAPTER 1. VECTORS AND THE GEOMETRY OF SPACE. v x. u y v z u z v y u y u z. v y v z

28 CHAPTER 1. VECTORS AND THE GEOMETRY OF SPACE 1.4 Cross Product 1.4.1 Definitions The cross product is the second multiplication operation between vectors we will study. The goal behind the definition

BILINEAR FORMS KEITH CONRAD The geometry of R n is controlled algebraically by the dot product. We will abstract the dot product on R n to a bilinear form on a vector space and study algebraic and geometric

### These axioms must hold for all vectors ū, v, and w in V and all scalars c and d.

DEFINITION: A vector space is a nonempty set V of objects, called vectors, on which are defined two operations, called addition and multiplication by scalars (real numbers), subject to the following axioms

### Orthogonal Bases and the QR Algorithm

Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries

### LEARNING OBJECTIVES FOR THIS CHAPTER

CHAPTER 6 Woman teaching geometry, from a fourteenth-century edition of Euclid s geometry book. Inner Product Spaces In making the definition of a vector space, we generalized the linear structure (addition

### Linear Algebra Test 2 Review by JC McNamara

Linear Algebra Test 2 Review by JC McNamara 2.3 Properties of determinants: det(a T ) = det(a) det(ka) = k n det(a) det(a + B) det(a) + det(b) (In some cases this is true but not always) A is invertible

### Vectors Math 122 Calculus III D Joyce, Fall 2012

Vectors Math 122 Calculus III D Joyce, Fall 2012 Vectors in the plane R 2. A vector v can be interpreted as an arro in the plane R 2 ith a certain length and a certain direction. The same vector can be

### x1 x 2 x 3 y 1 y 2 y 3 x 1 y 2 x 2 y 1 0.

Cross product 1 Chapter 7 Cross product We are getting ready to study integration in several variables. Until now we have been doing only differential calculus. One outcome of this study will be our ability

### Orthogonal Projections and Orthonormal Bases

CS 3, HANDOUT -A, 3 November 04 (adjusted on 7 November 04) Orthogonal Projections and Orthonormal Bases (continuation of Handout 07 of 6 September 04) Definition (Orthogonality, length, unit vectors).

### 3 Orthogonal Vectors and Matrices

3 Orthogonal Vectors and Matrices The linear algebra portion of this course focuses on three matrix factorizations: QR factorization, singular valued decomposition (SVD), and LU factorization The first

### Two vectors are equal if they have the same length and direction. They do not

Vectors define vectors Some physical quantities, such as temperature, length, and mass, can be specified by a single number called a scalar. Other physical quantities, such as force and velocity, must

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008 This chapter is available free to all individuals, on understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

### 13.4 THE CROSS PRODUCT

710 Chapter Thirteen A FUNDAMENTAL TOOL: VECTORS 62. Use the following steps and the results of Problems 59 60 to show (without trigonometry) that the geometric and algebraic definitions of the dot product

### MATH 240 Fall, Chapter 1: Linear Equations and Matrices

MATH 240 Fall, 2007 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 9th Ed. written by Prof. J. Beachy Sections 1.1 1.5, 2.1 2.3, 4.2 4.9, 3.1 3.5, 5.3 5.5, 6.1 6.3, 6.5, 7.1 7.3 DEFINITIONS

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

### Notice that v v w (4)( 15) ( 3)( 20) (0)(2) ( 2)( 15) (2)( 20) (5)(2)

The Cross Product When discussing the dot product, we showed how two vectors can be combined to get a number. Here, we shall see another way of combining vectors, this time resulting in a vector. This

### Inner product. Definition of inner product

Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product

### South Carolina College- and Career-Ready (SCCCR) Pre-Calculus

South Carolina College- and Career-Ready (SCCCR) Pre-Calculus Key Concepts Arithmetic with Polynomials and Rational Expressions PC.AAPR.2 PC.AAPR.3 PC.AAPR.4 PC.AAPR.5 PC.AAPR.6 PC.AAPR.7 Standards Know

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### 12.5 Equations of Lines and Planes

Instructor: Longfei Li Math 43 Lecture Notes.5 Equations of Lines and Planes What do we need to determine a line? D: a point on the line: P 0 (x 0, y 0 ) direction (slope): k 3D: a point on the line: P

### Equations Involving Lines and Planes Standard equations for lines in space

Equations Involving Lines and Planes In this section we will collect various important formulas regarding equations of lines and planes in three dimensional space Reminder regarding notation: any quantity

### v 1 v 3 u v = (( 1)4 (3)2, [1(4) ( 2)2], 1(3) ( 2)( 1)) = ( 10, 8, 1) (d) u (v w) = (u w)v (u v)w (Relationship between dot and cross product)

0.1 Cross Product The dot product of two vectors is a scalar, a number in R. Next we will define the cross product of two vectors in 3-space. This time the outcome will be a vector in 3-space. Definition

### Solutions to old Exam 1 problems

Solutions to old Exam 1 problems Hi students! I am putting this old version of my review for the first midterm review, place and time to be announced. Check for updates on the web site as to which sections

### Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain

Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain 1. Orthogonal matrices and orthonormal sets An n n real-valued matrix A is said to be an orthogonal

### 1 Norms and Vector Spaces

008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)

### Determinants, Areas and Volumes

Determinants, Areas and Volumes Theodore Voronov Part 2 Areas and Volumes The area of a two-dimensional object such as a region of the plane and the volume of a three-dimensional object such as a solid

### Figure 1.1 Vector A and Vector F

CHAPTER I VECTOR QUANTITIES Quantities are anything which can be measured, and stated with number. Quantities in physics are divided into two types; scalar and vector quantities. Scalar quantities have

### 1 Sets and Set Notation.

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

### Vector Algebra CHAPTER 13. Ü13.1. Basic Concepts

CHAPTER 13 ector Algebra Ü13.1. Basic Concepts A vector in the plane or in space is an arrow: it is determined by its length, denoted and its direction. Two arrows represent the same vector if they have

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### Math 4310 Handout - Quotient Vector Spaces

Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable

### Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

### 17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function

17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function, : V V R, which is symmetric, that is u, v = v, u. bilinear, that is linear (in both factors):

### Euclidean Geometry. We start with the idea of an axiomatic system. An axiomatic system has four parts:

Euclidean Geometry Students are often so challenged by the details of Euclidean geometry that they miss the rich structure of the subject. We give an overview of a piece of this structure below. We start

### Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks

Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks Welcome to Thinkwell s Homeschool Precalculus! We re thrilled that you ve decided to make us part of your homeschool curriculum. This lesson

### Math 215 HW #6 Solutions

Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 6. Eigenvalues and Singular Values In this section, we collect together the basic facts about eigenvalues and eigenvectors. From a geometrical viewpoint,

### z 0 and y even had the form

Gaussian Integers The concepts of divisibility, primality and factoring are actually more general than the discussion so far. For the moment, we have been working in the integers, which we denote by Z

### 1.5 Equations of Lines and Planes in 3-D

40 CHAPTER 1. VECTORS AND THE GEOMETRY OF SPACE Figure 1.16: Line through P 0 parallel to v 1.5 Equations of Lines and Planes in 3-D Recall that given a point P = (a, b, c), one can draw a vector from

### Example SECTION 13-1. X-AXIS - the horizontal number line. Y-AXIS - the vertical number line ORIGIN - the point where the x-axis and y-axis cross

CHAPTER 13 SECTION 13-1 Geometry and Algebra The Distance Formula COORDINATE PLANE consists of two perpendicular number lines, dividing the plane into four regions called quadrants X-AXIS - the horizontal

### Vector Math Computer Graphics Scott D. Anderson

Vector Math Computer Graphics Scott D. Anderson 1 Dot Product The notation v w means the dot product or scalar product or inner product of two vectors, v and w. In abstract mathematics, we can talk about

### Vector Spaces. Chapter 2. 2.1 R 2 through R n

Chapter 2 Vector Spaces One of my favorite dictionaries (the one from Oxford) defines a vector as A quantity having direction as well as magnitude, denoted by a line drawn from its original to its final

### MyMathLab ecourse for Developmental Mathematics

MyMathLab ecourse for Developmental Mathematics, North Shore Community College, University of New Orleans, Orange Coast College, Normandale Community College Table of Contents Module 1: Whole Numbers and

### 6. Vectors. 1 2009-2016 Scott Surgent (surgent@asu.edu)

6. Vectors For purposes of applications in calculus and physics, a vector has both a direction and a magnitude (length), and is usually represented as an arrow. The start of the arrow is the vector s foot,

### CHAPTER FIVE. 5. Equations of Lines in R 3

118 CHAPTER FIVE 5. Equations of Lines in R 3 In this chapter it is going to be very important to distinguish clearly between points and vectors. Frequently in the past the distinction has only been a

### Math 315: Linear Algebra Solutions to Midterm Exam I

Math 35: Linear Algebra s to Midterm Exam I # Consider the following two systems of linear equations (I) ax + by = k cx + dy = l (II) ax + by = 0 cx + dy = 0 (a) Prove: If x = x, y = y and x = x 2, y =

### discuss how to describe points, lines and planes in 3 space.

Chapter 2 3 Space: lines and planes In this chapter we discuss how to describe points, lines and planes in 3 space. introduce the language of vectors. discuss various matters concerning the relative position

### DEFINITION 5.1.1 A complex number is a matrix of the form. x y. , y x

Chapter 5 COMPLEX NUMBERS 5.1 Constructing the complex numbers One way of introducing the field C of complex numbers is via the arithmetic of matrices. DEFINITION 5.1.1 A complex number is a matrix of

### Section 9.5: Equations of Lines and Planes

Lines in 3D Space Section 9.5: Equations of Lines and Planes Practice HW from Stewart Textbook (not to hand in) p. 673 # 3-5 odd, 2-37 odd, 4, 47 Consider the line L through the point P = ( x, y, ) that

### v w is orthogonal to both v and w. the three vectors v, w and v w form a right-handed set of vectors.

3. Cross product Definition 3.1. Let v and w be two vectors in R 3. The cross product of v and w, denoted v w, is the vector defined as follows: the length of v w is the area of the parallelogram with

### Linear Algebra I. Ronald van Luijk, 2012

Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.

### A 0.9 0.9. Figure A: Maximum circle of compatibility for position A, related to B and C

MEASURING IN WEIGHTED ENVIRONMENTS (Moving from Metric to Order Topology) Claudio Garuti Fulcrum Ingenieria Ltda. claudiogaruti@fulcrum.cl Abstract: This article addresses the problem of measuring closeness

### Ideal Class Group and Units

Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals

### Math 0980 Chapter Objectives. Chapter 1: Introduction to Algebra: The Integers.

Math 0980 Chapter Objectives Chapter 1: Introduction to Algebra: The Integers. 1. Identify the place value of a digit. 2. Write a number in words or digits. 3. Write positive and negative numbers used

### Coefficient of Potential and Capacitance

Coefficient of Potential and Capacitance Lecture 12: Electromagnetic Theory Professor D. K. Ghosh, Physics Department, I.I.T., Bombay We know that inside a conductor there is no electric field and that

### Biggar High School Mathematics Department. National 5 Learning Intentions & Success Criteria: Assessing My Progress

Biggar High School Mathematics Department National 5 Learning Intentions & Success Criteria: Assessing My Progress Expressions & Formulae Topic Learning Intention Success Criteria I understand this Approximation

### Solutions to Exercises, Section 5.1

Instructor s Solutions Manual, Section 5.1 Exercise 1 Solutions to Exercises, Section 5.1 1. Find all numbers t such that ( 1 3,t) is a point on the unit circle. For ( 1 3,t)to be a point on the unit circle

### Factoring Patterns in the Gaussian Plane

Factoring Patterns in the Gaussian Plane Steve Phelps Introduction This paper describes discoveries made at the Park City Mathematics Institute, 00, as well as some proofs. Before the summer I understood