Section 4.4 Inner Product Spaces


 Delilah Lucinda Cain
 1 years ago
 Views:
Transcription
1 Section 4.4 Inner Product Spaces In our discussion of vector spaces the specific nature of F as a field, other than the fact that it is a field, has played virtually no role. In this section we no longer consider vector spaces V over arbitrary fields F; rather, we restrict F to be the field of real or complex numbers. In the first case V is called a real vector space, in the second, a complex vector space. We all have had some experience with real vector spaces in fact both analytic geometry and the subject matter of vector analysis deal with these. What concepts used there can we carry over to a more abstract setting? To begin with we had in these concrete examples the idea of length; secondly we had the idea of perpendicularity, or, more generally, that of angle. These became special cases of the notion of a dot product (often called a scalar or inner product). Let us recall some properties of dot product as it pertained to the special case of the threedimensional real vectors. Given the vectors v = (x 1, x 2, x 3 ) and w = (y 1, y 2, y 3 ), where the x s and y s are real numbers, the dot product of v and w, denoted by v w, was defined as v w = x 1 y 1 + x 2 y 2 + x 3 y 3 Note that the length of v is given by v v and the angle θ between v and w is determined by cos θ = v w v v w w What formal properties does this dot product enjoy? We list a few: (1) v v and v v = if and only if v = ; (2) v w = w v; (3) u (αv + βw) = α(u v) + β(u w); for any vectors u, v, w and real numbers α, β. Everything that has been said can be carried over to complex vector spaces. However, to get geometrically reasonable definitions we must make some modifications. If we simply define v w = x 1 y 1 + x 2 y 2 + x 3 y 3 for v = (x 1, x 2, x 3 ) and w = (y 1, y 2, y 3 ) where the x s and y s are complex numbers, then it is quite possible that v v = with v ; this is illustrated by the vector v = (1, i, ). In fact, v v need not even be real. If, as in the real case, we should want v v to represent somehow the length of v, we should like that this length be real and that a nonzero vector should not have zero length. We can achieve this much by altering the definition of dot product slightly. If α denotes the complex conjugate of the complex number α, returning to the v and w of the paragraph above let us define v w = x 1 y 1 + x 2 y 2 + x 3 y 3. For real vectors this new definition coincides with the old one; on the other hand, for arbitrary complex vectors v, not only is v v real, it is in fact positive. Thus we have the possibility of introducing, in a natural way, a nonnegative length. However, we do lose something; for instance it is no longer true that v w = w v. In fact the exact relationship between these is v w = w v. Let us list a few properties of this dot product: 1
2 (1) v w = w v; (2) v v, and v v = if and only if v = ; (3) (αu + βv) w = α(u w) + β(v w); (4) u (αv + βw) = α(u v) + β(u w); for all complex numbers α, β and all complex vectors u, v, w. We reiterate that in what follows F is either the field of real or complex numbers. DEFINITION. The vector space V over F is said to be an inner product space if there is defined for any two vectors u, v V an element (u, v) in F such that: (1) (u, v) = (v, u); (2) (u, u) and (u, u) = if and only if u = ; (3) (αu + βv, w) = α(u, w) + β(v, w); for any u, v, w V and α, β F. A few observations about properties (1), (2), and (3) are in order. A function satisfying them is called an inner product. If F is the field of complex numbers, property (1) implies that (u, u) is real, and so property (2) makes sense. Using (1) and (3) we see that (u, αv + βw) = (αv + βw, u) = α(v, u) + β(w, u) = α(v, u) + β(w, u) = α(u, v) + β(u, w). We pause to look at some examples of inner product spaces. EXAMPLE 4.4.1: In F (n) define, for u = (α 1,..., α n ) and v = (β 1,..., β n ), This defines an inner product on F (n). (u, v) = α 1 β 1 + α 2 β α n β n. EXAMPLE 4.4.2: In F (2) define for u = (α 1, α 2 ) and v = (β 1, β 2 ), (u, v) = 2α 1 β 1 + α 1 β 2 + α 2 β 1 + α 2 β 2. It is easy to verify that this defines an inner product on F (2). EXAMPLE 4.4.3: Let V be the set of all continuous complexvalued functions on the closed unit interval [, 1]. If f(t), g(t) V, define (f(t), g(t)) = f(t)g(t)dt. We leave it to the reader to verify that this defines an inner product on V. For the remainder of this section V will denote an inner product space. DEFINITION: If v V then the length of v (or norm of v), written as v, is defined by v = (v, v). 2
3 LEMMA 4.4.1: If u, v V and α, β F then (αu + βv, αu + βv) = αα(u, u) + αβ(u, v) + αβ(v, u) + ββ(v, v). Proof: By property (3) defining an inner product space, But and (αu + βv, αu + βv) = α(u, αu + βv) + β(v, αu + βv). (u, αu + βv) = α(u, u) + β(u, v) (v, αu + βv) = α(v, u) + β(v, v). Substituting these in the expression for (αu + βv, αu + βv) we get the desired result. COROLLARY: αu = α u. Proof: We have αu 2 = (αu, αu) = αα(u, u) by Lemma (with v = ). Since αα = α 2 and (u, u) = u 2, taking square roots yields αu = α u. We digress for a moment, and prove a very elementary and familiar result about real quadratic equations. LEMMA 4.4.2: If a, b, c are real numbers such that a > and aλ 2 + 2bλ + c for all real numbers λ, then b 2 ac. Proof: Completing the squares, aλ 2 + 2bλ + c = 1 a (aλ + b)2 + ) (c b2. a Since it is greater than or equal to for all λ, in particular this must be true for λ = b a. Thus c b2 a, and since a > we get b2 ac. We now proceed to an extremely important inequality, usually known as the Schwarz inequality THEOREM 4.4.1: If u, v V then (u, v) u v. Proof: If u = then both (u, v) = and u v =, so that the result is true there. Suppose, for the moment, that (u, v) is real and u. By Lemma 4.4.1, for any real number λ, (λu + v, λu + v) = λ 2 (u, u) + 2(u, v)λ + (v, v). Let a = (u, u), b = (u, v), and c = (v, v); for these the hypothesis of Lemma is satisfied, so that b 2 ac. That is, (u, v) 2 (u, u)(v, v); from this it is immediate that (u, v) u v. 3
4 If α = (u, v) is not real, then it certainly is not, so that u/α is meaningful. Now, ( u ) α, v = 1 1 (u, v) = (u, v) = 1, α (u, v) and so it is certainly real. By the case of the Schwarz inequality discussed in the paragraph above, ( u ) 1 = α, v u v ; α since u = 1 α α u, we get 1 u v α whence α u v. Putting in that α = (u, v) we obtain (u, v) u v, the desired result. Specific cases of the Schwarz inequality are themselves of great interest. We point out two of them. (1) If V = F (n) with (u, v) = α 1 β α n β n, where u = (α 1,..., α n ) and v = (β 1,..., β n ) then Theorem implies that α 1 β α n β n 2 ( α α n 2 )( β β n 2 ). (2) If V is the set of all continuous, complexvalued functions on [, 1] with inner product defined by then Theorem implies that (f(t), g(t)) = f(t)g(t)dt 2 f(t)g(t)dt, f(t) 2 dt g(t) 2 dt. The concept of perpendicularity is an extremely useful and important one in geometry. We introduce its analog in general inner product spaces. DEFINITION: If u, v V then u is said to be orthogonal to v if (u, v) =. Note that if u is orthogonal to v then v is orthogonal to u, for (v, u) = (u, v) = =. DEFINITION. If W is a subspace of V, the orthogonal complement of W, W, is defined by W = {x V (x, w) = for all w W }. LEMMA 4.4.3: W is a subspace of V. 4
5 Proof: If a, b W then for all α, β F and all w W, since a, b W. (αa + βb, w) = α(a, w) + β(b, w) = Note that W W = (), for if w W W it must be selforthogonal, that is (w, w) =. The defining properties of an inner product space rule out this possibility unless w =. One of our goals is to show that V = W + W. Once this is done, the remark made above will become of some interest, for it will imply that V is the direct sum of W and W. DEFINITION. The set of vectors {v i } in V is an orthonormal set if (1) each v i is of length 1 (i.e., (v i, v i ) = 1) (2) for i j, (v i, v j ) =. LEMMA 4.4.4: If {v i } is an orthonormal set, then the vectors in {v i } are linearly independent. If w = α 1 v α n v n, then α i = (w, v i ) for i = 1, 2,..., n. Proof: Suppose that Therefore α 1 v 1 + α 2 v α n v n =. = (α 1 v α n v n, v i ) = α 1 (v 1, v i ) α n (v n, v i ). Since (v j, v i ) = for j i while (v i, v i ) = 1, this equation reduces to α i =. Thus the v j s are linearly independent. If w = α 1 v α n v n then computing as above yields (w, v i ) = α i. Similar in spirit and in proof to Lemma is LEMMA 4.4.5: If {v 1,..., v n } is an orthonormal set in V and if w V, then u = w (w, v 1 )v 1 (w, v 2 )v 2... (w, v i )v i... (w, v n )v n is orthogonal to each of v 1, v 2,...,v n. Proof: Computing (u, v i ) for any i n, using the orthonormality of v 1,...,v n yields the result. The construction carried out in the proof of the next theorem is one which appears and reappears in many parts of mathematics. It is a basic procedure and is known as the Gram Schmidt orthogonalization process. Although we shall be working in a finitedimensional inner product space, the GramSchmidt process works equally well in infinitedimensional situations. THEOREM 4.4.2: Let V be a finitedimensional inner product space; then V has an orthonormal set as a basis. Proof: Let V be of dimension n over F and let v 1,...,v n be a basis of V. From this basis we shall construct an orthonormal set of n vectors; by Lemma this set is linearly independent so must form a basis of V. We proceed with the construction. We seek n vectors w 1,...,w n each of length 1 such that for i j, (w i, w j ) =. In fact we shall finally produce them in the following form: w 1 will 5
6 be a multiple of v 1, w 2 will be in the linear span of w 1 and v 2, w 3 in the linear span of w 1, w 2, and v 3, and more generally, w i in the linear span of w 1, w 2,...,w i, v i. Let w 1 = v 1 v 1, then ( ) v1 (w 1, w 1 ) = v 1, v 1 = 1 v 1 v 1 2(v 1, v 1 ) = 1, whence w 1 = 1. We now ask: for what value of α is αw 1 +v 2 orthogonal to w 1? All we need is that (αw 1 + v 2, w 1 ) =, that is α(w 1, w 1 ) + (v 2, w 1 ) =. Since (w 1, w 1 ) = 1, α = (v 2, w 1 ) will do the trick. Let u 2 = (v 2, w 1 )w 1 + v 2 ; u 2 is orthogonal to w 1 ; since v 1 and v 2 are linearly independent, w 1 and v 2 must be linearly independent, and so u 2. Let w 2 = (u 2 / u 2 ); then {w 1, w 2 } is an orthonormal set. We continue. Let u 3 = (v 3, w 1 )w 1 (v 3, w 2 )w 2 + v 3. A simple check verifies that (u 3, w 1 ) = (u 3, w 2 ) =. Since w 1, w 2, and v 3 are linearly independent (for w 1, w 2 are in the linear span of v 1 and v 2 ), u 3. Let w 3 = (u 3 / u 3 ); then {w 1, w 2, w 3 } is an orthonormal set. The road ahead is now clear. Suppose that we have constructed w 1, w 2,...,w i, in the linear span of v 1,..., v i, which form an orthonormal set. How do we construct the next one, w i+1? Merely put u i+1 = (v i+1, w 1 )w 1 (v i+1, w 2 )w 2... (v i+1, w i )w i + v i+1. That u i+1 and that it is orthogonal to each of w 1,...,w i we leave to the reader. Put w i+1 = (u i+1 / u i+1 ). In this way, given r linearly independent elements in V, we can construct an orthonormal set having r elements. If particular, when dim V = n, from any basis of V we can construct an orthonormal set having n elements. This provides us with the required basis for V. We illustrate the construction used in the last proof in a concrete case. Let F be the real field and let V be the set of polynomials, in a variable x, over F of degree 2 or less. In V we define an inner product by: if p(x), q(x) V, then (p(x), q(x)) = p(x)q(x)dx. Let us start with the basis v 1 = 1, v 2 = x, v 3 = x 2 of V. Following the construction used, w 1 = v 1 v 1 = 1 = 1, u 2 = (v 2, w 1 )w 1 + v 2, 1 2 1dx which after the computations reduces to u 2 = x, and so w 2 = u 2 u 2 = 6 x x 2 dx = 3 2 x;
7 finally, and so u 3 = (v 3, w 1 )w 1 (v 3, w 2 )w 2 + v 3 = 3 + x2, w 3 = u 3 u 3 = 3 + x2 ( ) x2 dx = 1 4 ( + 3x2 ). We mentioned the next theorem earlier as one of our goals. We are now able to prove it. THEOREM 4.4.3: If V is a finitedimensional inner product space and if W is a subspace of V, then V = W + W. More particularly, V is the direct sum of W and W. Proof: Because of the highly geometric nature of the result, and because it is so basic, we give several proofs. The first will make use of Theorem and some of the earlier lemmas. The second will be motivated geometrically. First Proof: As a subspace of the inner product space V, W is itself an inner product space (its inner product being that of V restricted to W). Thus we can find an orthonormal set w 1,...,w r in W which is a basis of W. If v V by Lemma 4.4.5, v = v (v, w 1 )w 1 (v, w 2 )w 2... (v, w r )w r is orthogonal to each of w 1,..., w r and so is orthogonal to W. Thus v W, and since v = v + ((v, w 1 )w (v, w r )w r ) it follows that v W +W. Therefore V = W +W. Since W W = (), this sum is direct. Second Proof: In this proof we shall assume that F is the field of real numbers. The proof works, in almost the same way, for the complex numbers; however, it entails a few extra details which might tend to obscure the essential ideas used. Let v V ; suppose that we could find a vector w W such that v w v w for all w W. We claim that then (v w, w) = for all w W, that is, v w W. If w W, then w + w W, in consequence of which However, the righthand side is (v w, v w ) (v (w + w), v (w + w)). (w, w) + (v w, v w ) 2(v w, w), leading to 2(v w, w) (w, w) for all w W. If m is any positive integer, since w/m W we have that 2 ( m (v w, w) = 2 v w, w ) ( w m m m), w = 1 m2(w, w), and so 2(v w, w) (1/m)(w, w) for any positive integer m. However (1/m)(w, w) as m, whence 2(v w, w). Similarly, w W, and so 2(v w, w) = 2(v w, w), yielding (v w, w) = for all w W. Thus v w W ; hence v w + W W + W. 7
8 To finish the second proof we must prove the existence of a w W such that v w v w for all w W. We indicate sketchily two ways of proving the existence of such a w. Let u 1,..., u k be a basis of W; thus any w W is of the form w = λ 1 u λ k u k. Let β ij = (u i, u j ) and let γ i = (v, u i ) for v V. Thus (v w, v w) = (v λ 1 u 1... λ k u k, v λ 1 w 1... λ k w k ) = (v, v) λ i λ j β ij 2 λ i γ i. This quadratic function in the λ s is nonnegative and so, by results from the calculus, has a minimum. The λ s for this minimum, λ () 1, λ () 2,..., λ () k give us the desired vector w = λ () 1 u λ () k u k in W. A second way of exhibiting such a minimizing w is as follows. In V define a metric ζ by ζ(x, y) = x y ; one shows that ζ is a proper metric on V, and V is now a metric space. Let S = {w W v w v }. In this metric S is a compact set (prove!) and so the continuous function f(w) = v w defined for w S takes on a minimum at some point w S. We leave it to the reader to verify that w is the desired vector satisfying v w v w for all w W. COROLLARY: If V is a finitedimensional inner product space and W is a subspace of V then (W ) = W. Proof: If w W then for any u W, (w, u) =, whence W (W ). Now V = W + W and V = W + (W ) ; from these we get, since the sums are direct, dim(w) = dim((w ) ). Since W (W ) and is of the same dimension as (W ), it follows that W = (W ). 8
Inner Product Spaces
Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and
More informationInner products and orthogonality
Chapter 5 Inner products and orthogonality Inner product spaces, norms, orthogonality, GramSchmidt process Reading The list below gives examples of relevant reading. (For full publication details, see
More informationSec 4.1 Vector Spaces and Subspaces
Sec 4. Vector Spaces and Subspaces Motivation Let S be the set of all solutions to the differential equation y + y =. Let T be the set of all 2 3 matrices with real entries. These two sets share many common
More informationMA106 Linear Algebra lecture notes
MA106 Linear Algebra lecture notes Lecturers: Martin Bright and Daan Krammer Warwick, January 2011 Contents 1 Number systems and fields 3 1.1 Axioms for number systems......................... 3 2 Vector
More informationInner Product Spaces
Inner Product Spaces Definition 0.. An inner product on a vector space V is an operation on V that assigns to each pair of vectors x, y V, a real number < x, y > satisfying: I. < x, y >< y, x > for all
More informationSection 6.1  Inner Products and Norms
Section 6.1  Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,
More informationLinear Algebra Notes for Marsden and Tromba Vector Calculus
Linear Algebra Notes for Marsden and Tromba Vector Calculus ndimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of
More information3. INNER PRODUCT SPACES
. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.
More informationSimilarity and Diagonalization. Similar Matrices
MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that
More informationINNER PRODUCTS. 1. Real Inner Products Definition 1. An inner product on a real vector space V is a function (u, v) u, v from V V to R satisfying
INNER PRODUCTS 1. Real Inner Products Definition 1. An inner product on a real vector space V is a function (u, v) u, v from V V to R satisfying (1) αu + βv, w = α u, w + β v, s for all u, v, w V and all
More informationOrthogonal Complements (Revised Version)
Orthogonal Complements (Revised Version) Math 108A: May 19, 2010 John Douglas Moore 1 The dot product You will recall that the dot product was discussed in earlier calculus courses. If x = (x 1...., x
More informationFiniteDimensional Vector Spaces
Chapter 2 FiniteDimensional Vector Spaces In the last chapter we learned about vector spaces. Linear algebra focuses not on arbitrary vector spaces, but on finitedimensional vector spaces, which we introduce
More information1 VECTOR SPACES AND SUBSPACES
1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such
More informationNOTES on LINEAR ALGEBRA 1
School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura
More informationInner Product Spaces and Orthogonality
Inner Product Spaces and Orthogonality week 34 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,
More informationProjections and orthonormal bases
Projections and orthonormal bases Yehonatan Sella Consider the following situation. Given a line L R, and given any other vector v R, we can project the vector v onto the line L by dropping a perpendicular
More informationOrthogonal Diagonalization of Symmetric Matrices
MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding
More informationRecall that two vectors in are perpendicular or orthogonal provided that their dot
Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal
More informationLecture 2: Linear Algebra and Fourier Series.
Lecture : Linear Algebra and Fourier Series. 1 Introduction. At the beginning of the first lecture we gave the definition of Fourier series. Here we begin with the same definition: Definition 1.1 The Fourier
More informationChapter 6. Orthogonality
6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be
More informationInner Product Spaces
CHAPTER 6 Inner Product Spaces CHAPTER CONTENTS 6.1. Inner Products 6.2. Angle and Orthogonality in Inner Product Spaces 6.3. Gram Schmidt Process; QRDecomposition 6.4. Best Approximation; Least Squares
More informationInner product. Definition of inner product
Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product
More informationNumerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number
More informationAdditional Problems Part 1: Review
Additional Problems Part 1: Review None Part 2: Subspaces and Direct Sums DEF. If T 1 and T 2 are nonempty subsets of a vector space V, then their sum, T 1 + T 2, is the subset of V consisting of all sums
More informationLECTURE NOTES FOR 416, INNER PRODUCTS AND SPECTRAL THEOREMS
LECTURE NOTES FOR 416, INNER PRODUCTS AND SPECTRAL THEOREMS CHARLES REZK Real inner product. Let V be a vector space over R. A (real) inner product is a function, : V V R such that x, y = y, x for all
More informationOrthonormal Bases in Hilbert Space.
Linear (Vector) Spaces. Orthonormal Bases in Hilbert Space. Definition 0.1 A linear space is a nonempty set L together with a mapping from L L into L called addition, denoted (x, y) x + y and a mapping
More informationTo provide background material in support of topics in Digital Image Processing that are based on matrices and/or vectors.
Review Matrices and Vectors Objective To provide background material in support of topics in Digital Image Processing that are based on matrices and/or vectors. Some Definitions An m n (read "m by n")
More information1.3. DOT PRODUCT 19. 6. If θ is the angle (between 0 and π) between two nonzero vectors u and v,
1.3. DOT PRODUCT 19 1.3 Dot Product 1.3.1 Definitions and Properties The dot product is the first way to multiply two vectors. The definition we will give below may appear arbitrary. But it is not. It
More information6 Inner Product Spaces
Lectures 16,17,18 6 Inner Product Spaces 6.1 Basic Definition Parallelogram law, the ability to measure angle between two vectors and in particular, the concept of perpendicularity make the euclidean space
More informationDirect Sums of Subspaces and Fundamental Subspaces
Direct Sums of Subspaces and Fundamental Subspaces S. F. Ellermeyer July, 008 Direct Sums Suppose that V is a vector space and that H and K are subspaces of V such that H \ K = f0g. The direct sum of H
More informationChapter 6 Inner Product Spaces
Chapter 6 Inner Product Spaces PerOlof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 110 Linear Algebra Inner Products An inner product on a vector space
More information17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function
17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function, : V V R, which is symmetric, that is u, v = v, u. bilinear, that is linear (in both factors):
More informationSUBSPACES. Chapter Introduction. 3.2 Subspaces of F n
Chapter 3 SUBSPACES 3. Introduction Throughout this chapter, we will be studying F n, the set of all n dimensional column vectors with components from a field F. We continue our study of matrices by considering
More informationDiagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions
Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential
More informationWe call this set an ndimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.
Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to
More informationScalar Product / Dot Product / Inner Product
Scalar Product / Dot Product / Inner Product Scalar product, dot product, and inner product all are different name for the same operation. Scalar product should not be confused with multiplying a vector
More informationOrthogonal Projections and Orthonormal Bases
CS 3, HANDOUT A, 3 November 04 (adjusted on 7 November 04) Orthogonal Projections and Orthonormal Bases (continuation of Handout 07 of 6 September 04) Definition (Orthogonality, length, unit vectors).
More informationα = u v. In other words, Orthogonal Projection
Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v
More information4.12 Orthogonal Sets of Vectors and the GramSchmidt Process
4 Orthogonal Sets of Vectors and the GramSchmidt Process 33 6 Using Equation (4), determine all vectors satisfying v, v > 0 Such vectors are called spacelike vectors 7 MakeasketchofR and indicate the
More informationMATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.
MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α
More informationThe Dot Product of Two Vectors in n
These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (rd edition) These notes are intended primarily for inclass presentation
More information5 =5. Since 5 > 0 Since 4 7 < 0 Since 0 0
a p p e n d i x e ABSOLUTE VALUE ABSOLUTE VALUE E.1 definition. The absolute value or magnitude of a real number a is denoted by a and is defined by { a if a 0 a = a if a
More informationMATH 304 Linear Algebra Lecture 28: Orthogonal bases. The GramSchmidt orthogonalization process.
MATH 304 Linear Algebra Lecture 28: Orthogonal bases. The GramSchmidt orthogonalization process. Orthogonal sets Let V be an inner product space with an inner product, and the induced norm v = v,v. Definition.
More informationLecture 16: CauchySchwarz, triangle inequality, orthogonal projection, and GramSchmidt orthogonalization (1)
Lecture 16: CauchySchwarz, triangle inequality, orthogonal projection, and GramSchmidt orthogonalization (1) Travis Schedler Thurs, Nov 4, 2010 (version: Thurs, Nov 4, 4:40 PM) Goals (2) Corrected changeofbasis
More informationRANK AND NULLITY. x 1. x m
RANK AND NULLITY. The row and column spaces Let A be an m n matrix. Then A has n columns, each of which is a vector in R m. The linear span of the columns is a subspace of R n. It s called the column space
More informationMathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 19967 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
More informationSpan, Linear Independence, and Dimension
Span, and Calculus III Summer 2013, Session II Thursday, July 18, 2013 Agenda 1. 2. 3. Recap of span Yesterday, we saw how to construct a subspace of a vector space as the span of a collection of vectors.
More informationA Linear Algebra Primer James Baugh
Introduction to Vectors and Matrices Vectors and Vector Spaces Vectors are elements of a vector space which is a set of mathematical objects which can be added and multiplied by numbers (scalars) subject
More informationBACKGROUND NOTES FOR APPROXIMATION THEORY MATH 441, FALL 2009
BACKGROUND NOTES FOR APPROXIMATION THEORY MATH 441, FALL 2009 THOMAS SHORES Contents 1. Vector Spaces 1 2. Norms 4 2.1. Unit Vectors 6 2.2. Convexity 9 3. Inner Product Spaces 9 3.1. Induced Norms and
More informationChapter 5: Complex Inner Product Spaces
Chapter 5: Complex Inner Product Spaces Let us recall the inner product for the real) n dimensional Euclidean space R n : for vectors x = x 1,x,...,x n ) and y = y 1,y,...,y n ) in R n, the inner product
More informationLecture 16: CauchySchwarz, triangle inequality, orthogonal projection, and GramSchmidt orthogonalization (1)
Lecture 6: CauchySchwarz, triangle inequality, orthogonal projection, and GramSchmidt orthogonalization () Travis Schedler Thurs, Nov 4, 200 (version: Thurs, Nov 4, 4:40 PM) Goals (2) Corrected changeofbasis
More informationVectors, Gradient, Divergence and Curl.
Vectors, Gradient, Divergence and Curl. 1 Introduction A vector is determined by its length and direction. They are usually denoted with letters with arrows on the top a or in bold letter a. We will use
More informationLinear Algebra A Summary
Linear Algebra A Summary Definition: A real vector space is a set V that is provided with an addition and a multiplication such that (a) u V and v V u + v V, (1) u + v = v + u for all u V en v V, (2) u
More information4.5 Linear Dependence and Linear Independence
4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then
More informationSummary of week 8 (Lectures 22, 23 and 24)
WEEK 8 Summary of week 8 (Lectures 22, 23 and 24) This week we completed our discussion of Chapter 5 of [VST] Recall that if V and W are inner product spaces then a linear map T : V W is called an isometry
More informationMATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).
MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0
More informationLinear Span and Bases
MAT067 University of California, Davis Winter 2007 Linear Span and Bases Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (January 23, 2007) Intuition probably tells you that the plane R 2 is of dimension
More informationTHE DIMENSION OF A VECTOR SPACE
THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field
More informationWHEN DOES A CROSS PRODUCT ON R n EXIST?
WHEN DOES A CROSS PRODUCT ON R n EXIST? PETER F. MCLOUGHLIN It is probably safe to say that just about everyone reading this article is familiar with the cross product and the dot product. However, what
More informationAu = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.
Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry
More informationChapter 17. Orthogonal Matrices and Symmetries of Space
Chapter 17. Orthogonal Matrices and Symmetries of Space Take a random matrix, say 1 3 A = 4 5 6, 7 8 9 and compare the lengths of e 1 and Ae 1. The vector e 1 has length 1, while Ae 1 = (1, 4, 7) has length
More information1 Inner Products and Norms on Real Vector Spaces
Math 373: Principles Techniques of Applied Mathematics Spring 29 The 2 Inner Product 1 Inner Products Norms on Real Vector Spaces Recall that an inner product on a real vector space V is a function from
More informationFALL 2010, FINAL EXAM SOLUTIONS. Part I: (1) Define the following (5 points each). Do not assume that any vector spaces are finitedimensional
18.700 FALL 010, FINAL EXAM SOLUTIONS TRAVIS SCHEDLER Part I: 1 Define the following 5 points each. Do not assume that any vector spaces are finitedimensional unless specified. i A direct sum V = U W of
More information2: LINEAR TRANSFORMATIONS AND MATRICES
2: LINEAR TRANSFORMATIONS AND MATRICES STEVEN HEILMAN Contents 1. Review 1 2. Linear Transformations 1 3. Null spaces, range, coordinate bases 2 4. Linear Transformations and Bases 4 5. Matrix Representation,
More informationTHE HURWITZ THEOREM ON SUMS OF SQUARES BY LINEAR ALGEBRA
THE HURWITZ THEOREM ON SUMS OF SQUARES BY LINEAR ALGEBRA KEITH CONRAD 1. Introduction From commutativity of multiplication (for numbers), a product of two squares is a square: x 2 y 2 (xy) 2. A more interesting
More information9 Multiplication of Vectors: The Scalar or Dot Product
Arkansas Tech University MATH 934: Calculus III Dr. Marcel B Finan 9 Multiplication of Vectors: The Scalar or Dot Product Up to this point we have defined what vectors are and discussed basic notation
More informationMath 108B Selected Homework Solutions
Math 108B Selected Homework Solutions Charles Martin March 5, 2013 Homework 1 5.1.7 (a) If matrices A, B both represent T under different bases, then for some invertible matrix Q we have B = QAQ 1. Then
More informationSection 1.1. Introduction to R n
The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to
More information1. The Natural Numbers Denition 1.1. Given sets a and b, the order pair of a and b is dened as the set
1. The Natural Numbers Denition 1.1. Given sets a and b, the order pair of a and b is dened as the set (a, b) = {{a}, {a, b}}. For two sets A and B we dene the Cartesian product of A and B as We often
More information(, ) : R n R n R. 1. It is bilinear, meaning it s linear in each argument: that is
7 Inner products Up until now, we have only examined the properties of vectors and matrices in R n. But normally, when we think of R n, we re really thinking of ndimensional Euclidean space  that is,
More informationSection 1.2. Angles and the Dot Product. The Calculus of Functions of Several Variables
The Calculus of Functions of Several Variables Section 1.2 Angles and the Dot Product Suppose x = (x 1, x 2 ) and y = (y 1, y 2 ) are two vectors in R 2, neither of which is the zero vector 0. Let α and
More informationMath 4310 Handout  Quotient Vector Spaces
Math 4310 Handout  Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable
More informationFunctional Analysis Review
9.520 Math Camp 2010 Functional Analysis Review Andre Wibisono February 8, 2010 These notes present a brief summary of some of the basic definitions from functional analysis that we will need in this class.
More informationLinear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University
Linear Algebra Done Wrong Sergei Treil Department of Mathematics, Brown University Copyright c Sergei Treil, 2004, 2009, 2011, 2014 Preface The title of the book sounds a bit mysterious. Why should anyone
More information(u, Av) = (A T u,v), (6.4)
216 SECTION 6.1 CHAPTER 6 6.1 Hermitian Operators HERMITIAN, ORTHOGONAL, AND UNITARY OPERATORS In Chapter 4, we saw advantages in using bases consisting of eigenvectors of linear operators in a number
More information= ( 1 + i)(2 3i) (1 4i)(3 + 2i) = i 3 + 2i 2 3i 5 2i 6 + 4i b det. = (5 2i)(7i) (6 + 4i)( 3 + i) = i 3 + i 7i.
. Chapter 4 4.. # 3. Compute the determinants of the following matrices in M C): + i 4i a det + i) 3i) 4i)3 + i) + 5i 3 + i 3i 5 i 6 + 4i b det 5 i)7i) 6 + 4i) 3 + i) 36 + 4i 3 + i 7i i 3 c det 4 4 6i
More informationOrthogonal matrices. Definition: A square matrix E such that E t = E 1 is called an orthogonal matrix. Example:
Orthogonal matrices Suppose we take an orthonormal (on) basis {e, e,, e n } of R n and form the n n matrix E = (e e n ) Then E t E = e t e t (e e n ) = I n, e t n because (E t E) ij = e t ie j = e i e
More informationBasic Linear algebra. Andrew Baker
Basic Linear algebra c A. Baker Andrew Baker [08/12/2009] Department of Mathematics, University of Glasgow. Email address: a.baker@maths.gla.ac.uk URL: http://www.maths.gla.ac.uk/ ajb Linear Algebra is
More informationMATH 304 Linear Algebra Lecture 34: Review for Test 2.
MATH 304 Linear Algebra Lecture 34: Review for Test 2. Topics for Test 2 Coordinates and linear transformations (Leon 3.5, 4.1 4.3) Coordinates relative to a basis Change of basis, transition matrix Matrix
More information12.3 The Dot Product. The Dot Product. The Dot Product. The Dot Product. Example 1. Vectors and the Geometry of Space
12 Vectors and the Geometry of Space 12.3 Copyright Cengage Learning. All rights reserved. Copyright Cengage Learning. All rights reserved. So far we have added two vectors and multiplied a vector by a
More informationLEARNING OBJECTIVES FOR THIS CHAPTER
CHAPTER 6 Woman teaching geometry, from a fourteenthcentury edition of Euclid s geometry book. Inner Product Spaces In making the definition of a vector space, we generalized the linear structure (addition
More information28 CHAPTER 1. VECTORS AND THE GEOMETRY OF SPACE. v x. u y v z u z v y u y u z. v y v z
28 CHAPTER 1. VECTORS AND THE GEOMETRY OF SPACE 1.4 Cross Product 1.4.1 Definitions The cross product is the second multiplication operation between vectors we will study. The goal behind the definition
More informationBANACH AND HILBERT SPACE REVIEW
BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but
More informationMath 1180 Solutions to Homework 7
Math 1180 Solutions to Homework 7 April 10, 01 LADW Chapter 5, Number 1.8 (a) Taking v = x gives so x = 0. x = x, x = 0, (b) Let v V. Since {v 1,..., v n } is a spanning set, there are scalars c 1,...,
More informationElementary Linear Algebra
Elementary Linear Algebra Kuttler January, Saylor URL: http://wwwsaylororg/courses/ma/ Saylor URL: http://wwwsaylororg/courses/ma/ Contents Some Prerequisite Topics Sets And Set Notation Functions Graphs
More informationChapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis
Chapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis 2. Polar coordinates A point P in a polar coordinate system is represented by an ordered pair of numbers (r, θ). If r >
More informationLinear Algebra I. Ronald van Luijk, 2012
Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.
More information1. Periodic Fourier series. The Fourier expansion of a 2πperiodic function f is:
CONVERGENCE OF FOURIER SERIES 1. Periodic Fourier series. The Fourier expansion of a 2πperiodic function f is: with coefficients given by: a n = 1 π f(x) a 0 2 + a n cos(nx) + b n sin(nx), n 1 f(x) cos(nx)dx
More informationSystems of Linear Equations
Systems of Linear Equations Math 108A: August 5, 2008 John Douglas Moore 1 Orthogonal complements and null spaces of linear maps Linear algebra is the theory behind solving systems of linear equations,
More information12. Inner Product Spaces
1. Inner roduct Spaces 1.1. Vector spaces A real vector space is a set of objects that you can do to things ith: you can add to of them together to get another such object, and you can multiply one of
More informationLecture 3: The Dot Product
Lecture 3: The Dot Product 31 The angle between vectors Suppose x = (x 1, x 2 ) y = (y 1, y 2 ) are two vectors in R 2, neither of which is the zero vector 0 Let α β be the angles between x y the positive
More informationMAT 1341: REVIEW II SANGHOON BAEK
MAT 1341: REVIEW II SANGHOON BAEK 1. Projections and Cross Product 1.1. Projections. Definition 1.1. Given a vector u, the rectangular (or perpendicular or orthogonal) components are two vectors u 1 and
More informationThe Dot Product. MATH 311, Calculus III. J. Robert Buchanan. Fall Department of Mathematics
The Dot Product MATH 311, Calculus III J. Robert Buchanan Department of Mathematics Fall 2015 Dot Product The dot product is an operation performed on two vectors (the inputs) which produces a scalar (the
More informationRANK PROPERTIES OF SUBSPACES OF SYMMETRIC AND HERMITIAN MATRICES OVER FINITE FIELDS
RANK PROPERTIES OF SUBSPACES OF SYMMETRIC AND HERMITIAN MATRICES OVER FINITE FIELDS JEANGUILLAUME DUMAS ROD GOW AND JOHN SHEEKEY Abstract. We investigate constant rank subspaces of symmetric and hermitian
More informationFunctional Analysis  Summary
Functional Analysis  Summary 1 ormed spaces 1 Triangle inequalities for R: x + y x + y, a i a i 2 Cauchy s inequality: For a i, b i R, ) 1/2 ) 1/2 a i b i a i 2 b i 2 3 Hölder s inequality: For p, q >
More informationBasics from linear algebra
Basics from linear algebra Definition. A vector space is a set V with the operations of addition + : V V V, denoted w + v = +( v, w), where v, w V and multiplication by a scalar : R V V denoted r v = (r,
More informationVector coordinates, matrix elements and changes of basis
Physics 6A Winter Vector coordinates, matrix elements and changes of basis. Coordinates of vectors and matrix elements of linear operators Let V be an ndimensional real (or complex) vector space. Vectors
More informationLEARNING OBJECTIVES FOR THIS CHAPTER
CHAPTER 2 American mathematician Paul Halmos (1916 2006), who in 1942 published the first modern linear algebra book. The title of Halmos s book was the same as the title of this chapter. FiniteDimensional
More informationInner Product Spaces and Orthogonal Functions
Inner Product Spaces Orthogonal Functions Charles Byrne (Charles Byrne@uml.edu) Department of Mathematical Sciences University of Massachusetts at Lowell Lowell, MA 01854, USA May 1, 2008 1 Background
More informationMath 110, Spring 2015: Homework 5 Solutions. Exercise 2.4.4: Let A and B be n n invertible matrices. Prove that AB is invertible and (AB) 1 = B 1 A 1.
Math, Spring 25: Homework 5 Solutions Section 2.4 Exercise 2.4.4: Let A and B be n n invertible matrices. Prove that AB is invertible and (AB) = B A. Proof. We first note that the matrices AB and B A are
More information