Scalar Product / Dot Product / Inner Product

Transcription

1 Scalar Product / Dot Product / Inner Product Scalar product, dot product, and inner product all are different name for the same operation. Scalar product should not be confused with multiplying a vector with a scalar. Scalar product refers to the product of two vectors whose result is a scalar. This operation is between two vectors of the same dimension. The notation for the operation is a DOT ( ). Scalar product between vectors a and b: a b = s Where s is a scalar.

2 How to compute scalar product? u = u u 2 u n and v = v v 2 v n u v = u u 2 u n v v 2 v n = u v + u 2 v u n v n 2

3 Properties of scalar product Property (a) : Commutative Property (b) : Distributive Property (d) : Let u = (u, u 2, u n ). u. u = u 2 + u u n 2 which is positive, and 0 only if all components of u are 0 3

4 Geometric interpretation The result of scalar product is proportional to the cosine of the angle between the vectors. a b = a b cos θ Example: a = 3, b = 0 2, a b = 2 2 cos 60 = 2 By earlier approach, a b = = 0+2=2 b θ a 4

5 Scalar product of perpendicular vectors Since a b = a b cos θ, the scalar product of two perpendicular vector is 0. (cos 90 = 0) If we are given two vectors, we can determine if they are perpendicular to each other or not. Example: u = 2 and v = 0 2 are perpendicular, because u v = 0. When two vectors are perpendicular they are linearly independent too! However, u = 2 and v = 0 are trivially perpendicular. u v = 0. 0 But they are linearly DEPENDENT! So when a question asked to find a perpendicular vector to u, DO NOT give this trivial answer!! 5

6 Scalar product of a vector with itself Since a a = a a cos 0, the scalar product of a vector with itself is a 2. (cos 0 = ) What if a is. In other words, a is a unit vector? a a = Let s make this claim stronger: a is a unit vector, if and only if a a = 6

7 Projection of vector b on vector a, and a on b a b = a b cos θ What if a is. In other words, a is a unit vector? a b = a b cos θ = b cos θ, which is the LENGTH (magnitude) of the perpendicular projection of b on a The dotted line is perpendicular to a What if a is not. b LENGTH of the perpendicular projection of b on a a b b cos θ = a LENGTH of the perpendicular projection of a on b a b a cos θ = b cos θ b 7 b sin θ

8 Projected VECTOR of vector b on vector a a b = a b cos θ Vector c is the image/perpendicular projection of b on a Direction of c is the same as a Magnitude of c is c = b cos θ = a b c = a b If a is the unit vector of a, vector c = a b a a a b a = a a a b cos θ = a b a a a c = a b a a a 8 b b sin θ

9 Orthogonal set Any set of vectors that are mutually orthogonal, is a an orthogonal set. 9

10 Orthonormal set Any set of unit vectors that are mutually orthogonal, is a an orthonormal set. In other words, any orthogonal set is an orthonormal set if all the vectors in the set are unit vectors. Example: u, u 2, u 3 is an orthonormal set, where, 3 u =, u 2 = 6 2 6, u 3 =

11 An orthogonal set is Linearly Independent

12 Orthogonal basis An orthogonal basis for a subspace W of R n is a basis for W that is also an orthogonal set. Example: 0 0, 0 0, 0 0 is basically the x, y, and z axis. It is an orthogonal basis in R 3, and it spans the whole R 3 space. It is also an orthogonal set. 2

13 Projected VECTOR of vector b on vector a a b = a b cos θ Vector c is the image/perpendicular projection of b on a Direction of c is the same as a Magnitude of c is c = b cos θ = a b c = a b If a is the unit vector of a, vector c = a b a a a b a = a a a b cos θ = a b a a a c = a b a a a 3 b b sin θ

14 Orthogonal basis We know that given a basis of a subspace, any vector in that subspace will be a linear combination of the basis vectors. For example, if u and v are linearly independent and form the basis for a subspace S, then any vector y in S can be expressed as: y = c u + c 2 v But computing c and c 2 is not straight forward. On the other hand, if u and v form an orthogonal basis, Then, c = y u = y u u u u c 2 = y v v = y v v v c 2 v y c u 4

15 Does not work if it is not Orthogonal basis y = c u + c 2 v But computing c and c 2 is not straight forward (yet). v What is computed is, d = y u u d 2 = y v v = y u u u = y v v v d 2 c 2 y u c d 5

16 For an orthogonal basis 6

17 Projection of a vector on a Subspace (span) 𝒖 and 𝒗 are orthogonal 3D vectors. They span a plane (green plane) in 3D 𝒚 is an arbitrary 3D vector out of the plane. 𝒚 is the projection of 𝒚 onto the plane. 𝒚 = 𝒚 𝒖 𝒖 𝒖 𝒖 + 𝒚 𝒗 𝒗 𝒗 𝒗 Perpendicular to 𝒚, 𝒖, 𝒗 𝒚 𝒗 𝒚 𝒖 The point 𝒚 is also the closest point to 𝒚 on the plane. 𝒚 𝒚 is perpendicular to 𝒚, Span{𝒖, 𝒗}, and hence 𝒖 𝑎𝑛𝑑 𝒗 7

18 Closest point of a vector to a span does not depend on the basis of that span 𝒚 is an arbitrary 3D vector out of the plane. 𝒚 𝒚 is the projection of 𝒚 onto the plane. 𝒖𝟒 𝒚 = 𝑥 𝒖𝟏 + 𝑥2 𝒖𝟐 𝒖𝟑 𝒚 = 𝑥3 𝒖𝟑 + 𝑥4 𝒖𝟒 𝒚 𝒚 = 𝑥5 𝒖𝟏 + 𝑥6 𝒖𝟑 𝒖𝟐 𝒖𝟏 Coordinates of 𝒚 change if the basis changes. But the vector 𝒚 itself does not change. Hence the the closest point to 𝒚 on the plane does not change. Even here, 𝒚 𝒚 is perpendicular to 𝒚 and Span{.,. } Perpendicular to 𝒚 and the span 8

19 Closest point of a vector to a span does not depend on the basis of that span FINDING THE CLOSEST POINT OF A VECTOR TO A SPAN means Finding the coordinates of the projection of the vector. So, if you want to compute the closest point of a vector to a span, then find an appropriate basis with which you can compute the coordinates of the projection easily. What would be that basis? Answer: An orthogonal basis! u 4 u 3 y y u 2 u Perpendicular to y and the span 9

20 Projection on a span of non-orthogonal vectors How to find projection of any arbitrary 3D vector onto the span of two non-orthogonal, linearly independent vectors? u and u 2 are not orthogonal, but linearly independent vectors in 3D. y is an arbitrary 3D vector. Find the projection of y in the space spanned by u and u 2. a) First, find the orthogonal set of vectors v and v 2 that span the same subspace as u and u 2. In other words, find an orthogonal basis. b) Project y onto the space spanned by orthogonal v and v 2 vectors, as we earlier. v 2 u 2 u v y y 20

21 How to find an orthogonal basis? Given two LI vectors {𝒖, 𝒖2 } find the orthogonal basis for Span{𝒖, 𝒖2 } Assume that the first vector 𝒖 is in the orthogonal basis. Other vector(s) of the basis are computed that are perpendicular to 𝒖 Let 𝒗 = 𝒖 𝒚 𝒖2 𝒗 Let 𝒗2 = 𝒖2 𝒗 𝒗 𝒗 We know that 𝒗2 is perpendicular to 𝒗. 𝒗2 is in the Span{𝒖, 𝒖2 } (Why?) So Span{𝒖, 𝒖2 } = Span{𝒗, 𝒗2 } And {𝒗, 𝒗2 } is an orthogonal basis Projection of 𝒚 on to the Span{𝒖, 𝒖2 } 𝒚 𝒗 𝒚 𝒗2 𝒚 = 𝒗 + 𝒗2 𝒗 𝒗 𝒗2 𝒖2 𝒚 𝒖 𝒗 𝒗2 𝒗2 2

22 The Gram-Schmidt process 22

23 Linear Least Squares Example A trader buys and/or sells tomatoes and potatoes. (Negative number means buys, positive number means sells.) In the process, he either makes profit (positive number) or loss (negative number). A week s transaction is shown; find the approximate cost of tomatoes and potatoes. Tomatoes (tons) Potatoes (tons) Profit/Loss (in thousands) 23

24 t - 6p = t - 2p = 2 t + p = t + 7p = 6 t p = 2 6 The above equation might not have a solution (values of t and p that would satisfy that equation). So the best we can do is to find the values of t and p that would result in a vector on the right hand side that is as close as possible to the desired right hand side vector. 24

25 25