Basic Linear Algebra

Basic Linear Algebra by: Dan Sunday, softsurfer.com Table of Contents Coordinate Systems 1 Points and Vectors Basic Definitions Vector Addition Scalar Multiplication 3 Affine Addition 3 Vector Length 4 Vector Products 5 The Dot Product 5 The D Perp Operator 7 The D Perp Product 7 The 3D Cross Product 9 The 3D Triple Product 11 Geometric Applications 13 Lines 13 Distance of a Point to a Line 13 Summary 15 Exercises 16 Coordinate Systems Points are positions in space. When first introduced in Greek geometry, there as no formal method for measuring here a point as located. Points ere primitie entities. Much later in history, in the early seenteenth century, Fermat and Descartes introduced the idea of using a linear coordinate system to specify point locations, using algebraic equations to describe geometric objects (such as lines), and using algebra to sole geometric problems (such as computing the intersection point of to lines). A coordinate system has an origin as an absolute reference point hose location is gien a priori, and axes (a set of ectors) that determine the directions in hich to make measurements. A coordinate system specifies points in n-dimensional space as a set of numbers (x 1, x,, x n ) and the numeric rule: start at the origin, go distance x 1 in the direction of the first axis, stop, no go distance x in the direction of the second axis, stop, and so on until done ith x n. The final location that one reaches is the point specified. This is much like finding the treasure on a pirate s map; for example, start at the old oak tree as the origin, first go east 10 paces, second go north 0 paces, and third dig don 3 feet to locate the treasure. The ordered set of numbers (x 1, x,, x n ) is called the coordinate of the point it specifies, and a coordinate system is said to span the set of points that it can specify. The set of all these points is called the coordinate space. A specific set of axes spanning a coordinate space is called a frame of reference or a basis for the space. Clearly, different coordinate systems can span the same coordinate space, just as the pirate may hae used any of seeral origin trees or orientation directions for his treasure map. Copyright 00 Dan Sunday, softsurfer.com

Points and Vectors Basic Definitions A scalar represents magnitude, and is gien by a real number: a, b, c,..., x, y, z. A point in n-dimensional space is gien by an n-tuple P = (p 1, p,, p n ) here each coordinate p i is a scalar number. We ill rite P = (p i ) as a shorthand for this n-tuple. The position of a point is relatie to a coordinate system ith an origin 0=(0,0,,0) and unit axes u 1 = (1,0,,0), u = (0,1,,0), and u n = (0,,0,1). Thus, a 3-dimensional (3D) point is gien by a triple P = (p 1, p, p 3 ) hose coordinates are relatie to the axes (1,0,0), (0,1,0), and (0,0,1) hich are usually referred to as the x, y, and z-axes. Because of this ell-knon conention, e sometimes rite P = (x, y) for D points and P = (x, y, z) for 3D points. A ector represents magnitude and direction in space, and is gien by an n-tuple = ( 1,,, n ) here each coordinate i is a scalar. We also rite = ( i ) as a shorthand for the ector s n-tuple. The ector is interpreted to be the magnitude and direction of the line segment going from the origin 0 = (0,0,,0) to the point ( 1,,, n ). Hoeer, the ector is not this point, hich only gies a standard ay of isualizing the ector. We can also isualize a ector as a directed line segment from an initial point P = (p 1,, p n ) = (p i ) to a final point Q = (q 1,, q n ) = (q i ). Then, the ector from P to Q is gien by: = Q P = (q 1 p 1,, q n p n ) = (q i p i ), shoing that the difference of any to points is a ector. In particular, ectors do not hae a fixed position in space, but can be located at any initial base point P. For example, a traeling ehicle can be said to be going east (direction) at 50 mph (magnitude) no matter here it is located. Vector Addition The sum of to ectors is gien by adding their corresponding coordinates. So, for ectors = ( i ) and = ( i ), e hae: + = ( i + i ). This can be ieed geometrically as: + One can also add a ector = ( i ) and a point P = (p i ) by adding their coordinates to get another point Q = P + = (p i + i ). The resulting point Q is the displacement, or translation, of the point P in the direction and by the magnitude of the ector = P Q as shon by: = Q P Q = P + P --

Vector addition satisfies the folloing properties: u + ( + ) = (u + ) + [Association] + = + [Commutation] Scalar Multiplication Multiplication of a ector by a scalar number is gien by the formula: a = (a i ), multiplying each component of = ( i ) by the scalar a. This represents scaling the size of a ector by a magnification factor of a. So, for example, is tice the size of, and / is half. Scalar multiplication has the properties: a(b) = (ab) [Scalar Association] (a + b) = a + b [Scalar Distribution] a( + ) = a + a [Vector Distribution] Scalar multiplication is often used to interpolate positions beteen to points P and Q. To get an intermediate point R part ay from P to Q, gien by a ratio r [0,1], one scales the ector = Q P to r, and adds it to P to get: R = P + r = P + r(q P) = (1 r)p + rq. Intermediate points are easily calculated; for example, to get the midpoint M beteen P and Q, use r = ½ to compute it as (P+Q)/. = Q P Q P M = P+ = ( P+ Q) This interpolation equation is also used to represent the line through P and Q as a function of a parameter (see the section on Geometric Applications for Lines ), namely as: P(t) = (1 t)p + tq hich is the line parametric equation. Affine Addition We hae already seen that the difference beteen to points can be considered as a ector. Hoeer, in general, it makes no sense to add to points together. Points denote an absolute position in space independent of any coordinate system describing them. Blindly adding indiidual coordinates together ould gie different ansers for different coordinate reference frames. Neertheless, there is one special case, knon as affine addition, here one can add points together as a eighted sum. In fact, in the preious section on scalar multiplication, e did just that to represent the points on a line going through to fixed points P and Q. More generally, gien m points P 0,, P m 1, one can define: -3-

Definition Affine Sum m 1 m 1 P= a P here a = 1 i i i i= 0 i= 0 here the coefficients must sum to 1. One interprets this sum as the center of mass of eights a i located at the points P i. And the center of mass is uniquely determined as the same point regardless of hat coordinate frame of reference is being used. For example, gien equal eights a 0 = a 1 = ½, e hae that P = ½P 0 + ½P 1 = (P 0 + P 1 )/ is alays the midpoint of the line segment from P 0 to P 1. Further, eery point on the line through P 0 and P 1 is uniquely represented by a pair (a 0,a 1 ) ith a 0 + a 1 = 1. Similarly, the affine sum of three non-collinear points P 0, P 1, P, defines a point in the unique plane going through these points. And also, eery point P in the plane P 0 P 1 P, is uniquely represented by a triple (a 0, a 1, a ) ith a 0 + a 1 + a = 1. This triple is called the barycentric coordinate of its associated point P on the plane defined by the triangle of points P 1 P P 3. Further, by substituting a 1 = s, a = t, and a 0 = 1 s t, one can rite the plane parametric equation to be: P(s,t) = (1 s t)p 0 + sp 1 + tp = P 0 + s(p 1 P 0 ) + t(p P 0 ) = P 0 + su + t, here u = (P 1 P 0 ) and = (P P 0 ) are independent ectors spanning the plane. The pair (s,t) is the parametric coordinate of P on the plane, and there is a unique parametric coordinate pair for each point on the plane. Vector Length The length of a ector is defined as: Definition Vector Length = i i This gies the standard Euclidean geometry (Pythagorean) length for the line segment representing a ector. For a D ector = ( 1, ), one has: It is easy to proe the formula: a = a 1 = +, hich is the Pythagorean theorem. shoing that scalar multiplication really does scale the length of a ector as one ould expect. A unit ector is one hose length = 1. One can scale any ector to get a unit ector u that points in the same direction as by computing: u =, and thus u = 1. The process of scaling to a unit ector u is called normalization, and one says that has been normalized. One thinks of u as the direction of since = u simply scales u to the magnitude. -4-

Vector Products The Dot Product The dot product (aka inner product or scalar product) of to ectors, and, is defined as the (scalar) real number gien by the sum of the products of their corresponding coordinates. This operation is denoted by a dot,, and gien by the equation: Definition = i i Dot Product 1 n i= For example, if and are D ectors, then: = 1 1 +. The dot product has the properties: = [Vector Length] (a) (b) = (ab)( ) [Scalar Association] = [Commutation] u ( + ) = (u ) + (u ) and (u + ) = (u ) + ( ) [Additie Distribution] An amazing mathematical formula for the dot product is: = cosθ here θ is the angle beteen the ectors and. This formula is used extensiely in computer graphics since it speeds up computation in many situations by aoiding direct usage of an inefficient trigonometric function. It is useful to note that hen these are unit ectors ith = = 1, then = cosθ. More generally, to compute cosθ of the angle θ beteen to ectors, the folloing formula is conenient: cosθ = = the dot product of the to normalized unit ectors. Additionally, the dot product formula can be interpreted geometrically as the projection of one ector onto the other. So, if u is a unit ector, then u is the length of the perpendicular projection of onto u, as shon in the diagram: -5-

P 1 θ P P = u 1 P u ith u = 1 Further, hen to ectors and are perpendicular, they are said to be normal to each other, and this is equialent to their dot product being zero, that is: = 0. So this is a ery simple and efficient test for perpendicularity. Because of this, for any ector = ( 1,,, n ) one can easily construct perpendicular ectors by zeroing all components except, flipping those to, and reersing the sign of one of them; for example, (, 1,0,,0), (0, 3,,0,,0), etc. The dot product of any of these ith the original ector is alays = 0, and thus they are all perpendicular to. For example, in 3D space ith = ( 1,, 3 ), the to ectors u 1 = (, 1,0) and u = (0, 3, ), hen nonzero, are a basis for the unique plane through the origin and perpendicular to. Beyond this, another important and useful consequence of the dot product cosine formula is that (for θ 180º): = 0 and are perpendicular; i.e. θ = 90 > 0 θ is an acute angle; i.e. θ < 90 < 0 θ is an obtuse angle; i.e. θ > 90 < 0 = 0 > 0 < 0 = 0 > 0-6-

The D Perp Operator Based on the preceding, e can define an operator on the D plane that gies a counterclockise (cc) normal (i.e.: perpendicular) ector of to be: Definition D Perp Operator = (, ) = (, ) 1 1 This operator, denoted by, is called the perp operator. The perp ector is the normal ector pointing to the left (cc) side of the ector as shon in the diagram: = Some properties of the perp operator are: = 0 [Perpendicular] = [Preseres length] (a + b) = a + b [Linearity] = ( ) = [Antipotent] The D Perp Product Also in D space, there is another useful scalar product of to ectors and, the perp product (aka the D exterior product, or outer product), defined by: Definition Perp Product = = = 1 1 1 1 here the x determinant is gien by: a c ad bc b d =. Some properties of the D perp product are: -7-

= 0 [Nilpotent] (a) (b) = (ab)( ) [Scalar Association] = [Antisymmetric] u ( + ) = (u ) + (u ) and (u + ) = (u ) + ( ) [Additie Distribution] ( ) + ( ) = Also, for the D perp product, e hae the formula: = sinθ hich can be used to compute sinθ from and. Further, geometrically the perp product gies the (signed) area of the D parallelogram spanned by and, as shon in the diagram: θ h = sinθ Area = bh = sinθ = To compute the area of a D triangle ith ertices V 0, V 1, V and V i = (x i, y i ) for i = 0,, define the edge ectors = V 1 V 0 and = V V 0. Then, since a triangle is half of a parallelogram, e can compute the (signed) area of V 0 V 1 V as: Area D 1 ( ) = ( ) = 1 ( x x )( x x ) 1 0 0 ( y y )( y y ) 1 0 0 V V0 1 V hich is a ery efficient computation. This signed area is positie hen the ertices V 0, V 1, V are oriented counterclockise and is negatie hen they are oriented clockise, so it can also be used to test for the orientation of a triangle. This is the same as testing for hich side of the directed line through V 0 V 1 the point V lies on: it is left of V 0 V 1 hen the area is positie, on the line hen the area = 0, and on the right side hen the area is negatie. -8-

The D perp product can also be used to determine hich side (left or right) of one ector another ector is pointing, since (for θ 180º): = 0 and are colinear; i.e. θ = 0 or 180 > 0 is left of ; i.e. 0 < θ < 180 < 0 is right of ; i.e. 0 > θ > 180 >0 > 0 = 0 <0 < 0 The 3D Cross Product The 3D cross product (aka 3D outer product or ector product) of to ectors, and, is only defined for 3D ectors, say = ( 1,, 3 ) and = ( 1,, 3 ). It is denoted by a cross,, and is defined by the equation: Definition Cross Product 3 3 1 1 =,, 3 3 1 1 here a c ad bc b d = The cross product has the properties: = 0 = (0,0,0) [Nilpotent] (a) (b) = (ab)( ) [Scalar Association] = ( ) [Antisymmetric] u ( + ) = (u ) + (u ) and (u + ) = (u ) + ( ) [Additie Distribution] ( ) = ( ) = 0 [Perpendicularity] + ( ) = Hoeer, the cross product is not associatie ith itself, and it is not distributie ith the dot product. Instead, one has the folloing formulas. These are not often used in computer graphics, but sometimes can streamline computations. Note that the formulas for left and right association are different. -9-

(u ) = (u ) ( ) u [Left Association] u ( ) = (u ) (u ) [Right Association] u ( ) = (u ) [Dot-Cross Association] (u ) ( x) = (u )( x) ( )(u x) [Lagrange s Identity] From Lagrange s Identity, e can also compute that: u = ( u ) ( u ) = ( uu )( ) ( u ) = u (1 cos ) = u sin θ hich demonstrates the important cross product formula: θ = sinθ ( θ ) = sin u ith u = 1 here θ is the angle beteen and. Note that u is perpendicular to both and. Geometrically, the cross product points outard from the -plane using a right-hand rule; and its magnitude is the area of the parallelogram spanned by and as shon in the folloing diagram: Area=Base Height = sinθ = θ Height = sinθ This fact makes the cross product ery useful for doing 3D area computations. For example, to get the area of a 3D triangle 3D = V 0 V 1 V ith ertices V 0, V 1, V, define the edge ectors = V 1 V 0 and = V V 0, and compute the area of V 0 V 1 V to be: -10-

1 Area ( 3D ) = 1 = ( V V ) ( V V ) 1 0 0 V V 1 Another important consequence of the cross product formula is that if and are perpendicular unit ectors, then is also a unit ector since sinθ = 1. Thus, the three ectors,, and form an orthogonal coordinate frame of reference (or basis) for 3D space. This is used in 3D graphics to simplify perspectie calculations from obserer iepoints. Finally, in D space, there is a relationship beteen the embedded cross product and the D perp product. One can embed a D ector = ( 1, ) in 3D space by appending a third coordinate equal to 0, namely: (,0) = ( 1,, 0). Then, for to ectors and, the embedded 3D cross product is: (,0) (,0) = (0, 0, 1 1 ) = (0, 0, ), hich only has a single non-zero third component that is equal to the perp product. The 3D Triple Product Another useful geometric computation is the triple product: V Definition Triple Product [ u] = u ( ) hich satisfies the equation: [ u] = u u u 1 3 1 3 1 3 The triple product is equal to the olume of the parallelepiped (the 3D analogue of a parallelogram) defined by the three ectors u, and starting from the same corner point as shon in the diagram: -11-

Volume(! ) u 3D = [ ] u = ( ) u From this e can get the olume of a 3D tetrahedron ith ertices V 0, V 1, V, V 3 gien by V i = (x i, y i, z i ) for i=0,3. The olume of this tetrahedron is 1/6 that of the parallelepiped spanned by the ectors u = V 1 V 0, = V V 0, and = V 3 V 0. This gies: V 1 [ ] Volume ( VVVV 0 1 3) = 1 6 ( V1 V0) ( V V0) ( V3 V0) ( x1 x0) ( y1 y0) ( z1 z0) = 1 6 ( x x0) ( y y0) ( z z0) ( x x ) ( y y ) ( z z ) 3 0 3 0 3 0 V 0 u V 3 V -1-

Geometric Applications Lines In any dimension, a line is uniquely defined by any to points P 0 and P 1 on it. The line consists of all points satisfying the linear parametric equation: P(t) = (1 t) P 0 + t P 1, here t is a real number. Each alue of t corresponds to one point P(t) on the line, for example P(0) = P 0 and P(1) = P 1, here P 0 and P 1 are the to points that defined the line. So, the (finite) line segment bounded by the endpoints P 0 and P 1 is gien by the points P(t) ith 0 t 1. Also, the line equation can be reritten as: P(t) = P 0 + t (P 1 P 0 ) = P 0 + t, here = (P 1 P 0 ) representing the line as starting at the point P 0 and extending in the direction of the ector. Distance of a Point to a Line From the properties of the dot, perp, and cross products, one can do many useful computations. One of these is computing the distance from a point to a line in D and 3D. There are a number of alternatie ays to compute this distance, depending on ho the line is represented and the dimension of the space. Here, e let the line L be gien by to points, P 0 and P 1, through hich it passes, and d(p, L) be the perpendicular distance from a point P to the line L. D Case: In the D plane e can use the dot product to compute d(p, L). A normal ector to the line is gien by n = (P 1 P 0 ). Then, d(p, L) is equal to the length of the projection of the ector from P 0 to P onto the normal n as shon in the diagram. Computing this e get: n yx ( x0) xy ( y0) dp (, L ) = P P0 cos θ = ( P P0) = n x + y here P = (x, y), P 0 = (x 0, y 0 ), P 1 = (x 1, y 1 ), x = (x 1 x 0 ), and y = (y 1 y 0 ). Note that e hae computed this as a signed alue that is positie hen P is on the left side of the directed line P 0 P 1, and negatie hen on the right side. n = ( P ) 1 P 0 P P P cosθ 0 θ d(p) P 0 P 1-13-

3D Case: In 3D space the cross product can be used to compute d(p, L). Consider the parallelogram defined by the ectors = P 1 P 0, and = P P 0. Its area is equal to. But also, ieing as the base of the parallelogram and d(p, L) as its height, then its area is equal to the base times the height, namely d( P, L ). Equating these and soling, one gets: ( P1 P0) ( P P0) dp (, L) = = P P 1 0 = Area = Base Height = P P 0 P = d( P, L) L Height = d( P, L) P 0 = P P 1 0 P 1 Hoeer, unlike the D case, this is an absolute unsigned alue, and there is no sense in saying that a point is to the left or right of a line in 3D space. The equialent concept in 3D space ould be the distance of a point to a plane, since a plane diides 3D space into to disjoint pieces, and one can compute the distance from a point to a plane as a signed quantity. -14-

Summary Notation Definitions Scalars loer case italic a, b,..., x, y,... Points UPPER CASE ITALIC P = (p 1, p,, p n ) Vectors loer case bold gothic = ( 1,,, n ) Point Difference: = Q P = (q i p i ) Vector Addition: ± = ( i ± i ) Scalar Multiplication: a = (a i ) Dot Product: = i i Vector Length: i = = i i D Perp Operator: = ( 1, ) = (, 1 ) 1 D Perp Product: = = 1 1 = 3D Cross Product: 3D Triple Product: [ ] 1 a c 3 3 1 1 b d 3 3 1 1 =,, here = ad u u u 1 3 u = u ( ) = ( u ) = 1 1 3 3 bc Basic Equations Dot Product: D Perp Product: 3D Cross Product: = cosθ u ( + ) = (u ) + (u ) = sinθ u ( + ) = (u ) + (u ) + = ( ) ( ) = sinθ = ( ) u ( + ) = (u ) + (u ) ( ) = ( ) = 0-15-

Exercises (1) Let = (8,6). Compute the length of. Gie the coordinates of a unit ector u normal to. () For to points P = (1,) and Q = (4,8), compute the intermediate points that are 1/3, 1/ and 3/4 the ay beteen P and Q. (3) Let = (8,6), = (0,5), and θ = the angle beteen and. Using the dot and perp products, compute cos( θ ) and sin( θ ). (4) Let = ( 1,, 3 ) and = ( 1,, 3 ) be any to 3D ectors. Proe that the equations: ( ) = 0 and ( ) = 0 are alays true, and thus that is alays perpendicular to both and. (5) Let u = (u 1, u, u 3 ), = ( 1,, 3 ) and = ( 1,, 3 ) be any three 3D ectors. Proe that the equation: u ( ) = (u ) is alays true. Sho, using geometric reasoning, that it is equal to the olume of the 3D parallelepiped defined by edge ectors u,, and starting at the origin. (6) Let a D (infinite) line pass through to points P 0 and P 1. Gien any arbitrary point P in the plane, sho that the perp product: (P P 0 ) (P 1 P 0 ) ill be positie for points P on one side of the line, and negatie for points on the other side of the line. (7) Using the preious exercise, deelop a test for hether a finite segment beteen points Q 0 and Q 1 crosses (i.e. intersects) the (infinite) line through P 0 and P 1. Use this to deelop another test for hether the finite segment Q 0 Q 1 intersects ith the finite segment P 0 P 1 ithout actually computing the point of intersection. (8) Let a 3D (infinite) line L be defined by to points P 0 = (x 0, y 0, z 0 ) and P 1 = (x 1, y 1, z 1 ) on it, and let P = (x, y, z) be an arbitrary point. Put x = (x 1 x 0 ), y = (y 1 y 0 ), z = (z 1 z 0 ); and r = (x x 0 ), s = (y y 0 ), t = (z z 0 ). Sho that the distance d(p, L) from P to the line L is gien by the formula: d( P, L) = ( ) ( ) ( ( ) z s y t + x t z r + y r x s) x + y + z (9) In the D plane, let a and b be to fixed perpendicular ectors, and let be any arbitrary ector. Sho that can be decomposed into the sum of to ectors in the directions of a and b. That is, sho that the linear equation = αa + βb can be soled by finding explicit formulas for the coefficients α and β in terms of a, b, and. [Hint: use the dot product.] -16-

(10) Using the preceding exercise, in the D plane, sho ho to compute the reflection of a ector off a fixed linear mirror. Write an explicit formula for the reflected ector. [Hint: let a be a ector along the mirror s surface, and let b be perpendicular to the mirror.] Note that the ector could represent the direction of trael of anything from a light ray to a billiard ball that can reflect off a straight line obstacle. (11) In 3D space, three distinct noncollinear points P 0, P 1, and P define a plane π. Let P be any 3D point, and d(p,π) be the perpendicular distance from P to the plane π. Derie a formula to compute d(p,π). [Hint: use a ector n normal to the plane π]. (1) In 3D space, let π be a plane through the points P 0, P 1, and P ; and let L be a line through the points Q 0 and Q 1. Using ector operations, derie a formula to compute the intersection of L ith π. -17-