# Least-Squares Intersection of Lines

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a single point. A least-squares solution can be derived that minimizes the sum of perpendicular distances from the unique solution point to all the lines. L2 regularization is also considered to bias the solution towards a reference point. The solution for the case when only the direction vector parameters of the lines are known is also derived. This involves solving for the null space eigenvector of a matrix. 1 Parameterization Consider a line in 2 dimensions. It can be fully described by a point on the line: a = [ a 1 a 2 ] T, (1) and a direction vector: n = [ n 1 n 2 ] T, n 2 = n T n = 1. (2) Thus, the line is of the form: p = a + t n, < t <. (3) 2 Distance from a point to a line The squared perpendicular distance from a point to a line can be derived from the geometry involved (see Figure 1): D (p ; a, n) = (a p) ( (a p) T n ) n 2 2 (4) = (a p) nn T (a p) 2 2 (5) = ( I nn T ) (a p) 2 2 = (a p) T ( I nn T ) ( I nn T ) T (a p) (7) = (a p) T ( I nn T ) (a p), (8) The last line follows since the matrix I nn T is a projector (onto the orthocomplement of n) and is thus idempotent. It removes from a vector its component in the direction of n. This operation will only cause a change the first time it is applied. Mathematically: ( I nn T ) ( I nn T ) T = I 2nn T + nn T nn T = I nn T. (9) (6)

2 Figure 1: Perpendicular distance from a point to a line. (Source: Wikipedia) 2.1 Interpretation of cost function as Gaussian error We can interpret the quadratic distance function as corresponding to a Gaussian error term. The matrix I nn T serves to project the vectors a and p into the space orthogonal to n. Thus, minimizing the distance to the line in this projected space corresponds to maximizing the pdf of a Gaussian distribution. 3 Least-squares solution to the intersection of lines If we have K lines, they may all intersect at a unique point, only in pairs, or in other ways. We can find the unique solution to the intersection of the set of lines in a least-squares sense. The idea is to minimize the sum of squared distances: D (p ; A, N) = D (p ; a j, n j ) = (a j p) T ( I n j n T ) j (aj p). (10) This cost function corresponds to a product of independent Gaussian pdf s. Thus, the objective is: which is quadratic in p. Taking derivatives with respect to p, we have: p = argmin D (p ; A, N), (11) p D K p = 2 ( I n j n T ) j (aj p) = 0. (12) Re-arranging, we get a linear system of equations: Rp = q, (13) ( R = I nj n T ) ( j, q = I nj n T ) j aj, (14) We can either solve the system directly or apply the Moore-Penrose pseudoinverse: p = R q. (15)

3 Figure 2: (Top) Least-squares solution to intersection of K = 5 lines in 2 dimensions. This generalizes to any number of dimensions. An example for K = 5 lines in 2 dimensions is shown in Figures 2 and 3. The latter plot shows that the objective is quadratic and that the optimum is achieved at the least-squares solution. 3.1 Line confidence We might want to incorporate a confidence value, c j, that captures how severe it is to be far away from each line. In that case, we can minimize an alternative cost function: D (p ; A, N, c) = c j (a j p) T ( I n j n T ) j (aj p), (16) which leads once again to a system of linear equations: Rp = q, (17) ( R = c j I nj n T ) ( j, q = c j I nj n T ) j aj. (18) In the probabilistic interpretation, the confidence scores are precisions (inverse variances) of the Gaussian distributions. If the confidence for a particular line is relatively very large, then the solution will lie very near that line. 3.2 Bias towards a reference point We may want the solution to be reasonably close to a reference point s. This might be useful in an adaptive algorithm that updates the intersection over time. We can enforce this by regularizing the solution with a quadratic term: D (p ; A, N, s, λ) = (a j p) T ( I n j n T ) j (aj p) + λ (s p) T (s p). (19)

4 Figure 3: Negative of quadratic objective function, D (p ; A, N), showing that the optimum is achieved. Following the derivation from before, we get the following system of linear equations: Rp = q, (20) ( R = I nj n T ) K ( j + λ I, q = I nj n T ) j aj + λ s. (21) From this, we can see that as λ, p s. 4 Null space solution for partially known line parameters We can derive an eigendecomposition solution for the case where only the direction vectors n j are known. For a solution to exist, we need at least 3 lines and 3 sets of direction vectors (3 for each line). The cost function then becomes: D (P, A ; N) = L i=1 (a j p i ) T (I n ij n ij ) T (a j p i ), (22) which we would like to minimize over all choices of: ] ] P = [p 1 p L, A = [a 1 a K. (23) For simplicity, let s consider the minimal case where L = K = 3. If we stack all the parameters to be learned in a single column vector: x = then Equation 22 can be written as a single quadratic term in x: [ a T 1 a T 2 a T 3 p T 1 p T 2 p T 3 ] T, (24) D (x ; N) = x T Cx, (25)

5 where and C = G T BG, (26) I 0 0 I 0 0 I I 0 I I 0 I 0 I 0 0 G = 0 I 0 0 I 0, (27) 0 I I 0 0 I I I 0 I I 0 0 I M M M M B = M , M ij = I n ij n T ij. (28) M M M M 33 This can be derived by stacking all the difference vectors: a 1 p 1 a 1 p 2 a 1 p 3 a 2 p 1 q = a 2 p 2 a 2 p 3 a 3 p 1 a 3 p 2 a 3 p 3, (29) and determining what matrix G is necessary to construct q from x, i.e. q = Gx. Then, we can write the cost function in the form: The matrix C turns out to be: q T Bq = (Gx) T B (Gx) = x T ( G T BG ) x = x T Cx. (30)

6 M 11 + M 21 + M M 11 M 21 M 31 0 M 12 + M 22 + M 32 0 M 12 M 22 M 32 C = 0 0 M 13 + M 23 + M 33 M 13 M 23 M 33 M 11 M 12 M 13 M 11 + M 12 + M M 21 M 22 M 23 0 M 21 + M 22 + M 23 0 M 31 M 32 M M 31 + M 32 + M 33 (31) Minimizing Equation (30) as is yields the trivial solution x = 0. There are also unresolved translation, scale, and reflection ambiguities that make C rank deficient by 3 (for the 2D case). So, we will impose a constraint to make the solution unique up to a scaling factor: x T Hx = 0, (32). where I H = (33) This removes the translation ambiguity by forcing a 1 = 0, which in turn removes two degrees of freedom from the solution. Now, the problem is to minimize: D (x ; N) = x T Cx + x T Hx. (34) Taking derivatives with respect to x and setting the result equal to zero, we get the homogeneous system of equations: (C + H) x = 0. (35) The solution for x is the eigenvector v that spans the null space of C+H. Its corresponding eigenvalue is λ = 0. The vector x = v minimizes the original cost function in Equation (22) and provides a unique solution up to a real-valued scaling factor. The reflection about a 1 (i.e. the origin) is resolved by ensuring that the direction vectors implied by the solution are oriented correctly: x = { v, (p 1 a 1 ) T n 11 > 0 v, (p 1 a 1 ) T n 11 < 0. (36) Thus, we can recover both the lines and the intersection points up to a positive scaling factor from just the 9 direction vectors n ij. 4.1 Handling errors in the direction vectors When the direction vectors are not exact, multiple eigenvectors may have very small eigenvalues. This might happen if the a vectors are collinear and/or the p vectors are collinear. Numerically speaking, this makes the solution ill-conditioned. In this case, we can regularize the solution to be robust to such uncertainty by adding a scaled matrix of ones to the system:

7 Figure 4: Poorly conditioned problem. The true and estimated solutions are denoted by the empty and filled symbols, respectively. Because of the geometry involved, the solution is very sensitive to small Gaussian errors (σ = 0.1) in the direction vectors. The condition number is measured as the ratio of the largest eigenvalue of C + H to its second smallest. (C + H + λ J) x = 0, J = 11 T. (37) An example is shown in Figures 4 and 5. Although the regularization makes the value of the cost function non-zero, it allows a reasonable solution to be found. 5 Least-squares solution for unknown line orientations Consider the case where the line position parameters p j are known but the direction vectors n ij are only known up to a rotation angle θ j. This means that the relative orientation of the direction vectors is only known for each line, but not globally for all the lines. We can represent this ambiguity via a rotation matrix for each line: Then, the cost function to minimize is: [ ] cos(θ j ) sin(θ j ) R (θ j ) = sin(θ j ) cos(θ j ). (38)

8 Figure 5: Regularized solution with λ = 0.3. The objective function value is non-zero, but the solution is more reasonable. D (P, θ ; N, A) = = = L i=1 L i=1 L i=1 ( ) (a j p i ) T I R (θ j ) n ij n T ijr (θ j ) T (a j p i ) (39) (a j p i ) T R (θ j ) ( I n ij n T ij) R (θj ) T (a j p i ) (40) (a j p i ) T R (θ j ) M ij R (θ j ) T (a j p i ). (41) We can solve for the intersections p i and orientations θ j in a least-squares sense with at least K = 3 lines and L = 2 intersections. The non-linear dependence on θ j makes this a more challenging optimization problem. The circular nature of the orientation angles evokes an objective function with many local minima. The optimization must be initialized quite close to the global minimum for it to be achieved. If the point parameters a j, orientations θ j, and intersections p i are all unknown, then the cost function is the same, but is highly non-convex in the full parameter set. So, it will most likely be very difficult to find the solution or even to determine if a reasonable solution exists in the case of errors in the only known information: the direction vectors n ij.

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

### Dot product and vector projections (Sect. 12.3) There are two main ways to introduce the dot product

Dot product and vector projections (Sect. 12.3) Two definitions for the dot product. Geometric definition of dot product. Orthogonal vectors. Dot product and orthogonal projections. Properties of the dot

### Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

### Equations Involving Lines and Planes Standard equations for lines in space

Equations Involving Lines and Planes In this section we will collect various important formulas regarding equations of lines and planes in three dimensional space Reminder regarding notation: any quantity

### Orthogonal Projections

Orthogonal Projections and Reflections (with exercises) by D. Klain Version.. Corrections and comments are welcome! Orthogonal Projections Let X,..., X k be a family of linearly independent (column) vectors

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Orthogonal Projections and Orthonormal Bases

CS 3, HANDOUT -A, 3 November 04 (adjusted on 7 November 04) Orthogonal Projections and Orthonormal Bases (continuation of Handout 07 of 6 September 04) Definition (Orthogonality, length, unit vectors).

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Math 215 HW #6 Solutions

Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T

### Solutions to old Exam 1 problems

Solutions to old Exam 1 problems Hi students! I am putting this old version of my review for the first midterm review, place and time to be announced. Check for updates on the web site as to which sections

### THREE DIMENSIONAL GEOMETRY

Chapter 8 THREE DIMENSIONAL GEOMETRY 8.1 Introduction In this chapter we present a vector algebra approach to three dimensional geometry. The aim is to present standard properties of lines and planes,

### 1.3. DOT PRODUCT 19. 6. If θ is the angle (between 0 and π) between two non-zero vectors u and v,

1.3. DOT PRODUCT 19 1.3 Dot Product 1.3.1 Definitions and Properties The dot product is the first way to multiply two vectors. The definition we will give below may appear arbitrary. But it is not. It

### Solving Simultaneous Equations and Matrices

Solving Simultaneous Equations and Matrices The following represents a systematic investigation for the steps used to solve two simultaneous linear equations in two unknowns. The motivation for considering

### Section 9.5: Equations of Lines and Planes

Lines in 3D Space Section 9.5: Equations of Lines and Planes Practice HW from Stewart Textbook (not to hand in) p. 673 # 3-5 odd, 2-37 odd, 4, 47 Consider the line L through the point P = ( x, y, ) that

### α = u v. In other words, Orthogonal Projection

Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

### State of Stress at Point

State of Stress at Point Einstein Notation The basic idea of Einstein notation is that a covector and a vector can form a scalar: This is typically written as an explicit sum: According to this convention,

### Recall that two vectors in are perpendicular or orthogonal provided that their dot

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Machine Learning and Pattern Recognition Logistic Regression

Machine Learning and Pattern Recognition Logistic Regression Course Lecturer:Amos J Storkey Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh Crichton Street,

### Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components

Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they

### Dimensionality Reduction: Principal Components Analysis

Dimensionality Reduction: Principal Components Analysis In data mining one often encounters situations where there are a large number of variables in the database. In such situations it is very likely

### 3 Orthogonal Vectors and Matrices

3 Orthogonal Vectors and Matrices The linear algebra portion of this course focuses on three matrix factorizations: QR factorization, singular valued decomposition (SVD), and LU factorization The first

### 9 Multiplication of Vectors: The Scalar or Dot Product

Arkansas Tech University MATH 934: Calculus III Dr. Marcel B Finan 9 Multiplication of Vectors: The Scalar or Dot Product Up to this point we have defined what vectors are and discussed basic notation

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

### Manifold Learning Examples PCA, LLE and ISOMAP

Manifold Learning Examples PCA, LLE and ISOMAP Dan Ventura October 14, 28 Abstract We try to give a helpful concrete example that demonstrates how to use PCA, LLE and Isomap, attempts to provide some intuition

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### Recall the basic property of the transpose (for any A): v A t Aw = v w, v, w R n.

ORTHOGONAL MATRICES Informally, an orthogonal n n matrix is the n-dimensional analogue of the rotation matrices R θ in R 2. When does a linear transformation of R 3 (or R n ) deserve to be called a rotation?

### A QUICK GUIDE TO THE FORMULAS OF MULTIVARIABLE CALCULUS

A QUIK GUIDE TO THE FOMULAS OF MULTIVAIABLE ALULUS ontents 1. Analytic Geometry 2 1.1. Definition of a Vector 2 1.2. Scalar Product 2 1.3. Properties of the Scalar Product 2 1.4. Length and Unit Vectors

### 1 Sets and Set Notation.

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### 521493S Computer Graphics. Exercise 2 & course schedule change

521493S Computer Graphics Exercise 2 & course schedule change Course Schedule Change Lecture from Wednesday 31th of March is moved to Tuesday 30th of March at 16-18 in TS128 Question 2.1 Given two nonparallel,

### LINES AND PLANES IN R 3

LINES AND PLANES IN R 3 In this handout we will summarize the properties of the dot product and cross product and use them to present arious descriptions of lines and planes in three dimensional space.

### Chapter 22: Electric Flux and Gauss s Law

22.1 ntroduction We have seen in chapter 21 that determining the electric field of a continuous charge distribution can become very complicated for some charge distributions. t would be desirable if we

### Weighted-Least-Square(WLS) State Estimation

Weighted-Least-Square(WLS) State Estimation Yousu Chen PNNL December 18, 2015 This document is a description of how to formulate the weighted-least squares (WLS) state estimation problem. Most of the formulation

### RESULTANT AND DISCRIMINANT OF POLYNOMIALS

RESULTANT AND DISCRIMINANT OF POLYNOMIALS SVANTE JANSON Abstract. This is a collection of classical results about resultants and discriminants for polynomials, compiled mainly for my own use. All results

### Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors

1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### GCE Mathematics (6360) Further Pure unit 4 (MFP4) Textbook

Version 36 klm GCE Mathematics (636) Further Pure unit 4 (MFP4) Textbook The Assessment and Qualifications Alliance (AQA) is a company limited by guarantee registered in England and Wales 364473 and a

### The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression

The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression The SVD is the most generally applicable of the orthogonal-diagonal-orthogonal type matrix decompositions Every

### Lecture 14: Section 3.3

Lecture 14: Section 3.3 Shuanglin Shao October 23, 2013 Definition. Two nonzero vectors u and v in R n are said to be orthogonal (or perpendicular) if u v = 0. We will also agree that the zero vector in

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008 This chapter is available free to all individuals, on understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

### Mathematics Notes for Class 12 chapter 10. Vector Algebra

1 P a g e Mathematics Notes for Class 12 chapter 10. Vector Algebra A vector has direction and magnitude both but scalar has only magnitude. Magnitude of a vector a is denoted by a or a. It is non-negative

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### Algebra I Vocabulary Cards

Algebra I Vocabulary Cards Table of Contents Expressions and Operations Natural Numbers Whole Numbers Integers Rational Numbers Irrational Numbers Real Numbers Absolute Value Order of Operations Expression

### Metrics on SO(3) and Inverse Kinematics

Mathematical Foundations of Computer Graphics and Vision Metrics on SO(3) and Inverse Kinematics Luca Ballan Institute of Visual Computing Optimization on Manifolds Descent approach d is a ascent direction

### Lecture notes on linear algebra

Lecture notes on linear algebra David Lerner Department of Mathematics University of Kansas These are notes of a course given in Fall, 2007 and 2008 to the Honors sections of our elementary linear algebra

### NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )

Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates

### ME 115(b): Solution to Homework #1

ME 115(b): Solution to Homework #1 Solution to Problem #1: To construct the hybrid Jacobian for a manipulator, you could either construct the body Jacobian, JST b, and then use the body-to-hybrid velocity

### Linear Codes. Chapter 3. 3.1 Basics

Chapter 3 Linear Codes In order to define codes that we can encode and decode efficiently, we add more structure to the codespace. We shall be mainly interested in linear codes. A linear code of length

### ISOMETRIES OF R n KEITH CONRAD

ISOMETRIES OF R n KEITH CONRAD 1. Introduction An isometry of R n is a function h: R n R n that preserves the distance between vectors: h(v) h(w) = v w for all v and w in R n, where (x 1,..., x n ) = x

### These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop

Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher

### 5 VECTOR GEOMETRY. 5.0 Introduction. Objectives. Activity 1

5 VECTOR GEOMETRY Chapter 5 Vector Geometry Objectives After studying this chapter you should be able to find and use the vector equation of a straight line; be able to find the equation of a plane in

### Numerisches Rechnen. (für Informatiker) M. Grepl J. Berger & J.T. Frings. Institut für Geometrie und Praktische Mathematik RWTH Aachen

(für Informatiker) M. Grepl J. Berger & J.T. Frings Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2010/11 Problem Statement Unconstrained Optimality Conditions Constrained

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,

### Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication

Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication Thomas Reilly Data Physics Corporation 1741 Technology Drive, Suite 260 San Jose, CA 95110 (408) 216-8440 This paper

### A vector is a directed line segment used to represent a vector quantity.

Chapters and 6 Introduction to Vectors A vector quantity has direction and magnitude. There are many examples of vector quantities in the natural world, such as force, velocity, and acceleration. A vector

### SALEM COMMUNITY COLLEGE Carneys Point, New Jersey 08069 COURSE SYLLABUS COVER SHEET. Action Taken (Please Check One) New Course Initiated

SALEM COMMUNITY COLLEGE Carneys Point, New Jersey 08069 COURSE SYLLABUS COVER SHEET Course Title Course Number Department Linear Algebra Mathematics MAT-240 Action Taken (Please Check One) New Course Initiated

### Lecture 6. Weight. Tension. Normal Force. Static Friction. Cutnell+Johnson: 4.8-4.12, second half of section 4.7

Lecture 6 Weight Tension Normal Force Static Friction Cutnell+Johnson: 4.8-4.12, second half of section 4.7 In this lecture, I m going to discuss four different kinds of forces: weight, tension, the normal

### Applied Multivariate Analysis

Neil H. Timm Applied Multivariate Analysis With 42 Figures Springer Contents Preface Acknowledgments List of Tables List of Figures vii ix xix xxiii 1 Introduction 1 1.1 Overview 1 1.2 Multivariate Models

### Component Ordering in Independent Component Analysis Based on Data Power

Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals

### Mechanics lecture 7 Moment of a force, torque, equilibrium of a body

G.1 EE1.el3 (EEE1023): Electronics III Mechanics lecture 7 Moment of a force, torque, equilibrium of a body Dr Philip Jackson http://www.ee.surrey.ac.uk/teaching/courses/ee1.el3/ G.2 Moments, torque and

### discuss how to describe points, lines and planes in 3 space.

Chapter 2 3 Space: lines and planes In this chapter we discuss how to describe points, lines and planes in 3 space. introduce the language of vectors. discuss various matters concerning the relative position

### MAT188H1S Lec0101 Burbulla

Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u

### ( ) which must be a vector

MATH 37 Linear Transformations from Rn to Rm Dr. Neal, WKU Let T : R n R m be a function which maps vectors from R n to R m. Then T is called a linear transformation if the following two properties are

### Section 1.1. Introduction to R n

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

### 3. Let A and B be two n n orthogonal matrices. Then prove that AB and BA are both orthogonal matrices. Prove a similar result for unitary matrices.

Exercise 1 1. Let A be an n n orthogonal matrix. Then prove that (a) the rows of A form an orthonormal basis of R n. (b) the columns of A form an orthonormal basis of R n. (c) for any two vectors x,y R

### 3. Regression & Exponential Smoothing

3. Regression & Exponential Smoothing 3.1 Forecasting a Single Time Series Two main approaches are traditionally used to model a single time series z 1, z 2,..., z n 1. Models the observation z t as a

### Content. Chapter 4 Functions 61 4.1 Basic concepts on real functions 62. Credits 11

Content Credits 11 Chapter 1 Arithmetic Refresher 13 1.1 Algebra 14 Real Numbers 14 Real Polynomials 19 1.2 Equations in one variable 21 Linear Equations 21 Quadratic Equations 22 1.3 Exercises 28 Chapter

### Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.

This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Solutions for Review Problems

olutions for Review Problems 1. Let be the triangle with vertices A (,, ), B (4,, 1) and C (,, 1). (a) Find the cosine of the angle BAC at vertex A. (b) Find the area of the triangle ABC. (c) Find a vector

### Partial Least Squares (PLS) Regression.

Partial Least Squares (PLS) Regression. Hervé Abdi 1 The University of Texas at Dallas Introduction Pls regression is a recent technique that generalizes and combines features from principal component

### Multivariate Normal Distribution

Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### Chapter 6: Multivariate Cointegration Analysis

Chapter 6: Multivariate Cointegration Analysis 1 Contents: Lehrstuhl für Department Empirische of Wirtschaftsforschung Empirical Research and und Econometrics Ökonometrie VI. Multivariate Cointegration

### (Quasi-)Newton methods

(Quasi-)Newton methods 1 Introduction 1.1 Newton method Newton method is a method to find the zeros of a differentiable non-linear function g, x such that g(x) = 0, where g : R n R n. Given a starting

### PEST - Beyond Basic Model Calibration. Presented by Jon Traum

PEST - Beyond Basic Model Calibration Presented by Jon Traum Purpose of Presentation Present advance techniques available in PEST for model calibration High level overview Inspire more people to use PEST!

### CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES. From Exploratory Factor Analysis Ledyard R Tucker and Robert C.

CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES From Exploratory Factor Analysis Ledyard R Tucker and Robert C MacCallum 1997 180 CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES In

### Vector Spaces; the Space R n

Vector Spaces; the Space R n Vector Spaces A vector space (over the real numbers) is a set V of mathematical entities, called vectors, U, V, W, etc, in which an addition operation + is defined and in which

### = 2 + 1 2 2 = 3 4, Now assume that P (k) is true for some fixed k 2. This means that

Instructions. Answer each of the questions on your own paper, and be sure to show your work so that partial credit can be adequately assessed. Credit will not be given for answers (even correct ones) without

### Integer Factorization using the Quadratic Sieve

Integer Factorization using the Quadratic Sieve Chad Seibert* Division of Science and Mathematics University of Minnesota, Morris Morris, MN 56567 seib0060@morris.umn.edu March 16, 2011 Abstract We give

### Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10

Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 10 Boundary Value Problems for Ordinary Differential Equations Copyright c 2001. Reproduction

### Dynamic data processing

Dynamic data processing recursive least-squares P.J.G. Teunissen Series on Mathematical Geodesy and Positioning Dynamic data processing recursive least-squares Dynamic data processing recursive least-squares

### Math 312 Homework 1 Solutions

Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

### Profile analysis is the multivariate equivalent of repeated measures or mixed ANOVA. Profile analysis is most commonly used in two cases:

Profile Analysis Introduction Profile analysis is the multivariate equivalent of repeated measures or mixed ANOVA. Profile analysis is most commonly used in two cases: ) Comparing the same dependent variables

### Section 1.4. Lines, Planes, and Hyperplanes. The Calculus of Functions of Several Variables

The Calculus of Functions of Several Variables Section 1.4 Lines, Planes, Hyperplanes In this section we will add to our basic geometric understing of R n by studying lines planes. If we do this carefully,

### Math 241, Exam 1 Information.

Math 241, Exam 1 Information. 9/24/12, LC 310, 11:15-12:05. Exam 1 will be based on: Sections 12.1-12.5, 14.1-14.3. The corresponding assigned homework problems (see http://www.math.sc.edu/ boylan/sccourses/241fa12/241.html)

### http://www.jstor.org This content downloaded on Tue, 19 Feb 2013 17:28:43 PM All use subject to JSTOR Terms and Conditions

A Significance Test for Time Series Analysis Author(s): W. Allen Wallis and Geoffrey H. Moore Reviewed work(s): Source: Journal of the American Statistical Association, Vol. 36, No. 215 (Sep., 1941), pp.

### Factor Analysis. Principal components factor analysis. Use of extracted factors in multivariate dependency models

Factor Analysis Principal components factor analysis Use of extracted factors in multivariate dependency models 2 KEY CONCEPTS ***** Factor Analysis Interdependency technique Assumptions of factor analysis

### ON THE DEGREES OF FREEDOM OF SIGNALS ON GRAPHS. Mikhail Tsitsvero and Sergio Barbarossa

ON THE DEGREES OF FREEDOM OF SIGNALS ON GRAPHS Mikhail Tsitsvero and Sergio Barbarossa Sapienza Univ. of Rome, DIET Dept., Via Eudossiana 18, 00184 Rome, Italy E-mail: tsitsvero@gmail.com, sergio.barbarossa@uniroma1.it

### constraint. Let us penalize ourselves for making the constraint too big. We end up with a

Chapter 4 Constrained Optimization 4.1 Equality Constraints (Lagrangians) Suppose we have a problem: Maximize 5, (x 1, 2) 2, 2(x 2, 1) 2 subject to x 1 +4x 2 =3 If we ignore the constraint, we get the

### Background: State Estimation

State Estimation Cyber Security of the Smart Grid Dr. Deepa Kundur Background: State Estimation University of Toronto Dr. Deepa Kundur (University of Toronto) Cyber Security of the Smart Grid 1 / 81 Dr.

### Machine Learning in Statistical Arbitrage

Machine Learning in Statistical Arbitrage Xing Fu, Avinash Patra December 11, 2009 Abstract We apply machine learning methods to obtain an index arbitrage strategy. In particular, we employ linear regression

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

### Computational Optical Imaging - Optique Numerique. -- Deconvolution --

Computational Optical Imaging - Optique Numerique -- Deconvolution -- Winter 2014 Ivo Ihrke Deconvolution Ivo Ihrke Outline Deconvolution Theory example 1D deconvolution Fourier method Algebraic method