F Matrix Calculus F 1

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "F Matrix Calculus F 1"

Transcription

1 F Matrix Calculus F 1

2 Appendix F: MATRIX CALCULUS TABLE OF CONTENTS Page F1 Introduction F 3 F2 The Derivatives of Vector Functions F 3 F21 Derivative of Vector with Respect to Vector F 3 F22 Derivative of a Scalar with Respect to Vector F 3 F23 Derivative of Vector with Respect to Scalar F 3 F24 Jacobian of a Variable Transformation F 4 F3 The Chain Rule for Vector Functions F 5 F4 The Derivative of Scalar Functions of a Matrix F 6 F41 Functions of a Matrix Determinant F 7 F5 The Matrix Differential F 8 F 2

3 F2 THE DERIVATIVES OF VECTOR FUNCTIONS F1 Introduction In this Appendix we collect some useful formulas of matrix calculus that often appear in finite element derivations F2 The Derivatives of Vector Functions Let x and y be vectors of orders n and m respectively: x 1 x 2 x x n, y y 1 y 2 y m, (F1) where each component y i may be a function of all the x j, a fact represented by saying that y is a function of x,or y y(x) (F2) If n 1, x reduces to a scalar, which we call x Ifm 1, y reduces to a scalar, which we call y Various applications are studied in the following subsections F21 Derivative of Vector with Respect to Vector The derivative of the vector y with respect to vector x is the n m matrix m 1 1 def m n m n n (F3) F22 Derivative of a Scalar with Respect to Vector If y is a scalar, def 1 2 n (F4) F23 Derivative of Vector with Respect to Scalar If x is a scalar, def [ 1 2 m ] (F5) F 3

4 Appendix F: MATRIX CALCULUS Remark F1 Many authors, notably in statistics and economics, define the derivatives as the transposes of those given above 1 This has the advantage of better agreement of matrix products with composition schemes such as the chain rule Evidently the notation is not yet stable Example F1 Given and y [ y1 ], x y 2 y 1 x 2 1 x 2 y 2 x x 2 [ x1 x 2 x 3 ] (F6) (F7) the partial derivative matrix / is computed as follows: [ 2x1 0 ] x 3 (F8) F24 Jacobian of a Variable Transformation In multivariate analysis, if x and y are of the same order, the determinant of the square matrix /, that is J (F9) is called the Jacobian of the transformation determined by y y(x) The inverse determinant is J 1 (F10) Example F2 The transformation from spherical to Cartesian coordinates is defined by x r sin θ cos ψ, y r sin θ sin ψ, z r cos θ (F11) where r > 0, 0 < θ < π and 0 ψ < 2π To obtain the Jacobian of the transformation, let Then x x 1, y x 2, z x 3 (F12) r y 1, θ y 2, ψ y 3 J sin y 2 cos y 3 sin y 2 sin y 3 cos y 2 y 1 cos y 2 cos y 3 y 1 cos y 2 sin y 3 y 1 sin y 2 y 1 sin y 2 sin y 3 y 1 sin y 2 cos y 3 0 (F13) y 2 1 sin y 2 r 2 sin θ The foregoing definitions can be used to obtain derivatives to many frequently used expressions, including quadratic and bilinear forms 1 One author puts it this way: When one does matrix calculus, one quickly finds that there are two kinds of people in this world: those who think the gradient is a row vector, and those who think it is a column vector F 4

5 F3 THE CHAIN RULE FOR VECTOR FUNCTIONS Example F3 Consider the quadratic form y x T Ax where A is a square matrix of order n Using the definition (D3) one obtains (F14) Ax + AT x (F15) and if A is symmetric, 2Ax (F16) We can of course continue the differentiation process: 2 y ( ) A + A T, (F17) 2 and if A is symmetric, 2 y 2 2A (F18) The following table collects several useful vector derivative formulas y Ax x T A x T x x T Ax A T A 2x Ax + A T x F3 The Chain Rule for Vector Functions Let x 1 y 1 z 1 x x 2, y y 2 and z z 2 (F19) x n y r z m where z is a function of y, which is in turn a function of x Using the definition (D2), we can write z 1 z 1 z n ( ) z T z 2 z 2 z n (F20) 2 Each entry of this matrix may be expanded as z i j r q1 1 z i q q j F 5 n { i 1, 2,,m j 1, 2,,n (F21)

6 Appendix F: MATRIX CALCULUS Then ( ) z T z1 y q q 1 z2 q q 1 zm q z 1 z 1 1 q 1 2 z 2 1 z ( z 2 ) T ( On transposing both sides, we finally obtain z1 q q 2 z2 q q 2 zm q q z2 q q n z2 q q n zm q q n 2 z r n z 2 r n r r r 1 2 r n ) T ( ) z T (F22) z z, (F23) which is the chain rule for vectors If all vectors reduce to scalars, z z z, (F24) which is the conventional chain rule of calculus Note, however, that when we are dealing with vectors, the chain of matrices builds toward the left For example, if w is a function of z, which is a function of y, which is a function of x, w z w z (F25) On the other hand, in the ordinary chain rule one can indistictly build the product to the right or to the left because scalar multiplication is commutative F4 The Derivative of Scalar Functions of a Matrix Let X (x ij ) be a matrix of order (m n) and let y f (X), (F26) be a scalar function of X The derivative of y with respect to X, denoted by X, (F27) F 6

7 is defined as the following matrix of order (m n): G X m2 F4 THE DERIVATIVE OF SCALAR FUNCTIONS OF A MATRIX m1 1n 2n mn [ ] ij i, j E ij ij, (F28) where E ij denotes the elementary matrix* of order (m n) This matrix G is also known as a gradient matrix Example F4 Find the gradient matrix if y is the trace of a square matrix X of order n, that is n y tr(x) x ii i1 (F29) Obviously all non-diagonal partials vanish whereas the diagonal partials equal one, thus where I denotes the identity matrix of order n G X I, (F30) F41 Functions of a Matrix Determinant An important family of derivatives with respect to a matrix involves functions of the determinant of a matrix, for example y X or y AX Suppose that we have a matrix Y [y ij ] whose components are functions of a matrix X [x rs ], that is y ij f ij (x rs ), and set out to build the matrix X (F31) Using the chain rule we can write rs i j Y ij ij ij rs (F32) But Y j y ij Y ij, (F33) where Y ij is the cofactor of the element y ij in Y Since the cofactors Y i1, Y i2, are independent of the element y ij,wehave ij Y ij (F34) It follows that rs i j Y ij ij rs (F35) * The elementary matrix E ij of order m n has all zero entries except for the (i, j) entry, which is one F 7

8 Appendix F: MATRIX CALCULUS There is an alternative form of this result which is ocassionally useful Define a ij Y ij, A [a ij ], b ij ij rs, B [b ij ] (F36) Then it can be shown that rs tr(ab T ) tr(b T A) (F37) Example F5 IfX is a nonsingular square matrix and Z X X 1 its cofactor matrix, If X is also symmetric, G X X ZT G X X 2ZT diag(z T ) (F38) (F39) F5 The Matrix Differential For a scalar function f (x), where x is an n-vector, the ordinary differential of multivariate calculus is defined as n f df dx i (F40) i1 i In harmony with this formula, we define the differential of an m n matrix X [x ij ]tobe dx def dx 11 dx 12 dx 1n dx 21 dx 22 dx 2n dx m1 dx m2 dx mn This definition complies with the multiplicative and associative rules (F41) d(αx) α dx, d(x + Y) dx + dy (F42) If X and Y are product-conforming matrices, it can be verified that the differential of their product is d(xy) (dx)y + X(dY) (F43) which is an extension of the well known rule d(xy) ydx+ xdyfor scalar functions Example F6 If X [x ij ] is a square nonsingular matrix of order n, and denote Z X X 1 Find the differential of the determinant of X: X d X dx ij X ij dx ij tr( X X 1 ) T dx) tr(z T dx), (F44) ij where X ij denotes the cofactor of x ij in X i, j i, j F 8

9 F5 THE MATRIX DIFFERENTIAL Example F7 With the same assumptions as above, find d(x 1 ) The quickest derivation follows by differentiating both sides of the identity X 1 X I: from which If X reduces to the scalar x we have d(x 1 )X + X 1 dx 0, d(x 1 ) X 1 dxx 1 d ( ) 1 dx x x 2 (F45) (F46) (F47) F 9

1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES 1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

More information

A QUICK GUIDE TO THE FORMULAS OF MULTIVARIABLE CALCULUS

A QUICK GUIDE TO THE FORMULAS OF MULTIVARIABLE CALCULUS A QUIK GUIDE TO THE FOMULAS OF MULTIVAIABLE ALULUS ontents 1. Analytic Geometry 2 1.1. Definition of a Vector 2 1.2. Scalar Product 2 1.3. Properties of the Scalar Product 2 1.4. Length and Unit Vectors

More information

Math 312 Homework 1 Solutions

Math 312 Homework 1 Solutions Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

More information

9 MATRICES AND TRANSFORMATIONS

9 MATRICES AND TRANSFORMATIONS 9 MATRICES AND TRANSFORMATIONS Chapter 9 Matrices and Transformations Objectives After studying this chapter you should be able to handle matrix (and vector) algebra with confidence, and understand the

More information

Math 241, Exam 1 Information.

Math 241, Exam 1 Information. Math 241, Exam 1 Information. 9/24/12, LC 310, 11:15-12:05. Exam 1 will be based on: Sections 12.1-12.5, 14.1-14.3. The corresponding assigned homework problems (see http://www.math.sc.edu/ boylan/sccourses/241fa12/241.html)

More information

MAT188H1S Lec0101 Burbulla

MAT188H1S Lec0101 Burbulla Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u

More information

6. Cholesky factorization

6. Cholesky factorization 6. Cholesky factorization EE103 (Fall 2011-12) triangular matrices forward and backward substitution the Cholesky factorization solving Ax = b with A positive definite inverse of a positive definite matrix

More information

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,

More information

Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components

Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they

More information

Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

More information

Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

More information

Vector and Tensor Algebra (including Column and Matrix Notation)

Vector and Tensor Algebra (including Column and Matrix Notation) Vector and Tensor Algebra (including Column and Matrix Notation) 2 Vectors and tensors In mechanics and other fields of physics, quantities are represented by vectors and tensors. Essential manipulations

More information

Lecture notes on linear algebra

Lecture notes on linear algebra Lecture notes on linear algebra David Lerner Department of Mathematics University of Kansas These are notes of a course given in Fall, 2007 and 2008 to the Honors sections of our elementary linear algebra

More information

Recall that two vectors in are perpendicular or orthogonal provided that their dot

Recall that two vectors in are perpendicular or orthogonal provided that their dot Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

More information

1.3. DOT PRODUCT 19. 6. If θ is the angle (between 0 and π) between two non-zero vectors u and v,

1.3. DOT PRODUCT 19. 6. If θ is the angle (between 0 and π) between two non-zero vectors u and v, 1.3. DOT PRODUCT 19 1.3 Dot Product 1.3.1 Definitions and Properties The dot product is the first way to multiply two vectors. The definition we will give below may appear arbitrary. But it is not. It

More information

26. Determinants I. 1. Prehistory

26. Determinants I. 1. Prehistory 26. Determinants I 26.1 Prehistory 26.2 Definitions 26.3 Uniqueness and other properties 26.4 Existence Both as a careful review of a more pedestrian viewpoint, and as a transition to a coordinate-independent

More information

Dot product and vector projections (Sect. 12.3) There are two main ways to introduce the dot product

Dot product and vector projections (Sect. 12.3) There are two main ways to introduce the dot product Dot product and vector projections (Sect. 12.3) Two definitions for the dot product. Geometric definition of dot product. Orthogonal vectors. Dot product and orthogonal projections. Properties of the dot

More information

BILINEAR FORMS KEITH CONRAD

BILINEAR FORMS KEITH CONRAD BILINEAR FORMS KEITH CONRAD The geometry of R n is controlled algebraically by the dot product. We will abstract the dot product on R n to a bilinear form on a vector space and study algebraic and geometric

More information

Similar matrices and Jordan form

Similar matrices and Jordan form Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

More information

Electrical Engineering 103 Applied Numerical Computing

Electrical Engineering 103 Applied Numerical Computing UCLA Fall Quarter 2011-12 Electrical Engineering 103 Applied Numerical Computing Professor L Vandenberghe Notes written in collaboration with S Boyd (Stanford Univ) Contents I Matrix theory 1 1 Vectors

More information

LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008 This chapter is available free to all individuals, on understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

More information

The Characteristic Polynomial

The Characteristic Polynomial Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

More information

Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors

Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors 1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number

More information

LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN LINEAR ALGEBRA W W L CHEN c W W L Chen, 1982, 2008. This chapter originates from material used by author at Imperial College, University of London, between 1981 and 1990. It is available free to all individuals,

More information

α = u v. In other words, Orthogonal Projection

α = u v. In other words, Orthogonal Projection Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

More information

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

More information

Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

More information

Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University

Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University Linear Algebra Done Wrong Sergei Treil Department of Mathematics, Brown University Copyright c Sergei Treil, 2004, 2009, 2011, 2014 Preface The title of the book sounds a bit mysterious. Why should anyone

More information

Chapter 7. Matrices. Definition. An m n matrix is an array of numbers set out in m rows and n columns. Examples. ( 1 1 5 2 0 6

Chapter 7. Matrices. Definition. An m n matrix is an array of numbers set out in m rows and n columns. Examples. ( 1 1 5 2 0 6 Chapter 7 Matrices Definition An m n matrix is an array of numbers set out in m rows and n columns Examples (i ( 1 1 5 2 0 6 has 2 rows and 3 columns and so it is a 2 3 matrix (ii 1 0 7 1 2 3 3 1 is a

More information

Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

More information

1 Local Brouwer degree

1 Local Brouwer degree 1 Local Brouwer degree Let D R n be an open set and f : S R n be continuous, D S and c R n. Suppose that the set f 1 (c) D is compact. (1) Then the local Brouwer degree of f at c in the set D is defined.

More information

LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA. September 23, 2010 LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

More information

The Analysis of Data. Volume 1. Probability. Guy Lebanon

The Analysis of Data. Volume 1. Probability. Guy Lebanon The Analysis of Data Volume 1 Probability Guy Lebanon First Edition 2012 Probability. The Analysis of Data, Volume 1. First Edition, First Printing, 2013 http://theanalysisofdata.com Copyright 2013 by

More information

Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University

Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University Linear Algebra Done Wrong Sergei Treil Department of Mathematics, Brown University Copyright c Sergei Treil, 2004, 2009, 2011, 2014 Preface The title of the book sounds a bit mysterious. Why should anyone

More information

Review of Vector Analysis in Cartesian Coordinates

Review of Vector Analysis in Cartesian Coordinates R. evicky, CBE 6333 Review of Vector Analysis in Cartesian Coordinates Scalar: A quantity that has magnitude, but no direction. Examples are mass, temperature, pressure, time, distance, and real numbers.

More information

Section 9.5: Equations of Lines and Planes

Section 9.5: Equations of Lines and Planes Lines in 3D Space Section 9.5: Equations of Lines and Planes Practice HW from Stewart Textbook (not to hand in) p. 673 # 3-5 odd, 2-37 odd, 4, 47 Consider the line L through the point P = ( x, y, ) that

More information

1 Sets and Set Notation.

1 Sets and Set Notation. LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

More information

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10 Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 10 Boundary Value Problems for Ordinary Differential Equations Copyright c 2001. Reproduction

More information

CONTROLLABILITY. Chapter 2. 2.1 Reachable Set and Controllability. Suppose we have a linear system described by the state equation

CONTROLLABILITY. Chapter 2. 2.1 Reachable Set and Controllability. Suppose we have a linear system described by the state equation Chapter 2 CONTROLLABILITY 2 Reachable Set and Controllability Suppose we have a linear system described by the state equation ẋ Ax + Bu (2) x() x Consider the following problem For a given vector x in

More information

Recall the basic property of the transpose (for any A): v A t Aw = v w, v, w R n.

Recall the basic property of the transpose (for any A): v A t Aw = v w, v, w R n. ORTHOGONAL MATRICES Informally, an orthogonal n n matrix is the n-dimensional analogue of the rotation matrices R θ in R 2. When does a linear transformation of R 3 (or R n ) deserve to be called a rotation?

More information

Introduction to Partial Differential Equations. John Douglas Moore

Introduction to Partial Differential Equations. John Douglas Moore Introduction to Partial Differential Equations John Douglas Moore May 2, 2003 Preface Partial differential equations are often used to construct models of the most basic theories underlying physics and

More information

7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix

7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix 7. LU factorization EE103 (Fall 2011-12) factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization algorithm effect of rounding error sparse

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

More information

RESULTANT AND DISCRIMINANT OF POLYNOMIALS

RESULTANT AND DISCRIMINANT OF POLYNOMIALS RESULTANT AND DISCRIMINANT OF POLYNOMIALS SVANTE JANSON Abstract. This is a collection of classical results about resultants and discriminants for polynomials, compiled mainly for my own use. All results

More information

CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

More information

Matrices and Polynomials

Matrices and Polynomials APPENDIX 9 Matrices and Polynomials he Multiplication of Polynomials Let α(z) =α 0 +α 1 z+α 2 z 2 + α p z p and y(z) =y 0 +y 1 z+y 2 z 2 + y n z n be two polynomials of degrees p and n respectively. hen,

More information

Solutions for Review Problems

Solutions for Review Problems olutions for Review Problems 1. Let be the triangle with vertices A (,, ), B (4,, 1) and C (,, 1). (a) Find the cosine of the angle BAC at vertex A. (b) Find the area of the triangle ABC. (c) Find a vector

More information

Vector Spaces. Chapter 2. 2.1 R 2 through R n

Vector Spaces. Chapter 2. 2.1 R 2 through R n Chapter 2 Vector Spaces One of my favorite dictionaries (the one from Oxford) defines a vector as A quantity having direction as well as magnitude, denoted by a line drawn from its original to its final

More information

x1 x 2 x 3 y 1 y 2 y 3 x 1 y 2 x 2 y 1 0.

x1 x 2 x 3 y 1 y 2 y 3 x 1 y 2 x 2 y 1 0. Cross product 1 Chapter 7 Cross product We are getting ready to study integration in several variables. Until now we have been doing only differential calculus. One outcome of this study will be our ability

More information

LECTURE 1: DIFFERENTIAL FORMS. 1. 1-forms on R n

LECTURE 1: DIFFERENTIAL FORMS. 1. 1-forms on R n LECTURE 1: DIFFERENTIAL FORMS 1. 1-forms on R n In calculus, you may have seen the differential or exterior derivative df of a function f(x, y, z) defined to be df = f f f dx + dy + x y z dz. The expression

More information

Modélisation et résolutions numérique et symbolique

Modélisation et résolutions numérique et symbolique Modélisation et résolutions numérique et symbolique via les logiciels Maple et Matlab Jeremy Berthomieu Mohab Safey El Din Stef Graillat Mohab.Safey@lip6.fr Outline Previous course: partial review of what

More information

Factorization Theorems

Factorization Theorems Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

More information

GCE Mathematics (6360) Further Pure unit 4 (MFP4) Textbook

GCE Mathematics (6360) Further Pure unit 4 (MFP4) Textbook Version 36 klm GCE Mathematics (636) Further Pure unit 4 (MFP4) Textbook The Assessment and Qualifications Alliance (AQA) is a company limited by guarantee registered in England and Wales 364473 and a

More information

Discrete Convolution and the Discrete Fourier Transform

Discrete Convolution and the Discrete Fourier Transform Discrete Convolution and the Discrete Fourier Transform Discrete Convolution First of all we need to introduce what we might call the wraparound convention Because the complex numbers w j e i πj N have

More information

STATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. Clarificationof zonationprocedure described onpp. 238-239

STATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. Clarificationof zonationprocedure described onpp. 238-239 STATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. by John C. Davis Clarificationof zonationprocedure described onpp. 38-39 Because the notation used in this section (Eqs. 4.8 through 4.84) is inconsistent

More information

Chapter 9 Unitary Groups and SU(N)

Chapter 9 Unitary Groups and SU(N) Chapter 9 Unitary Groups and SU(N) The irreducible representations of SO(3) are appropriate for describing the degeneracies of states of quantum mechanical systems which have rotational symmetry in three

More information

CS 295 (UVM) The LMS Algorithm Fall 2013 1 / 23

CS 295 (UVM) The LMS Algorithm Fall 2013 1 / 23 The LMS Algorithm The LMS Objective Function. Global solution. Pseudoinverse of a matrix. Optimization (learning) by gradient descent. LMS or Widrow-Hoff Algorithms. Convergence of the Batch LMS Rule.

More information

ISOMETRIES OF R n KEITH CONRAD

ISOMETRIES OF R n KEITH CONRAD ISOMETRIES OF R n KEITH CONRAD 1. Introduction An isometry of R n is a function h: R n R n that preserves the distance between vectors: h(v) h(w) = v w for all v and w in R n, where (x 1,..., x n ) = x

More information

The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression

The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression The SVD is the most generally applicable of the orthogonal-diagonal-orthogonal type matrix decompositions Every

More information

3 Orthogonal Vectors and Matrices

3 Orthogonal Vectors and Matrices 3 Orthogonal Vectors and Matrices The linear algebra portion of this course focuses on three matrix factorizations: QR factorization, singular valued decomposition (SVD), and LU factorization The first

More information

Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i. Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

More information

Section 5.3. Section 5.3. u m ] l jj. = l jj u j + + l mj u m. v j = [ u 1 u j. l mj

Section 5.3. Section 5.3. u m ] l jj. = l jj u j + + l mj u m. v j = [ u 1 u j. l mj Section 5. l j v j = [ u u j u m ] l jj = l jj u j + + l mj u m. l mj Section 5. 5.. Not orthogonal, the column vectors fail to be perpendicular to each other. 5..2 his matrix is orthogonal. Check that

More information

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan

More information

Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University

Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision

More information

Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems. Matrix Factorization Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

More information

4.1. COMPLEX NUMBERS

4.1. COMPLEX NUMBERS 4.1. COMPLEX NUMBERS What You Should Learn Use the imaginary unit i to write complex numbers. Add, subtract, and multiply complex numbers. Use complex conjugates to write the quotient of two complex numbers

More information

CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES. From Exploratory Factor Analysis Ledyard R Tucker and Robert C.

CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES. From Exploratory Factor Analysis Ledyard R Tucker and Robert C. CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES From Exploratory Factor Analysis Ledyard R Tucker and Robert C MacCallum 1997 180 CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES In

More information

Cryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur

Cryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 12 Block Cipher Standards

More information

A SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS

A SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS A SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS Eusebio GÓMEZ, Miguel A. GÓMEZ-VILLEGAS and J. Miguel MARÍN Abstract In this paper it is taken up a revision and characterization of the class of

More information

MATH1231 Algebra, 2015 Chapter 7: Linear maps

MATH1231 Algebra, 2015 Chapter 7: Linear maps MATH1231 Algebra, 2015 Chapter 7: Linear maps A/Prof. Daniel Chan School of Mathematics and Statistics University of New South Wales danielc@unsw.edu.au Daniel Chan (UNSW) MATH1231 Algebra 1 / 43 Chapter

More information

( ) which must be a vector

( ) which must be a vector MATH 37 Linear Transformations from Rn to Rm Dr. Neal, WKU Let T : R n R m be a function which maps vectors from R n to R m. Then T is called a linear transformation if the following two properties are

More information

The Convolution Operation

The Convolution Operation The Convolution Operation Convolution is a very natural mathematical operation which occurs in both discrete and continuous modes of various kinds. We often encounter it in the course of doing other operations

More information

Exam 1 Sample Question SOLUTIONS. y = 2x

Exam 1 Sample Question SOLUTIONS. y = 2x Exam Sample Question SOLUTIONS. Eliminate the parameter to find a Cartesian equation for the curve: x e t, y e t. SOLUTION: You might look at the coordinates and notice that If you don t see it, we can

More information

ME 115(b): Solution to Homework #1

ME 115(b): Solution to Homework #1 ME 115(b): Solution to Homework #1 Solution to Problem #1: To construct the hybrid Jacobian for a manipulator, you could either construct the body Jacobian, JST b, and then use the body-to-hybrid velocity

More information

Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

More information

Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

More information

Continuous Groups, Lie Groups, and Lie Algebras

Continuous Groups, Lie Groups, and Lie Algebras Chapter 7 Continuous Groups, Lie Groups, and Lie Algebras Zeno was concerned with three problems... These are the problem of the infinitesimal, the infinite, and continuity... Bertrand Russell The groups

More information

Solutions to old Exam 1 problems

Solutions to old Exam 1 problems Solutions to old Exam 1 problems Hi students! I am putting this old version of my review for the first midterm review, place and time to be announced. Check for updates on the web site as to which sections

More information

Orthogonal Bases and the QR Algorithm

Orthogonal Bases and the QR Algorithm Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries

More information

Understanding Basic Calculus

Understanding Basic Calculus Understanding Basic Calculus S.K. Chung Dedicated to all the people who have helped me in my life. i Preface This book is a revised and expanded version of the lecture notes for Basic Calculus and other

More information

Jim Lambers MAT 169 Fall Semester 2009-10 Lecture 25 Notes

Jim Lambers MAT 169 Fall Semester 2009-10 Lecture 25 Notes Jim Lambers MAT 169 Fall Semester 009-10 Lecture 5 Notes These notes correspond to Section 10.5 in the text. Equations of Lines A line can be viewed, conceptually, as the set of all points in space that

More information

Factoring Algorithms

Factoring Algorithms Factoring Algorithms The p 1 Method and Quadratic Sieve November 17, 2008 () Factoring Algorithms November 17, 2008 1 / 12 Fermat s factoring method Fermat made the observation that if n has two factors

More information

Regression Analysis LECTURE 2. The Multiple Regression Model in Matrices Consider the regression equation. (1) y = β 0 + β 1 x 1 + + β k x k + ε,

Regression Analysis LECTURE 2. The Multiple Regression Model in Matrices Consider the regression equation. (1) y = β 0 + β 1 x 1 + + β k x k + ε, LECTURE 2 Regression Analysis The Multiple Regression Model in Matrices Consider the regression equation (1) y = β 0 + β 1 x 1 + + β k x k + ε, and imagine that T observations on the variables y, x 1,,x

More information

Solutions to Homework 5

Solutions to Homework 5 Solutions to Homework 5 1. Let z = f(x, y) be a twice continously differentiable function of x and y. Let x = r cos θ and y = r sin θ be the equations which transform polar coordinates into rectangular

More information

Linear Programming: Chapter 11 Game Theory

Linear Programming: Chapter 11 Game Theory Linear Programming: Chapter 11 Game Theory Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Rock-Paper-Scissors

More information

Network Traffic Modelling

Network Traffic Modelling University of York Dissertation submitted for the MSc in Mathematics with Modern Applications, Department of Mathematics, University of York, UK. August 009 Network Traffic Modelling Author: David Slade

More information

Dynamic Analysis. Mass Matrices and External Forces

Dynamic Analysis. Mass Matrices and External Forces 4 Dynamic Analysis. Mass Matrices and External Forces The formulation of the inertia and external forces appearing at any of the elements of a multibody system, in terms of the dependent coordinates that

More information

LINES AND PLANES IN R 3

LINES AND PLANES IN R 3 LINES AND PLANES IN R 3 In this handout we will summarize the properties of the dot product and cross product and use them to present arious descriptions of lines and planes in three dimensional space.

More information

Lecture 14: Section 3.3

Lecture 14: Section 3.3 Lecture 14: Section 3.3 Shuanglin Shao October 23, 2013 Definition. Two nonzero vectors u and v in R n are said to be orthogonal (or perpendicular) if u v = 0. We will also agree that the zero vector in

More information

Lecture 8 February 4

Lecture 8 February 4 ICS273A: Machine Learning Winter 2008 Lecture 8 February 4 Scribe: Carlos Agell (Student) Lecturer: Deva Ramanan 8.1 Neural Nets 8.1.1 Logistic Regression Recall the logistic function: g(x) = 1 1 + e θt

More information

Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

More information

Content. Chapter 4 Functions 61 4.1 Basic concepts on real functions 62. Credits 11

Content. Chapter 4 Functions 61 4.1 Basic concepts on real functions 62. Credits 11 Content Credits 11 Chapter 1 Arithmetic Refresher 13 1.1 Algebra 14 Real Numbers 14 Real Polynomials 19 1.2 Equations in one variable 21 Linear Equations 21 Quadratic Equations 22 1.3 Exercises 28 Chapter

More information

LINES AND PLANES CHRIS JOHNSON

LINES AND PLANES CHRIS JOHNSON LINES AND PLANES CHRIS JOHNSON Abstract. In this lecture we derive the equations for lines and planes living in 3-space, as well as define the angle between two non-parallel planes, and determine the distance

More information

Structure of the Root Spaces for Simple Lie Algebras

Structure of the Root Spaces for Simple Lie Algebras Structure of the Root Spaces for Simple Lie Algebras I. Introduction A Cartan subalgebra, H, of a Lie algebra, G, is a subalgebra, H G, such that a. H is nilpotent, i.e., there is some n such that (H)

More information

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

More information

3. Let A and B be two n n orthogonal matrices. Then prove that AB and BA are both orthogonal matrices. Prove a similar result for unitary matrices.

3. Let A and B be two n n orthogonal matrices. Then prove that AB and BA are both orthogonal matrices. Prove a similar result for unitary matrices. Exercise 1 1. Let A be an n n orthogonal matrix. Then prove that (a) the rows of A form an orthonormal basis of R n. (b) the columns of A form an orthonormal basis of R n. (c) for any two vectors x,y R

More information

Math 215 HW #6 Solutions

Math 215 HW #6 Solutions Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T

More information

Row Ideals and Fibers of Morphisms

Row Ideals and Fibers of Morphisms Michigan Math. J. 57 (2008) Row Ideals and Fibers of Morphisms David Eisenbud & Bernd Ulrich Affectionately dedicated to Mel Hochster, who has been an inspiration to us for many years, on the occasion

More information

SOME PROPERTIES OF SYMBOL ALGEBRAS OF DEGREE THREE

SOME PROPERTIES OF SYMBOL ALGEBRAS OF DEGREE THREE SOME PROPERTIES OF SYMBOL ALGEBRAS OF DEGREE THREE CRISTINA FLAUT and DIANA SAVIN Communicated by the former editorial board In this paper, we study some properties of the matrix representations of the

More information

State of Stress at Point

State of Stress at Point State of Stress at Point Einstein Notation The basic idea of Einstein notation is that a covector and a vector can form a scalar: This is typically written as an explicit sum: According to this convention,

More information