# F Matrix Calculus F 1

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 F Matrix Calculus F 1

2 Appendix F: MATRIX CALCULUS TABLE OF CONTENTS Page F1 Introduction F 3 F2 The Derivatives of Vector Functions F 3 F21 Derivative of Vector with Respect to Vector F 3 F22 Derivative of a Scalar with Respect to Vector F 3 F23 Derivative of Vector with Respect to Scalar F 3 F24 Jacobian of a Variable Transformation F 4 F3 The Chain Rule for Vector Functions F 5 F4 The Derivative of Scalar Functions of a Matrix F 6 F41 Functions of a Matrix Determinant F 7 F5 The Matrix Differential F 8 F 2

3 F2 THE DERIVATIVES OF VECTOR FUNCTIONS F1 Introduction In this Appendix we collect some useful formulas of matrix calculus that often appear in finite element derivations F2 The Derivatives of Vector Functions Let x and y be vectors of orders n and m respectively: x 1 x 2 x x n, y y 1 y 2 y m, (F1) where each component y i may be a function of all the x j, a fact represented by saying that y is a function of x,or y y(x) (F2) If n 1, x reduces to a scalar, which we call x Ifm 1, y reduces to a scalar, which we call y Various applications are studied in the following subsections F21 Derivative of Vector with Respect to Vector The derivative of the vector y with respect to vector x is the n m matrix m 1 1 def m n m n n (F3) F22 Derivative of a Scalar with Respect to Vector If y is a scalar, def 1 2 n (F4) F23 Derivative of Vector with Respect to Scalar If x is a scalar, def [ 1 2 m ] (F5) F 3

4 Appendix F: MATRIX CALCULUS Remark F1 Many authors, notably in statistics and economics, define the derivatives as the transposes of those given above 1 This has the advantage of better agreement of matrix products with composition schemes such as the chain rule Evidently the notation is not yet stable Example F1 Given and y [ y1 ], x y 2 y 1 x 2 1 x 2 y 2 x x 2 [ x1 x 2 x 3 ] (F6) (F7) the partial derivative matrix / is computed as follows: [ 2x1 0 ] x 3 (F8) F24 Jacobian of a Variable Transformation In multivariate analysis, if x and y are of the same order, the determinant of the square matrix /, that is J (F9) is called the Jacobian of the transformation determined by y y(x) The inverse determinant is J 1 (F10) Example F2 The transformation from spherical to Cartesian coordinates is defined by x r sin θ cos ψ, y r sin θ sin ψ, z r cos θ (F11) where r > 0, 0 < θ < π and 0 ψ < 2π To obtain the Jacobian of the transformation, let Then x x 1, y x 2, z x 3 (F12) r y 1, θ y 2, ψ y 3 J sin y 2 cos y 3 sin y 2 sin y 3 cos y 2 y 1 cos y 2 cos y 3 y 1 cos y 2 sin y 3 y 1 sin y 2 y 1 sin y 2 sin y 3 y 1 sin y 2 cos y 3 0 (F13) y 2 1 sin y 2 r 2 sin θ The foregoing definitions can be used to obtain derivatives to many frequently used expressions, including quadratic and bilinear forms 1 One author puts it this way: When one does matrix calculus, one quickly finds that there are two kinds of people in this world: those who think the gradient is a row vector, and those who think it is a column vector F 4

5 F3 THE CHAIN RULE FOR VECTOR FUNCTIONS Example F3 Consider the quadratic form y x T Ax where A is a square matrix of order n Using the definition (D3) one obtains (F14) Ax + AT x (F15) and if A is symmetric, 2Ax (F16) We can of course continue the differentiation process: 2 y ( ) A + A T, (F17) 2 and if A is symmetric, 2 y 2 2A (F18) The following table collects several useful vector derivative formulas y Ax x T A x T x x T Ax A T A 2x Ax + A T x F3 The Chain Rule for Vector Functions Let x 1 y 1 z 1 x x 2, y y 2 and z z 2 (F19) x n y r z m where z is a function of y, which is in turn a function of x Using the definition (D2), we can write z 1 z 1 z n ( ) z T z 2 z 2 z n (F20) 2 Each entry of this matrix may be expanded as z i j r q1 1 z i q q j F 5 n { i 1, 2,,m j 1, 2,,n (F21)

6 Appendix F: MATRIX CALCULUS Then ( ) z T z1 y q q 1 z2 q q 1 zm q z 1 z 1 1 q 1 2 z 2 1 z ( z 2 ) T ( On transposing both sides, we finally obtain z1 q q 2 z2 q q 2 zm q q z2 q q n z2 q q n zm q q n 2 z r n z 2 r n r r r 1 2 r n ) T ( ) z T (F22) z z, (F23) which is the chain rule for vectors If all vectors reduce to scalars, z z z, (F24) which is the conventional chain rule of calculus Note, however, that when we are dealing with vectors, the chain of matrices builds toward the left For example, if w is a function of z, which is a function of y, which is a function of x, w z w z (F25) On the other hand, in the ordinary chain rule one can indistictly build the product to the right or to the left because scalar multiplication is commutative F4 The Derivative of Scalar Functions of a Matrix Let X (x ij ) be a matrix of order (m n) and let y f (X), (F26) be a scalar function of X The derivative of y with respect to X, denoted by X, (F27) F 6

7 is defined as the following matrix of order (m n): G X m2 F4 THE DERIVATIVE OF SCALAR FUNCTIONS OF A MATRIX m1 1n 2n mn [ ] ij i, j E ij ij, (F28) where E ij denotes the elementary matrix* of order (m n) This matrix G is also known as a gradient matrix Example F4 Find the gradient matrix if y is the trace of a square matrix X of order n, that is n y tr(x) x ii i1 (F29) Obviously all non-diagonal partials vanish whereas the diagonal partials equal one, thus where I denotes the identity matrix of order n G X I, (F30) F41 Functions of a Matrix Determinant An important family of derivatives with respect to a matrix involves functions of the determinant of a matrix, for example y X or y AX Suppose that we have a matrix Y [y ij ] whose components are functions of a matrix X [x rs ], that is y ij f ij (x rs ), and set out to build the matrix X (F31) Using the chain rule we can write rs i j Y ij ij ij rs (F32) But Y j y ij Y ij, (F33) where Y ij is the cofactor of the element y ij in Y Since the cofactors Y i1, Y i2, are independent of the element y ij,wehave ij Y ij (F34) It follows that rs i j Y ij ij rs (F35) * The elementary matrix E ij of order m n has all zero entries except for the (i, j) entry, which is one F 7

8 Appendix F: MATRIX CALCULUS There is an alternative form of this result which is ocassionally useful Define a ij Y ij, A [a ij ], b ij ij rs, B [b ij ] (F36) Then it can be shown that rs tr(ab T ) tr(b T A) (F37) Example F5 IfX is a nonsingular square matrix and Z X X 1 its cofactor matrix, If X is also symmetric, G X X ZT G X X 2ZT diag(z T ) (F38) (F39) F5 The Matrix Differential For a scalar function f (x), where x is an n-vector, the ordinary differential of multivariate calculus is defined as n f df dx i (F40) i1 i In harmony with this formula, we define the differential of an m n matrix X [x ij ]tobe dx def dx 11 dx 12 dx 1n dx 21 dx 22 dx 2n dx m1 dx m2 dx mn This definition complies with the multiplicative and associative rules (F41) d(αx) α dx, d(x + Y) dx + dy (F42) If X and Y are product-conforming matrices, it can be verified that the differential of their product is d(xy) (dx)y + X(dY) (F43) which is an extension of the well known rule d(xy) ydx+ xdyfor scalar functions Example F6 If X [x ij ] is a square nonsingular matrix of order n, and denote Z X X 1 Find the differential of the determinant of X: X d X dx ij X ij dx ij tr( X X 1 ) T dx) tr(z T dx), (F44) ij where X ij denotes the cofactor of x ij in X i, j i, j F 8

9 F5 THE MATRIX DIFFERENTIAL Example F7 With the same assumptions as above, find d(x 1 ) The quickest derivation follows by differentiating both sides of the identity X 1 X I: from which If X reduces to the scalar x we have d(x 1 )X + X 1 dx 0, d(x 1 ) X 1 dxx 1 d ( ) 1 dx x x 2 (F45) (F46) (F47) F 9

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### A QUICK GUIDE TO THE FORMULAS OF MULTIVARIABLE CALCULUS

A QUIK GUIDE TO THE FOMULAS OF MULTIVAIABLE ALULUS ontents 1. Analytic Geometry 2 1.1. Definition of a Vector 2 1.2. Scalar Product 2 1.3. Properties of the Scalar Product 2 1.4. Length and Unit Vectors

### Math 312 Homework 1 Solutions

Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

### 9 MATRICES AND TRANSFORMATIONS

9 MATRICES AND TRANSFORMATIONS Chapter 9 Matrices and Transformations Objectives After studying this chapter you should be able to handle matrix (and vector) algebra with confidence, and understand the

### Math 241, Exam 1 Information.

Math 241, Exam 1 Information. 9/24/12, LC 310, 11:15-12:05. Exam 1 will be based on: Sections 12.1-12.5, 14.1-14.3. The corresponding assigned homework problems (see http://www.math.sc.edu/ boylan/sccourses/241fa12/241.html)

### MAT188H1S Lec0101 Burbulla

Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u

### 6. Cholesky factorization

6. Cholesky factorization EE103 (Fall 2011-12) triangular matrices forward and backward substitution the Cholesky factorization solving Ax = b with A positive definite inverse of a positive definite matrix

### SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,

### Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components

Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

### Vector and Tensor Algebra (including Column and Matrix Notation)

Vector and Tensor Algebra (including Column and Matrix Notation) 2 Vectors and tensors In mechanics and other fields of physics, quantities are represented by vectors and tensors. Essential manipulations

### Lecture notes on linear algebra

Lecture notes on linear algebra David Lerner Department of Mathematics University of Kansas These are notes of a course given in Fall, 2007 and 2008 to the Honors sections of our elementary linear algebra

### Recall that two vectors in are perpendicular or orthogonal provided that their dot

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

### 1.3. DOT PRODUCT 19. 6. If θ is the angle (between 0 and π) between two non-zero vectors u and v,

1.3. DOT PRODUCT 19 1.3 Dot Product 1.3.1 Definitions and Properties The dot product is the first way to multiply two vectors. The definition we will give below may appear arbitrary. But it is not. It

### 26. Determinants I. 1. Prehistory

26. Determinants I 26.1 Prehistory 26.2 Definitions 26.3 Uniqueness and other properties 26.4 Existence Both as a careful review of a more pedestrian viewpoint, and as a transition to a coordinate-independent

### Dot product and vector projections (Sect. 12.3) There are two main ways to introduce the dot product

Dot product and vector projections (Sect. 12.3) Two definitions for the dot product. Geometric definition of dot product. Orthogonal vectors. Dot product and orthogonal projections. Properties of the dot

BILINEAR FORMS KEITH CONRAD The geometry of R n is controlled algebraically by the dot product. We will abstract the dot product on R n to a bilinear form on a vector space and study algebraic and geometric

### Similar matrices and Jordan form

Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

### Electrical Engineering 103 Applied Numerical Computing

UCLA Fall Quarter 2011-12 Electrical Engineering 103 Applied Numerical Computing Professor L Vandenberghe Notes written in collaboration with S Boyd (Stanford Univ) Contents I Matrix theory 1 1 Vectors

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008 This chapter is available free to all individuals, on understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors

1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1982, 2008. This chapter originates from material used by author at Imperial College, University of London, between 1981 and 1990. It is available free to all individuals,

### α = u v. In other words, Orthogonal Projection

Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University

Linear Algebra Done Wrong Sergei Treil Department of Mathematics, Brown University Copyright c Sergei Treil, 2004, 2009, 2011, 2014 Preface The title of the book sounds a bit mysterious. Why should anyone

### Chapter 7. Matrices. Definition. An m n matrix is an array of numbers set out in m rows and n columns. Examples. ( 1 1 5 2 0 6

Chapter 7 Matrices Definition An m n matrix is an array of numbers set out in m rows and n columns Examples (i ( 1 1 5 2 0 6 has 2 rows and 3 columns and so it is a 2 3 matrix (ii 1 0 7 1 2 3 3 1 is a

### Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

### 1 Local Brouwer degree

1 Local Brouwer degree Let D R n be an open set and f : S R n be continuous, D S and c R n. Suppose that the set f 1 (c) D is compact. (1) Then the local Brouwer degree of f at c in the set D is defined.

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### The Analysis of Data. Volume 1. Probability. Guy Lebanon

The Analysis of Data Volume 1 Probability Guy Lebanon First Edition 2012 Probability. The Analysis of Data, Volume 1. First Edition, First Printing, 2013 http://theanalysisofdata.com Copyright 2013 by

### Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University

Linear Algebra Done Wrong Sergei Treil Department of Mathematics, Brown University Copyright c Sergei Treil, 2004, 2009, 2011, 2014 Preface The title of the book sounds a bit mysterious. Why should anyone

### Review of Vector Analysis in Cartesian Coordinates

R. evicky, CBE 6333 Review of Vector Analysis in Cartesian Coordinates Scalar: A quantity that has magnitude, but no direction. Examples are mass, temperature, pressure, time, distance, and real numbers.

### Section 9.5: Equations of Lines and Planes

Lines in 3D Space Section 9.5: Equations of Lines and Planes Practice HW from Stewart Textbook (not to hand in) p. 673 # 3-5 odd, 2-37 odd, 4, 47 Consider the line L through the point P = ( x, y, ) that

### 1 Sets and Set Notation.

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

### Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10

Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 10 Boundary Value Problems for Ordinary Differential Equations Copyright c 2001. Reproduction

### CONTROLLABILITY. Chapter 2. 2.1 Reachable Set and Controllability. Suppose we have a linear system described by the state equation

Chapter 2 CONTROLLABILITY 2 Reachable Set and Controllability Suppose we have a linear system described by the state equation ẋ Ax + Bu (2) x() x Consider the following problem For a given vector x in

### Recall the basic property of the transpose (for any A): v A t Aw = v w, v, w R n.

ORTHOGONAL MATRICES Informally, an orthogonal n n matrix is the n-dimensional analogue of the rotation matrices R θ in R 2. When does a linear transformation of R 3 (or R n ) deserve to be called a rotation?

### Introduction to Partial Differential Equations. John Douglas Moore

Introduction to Partial Differential Equations John Douglas Moore May 2, 2003 Preface Partial differential equations are often used to construct models of the most basic theories underlying physics and

### 7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix

7. LU factorization EE103 (Fall 2011-12) factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization algorithm effect of rounding error sparse

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### RESULTANT AND DISCRIMINANT OF POLYNOMIALS

RESULTANT AND DISCRIMINANT OF POLYNOMIALS SVANTE JANSON Abstract. This is a collection of classical results about resultants and discriminants for polynomials, compiled mainly for my own use. All results

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### Matrices and Polynomials

APPENDIX 9 Matrices and Polynomials he Multiplication of Polynomials Let α(z) =α 0 +α 1 z+α 2 z 2 + α p z p and y(z) =y 0 +y 1 z+y 2 z 2 + y n z n be two polynomials of degrees p and n respectively. hen,

### Solutions for Review Problems

olutions for Review Problems 1. Let be the triangle with vertices A (,, ), B (4,, 1) and C (,, 1). (a) Find the cosine of the angle BAC at vertex A. (b) Find the area of the triangle ABC. (c) Find a vector

### Vector Spaces. Chapter 2. 2.1 R 2 through R n

Chapter 2 Vector Spaces One of my favorite dictionaries (the one from Oxford) defines a vector as A quantity having direction as well as magnitude, denoted by a line drawn from its original to its final

### x1 x 2 x 3 y 1 y 2 y 3 x 1 y 2 x 2 y 1 0.

Cross product 1 Chapter 7 Cross product We are getting ready to study integration in several variables. Until now we have been doing only differential calculus. One outcome of this study will be our ability

### LECTURE 1: DIFFERENTIAL FORMS. 1. 1-forms on R n

LECTURE 1: DIFFERENTIAL FORMS 1. 1-forms on R n In calculus, you may have seen the differential or exterior derivative df of a function f(x, y, z) defined to be df = f f f dx + dy + x y z dz. The expression

### Modélisation et résolutions numérique et symbolique

Modélisation et résolutions numérique et symbolique via les logiciels Maple et Matlab Jeremy Berthomieu Mohab Safey El Din Stef Graillat Mohab.Safey@lip6.fr Outline Previous course: partial review of what

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### GCE Mathematics (6360) Further Pure unit 4 (MFP4) Textbook

Version 36 klm GCE Mathematics (636) Further Pure unit 4 (MFP4) Textbook The Assessment and Qualifications Alliance (AQA) is a company limited by guarantee registered in England and Wales 364473 and a

### Discrete Convolution and the Discrete Fourier Transform

Discrete Convolution and the Discrete Fourier Transform Discrete Convolution First of all we need to introduce what we might call the wraparound convention Because the complex numbers w j e i πj N have

### STATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. Clarificationof zonationprocedure described onpp. 238-239

STATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. by John C. Davis Clarificationof zonationprocedure described onpp. 38-39 Because the notation used in this section (Eqs. 4.8 through 4.84) is inconsistent

### Chapter 9 Unitary Groups and SU(N)

Chapter 9 Unitary Groups and SU(N) The irreducible representations of SO(3) are appropriate for describing the degeneracies of states of quantum mechanical systems which have rotational symmetry in three

### CS 295 (UVM) The LMS Algorithm Fall 2013 1 / 23

The LMS Algorithm The LMS Objective Function. Global solution. Pseudoinverse of a matrix. Optimization (learning) by gradient descent. LMS or Widrow-Hoff Algorithms. Convergence of the Batch LMS Rule.

### ISOMETRIES OF R n KEITH CONRAD

ISOMETRIES OF R n KEITH CONRAD 1. Introduction An isometry of R n is a function h: R n R n that preserves the distance between vectors: h(v) h(w) = v w for all v and w in R n, where (x 1,..., x n ) = x

### The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression

The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression The SVD is the most generally applicable of the orthogonal-diagonal-orthogonal type matrix decompositions Every

### 3 Orthogonal Vectors and Matrices

3 Orthogonal Vectors and Matrices The linear algebra portion of this course focuses on three matrix factorizations: QR factorization, singular valued decomposition (SVD), and LU factorization The first

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### Section 5.3. Section 5.3. u m ] l jj. = l jj u j + + l mj u m. v j = [ u 1 u j. l mj

Section 5. l j v j = [ u u j u m ] l jj = l jj u j + + l mj u m. l mj Section 5. 5.. Not orthogonal, the column vectors fail to be perpendicular to each other. 5..2 his matrix is orthogonal. Check that

### THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan

### Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University

Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision

### Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### 4.1. COMPLEX NUMBERS

4.1. COMPLEX NUMBERS What You Should Learn Use the imaginary unit i to write complex numbers. Add, subtract, and multiply complex numbers. Use complex conjugates to write the quotient of two complex numbers

### CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES. From Exploratory Factor Analysis Ledyard R Tucker and Robert C.

CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES From Exploratory Factor Analysis Ledyard R Tucker and Robert C MacCallum 1997 180 CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES In

### Cryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur

Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 12 Block Cipher Standards

### A SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS

A SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS Eusebio GÓMEZ, Miguel A. GÓMEZ-VILLEGAS and J. Miguel MARÍN Abstract In this paper it is taken up a revision and characterization of the class of

### MATH1231 Algebra, 2015 Chapter 7: Linear maps

MATH1231 Algebra, 2015 Chapter 7: Linear maps A/Prof. Daniel Chan School of Mathematics and Statistics University of New South Wales danielc@unsw.edu.au Daniel Chan (UNSW) MATH1231 Algebra 1 / 43 Chapter

### ( ) which must be a vector

MATH 37 Linear Transformations from Rn to Rm Dr. Neal, WKU Let T : R n R m be a function which maps vectors from R n to R m. Then T is called a linear transformation if the following two properties are

### The Convolution Operation

The Convolution Operation Convolution is a very natural mathematical operation which occurs in both discrete and continuous modes of various kinds. We often encounter it in the course of doing other operations

### Exam 1 Sample Question SOLUTIONS. y = 2x

Exam Sample Question SOLUTIONS. Eliminate the parameter to find a Cartesian equation for the curve: x e t, y e t. SOLUTION: You might look at the coordinates and notice that If you don t see it, we can

### ME 115(b): Solution to Homework #1

ME 115(b): Solution to Homework #1 Solution to Problem #1: To construct the hybrid Jacobian for a manipulator, you could either construct the body Jacobian, JST b, and then use the body-to-hybrid velocity

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

### Continuous Groups, Lie Groups, and Lie Algebras

Chapter 7 Continuous Groups, Lie Groups, and Lie Algebras Zeno was concerned with three problems... These are the problem of the infinitesimal, the infinite, and continuity... Bertrand Russell The groups

### Solutions to old Exam 1 problems

Solutions to old Exam 1 problems Hi students! I am putting this old version of my review for the first midterm review, place and time to be announced. Check for updates on the web site as to which sections

### Orthogonal Bases and the QR Algorithm

Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries

### Understanding Basic Calculus

Understanding Basic Calculus S.K. Chung Dedicated to all the people who have helped me in my life. i Preface This book is a revised and expanded version of the lecture notes for Basic Calculus and other

### Jim Lambers MAT 169 Fall Semester 2009-10 Lecture 25 Notes

Jim Lambers MAT 169 Fall Semester 009-10 Lecture 5 Notes These notes correspond to Section 10.5 in the text. Equations of Lines A line can be viewed, conceptually, as the set of all points in space that

### Factoring Algorithms

Factoring Algorithms The p 1 Method and Quadratic Sieve November 17, 2008 () Factoring Algorithms November 17, 2008 1 / 12 Fermat s factoring method Fermat made the observation that if n has two factors

### Regression Analysis LECTURE 2. The Multiple Regression Model in Matrices Consider the regression equation. (1) y = β 0 + β 1 x 1 + + β k x k + ε,

LECTURE 2 Regression Analysis The Multiple Regression Model in Matrices Consider the regression equation (1) y = β 0 + β 1 x 1 + + β k x k + ε, and imagine that T observations on the variables y, x 1,,x

### Solutions to Homework 5

Solutions to Homework 5 1. Let z = f(x, y) be a twice continously differentiable function of x and y. Let x = r cos θ and y = r sin θ be the equations which transform polar coordinates into rectangular

### Linear Programming: Chapter 11 Game Theory

Linear Programming: Chapter 11 Game Theory Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Rock-Paper-Scissors

### Network Traffic Modelling

University of York Dissertation submitted for the MSc in Mathematics with Modern Applications, Department of Mathematics, University of York, UK. August 009 Network Traffic Modelling Author: David Slade

### Dynamic Analysis. Mass Matrices and External Forces

4 Dynamic Analysis. Mass Matrices and External Forces The formulation of the inertia and external forces appearing at any of the elements of a multibody system, in terms of the dependent coordinates that

### LINES AND PLANES IN R 3

LINES AND PLANES IN R 3 In this handout we will summarize the properties of the dot product and cross product and use them to present arious descriptions of lines and planes in three dimensional space.

### Lecture 14: Section 3.3

Lecture 14: Section 3.3 Shuanglin Shao October 23, 2013 Definition. Two nonzero vectors u and v in R n are said to be orthogonal (or perpendicular) if u v = 0. We will also agree that the zero vector in

### Lecture 8 February 4

ICS273A: Machine Learning Winter 2008 Lecture 8 February 4 Scribe: Carlos Agell (Student) Lecturer: Deva Ramanan 8.1 Neural Nets 8.1.1 Logistic Regression Recall the logistic function: g(x) = 1 1 + e θt

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

### Content. Chapter 4 Functions 61 4.1 Basic concepts on real functions 62. Credits 11

Content Credits 11 Chapter 1 Arithmetic Refresher 13 1.1 Algebra 14 Real Numbers 14 Real Polynomials 19 1.2 Equations in one variable 21 Linear Equations 21 Quadratic Equations 22 1.3 Exercises 28 Chapter

### LINES AND PLANES CHRIS JOHNSON

LINES AND PLANES CHRIS JOHNSON Abstract. In this lecture we derive the equations for lines and planes living in 3-space, as well as define the angle between two non-parallel planes, and determine the distance

### Structure of the Root Spaces for Simple Lie Algebras

Structure of the Root Spaces for Simple Lie Algebras I. Introduction A Cartan subalgebra, H, of a Lie algebra, G, is a subalgebra, H G, such that a. H is nilpotent, i.e., there is some n such that (H)

### THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

### 3. Let A and B be two n n orthogonal matrices. Then prove that AB and BA are both orthogonal matrices. Prove a similar result for unitary matrices.

Exercise 1 1. Let A be an n n orthogonal matrix. Then prove that (a) the rows of A form an orthonormal basis of R n. (b) the columns of A form an orthonormal basis of R n. (c) for any two vectors x,y R

### Math 215 HW #6 Solutions

Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T

### Row Ideals and Fibers of Morphisms

Michigan Math. J. 57 (2008) Row Ideals and Fibers of Morphisms David Eisenbud & Bernd Ulrich Affectionately dedicated to Mel Hochster, who has been an inspiration to us for many years, on the occasion