Some probability and statistics
|
|
- Antonia Simpson
- 7 years ago
- Views:
Transcription
1 Appendix A Some probability and statistics A Probabilities, random variables and their distribution We summarize a few of the basic concepts of random variables, usually denoted by capital letters, X,Y, Z, etc, and their probability distributions, defined by the cumulative distribution function CDF) F X x)=px x), etc To a random experiment we define a sample space Ω, that contains all the outcomes that can occur in the experiment A random variable, X, is a function defined on a sample space Ω To each outcome ω Ω it defines a real number Xω), that represents the value of a numeric quantity that can be measured in the experiment For X to be called a random variable, the probability PX x) has to be defined for all real x Distributions and moments A probability distribution with cumulative distribution function F X x) can be discrete or continuous with probability function p X x) and probability density function f X x), respectively, such that F X x)=px x)= k x p X k), if X takes only integer values, x f Xy)dy A distribution can be of mixed type The distribution function is then an integral plus discrete jumps; see Appendix B The expectation of a random variable X is defined as the center of gravity in the distribution, k kp X k), E[X]=m X = x= x f Xx)dx The variance is a simple measure of the spreading of the distribution and is 26
2 262 SOME PROBABILITY AND STATISTICS defined as V[X]=E[X m X ) 2 ]=E[X 2 ] m 2 X = Chebyshev s inequality states that, for all ε > 0, k k m X ) 2 p X k), P X m X >ε) E[X m X) 2 ] ε 2 x= x m X) 2 f X x)dx In order to describe the statistical properties of a random function one needs the notion of multivariate distributions If the result of a experiment is described by two different quantities, denoted by X and Y, eg length and height of randomly chosen individual in a population, or the value of a random function at two different time points, one has to deal with a two-dimensional random variable This is described by its two-dimensional distribution function F X,Y x,y)= PX x,y y), or the corresponding two-dimensional probability or density function, f X,Y x,y)= 2 F X,Y x,y) x y Two random variables X,Y are independent if, for all x, y, F X,Y x,y)=f X x)f Y y) An important concept is the covariance between two random variables X and Y, defined as C[X,Y]=E[X m X )Y m Y )]=E[XY] m X m Y The correlation coefficient is equal to the dimensionless, normalized covariance, ρ[x,y]= C[X,Y] V[X]V[Y] Two random variable with zero correlation, ρ[x,y] = 0, are called uncorrelated Note that if two random variables X and Y are independent, then they are also uncorrelated, but the reverse does not hold It can happen that two uncorrelated variables are dependent Speaking about the correlation between two random quantities, one often means the degree of covariation between the two However, one has to remember that the correlation only measures the degree of linear covariation Another meaning of the term correlation is used in connection with two data series, x,,x n ) and y,,y n ) Then sometimes the sum of products, n x ky k, can be called correlation, and a device that produces this sum is called a correlator
3 MULTIDIMENSIONAL NORMAL DISTRIBUTION 263 Conditional distributions If X and Y are two random variables with bivariate density function f X,Y x,y), we can define the conditional distribution for X given Y = y, by the conditional density, f X Y=y x)= f X,Yx,y), f Y y) for every y where the marginal density f Y y) is non-zero The expectation in this distribution, the conditional expectation, is a function of y, and is denoted and defined as E[X Y = y]= x f X Y=y x)dx=my) The conditional variance is defined as x V[X Y = y]= x my)) 2 f X Y=y x)dx= σ X Y y) x The unconditional expectation of X can be obtained from the law of total probability, and computed as } E[X]= x f X Y=y x)dx f Y y)dy=e[e[x Y]] y x The unconditional variance of X is given by V[X]=E[V[X Y]]+V[E[X Y]] A2 Multidimensional normal distribution A one-dimensional normal random variable X with expectation m and variance σ 2 has probability density function f X x)= exp 2πσ 2 2 ) } x m 2, σ and we write X Nm,σ 2 ) If m=0 and σ =, the normal distribution is standardized If X N0,), then σx+ m Nm,σ 2 ), and if X Nm,σ 2 ) then X m)/σ N0, ) We accept a constant random variable, X m, as a normal variable, X Nm,0)
4 264 SOME PROBABILITY AND STATISTICS Now, let X,,X n have expectation m k = E[X k ] and covariances σ jk = C[X j,x k ], and define, with for transpose), µ = m,,m n ), Σ = σ jk )=the covariance matrix for X,,X n It is a characteristic property of the normal distributions that all linear combinations of a multivariate normal variable also has a normal distribution To formulate the definition, write a=a,,a n ) and X=X,,X n ), with a X=a X + +a n X n Then, E[a X]=a µ = V[a X]=a Σa= n j= n j,k a j m j, a j a k σ jk A) Definition A The random variables X,,X n, are said to have an n-dimensional normal distribution is every linear combination a X + +a n X n has a normal distribution From A) we have that X=X,,X n ) is n-dimensional normal, if and only if a X Na µ,a Σa) for all a=a,,a n ) Obviously, X k = 0 X + + X k + +0 X n, is normal, ie, all marginal distributions in an n-dimensional normal distribution are onedimensional normal However, the reverse is not necessarily true; there are variables X,,X n, each of which is one-dimensional normal, but the vector X,,X n ) is not n-dimensional normal It is an important consequence of the definition that sums and differences of n-dimensional normal variables have a normal distribution If the covariance matrix Σ is non-singular, the n-dimensional normal distribution has a probability density function with x=x,,x n ) ) 2π) n/2 det Σ exp } 2 x µ) Σ x µ) A2) The distribution is said to be non-singular The density A2) is constant on every ellipsoid x µ) Σ x µ)= C in R n
5 MULTIDIMENSIONAL NORMAL DISTRIBUTION 265 Note: the density function of an n-dimensional normal distribution is uniquely determined by the expectations and covariances Example A Suppose X,X 2 have a two-dimensional normal distribution If then Σ is non-singular, and det Σ=σ σ 22 σ 2 2 > 0, Σ = det Σ With Qx,x 2 )=x µ) Σ x µ), σ22 σ 2 σ 2 σ Qx,x 2 )= = σ σ 22 σ2 2 x m ) 2 σ 22 2x m )x 2 m 2 )σ 2 +x 2 m 2 ) 2 } σ = ) = ρ 2 x m σ ) 2 2ρ x m σ ) x2 m 2 σ22 )+ x2 m 2 σ22 ) 2 }, where we also used the correlation coefficient ρ = σ 2 σ σ 22, and, f X,X 2 x,x 2 )= 2π σ σ 22 ρ 2 ) exp ) 2 Qx,x 2 ) A3) For variables with m = m 2 = 0 and σ = σ 22 = σ 2, the bivariate density is ) f X,X 2 x,x 2 )= 2πσ 2 ρ exp 2 2σ 2 ρ 2 ) x2 2ρx x 2 + x 2 2 ) We see that, if ρ = 0, this is the density of two independent normal variables Figure A shows the density function f X,X 2 x,x 2 ) and level curves for a bivariate normal distribution with expectation µ = 0, 0), and covariance matrix ) 05 Σ= 05 The correlation coefficient is ρ = 05 Remark A If the covariance matrix Σ is singular and non-invertible, then there exists at least one set of constants a,,a n, not all equal to 0, such that a Σa=0 From A) it follows thatv[a X]=0, which means that a X is constant equal to a µ The distribution of X is concentrated to a hyper plane a x = constant in R n The distribution is said to be singular and it has no density function inr n
6 266 SOME PROBABILITY AND STATISTICS Figure A Two-dimensional normal density Left: density function; Right: elliptic level curves at levels 00, 002, 005, 0, 05 Remark A2 Formula A) implies that every covariance matrix Σ is positive definite or positive semi-definite, ie, j,k a j a k σ jk 0 for all a,,a n Conversely, if Σ is a symmetric, positive definite matrix of size n n, ie, if j,k a j a k σ jk > 0 for all a,,a n 0,,0, then A2) defines the density function for an n-dimensional normal distribution with expectation m k and covariances σ jk Every symmetric, positive definite matrix is a covariance matrix for a non-singular distribution Furthermore, for every symmetric, positive semi-definite matrix, Σ, ie, such that a j a k σ jk 0 j,k for all a,,a n with equality holding for some choice of a,,a n 0,,0, there exists an n-dimensional normal distribution that has Σ as its covariance matrix For n-dimensional normal variables, uncorrelated and independent are equivalent Theorem A If the random variables X,,X n are n- dimensional normal and uncorrelated, then they are independent Proof We show the theorem only for non-singular variables with density function It is true also for singular normal variables If X,,X n are uncorrelated, σ jk = 0 for j k, then Σ, and also Σ are
7 MULTIDIMENSIONAL NORMAL DISTRIBUTION 267 diagonal matrices, ie, note that σ j j =V[X j ]), σ 0 det Σ= σ j j, Σ = j 0 σ nn This means that x µ) Σ x µ) = j x j µ j ) 2 /σ j j, and the density A2) is exp x j m j ) 2 } 2πσ j j j 2σ j j Hence, the joint density function for X,,X n is a product of the marginal densities, which says that the variables are independent A2 Conditional normal distribution This section deals with partial observations in a multivariate normal distribution It is a special property of this distribution, that conditioning on observed values of a subset of variables, leads to a conditional distribution for the unobserved variables that is also normal Furthermore, the expectation in the conditional distribution is linear in the observations, and the covariance matrix does not depend on the observed values This property is particularly useful in prediction of Gaussian time series, as formulated by the Kalman filter Conditioning in the bivariate normal distribution Let X and Y have a bivariate normal distribution with expectations m X and m Y, variances σx 2 and σ Y 2, respectively, and with correlation coefficient ρ = C[X,Y]/σ X σ Y ) The simultaneous density function is given by A3) The conditional density function for X given that Y = y is f X Y=y x)= f X,Yx,y) f Y y) = σ X ρ 2 2π exp x m X+ σ X ρy m Y )/σ Y )) 2 2σ 2 X ρ2 ) Hence, the conditional distribution of X given Y = y is normal with expectation and variance m X Y=y = m X + σ X ρy m Y )/σ Y, σ 2 X Y=y = σ 2 X ρ2 ) Note: the conditional expectation depends linearly on the observed y-value, and the conditional variance is constant, independent of Y = y }
8 268 SOME PROBABILITY AND STATISTICS Conditioning in the multivariate normal distribution Let X=X,,X n ) and Y=Y,,Y m ) be two multivariate normal variables, of size n and m, respectively, such that Z=X,,X n,y,,y m ) is n + m)-dimensional normal Denote the expectations E[X]=m X, E[Y]=m Y, and partition the covariance matrix for Z with Σ XY = Σ YX), [ ) )] ) X X ΣXX Σ Σ=Cov ; = XY A4) Y Y Σ YX Σ YY If the covariance matrix Σ is positive definite, the distribution of X, Y) has the density function f XY x,y)= 2π) m+n)/2 det Σ e 2 x m X,y m Y )Σ x m X,y m Y ), while the m-dimensional density of Y is f Y y)= 2π) m/2 e 2 y m Y) Σ det Σ YY To find the conditional density of X given that Y = y, we need the following matrix property YY y my) f X Y x y)= f YXy,x), A5) f Y y) Theorem A2 Matrix inversions lemma ) Let B be a p p-matrix p = n+m): ) B B B= 2, B 2 B 22 where the sub-matrices have dimension n n, n m, etc Suppose B,B,B 22 are non-singular, and partition the inverse in the same way as B, ) A=B A A = 2 A 2 A 22 Then A= B B 2 B22 B 2) B B 2 B 22 B ) 2) B 2 B 22 B 22 B 2 B B 2) B 2 B B 22 B 2 B B 2)
9 MULTIDIMENSIONAL NORMAL DISTRIBUTION 269 Proof For the proof, see a matrix theory textbook, for example, [22] Theorem A3 Conditional normal distribution ) The conditional normal distribution for X, given that Y = y, is n-dimensional normal with expectation and covariance matrix E[X Y=y] = m X Y=y = m X + Σ XY Σ YY y m Y), A6) C[X Y=y] = Σ XX Y = Σ XX Σ XY Σ YY Σ YX A7) These formulas are easy to remember: the dimension of the submatrices, for example in the covariance matrix Σ XX Y, are the only possible for the matrix multiplications in the right hand side to be meaningful Proof To simplify calculations, we start with m X = m Y = 0, and add the expectations afterwards The conditional distribution of X given that Y = y is, according to A5), the ratio between two multivariate normal densities, and hence it is of the form, c exp 2 x,y ) Σ x y ) + 2 y Σ YY y }=c exp 2 Qx,y)}, where c is a normalization constant, independent of x and y The matrix Σ can be partitioned as in A4), and if we use the matrix inversion lemma, we find that ) Σ A A = A= 2 A8) A 2 A 22 = Σ XX Σ XY Σ YY Σ YX ) Σ YY Σ YX Σ XX Σ XY ) Σ YX Σ XX We also see that Σ XX Σ XY Σ YY Σ YX ) Σ XY Σ YY Σ YY Σ YX Σ XX Σ XY ) Qx,y) = x A x+x A 2 y+y A 2 x+y A 22 y y Σ YY y = x y C )A x Cy)+ Qy) = x A x x A Cy y C A x+y C A Cy+ Qy), for some matrix C and quadratic form Qy) in y )
10 270 SOME PROBABILITY AND STATISTICS Here A = Σ solving XX Y, according to A7) and A8), while we can find C by A C=A 2, ie C= A A 2 = Σ XY Σ YY, according to A8) This is precisely the matrix in A6) If we reinstall the deleted m X and m Y, we get the conditional density for X given Y=y to be of the form c exp 2 Qx,y)}=c exp 2 x m X Y=y )Σ XX Y x m X Y=y)}, which is the normal density we were looking for A22 Complex normal variables In most of the book, we have assumed all random variables to be real valued In many applications, and also in the mathematical background, it is advantageous to consider complex variables, simply defined as Z = X+ iy, where X and Y have a bivariate distribution The mean value of a complex random variable is simply E[Z]=E[RZ]+iE[IZ], while the variance and covariances are defined with complex conjugate on the second variable, C[Z,Z 2 ]=E[Z Z 2 ] m Z m Z2, V[Z]=C[Z,Z]=E[ Z 2 ] m Z 2 Note, that for a complex Z = X+ iy, withv[x]= V[Y]=σ 2, C[Z,Z]=V[X]+V[Y]=2σ 2, C[Z,Z]=V[X] V[Y]+2iC[X,Y]=2iC[X,Y] Hence, if the real and imaginary parts are uncorrelated with the same variance, then the complex variable Z is uncorrelated with its own complex conjugate, Z Often, one uses the term orthogonal, instead of uncorrelated for complex variables
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,
More informationThe Bivariate Normal Distribution
The Bivariate Normal Distribution This is Section 4.7 of the st edition (2002) of the book Introduction to Probability, by D. P. Bertsekas and J. N. Tsitsiklis. The material in this section was not included
More informationSF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2014 Timo Koski () Mathematisk statistik 24.09.2014 1 / 75 Learning outcomes Random vectors, mean vector, covariance
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +
More informationCovariance and Correlation
Covariance and Correlation ( c Robert J. Serfling Not for reproduction or distribution) We have seen how to summarize a data-based relative frequency distribution by measures of location and spread, such
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationOctober 3rd, 2012. Linear Algebra & Properties of the Covariance Matrix
Linear Algebra & Properties of the Covariance Matrix October 3rd, 2012 Estimation of r and C Let rn 1, rn, t..., rn T be the historical return rates on the n th asset. rn 1 rṇ 2 r n =. r T n n = 1, 2,...,
More informationDepartment of Mathematics, Indian Institute of Technology, Kharagpur Assignment 2-3, Probability and Statistics, March 2015. Due:-March 25, 2015.
Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment -3, Probability and Statistics, March 05. Due:-March 5, 05.. Show that the function 0 for x < x+ F (x) = 4 for x < for x
More informationQuadratic forms Cochran s theorem, degrees of freedom, and all that
Quadratic forms Cochran s theorem, degrees of freedom, and all that Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 1, Slide 1 Why We Care Cochran s theorem tells us
More informationIntroduction to Matrix Algebra
Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary
More informationMultivariate normal distribution and testing for means (see MKB Ch 3)
Multivariate normal distribution and testing for means (see MKB Ch 3) Where are we going? 2 One-sample t-test (univariate).................................................. 3 Two-sample t-test (univariate).................................................
More informationSummary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)
Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume
More informationDecember 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS
December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation
More informationLinear Algebra Review. Vectors
Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length
More informationVector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty
More informationMULTIVARIATE PROBABILITY DISTRIBUTIONS
MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined
More informationElliptical copulae. Dorota Kurowicka, Jolanta Misiewicz, Roger Cooke
Elliptical copulae Dorota Kurowicka, Jolanta Misiewicz, Roger Cooke Abstract: In this paper we construct a copula, that is, a distribution with uniform marginals. This copula is continuous and can realize
More informationNOTES ON LINEAR TRANSFORMATIONS
NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all
More informationFactorization Theorems
Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization
More informationMATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.
MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α
More informationEconometrics Simple Linear Regression
Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight
More information13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.
3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms
More informationA SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS
A SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS Eusebio GÓMEZ, Miguel A. GÓMEZ-VILLEGAS and J. Miguel MARÍN Abstract In this paper it is taken up a revision and characterization of the class of
More information1 VECTOR SPACES AND SUBSPACES
1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such
More informationMultivariate Normal Distribution Rebecca Jennings, Mary Wakeman-Linn, Xin Zhao November 11, 2010
Multivariate Normal Distribution Rebecca Jeings, Mary Wakeman-Li, Xin Zhao November, 00. Basics. Parameters We say X ~ N n (µ, ) with parameters µ = [E[X ],.E[X n ]] and = Cov[X i X j ] i=..n, j= n. The
More information1 Introduction to Matrices
1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns
More informationFinite dimensional C -algebras
Finite dimensional C -algebras S. Sundar September 14, 2012 Throughout H, K stand for finite dimensional Hilbert spaces. 1 Spectral theorem for self-adjoint opertors Let A B(H) and let {ξ 1, ξ 2,, ξ n
More informationMathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
More informationE3: PROBABILITY AND STATISTICS lecture notes
E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................
More informationSection 6.1 - Inner Products and Norms
Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,
More informationSolving Linear Systems, Continued and The Inverse of a Matrix
, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing
More informationLecture 21. The Multivariate Normal Distribution
Lecture. The Multivariate Normal Distribution. Definitions and Comments The joint moment-generating function of X,...,X n [also called the moment-generating function of the random vector (X,...,X n )]
More informationNotes on Determinant
ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without
More informationNotes on Orthogonal and Symmetric Matrices MENU, Winter 2013
Notes on Orthogonal and Symmetric Matrices MENU, Winter 201 These notes summarize the main properties and uses of orthogonal and symmetric matrices. We covered quite a bit of material regarding these topics,
More informationST 371 (IV): Discrete Random Variables
ST 371 (IV): Discrete Random Variables 1 Random Variables A random variable (rv) is a function that is defined on the sample space of the experiment and that assigns a numerical variable to each possible
More informationPrinciple of Data Reduction
Chapter 6 Principle of Data Reduction 6.1 Introduction An experimenter uses the information in a sample X 1,..., X n to make inferences about an unknown parameter θ. If the sample size n is large, then
More informationLecture 9: Introduction to Pattern Analysis
Lecture 9: Introduction to Pattern Analysis g Features, patterns and classifiers g Components of a PR system g An example g Probability definitions g Bayes Theorem g Gaussian densities Features, patterns
More informationTHE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok
THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan
More informationLecture 5 Principal Minors and the Hessian
Lecture 5 Principal Minors and the Hessian Eivind Eriksen BI Norwegian School of Management Department of Economics October 01, 2010 Eivind Eriksen (BI Dept of Economics) Lecture 5 Principal Minors and
More informationLinear Algebra Notes
Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note
More informationProbability Calculator
Chapter 95 Introduction Most statisticians have a set of probability tables that they refer to in doing their statistical wor. This procedure provides you with a set of electronic statistical tables that
More informationMATHEMATICAL METHODS OF STATISTICS
MATHEMATICAL METHODS OF STATISTICS By HARALD CRAMER TROFESSOK IN THE UNIVERSITY OF STOCKHOLM Princeton PRINCETON UNIVERSITY PRESS 1946 TABLE OF CONTENTS. First Part. MATHEMATICAL INTRODUCTION. CHAPTERS
More informationNumerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J. Olver 6. Eigenvalues and Singular Values In this section, we collect together the basic facts about eigenvalues and eigenvectors. From a geometrical viewpoint,
More information8 Square matrices continued: Determinants
8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You
More informationLinear Algebra Notes for Marsden and Tromba Vector Calculus
Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of
More informationCONDITIONAL, PARTIAL AND RANK CORRELATION FOR THE ELLIPTICAL COPULA; DEPENDENCE MODELLING IN UNCERTAINTY ANALYSIS
CONDITIONAL, PARTIAL AND RANK CORRELATION FOR THE ELLIPTICAL COPULA; DEPENDENCE MODELLING IN UNCERTAINTY ANALYSIS D. Kurowicka, R.M. Cooke Delft University of Technology, Mekelweg 4, 68CD Delft, Netherlands
More informationTHE DYING FIBONACCI TREE. 1. Introduction. Consider a tree with two types of nodes, say A and B, and the following properties:
THE DYING FIBONACCI TREE BERNHARD GITTENBERGER 1. Introduction Consider a tree with two types of nodes, say A and B, and the following properties: 1. Let the root be of type A.. Each node of type A produces
More informationThe Characteristic Polynomial
Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem
More informationProbability and Random Variables. Generation of random variables (r.v.)
Probability and Random Variables Method for generating random variables with a specified probability distribution function. Gaussian And Markov Processes Characterization of Stationary Random Process Linearly
More informationModern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem
More informationAdding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors
1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number
More informationMATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column
More informationDIFFERENTIABILITY OF COMPLEX FUNCTIONS. Contents
DIFFERENTIABILITY OF COMPLEX FUNCTIONS Contents 1. Limit definition of a derivative 1 2. Holomorphic functions, the Cauchy-Riemann equations 3 3. Differentiability of real functions 5 4. A sufficient condition
More information3. INNER PRODUCT SPACES
. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.
More informationSolving Systems of Linear Equations
LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how
More information18.06 Problem Set 4 Solution Due Wednesday, 11 March 2009 at 4 pm in 2-106. Total: 175 points.
806 Problem Set 4 Solution Due Wednesday, March 2009 at 4 pm in 2-06 Total: 75 points Problem : A is an m n matrix of rank r Suppose there are right-hand-sides b for which A x = b has no solution (a) What
More informationLecture 13: Martingales
Lecture 13: Martingales 1. Definition of a Martingale 1.1 Filtrations 1.2 Definition of a martingale and its basic properties 1.3 Sums of independent random variables and related models 1.4 Products of
More information15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
More information5. Orthogonal matrices
L Vandenberghe EE133A (Spring 2016) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal
More informationLINEAR ALGEBRA. September 23, 2010
LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................
More informationInner Product Spaces
Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and
More informationReview Jeopardy. Blue vs. Orange. Review Jeopardy
Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round $200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?
More informationEigenvalues, Eigenvectors, Matrix Factoring, and Principal Components
Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they
More informationMATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.
MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column
More informationGeneral Sampling Methods
General Sampling Methods Reference: Glasserman, 2.2 and 2.3 Claudio Pacati academic year 2016 17 1 Inverse Transform Method Assume U U(0, 1) and let F be the cumulative distribution function of a distribution
More informationA characterization of trace zero symmetric nonnegative 5x5 matrices
A characterization of trace zero symmetric nonnegative 5x5 matrices Oren Spector June 1, 009 Abstract The problem of determining necessary and sufficient conditions for a set of real numbers to be the
More informationGaussian Conjugate Prior Cheat Sheet
Gaussian Conjugate Prior Cheat Sheet Tom SF Haines 1 Purpose This document contains notes on how to handle the multivariate Gaussian 1 in a Bayesian setting. It focuses on the conjugate prior, its Bayesian
More informationFOUNDATIONS OF ALGEBRAIC GEOMETRY CLASS 22
FOUNDATIONS OF ALGEBRAIC GEOMETRY CLASS 22 RAVI VAKIL CONTENTS 1. Discrete valuation rings: Dimension 1 Noetherian regular local rings 1 Last day, we discussed the Zariski tangent space, and saw that it
More informationLinear Algebra I. Ronald van Luijk, 2012
Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.
More informationCorrelation in Random Variables
Correlation in Random Variables Lecture 11 Spring 2002 Correlation in Random Variables Suppose that an experiment produces two random variables, X and Y. What can we say about the relationship between
More informationSystems of Linear Equations
Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and
More informationMulti-variable Calculus and Optimization
Multi-variable Calculus and Optimization Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Multi-variable Calculus and Optimization 1 / 51 EC2040 Topic 3 - Multi-variable Calculus
More informationLecture Notes 1. Brief Review of Basic Probability
Probability Review Lecture Notes Brief Review of Basic Probability I assume you know basic probability. Chapters -3 are a review. I will assume you have read and understood Chapters -3. Here is a very
More informationSimilarity and Diagonalization. Similar Matrices
MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that
More informationHow To Prove The Dirichlet Unit Theorem
Chapter 6 The Dirichlet Unit Theorem As usual, we will be working in the ring B of algebraic integers of a number field L. Two factorizations of an element of B are regarded as essentially the same if
More informationIntroduction to Algebraic Geometry. Bézout s Theorem and Inflection Points
Introduction to Algebraic Geometry Bézout s Theorem and Inflection Points 1. The resultant. Let K be a field. Then the polynomial ring K[x] is a unique factorisation domain (UFD). Another example of a
More informationSections 2.11 and 5.8
Sections 211 and 58 Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1/25 Gesell data Let X be the age in in months a child speaks his/her first word and
More informationAn Introduction to Basic Statistics and Probability
An Introduction to Basic Statistics and Probability Shenek Heyward NCSU An Introduction to Basic Statistics and Probability p. 1/4 Outline Basic probability concepts Conditional probability Discrete Random
More informationLinear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear
More informationExample: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C
More informationChapter 17. Orthogonal Matrices and Symmetries of Space
Chapter 17. Orthogonal Matrices and Symmetries of Space Take a random matrix, say 1 3 A = 4 5 6, 7 8 9 and compare the lengths of e 1 and Ae 1. The vector e 1 has length 1, while Ae 1 = (1, 4, 7) has length
More informationNotes on Continuous Random Variables
Notes on Continuous Random Variables Continuous random variables are random quantities that are measured on a continuous scale. They can usually take on any value over some interval, which distinguishes
More informationLecture 8. Confidence intervals and the central limit theorem
Lecture 8. Confidence intervals and the central limit theorem Mathematical Statistics and Discrete Mathematics November 25th, 2015 1 / 15 Central limit theorem Let X 1, X 2,... X n be a random sample of
More informationLecture L3 - Vectors, Matrices and Coordinate Transformations
S. Widnall 16.07 Dynamics Fall 2009 Lecture notes based on J. Peraire Version 2.0 Lecture L3 - Vectors, Matrices and Coordinate Transformations By using vectors and defining appropriate operations between
More informationPolynomial Invariants
Polynomial Invariants Dylan Wilson October 9, 2014 (1) Today we will be interested in the following Question 1.1. What are all the possible polynomials in two variables f(x, y) such that f(x, y) = f(y,
More informationBEGINNING ALGEBRA ACKNOWLEDMENTS
BEGINNING ALGEBRA The Nursing Department of Labouré College requested the Department of Academic Planning and Support Services to help with mathematics preparatory materials for its Bachelor of Science
More informationMAT188H1S Lec0101 Burbulla
Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u
More informationCITY UNIVERSITY LONDON. BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION
No: CITY UNIVERSITY LONDON BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION ENGINEERING MATHEMATICS 2 (resit) EX2005 Date: August
More informationTHREE DIMENSIONAL GEOMETRY
Chapter 8 THREE DIMENSIONAL GEOMETRY 8.1 Introduction In this chapter we present a vector algebra approach to three dimensional geometry. The aim is to present standard properties of lines and planes,
More informationOverview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationProbability and Statistics Vocabulary List (Definitions for Middle School Teachers)
Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) B Bar graph a diagram representing the frequency distribution for nominal or discrete data. It consists of a sequence
More informationMATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.
MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar
More informationSTAT 830 Convergence in Distribution
STAT 830 Convergence in Distribution Richard Lockhart Simon Fraser University STAT 830 Fall 2011 Richard Lockhart (Simon Fraser University) STAT 830 Convergence in Distribution STAT 830 Fall 2011 1 / 31
More informationMethods for Finding Bases
Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,
More informationThe Determinant: a Means to Calculate Volume
The Determinant: a Means to Calculate Volume Bo Peng August 20, 2007 Abstract This paper gives a definition of the determinant and lists many of its well-known properties Volumes of parallelepipeds are
More informationFUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES
FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES CHRISTOPHER HEIL 1. Cosets and the Quotient Space Any vector space is an abelian group under the operation of vector addition. So, if you are have studied
More informationName: Section Registered In:
Name: Section Registered In: Math 125 Exam 3 Version 1 April 24, 2006 60 total points possible 1. (5pts) Use Cramer s Rule to solve 3x + 4y = 30 x 2y = 8. Be sure to show enough detail that shows you are
More informationDiscrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 18. A Brief Introduction to Continuous Probability
CS 7 Discrete Mathematics and Probability Theory Fall 29 Satish Rao, David Tse Note 8 A Brief Introduction to Continuous Probability Up to now we have focused exclusively on discrete probability spaces
More informationNonlinear Programming Methods.S2 Quadratic Programming
Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective
More information