Sections 2.11 and 5.8


 Diane Burke
 2 years ago
 Views:
Transcription
1 Sections 211 and 58 Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1/25
2 Gesell data Let X be the age in in months a child speaks his/her first word and let Y be the Gesell adaptive score, a measure of a child s aptitude (observed later on) Are X and Y related? How does the child s aptitude change with how long it takes them to speak? Here s the Gesell score y i and age at first word in months x i data, i = 1,,21 x i y i x i y i x i y i x i y i x i y i In R, we compute r = 0640, a moderately strong negative relationship between age at first word spoken and Gesell score > age=c(15,26,10,9,15,20,18,11,8,20,7,9,10,11,11,10,12,42,17,11,10) > Gesell=c(95,71,83,91,102,87,93,100,104,94,113,96,83,84,102,100,105,57,121,86,100) > plot(age,gesell) > cor(age,gesell) [1] /25
3 Scatterplot of (x 1,y 1 ),,(x 21,y 21 ) Gesell age 3/25
4 Random vectors A random vector X = variables X 1 X 2 X k is made up of, say, k random A random vector has a joint distribution, eg a density f(x), that gives probabilities P(X A) = f(x)dx Just as a random variable X has a mean E(X) and variance var(x), a random vector also has a mean vector E(X) and a covariance matrix cov(x) A 4/25
5 Mean vector & covariance matrix Let X = (X 1,,X k ) be a random vector with density f(x 1,,x k ) The mean of X is the vector of marginal means X 1 E(X 1 ) X 2 E(X) = E = E(X 2 ) (538) X k E(X k ) The covariance matrix of X is given by cov(x 1,X 1 ) cov(x 1,X 2 ) cov(x 1,X k ) cov(x 2,X 1 ) cov(x 2,X 2 ) cov(x 2,X k ) cov(x) = cov(x k,x 1 ) cov(x k,x 2 ) cov(x k,x k ) (542) 5/25
6 Multivariate normal distribution The normal distribution generalizes to multiple dimensions We ll first look at two jointly distributed normal random variables, then discuss three or more The bivariate normal density for (X 1,X 2 ) is given by f(x 1,x 2 ) = { [ (x1 ) 1 2πσ 1 σ 2 1 ρ 2 exp 1 µ 2 ( )( ) ( ) ]} 1 x1 µ 1 x2 µ 2 x2 µ 2 2 2(1 ρ 2 2ρ + ) σ 1 σ 1 σ 2 σ 2 There are 5 parameters: (µ 1,µ 2,σ 1,σ 2,ρ) Besides 58, also see 211 pp /25
7 Bivariate normal distribution This density jointly defines X 1 and X 2, which live in R 2 = (, ) (, ) Marginally, X 1 N(µ 1,σ1 2) and X 2 N(µ 2,σ2 2 ) (p 79) The correlation between X 1 and X 2 is given by corr(x 1,X 2 ) = ρ (p 80) For jointly normal random variables, if the correlation is zero then they are independent This is not true in general for jointly defined random variables E(X) = [ µ1 ], cov(x) = [ σ 2 1 σ 1 σ 2 ρ µ 2 σ 1 σ 2 ρ σ2 2 Next slide: µ 1 = 0,1; µ 2 = 0,2; σ1 2 = σ2 2 = 1; ρ = 0,09, 06 ] 7/25
8 Bivariate normal PDF level curves /25
9 Proof that X 1 indpendent X 2 when ρ = 0 When ρ = 0 the joint density for (X 1,X 2 ) simplifies to { [ 1 f(x 1,x 2 ) = exp 1 (x1 ) µ 2 ( ) ]} 1 x2 µ πσ 1 σ 2 2 σ 1 σ 2 [ ( ) 1 = e 05 x1 µ 2 ][ ( ) 1 1 σ 1 e 05 x2 µ ] σ 2 2πσ1 2πσ2 Since these are each respectively functions of x 1 and x 2 only, and the range of (X 1,X 2 ) factors into the produce of two sets, X 1 and X 2 are independent and in fact X 1 N(µ 1,σ 2 1 ) and X 2 N(µ 2,σ 2 2 ) 9/25
10 Conditional distributions [X 1 X 2 = x 2 ] and [X 2 X 1 = x 1 ] (pp 80 81) The conditional distribution of X 1 given X 2 = x 2 is ( [X 1 X 2 = x 2 ] N µ 1 + σ ) 1 ρ(x 2 µ 2 ),σ 2 σ 1(1 ρ 2 ) 2 Similarly, [X 2 X 1 = x 1 ] N This ties directly to linear regression: ( µ 2 + σ ) 2 ρ(x 1 µ 1 ),σ 2 σ 2(1 ρ 2 ) 1 To predict X 2 X 1 = x 1, we have [ E(X 2 X 1 = x 1 ) = µ 2 σ ] [ ] 2 σ2 ρµ 1 + ρ x 1 = β 0 +β 1 x 1 σ 1 σ 1 10/25
11 Bivariate normal distribution as data model Here we assume or succinctly, [ Xi1 X i2 ] iid N2 ([ µ1 µ 2 ], X i iid N2 (µ,σ) [ ]) σ11 σ 12, σ 21 σ 22 If the bivariate normal model is appropriate for paired outcomes, it provides a convenient probability model with some nice properties The sample mean X = 1 n n i=1 X i is the MLE of µ and the sample covariance matrix S = 1 n n 1 i=1 (X i X)(X i X) is unbiased for Σ It can be shown that X N 2 ( µ, 1 n Σ ) The matrix (n 1)S has a Wishart distribution (generalizes χ 2 ) 11/25
12 Sample mean vector & covariance matrix Say n outcome pairs are to be recorded: {(X 11,X 12 ),(X 21,X 22 ),,(X n1,x n2 )} The i th pair is (X i1,x i2 ) The sample mean vector is given elementwise by X = [ X1 X 2 ] = [ 1 n n i=1 X i1 1 n n i=1 X i2 and the sample covariance matrix is given elementwise by [ 1 n S = n 1 i=1 (X i1 X 1 ) 2 1 n n 1 i=1 (X ] i1 X 1 )(X i2 X 2 ) 1 n n 1 i=1 (X i1 X 1 )(X i2 X 1 n 2 ) n 1 i=1 (X i2 X 2 ) 2 ], 12/25
13 Estimation The sample mean vector X estimates µ = [ µ1 µ 2 ] and the sample covariance [ matrix S estimates ] [ ] σ1 Σ = 2 ρσ 1 σ 2 σ11 σ ρσ 1 σ 2 σ2 2 = 12 σ 21 σ 22 We will place hats on parameter estimators based on the data So ˆµ 1 = X 1, ˆµ 2 = X 2, ˆσ 2 1 = s 2 1 = 1 n 1 n (X i1 X 1 ) 2, i=1 Also, ˆσ 2 2 = s 2 2 = 1 n 1 ĉov(x 1,X 2 ) = 1 n 1 n (X i2 X 2 ) 2 i=1 n (X i1 X)(X i2 X 2 ) i=1 13/25
14 Correlation coefficient r So a natural estimate of ρ is then ˆρ = ĉov(x 1,X 2 ) ˆσ 1ˆσ 2 = 1 n 1 1 n n 1 i=1 (X i1 X 1 )(X i2 X 2 ) n i=1 (X i1 X 1 ) 2 n i=1 (X i1 X 1 ) 2 1 n 1 This is in fact the MLE estimate based on the bivariate normal model It is also a plugin estimator based on the methodofmoments too It is commonly referred to as the Pearson correlation coefficient You can get it as, eg, cor(age,gesell) in R This estimate of correlation can be unduly influenced by outliers in the sample An alternative measure of linear association is the Spearman correlation based on ranks, discussed a few lectures ago > cor(age,gesell) [1] > cor(age,gesell,method="spearman") [1] /25
15 Gesell data Recall: X is age in months a child speaks his/her first word and let Y is Gesell adaptive score, a measure of a child s aptitude Question: how does the child s aptitude change with how long it takes them to speak? [ Here, ] n = 21 [ ] In R we find X = Also, S = Assuming a bivariate model, we plug in the estimates and obtain the estimated PDF for (X,Y): f(x,y) = exp( x 00134x y 00098xy 00043y 2 ) We can further find from Y N(9367,18632), f Y (y) = exp( (y 9367) 2 ) 15/25
16 3D plot of f(x,y) for (X,Y) estimated from data /25
17 Gesell conditional distribution Solid is f Y (y); left dashed is f Y X (y 25) the right dashed is f Y X (y 10) As the age in months of first words X = x increases, the distribution of Gesell Adaptive Scores Y decreases 17/25
18 Density estimate with actual data Gesell age 18/25
19 Multivariate normal distribution In general, a kvariate normal is defined through the mean and covariance matrix: X 1 µ 1 σ 11 σ 12 σ 1k X 2 N µ 2 k, σ 21 σ 22 σ 2k X k µ k σ k1 σ k2 σ kk Succinctly, X N k (µ,σ) Recall that if Z N(0,1), then X = µ+σz N(µ,σ 2 ) The definition of the multivariate normal distribution just extends this idea 19/25
20 Multivariate normal made from independent normals Instead of one standard normal, we have a list of k independent standard normals Z = (Z 1,,Z k ), and consider the same sort of transformation in the multivariate case using matrices and vectors Let Z 1,,Z k iid N(0,1) The joint pdf of (Z1,,Z k ) is given by Let µ = f(z 1,,z k ) = µ 1 µ 2 µ k k exp( 05zi 2 )/ 2π i=1 σ 11 σ 12 σ 1k and Σ = σ 21 σ 22 σ 2k, σ k1 σ k2 σ kk where Σ is symmetric (ie Σ = Σ, which implies σ ij = σ ji for all 1 i,j k) 20/25
21 Multivariate normal made from independent normals Let Σ 1/2 be any matrix such that Σ 1/2 Σ 1/2 = Σ Then X = µ+σ 1/2 Z is said to have a multivariate normal distribution with mean vector µ and covariance matrix Σ, written Written in terms of matrices X N k (µ,σ) X = X 1 X 2 X k = µ 1 µ 2 µ k + σ 11 σ 12 σ 1k σ 21 σ 22 σ 2k σ k1 σ k2 σ kk 1/2 Z 1 Z 2 Z k 21/25
22 Joint PDF Using some math, it can be shown that the pdf of the new vector X = (X 1,,X k ) is given by f(x 1,,x k µ,σ) = 2πΣ 1/2 exp{ 05(x µ) Σ 1 (x µ)} In the onedimensional case, this simplifies to our old friend f(x 1 µ,σ 2 ) = (2πσ 2 ) 1/2 exp{ 05(x µ)(σ 2 ) 1 (x µ)}, the pdf of a N(µ,σ 2 ) random variable X A is the determinant of the matrix A, and is a function of the elements of A, but beyond this course 22/25
23 Properties of multivariate normal vectors Let Then X N k (µ,σ) 1 For each X i in X = (X 1,,X k ), E(X i ) = µ i and var(x i ) = σ ii That is, marginally, X i N(µ i,σ ii ) 2 For any r k matrix M, MX N r (Mµ,MΣM ) 3 For any two (X i,x j ) where 1 i < j k, cov(x i,x j ) = σ ij The offdiagonal elements of Σ give the covariance between two elements of (X 1,,X k ) Note then ρ(x i,x j ) = σ ij / σ ii σ jj 4 For any k 1 vector m = (m 1,,m k ) and Y N k (µ,σ), m+y N k (m+µ,σ) 23/25
24 Example Let Define X 1 X 2 N 3 2 5, X [ ] [ ] Y1 M = and Y = = MX = Y 2 [ ] X 1 X 2 X 3 Then X 2 N(5,3), cov(x 2,X 3 ) = 1 and [ ] X 1 X X 3 [ ] N [ ] , or simplifying, [ Y1 Y 2 ] = [ ] X 1 X 2 X 3 N 2 ([ 2 1 ] [ 4 0, Note that for the transformed vector Y = (Y 1,Y 2 ), cov(y 1,Y 2 ) = 0 and therefore Y 1 and Y 2 are uncorrelated, ie ρ(y 1,Y 2 ) = 0 ]), 24/25
25 Simple linear regression For the linear model (eg simple linear regression or the twosample model) Y = Xβ +ǫ, the error vector is assumed (pp ) ǫ N n (0,I n n σ 2 ) Then the least squares estimators have a multivariate normal distribution β N p (β,(x X) 1 σ 2 ) p = 2 is the number of mean parameters (The MSE has a gamma distribution) We ll discuss this shortly! 25/25
Multivariate Normal Distribution
Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #47/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues
More informationRandom Vectors and the Variance Covariance Matrix
Random Vectors and the Variance Covariance Matrix Definition 1. A random vector X is a vector (X 1, X 2,..., X p ) of jointly distributed random variables. As is customary in linear algebra, we will write
More informationCS229 Lecture notes. Andrew Ng
CS229 Lecture notes Andrew Ng Part X Factor analysis Whenwehavedatax (i) R n thatcomesfromamixtureofseveral Gaussians, the EM algorithm can be applied to fit a mixture model. In this setting, we usually
More informationLecture Notes 1. Brief Review of Basic Probability
Probability Review Lecture Notes Brief Review of Basic Probability I assume you know basic probability. Chapters 3 are a review. I will assume you have read and understood Chapters 3. Here is a very
More informationNotes for STA 437/1005 Methods for Multivariate Data
Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.
More informationStat 704 Data Analysis I Probability Review
1 / 30 Stat 704 Data Analysis I Probability Review Timothy Hanson Department of Statistics, University of South Carolina Course information 2 / 30 Logistics: Tuesday/Thursday 11:40am to 12:55pm in LeConte
More informationBivariate Distributions
Chapter 4 Bivariate Distributions 4.1 Distributions of Two Random Variables In many practical cases it is desirable to take more than one measurement of a random observation: (brief examples) 1. What is
More information15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
More informationStatistiek (WISB361)
Statistiek (WISB361) Final exam June 29, 2015 Schrijf uw naam op elk in te leveren vel. Schrijf ook uw studentnummer op blad 1. The maximum number of points is 100. Points distribution: 23 20 20 20 17
More information4. Joint Distributions of Two Random Variables
4. Joint Distributions of Two Random Variables 4.1 Joint Distributions of Two Discrete Random Variables Suppose the discrete random variables X and Y have supports S X and S Y, respectively. The joint
More informationSF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,
More informationP (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i )
Discrete random variables Probability mass function Given a discrete random variable X taking values in X = {v 1,..., v m }, its probability mass function P : X [0, 1] is defined as: P (v i ) = Pr[X =
More informationExample: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation:  Feature vector X,  qualitative response Y, taking values in C
More informationOutline. Correlation & Regression, III. Review. Relationship between r and regression
Outline Correlation & Regression, III 9.07 4/6/004 Relationship between correlation and regression, along with notes on the correlation coefficient Effect size, and the meaning of r Other kinds of correlation
More informationChapter 6: Point Estimation. Fall 2011.  Probability & Statistics
STAT355 Chapter 6: Point Estimation Fall 2011 Chapter Fall 2011 6: Point1 Estimat / 18 Chap 6  Point Estimation 1 6.1 Some general Concepts of Point Estimation Point Estimate Unbiasedness Principle of
More informationJoint Probability Distributions and Random Samples (Devore Chapter Five)
Joint Probability Distributions and Random Samples (Devore Chapter Five) 101634501 Probability and Statistics for Engineers Winter 20102011 Contents 1 Joint Probability Distributions 1 1.1 Two Discrete
More information3. The Multivariate Normal Distribution
3. The Multivariate Normal Distribution 3.1 Introduction A generalization of the familiar bell shaped normal density to several dimensions plays a fundamental role in multivariate analysis While real data
More informationSF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2014 Timo Koski () Mathematisk statistik 24.09.2014 1 / 75 Learning outcomes Random vectors, mean vector, covariance
More informationLesson 5 Chapter 4: Jointly Distributed Random Variables
Lesson 5 Chapter 4: Jointly Distributed Random Variables Department of Statistics The Pennsylvania State University 1 Marginal and Conditional Probability Mass Functions The Regression Function Independence
More informationJoint Probability Distributions and Random Samples. Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage
5 Joint Probability Distributions and Random Samples Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage Two Discrete Random Variables The probability mass function (pmf) of a single
More informationThe Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables
Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh The Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables 1 The Monte Carlo Framework Suppose we wish
More informationRegression III: Advanced Methods
Lecture 5: Linear leastsquares Regression III: Advanced Methods William G. Jacoby Department of Political Science Michigan State University http://polisci.msu.edu/jacoby/icpsr/regress3 Simple Linear Regression
More informationSlides for Risk Management
Slides for Risk Management VaR and Expected Shortfall Groll Seminar für Finanzökonometrie Prof. Mittnik, PhD 1 Introduction ValueatRisk Expected Shortfall Model risk Multiperiod / multiasset case 2
More informationSlides for Risk Management VaR and Expected Shortfall
Slides for Risk Management VaR and Expected Shortfall Groll Seminar für Finanzökonometrie Prof. Mittnik, PhD Groll (Seminar für Finanzökonometrie) Slides for Risk Management Prof. Mittnik, PhD 1 / 133
More informationTopic 4: Multivariate random variables. Multiple random variables
Topic 4: Multivariate random variables Joint, marginal, and conditional pmf Joint, marginal, and conditional pdf and cdf Independence Expectation, covariance, correlation Conditional expectation Two jointly
More informationQuadratic forms Cochran s theorem, degrees of freedom, and all that
Quadratic forms Cochran s theorem, degrees of freedom, and all that Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 1, Slide 1 Why We Care Cochran s theorem tells us
More informationExploratory Factor Analysis and Principal Components. Pekka Malo & Anton Frantsev 30E00500 Quantitative Empirical Research Spring 2016
and Principal Components Pekka Malo & Anton Frantsev 30E00500 Quantitative Empirical Research Spring 2016 Agenda Brief History and Introductory Example Factor Model Factor Equation Estimation of Loadings
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationThe Big 50 Revision Guidelines for S1
The Big 50 Revision Guidelines for S1 If you can understand all of these you ll do very well 1. Know what is meant by a statistical model and the Modelling cycle of continuous refinement 2. Understand
More informationFactor analysis. Angela Montanari
Factor analysis Angela Montanari 1 Introduction Factor analysis is a statistical model that allows to explain the correlations between a large number of observed correlated variables through a small number
More informationSummary of Formulas and Concepts. Descriptive Statistics (Ch. 14)
Summary of Formulas and Concepts Descriptive Statistics (Ch. 14) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume
More informationMULTIVARIATE PROBABILITY DISTRIBUTIONS
MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined
More informationMultivariate Analysis (Slides 13)
Multivariate Analysis (Slides 13) The final topic we consider is Factor Analysis. A Factor Analysis is a mathematical approach for attempting to explain the correlation between a large set of variables
More informationJoint Exam 1/P Sample Exam 1
Joint Exam 1/P Sample Exam 1 Take this practice exam under strict exam conditions: Set a timer for 3 hours; Do not stop the timer for restroom breaks; Do not look at your notes. If you believe a question
More informationJoint Distributions. Tieming Ji. Fall 2012
Joint Distributions Tieming Ji Fall 2012 1 / 33 X : univariate random variable. (X, Y ): bivariate random variable. In this chapter, we are going to study the distributions of bivariate random variables
More informationCorrelation in Random Variables
Correlation in Random Variables Lecture 11 Spring 2002 Correlation in Random Variables Suppose that an experiment produces two random variables, X and Y. What can we say about the relationship between
More informationA Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution
A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 4: September
More informationIntroduction to Multivariate Analysis
Introduction to Multivariate Analysis Lecture 1 August 24, 2005 Multivariate Analysis Lecture #18/24/2005 Slide 1 of 30 Today s Lecture Today s Lecture Syllabus and course overview Chapter 1 (a brief
More informationDepartment of Mathematics, Indian Institute of Technology, Kharagpur Assignment 23, Probability and Statistics, March 2015. Due:March 25, 2015.
Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment 3, Probability and Statistics, March 05. Due:March 5, 05.. Show that the function 0 for x < x+ F (x) = 4 for x < for x
More informationST 371 (VIII): Theory of Joint Distributions
ST 371 (VIII): Theory of Joint Distributions So far we have focused on probability distributions for single random variables. However, we are often interested in probability statements concerning two or
More informationThe Delta Method and Applications
Chapter 5 The Delta Method and Applications 5.1 Linear approximations of functions In the simplest form of the central limit theorem, Theorem 4.18, we consider a sequence X 1, X,... of independent and
More informationElementary Statistics. Scatter Plot, Regression Line, Linear Correlation Coefficient, and Coefficient of Determination
Scatter Plot, Regression Line, Linear Correlation Coefficient, and Coefficient of Determination What is a Scatter Plot? A Scatter Plot is a plot of ordered pairs (x, y) where the horizontal axis is used
More informationSome probability and statistics
Appendix A Some probability and statistics A Probabilities, random variables and their distribution We summarize a few of the basic concepts of random variables, usually denoted by capital letters, X,Y,
More informationMultivariate Normal Distribution Rebecca Jennings, Mary WakemanLinn, Xin Zhao November 11, 2010
Multivariate Normal Distribution Rebecca Jeings, Mary WakemanLi, Xin Zhao November, 00. Basics. Parameters We say X ~ N n (µ, ) with parameters µ = [E[X ],.E[X n ]] and = Cov[X i X j ] i=..n, j= n. The
More informationAn Introduction to Machine Learning
An Introduction to Machine Learning L5: Novelty Detection and Regression Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia Alex.Smola@nicta.com.au Tata Institute, Pune,
More informationSpatial Statistics Chapter 3 Basics of areal data and areal data modeling
Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Recall areal data also known as lattice data are data Y (s), s D where D is a discrete index set. This usually corresponds to data
More informationLeast Squares Estimation
Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN13: 9780470860809 ISBN10: 0470860804 Editors Brian S Everitt & David
More informationMultivariate normal distribution and testing for means (see MKB Ch 3)
Multivariate normal distribution and testing for means (see MKB Ch 3) Where are we going? 2 Onesample ttest (univariate).................................................. 3 Twosample ttest (univariate).................................................
More informationL10: Probability, statistics, and estimation theory
L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian
More informationRegression, least squares
Regression, least squares Joe Felsenstein Department of Genome Sciences and Department of Biology Regression, least squares p.1/24 Fitting a straight line X Two distinct cases: The X values are chosen
More informationStatistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
More informationLinear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear
More informationChapters 5. Multivariate Probability Distributions
Chapters 5. Multivariate Probability Distributions Random vectors are collection of random variables defined on the same sample space. Whenever a collection of random variables are mentioned, they are
More informationCovariance and Correlation
Covariance and Correlation ( c Robert J. Serfling Not for reproduction or distribution) We have seen how to summarize a databased relative frequency distribution by measures of location and spread, such
More informationLecture 8: Signal Detection and Noise Assumption
ECE 83 Fall Statistical Signal Processing instructor: R. Nowak, scribe: Feng Ju Lecture 8: Signal Detection and Noise Assumption Signal Detection : X = W H : X = S + W where W N(, σ I n n and S = [s, s,...,
More informationCorrelation key concepts:
CORRELATION Correlation key concepts: Types of correlation Methods of studying correlation a) Scatter diagram b) Karl pearson s coefficient of correlation c) Spearman s Rank correlation coefficient d)
More informationLecture 9: Introduction to Pattern Analysis
Lecture 9: Introduction to Pattern Analysis g Features, patterns and classifiers g Components of a PR system g An example g Probability definitions g Bayes Theorem g Gaussian densities Features, patterns
More informationLecture 21. The Multivariate Normal Distribution
Lecture. The Multivariate Normal Distribution. Definitions and Comments The joint momentgenerating function of X,...,X n [also called the momentgenerating function of the random vector (X,...,X n )]
More informationStatistical Foundations: Measures of Location and Central Tendency and Summation and Expectation
Statistical Foundations: and Central Tendency and and Lecture 4 September 5, 2006 Psychology 790 Lecture #49/05/2006 Slide 1 of 26 Today s Lecture Today s Lecture Where this Fits central tendency/location
More information2. Default correlation. Correlation of defaults of a pair of risky assets
2. Default correlation Correlation of defaults of a pair of risky assets Consider two obligors A and B and a fixed time horizon T. p A = probability of default of A before T p B = probability of default
More informationECE302 Spring 2006 HW7 Solutions March 11, 2006 1
ECE32 Spring 26 HW7 Solutions March, 26 Solutions to HW7 Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where
More informationInstitute of Actuaries of India Subject CT3 Probability and Mathematical Statistics
Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in
More informationStatistics in Psychosocial Research Lecture 8 Factor Analysis I. Lecturer: Elizabeth GarrettMayer
This work is licensed under a Creative Commons AttributionNonCommercialShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More information3.1 Least squares in matrix form
118 3 Multiple Regression 3.1 Least squares in matrix form E Uses Appendix A.2 A.4, A.6, A.7. 3.1.1 Introduction More than one explanatory variable In the foregoing chapter we considered the simple regression
More informationJointly Distributed Random Variables
Jointly Distributed Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Jointly Distributed Random Variables 1 1.1 Definition......................................... 1 1.2 Joint cdfs..........................................
More informationMultivariate Statistical Inference and Applications
Multivariate Statistical Inference and Applications ALVIN C. RENCHER Department of Statistics Brigham Young University A WileyInterscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim
More informationWhat is Statistics? Lecture 1. Introduction and probability review. Idea of parametric inference
0. 1. Introduction and probability review 1.1. What is Statistics? What is Statistics? Lecture 1. Introduction and probability review There are many definitions: I will use A set of principle and procedures
More informationOverview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationCS395T Computational Statistics with Application to Bioinformatics
CS395T Computational Statistics with Application to Bioinformatics Prof. William H. Press Spring Term, 2010 The University of Texas at Austin Unit 6: Multivariate Normal Distributions and Chi Square The
More informationThe Bivariate Normal Distribution
The Bivariate Normal Distribution This is Section 4.7 of the st edition (2002) of the book Introduction to Probability, by D. P. Bertsekas and J. N. Tsitsiklis. The material in this section was not included
More informationVariance Reduction. Pricing American Options. Monte Carlo Option Pricing. Delta and Common Random Numbers
Variance Reduction The statistical efficiency of Monte Carlo simulation can be measured by the variance of its output If this variance can be lowered without changing the expected value, fewer replications
More informationELECE8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems
Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed
More informationPITFALLS IN TIME SERIES ANALYSIS. Cliff Hurvich Stern School, NYU
PITFALLS IN TIME SERIES ANALYSIS Cliff Hurvich Stern School, NYU The t Test If x 1,..., x n are independent and identically distributed with mean 0, and n is not too small, then t = x 0 s n has a standard
More informationMath 141. Lecture 7: Variance, Covariance, and Sums. Albyn Jones 1. 1 Library 304. jones/courses/141
Math 141 Lecture 7: Variance, Covariance, and Sums Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141 Last Time Variance: expected squared deviation from the mean: Standard
More informationDepartment of Economics
Department of Economics On Testing for Diagonality of Large Dimensional Covariance Matrices George Kapetanios Working Paper No. 526 October 2004 ISSN 14730278 On Testing for Diagonality of Large Dimensional
More informationi=1 In practice, the natural logarithm of the likelihood function, called the loglikelihood function and denoted by
Statistics 580 Maximum Likelihood Estimation Introduction Let y (y 1, y 2,..., y n be a vector of iid, random variables from one of a family of distributions on R n and indexed by a pdimensional parameter
More informationSection 3 Part 1. Relationships between two numerical variables
Section 3 Part 1 Relationships between two numerical variables 1 Relationship between two variables The summary statistics covered in the previous lessons are appropriate for describing a single variable.
More informationOn Mardia s Tests of Multinormality
On Mardia s Tests of Multinormality Kankainen, A., Taskinen, S., Oja, H. Abstract. Classical multivariate analysis is based on the assumption that the data come from a multivariate normal distribution.
More informationChapter 4. Multivariate Distributions
1 Chapter 4. Multivariate Distributions Joint p.m.f. (p.d.f.) Independent Random Variables Covariance and Correlation Coefficient Expectation and Covariance Matrix Multivariate (Normal) Distributions Matlab
More informationFactor Analysis. Factor Analysis
Factor Analysis Principal Components Analysis, e.g. of stock price movements, sometimes suggests that several variables may be responding to a small number of underlying forces. In the factor model, we
More informationRegression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur. Lecture  2 Simple Linear Regression
Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur Lecture  2 Simple Linear Regression Hi, this is my second lecture in module one and on simple
More informationProbability and Statistics
CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b  0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute  Systems and Modeling GIGA  Bioinformatics ULg kristel.vansteen@ulg.ac.be
More informationUniversity of Ljubljana Doctoral Programme in Statistics Methodology of Statistical Research Written examination February 14 th, 2014.
University of Ljubljana Doctoral Programme in Statistics ethodology of Statistical Research Written examination February 14 th, 2014 Name and surname: ID number: Instructions Read carefully the wording
More informationMaster s Theory Exam Spring 2006
Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem
More informationSpearman s correlation
Spearman s correlation Introduction Before learning about Spearman s correllation it is important to understand Pearson s correlation which is a statistical measure of the strength of a linear relationship
More informationChapter 5: Joint Probability Distributions. Chapter Learning Objectives. The Joint Probability Distribution for a Pair of Discrete Random
Chapter 5: Joint Probability Distributions 51 Two or More Random Variables 51.1 Joint Probability Distributions 51.2 Marginal Probability Distributions 51.3 Conditional Probability Distributions 51.4
More informationA Tutorial on Probability Theory
Paola Sebastiani Department of Mathematics and Statistics University of Massachusetts at Amherst Corresponding Author: Paola Sebastiani. Department of Mathematics and Statistics, University of Massachusetts,
More informationDefinition: Suppose that two random variables, either continuous or discrete, X and Y have joint density
HW MATH 461/561 Lecture Notes 15 1 Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density and marginal densities f(x, y), (x, y) Λ X,Y f X (x), x Λ X,
More informationData visualization and clustering. Genomics is to no small extend a data science
Data visualization and clustering Genomics is to no small extend a data science [www.data2discovery.org] Data visualization and clustering Genomics is to no small extend a data science [Andersson et al.,
More informationExamination 110 Probability and Statistics Examination
Examination 0 Probability and Statistics Examination Sample Examination Questions The Probability and Statistics Examination consists of 5 multiplechoice test questions. The test is a threehour examination
More informationEigenvalues, Eigenvectors, Matrix Factoring, and Principal Components
Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they
More informationModule 3: Correlation and Covariance
Using Statistical Data to Make Decisions Module 3: Correlation and Covariance Tom Ilvento Dr. Mugdim Pašiƒ University of Delaware Sarajevo Graduate School of Business O ften our interest in data analysis
More informationBNG 202 Biomechanics Lab. Descriptive statistics and probability distributions I
BNG 202 Biomechanics Lab Descriptive statistics and probability distributions I Overview The overall goal of this short course in statistics is to provide an introduction to descriptive and inferential
More informationGeostatistics Exploratory Analysis
Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa Master of Science in Geospatial Technologies Geostatistics Exploratory Analysis Carlos Alberto Felgueiras cfelgueiras@isegi.unl.pt
More informationNumerical Summarization of Data OPRE 6301
Numerical Summarization of Data OPRE 6301 Motivation... In the previous session, we used graphical techniques to describe data. For example: While this histogram provides useful insight, other interesting
More informationWeek 5: Multiple Linear Regression
BUS41100 Applied Regression Analysis Week 5: Multiple Linear Regression Parameter estimation and inference, forecasting, diagnostics, dummy variables Robert B. Gramacy The University of Chicago Booth School
More information1.5. Factorisation. Introduction. Prerequisites. Learning Outcomes. Learning Style
Factorisation 1.5 Introduction In Block 4 we showed the way in which brackets were removed from algebraic expressions. Factorisation, which can be considered as the reverse of this process, is dealt with
More informationSpectral Measure of Large Random Toeplitz Matrices
Spectral Measure of Large Random Toeplitz Matrices Yongwhan Lim June 5, 2012 Definition (Toepliz Matrix) The symmetric Toeplitz matrix is defined to be [X i j ] where 1 i, j n; that is, X 0 X 1 X 2 X n
More informationThe Scalar Algebra of Means, Covariances, and Correlations
3 The Scalar Algebra of Means, Covariances, and Correlations In this chapter, we review the definitions of some key statistical concepts: means, covariances, and correlations. We show how the means, variances,
More informationRegression Analysis. Regression Analysis MIT 18.S096. Dr. Kempthorne. Fall 2013
Lecture 6: Regression Analysis MIT 18.S096 Dr. Kempthorne Fall 2013 MIT 18.S096 Regression Analysis 1 Outline Regression Analysis 1 Regression Analysis MIT 18.S096 Regression Analysis 2 Multiple Linear
More information