Sections 2.11 and 5.8
|
|
- Diane Burke
- 8 years ago
- Views:
Transcription
1 Sections 211 and 58 Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1/25
2 Gesell data Let X be the age in in months a child speaks his/her first word and let Y be the Gesell adaptive score, a measure of a child s aptitude (observed later on) Are X and Y related? How does the child s aptitude change with how long it takes them to speak? Here s the Gesell score y i and age at first word in months x i data, i = 1,,21 x i y i x i y i x i y i x i y i x i y i In R, we compute r = 0640, a moderately strong negative relationship between age at first word spoken and Gesell score > age=c(15,26,10,9,15,20,18,11,8,20,7,9,10,11,11,10,12,42,17,11,10) > Gesell=c(95,71,83,91,102,87,93,100,104,94,113,96,83,84,102,100,105,57,121,86,100) > plot(age,gesell) > cor(age,gesell) [1] /25
3 Scatterplot of (x 1,y 1 ),,(x 21,y 21 ) Gesell age 3/25
4 Random vectors A random vector X = variables X 1 X 2 X k is made up of, say, k random A random vector has a joint distribution, eg a density f(x), that gives probabilities P(X A) = f(x)dx Just as a random variable X has a mean E(X) and variance var(x), a random vector also has a mean vector E(X) and a covariance matrix cov(x) A 4/25
5 Mean vector & covariance matrix Let X = (X 1,,X k ) be a random vector with density f(x 1,,x k ) The mean of X is the vector of marginal means X 1 E(X 1 ) X 2 E(X) = E = E(X 2 ) (538) X k E(X k ) The covariance matrix of X is given by cov(x 1,X 1 ) cov(x 1,X 2 ) cov(x 1,X k ) cov(x 2,X 1 ) cov(x 2,X 2 ) cov(x 2,X k ) cov(x) = cov(x k,x 1 ) cov(x k,x 2 ) cov(x k,x k ) (542) 5/25
6 Multivariate normal distribution The normal distribution generalizes to multiple dimensions We ll first look at two jointly distributed normal random variables, then discuss three or more The bivariate normal density for (X 1,X 2 ) is given by f(x 1,x 2 ) = { [ (x1 ) 1 2πσ 1 σ 2 1 ρ 2 exp 1 µ 2 ( )( ) ( ) ]} 1 x1 µ 1 x2 µ 2 x2 µ 2 2 2(1 ρ 2 2ρ + ) σ 1 σ 1 σ 2 σ 2 There are 5 parameters: (µ 1,µ 2,σ 1,σ 2,ρ) Besides 58, also see 211 pp /25
7 Bivariate normal distribution This density jointly defines X 1 and X 2, which live in R 2 = (, ) (, ) Marginally, X 1 N(µ 1,σ1 2) and X 2 N(µ 2,σ2 2 ) (p 79) The correlation between X 1 and X 2 is given by corr(x 1,X 2 ) = ρ (p 80) For jointly normal random variables, if the correlation is zero then they are independent This is not true in general for jointly defined random variables E(X) = [ µ1 ], cov(x) = [ σ 2 1 σ 1 σ 2 ρ µ 2 σ 1 σ 2 ρ σ2 2 Next slide: µ 1 = 0,1; µ 2 = 0,2; σ1 2 = σ2 2 = 1; ρ = 0,09, 06 ] 7/25
8 Bivariate normal PDF level curves /25
9 Proof that X 1 indpendent X 2 when ρ = 0 When ρ = 0 the joint density for (X 1,X 2 ) simplifies to { [ 1 f(x 1,x 2 ) = exp 1 (x1 ) µ 2 ( ) ]} 1 x2 µ πσ 1 σ 2 2 σ 1 σ 2 [ ( ) 1 = e 05 x1 µ 2 ][ ( ) 1 1 σ 1 e 05 x2 µ ] σ 2 2πσ1 2πσ2 Since these are each respectively functions of x 1 and x 2 only, and the range of (X 1,X 2 ) factors into the produce of two sets, X 1 and X 2 are independent and in fact X 1 N(µ 1,σ 2 1 ) and X 2 N(µ 2,σ 2 2 ) 9/25
10 Conditional distributions [X 1 X 2 = x 2 ] and [X 2 X 1 = x 1 ] (pp 80 81) The conditional distribution of X 1 given X 2 = x 2 is ( [X 1 X 2 = x 2 ] N µ 1 + σ ) 1 ρ(x 2 µ 2 ),σ 2 σ 1(1 ρ 2 ) 2 Similarly, [X 2 X 1 = x 1 ] N This ties directly to linear regression: ( µ 2 + σ ) 2 ρ(x 1 µ 1 ),σ 2 σ 2(1 ρ 2 ) 1 To predict X 2 X 1 = x 1, we have [ E(X 2 X 1 = x 1 ) = µ 2 σ ] [ ] 2 σ2 ρµ 1 + ρ x 1 = β 0 +β 1 x 1 σ 1 σ 1 10/25
11 Bivariate normal distribution as data model Here we assume or succinctly, [ Xi1 X i2 ] iid N2 ([ µ1 µ 2 ], X i iid N2 (µ,σ) [ ]) σ11 σ 12, σ 21 σ 22 If the bivariate normal model is appropriate for paired outcomes, it provides a convenient probability model with some nice properties The sample mean X = 1 n n i=1 X i is the MLE of µ and the sample covariance matrix S = 1 n n 1 i=1 (X i X)(X i X) is unbiased for Σ It can be shown that X N 2 ( µ, 1 n Σ ) The matrix (n 1)S has a Wishart distribution (generalizes χ 2 ) 11/25
12 Sample mean vector & covariance matrix Say n outcome pairs are to be recorded: {(X 11,X 12 ),(X 21,X 22 ),,(X n1,x n2 )} The i th pair is (X i1,x i2 ) The sample mean vector is given elementwise by X = [ X1 X 2 ] = [ 1 n n i=1 X i1 1 n n i=1 X i2 and the sample covariance matrix is given elementwise by [ 1 n S = n 1 i=1 (X i1 X 1 ) 2 1 n n 1 i=1 (X ] i1 X 1 )(X i2 X 2 ) 1 n n 1 i=1 (X i1 X 1 )(X i2 X 1 n 2 ) n 1 i=1 (X i2 X 2 ) 2 ], 12/25
13 Estimation The sample mean vector X estimates µ = [ µ1 µ 2 ] and the sample covariance [ matrix S estimates ] [ ] σ1 Σ = 2 ρσ 1 σ 2 σ11 σ ρσ 1 σ 2 σ2 2 = 12 σ 21 σ 22 We will place hats on parameter estimators based on the data So ˆµ 1 = X 1, ˆµ 2 = X 2, ˆσ 2 1 = s 2 1 = 1 n 1 n (X i1 X 1 ) 2, i=1 Also, ˆσ 2 2 = s 2 2 = 1 n 1 ĉov(x 1,X 2 ) = 1 n 1 n (X i2 X 2 ) 2 i=1 n (X i1 X)(X i2 X 2 ) i=1 13/25
14 Correlation coefficient r So a natural estimate of ρ is then ˆρ = ĉov(x 1,X 2 ) ˆσ 1ˆσ 2 = 1 n 1 1 n n 1 i=1 (X i1 X 1 )(X i2 X 2 ) n i=1 (X i1 X 1 ) 2 n i=1 (X i1 X 1 ) 2 1 n 1 This is in fact the MLE estimate based on the bivariate normal model It is also a plug-in estimator based on the method-of-moments too It is commonly referred to as the Pearson correlation coefficient You can get it as, eg, cor(age,gesell) in R This estimate of correlation can be unduly influenced by outliers in the sample An alternative measure of linear association is the Spearman correlation based on ranks, discussed a few lectures ago > cor(age,gesell) [1] > cor(age,gesell,method="spearman") [1] /25
15 Gesell data Recall: X is age in months a child speaks his/her first word and let Y is Gesell adaptive score, a measure of a child s aptitude Question: how does the child s aptitude change with how long it takes them to speak? [ Here, ] n = 21 [ ] In R we find X = Also, S = Assuming a bivariate model, we plug in the estimates and obtain the estimated PDF for (X,Y): f(x,y) = exp( x 00134x y 00098xy 00043y 2 ) We can further find from Y N(9367,18632), f Y (y) = exp( (y 9367) 2 ) 15/25
16 3D plot of f(x,y) for (X,Y) estimated from data /25
17 Gesell conditional distribution Solid is f Y (y); left dashed is f Y X (y 25) the right dashed is f Y X (y 10) As the age in months of first words X = x increases, the distribution of Gesell Adaptive Scores Y decreases 17/25
18 Density estimate with actual data Gesell age 18/25
19 Multivariate normal distribution In general, a k-variate normal is defined through the mean and covariance matrix: X 1 µ 1 σ 11 σ 12 σ 1k X 2 N µ 2 k, σ 21 σ 22 σ 2k X k µ k σ k1 σ k2 σ kk Succinctly, X N k (µ,σ) Recall that if Z N(0,1), then X = µ+σz N(µ,σ 2 ) The definition of the multivariate normal distribution just extends this idea 19/25
20 Multivariate normal made from independent normals Instead of one standard normal, we have a list of k independent standard normals Z = (Z 1,,Z k ), and consider the same sort of transformation in the multivariate case using matrices and vectors Let Z 1,,Z k iid N(0,1) The joint pdf of (Z1,,Z k ) is given by Let µ = f(z 1,,z k ) = µ 1 µ 2 µ k k exp( 05zi 2 )/ 2π i=1 σ 11 σ 12 σ 1k and Σ = σ 21 σ 22 σ 2k, σ k1 σ k2 σ kk where Σ is symmetric (ie Σ = Σ, which implies σ ij = σ ji for all 1 i,j k) 20/25
21 Multivariate normal made from independent normals Let Σ 1/2 be any matrix such that Σ 1/2 Σ 1/2 = Σ Then X = µ+σ 1/2 Z is said to have a multivariate normal distribution with mean vector µ and covariance matrix Σ, written Written in terms of matrices X N k (µ,σ) X = X 1 X 2 X k = µ 1 µ 2 µ k + σ 11 σ 12 σ 1k σ 21 σ 22 σ 2k σ k1 σ k2 σ kk 1/2 Z 1 Z 2 Z k 21/25
22 Joint PDF Using some math, it can be shown that the pdf of the new vector X = (X 1,,X k ) is given by f(x 1,,x k µ,σ) = 2πΣ 1/2 exp{ 05(x µ) Σ 1 (x µ)} In the one-dimensional case, this simplifies to our old friend f(x 1 µ,σ 2 ) = (2πσ 2 ) 1/2 exp{ 05(x µ)(σ 2 ) 1 (x µ)}, the pdf of a N(µ,σ 2 ) random variable X A is the determinant of the matrix A, and is a function of the elements of A, but beyond this course 22/25
23 Properties of multivariate normal vectors Let Then X N k (µ,σ) 1 For each X i in X = (X 1,,X k ), E(X i ) = µ i and var(x i ) = σ ii That is, marginally, X i N(µ i,σ ii ) 2 For any r k matrix M, MX N r (Mµ,MΣM ) 3 For any two (X i,x j ) where 1 i < j k, cov(x i,x j ) = σ ij The off-diagonal elements of Σ give the covariance between two elements of (X 1,,X k ) Note then ρ(x i,x j ) = σ ij / σ ii σ jj 4 For any k 1 vector m = (m 1,,m k ) and Y N k (µ,σ), m+y N k (m+µ,σ) 23/25
24 Example Let Define X 1 X 2 N 3 2 5, X [ ] [ ] Y1 M = and Y = = MX = Y 2 [ ] X 1 X 2 X 3 Then X 2 N(5,3), cov(x 2,X 3 ) = 1 and [ ] X 1 X X 3 [ ] N [ ] , or simplifying, [ Y1 Y 2 ] = [ ] X 1 X 2 X 3 N 2 ([ 2 1 ] [ 4 0, Note that for the transformed vector Y = (Y 1,Y 2 ), cov(y 1,Y 2 ) = 0 and therefore Y 1 and Y 2 are uncorrelated, ie ρ(y 1,Y 2 ) = 0 ]), 24/25
25 Simple linear regression For the linear model (eg simple linear regression or the two-sample model) Y = Xβ +ǫ, the error vector is assumed (pp ) ǫ N n (0,I n n σ 2 ) Then the least squares estimators have a multivariate normal distribution β N p (β,(x X) 1 σ 2 ) p = 2 is the number of mean parameters (The MSE has a gamma distribution) We ll discuss this shortly! 25/25
Multivariate Normal Distribution
Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues
More informationCS229 Lecture notes. Andrew Ng
CS229 Lecture notes Andrew Ng Part X Factor analysis Whenwehavedatax (i) R n thatcomesfromamixtureofseveral Gaussians, the EM algorithm can be applied to fit a mixture model. In this setting, we usually
More informationLecture Notes 1. Brief Review of Basic Probability
Probability Review Lecture Notes Brief Review of Basic Probability I assume you know basic probability. Chapters -3 are a review. I will assume you have read and understood Chapters -3. Here is a very
More informationStat 704 Data Analysis I Probability Review
1 / 30 Stat 704 Data Analysis I Probability Review Timothy Hanson Department of Statistics, University of South Carolina Course information 2 / 30 Logistics: Tuesday/Thursday 11:40am to 12:55pm in LeConte
More information15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
More informationSF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,
More informationSF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2014 Timo Koski () Mathematisk statistik 24.09.2014 1 / 75 Learning outcomes Random vectors, mean vector, covariance
More informationExample: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C
More informationChapter 6: Point Estimation. Fall 2011. - Probability & Statistics
STAT355 Chapter 6: Point Estimation Fall 2011 Chapter Fall 2011 6: Point1 Estimat / 18 Chap 6 - Point Estimation 1 6.1 Some general Concepts of Point Estimation Point Estimate Unbiasedness Principle of
More informationThe Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables
Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh The Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables 1 The Monte Carlo Framework Suppose we wish
More informationSlides for Risk Management
Slides for Risk Management VaR and Expected Shortfall Groll Seminar für Finanzökonometrie Prof. Mittnik, PhD 1 Introduction Value-at-Risk Expected Shortfall Model risk Multi-period / multi-asset case 2
More informationSlides for Risk Management VaR and Expected Shortfall
Slides for Risk Management VaR and Expected Shortfall Groll Seminar für Finanzökonometrie Prof. Mittnik, PhD Groll (Seminar für Finanzökonometrie) Slides for Risk Management Prof. Mittnik, PhD 1 / 133
More informationQuadratic forms Cochran s theorem, degrees of freedom, and all that
Quadratic forms Cochran s theorem, degrees of freedom, and all that Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 1, Slide 1 Why We Care Cochran s theorem tells us
More informationExploratory Factor Analysis and Principal Components. Pekka Malo & Anton Frantsev 30E00500 Quantitative Empirical Research Spring 2016
and Principal Components Pekka Malo & Anton Frantsev 30E00500 Quantitative Empirical Research Spring 2016 Agenda Brief History and Introductory Example Factor Model Factor Equation Estimation of Loadings
More informationSummary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)
Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationJoint Exam 1/P Sample Exam 1
Joint Exam 1/P Sample Exam 1 Take this practice exam under strict exam conditions: Set a timer for 3 hours; Do not stop the timer for restroom breaks; Do not look at your notes. If you believe a question
More informationMultivariate Analysis (Slides 13)
Multivariate Analysis (Slides 13) The final topic we consider is Factor Analysis. A Factor Analysis is a mathematical approach for attempting to explain the correlation between a large set of variables
More informationIntroduction to Multivariate Analysis
Introduction to Multivariate Analysis Lecture 1 August 24, 2005 Multivariate Analysis Lecture #1-8/24/2005 Slide 1 of 30 Today s Lecture Today s Lecture Syllabus and course overview Chapter 1 (a brief
More informationCorrelation in Random Variables
Correlation in Random Variables Lecture 11 Spring 2002 Correlation in Random Variables Suppose that an experiment produces two random variables, X and Y. What can we say about the relationship between
More informationA Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution
A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 4: September
More informationFactor analysis. Angela Montanari
Factor analysis Angela Montanari 1 Introduction Factor analysis is a statistical model that allows to explain the correlations between a large number of observed correlated variables through a small number
More informationDepartment of Mathematics, Indian Institute of Technology, Kharagpur Assignment 2-3, Probability and Statistics, March 2015. Due:-March 25, 2015.
Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment -3, Probability and Statistics, March 05. Due:-March 5, 05.. Show that the function 0 for x < x+ F (x) = 4 for x < for x
More informationMULTIVARIATE PROBABILITY DISTRIBUTIONS
MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined
More informationSome probability and statistics
Appendix A Some probability and statistics A Probabilities, random variables and their distribution We summarize a few of the basic concepts of random variables, usually denoted by capital letters, X,Y,
More informationMultivariate Normal Distribution Rebecca Jennings, Mary Wakeman-Linn, Xin Zhao November 11, 2010
Multivariate Normal Distribution Rebecca Jeings, Mary Wakeman-Li, Xin Zhao November, 00. Basics. Parameters We say X ~ N n (µ, ) with parameters µ = [E[X ],.E[X n ]] and = Cov[X i X j ] i=..n, j= n. The
More informationMultivariate normal distribution and testing for means (see MKB Ch 3)
Multivariate normal distribution and testing for means (see MKB Ch 3) Where are we going? 2 One-sample t-test (univariate).................................................. 3 Two-sample t-test (univariate).................................................
More informationStatistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
More informationLecture 8: Signal Detection and Noise Assumption
ECE 83 Fall Statistical Signal Processing instructor: R. Nowak, scribe: Feng Ju Lecture 8: Signal Detection and Noise Assumption Signal Detection : X = W H : X = S + W where W N(, σ I n n and S = [s, s,...,
More informationCovariance and Correlation
Covariance and Correlation ( c Robert J. Serfling Not for reproduction or distribution) We have seen how to summarize a data-based relative frequency distribution by measures of location and spread, such
More informationLecture 9: Introduction to Pattern Analysis
Lecture 9: Introduction to Pattern Analysis g Features, patterns and classifiers g Components of a PR system g An example g Probability definitions g Bayes Theorem g Gaussian densities Features, patterns
More informationLecture 21. The Multivariate Normal Distribution
Lecture. The Multivariate Normal Distribution. Definitions and Comments The joint moment-generating function of X,...,X n [also called the moment-generating function of the random vector (X,...,X n )]
More informationAn Introduction to Machine Learning
An Introduction to Machine Learning L5: Novelty Detection and Regression Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia Alex.Smola@nicta.com.au Tata Institute, Pune,
More informationStatistics in Psychosocial Research Lecture 8 Factor Analysis I. Lecturer: Elizabeth Garrett-Mayer
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationCorrelation key concepts:
CORRELATION Correlation key concepts: Types of correlation Methods of studying correlation a) Scatter diagram b) Karl pearson s coefficient of correlation c) Spearman s Rank correlation coefficient d)
More informationMultivariate Statistical Inference and Applications
Multivariate Statistical Inference and Applications ALVIN C. RENCHER Department of Statistics Brigham Young University A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim
More informationLeast Squares Estimation
Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David
More informationThe Bivariate Normal Distribution
The Bivariate Normal Distribution This is Section 4.7 of the st edition (2002) of the book Introduction to Probability, by D. P. Bertsekas and J. N. Tsitsiklis. The material in this section was not included
More informationOverview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationLinear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear
More informationSpatial Statistics Chapter 3 Basics of areal data and areal data modeling
Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Recall areal data also known as lattice data are data Y (s), s D where D is a discrete index set. This usually corresponds to data
More informationi=1 In practice, the natural logarithm of the likelihood function, called the log-likelihood function and denoted by
Statistics 580 Maximum Likelihood Estimation Introduction Let y (y 1, y 2,..., y n be a vector of iid, random variables from one of a family of distributions on R n and indexed by a p-dimensional parameter
More informationSection 3 Part 1. Relationships between two numerical variables
Section 3 Part 1 Relationships between two numerical variables 1 Relationship between two variables The summary statistics covered in the previous lessons are appropriate for describing a single variable.
More informationMaster s Theory Exam Spring 2006
Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem
More informationInstitute of Actuaries of India Subject CT3 Probability and Mathematical Statistics
Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in
More information3.1 Least squares in matrix form
118 3 Multiple Regression 3.1 Least squares in matrix form E Uses Appendix A.2 A.4, A.6, A.7. 3.1.1 Introduction More than one explanatory variable In the foregoing chapter we considered the simple regression
More informationA Tutorial on Probability Theory
Paola Sebastiani Department of Mathematics and Statistics University of Massachusetts at Amherst Corresponding Author: Paola Sebastiani. Department of Mathematics and Statistics, University of Massachusetts,
More informationFactor Analysis. Factor Analysis
Factor Analysis Principal Components Analysis, e.g. of stock price movements, sometimes suggests that several variables may be responding to a small number of underlying forces. In the factor model, we
More informationUniversity of Ljubljana Doctoral Programme in Statistics Methodology of Statistical Research Written examination February 14 th, 2014.
University of Ljubljana Doctoral Programme in Statistics ethodology of Statistical Research Written examination February 14 th, 2014 Name and surname: ID number: Instructions Read carefully the wording
More informationDefinition: Suppose that two random variables, either continuous or discrete, X and Y have joint density
HW MATH 461/561 Lecture Notes 15 1 Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density and marginal densities f(x, y), (x, y) Λ X,Y f X (x), x Λ X,
More informationDepartment of Economics
Department of Economics On Testing for Diagonality of Large Dimensional Covariance Matrices George Kapetanios Working Paper No. 526 October 2004 ISSN 1473-0278 On Testing for Diagonality of Large Dimensional
More informationOn Mardia s Tests of Multinormality
On Mardia s Tests of Multinormality Kankainen, A., Taskinen, S., Oja, H. Abstract. Classical multivariate analysis is based on the assumption that the data come from a multivariate normal distribution.
More informationModule 3: Correlation and Covariance
Using Statistical Data to Make Decisions Module 3: Correlation and Covariance Tom Ilvento Dr. Mugdim Pašiƒ University of Delaware Sarajevo Graduate School of Business O ften our interest in data analysis
More informationRegression Analysis. Regression Analysis MIT 18.S096. Dr. Kempthorne. Fall 2013
Lecture 6: Regression Analysis MIT 18.S096 Dr. Kempthorne Fall 2013 MIT 18.S096 Regression Analysis 1 Outline Regression Analysis 1 Regression Analysis MIT 18.S096 Regression Analysis 2 Multiple Linear
More informationHow To Find The Correlation Of Random Bits With The Xor Operator
Exclusive OR (XOR) and hardware random number generators Robert B Davies February 28, 2002 1 Introduction The exclusive or (XOR) operation is commonly used to reduce the bias from the bits generated by
More informationWhat is Statistics? Lecture 1. Introduction and probability review. Idea of parametric inference
0. 1. Introduction and probability review 1.1. What is Statistics? What is Statistics? Lecture 1. Introduction and probability review There are many definitions: I will use A set of principle and procedures
More informationVariance Reduction. Pricing American Options. Monte Carlo Option Pricing. Delta and Common Random Numbers
Variance Reduction The statistical efficiency of Monte Carlo simulation can be measured by the variance of its output If this variance can be lowered without changing the expected value, fewer replications
More informationInner Product Spaces
Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and
More informationOverview of Monte Carlo Simulation, Probability Review and Introduction to Matlab
Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab 1 Overview of Monte Carlo Simulation 1.1 Why use simulation?
More informationGeostatistics Exploratory Analysis
Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa Master of Science in Geospatial Technologies Geostatistics Exploratory Analysis Carlos Alberto Felgueiras cfelgueiras@isegi.unl.pt
More informationOutline. Topic 4 - Analysis of Variance Approach to Regression. Partitioning Sums of Squares. Total Sum of Squares. Partitioning sums of squares
Topic 4 - Analysis of Variance Approach to Regression Outline Partitioning sums of squares Degrees of freedom Expected mean squares General linear test - Fall 2013 R 2 and the coefficient of correlation
More informationAutocovariance and Autocorrelation
Chapter 3 Autocovariance and Autocorrelation If the {X n } process is weakly stationary, the covariance of X n and X n+k depends only on the lag k. This leads to the following definition of the autocovariance
More informationWeek 5: Multiple Linear Regression
BUS41100 Applied Regression Analysis Week 5: Multiple Linear Regression Parameter estimation and inference, forecasting, diagnostics, dummy variables Robert B. Gramacy The University of Chicago Booth School
More informationproblem arises when only a non-random sample is available differs from censored regression model in that x i is also unobserved
4 Data Issues 4.1 Truncated Regression population model y i = x i β + ε i, ε i N(0, σ 2 ) given a random sample, {y i, x i } N i=1, then OLS is consistent and efficient problem arises when only a non-random
More informationLogistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression
Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max
More informationPrincipal Component Analysis
Principal Component Analysis ERS70D George Fernandez INTRODUCTION Analysis of multivariate data plays a key role in data analysis. Multivariate data consists of many different attributes or variables recorded
More informationCorrelation Coefficient The correlation coefficient is a summary statistic that describes the linear relationship between two numerical variables 2
Lesson 4 Part 1 Relationships between two numerical variables 1 Correlation Coefficient The correlation coefficient is a summary statistic that describes the linear relationship between two numerical variables
More informationPITFALLS IN TIME SERIES ANALYSIS. Cliff Hurvich Stern School, NYU
PITFALLS IN TIME SERIES ANALYSIS Cliff Hurvich Stern School, NYU The t -Test If x 1,..., x n are independent and identically distributed with mean 0, and n is not too small, then t = x 0 s n has a standard
More informationLinear Regression. Chapter 5. Prediction via Regression Line Number of new birds and Percent returning. Least Squares
Linear Regression Chapter 5 Regression Objective: To quantify the linear relationship between an explanatory variable (x) and response variable (y). We can then predict the average response for all subjects
More informationEconometrics Simple Linear Regression
Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight
More informationSTAT 830 Convergence in Distribution
STAT 830 Convergence in Distribution Richard Lockhart Simon Fraser University STAT 830 Fall 2011 Richard Lockhart (Simon Fraser University) STAT 830 Convergence in Distribution STAT 830 Fall 2011 1 / 31
More informationNew Methods Providing High Degree Polynomials with Small Mahler Measure
New Methods Providing High Degree Polynomials with Small Mahler Measure G. Rhin and J.-M. Sac-Épée CONTENTS 1. Introduction 2. A Statistical Method 3. A Minimization Method 4. Conclusion and Prospects
More informationEigenvalues, Eigenvectors, Matrix Factoring, and Principal Components
Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they
More informationExploratory Factor Analysis
Exploratory Factor Analysis Definition Exploratory factor analysis (EFA) is a procedure for learning the extent to which k observed variables might measure m abstract variables, wherein m is less than
More informationHomework 11. Part 1. Name: Score: / null
Name: Score: / Homework 11 Part 1 null 1 For which of the following correlations would the data points be clustered most closely around a straight line? A. r = 0.50 B. r = -0.80 C. r = 0.10 D. There is
More informationCONDITIONAL, PARTIAL AND RANK CORRELATION FOR THE ELLIPTICAL COPULA; DEPENDENCE MODELLING IN UNCERTAINTY ANALYSIS
CONDITIONAL, PARTIAL AND RANK CORRELATION FOR THE ELLIPTICAL COPULA; DEPENDENCE MODELLING IN UNCERTAINTY ANALYSIS D. Kurowicka, R.M. Cooke Delft University of Technology, Mekelweg 4, 68CD Delft, Netherlands
More informationPrinciple Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression
Principle Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression Saikat Maitra and Jun Yan Abstract: Dimension reduction is one of the major tasks for multivariate
More informationIntroduction: Overview of Kernel Methods
Introduction: Overview of Kernel Methods Statistical Data Analysis with Positive Definite Kernels Kenji Fukumizu Institute of Statistical Mathematics, ROIS Department of Statistical Science, Graduate University
More informationLinear Classification. Volker Tresp Summer 2015
Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong
More information1.5. Factorisation. Introduction. Prerequisites. Learning Outcomes. Learning Style
Factorisation 1.5 Introduction In Block 4 we showed the way in which brackets were removed from algebraic expressions. Factorisation, which can be considered as the reverse of this process, is dealt with
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby
More informationDimensionality Reduction: Principal Components Analysis
Dimensionality Reduction: Principal Components Analysis In data mining one often encounters situations where there are a large number of variables in the database. In such situations it is very likely
More informationChapter 13 Introduction to Nonlinear Regression( 非 線 性 迴 歸 )
Chapter 13 Introduction to Nonlinear Regression( 非 線 性 迴 歸 ) and Neural Networks( 類 神 經 網 路 ) 許 湘 伶 Applied Linear Regression Models (Kutner, Nachtsheim, Neter, Li) hsuhl (NUK) LR Chap 10 1 / 35 13 Examples
More informationDegrees of Freedom and Model Search
Degrees of Freedom and Model Search Ryan J. Tibshirani Abstract Degrees of freedom is a fundamental concept in statistical modeling, as it provides a quantitative description of the amount of fitting performed
More informationThe 3-Level HLM Model
James H. Steiger Department of Psychology and Human Development Vanderbilt University Regression Modeling, 2009 1 2 Basic Characteristics of the 3-level Model Level-1 Model Level-2 Model Level-3 Model
More informationCenter for Advanced Studies in Measurement and Assessment. CASMA Research Report
Center for Advanced Studies in Measurement and Assessment CASMA Research Report Number 13 and Accuracy Under the Compound Multinomial Model Won-Chan Lee November 2005 Revised April 2007 Revised April 2008
More informationAn extension of the factoring likelihood approach for non-monotone missing data
An extension of the factoring likelihood approach for non-monotone missing data Jae Kwang Kim Dong Wan Shin January 14, 2010 ABSTRACT We address the problem of parameter estimation in multivariate distributions
More informationChapter 2 Fundamentals of Statistical Analysis
Chapter Fundamentals of Statistical Analysis To make this book self-contained, this chapter will review relevant mathematical concepts used in this book. We first review basic probability and statistical
More informationCOMP6053 lecture: Relationship between two variables: correlation, covariance and r-squared. jn2@ecs.soton.ac.uk
COMP6053 lecture: Relationship between two variables: correlation, covariance and r-squared jn2@ecs.soton.ac.uk Relationships between variables So far we have looked at ways of characterizing the distribution
More informationQuestion 2: How do you solve a matrix equation using the matrix inverse?
Question : How do you solve a matrix equation using the matrix inverse? In the previous question, we wrote systems of equations as a matrix equation AX B. In this format, the matrix A contains the coefficients
More informationCORRELATION ANALYSIS
CORRELATION ANALYSIS Learning Objectives Understand how correlation can be used to demonstrate a relationship between two factors. Know how to perform a correlation analysis and calculate the coefficient
More informationBNG 202 Biomechanics Lab. Descriptive statistics and probability distributions I
BNG 202 Biomechanics Lab Descriptive statistics and probability distributions I Overview The overall goal of this short course in statistics is to provide an introduction to descriptive and inferential
More informationPUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.
PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include
More informationSTT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012)
STT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012) TA: Zhen (Alan) Zhang zhangz19@stt.msu.edu Office hour: (C500 WH) 1:45 2:45PM Tuesday (office tel.: 432-3342) Help-room: (A102 WH) 11:20AM-12:30PM,
More information1 Prior Probability and Posterior Probability
Math 541: Statistical Theory II Bayesian Approach to Parameter Estimation Lecturer: Songfeng Zheng 1 Prior Probability and Posterior Probability Consider now a problem of statistical inference in which
More informationChapter 9 Descriptive Statistics for Bivariate Data
9.1 Introduction 215 Chapter 9 Descriptive Statistics for Bivariate Data 9.1 Introduction We discussed univariate data description (methods used to eplore the distribution of the values of a single variable)
More informationStatistical Models in Data Mining
Statistical Models in Data Mining Sargur N. Srihari University at Buffalo The State University of New York Department of Computer Science and Engineering Department of Biostatistics 1 Srihari Flood of
More informationMaximum likelihood estimation of mean reverting processes
Maximum likelihood estimation of mean reverting processes José Carlos García Franco Onward, Inc. jcpollo@onwardinc.com Abstract Mean reverting processes are frequently used models in real options. For
More informationTopic 3. Chapter 5: Linear Regression in Matrix Form
Topic Overview Statistics 512: Applied Linear Models Topic 3 This topic will cover thinking in terms of matrices regression on multiple predictor variables case study: CS majors Text Example (NKNW 241)
More informationMATH4427 Notebook 2 Spring 2016. 2 MATH4427 Notebook 2 3. 2.1 Definitions and Examples... 3. 2.2 Performance Measures for Estimators...
MATH4427 Notebook 2 Spring 2016 prepared by Professor Jenny Baglivo c Copyright 2009-2016 by Jenny A. Baglivo. All Rights Reserved. Contents 2 MATH4427 Notebook 2 3 2.1 Definitions and Examples...................................
More information