Data Reduction I : PCA and Factor Analysis
|
|
|
- Reynold Bertram Beasley
- 7 years ago
- Views:
Transcription
1 Data Reduction I : PCA and Factor Analysis Carlos Pinto 1 1 Neuropsychiatric Genetics Group University of Dublin Data Analysis Seminars 11 November 2009
2 Overview 1. Distortions in Data 2. Variance and Covariance 3. PCA: A Toy Example 4. Running PCA in R 5. Factor Analysis 6. Running Factor Analysis in R 7. Conclusions
3 Sources of Confusion in Data 1. Noise 2. Rotation 3. Redundancy in linear systems...
4 Sources of Confusion in Data 1. Noise 2. Rotation 3. Redundancy in linear systems...
5 Sources of Confusion in Data 1. Noise 2. Rotation 3. Redundancy in linear systems...
6 Sources of Confusion in Data 1. Noise 2. Rotation 3. Redundancy in linear systems...
7 Noise 1. Noise arises from inaccuracies in our measurements 2. Noise is measured relative to the signal SNR = σ2 signal σ 2 noise The SNR must be large if we are to extract useful information from our experiment.
8 Noise 1. Noise arises from inaccuracies in our measurements 2. Noise is measured relative to the signal SNR = σ2 signal σ 2 noise The SNR must be large if we are to extract useful information from our experiment.
9 Noise 1. Noise arises from inaccuracies in our measurements 2. Noise is measured relative to the signal SNR = σ2 signal σ 2 noise The SNR must be large if we are to extract useful information from our experiment.
10 Noise 1. Noise arises from inaccuracies in our measurements 2. Noise is measured relative to the signal SNR = σ2 signal σ 2 noise The SNR must be large if we are to extract useful information from our experiment.
11 Variance in Signal and Noise
12 Rotation 1. We don t always know the true underlying variables in the system we are studying 2. We may actually be measuring some (hopefully linear!) combination of the true variables.
13 Rotation 1. We don t always know the true underlying variables in the system we are studying 2. We may actually be measuring some (hopefully linear!) combination of the true variables.
14 Redundancy 1. We don t always know whether the variables we are measuring are independent. 2. We may actually be measuring one or more (hopefully linear!) combinations of the same set of variables.
15 Redundancy 1. We don t always know whether the variables we are measuring are independent. 2. We may actually be measuring one or more (hopefully linear!) combinations of the same set of variables.
16 Degrees of Redundancy
17 An Illustrative Experiment
18 Goals of PCA 1. To isolate noise 2. To separate out the redundant degrees of freedom 3. To eliminate the effects of rotation
19 Goals of PCA 1. To isolate noise 2. To separate out the redundant degrees of freedom 3. To eliminate the effects of rotation
20 Goals of PCA 1. To isolate noise 2. To separate out the redundant degrees of freedom 3. To eliminate the effects of rotation
21 Example: Measuring Two Correlated Variables
22 Reduced Data
23 Variance Variable : X Measurements: {x 1... x n } Variable : Y Measurements: {y 1... y n } Variance of X: Variance of Y: σ 2 XX = P i (x i x)(x i x) n 1 σ 2 YY = P i (y i y)(y i y) n 1
24 Covariance Covariance of X and Y: σ 2 XY = i (x i x)(y i y) = σyx 2 n 1 The covariance measures the degree of the (linear!) relationship between X and Y Small values of the covariance indicate that the variables are independent (uncorrelated)
25 Covariance Matrix We can write the four variances associated with our two variables in the form of a matrix: ( ) CXX C XY C YX C YY The diagonal elements are the variances and the off diagonal elements are the covariances The covariance matrix is symmetric, since C XY = C YX
26 Three Variables For 3 variables, we get a 3 3 matrix: C 11 C 12 C 13 C 21 C 22 C 23 C 31 C 32 C 33 As before, the diagonal elements are the variances and the off diagonal elements are the covariances, and the matrix is symmetric
27 Interpretation of the Covariance Matrix 1. The diagonal terms are the variances of our different variables. 2. Large diagonal values correspond to strong signals 3. The off-diagonal terms are the covariances between different variables. They reflect distortions in the data (noise, rotation, redundancy). 4. Large off-diagonal values correspond to distortions in our data
28 Interpretation of the Covariance Matrix 1. The diagonal terms are the variances of our different variables. 2. Large diagonal values correspond to strong signals 3. The off-diagonal terms are the covariances between different variables. They reflect distortions in the data (noise, rotation, redundancy). 4. Large off-diagonal values correspond to distortions in our data
29 Minimizing Distortion 1. If we redefine our variables (as linear combinations of each other) the covariance matrix will change. 2. We want to change the covariance matrix so that the off diagonal elements are close to zero. 3. That is, we want to diagonalize the covariance matrix.
30 Minimizing Distortion 1. If we redefine our variables (as linear combinations of each other) the covariance matrix will change. 2. We want to change the covariance matrix so that the off diagonal elements are close to zero. 3. That is, we want to diagonalize the covariance matrix.
31 Minimizing Distortion 1. If we redefine our variables (as linear combinations of each other) the covariance matrix will change. 2. We want to change the covariance matrix so that the off diagonal elements are close to zero. 3. That is, we want to diagonalize the covariance matrix.
32 Example 1: Toy Example X = (1, 2) X = 3/2 Y = (3, 6) Y = 9/2 Subtract the means, since we are only interested in the variances X = Y = ( 1 2 ( 3 2,, ) 1 2 ) 3 2
33 Example 1: Toy Example X = (1, 2) X = 3/2 Y = (3, 6) Y = 9/2 Subtract the means, since we are only interested in the variances X = Y = ( 1 2 ( 3 2,, ) 1 2 ) 3 2
34 Variances C 11 = ( ) + ( C 12 = ( ) + ( C 21 = ( ) + ( C 22 = ( ) + ( ) = 1 2 ) = 3 2 ) = 3 2 ) = 9 2
35 Covariance Matrix The covariance matrix is: C = This is not diagonal the non zero off-diagonal terms indicate the presence of distortion, in this case because the two variables are correlated
36 Rotation of Axes I We need to redefine our variables to diagonalize the covariance matrix. Choose new variables which are a linear combination of the old ones: X = a 1 X + a 2 Y Y = b 1 X + b 2 Y We have to determine the four constants a 1, a 2, b 1, b 2 such that the covariance matrix is diagonal.
37 Rotation of Axes I We need to redefine our variables to diagonalize the covariance matrix. Choose new variables which are a linear combination of the old ones: X = a 1 X + a 2 Y Y = b 1 X + b 2 Y We have to determine the four constants a 1, a 2, b 1, b 2 such that the covariance matrix is diagonal.
38 Rotation of Axes I We need to redefine our variables to diagonalize the covariance matrix. Choose new variables which are a linear combination of the old ones: X = a 1 X + a 2 Y Y = b 1 X + b 2 Y We have to determine the four constants a 1, a 2, b 1, b 2 such that the covariance matrix is diagonal.
39 Rotation of Axes II In this case, the constants are easy to find: X = X + 3Y Y = 3X + Y
40 Recalculating the Data Points Our data points relative to the new axes are now: X 1 = ( 3 3 ) 2 X 2 = ( 3 3 ) 2 = 5 = 5 Y 1 = ( 3 1 ) = 0 Y 2 = ( 3 1 ) = 0
41 Recalculating the Variances The new variances are now: C 11 = ( 5 5) + (5 5) = 25 C 12 = ( 5 0) + (5 0) = 0 C 21 = (0 5) + (0 5) = 0 C 22 = (0 0) + (0 0) = 0
42 Diagonalised Covariance Matrix The covariance matrix is now: C = All off diagonal entries are now zero, so data distortions have been removed 2. The variance of Y (second entry on the diagonal) is now zero. This variable has now been eliminated, so the data has been dimensionally reduced (from 2 dimensions to 1)
43 Eigenvalues 1. The numbers on the diagonal of C are called the eigenvalues of the covariance matrix. 2. Large eigenvalues correspond to large variances these are the important variables to consider. 3. Small eigenvalues correspond to small variances these variables can sometimes be neglected.
44 Eigenvectors The directions of the new rotated axes are called the eigenvectors of the covariance matrix Etymological Note: vector: from Latin, carrier eigen: from German, proper to, belonging to
45 Example 2: A More Realistic Example 3 variables, 5 observations Finance, X Marketing, Y Policy, Z
46 Running PCA in R
47 PCA Output from R
48 Assumptions and Limitations of PCA 1. Large variances represent signal. Small signals represent noise. 2. The new axes are linear combinations of the original axes 3. The principal components are orthogonal (perpendicular) to each other 4. The distributions of our measurements are fully described by their mean and variance
49 Advantages of PCA 1. The procedure is hypothesis free. 2. The procedure is well defined 3. The solution is (essentially) unique
50 Factor Analysis There are a multiplicity of procedures which go under the name of factor analysis. Factor analysis is hypothesis driven and tries to find a priori structure in the data. We ll use the data from Example 2 in the next few slides...
51 Factor Model Hypothesis: A significant part of the variance of our 3 variables can be expressed as linear combinations of two underlying factors, F 1 and F 2 : X = a 1 F 1 + a 2 F 2 + e X Y = b 1 F 1 + b 2 F 2 + e Y Z = c 1 F 1 + c 2 F 2 + e Z e X, e Y and e Z allow for errors in the measuements of our 3 variables.
52 Factor Model Hypothesis: A significant part of the variance of our 3 variables can be expressed as linear combinations of two underlying factors, F 1 and F 2 : X = a 1 F 1 + a 2 F 2 + e X Y = b 1 F 1 + b 2 F 2 + e Y Z = c 1 F 1 + c 2 F 2 + e Z e X, e Y and e Z allow for errors in the measuements of our 3 variables.
53 Factor Model Hypothesis: A significant part of the variance of our 3 variables can be expressed as linear combinations of two underlying factors, F 1 and F 2 : X = a 1 F 1 + a 2 F 2 + e X Y = b 1 F 1 + b 2 F 2 + e Y Z = c 1 F 1 + c 2 F 2 + e Z e X, e Y and e Z allow for errors in the measuements of our 3 variables.
54 Assumptions 1. The factors F i are independent with mean 0 and variance 1 2. The error terms are independent with mean 0 and variance σ 2 i With these assumptions, we can calculate the variances in terms of our hypothesised loadings.
55 Assumptions 1. The factors F i are independent with mean 0 and variance 1 2. The error terms are independent with mean 0 and variance σ 2 i With these assumptions, we can calculate the variances in terms of our hypothesised loadings.
56 Factor Model Variances C XX = a a σ 2 e X C YY = b b σ 2 e Y C ZZ = c c σ 2 e Z C XY = a 1 b 1 + a 2 b 2 C XZ = a 1 c 1 + a 2 c 2 C YZ = b 1 c 1 + b 2 c 2
57 Note: Observations are not mean-adjusted Observed Variances for Example 2 C XX = 9.84 C XX = 5.04 C XX = 3.04 C XY = 0.36 C XZ = 0.44 C YZ = 3.84
58 Fitting the Model To fit the model we would like to choose the loadings a i, b i and c i so that the predicted variances match the observed variances. Problem: The solution is not unique That is, there are different choices for the loadings that yield the same values for the variances; in fact, an infinite number...
59 Fitting the Model To fit the model we would like to choose the loadings a i, b i and c i so that the predicted variances match the observed variances. Problem: The solution is not unique That is, there are different choices for the loadings that yield the same values for the variances; in fact, an infinite number...
60 So... what does the computer do? In essence, it finds an initial set of loadings and then rotates these to find the best solution according to a user specified criterion Varimax: Tends to maximise the difference in loading between factors. Quartimax: Tends to maximise the difference in loading between variables.
61 So... what does the computer do? In essence, it finds an initial set of loadings and then rotates these to find the best solution according to a user specified criterion Varimax: Tends to maximise the difference in loading between factors. Quartimax: Tends to maximise the difference in loading between variables.
62 Example 3 Physics History English Maths Politics
63 Running Factor Analysis in R
64 PCA Analysis of Example 3
65 Factor Analysis of Example 3
66 Some Opinions on Factor Analysis...
67 Take Home 1. Measurements are subject to many distortions including noise, rotation and redundancy 2. PCA is a robust, well defined method for reducing data and investigating underlying structure. 3. There are many approaches to factor analysis; all of them are subject to a certain degree of arbitrariness and should be used with caution
68 Further Reading A Tutorial on Principal Components Analysis Jonathon Shlens shlens/notes.html Factor Analysis Peter Tryfos
Review Jeopardy. Blue vs. Orange. Review Jeopardy
Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round $200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?
Principal Component Analysis
Principal Component Analysis Principle Component Analysis: A statistical technique used to examine the interrelations among a set of variables in order to identify the underlying structure of those variables.
Factor Analysis. Chapter 420. Introduction
Chapter 420 Introduction (FA) is an exploratory technique applied to a set of observed variables that seeks to find underlying factors (subsets of variables) from which the observed variables were generated.
Adding vectors We can do arithmetic with vectors. We ll start with vector addition and related operations. Suppose you have two vectors
1 Chapter 13. VECTORS IN THREE DIMENSIONAL SPACE Let s begin with some names and notation for things: R is the set (collection) of real numbers. We write x R to mean that x is a real number. A real number
Factor Analysis. Advanced Financial Accounting II Åbo Akademi School of Business
Factor Analysis Advanced Financial Accounting II Åbo Akademi School of Business Factor analysis A statistical method used to describe variability among observed variables in terms of fewer unobserved variables
Introduction to Principal Components and FactorAnalysis
Introduction to Principal Components and FactorAnalysis Multivariate Analysis often starts out with data involving a substantial number of correlated variables. Principal Component Analysis (PCA) is a
Component Ordering in Independent Component Analysis Based on Data Power
Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals
[1] Diagonal factorization
![[1] Diagonal factorization](/thumbs/40/21671034.jpg "[1] Diagonal factorization")
8.03 LA.6: Diagonalization and Orthogonal Matrices [ Diagonal factorization [2 Solving systems of first order differential equations [3 Symmetric and Orthonormal Matrices [ Diagonal factorization Recall:
Exploratory Factor Analysis and Principal Components. Pekka Malo & Anton Frantsev 30E00500 Quantitative Empirical Research Spring 2016
and Principal Components Pekka Malo & Anton Frantsev 30E00500 Quantitative Empirical Research Spring 2016 Agenda Brief History and Introductory Example Factor Model Factor Equation Estimation of Loadings
BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES
BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES 123 CHAPTER 7 BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES 7.1 Introduction Even though using SVM presents
by the matrix A results in a vector which is a reflection of the given
Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that
Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/
Basic Filters (7) Convolution/correlation/Linear filtering Gaussian filters Smoothing and noise reduction First derivatives of Gaussian Second derivative of Gaussian: Laplacian Oriented Gaussian filters
Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components
Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they
Factor analysis. Angela Montanari
Factor analysis Angela Montanari 1 Introduction Factor analysis is a statistical model that allows to explain the correlations between a large number of observed correlated variables through a small number
Exploratory Factor Analysis
Exploratory Factor Analysis Definition Exploratory factor analysis (EFA) is a procedure for learning the extent to which k observed variables might measure m abstract variables, wherein m is less than
Introduction to Matrix Algebra
Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary
Statistics in Psychosocial Research Lecture 8 Factor Analysis I. Lecturer: Elizabeth Garrett-Mayer
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES. From Exploratory Factor Analysis Ledyard R Tucker and Robert C.
CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES From Exploratory Factor Analysis Ledyard R Tucker and Robert C MacCallum 1997 180 CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES In
FACTOR ANALYSIS NASC
FACTOR ANALYSIS NASC Factor Analysis A data reduction technique designed to represent a wide range of attributes on a smaller number of dimensions. Aim is to identify groups of variables which are relatively
Multivariate Analysis (Slides 13)
Multivariate Analysis (Slides 13) The final topic we consider is Factor Analysis. A Factor Analysis is a mathematical approach for attempting to explain the correlation between a large set of variables
Common factor analysis
Common factor analysis This is what people generally mean when they say "factor analysis" This family of techniques uses an estimate of common variance among the original variables to generate the factor
October 3rd, 2012. Linear Algebra & Properties of the Covariance Matrix
Linear Algebra & Properties of the Covariance Matrix October 3rd, 2012 Estimation of r and C Let rn 1, rn, t..., rn T be the historical return rates on the n th asset. rn 1 rṇ 2 r n =. r T n n = 1, 2,...,
Dimensionality Reduction: Principal Components Analysis
Dimensionality Reduction: Principal Components Analysis In data mining one often encounters situations where there are a large number of variables in the database. In such situations it is very likely
Rachel J. Goldberg, Guideline Research/Atlanta, Inc., Duluth, GA
PROC FACTOR: How to Interpret the Output of a Real-World Example Rachel J. Goldberg, Guideline Research/Atlanta, Inc., Duluth, GA ABSTRACT THE METHOD This paper summarizes a real-world example of a factor
Linear Algebra Review. Vectors
Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length
Vector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty
Principal Component Analysis
Principal Component Analysis ERS70D George Fernandez INTRODUCTION Analysis of multivariate data plays a key role in data analysis. Multivariate data consists of many different attributes or variables recorded
Understanding and Applying Kalman Filtering
Understanding and Applying Kalman Filtering Lindsay Kleeman Department of Electrical and Computer Systems Engineering Monash University, Clayton 1 Introduction Objectives: 1. Provide a basic understanding
arxiv:1404.1100v1 [cs.lg] 3 Apr 2014
![arxiv:1404.1100v1 [cs.lg] 3 Apr 2014](/thumbs/40/20925558.jpg "arxiv:1404.1100v1 [cs.lg] 3 Apr 2014")
A Tutorial on Principal Component Analysis Jonathon Shlens Google Research Mountain View, CA 94043 (Dated: April 7, 2014; Version 3.02) arxiv:1404.1100v1 [cs.lg] 3 Apr 2014 Principal component analysis
Rotation Matrices and Homogeneous Transformations
Rotation Matrices and Homogeneous Transformations A coordinate frame in an n-dimensional space is defined by n mutually orthogonal unit vectors. In particular, for a two-dimensional (2D) space, i.e., n
Nonlinear Iterative Partial Least Squares Method
Numerical Methods for Determining Principal Component Analysis Abstract Factors Béchu, S., Richard-Plouet, M., Fernandez, V., Walton, J., and Fairley, N. (2016) Developments in numerical treatments for
T-61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier. Santosh Tirunagari : 245577
T-61.3050 : Email Classification as Spam or Ham using Naive Bayes Classifier Santosh Tirunagari : 245577 January 20, 2011 Abstract This term project gives a solution how to classify an email as spam or
Overview of Factor Analysis
Overview of Factor Analysis Jamie DeCoster Department of Psychology University of Alabama 348 Gordon Palmer Hall Box 870348 Tuscaloosa, AL 35487-0348 Phone: (205) 348-4431 Fax: (205) 348-8648 August 1,
Introduction to Principal Component Analysis: Stock Market Values
Chapter 10 Introduction to Principal Component Analysis: Stock Market Values The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from
Exploratory Factor Analysis
Introduction Principal components: explain many variables using few new variables. Not many assumptions attached. Exploratory Factor Analysis Exploratory factor analysis: similar idea, but based on model.
Factor Rotations in Factor Analyses.
Factor Rotations in Factor Analyses. Hervé Abdi 1 The University of Texas at Dallas Introduction The different methods of factor analysis first extract a set a factors from a data set. These factors are
4. There are no dependent variables specified... Instead, the model is: VAR 1. Or, in terms of basic measurement theory, we could model it as:
1 Neuendorf Factor Analysis Assumptions: 1. Metric (interval/ratio) data 2. Linearity (in the relationships among the variables--factors are linear constructions of the set of variables; the critical source
NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )
Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates
FACTOR ANALYSIS. Factor Analysis is similar to PCA in that it is a technique for studying the interrelationships among variables.
FACTOR ANALYSIS Introduction Factor Analysis is similar to PCA in that it is a technique for studying the interrelationships among variables Both methods differ from regression in that they don t have
Multivariate Analysis
Table Of Contents Multivariate Analysis... 1 Overview... 1 Principal Components... 2 Factor Analysis... 5 Cluster Observations... 12 Cluster Variables... 17 Cluster K-Means... 20 Discriminant Analysis...
Introduction: Overview of Kernel Methods
Introduction: Overview of Kernel Methods Statistical Data Analysis with Positive Definite Kernels Kenji Fukumizu Institute of Statistical Mathematics, ROIS Department of Statistical Science, Graduate University
Canonical Correlation Analysis
Canonical Correlation Analysis Lecture 11 August 4, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #11-8/4/2011 Slide 1 of 39 Today s Lecture Canonical Correlation Analysis
Principal components analysis
CS229 Lecture notes Andrew Ng Part XI Principal components analysis In our discussion of factor analysis, we gave a way to model data x R n as approximately lying in some k-dimension subspace, where k
2. Linearity (in relationships among the variables--factors are linear constructions of the set of variables) F 2 X 4 U 4
1 Neuendorf Factor Analysis Assumptions: 1. Metric (interval/ratio) data. Linearity (in relationships among the variables--factors are linear constructions of the set of variables) 3. Univariate and multivariate
EXPLORATORY FACTOR ANALYSIS IN MPLUS, R AND SPSS. sigbert@wiwi.hu-berlin.de
EXPLORATORY FACTOR ANALYSIS IN MPLUS, R AND SPSS Sigbert Klinke 1,2 Andrija Mihoci 1,3 and Wolfgang Härdle 1,3 1 School of Business and Economics, Humboldt-Universität zu Berlin, Germany 2 Department of
1 2 3 1 1 2 x = + x 2 + x 4 1 0 1
(d) If the vector b is the sum of the four columns of A, write down the complete solution to Ax = b. 1 2 3 1 1 2 x = + x 2 + x 4 1 0 0 1 0 1 2. (11 points) This problem finds the curve y = C + D 2 t which
Lecture L26-3D Rigid Body Dynamics: The Inertia Tensor
J. Peraire, S. Widnall 16.07 Dynaics Fall 008 Lecture L6-3D Rigid Body Dynaics: The Inertia Tensor Version.1 In this lecture, we will derive an expression for the angular oentu of a 3D rigid body. We shall
MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).
MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0
Object Recognition and Template Matching
Object Recognition and Template Matching Template Matching A template is a small image (sub-image) The goal is to find occurrences of this template in a larger image That is, you want to find matches of
Orthogonal Diagonalization of Symmetric Matrices
MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding
Principle Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression
Principle Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression Saikat Maitra and Jun Yan Abstract: Dimension reduction is one of the major tasks for multivariate
Statistics for Business Decision Making
Statistics for Business Decision Making Faculty of Economics University of Siena 1 / 62 You should be able to: ˆ Summarize and uncover any patterns in a set of multivariate data using the (FM) ˆ Apply
T-test & factor analysis
Parametric tests T-test & factor analysis Better than non parametric tests Stringent assumptions More strings attached Assumes population distribution of sample is normal Major problem Alternatives Continue
Mehtap Ergüven Abstract of Ph.D. Dissertation for the degree of PhD of Engineering in Informatics
INTERNATIONAL BLACK SEA UNIVERSITY COMPUTER TECHNOLOGIES AND ENGINEERING FACULTY ELABORATION OF AN ALGORITHM OF DETECTING TESTS DIMENSIONALITY Mehtap Ergüven Abstract of Ph.D. Dissertation for the degree
Linear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear
Solutions to Exam in Speech Signal Processing EN2300
Solutions to Exam in Speech Signal Processing EN23 Date: Thursday, Dec 2, 8: 3: Place: Allowed: Grades: Language: Solutions: Q34, Q36 Beta Math Handbook (or corresponding), calculator with empty memory.
Subspace Analysis and Optimization for AAM Based Face Alignment
Subspace Analysis and Optimization for AAM Based Face Alignment Ming Zhao Chun Chen College of Computer Science Zhejiang University Hangzhou, 310027, P.R.China zhaoming1999@zju.edu.cn Stan Z. Li Microsoft
Psychology 7291, Multivariate Analysis, Spring 2003. SAS PROC FACTOR: Suggestions on Use
: Suggestions on Use Background: Factor analysis requires several arbitrary decisions. The choices you make are the options that you must insert in the following SAS statements: PROC FACTOR METHOD=????
Lecture 5: Singular Value Decomposition SVD (1)
EEM3L1: Numerical and Analytical Techniques Lecture 5: Singular Value Decomposition SVD (1) EE3L1, slide 1, Version 4: 25-Sep-02 Motivation for SVD (1) SVD = Singular Value Decomposition Consider the system
Factor Analysis Example: SAS program (in blue) and output (in black) interleaved with comments (in red)
Factor Analysis Example: SAS program (in blue) and output (in black) interleaved with comments (in red) The following DATA procedure is to read input data. This will create a SAS dataset named CORRMATR
Factor Analysis. Sample StatFolio: factor analysis.sgp
STATGRAPHICS Rev. 1/10/005 Factor Analysis Summary The Factor Analysis procedure is designed to extract m common factors from a set of p quantitative variables X. In many situations, a small number of
Math 241, Exam 1 Information.
Math 241, Exam 1 Information. 9/24/12, LC 310, 11:15-12:05. Exam 1 will be based on: Sections 12.1-12.5, 14.1-14.3. The corresponding assigned homework problems (see http://www.math.sc.edu/ boylan/sccourses/241fa12/241.html)
Manifold Learning Examples PCA, LLE and ISOMAP
Manifold Learning Examples PCA, LLE and ISOMAP Dan Ventura October 14, 28 Abstract We try to give a helpful concrete example that demonstrates how to use PCA, LLE and Isomap, attempts to provide some intuition
Chapter 6. Orthogonality
6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be
What is Rotating in Exploratory Factor Analysis?
A peer-reviewed electronic journal. Copyright is retained by the first or sole author, who grants right of first publication to the Practical Assessment, Research & Evaluation. Permission is granted to
How to report the percentage of explained common variance in exploratory factor analysis
UNIVERSITAT ROVIRA I VIRGILI How to report the percentage of explained common variance in exploratory factor analysis Tarragona 2013 Please reference this document as: Lorenzo-Seva, U. (2013). How to report
Part 2: Analysis of Relationship Between Two Variables
Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable
CHAPTER 2. Eigenvalue Problems (EVP s) for ODE s
A SERIES OF CLASS NOTES FOR 005-006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 4 A COLLECTION OF HANDOUTS ON PARTIAL DIFFERENTIAL EQUATIONS
CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen
CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 3: DATA TRANSFORMATION AND DIMENSIONALITY REDUCTION Chapter 3: Data Preprocessing Data Preprocessing: An Overview Data Quality Major
SECOND DERIVATIVE TEST FOR CONSTRAINED EXTREMA
SECOND DERIVATIVE TEST FOR CONSTRAINED EXTREMA This handout presents the second derivative test for a local extrema of a Lagrange multiplier problem. The Section 1 presents a geometric motivation for the
Risk Decomposition of Investment Portfolios. Dan dibartolomeo Northfield Webinar January 2014
Risk Decomposition of Investment Portfolios Dan dibartolomeo Northfield Webinar January 2014 Main Concepts for Today Investment practitioners rely on a decomposition of portfolio risk into factors to guide
Multivariate Analysis of Variance (MANOVA)
Multivariate Analysis of Variance (MANOVA) Aaron French, Marcelo Macedo, John Poulsen, Tyler Waterson and Angela Yu Keywords: MANCOVA, special cases, assumptions, further reading, computations Introduction
Statistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
1 of 79 Erik Eberhardt UBC Geological Engineering EOSC 433
Stress & Strain: A review xx yz zz zx zy xy xz yx yy xx yy zz 1 of 79 Erik Eberhardt UBC Geological Engineering EOSC 433 Disclaimer before beginning your problem assignment: Pick up and compare any set
Similar matrices and Jordan form
Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive
How To Run Factor Analysis
Getting Started in Factor Analysis (using Stata 10) (ver. 1.5) Oscar Torres-Reyna Data Consultant otorres@princeton.edu http://dss.princeton.edu/training/ Factor analysis is used mostly for data reduction
Applications to Data Smoothing and Image Processing I
Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is
Math 215 HW #6 Solutions
Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T
15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
Module 3: Correlation and Covariance
Using Statistical Data to Make Decisions Module 3: Correlation and Covariance Tom Ilvento Dr. Mugdim Pašiƒ University of Delaware Sarajevo Graduate School of Business O ften our interest in data analysis
Probability and Random Variables. Generation of random variables (r.v.)
Probability and Random Variables Method for generating random variables with a specified probability distribution function. Gaussian And Markov Processes Characterization of Stationary Random Process Linearly
Singular Spectrum Analysis with Rssa
Singular Spectrum Analysis with Rssa Maurizio Sanarico Chief Data Scientist SDG Consulting Milano R June 4, 2014 Motivation Discover structural components in complex time series Working hypothesis: a signal
A Solution Manual and Notes for: Exploratory Data Analysis with MATLAB by Wendy L. Martinez and Angel R. Martinez.
A Solution Manual and Notes for: Exploratory Data Analysis with MATLAB by Wendy L. Martinez and Angel R. Martinez. John L. Weatherwax May 7, 9 Introduction Here you ll find various notes and derivations
4. Matrix Methods for Analysis of Structure in Data Sets:
ATM 552 Notes: Matrix Methods: EOF, SVD, ETC. D.L.Hartmann Page 64 4. Matrix Methods for Analysis of Structure in Data Sets: Empirical Orthogonal Functions, Principal Component Analysis, Singular Value
CHAPTER 4 EXAMPLES: EXPLORATORY FACTOR ANALYSIS
Examples: Exploratory Factor Analysis CHAPTER 4 EXAMPLES: EXPLORATORY FACTOR ANALYSIS Exploratory factor analysis (EFA) is used to determine the number of continuous latent variables that are needed to
5.04 Principles of Inorganic Chemistry II
MIT OpenourseWare http://ocw.mit.edu 5.4 Principles of Inorganic hemistry II Fall 8 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 5.4, Principles of
To do a factor analysis, we need to select an extraction method and a rotation method. Hit the Extraction button to specify your extraction method.
Factor Analysis in SPSS To conduct a Factor Analysis, start from the Analyze menu. This procedure is intended to reduce the complexity in a set of data, so we choose Data Reduction from the menu. And the
Quadratic forms Cochran s theorem, degrees of freedom, and all that
Quadratic forms Cochran s theorem, degrees of freedom, and all that Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 1, Slide 1 Why We Care Cochran s theorem tells us
Sections 2.11 and 5.8
Sections 211 and 58 Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1/25 Gesell data Let X be the age in in months a child speaks his/her first word and
How To Cluster
Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical Main
Department of Economics
Department of Economics On Testing for Diagonality of Large Dimensional Covariance Matrices George Kapetanios Working Paper No. 526 October 2004 ISSN 1473-0278 On Testing for Diagonality of Large Dimensional
STA 4107/5107. Chapter 3
STA 4107/5107 Chapter 3 Factor Analysis 1 Key Terms Please review and learn these terms. 2 What is Factor Analysis? Factor analysis is an interdependence technique (see chapter 1) that primarily uses metric
3. Let A and B be two n n orthogonal matrices. Then prove that AB and BA are both orthogonal matrices. Prove a similar result for unitary matrices.
Exercise 1 1. Let A be an n n orthogonal matrix. Then prove that (a) the rows of A form an orthonormal basis of R n. (b) the columns of A form an orthonormal basis of R n. (c) for any two vectors x,y R
Exploratory Factor Analysis: rotation. Psychology 588: Covariance structure and factor models
Exploratory Factor Analysis: rotation Psychology 588: Covariance structure and factor models Rotational indeterminacy Given an initial (orthogonal) solution (i.e., Φ = I), there exist infinite pairs of
Factor Analysis. Principal components factor analysis. Use of extracted factors in multivariate dependency models
Factor Analysis Principal components factor analysis Use of extracted factors in multivariate dependency models 2 KEY CONCEPTS ***** Factor Analysis Interdependency technique Assumptions of factor analysis
Solutions to Math 51 First Exam January 29, 2015
Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not
MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.
MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar
Least-Squares Intersection of Lines
Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a
Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.
Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry
Notes on Orthogonal and Symmetric Matrices MENU, Winter 2013
Notes on Orthogonal and Symmetric Matrices MENU, Winter 201 These notes summarize the main properties and uses of orthogonal and symmetric matrices. We covered quite a bit of material regarding these topics,