Exploratory Factor Analysis


 Eugene Preston
 2 years ago
 Views:
Transcription
1 Introduction Principal components: explain many variables using few new variables. Not many assumptions attached. Exploratory Factor Analysis Exploratory factor analysis: similar idea, but based on model. Idea: variance of each variable depends on common factors (shared among variables) plus specific factor for that variable. Aim: identify common factors, relate to original variables. Want small variance of specific factors ( error ). Extra: use rotation to get clearer answers Examples One aim of factor analysis: identify unobservable characteristics (eg. attitudes, beliefs, perceptions). Observe measurable variables, try to relate. Example: give children several tests of different kinds: reading comprehension, spelling, sentence completion addition and subtraction, counting Hope to find factors picking out first 3 tests ( verbal ability ), last 2 ( numerical ability ). Another: perceptions of luxury cars. Ask potential customers to rate many luxury cars on many features ( style, reliability, performance ), look for factors relating to features. Might get factor based on reliability, fuel economy, maintenance, quality, durability call it sensible. Then another based on luxury, style, performance call it eg. appealing. Aim: pick out common features of variables
2 How it works Measure p variables, X 1,..., X p. Assume standardized: E(X i ) = 0, var(x i ) = 1 for all i. One common factor to start: Observed X i depends on common factor ξ ( xi ) plus specific factor δ i. Write X i = λ i ξ + δ i, like regression except ξ, δ i not observable. Assume ξ also standardized, ξ, δ i independent. Take variances: Interpret λ 2 i as proportion of variation in X i explained by common factor (like R 2 ). Called communality of X i. Rest of variance down to specific factor. Write θ 2 ii = var(δ i ); communality then 1 θ 2 ii. Communality near 1: X i nearperfect measure of ξ. Near 0: nothing to do with common factor. Want communalities not too small. var(x i ) = var(λ i ξ +δ i ) = λ 2 i var(ξ)+var(δ i ) = λ 2 i +var(δ i ). But this is Two common factors In principal components, cannot always use only one. Likewise, in factor analysis, may need 2 or more common factors. Assume ξ 1, ξ 2 independent, standardized. Write X i = λ i1 ξ 1 + λ i2 ξ 2 + δ i ; var(x i ) = λ 2 i1 + λ 2 i2 + θii 2 = 1. Specific variances θii 2 only appear with X i, so only affect variances (not covariances) of X i. Communality now 1 θ 2 ii = λ 2 i1 + λ 2 i2. Hope that one of the λ ij is reasonably large (eg. if factor 1 is verbal ability and factor 2 mathematical, that each test has something to do with one of these). Finding a solution: Principal Factor Analysis Specific factors only impact variances, not covariances. So if we knew the θ 2 ii = var δ i, would only impact diagonal of varcov matrix. Also, since we standardized X i, varcov matrix is same as correlation matrix R. So work with matrix like R, but with the θii 2 subtracted off the diagonal. Then can solve problem using same ideas as principal components: find eigenvalues, then use eigenvectors as factors. (Not only solution, but will work.) 73 74
3 Example: correlation matrix for psychological test data, page 132. Read into IML matrix R. Suppose the specific factor variances are all θ 2 ii = 0.4 (actually value used in text). Then following lines subtract them from matrix diagonal and get eigenvalues/vectors: Eigenvalues are: EVAL theta={ 0.40, 0.40, 0.40, 0.40, 0.40 }; m=rdiag(theta); print m; call eigen(eval,evec,m); print eval; print evec; (as text) and eigenvectors (different from text) EVEC Now, 1 θ 2 ii = λ2 i1 + λ2 i2, so use current estimates of λ ij. These are the jth eigenvalue times the eigenvector coefficient squared, Not all eigenvalues positive. Use those that look meaningfully so (first 2). Eigenvectors give factor loadings as in principal components. First factor mostly 1st 3 tests (para, sent, word); second factor last 2 tests (add, dots). (Verbal, numerical skills.) But... were the communalities correct? added up. Thus: 1 θ11 2 = 2.187(0.5345) ( ) 2 = θ22 2 = 2.187(0.5424) ( ) 2 = θ33 2 = 2.187(0.5234) ( ) 2 = θ44 2 = 2.187(0.2971) (0.6268) 2 = θ55 2 = 2.187(0.2406) (0.6776) 2 = Probably not, since they were just guesses. so go back and repeat the process with values 0.314, 0.329, 0.339, 0.405,
4 Updated values of 1 θ 2 ii are 0.720, 0.694, 0.679, 0.592, Go Eigenvalues and eigenvectors changed slightly: EVAL EVEC back and repeat process with 1 minus these. Continue until no change in eigenvalues, eigenvectors, 1 θ 2 ii. Final answers (basically as text): EVAL EVEC Using SAS s PROC FACTOR Of course, only did above to show method. In practice, use canned procedure. SAS can use original data or correlation matrix. For latter, type matrix into file like this: para sent word add dots data rmat(type=corr); _type_= corr ; infile "rex2.dat"; input _name_ $ para sent word add dots; Following does one step of cycle starting from 1 θii 2 = 0.6: proc factor; priors ; and produces this output: with variable name on front of line, then read in file in special way with same variable names: 81 82
5 To do iterated principal factor analysis, change the code to contain Prior Communality Estimates: NUMERIC PARA SENT WORD ADD DOTS Eigenvalues of the Reduced Correlation Matrix: Total = 3 Average = Eigenvalue Difference Proportion Cumulative Eigenvalue Difference Proportion Cumulative which has the same eigenvalues as we calculated. proc factor method=prinit; which gives this (edited): Iter Change Communalities Convergence criterion satisfied. Eigenvalues of the Reduced Correlation Matrix: Total = Average = Eigenvalue Eigenvalue Initial Factor Method: Iterated Principal Factor Analysis Factor Pattern PARA SENT WORD ADD DOTS Variance explained by each factor Final Communality Estimates: Total = PARA SENT WORD ADD DOTS This is the same (numerically) as the text and our previous calculation. Mathematics of the common factor model With c factors, have model like X i = λ i1 ξ λ ic ξ c + δ i for each i, i = 1, 2,..., p. Easier to write in matrix terms: X = ΞΛ c +. Assumptions: 1. Common factors ξ uncorrelated, variance 1: Ξ Ξ/(n 1) = I. 2. Specific factors δ uncorrelated with variances θii 2 : Θ = /(n 1) diagonal
6 Invariance under rotation 3. Common and specific factors uncorrelated: Ξ = 0. Since X i standardized, correlation matrix R = X X/(n 1). Substitute for X, and remove any terms 0 by assumption. Finally R Θ = Λ c Λ c. In principal components, choose each component to maximize variance (while being uncorrelated with previous components). But here, no such restriction: any matrix Λ c satisfying equation is OK. Consider orthogonal matrix T, representing rotation. Let Λ c = Λ c T. Then Λ cλ c = Λ c T T Λ c = Λ c Λ c, because T T = I. That is, for any matrix of factor loadings solving the problem, any rotated version of it also solves the problem. In principal components, difficult to interpret mediumsized component loading. Idea: find rotation method that makes factors easy to interpret Kaiser s varimax rotation Want factor loadings (elements of rotated Λ c ) to be close to 0 or 1. Then each factor clearly depends (or does not depend) on each variable. Varimax: find rotation that maximizes sum of column variances. Maximizing variances drives values towards extremes. In SAS, change FACTOR line to read proc factor method=prinit rotate=varimax; Can eliminate PRIORS line. Results: Rotation Method: Varimax Orthogonal Transformation Matrix Rotated Factor Pattern PARA SENT WORD ADD DOTS Variance explained by each factor
7 Factor 1 now clearly depends on the three verbal tests (paragraph comprehension, sentence completion, word meaning); factor 2 now clearly based on numerical tests (addition, counting dots). Variance explained by 1st factor now no longer largest possible, but 2 factors together still explain same amount of variance. Quartimax rotation Similar idea to varimax: but now maximize total row variance. Makes each variable load on as few factors as possible. (In varimax, could still have variable appearing in several different factors.) In example data set, results very similar (because each variable only loaded on one factor anyway in varimax) Factor scores In principal components, obtained component scores: values for each observation representing where that observation falls on each component. Provided way to plot multidimensional data in 2 dimensions. Used component loadings to make linear combinations of original variables. Same idea in factor analysis, but difficulty: don t know specific factors exactly. So estimate factor scores by assuming specific factor δ i = 0: Ξ = XR 1 Λ c. Λ c depends on rotation, therefore factor scores do too. Saving and plotting factor scores Factor scores depend on original observations, so need original data not just correlation matrix. Data in file stock_returns.dat are weekly rates of return for 5 stocks on NYSE: Allied Chemical, du Pont, Union Carbide, Exxon, Texaco. Collected over 100 weeks. SAS FACTOR line: proc factor method=prinit rotate=varimax priors=smc nfact Variations: priors command specifies communalities that should be closer to truth; nfactors says to get 2 factors; out says to create output dataset with factor scores
8 Factor pattern before rotation: ALLCHEM DUPONT UNIONCAR EXXON TEXACO Variance explained by each factor Factor 1 basically average of all; factor 2 contrasts Texaco with du Pont. Factor 1 explains most of variance. Compare pattern after rotation: ALLCHEM DUPONT UNIONCAR EXXON TEXACO Variance explained by each factor Factor 1 picks out first 3 (chemical) companies; factor 2 picks out last 2 (oil). Explained variance shared out more evenly between factors Can print out new dataset (including factor scores) and plot factor scores. (SAS by default uses newest dataset.) proc print; proc plot; plot Factor1 * Factor2; On plot, pick out good/bad days for chemical (top/bottom), good/bad days for oil (right/left). Example: day 13. Factor 1 score 2.64, factor 2 score Good day for chem companies: du Pont, Union Carbide big gains. Average day for oil: small gains for both Exxon, Texaco. Day 20: factor , factor No gain for chem companies, both Exxon, Texaco solid gain. 97
Factor Analysis. Advanced Financial Accounting II Åbo Akademi School of Business
Factor Analysis Advanced Financial Accounting II Åbo Akademi School of Business Factor analysis A statistical method used to describe variability among observed variables in terms of fewer unobserved variables
More informationFactor Analysis Example: SAS program (in blue) and output (in black) interleaved with comments (in red)
Factor Analysis Example: SAS program (in blue) and output (in black) interleaved with comments (in red) The following DATA procedure is to read input data. This will create a SAS dataset named CORRMATR
More informationFACTOR ANALYSIS. Factor Analysis is similar to PCA in that it is a technique for studying the interrelationships among variables.
FACTOR ANALYSIS Introduction Factor Analysis is similar to PCA in that it is a technique for studying the interrelationships among variables Both methods differ from regression in that they don t have
More informationRachel J. Goldberg, Guideline Research/Atlanta, Inc., Duluth, GA
PROC FACTOR: How to Interpret the Output of a RealWorld Example Rachel J. Goldberg, Guideline Research/Atlanta, Inc., Duluth, GA ABSTRACT THE METHOD This paper summarizes a realworld example of a factor
More informationFactor Analysis. Factor Analysis
Factor Analysis Principal Components Analysis, e.g. of stock price movements, sometimes suggests that several variables may be responding to a small number of underlying forces. In the factor model, we
More informationFACTOR ANALYSIS NASC
FACTOR ANALYSIS NASC Factor Analysis A data reduction technique designed to represent a wide range of attributes on a smaller number of dimensions. Aim is to identify groups of variables which are relatively
More informationCommon factor analysis
Common factor analysis This is what people generally mean when they say "factor analysis" This family of techniques uses an estimate of common variance among the original variables to generate the factor
More informationFactor Analysis. Chapter 420. Introduction
Chapter 420 Introduction (FA) is an exploratory technique applied to a set of observed variables that seeks to find underlying factors (subsets of variables) from which the observed variables were generated.
More informationMultivariate Analysis (Slides 13)
Multivariate Analysis (Slides 13) The final topic we consider is Factor Analysis. A Factor Analysis is a mathematical approach for attempting to explain the correlation between a large set of variables
More informationPsychology 7291, Multivariate Analysis, Spring 2003. SAS PROC FACTOR: Suggestions on Use
: Suggestions on Use Background: Factor analysis requires several arbitrary decisions. The choices you make are the options that you must insert in the following SAS statements: PROC FACTOR METHOD=????
More informationThe president of a Fortune 500 firm wants to measure the firm s image.
4. Factor Analysis A related method to the PCA is the Factor Analysis (FA) with the crucial difference that in FA a statistical model is constructed to explain the interrelations (correlations) between
More informationPart 2: EFA Outline. Exploratory and Confirmatory Factor Analysis. Basic Ideas of Factor Analysis. Basic ideas: 1. Linear regression on common factors
Exploratory and Confirmatory Factor Analysis Michael Friendly Feb. 25, Mar. 3, 10, 2008 SCS Short Course web notes: http://www.math.yorku.ca/scs/courses/factor/ Part 2: EFA Outline 1 Linear regression
More informationExploratory Factor Analysis and Principal Components. Pekka Malo & Anton Frantsev 30E00500 Quantitative Empirical Research Spring 2016
and Principal Components Pekka Malo & Anton Frantsev 30E00500 Quantitative Empirical Research Spring 2016 Agenda Brief History and Introductory Example Factor Model Factor Equation Estimation of Loadings
More informationFactor Analysis. Principal components factor analysis. Use of extracted factors in multivariate dependency models
Factor Analysis Principal components factor analysis Use of extracted factors in multivariate dependency models 2 KEY CONCEPTS ***** Factor Analysis Interdependency technique Assumptions of factor analysis
More informationFactor Analysis. Sample StatFolio: factor analysis.sgp
STATGRAPHICS Rev. 1/10/005 Factor Analysis Summary The Factor Analysis procedure is designed to extract m common factors from a set of p quantitative variables X. In many situations, a small number of
More informationIntroduction to Matrix Algebra
Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra  1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary
More informationReview Jeopardy. Blue vs. Orange. Review Jeopardy
Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 03 Jeopardy Round $200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?
More informationStatistics for Business Decision Making
Statistics for Business Decision Making Faculty of Economics University of Siena 1 / 62 You should be able to: ˆ Summarize and uncover any patterns in a set of multivariate data using the (FM) ˆ Apply
More informationTo do a factor analysis, we need to select an extraction method and a rotation method. Hit the Extraction button to specify your extraction method.
Factor Analysis in SPSS To conduct a Factor Analysis, start from the Analyze menu. This procedure is intended to reduce the complexity in a set of data, so we choose Data Reduction from the menu. And the
More informationExploratory Factor Analysis
Exploratory Factor Analysis Definition Exploratory factor analysis (EFA) is a procedure for learning the extent to which k observed variables might measure m abstract variables, wherein m is less than
More informationSTA 4107/5107. Chapter 3
STA 4107/5107 Chapter 3 Factor Analysis 1 Key Terms Please review and learn these terms. 2 What is Factor Analysis? Factor analysis is an interdependence technique (see chapter 1) that primarily uses metric
More informationExploratory Factor Analysis: rotation. Psychology 588: Covariance structure and factor models
Exploratory Factor Analysis: rotation Psychology 588: Covariance structure and factor models Rotational indeterminacy Given an initial (orthogonal) solution (i.e., Φ = I), there exist infinite pairs of
More informationDimensionality Reduction: Principal Components Analysis
Dimensionality Reduction: Principal Components Analysis In data mining one often encounters situations where there are a large number of variables in the database. In such situations it is very likely
More informationDoing Quantitative Research 26E02900, 6 ECTS Lecture 2: Measurement Scales. OlliPekka Kauppila Rilana Riikkinen
Doing Quantitative Research 26E02900, 6 ECTS Lecture 2: Measurement Scales OlliPekka Kauppila Rilana Riikkinen Learning Objectives 1. Develop the ability to assess a quality of measurement instruments
More informationIntroduction to Principal Component Analysis: Stock Market Values
Chapter 10 Introduction to Principal Component Analysis: Stock Market Values The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from
More informationPrincipal Components Analysis (PCA)
Principal Components Analysis (PCA) Janette Walde janette.walde@uibk.ac.at Department of Statistics University of Innsbruck Outline I Introduction Idea of PCA Principle of the Method Decomposing an Association
More informationPRINCIPAL COMPONENT ANALYSIS
1 Chapter 1 PRINCIPAL COMPONENT ANALYSIS Introduction: The Basics of Principal Component Analysis........................... 2 A Variable Reduction Procedure.......................................... 2
More informationTtest & factor analysis
Parametric tests Ttest & factor analysis Better than non parametric tests Stringent assumptions More strings attached Assumes population distribution of sample is normal Major problem Alternatives Continue
More informationGetting Started in Factor Analysis (using Stata 10) (ver. 1.5)
Getting Started in Factor Analysis (using Stata 10) (ver. 1.5) Oscar TorresReyna Data Consultant otorres@princeton.edu http://dss.princeton.edu/training/ Factor analysis is used mostly for data reduction
More informationA Introduction to Matrix Algebra and Principal Components Analysis
A Introduction to Matrix Algebra and Principal Components Analysis Multivariate Methods in Education ERSH 8350 Lecture #2 August 24, 2011 ERSH 8350: Lecture 2 Today s Class An introduction to matrix algebra
More informationCanonical Correlation
Chapter 400 Introduction Canonical correlation analysis is the study of the linear relations between two sets of variables. It is the multivariate extension of correlation analysis. Although we will present
More informationSimilarity and Diagonalization. Similar Matrices
MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that
More informationIntroduction to Principal Components and FactorAnalysis
Introduction to Principal Components and FactorAnalysis Multivariate Analysis often starts out with data involving a substantial number of correlated variables. Principal Component Analysis (PCA) is a
More informationFACTOR ANALYSIS EXPLORATORY APPROACHES. Kristofer Årestedt
FACTOR ANALYSIS EXPLORATORY APPROACHES Kristofer Årestedt 20130428 UNIDIMENSIONALITY Unidimensionality imply that a set of items forming an instrument measure one thing in common Unidimensionality is
More informationPrincipal Component Analysis
Principal Component Analysis Principle Component Analysis: A statistical technique used to examine the interrelations among a set of variables in order to identify the underlying structure of those variables.
More informationA Brief Introduction to SPSS Factor Analysis
A Brief Introduction to SPSS Factor Analysis SPSS has a procedure that conducts exploratory factor analysis. Before launching into a step by step example of how to use this procedure, it is recommended
More informationStatistics in Psychosocial Research Lecture 8 Factor Analysis I. Lecturer: Elizabeth GarrettMayer
This work is licensed under a Creative Commons AttributionNonCommercialShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationCHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES. From Exploratory Factor Analysis Ledyard R Tucker and Robert C.
CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES From Exploratory Factor Analysis Ledyard R Tucker and Robert C MacCallum 1997 180 CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES In
More informationFactor Analysis  2 nd TUTORIAL
Factor Analysis  2 nd TUTORIAL Subject marks File sub_marks.csv shows correlation coefficients between subject scores for a sample of 220 boys. sub_marks
More informationRegression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur
Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur Lecture  7 Multiple Linear Regression (Contd.) This is my second lecture on Multiple Linear Regression
More information[1] Diagonal factorization
8.03 LA.6: Diagonalization and Orthogonal Matrices [ Diagonal factorization [2 Solving systems of first order differential equations [3 Symmetric and Orthonormal Matrices [ Diagonal factorization Recall:
More informationFactor analysis. Angela Montanari
Factor analysis Angela Montanari 1 Introduction Factor analysis is a statistical model that allows to explain the correlations between a large number of observed correlated variables through a small number
More informationMISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group
MISSING DATA TECHNIQUES WITH SAS IDRE Statistical Consulting Group ROAD MAP FOR TODAY To discuss: 1. Commonly used techniques for handling missing data, focusing on multiple imputation 2. Issues that could
More informationNCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )
Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates
More informationChapter 7 Factor Analysis SPSS
Chapter 7 Factor Analysis SPSS Factor analysis attempts to identify underlying variables, or factors, that explain the pattern of correlations within a set of observed variables. Factor analysis is often
More informationTopic 10: Factor Analysis
Topic 10: Factor Analysis Introduction Factor analysis is a statistical method used to describe variability among observed variables in terms of a potentially lower number of unobserved variables called
More information4. There are no dependent variables specified... Instead, the model is: VAR 1. Or, in terms of basic measurement theory, we could model it as:
1 Neuendorf Factor Analysis Assumptions: 1. Metric (interval/ratio) data 2. Linearity (in the relationships among the variablesfactors are linear constructions of the set of variables; the critical source
More informationCanonical Correlation Analysis
Canonical Correlation Analysis Lecture 11 August 4, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #118/4/2011 Slide 1 of 39 Today s Lecture Canonical Correlation Analysis
More informationRegression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur. Lecture  2 Simple Linear Regression
Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur Lecture  2 Simple Linear Regression Hi, this is my second lecture in module one and on simple
More informationRegression III: Advanced Methods
Lecture 16: Generalized Additive Models Regression III: Advanced Methods Bill Jacoby Michigan State University http://polisci.msu.edu/jacoby/icpsr/regress3 Goals of the Lecture Introduce Additive Models
More informationMultivariate Analysis
Table Of Contents Multivariate Analysis... 1 Overview... 1 Principal Components... 2 Factor Analysis... 5 Cluster Observations... 12 Cluster Variables... 17 Cluster KMeans... 20 Discriminant Analysis...
More informationSummary of week 8 (Lectures 22, 23 and 24)
WEEK 8 Summary of week 8 (Lectures 22, 23 and 24) This week we completed our discussion of Chapter 5 of [VST] Recall that if V and W are inner product spaces then a linear map T : V W is called an isometry
More information2. Linearity (in relationships among the variablesfactors are linear constructions of the set of variables) F 2 X 4 U 4
1 Neuendorf Factor Analysis Assumptions: 1. Metric (interval/ratio) data. Linearity (in relationships among the variablesfactors are linear constructions of the set of variables) 3. Univariate and multivariate
More informationOrthogonal Diagonalization of Symmetric Matrices
MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding
More informationMultivariate Analysis (Slides 4)
Multivariate Analysis (Slides 4) In today s class we examine examples of principal components analysis. We shall consider a difficulty in applying the analysis and consider a method for resolving this.
More informationChapter 6. Orthogonality
6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be
More informationAssociation Between Variables
Contents 11 Association Between Variables 767 11.1 Introduction............................ 767 11.1.1 Measure of Association................. 768 11.1.2 Chapter Summary.................... 769 11.2 Chi
More informationExploratory Factor Analysis Brian Habing  University of South Carolina  October 15, 2003
Exploratory Factor Analysis Brian Habing  University of South Carolina  October 15, 2003 FA is not worth the time necessary to understand it and carry it out. Hills, 1977 Factor analysis should not
More informationIntroduction to Regression. Dr. Tom Pierce Radford University
Introduction to Regression Dr. Tom Pierce Radford University In the chapter on correlational techniques we focused on the Pearson R as a tool for learning about the relationship between two variables.
More informationPrincipal Component Analysis
Principal Component Analysis ERS70D George Fernandez INTRODUCTION Analysis of multivariate data plays a key role in data analysis. Multivariate data consists of many different attributes or variables recorded
More informationA Brief Introduction to Factor Analysis
1. Introduction A Brief Introduction to Factor Analysis Factor analysis attempts to represent a set of observed variables X 1, X 2. X n in terms of a number of 'common' factors plus a factor which is unique
More informationSCHOOL OF MATHEMATICS MATHEMATICS FOR PART I ENGINEERING. Self Study Course
SCHOOL OF MATHEMATICS MATHEMATICS FOR PART I ENGINEERING Self Study Course MODULE 17 MATRICES II Module Topics 1. Inverse of matrix using cofactors 2. Sets of linear equations 3. Solution of sets of linear
More informationCOMP6053 lecture: Relationship between two variables: correlation, covariance and rsquared. jn2@ecs.soton.ac.uk
COMP6053 lecture: Relationship between two variables: correlation, covariance and rsquared jn2@ecs.soton.ac.uk Relationships between variables So far we have looked at ways of characterizing the distribution
More informationMatrices provide a compact notation for expressing systems of equations or variables. For instance, a linear function might be written as: b k
MATRIX ALGEBRA FOR STATISTICS: PART 1 Matrices provide a compact notation for expressing systems of equations or variables. For instance, a linear function might be written as: y = x 1 b 1 + x 2 + x 3
More information15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
More informationMehtap Ergüven Abstract of Ph.D. Dissertation for the degree of PhD of Engineering in Informatics
INTERNATIONAL BLACK SEA UNIVERSITY COMPUTER TECHNOLOGIES AND ENGINEERING FACULTY ELABORATION OF AN ALGORITHM OF DETECTING TESTS DIMENSIONALITY Mehtap Ergüven Abstract of Ph.D. Dissertation for the degree
More informationOverview of Factor Analysis
Overview of Factor Analysis Jamie DeCoster Department of Psychology University of Alabama 348 Gordon Palmer Hall Box 870348 Tuscaloosa, AL 354870348 Phone: (205) 3484431 Fax: (205) 3488648 August 1,
More informationSimple Regression Theory II 2010 Samuel L. Baker
SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the
More informationPRINCIPAL COMPONENTS AND THE MAXIMUM LIKELIHOOD METHODS AS TOOLS TO ANALYZE LARGE DATA WITH A PSYCHOLOGICAL TESTING EXAMPLE
PRINCIPAL COMPONENTS AND THE MAXIMUM LIKELIHOOD METHODS AS TOOLS TO ANALYZE LARGE DATA WITH A PSYCHOLOGICAL TESTING EXAMPLE Markela Muca Llukan Puka Klodiana Bani Department of Mathematics, Faculty of
More informationExample: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation:  Feature vector X,  qualitative response Y, taking values in C
More information2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system
1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables
More informationLinear Algebra Review. Vectors
Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length
More informationManifold Learning Examples PCA, LLE and ISOMAP
Manifold Learning Examples PCA, LLE and ISOMAP Dan Ventura October 14, 28 Abstract We try to give a helpful concrete example that demonstrates how to use PCA, LLE and Isomap, attempts to provide some intuition
More informationRow and column operations
Row and column operations It is often very useful to apply row and column operations to a matrix. Let us list what operations we re going to be using. 3 We ll illustrate these using the example matrix
More informationThe basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23
(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, highdimensional
More informationFactor Rotations in Factor Analyses.
Factor Rotations in Factor Analyses. Hervé Abdi 1 The University of Texas at Dallas Introduction The different methods of factor analysis first extract a set a factors from a data set. These factors are
More informationNotes for STA 437/1005 Methods for Multivariate Data
Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.
More informationModule 3: Correlation and Covariance
Using Statistical Data to Make Decisions Module 3: Correlation and Covariance Tom Ilvento Dr. Mugdim Pašiƒ University of Delaware Sarajevo Graduate School of Business O ften our interest in data analysis
More informationDATA ANALYSIS II. Matrix Algorithms
DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where
More informationLinear Dependence Tests
Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks
More information9.2 User s Guide SAS/STAT. The FACTOR Procedure. (Book Excerpt) SAS Documentation
SAS/STAT 9.2 User s Guide The FACTOR Procedure (Book Excerpt) SAS Documentation This document is an individual chapter from SAS/STAT 9.2 User s Guide. The correct bibliographic citation for the complete
More informationUnderstanding and Using Factor Scores: Considerations for the Applied Researcher
A peerreviewed electronic journal. Copyright is retained by the first or sole author, who grants right of first publication to the Practical Assessment, Research & Evaluation. Permission is granted to
More informationCHAPTER 17. Linear Programming: Simplex Method
CHAPTER 17 Linear Programming: Simplex Method CONTENTS 17.1 AN ALGEBRAIC OVERVIEW OF THE SIMPLEX METHOD Algebraic Properties of the Simplex Method Determining a Basic Solution Basic Feasible Solution 17.2
More informationThe ith principal component (PC) is the line that follows the eigenvector associated with the ith largest eigenvalue.
More Principal Components Summary Principal Components (PCs) are associated with the eigenvectors of either the covariance or correlation matrix of the data. The ith principal component (PC) is the line
More informationLeastSquares Intersection of Lines
LeastSquares Intersection of Lines Johannes Traa  UIUC 2013 This writeup derives the leastsquares solution for the intersection of lines. In the general case, a set of lines will not intersect at a
More informationPrincipal components analysis
CS229 Lecture notes Andrew Ng Part XI Principal components analysis In our discussion of factor analysis, we gave a way to model data x R n as approximately lying in some kdimension subspace, where k
More informationOLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique  ie the estimator has the smallest variance
Lecture 5: Hypothesis Testing What we know now: OLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique  ie the estimator has the smallest variance (if the GaussMarkov
More informationDiscriminant Function Analysis in SPSS To do DFA in SPSS, start from Classify in the Analyze menu (because we re trying to classify participants into
Discriminant Function Analysis in SPSS To do DFA in SPSS, start from Classify in the Analyze menu (because we re trying to classify participants into different groups). In this case we re looking at a
More informationPower System Analysis Prof. A. K. Sinha Department of Electrical Engineering Indian Institute of Technology, Kharagpur. Lecture  19 Power Flow IV
Power System Analysis Prof. A. K. Sinha Department of Electrical Engineering Indian Institute of Technology, Kharagpur Lecture  19 Power Flow IV Welcome to lesson 19 on Power System Analysis. In this
More informationExploratory Factor Analysis of Demographic Characteristics of Antenatal Clinic Attendees and their Association with HIV Risk
Doi:10.5901/mjss.2014.v5n20p303 Abstract Exploratory Factor Analysis of Demographic Characteristics of Antenatal Clinic Attendees and their Association with HIV Risk Wilbert Sibanda Philip D. Pretorius
More informationNotes on Applied Linear Regression
Notes on Applied Linear Regression Jamie DeCoster Department of Social Psychology Free University Amsterdam Van der Boechorststraat 1 1081 BT Amsterdam The Netherlands phone: +31 (0)20 4448935 email:
More information( % . This matrix consists of $ 4 5 " 5' the coefficients of the variables as they appear in the original system. The augmented 3 " 2 2 # 2 " 3 4&
Matrices define matrix We will use matrices to help us solve systems of equations. A matrix is a rectangular array of numbers enclosed in parentheses or brackets. In linear algebra, matrices are important
More informationELECE8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems
Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed
More information12/31/2016. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2
PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 Understand linear regression with a single predictor Understand how we assess the fit of a regression model Total Sum of Squares
More informationStandard Deviation Calculator
CSS.com Chapter 35 Standard Deviation Calculator Introduction The is a tool to calculate the standard deviation from the data, the standard error, the range, percentiles, the COV, confidence limits, or
More information13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.
3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in threespace, we write a vector in terms
More informationMATH 551  APPLIED MATRIX THEORY
MATH 55  APPLIED MATRIX THEORY FINAL TEST: SAMPLE with SOLUTIONS (25 points NAME: PROBLEM (3 points A web of 5 pages is described by a directed graph whose matrix is given by A Do the following ( points
More informationData analysis process
Data analysis process Data collection and preparation Collect data Prepare codebook Set up structure of data Enter data Screen data for errors Exploration of data Descriptive Statistics Graphs Analysis
More informationFactor Analysis and Structural equation modelling
Factor Analysis and Structural equation modelling Herman Adèr Previously: Department Clinical Epidemiology and Biostatistics, VU University medical center, Amsterdam Stavanger July 4 13, 2006 Herman Adèr
More informationWooldridge, Introductory Econometrics, 4th ed. Chapter 15: Instrumental variables and two stage least squares
Wooldridge, Introductory Econometrics, 4th ed. Chapter 15: Instrumental variables and two stage least squares Many economic models involve endogeneity: that is, a theoretical relationship does not fit
More information