Factor Analysis. Sample StatFolio: factor analysis.sgp

Transcription

1 STATGRAPHICS Rev. 1/10/005 Factor Analysis Summary The Factor Analysis procedure is designed to extract m common factors from a set of p quantitative variables X. In many situations, a small number of common factors may be able to represent a large percentage of the variability in the original variables. The ability to express the covariances amongst the variables in terms of a small number of meaningful factors often leads to important insights about the data being analyzed. This procedure supports both principal components and classical factor analysis. Factor loadings may be extracted from either the sample covariance or sample correlation matrix. The initial loadings may be rotated using either varimax, equimax, or quartimax rotation. Sample StatFolio: factor analysis.sgp Sample Data: The file 93cars.sf6 contains information on 6 variables for n = 93 makes and models of automobiles, taken from Lock (1993). The table below shows a partial list of the data in that file: Make Model Engine Horsepower Fuel Passengers Length Size Tank Acura Integra Acura Legend Audi Audi BMW 535i Buick Century Buick LeSabre Buick Roadmaster Buick Riviera Cadillac DeVille Cadillac Seville Chevrolet Cavalier It is desired to perform a factor analysis on the following variables: Engine Size Horsepower Fueltank Passengers Length Wheelbase Width U Turn Space Rear seat Luggage Weight 005 by StatPoint, Inc. Factor Analysis - 1

2 A matrix plot of the data is shown below: STATGRAPHICS Rev. 1/10/005 Engine Size Horsepower Fueltank Passengers Length Wheelbase Width U Turn Space Rear seat Luggage Weight As might be expected, the variables are highly correlated, since most are related to vehicle size. Data Input The data input dialog box requests the names of the columns containing the data: Data: either the original observations or the sample covariance matrix Σˆ. If entering the original observations, enter p numeric columns containing the n values for each column of X. 005 by StatPoint, Inc. Factor Analysis -

3 STATGRAPHICS Rev. 1/10/005 If entering the sample covariance matrix, enter p numeric columns containing the p values for each column of Σˆ. If the covariance matrix is entered, some of the tables and plots will not be available. Point Labels: optional labels for each observation. Select: subset selection. Statistical Model The goal of a factor analysis is to characterize the p variables in X in terms of a small number m of common factors F, which impact all of the variables, and a set of errors or specific factors ε, which affect only a single X variable. Following Johnson and Wichern (00), the orthogonal common factor model expresses the observed variables as X X X 1 µ 1 = l11f1 + l1f l1 mfm + ε1 µ = l F l F l F m m + ε p µ = l... + p p1 F1 + l pf + + l pmfm ε p (1) In matrix notation, X µ = LF + ε () where µ is a vector of means and L is called the factor loading matrix. It is assumed that the common factors and specific factors are all independent of one another. To avoid ambiguity in scaling, the variances of the common factors are assumed to equal 1, while the covariance matrix of the specific factors Ψ is a diagonal matrix with diagonal elements Ψ j. The covariance matrix Σ of the original observations X is related to the factor loading matrix by Σ = L L + Ψ (3) An important result of the above model is the relationship between the variances of the original X variables and the variances of the derived factors. In particular, Var( X ) = l 1 + l l + Ψ (4) j j j jm j This variance is expressed as the sum of two quantities: 1. the communality: l + l + + l j1 j... jm. the specific variance: Ψ j The communality is the variance attributable to factors that all the X variables have in common, while the specific variance is specific to a single factor. It should also be noted that the factor loadings L are not unique. Multiplication by any orthogonal matrix yields another acceptable set of factor loadings. Subsequent to the initial factor 005 by StatPoint, Inc. Factor Analysis - 3

4 STATGRAPHICS Rev. 1/10/005 extraction, it is therefore common to rotate the factor loadings until they can be most easily interpreted. Analysis Summary The Analysis Summary table is shown below: Factor Analysis Data variables: Engine Size Horsepower Fueltank Passengers Length Wheelbase Width U Turn Space Rear seat Luggage Weight Data input: observations Number of complete cases: 8 Missing value treatment: listwise Standardized: yes Type of factoring: principal components Number of factors extracted: Factor Analysis Factor Percent of Cumulative Number Eigenvalue Variance Percentage Initial Variable Communality Engine Size 1.0 Horsepower 1.0 Fueltank 1.0 Passengers 1.0 Length 1.0 Wheelbase 1.0 Width 1.0 U Turn Space 1.0 Rear seat 1.0 Luggage 1.0 Weight 1.0 Displayed in the table are: Data variables: the names of the p input columns. 005 by StatPoint, Inc. Factor Analysis - 4

5 STATGRAPHICS Rev. 1/10/005 Data input: either observations or matrix, depending upon whether the input columns contain the original observations or the sample covariance matrix. Number of complete cases: the number of cases n for which none of the observations were missing. Missing value treatment: how missing values were treated in estimating the covariance or correlation matrix. If listwise, the estimates were based on complete cases only. If pairwise, all pairs of non-missing data values were used to obtain the estimates. Standardized: yes if the analysis was based on the correlation matrix. No if it was based on the covariance matrix. Type of factoring: either principal components if the factor extraction was done directly on the sample covariance or correlation matrix, or classical if the diagonal elements were adjusted using estimates of the communalities. Number of components extracted: the number of components m extracted from the data. This number is based on the settings on the Analysis Options dialog box. A table is also displayed showing information for each of the p possible factors: Factor number: the factor number j, from 1 to p. Eigenvalue: the eigenvalue of the estimated covariance or correlation matrix, λˆ j, after adjusting for the estimated communalities if using the classical method. Percentage of variance: the percent of the total estimated population variance represented by this factor, equal to 100 ˆ λ1 ˆ λ j % ˆ λ... ˆ + + λ + m (5) Cumulative percentage: the cumulative percentage of the total estimated population variance accounted for by the first j factors. Initial communality: the initial communality used in the calculations, either input by the user or estimated from the sample covariances or correlations. In the example, the first m = factors account for over 84% of the overall variance amongst the 11 variables. 005 by StatPoint, Inc. Factor Analysis - 5

6 Analysis Options STATGRAPHICS Rev. 1/10/005 Missing Values Treatment: method of handling missing values when estimating the sample covariances or correlations. Specify Listwise to use only cases with no missing values for any of the input variables. Specify Pairwise to use all pairs of observations in which neither value is missing. Standardize: Check this box to base the analysis on the sample correlation matrix rather than the sample covariance matrix. This corresponds to standardizing each input variable before calculating the covariances, by subtracting its mean and dividing by its standard deviation. Type of Factoring: Select principal components to extract the factors directly from the covariance or correlation matrix. Select classical to replace the diagonal elements with estimated communalities. If using the classical method, you can specify the communalities by pressing the Communalities button or let the program use an iterative procedure to estimate them. Rotation: the method used to rotate the factor loading matrix after it has been extracted. Varimax rotation maximizes the variance of the squared loadings in each column. Quartimax maximizes the variance of the squared loadings in each row. Equimax attempts to achieve a balance between rows and columns. Extract By: the criterion used to determine the number of factors to extract. Minimum Eigenvalue: if extracting by magnitude of the eigenvalues, the smallest eigenvalue for which a factor will be extracted. Number of Factors: if extracting by number of factors, the number k. 005 by StatPoint, Inc. Factor Analysis - 6

7 There are also two buttons that access additional dialog boxes: Estimation Button STATGRAPHICS Rev. 1/10/005 These fields control the iterations used in: 1. The classical method of factor extraction. Estimated communalities are revised until the proportional change in their sum is less than the stopping criterion, or the maximum iterations is reached.. Rotation of the factor loadings. The stopping criteria apply to the variance of the squared elements on the diagonal of the factor loading matrix. Communalities Button When using the classical estimation method, you may specify a column containing the communalities instead of having the program estimate them iteratively. 005 by StatPoint, Inc. Factor Analysis - 7

8 STATGRAPHICS Rev. 1/10/005 Scree Plot The Scree Plot can be very helpful in determining the number of factors to extract. By default, it plots the size of the eigenvalues corresponding to each of the p possible factors: 8 Scree Plot Eigenvalue Factor A horizontal line is added at the minimum eigenvalue specified on the Analysis Options dialog box. In the plot above, note that only the first factors have large eigenvalues. Pane Options Plot: value plotted on the vertical axis. 005 by StatPoint, Inc. Factor Analysis - 8

9 STATGRAPHICS Rev. 1/10/005 Extraction Statistics The Extraction Statistics pane shows the estimated value of the coefficients l for each factor extracted, before any rotation is applied: Factor Loading Matrix Before Rotation Factor Factor 1 Engine Size Horsepower Fueltank Passengers Length Wheelbase Width U Turn Space Rear seat Luggage Weight Estimated Specific Variable Communality Variance Engine Size Horsepower Fueltank Passengers Length Wheelbase Width U Turn Space Rear seat Luggage Weight Also displayed are the communalities and the specific variances. The weights within each column often have interesting interpretations. In the example, note that the weights in the first column are all approximately the same. This implies that the first component is basically an average of all of the input variables. The second component is weighted most heavily in a positive direction on the number of Passengers, the Rear Seat room, and the amount of Luggage space, and in a negative direction on Horsepower. It likely differentiates amongst the different types of vehicles. Note also that U Turn Space and Luggage have a larger specific variance than the others, implying that they are not as well accounted for by the two extracted factors. 005 by StatPoint, Inc. Factor Analysis - 9

10 STATGRAPHICS Rev. 1/10/005 Rotation Statistics The Rotation Statistics pane shows the estimated value of the coefficients l after the requested rotation is applied: Factor Loading Matrix After Varimax Rotation Factor Factor 1 Engine Size Horsepower Fueltank Passengers Length Wheelbase Width U Turn Space Rear seat Luggage Weight Estimated Specific Variable Communality Variance Engine Size Horsepower Fueltank Passengers Length Wheelbase Width U Turn Space Rear seat Luggage Weight Note that the rotation has substantially decreased the loading of Passengers, Rear seat, and Luggage on the first factor and made them the dominant variables in the second factor. The second factor seems to distinguish large family vehicles such as minivans and SUV s from the other automobiles. 005 by StatPoint, Inc. Factor Analysis - 10

11 STATGRAPHICS Rev. 1/10/005 D and 3D Scatterplots These plots display the values of or 3 selected factors for each of the n cases, after rotation Scatterplot Factor Dodge Stealth Factor 1 It is useful to examine any points far away from the others, such as the highlighted Dodge Stealth, which has a very low value for the second factor. An interesting variation of this plot is one in which the variables are coded according to another column, such as the type of vehicle: Plot of SCORE_ vs SCORE_1 SCORE_ SCORE_1 Type Compact Large Midsize Small Sporty To produce the above plot: 1. Press the Save Results button and save the Factor Scores to new columns of the datasheet.. Select the X-Y Plot procedure from the top menu and input the new columns. 3. Select Analysis Options and specify Type in the Point Codes field. It is now clear that the first factor is related to the size of the vehicle, while the second factor separates the sporty cars from the others. 005 by StatPoint, Inc. Factor Analysis - 11

12 Pane Options STATGRAPHICS Rev. 1/10/005 Specify the factors to plot on each axis. Factor Scores The Factor Scores pane displays the values of the rotated factor scores for each of the n cases. Table of Factor Scores Factor Factor Row Label 1 1 Integra Legend i Century LeSabre Roadmaster Riviera The factor scores show where each observation falls with respect to the extracted factors. Factor Score Coefficients The table of Factor Score Coefficients shows the coefficients used to create the factor scores from the original variables. Factor Score Coefficients Factor Factor 1 Engine Size Horsepower Fueltank Passengers Length Wheelbase Width U Turn Space Rear seat Luggage Weight If the sample covariance matrix S has been factored, then the coefficients are the leading terms multiplying the deviation of each variable from its mean in 005 by StatPoint, Inc. Factor Analysis - 1

13 ˆ ˆ 1 f j = L S ( x j x) STATGRAPHICS Rev. 1/10/005 (6) If the correlation matrix R has been factored, then the coefficients are the leading terms multiplying the standardized values of each variable according in fˆ = Lˆ R (7) 1 j z j D and 3D Factor Plots The Factor Plots show the location of each variable in the space of or 3 selected factors: Plot of Factor Loadings Rear seat Passengers Luggage Factor Wheelbase Length U Turn SpaceWidth Engine Size Weight Fueltank 0 Horsepower Factor 1 Variables furthest from the reference lines at 0 make the largest contribution to the factors. Save Results The following results may be saved to the datasheet: 1. Eigenvalues the m eigenvalues.. Factor Matrix m columns, each containing p estimates of the coefficients l before rotation. 3. Rotated Factor Matrix m columns, each containing p estimates of the coefficients l after rotation. 4. Transition Matrix the m by m matrix that multiplies the original factor loadings to yield the rotated factor loadings. 5. Communalities the p estimated communalities after rotation. 6. Specific variances the p specific variances after rotation. 005 by StatPoint, Inc. Factor Analysis - 13

14 STATGRAPHICS Rev. 1/10/ Factor Scores m columns, each containing n values corresponding to the extracted factors. 8. Factor Score Coefficients m columns, each containing the p values of the factor score coefficients. 005 by StatPoint, Inc. Factor Analysis - 14