Factor Analysis and Structural equation modelling Herman Adèr Previously: Department Clinical Epidemiology and Biostatistics, VU University medical center, Amsterdam Stavanger July 4 13, 2006 Herman Adèr 1
Overview 1 E C 2 EFA 3 CFA 4 SEM Exploratory Confirmatory methods Exploratory Factor Analysis Confirmatory Factor Analysis Structural Equation Modelling Herman Adèr 2
Overview 1 E C 2 EFA 3 CFA 4 SEM Exploratory Confirmatory methods Exploratory Factor Analysis Confirmatory Factor Analysis Structural Equation Modelling Herman Adèr 2
Overview 1 E C 2 EFA 3 CFA 4 SEM Exploratory Confirmatory methods Exploratory Factor Analysis Confirmatory Factor Analysis Structural Equation Modelling Herman Adèr 2
Overview 1 E C 2 EFA 3 CFA 4 SEM Exploratory Confirmatory methods Exploratory Factor Analysis Confirmatory Factor Analysis Structural Equation Modelling Herman Adèr 2
Overview 1 E C 2 EFA 3 CFA 4 SEM Exploratory Confirmatory methods Exploratory Factor Analysis Confirmatory Factor Analysis Structural Equation Modelling Herman Adèr 2
Part VI Factor analysis and Herman Adèr Factor Analysis and SEM 3
Detective Judge Herman Adèr Factor Analysis and SEM 5
Characterization of Tukey (1977) Exploratory data analysis is detective in character. Confirmatory data analysis is judicial or quasi-judicial in character... Unless the detective finds the clues, judge or jury has nothing to consider. Unless exploratory data analysis uncovers indications, usually quantitative ones, there is likely to be nothing for confirmatory data analysis to consider. On the other hand: Exploratory data analysis can never be the whole story, but nothing else can serve as the foundation stone the first step. Herman Adèr Factor Analysis and SEM 7
Characterization of Tukey (1977) Exploratory data analysis is detective in character. Confirmatory data analysis is judicial or quasi-judicial in character... Unless the detective finds the clues, judge or jury has nothing to consider. Unless exploratory data analysis uncovers indications, usually quantitative ones, there is likely to be nothing for confirmatory data analysis to consider. On the other hand: Exploratory data analysis can never be the whole story, but nothing else can serve as the foundation stone the first step. Herman Adèr Factor Analysis and SEM 7
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure F = M U E, of which M indicate the meaningful factors, U so-called unique factors (factors on which only one item loads) and E error factors. Questions we try to settle using EFA 1 How many meaningful dimensions are present? 2 What is the structure of those dimensions? Herman Adèr Factor Analysis and SEM 8
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure F = M U E, of which M indicate the meaningful factors, U so-called unique factors (factors on which only one item loads) and E error factors. Questions we try to settle using EFA 1 How many meaningful dimensions are present? 2 What is the structure of those dimensions? Herman Adèr Factor Analysis and SEM 8
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Data set: { } D = Subject Items Exploratory Factor Analysis factor structure determine meaningful factors unrotated Three steps 1 Determine the meaningful factors rotate factor structure k meaningful factors 2 Rotate 3 Interpret the factor structure determine factor meaning final factor solution Herman Adèr Factor Analysis and SEM 9
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure determine meaningful factors: scree plot eigenvalues larger than 1 GoF change significant between solutions total variance explained more than 50% select factors with intelligible loading patterns and name them communalities reliabilities doubtful Herman Adèr Factor Analysis and SEM 10
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure CES-D: A multidimensional scale to assess Depression FACTOR /VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5 cesd6 cesd7 cesd8 cesd9 cesd10 cesd11 cesd12 cesd13 cesd14 cesd15 cesd16 cesd17 cesd18 cesd19 cesd20 /MISSING LISTWISE /ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5 cesd6 cesd7 cesd8 cesd9 cesd10 cesd11 cesd12 cesd13 cesd14 cesd15 cesd16 cesd17 cesd18 cesd19 cesd20 /PRINT all /FORMAT BLANK(.30) /PLOT EIGEN /CRITERIA MINEIGEN(0) ITERATE(100) factors(3) /EXTRACTION pc /ROTATION norotate. Herman Adèr Factor Analysis and SEM 12
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure CES-D: A multidimensional scale to assess Depression FACTOR /VARIABLES cesd1 cesd2 cesd3 cesd4 cesd5 cesd6 cesd7 cesd8 cesd9 cesd10 cesd11 cesd12 cesd13 cesd14 cesd15 cesd16 cesd17 cesd18 cesd19 cesd20 /MISSING LISTWISE /ANALYSIS cesd1 cesd2 cesd3 cesd4 cesd5 cesd6 cesd7 cesd8 cesd9 cesd10 cesd11 cesd12 cesd13 cesd14 cesd15 cesd16 cesd17 cesd18 cesd19 cesd20 /PRINT all /FORMAT BLANK(.30) /PLOT EIGEN /CRITERIA MINEIGEN(0) ITERATE(100) factors(3) /EXTRACTION pc /ROTATION norotate. Herman Adèr Factor Analysis and SEM 12
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure 10 Scree Plot 8 6 4 Eigenvalue 2 0 1 3 5 7 9 11 13 15 17 19 2 4 6 8 10 12 14 16 18 20 Component Number Herman Adèr Factor Analysis and SEM 13
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Component 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Initial Eigenvalues Total % of Variance Cumulative % 8.516 42.578 42.578 1.775 8.876 51.454 1.229 6.147 57.600 1.040 5.201 62.802.944 4.719 67.520.767 3.836 71.356.738 3.689 75.045.684 3.419 78.465.606 3.030 81.495.577 2.886 84.381.508 2.542 86.923.442 2.211 89.134.426 2.129 91.263.339 1.697 92.960.318 1.592 94.552.286 1.429 95.981.274 1.371 97.352.202 1.010 98.362.178.891 99.253.149.747 100.000 Extraction Method: Principal Component Analysis. Herman Adèr Factor Analysis and SEM 14
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Interpretation of the unrotated factor structure Component 1 2 3 4 5 6 CESD1.525.372 -.373 CESD2.757 -.318 CESD3.802.389 CESD4 -.573.538 CESD5.687.336 CESD6 CESD7.689.726.349 CESD8 -.635.531 CESD9.721 -.353 CESD10.672.398 CESD11.623.415 CESD12 -.690.489 CESD13.599 -.513 CESD14.710 CESD15.432.490 -.627 CESD16 -.758.383 CESD17.657.500 CESD18.763.349 CESD19.482 -.447 -.376 CESD20.784.359 Extraction Method: Principal Component Analysis. Herman Adèr Factor Analysis and SEM 15
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Communalities: four factors and all factors Communalities Initial Extraction CESD1 1.000.418 CESD2 1.000.751 CESD3 1.000.798 CESD4 1.000.683 CESD5 1.000.492 CESD6 1.000.526 CESD7 1.000.533 CESD8 1.000.763 CESD9 1.000.695 CESD10 1.000.543 CESD11 1.000.597 CESD12 1.000.734 CESD13 1.000.655 CESD14 1.000.546 CESD15 1.000.442 CESD16 1.000.721 CESD17 1.000.699 CESD18 1.000.709 CESD19 1.000.631 CESD20 1.000.624 Extraction Method: Principal Component Analysis. Communalities Initial Extraction CESD1 1.000.966 CESD2 1.000.970 CESD3 1.000.868 CESD4 1.000.817 CESD5 1.000.919 CESD6 1.000.912 CESD7 1.000.830 CESD8 1.000.834 CESD9 1.000.812 CESD10 1.000.883 CESD11 1.000.896 CESD12 1.000.841 CESD13 1.000.861 CESD14 1.000.829 CESD15 1.000.961 CESD16 1.000.829 CESD17 1.000.828 CESD18 1.000.793 CESD19 1.000.963 CESD20 1.000.773 Extraction Method: Principal Component Analysis. Herman Adèr Factor Analysis and SEM 16
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Communalities: two and three factors Communalities Initial Extraction CESD1 1.000.414 CESD2 1.000 7.631E-02 CESD3 1.000.795 CESD4 1.000.617 CESD5 1.000.481 CESD6 1.000.489 CESD7 1.000.531 CESD8 1.000.686 CESD9 1.000.544 CESD10 1.000.539 CESD11 1.000.399 CESD12 1.000.716 CESD13 1.000.389 CESD14 1.000.540 CESD15 1.000.427 CESD16 1.000.721 CESD17 1.000.434 CESD18 1.000.587 CESD19 1.000.291 CESD20 1.000.615 Extraction Method: Principal Component Analysis. Communalities Initial Extraction CESD1 1.000.418 CESD2 1.000.650 CESD3 1.000.796 CESD4 1.000.639 CESD5 1.000.491 CESD6 1.000.517 CESD7 1.000.532 CESD8 1.000.720 CESD9 1.000.669 CESD10 1.000.541 CESD11 1.000.571 CESD12 1.000.733 CESD13 1.000.392 CESD14 1.000.546 CESD15 1.000.437 CESD16 1.000.721 CESD17 1.000.449 CESD18 1.000.587 CESD19 1.000.490 CESD20 1.000.621 Extraction Method: Principal Component Analysis. Herman Adèr Factor Analysis and SEM 17
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Goodness-of-fit criterium # Facts GoF χ 2 Df χ 2 change Df 1 444.399 170 χ 2 α=.05 2 278.177 151 166.222 19 30.144 3 218.422 133 59.755 18 28.869 4 170.845 116 47.577 17 27.587 5 126.361 100 44.484 16 26.296 As one can see by comparing the 0.05 threshold χ 2 in the last column Herman Adèr Factor Analysis and SEM 18
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Unrotated Rotated Factor Factor Item I II III I II III CESD01 + + + CESD02 + + CESD03 + + + CESD04 + CESD05 + + + CESD06 + + + CESD07 + + + CESD08 + CESD09 + + + CESD10 + + CESD11 + + + + CESD12 + CESD13 + + CESD14 + + CESD15 + + + CESD16 + CESD17 + + + CESD18 + + + CESD19 + + CESD20 + + + Herman Adèr Factor Analysis and SEM 19
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Interpretation of table I Unrotated Rotated Factor Factor Item I II III I II III CESD01 + + + CESD02 + + CESD03 + + + CESD04 + CESD05 + + + CESD06 + + + CESD07 + + + CESD08 + CESD09 + + + CESD10 + + CESD11 + + + + CESD12 + CESD13 + + CESD14 + + CESD15 + + + CESD16 + CESD17 + + + CESD18 + + + CESD19 + + CESD20 + + + All items with high positive loadings on the unrotated Factor I still load on this factor after rotation, but some of them (5, 6, 7, 9, 17, 18 and 20) also load on factor II. This makes that Factor I is now separated into two sublists, of which the above could be called the pure depression scale while (1, 3, 10, 11, 13, 14, 15, 19) includes a sublist indicating less severe symptoms of depression. Herman Adèr Factor Analysis and SEM 20
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Interpretation of table II Unrotated Rotated Factor Factor Item I II III I II III CESD01 + + + CESD02 + + CESD03 + + + CESD04 + CESD05 + + + CESD06 + + + CESD07 + + + CESD08 + CESD09 + + + CESD10 + + CESD11 + + + + CESD12 + CESD13 + + CESD14 + + CESD15 + + + CESD16 + CESD17 + + + CESD18 + + + CESD19 + + CESD20 + + + Factor II now also contains all the items that loaded negatively on Factor I before (4, 8, 12 and 16). We called this a factor that measures: general attitude towards life. Herman Adèr Factor Analysis and SEM 21
Questions and answers Three steps Determining the meaningful factors Rotation Interpretation of the factor structure Interpretation of table III Unrotated Rotated Factor Factor Item I II III I II III CESD01 + + + CESD02 + + CESD03 + + + CESD04 + CESD05 + + + CESD06 + + + CESD07 + + + CESD08 + CESD09 + + + CESD10 + + CESD11 + + + + CESD12 + CESD13 + + CESD14 + + CESD15 + + + CESD16 + CESD17 + + + CESD18 + + + CESD19 + + CESD20 + + + After rotation, Factor III now contains only item 2 ( I did not feel like eating: my appetite was poor ) and item 11 ( My sleep was restless ), suggesting that it is a factor that has to do with physical aspects of depression. Herman Adèr Factor Analysis and SEM 22
Example of EQS input Example of EQS input Conclusions of EFA and CFA combined Example of EQS input /TITLE Cesd; /SPECIFICATIONS VARIABLES=6; CASES= 200; DATAFILE= cesd.ess ; MATRIX=RAW; ME = ML; /EQUATIONS V1 = 2 * F1 + E1; V2 = 2 * F1 + E2; V3 = 2 * F2 + E3; V4 = 2 * F2 + E4; V5 = 2 * F3 + E5; V6 = 2 * F3 + E6; F1 = 2 * F2 + 2*F3+ D1; /VARIANCES E1 to E10 = 0.2*; D1, D2, D3 = 0.2*; /COVARIANCES F2,F3 =.5*; /END Herman Adèr Factor Analysis and SEM 23
Example of EQS input Example of EQS input Conclusions of EFA and CFA combined CES-D: Conclusions of the CFA The varimax solution is not fully confirmed: item 5, 6, 7, 18 and 20 have negligible coefficients on the second factor. The assumption of a orthogonal factor structure is unfounded: The correlations between the factors are (F1, F2) = 0.241; (F1, F3) = 0.382 and (F2, F3) = 0.025. Herman Adèr Factor Analysis and SEM 24
Example of EQS input Example of EQS input Conclusions of EFA and CFA combined CES-D: Conclusions of the CFA The varimax solution is not fully confirmed: item 5, 6, 7, 18 and 20 have negligible coefficients on the second factor. The assumption of a orthogonal factor structure is unfounded: The correlations between the factors are (F1, F2) = 0.241; (F1, F3) = 0.382 and (F2, F3) = 0.025. Herman Adèr Factor Analysis and SEM 24
Example of EQS input Example of EQS input Conclusions of EFA and CFA combined Conclusions of EFA and CFA combined 1 The first factor is a general depression factor (item 1, 3, 5, 6, 7, 10, 13, 14, 15, 18, 19 and 20) 2 The second factor contains the positively formulated items (4, 8, 9, 12, 16, 17). It represents a general attitude towards life. 3 The third factor contains items that have to do with physical aspects of depression. Herman Adèr Factor Analysis and SEM 25
Example: Post-traumatic stress disorder Example of a postulated structure Draw pictures! Herman Adèr Factor Analysis and SEM 26
Example: Post-traumatic stress disorder Example of a postulated structure Draw pictures! Herman Adèr Factor Analysis and SEM 26
Example: Post-traumatic stress disorder Example of a postulated structure Draw pictures! Herman Adèr Factor Analysis and SEM 26
Postulated structure Structure found Herman Adèr Factor Analysis and SEM 27
Recommended literature Factor analysis: Friendly (1995) (http://www.psych.yorku.ca/lab/psy6140/fa/factorbi.htm) Principal component analysis: Jackson (1991) : Bollen (1989), Kaplan (2000) Herman Adèr Factor Analysis and SEM 28
Summary Herman Adèr Factor Analysis and SEM 29
Summary In factor analysis the contrast between exploratory and confirmatory approaches is quite clear: the available techniques are different although related. can well be performed with principal component analysis, possibly combined with Maximum likelihood factor analysis. PCA is first used to determine the meaningful factors, MLFA for the final factor structure. Structural Equation Modelling makes it possible to analyze a research problem that has been represented as a diagram. Herman Adèr Factor Analysis and SEM 30
Bollen, K. A. (1989). Structural equations with latent variables. New York: John Wiley and Sons. Friendly, M. (1995). Annotated Factor Analysis Bibliography. Jackson, J. E. (1991). A user s guide to principal components. New York: Wiley. Kaplan, D. (2000). Structural Equation Modeling. Foundations and Extensions. Thousand Oaks London New Delhi: Sage Publications. Tukey, J. W. (1977). Exploratory data analysis. Reading, MA: Addison Wesley. Herman Adèr Factor Analysis and SEM 31