6/17/2016. Introduction

Transcription

1 FACTOR ANALYTIC METHODS: CONCEPTUAL FOUNDATIONS FOR EXPLORING THE STRUCTURE OF INTELLIGENCE TESTS AND RELATED ABILITY MEASURES Ryan J. McGill, Ph.D., BCBA-D, NCSP Assistant Professor of Psychology Co-Director Applied Psychometric Laboratory Texas Woman s University Inaugural TWU Applied Psychometric Laboratory Symposium, Denton TX Factor analysis rooted in abstract statistical theory and based on an attempt to discover underlying structure in large matrices of data, is, to put it bluntly, a bitch Gould (1981) Introduction In many disciplines we study phenomena or constructs that cannot be directly measured (self-esteem, personality, intelligence) It often is required to take multiple observations for each case, and in the end we may have more data than can be readily interpreted Items are representations of underlying or latent factors. We want to know what these factors are We have an idea of the phenomena that a set of items represent (construct validity). Because of this, we ll want to reduce them to a smaller set of factors 1

2 Purpose of Factor Analysis To find underlying latent constructs As manifested in multiple items/variables To assess the association between multiple factors To produce usable scores that reflect critical aspects of any complex phenomenon As an end in itself and a major step toward creating error free measures Basic Concepts If two items are highly correlated They may represent the same phenomenon If they tell us about the same underlying variance, combining them to form a single measure is reasonable for two reasons Parsimony Reduction in Error FA looks for the phenomena underlying the observed variance and covariance in a set of variables. These phenomena are called factors. Mathematical artifacts versus latent dimensions. PCA/FA While often used similarly, PCA and FA are distinct from one another Principal Components Analysis Extracts all the components underlying a set of variables The number of components = the number of variables Completely explains the variance in each variable Default in SPSS (Caveat emptor) Factor Analysis Analyzes only the shared variance Error is estimated apart from shared variance 2

3 FA vs. PCA conceptually FA produces factors; PCA produces components Factors cause variables; components are aggregates of the variables The underlying causal model is fundamentally distinct between the two Some do not consider PCA as part of the FA family* FA PCA I1 I2 I3 I1 I2 I3 Contrasting the underlying models PCA Extraction is the process of forming PCs as linear combinations of the measured variables: PC 1 = b 11 X 1 + b 21 X b k1 X k PC 2 = b 12 X 1 + b 22 X b k2 X k PC f = b 1f X 1 + b 2f X b kf X k Common factor model Each measure X has two contributing sources of variation: the common factor ξ and the specific or unique factor δ: X 1 = λ 1 ξ + δ 1 X 2 = λ 2 ξ + δ 2 X f = λ f ξ + δ f Terminology Manifest variable or measured variable Commonality Uniqueness Error variance (sources) First-order Second-order 3

4 Common Factor Model Unidimensional Model Orthogonal Factors Model 4

5 Correlated Factors Model Indirect Hierarchical Model Direct Hierarchical (Bifactor) Model 5

6 FA and Construct Validity Constructs are either univocal or multidimensional Ability measures should measure the relevant dimensions thought to underlie a construct. Exploratory Factor Analysis (EFA) Confirmatory Factor Analysis (CFA) EFA/CFA EFA Let s the data speak for themselves CFA Compare various models against one another to determine model that best explains internal structure. EFA and CFA are different methods that are designed to answer different empirical questions when they are in agreement better confidence in viability of the model (Gorsuch, 1983) CFA is not as robust ad advertised. All structural models, no matter how well fitting, must be evaluated against external criteria (Lubinski & Dawis, 1992) FA validity (Messick, 1988) FA Trends Trends for IQ tests (Frazier & Youngstrom, 2007) past 55 years-increasing number of factors claimed to be measures by IQ tests. Overuse of CFA, liberal factor extraction criteria. 6

7 Are we Overfactoring? EFA Steps What variables to include 3-6 variables per factor, avoid low communality and low reliability, variables not linearly dependent yet provide good representation of domain. What subjects to include Sample size is dependent on number of factors (minimum guidelines). Examine data to be analyzed Missing data, outliers, univariate and multivariate normality Is FA viable? FA input, Corelations >.30, Bartlett s test of Sphericity (is the correlation matrix random), KMO (sampling adequacy, >.60) EFA model PCA versus common factor analysis EFA steps Factor extraction method Principal factors versus maximum likelihood How many factors to retain Eigenvalues > 1, ccree test (visual, SEscree), parallel analysis, minimum average partials, theoretical convergence or parsimony Factor rotation Orthogonal (varimax, quartimax, equimax) uncorrelated factors Oblique (oblimin, promax) correlated factors Interpreting results Simple structure, theoretical convergence and parsimony, factor identification and naming 7

8 Reification Fallacy Just because a factor is named does not imply correct label Example: KABC-II Gv Hypothetical construct does not always correspond to a real thing (Kline, 2011). EFA Example: McGill & Spurgin (2016) Correlation Matrix 8

9 Extraction Test Results Criteria Number of Factors Suggested Eigenvalue >1 2 Visual Scree 2 SEscree 1 HPA 1 MAP 1 Publisher Theory 4 Scree Plot 5 4 Random Data Eigenvalues 3 2 Actual Factors First-Run EFA (4 Factors) 9

10 First-Run EFA (4 Factors) Factor Identification: First-Order Loadings Higher-Order Structure 10

11 Higher-Order Structure First-order EFA is equivalent to correlated factors model Moderate to large factor correlations imply higher-order dimension that must be explicated. Failure to do so will overestimate importance of first-order factors at the expense of the higher-order dimension (Watkins, 2006). How do we account for HO structure in EFA? Schmid-Leiman Transformation (1957) Second Run EFA (4 Factors) 11

12 Higher-Order EFA Results (4 Factors) KABC-II Luria Model Variance EFA Summary Multiple extraction criteria required: HPA and MAP although this has been debated (Keith et al., 2016) First-run insufficient for examining cognitive data, must explicate higher-order structure. SL transformation is recommended for sourcing variance and providing results that can be used to aid clinical interpretation. Just identified factors and the danger of weak loadings. Common factor must have at least two indicators: saliency versus alignment. Bifactor modeling in EFA 12

13 CFA Features The number of factors and the observed variables (indicators) that load on each construct (factor or latent variable) are specified in advance of the analysis Generally indicators load on only one construct (factor) Each indicator is represented as having two causes, a single factor that it is suppose to measure and all other unique sources of variance represented by measurement error CFA Features The measurement error terms are independent of each other and of the factors All associations between factors are unanalyzed Advantages of CFA Test nested models Test relationships among error variables or constraints on factor loadings (e.g., equality) Test equivalent measurement models in two or more groups or at two or more times. 13

14 Disadvantages of CFA Un-modeled complexity Viability of measurement models Modifications and capitalizing on chance Fit statistics (Constrained CFA) Confirmation bias and unethical practices HO structure difficult to explicate with less than three common factors and/or factors that are just identified CFA Model Identification Identification pertains to the difference between the number of estimated model parameters and the number of pieces of information in the variance/covariance matrix. Every latent variable needs to have its scale identified. Fix one loading of an observed variable on the latent variable to one Fix the variance of the latent variable to one Identification of CFA Sufficient : At least three (3) indicators per factor to make the model identified Two-indicator rule prone to estimation problems (esp. with small sample size) 14

15 Problems in estimation of CFA Heywood cases negative variance estimated or correlations > 1. Ratio of the sample size to the free parameters 10:1 ( better 20:1) Nonnormality affects ML estimation Suggestions by March and Hau(1999) when sample size is small: indicators with high standardized loadings( >0.6) constrain the factor loadings CFA Programs AMOS Lisrel EQS Mplus Lavaan Evaluating CFA Results Model fit Parameter estimates and standardized loadings Heywood cases 15

16 Common Fit Statistics Must evaluate fit using multiple indicators!!! Chi-square Lower values imply better fit Statistical significance can also be assessed Zero indicates completely saturated model Bias for complexity Incremental indices CFI, NFI, TLI Values of.95 or greater desired Absolute indices SRMR/RMSEA Lower values better,.05 or less preferred. Other fit indices AIC Lower values better Only when comparing two or more models Example: WISC-IV CFA (Canivez, 2016) Correlated Factors Model 16

17 IH Model Bifactor Model Sources of Variance/Model-Based Reliability 17

18 CFA Summary Value of CFA is only as good as the quality of the analyses: garbage in/garbage out CFA tends to favor complexity IH versus bifactor model Model-based reliability: alpha versus omega Example: KABC-II HO Model title: four direct hierarchical factor model KABC-II 8 Subtests (Ages 13-18) data: file = "C:\Users\laptop\Desktop\KABC.dat"; nobservations = 883;!V1-AT V2-BC V3-NR V4-PC V5-RL V6-RO V7-SC V8-WO variable: names = v1-v8; analysis: estimator is ML; model:! First-order factors L by v1 v5;! learning factor SQ by v3 v8;! sequential processing factor SM by v2 v6;! simultaneous processing factor P by v4 v7;! planning factor! Second-order factor G by L SQ SM P; output: STDYX MODINDICES RESIDUAL; Model Fit Statistics 18

19 Standardized Model Results Two-Tailed Estimate S.E. Est./S.E. P-Value L BY V V SQ BY V V SM BY V V P BY V V G BY L SQ SM P Factor Analysis Summary Factor analysis is a versatile analytic technique that is useful for uncovering the structure of latent dimensions EFA versus CFA Un-modeled complexity Bayesian alternatives Multidimensionality, clinical interpretation HO versus Bifactor model Method variance Contact Information: Ryan J. McGill, Ph.D., BCBA-D, NCSP Department of Psychology and Philosophy Texas Woman s University Professional website: rmcgill@twu.edu 19