How To Understand Multivariate Models

Transcription

1 Neil H. Timm Applied Multivariate Analysis With 42 Figures Springer

2 Contents Preface Acknowledgments List of Tables List of Figures vii ix xix xxiii 1 Introduction Overview Multivariate Models and Methods Scope of the Book 3 2 Vectors and Matrices Introduction Vectors, Vector Spaces, and Vector Subspaces 7 a. Vectors 7 b. Vector Spaces 8 c. Vector Subspaces Bases, Vector Norms, and the Algebra of Vector Spaces 12 a. Bases 13 b. Lengths, Distances, and Angles 13 c. Gram-Schmidt Orthogonalization Process 15 d. Orthogonal Spaces 17 e. Vector Inequalities, Vector Norms, and Statistical Distance 21

3 xii Contents 2.4 Basic Matrix Operations 25 a. Equality, Addition, and Multiplication of Matrices 26 b. Matrix Transposition 28 c. Some Special Matrices 29 d. Trace and the Euclidean Matrix Norm 30 e. Kronecker and Hadamard Products 32 f. Direct Sums 35 g. The Vec(-) and Vech(-) Operators Rank, Inverse, and Determinant 41 a. Rank and Inverse 41 b. Generalized Inverses 47 c. Determinants Systems of Equations, Transformations, and Quadratic Forms 55 a. Systems of Equations 55 b. Linear Transformations 61 c. Projection Transformations 63 d. Eigenvalues and Eigenvectors 67 e. Matrix Norms 71 f. Quadratic Forms and Extrema 72 g. Generalized Projectors Limits and Asymptotics 76 3 Multivariate Distributions and the Linear Model Introduction Random Vectors and Matrices The Multivariate Normal (MVN) Distribution 84 a. Properties of the Multivariate Normal Distribution 86 b. Estimating /x and E 88 c. The Matrix Normal Distribution The Chi-Square and Wishart Distributions 93 a. Chi-Square Distribution 93 b. The Wishart Distribution Other Multivariate Distributions 99 a. The Univariate t and F Distributions 99 b. Hotelling's T 2 Distribution 99 c. The Beta Distribution 101 d. Multivariate t, F, and / 2 Distributions The General Linear Model 106 a. Regression, ANOVA, and ANCOVA Models 107 b. Multivariate Regression, MANOVA, and MANCOVA Models c. The Seemingly Unrelated Regression (SUR) Model 114 d. The General MANOVA Model (GMANOVA) Evaluating Normality Tests of Covariance Matrices 133 a. Tests of Covariance Matrices 133

4 Contents xiii b. Equality of Covariance Matrices 133 c. Testing for a Specific Covariance Matrix 137 d. Testing for Compound Symmetry 138 e. Tests of Sphericity 139 f. Tests of Independence 143 g. Tests for Linear Structure Tests of Location 149 a. Two-Sample Case, Ei = E 2 = 149 b. Two-Sample Case, Ei ^ E c. Two-Sample Case, Nonnormality 160 d. Profile Analysis, One Group 160 e. Profile Analysis, Two Groups 165 f. Profile Analysis, Ei ^ E Univariate Profile Analysis 181 a. Univariate One-Group Profile Analysis 182 b. Univariate Two-Group Profile Analysis Power Calculations 182 Multivariate Regression Models Introduction Multivariate Regression 186 a. Multiple Linear Regression 186 b. Multivariate Regression Estimation and Testing Hypotheses 187 c. Multivariate Influence Measures 193 d. Measures of Association, Variable Selection and Lack-of-Fit Tests e. Simultaneous Confidence Sets for a New Observation y new and the Elements of B 204 f. Random X Matrix and Model Validation: Mean Squared Error of Prediction in Multivariate Regression 206 g. Exogeniety in Regression Multivariate Regression Example One-Way MANOVA and MANCOVA 218 a. One-Way MANOVA 218 b. One-Way MANCOVA 225 c. Simultaneous Test Procedures (STP) for One-Way MANOVA /MANCOVA One-Way MANOVA/MANCOVA Examples 234 a. MANOVA (Example 4.5.1) 234 b. MANCOVA (Example 4.5.2) MANOVA/MANCOVA with Unequal E, or Nonnormal Data One-Way MANOVA with Unequal E, Example Two-Way MANOVA/MANCOVA 246 a. Two-Way MANOVA with Interaction 246 b. Additive Two-Way MANOVA 252 c. Two-Way MANCOVA 256

5 xiv Contents d. Tests of Nonadditivity Two-Way MANOVA/MANCOVA Example 257 a. Two-Way MANOVA (Example 4.9.1) 257 b. Two-Way MANCOVA (Example 4.9.2) Nonorthogonal Two-Way MANOVA Designs 264 a. Nonorthogonal Two-Way MANOVA Designs with and Without Empty Cells, and Interaction 265 b. Additive Two-Way MANOVA Designs With Empty Cells Unbalance, Nonorthogonal Designs Example Higher Ordered Fixed Effect, Nested and Other Designs Complex Design Examples 276 a. Nested Design (Example ) 276 b. Latin Square Design (Example ) Repeated Measurement Designs 282 a. One-Way Repeated Measures Design 282 b. Extended Linear Hypotheses Repeated Measurements and Extended Linear Hypotheses Example a. Repeated Measures (Example ) 294 b. Extended Linear Hypotheses (Example ) Robustness and Power Analysis for MR Models Power Calculations Power.sas Testing for Mean Differences with Unequal Covariance Matrices Seemingly Unrelated Regression Models Introduction The SUR Model 312 a. Estimation and Hypothesis Testing 312 b. Prediction Seeming Unrelated Regression Example The CGMANOVA Model CGMANOVA Example The GMANOVA Model 320 a. Overview 320 b. Estimation and Hypothesis Testing 321 c. Test of Fit 324 d. Subsets of Covariates 324 e. GMANOVA vs SUR 326 f. Missing Data GMANOVA Example 327 a. One Group Design (Example 5.7.1) 328 b. Two Group Design (Example 5.7.2) Tests of Nonadditivity Testing for Nonadditivity Example Lack of Fit Test Sum of Profile Designs 338

6 Contents xv 5.12 The Multivariate SUR (MSUR) Model Sum of Profile Example Testing Model Specification in SUR Models Miscellanea 348 Multivariate Random and Mixed Models Introduction Random Coefficient Regression Models 352 a. Model Specification 352 b. Estimating the Parameters 353 c. Hypothesis Testing Univariate General Linear Mixed Models 357 a. Model Specification 357 b. Covariance Structures and Model Fit 359 c. Model Checking 361 d. Balanced Variance Component Experimental Design Models 366 e. Multilevel Hierarchical Models 367 f. Prediction Mixed Model Examples 369 a. Random Coefficient Regression (Example 6.4.1) 371 b. Generalized Randomized Block Design (Example 6.4.2) 376 c. Repeated Measurements (Example 6.4.3) 380 d. HLM Model (Example 6.4.4) Mixed Multivariate Models 385 a. Model Specification 386 b. Hypothesis Testing 388 c. Evaluating Expected Mean Square 391 d. Estimating the Mean 392 e. Repeated Measurements Model Balanced Mixed Multivariate Models Examples 394 a. Two-way Mixed MANOVA 395 b. Multivariate Split-Plot Design Double Multivariate Model (DMM) Double Multivariate Model Examples 403 a. Double Multivariate MANOVA (Example 6.8.1) 404 b. Split-Plot Design (Example 6.8.2) Multivariate Hierarchical Linear Models Tests of Means with Unequal Covariance Matrices 417 Discriminant and Classification Analysis Introduction Two Group Discrimination and Classification 420 a. Fisher's Linear Discriminant Function 421 b. Testing Discriminant Function Coefficients 422 c. Classification Rules 424

7 xvi Contents d. Evaluating Classification Rules Two Group Discriminant Analysis Example 429 a. Egyptian Skull Data (Example 7.3.1) 429 b. Brain Size (Example 7.3.2) Multiple Group Discrimination and Classification 434 a. Fisher's Linear Discriminant Function 434 b. Testing Discriminant Functions for Significance 435 c. Variable Selection 437 d. Classification Rules 438 e. Logistic Discrimination and Other Topics Multiple Group Discriminant Analysis Example Principal Component, Canonical Correlation, and Exploratory Factor Analysis Introduction Principal Component Analysis 445 a. Population Model for PCA 446 b. Number of Components and Component Structure 449 c. Principal Components with Covariates 453 d. Sample PCA 455 e. Plotting Components 458 f. Additional Comments 458 g. Outlier Detection Principal Component Analysis Examples 460 a. Test Battery (Example 8.3.1) 460 b. Semantic Differential Ratings (Example 8.3.2) 461 c. Performance Assessment Program (Example 8.3.3) Statistical Tests in Principal Component Analysis 468 a. Tests Using the Covariance Matrix 468 b. Tests Using a Correlation Matrix Regression on Principal Components 474 a. GMANOVA Model 475 b. The PCA Model Multivariate Regression on Principal Components Example Canonical Correlation Analysis 477 a. Population Model for CCA 477 b. Sample CCA 482 c. Tests of Significance 483 d. Association and Redundancy 485 e. Partial, Part and Bipartial Canonical Correlation 487 f. Predictive Validity in Multivariate Regression using CCA 490 g. Variable Selection and Generalized Constrained CCA Canonical Correlation Analysis Examples 492 a. Rohwer CCA (Example 8.8.1) 492 b. Partial and Part CCA (Example 8.8.2) 494

8 Contents xvii 8.9 Exploratory Factor Analysis 496 a. Population Model for EFA 497 b. Estimating Model Parameters 502 c. Determining Model Fit 506 d. Factor Rotation 507 e. Estimating Factor Scores 509 f. Additional Comments Exploratory Factor Analysis Examples 511 a. Performance Assessment Program (PAP Example ) 511 b. Di Vesta and Walls (Example ) 512 c. Shin (Example ) Cluster Analysis and Multidimensional Scaling Introduction Proximity Measures 516 a. Dissimilarity Measures 516 b. Similarity Measures 519 c. Clustering Variables Cluster Analysis 522 a. Agglomerative Hierarchical Clustering Methods 523 b. Nonhierarchical Clustering Methods 530 c. Number of Clusters 531 d. Additional Comments Cluster Analysis Examples 533 a. Protein Consumption (Example 9.4.1) 534 b. Nonhierarchical Method (Example 9.4.2) 536 c. Teacher Perception (Example 9.4.3) 538 d. Cedar Project (Example 9.4.4) Multidimensional Scaling 541 a. Classical Metric Scaling 542 b. Nonmetric Scaling 544 c. Additional Comments Multidimensional Scaling Examples 548 a. Classical Metric Scaling (Example 9.6.1) 549 b. Teacher Perception (Example 9.6.2) 550 c. Nation (Example 9.6.3) Structural Equation Models Introduction Path Diagrams, Basic Notation, and the General Approach Confirmatory Factor Analysis Confirmatory Factor Analysis Examples 575 a. Performance Assessment 3 - Factor Model (Example ) 575 b. Performance Assessment 5-Factor Model (Example ) Path Analysis 580

9 xviii Contents 10.6 Path Analysis Examples 586 a. Community Structure and Industrial Conflict (Example ) b. Nonrecursive Model (Example ) Structural Equations with Manifest and Latent Variables Structural Equations with Manifest and Latent Variables Example Longitudinal Analysis with Latent Variables Exogeniety in Structural Equation Models 604 Appendix 609 References 625 Author Index 667 Subject Index 675