Multivariate Statistical Inference and Applications

Transcription

1 Multivariate Statistical Inference and Applications ALVIN C. RENCHER Department of Statistics Brigham Young University A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane Singapore Toronto

2 Contents Some Properties of Random Vectors and Matrices Introduction, Univariate and Bivariate Random Variables, Univariate Random Variables, Bivariate Random Variables, Mean Vectors and Covariance Matrices for Random Vectors, Correlation Matrices, Partitioned Mean Vectors and Covariance Matrices, Linear Functions of Random Variables, Sample Means, Variances, and Covariances, Population Means, Variances, and Covariances, Measuring Intercorrelation, Mahalanobis Distance, MissingData, Robust Estimators of fi and 2, 27 The Multivariate Normal Distribution Univariate and Multivariate Normal Density Functions, Univariate Normal, Multivariate Normal, Constant Density Ellipsoids, Generating Multivariate Normal Data, Moments, Properties of Multivariate Normal Random Vectors, Estimation of Parameters in the Multivariate Normal Distribution, 49 v

3 vi CONTENTS Maximum Likelihood Method, PropertiesofyandS, Wishart Distribution, Additional Topics, Hotelling's T 2 -Tests Introduction, Test for HQ: fi = fi 0 with 2 Known, Hotelling's T 2 -test for // 0 : fi = /x 0 with 2 Unknown, Univariate f-test for Ho: ix IM> with a 2 Unknown, Likelihood Ratio Method of Test Construction, One-Sample r 2 -Test, Formal Definition of T 2 and Relationship to F, Effect on 7 2 of Adding a Variable, Propertiesofthe7 2 -Test, Likelihood Ratio Test, Union-Intersection Test, Confidence Intervals and Tests for Linear Functions of fi, Confidence Region for ft, Confidence Interval for a Single Linear Combination a'fi, Simultaneous Confidence Intervals for IJLJ and a'/m, Bonferroni Confidence Intervals for /x, and a'/u, Tests for H 0 : a'/u, = a'jto and H 0 : ju, 7 = /xo ; -, Tests for H 0 : Cp = 0, Tests of H 0 : fx { = ft 2 Assuming 21 = 2 2, Review of Univariate Likelihood Ratio Test for H 0 : ix\ n>2 When a 2 = er 2, Test for H 0 : (JL { = fi 2 When 21 = 2 2, Effect on T 2 of Adding a Variable, Properties of thetwo-sample r 2 -Statistic, Likelihood Ratio and Union-Intersection Tests, Confidence Intervals and Tests for Linear Functions of Two Mean Vectors, Confidence Region for fn x fi, 2, Simultaneous Confidence Intervals for a'ipi ~~ M2) anc^ Mi; - M2;, Bonferroni Confidence Intervals for a'(/*i _ M2) anc * Mi; _ M2;, 94

4 CONTENTS vii Tests for H 0 :a'(fi 1 fi 2 ) = a '^o and H 0j : tnj ~ Mj = 0, Test for H 0 : C(/A, - fi 2 ) = 0, Robustness of the r 2 -test, Robustness to 2, + X 2, Robustness to Nonnormality, Paired Observation Test, Testing H 0 : Mi = M 2 When 21 = X 2, Univariate Case, Multivariate Case, Power and Sample Size, Tests on a Subvector, Two-Sample Case, Step-Down Test, Selectionof Variables, One-Sample Case, Nonnormal Approaches to Hypothesis Testing, Elliptically Contoured Distributions, Nonparametric Tests, Robust Versions of T 2, Application of T 2 In Multivariate Quality Control, Multivariate Analysis of Variance One-Way Classification, Model for One-Way Multivariate Analysis of Variance, Wilks' Likelihood Ratio Test, Roy's Union-Intersection Test, The Pillai and Lawley-Hotelling Test Statistics, Summary of the Four Test Statistics, Effect of an Additional Variable on Wilks' A, Tests on Individual Variables, Power and Robustness Comparisons for the Four MANOVA Test Statistics, Tests for Equality of Covariance Matrices, Power and Sample Size for the Four MANOVA Tests, Contrasts Among Mean Vectors, Univariate Contrasts, Multivariate Contrasts, 145

5 viii CONTENTS 4.6. Two-Way Multivariate Analysis of Variance, Higher Order Models, Unbalanced Data, Introduction, Univariate One-Way Model, Multivariate One-Way Model, Univariate Two-Way Model, Multivariate Two-Way Model, Tests on a Subvector, Testing a Single Subvector, Step-Down Test, Stepwise Selection of Variables, Multivariate Analysis of Covariance, Introduction, Univariate Analysis of Covariance: One-Way Model with One Covariate, Univariate Analysis of Covariance: Two-Way Model with One Covariate, Additional Topics in Univariate Analysis of Covariance, Multivariate Analysis of Covariance, Alternative Approaches to Testing Ho-t*>i ~ J*2 = - Pk> Discriminant Functions for Descriptive Group Separation Introduction, TwoGroups, Several Groups, Discriminant Functions, Assumptions, Standardized Coefficients, Tests of Hypotheses, TwoGroups, Several Groups, Discriminant Analysis for Higher Order Designs, Interpretation of Discriminant Functions, Standardized Coefficients and Partial F-Values, Correlations between Variables and Discriminant Functions, 211

6 CONTENTS ix Other Approaches, Confidence Intervals, Subset Selection, Discriminant Function Approach to Selection, Stepwise Selection, All Possible Subsets, Selection in Higher Order Designs, Bias in Subset Selection, Other Estimators of Discriminant Functions, Ridge Discriminant Analysis and Related Techniques, Robust Discriminant Analysis, Classification of Observations into Groups Introduction, Two Groups, Equal Population Covariance Matrices, Unequal Population Covariance Matrices, Unequal Costs of Misclassification, Posterior Probability Approach, Robustness to Departures from the Assumptions, Robust Procedures, Several Groups, Equal Population Covariance Matrices, Unequal Population Covariance Matrices, Use of Linear Discriminant Functions for Classification, Estimation of Error Rates, Correcting for Bias in the Apparent Error Rate, Partitioning the Sample, Holdout Method, Bootstrap Estimator, Comparison of Error Estimators, Subset Selection, Selection Based on Separation of Groups, Selection Based on Allocation, Selection in the Heteroscedastic Case, Bias in Stepwise Classification Analysis, Logistic and Probit Classification, 254

7 X CONTENTS The Logistic Model for Two Groups with 2, = 2 2 > Comparison of Logistic Classification with Linear Classification Functions, Quadratic Logistic Functions When X i = X2, Logistic Classification for Several Groups, Additional Topics in Logistic Classification, Probit Classification, Additional Topics in Classification, Multivariate Regression Introduction, Multiple Regression: Fixedx's, Least Squares Estimators and Properties, An Estimator for er 2, The Model in Centered Form, Hypothesis Tests and Confidence Intervals, R 2 in Fixed-Jt Regression, Model Validation, Multiple Regression: Random x's, Model for Random x's, Estimation of ßo> ß\, and CT 2, R 2 in Random-x Regression, Tests and Confidence Intervals, Estimation in the Multivariate Multiple Regression Model: Fixedx's, The Multivariate Model, Least Squares Estimator for B, Properties of B, An Estimator for X, Normal Model for the y,'s, The Multivariate Model in Centered Form, Measures of Multivariate Association, Hypothesis Tests in the Multivariate Multiple Regression Model: Fixedx's, Test for Significance of Regression, Test onasubsetof the RowsofB, General Linear Hypotheses CB = O and CBM = O, Tests and Confidence Intervals for a Single ßß and a Bilinear Function a'bb, 297

8 CONTENTS xi Simultaneous Tests and Confidence Intervals for the ß jk 's and Bilinear Functions a'bb, Tests in the Presence of Missing Data, Multivariate Model Validation: Fixedx's, Lack-of-Fit Tests, Residuais, Influence and Outliers, Measurement Errors, Multivariate Regression: Random x's, Multivariate Normal Model for Random x's, Estimationofßo, Bi.andX, Tests and Confidence Intervals in the Multivariate Random-x Case, Additional Topics, Correlated Response Methods, Categorical Data, Subset Selection, Other Topics, Canonical Correlation Introduction, Canonical Correlations and Canonical Variates, Properties of Canonical Correlations and Variates, Properties of Canonical Correlations, Properties of Canonical Variates, Tests of Significance for Canonical Correlations, Tests of Independence of y and x, Test of Dimension of Relationship between the y's and the x's, Validation, Interpretation of Canonical Variates, Standardized Coefficients, Rotation of Canonical Variate Coefficients, Correlations between Variables and Canonical Variates, Redundancy Analysis, Additional Topics, 333

9 xii CONTENTS 9. Principal Component Analysis Introduction, Definition and Properties of Principal Components, Maximum Variance Property, Principal Components as Projections, Properties of Principal Components, Principal Components as a Rotation of Axes, Principal Components from the Correlation Matrix, Methods for Discarding Components, Percent of Variance, Average Eigenvalue, Scree Graph, Significance Tests, Other Methods, Information in the Last Few Principal Components, Interpretation of Principal Components, Special Patterns in S or R, Testing H 0 : X = er 2 [(1 - p)i + pj] and P p = (1 - p)i + pj, Additional Rotation, Correlations between Variables and Principal Components, Relationship Between Principal Components and Regression, Principal Component Regression, Latent Root Regression, Principal Component Analysis with Grouped Data, Additional Topics, Factor Analysis Introduction, Basic Factor Model, Model and Assumptions, Scale Invariance of the Model, Rotation of Factor Loadings in the Model, Estimation of Loadings and Communalities, Principal Component Method, Principal Factor Method, Iterated Principal Factor Method, 384

10 CONTENTS xiii Maximum Likelihood Method, Other Methods, Comparison of Methods, Determining the Number of Factors, m, Rotation of Factor Loadings, Introduction, Orthogonal Rotation, Oblique Rotations, Interpretation of the Factors, Factor Scores, Applicability of the Factor Analysis Model, Factor Analysis and Grouped Data, Additional Topics, 394 Appendix A. Review of Matrix Algebra 399 A.l. Introduction, 399 A.l.l. Basic Definitions, 399 A.1.2. Matrices with Special Patterns, 400 A.2. Properties of Matrix Addition and Multiphcation, 401 A.3. Partitioned Matrices, 404 A.4. Rank of Matrices, 406 A.5. Inverse Matrices, 407 A.6. Positive Definite and Positive Semidefinite Matrices, 408 A.7. Determinants, 409 A.8. Traceofa Matrix, 410 A.9. Orthogonal Vectors and Matrices, 410 A.10. Eigenvalues and Eigenvectors, 411 A. 11. Eigenstructure of Symmetrie and Positive Definite Matrices, 412 A.12. Idempotent Matrices, 414 A.B. Differentiation, 414 Appendix B. Tables 417 Appendix C. Answers and Hints to Selected Problems 449 Appendix D. About the Diskette 505 Bibliography 507 Index 549