Applied Multivariate Analysis

Like most academic authors, my views are a joint product of my teaching and my research. Needless to say, my views reflect the biases that I have acquired. One way to articulate the rationale (and limitations) of my biases is through the preface of a truly great text of a previous era, Cooley and Lohnes (1971, p. v). They draw a distinction between mathematical statisticians whose intel­ lect gave birth to the field of multivariate analysis, such as Hotelling, Bartlett, and Wilks, and those who chose to "concentrate much of their attention on methods of analyzing data in the sciences and of interpreting the results of statistical analysis . . . . (and) . . . who are more interested in the sciences than in mathematics, among other characteristics. " I find the distinction between individuals who are temperamentally "mathe­ maticians" (whom philosophy students might call "Platonists") and "scientists" ("Aristotelians") useful as long as it is not pushed to the point where one assumes "mathematicians" completely disdain data and "scientists" are never interested in contributing to the mathematical foundations of their discipline. I certainly feel more comfortable attempting to contribute in the "scientist" rather than the "mathematician" role. As a consequence, this book is primarily written for individuals concerned with data analysis. However, as noted in Chapter 1, true expertise demands familiarity with both traditions.

1 Introduction and Preview.- Overview.- Multivariate Analysis: A Broad Definition.- Multivariate Analysis: A Narrow Definition.- Some Important Themes.- Obtaining Meaningful Relations.- Selecting Cutoffs.- Questions of Statistical Inference.- Outliers.- The Importance of Theory.- Problems Peculiar to the Analysis of Scales.- The Role of Computers in Multivariate Analysis.- Multivariate Analysis and the Personal Computer.- Choosing a Computer Package.- Problems in the Use of Computer Packages.- The Importance of Matrix Procedures.- 2 Some Basic Statistical Concepts.- Overview.- Univariate Data Analysis.- Frequency Distributions.- Normal Distributions.- Standard Normal Distributions.- Parameters and Statistics.- Locational Parameters and Statistics.- Measures of Variability.- A Note on Estimation.- Binary Data and the Binomial Distribution.- Data Transformation.- Bivariate Data Analysis.- Characteristics of Bivariate Relationships.- Bivariate Normality.- Measures of Bivariate Relation.- Range Restriction.- Pearson Correlation Formulas in Special Cases.- Non-Pearson Estimates of Pearson Correlations.- The Eta-Square Measure.- Phi Coefficients with Unequal Probabilities.- Sampling Error of a Correlation.- The Z’ Transformation.- Linear Regression.- The Geometry of Regression.- Raw-Score Formulas for the Slope.- Raw-Score Formulas for the Intercept.- Residuals.- The Standard Error of Estimate.- Why the Term “Regression”?.- A Summary of Some Basic Relations.- Statistical Control: A First Look at Multivariate Relations.- Partial and Part Correlation.- Statistical versus Experimental Control.- Multiple Partialling.- Within-Group, Between-Group, and Total Correlations.- 3 Some Matrix Concepts.- Overview.- Basic Definitions.- Square Matrices.- Transposition.- Matrix Equality.- Basic Matrix Operations.- Matrix Addition and Subtraction.- Matrix Multiplication.- Correlation Matrices and Matrix Multiplication.- Partitioned Matrices and Their Multiplication.- Some Rules and Theorems Involved in Matrix Algebra.- Products of Symmetric Matrices.- More about Vector Products.- Exponentiation.- Determinants.- Matrix Singularity and Linear Dependency.- Matrix Rank.- Matrix “Division”.- The Inverse of a 2 × 2 Matrix.- Inverses of Higher-Order Matrices.- Recalculation of an Inverse Following Deletion of Variable(s).- An Application of Matrix Algebra.- More about Linear Combinations.- The Mean of a Linear Combination.- The Variance of a Linear Combination.- Covariances between Linear Combination.- The Correlation between Two Different Linear Combinations.- Correlations between Linear Combinations and Matrix Notation.- The “It Don’t Make No Nevermind” Principle.- Eigenvalues and Eigenvectors.- A Simple Eigenanalysis.- Eigenanalysis of Gramian Matrices.- 4 Multiple Regression and Correlation―Part 1. Basic concepts.- Overview.- Assumptions Underlying Multiple Regression.- The Multivariate Normal Distribution.- A Bit of the Geometry of Multiple Regression.- Basic Goals of Regression Analysis.- The Case of Two Predictors.- A Visual Example.- A Note on Suppressor Variables.- Computational Formulas.- Raw-Score Formulas.- Other Equations for R2.- Determining the Relative Importance of the Two Predictors.- Bias in Multiple Correlation.- The Case of More Than Two Predictors.- Checking for Multicollinearity.- Another Way to Obtain R2.- Residuals.- Inferential Tests.- Testing R.- Testing Beta Weights.- Testing the Uniqueness of Predictors.- Evaluating Alternative Equations.- Cross Validation.- Computing a Correlation from a priori Weights.- Testing the Difference between R2 and r2 Derived from a priori Weights.- Hierarchical Inclusion of Predictors.- Stepwise Inclusion of Predictors.- Other Ways to Handle Multicollinearity.- Comparing Alternative Equations.- Example 1―Perfect Prediction.- Example 2―Imperfect Prediction plus a Look at Residuals.- Example 3―Real Personality Assessment Data.- Alternative Approaches to Data Aggregation.- 5 Multiple Regression and Correlation―Part 2. Advanced Applications.- Overview.- Nonquantitative Variables.- Dummy Coding.- Effect Coding.- Orthogonal Coding.- The Simple Analysis of Variance (ANOVA).- Fixed Effects versus Random Effects.- The Simple ANOVA as a Formal Model.- Results of Regression ANOVAs.- Multiple Comparisons.- Orthogonal versus Nonorthogonal Contrasts.- Planned versus Unplanned Comparisons.- Individual Alpha Levels versus Groupwise Alpha Levels.- Evaluation of Quantitative Relations.- Method I.- Method II.- The Two-Way ANOVA.- Equal-N Analysis.- Unequal-N Analysis.- Fitting Parallel Lines.- Simple Effect Models.- The Analysis of Covariance (ANCOVA).- Effects of the ANCOVA on the Treatment Sum of Squares.- Using Dummy Codes to Plot Group Means.- Repeated Measures, Blocked and Matched Designs.- Higher-Order Designs.- 6 Exploratory Factor Analysis.- Overview.- The Basic Factor Analytic Model.- The Factor Equation.- The Raw-Score Matrix.- Factor Scores and the Factor Score Matrix.- Pattern Elements and the Pattern Matrix.- Error Scores and Error Loadings.- The Covariance Equation.- The First Form of Factor Indeterminacy.- An Important Special Case.- Common Uses of Factor Analysis.- Orthogonalization.- Reduction in the Number of Variables.- Dimensional Analysis.- Determination of Factor Scores.- An Overview of the Exploratory Factoring Process.- Principal Components.- The Eigenvectors and Eigenvalues of a Gramian Matrix.- A Note on the Orthogonality of PCs.- How an Eigenanalysis Is Performed.- Determining How Many Components to Retain.- Other Initial Component Solutions.- Factor Definition and Rotation.- Factor Definition.- Simple Structure.- PC versus Simple Structure.- Graphic Representation.- Analytic Orthogonal Rotation.- Oblique Rotations.- Reference Vectors.- Analytic Oblique Rotation.- The Common Factor Model.- An Example of the Common Factor Model.- A Second Form of Factor Indeterminacy.- Factor Scores.- “Exact” Procedures.- Estimation Procedures.- Approximation Procedures.- Addendum: Constructing Correlation Matrices with a Desired Factor Structure.- 7 Confirmatory Factor Analysis.- Overview.- Comparing Factor Structures.- Similarity of Individual Factors versus Similarity of the Overall Solution.- Case I―Comparing Alternate Solutions Derived from the Same Data.- Case II―Comparing Solutions Obtained from the Same Subjects but on Different Variables.- Case III―Comparing Solutions with the Same Variables but on Different Individuals; Matrix Information Available.- Case IV―Comparing Solutions with the Same Variables but Different Individuals; Matrix Information Unavailable.- Case V―Factor Matching.- Oblique Multiple Groups Tests of Weak Structure.- Basic Approach.- Evaluating the Substantive Model.- Alternative Models.- Performing the Actual OMG Analysis.- Computational Steps.- OMG Common Factor Solutions.- A Numerical Example.- LISREL Tests of Weak Substantive Models.- Specification of Weak Models.- Estimation.- Identification.- Assessment of Fit.- Numerical Examples.- LISREL Tests of Strong Substantive Models.- Causal Models and Path Analysis.- Fundamental Path Analytic Concepts.- Example I: One Exogenous and Two Endogenous Variables.- Example II: Two Exogenous and One Endogenous Variables.- Example III: One Exogenous and Two Endogenous Variables.- Decomposing Observed Correlations.- Causal Models and LISREL.- A Formal Separation of Exogenous and Endogenous Variables.- The Confirmatory Factor Model for Exogenous Variables.- The Confirmatory Factor Model for Endogenous Variables.- The Structural Model.- Addendum: A Program to Obtain Oblique Multiple Groups Solutions.- 8 Classification Methods―Part 1. Forming Discriminant Axes.- Overview.- Discriminant Analysis with Two Groups and Two Predictors.- Graphic Representation.- One Way to Obtain Discriminant Weights.- Another Way to Obtain Discriminant Weights.- Computation.- A Third Way to Obtain Discriminant Weights.- Eigenanalysis of W-1B versus Eigenanalysis of SPw-1SPb.- Strength of Relation.- Discriminant Scores, Discriminant Means, and the Mahalanobis Distance Measure.- The Discriminant Structure.- Interpreting a Discriminant Axis.- Discriminant Analysis with Two Predictors and Three Groups.- Extracting Multiple Discriminant Axes.- Inferring Concentration from the Eigenanalysis.- Numerical Examples.- Depiction of Group Differences in the Diffuse Example.- Varimax Rotation.- Use of Prior Weights.- Heteroscedasticity.- Discriminant Analysis―The General Case.- Components as Discriminant Variables.- Hypothesis Testing.- Confirmatory Factor Analysis.- Stepwise Discriminant Analysis.- 9 Classification Methods―Part 2. Methods of Assignment.- Overview.- The Equal Variance Gaussian Model.- Decision Rules and Cutoff Rules.- Other Properties.- Why a Cutoff Rule?.- Bayesian Considerations.- Why Bayesian Considerations Are Important.- Varying Base Rates.- Bayes’ Theorem in Binary Classification.- Receiver (Relative) Operating Characteristic (ROC) Curves.- Describing Accuracy of Classification (Sensitivity).- The Unequal Variance Gaussian Model.- Other Signal Detection Models.- Strategies for Individual Classification.- Describing Performance.- Bayesian Considerations with Multiple Groups.- Alternative Strategies―An Overview.- A Numerical Example.- Calibration Study.- The Context of Classification.- Classification Based on Salient Variables.- Homoscedasticity Assumed.- Normality Assumed.- Dealing with Heteroscedasticity.- Discriminant Functions and Classification.- Homoscedasticity Assumed.- Normality Assumed.- Dealing with Heteroscedasticity.- Using Multiple Discriminant Scores.- Normality and Multiple Discriminant Functions.- Classification Based on Distance Measures.- Simple Distance.- Fisher’s Classification Functions.- Mahalanobis Distances.- A Summary of Strategic Considerations in Classification.- 10 Classification Methods―Part 3. Inferential Considerations in the Manova.- Overview.- The Two-Group MANOVA and Hotelling’s T2.- The Formal MANOVA Model.- Hotelling’s T2.- Post Hoc Comparisons.- Application of Hotelling’s T2 to a Single Group.- Testing for Equality of Means.- Single Group MANOVA versus Repeated Measures ANOVA.- Tests of Vector Means with Multiple Groups.- The Fundamental Problem.- Testing a Concentrated Structure.- Testing a Diffuse Structure.- Testing for Homogeneity of Covariance.- The Simple MANOVA with Multiple Groups.- SP and Variance-Covariance Matrices.- Applying Box’s M Test.- Specifying the Nature of the Group Differences.- The Multivariate MANOVA.- Terminology and Basic Logic.- A Numerical Example.- The A Effect.- The B Effect.- The AB Interaction.- The MANCOVA.- Numerical Example.- 11 Profile and Canonical Analysis.- Overview.- Profile Similarity.- Similarity of Scalars.- Similarity of Vectors.- Measuring Profile Elevation.- Profile Similarity Based on Elevation Alone.- Numerical Example.- Profile Similarity Based on Shape Information.- A Numerical Example.- Correcting for Elevation.- Correlations, Covariances, and Cross Products as Alternative Indices of Similarity.- Simple and Hierarchical Clustering.- Numerical Example.- Canonical Analysis.- Goals of Canonical Analysis.- Basic Logic.- Redundancy Analysis.- Basic Matrix Operations.- Statistical Inference.- Numerical Example.- Alternatives to Canonical Analysis.- 12 Analysis of Scales.- Overview.- Properties of Individual Items.- Item Formats and Scoring.- Correction for Guessing.- Item Distributions.- Relation of Items to the Scale as a Whole.- Numerical Example.- Test Reliability.- The Logic of Internal Consistency Measures.- The Domain Sampling Model.- Computing rxx and Cronbach’s Alpha.- Numerical Example.- The Standard Error of Measurement.- Estimating What True Scores Would Be in the Absence of Unreliability.- Effects of Changes in Test Length.- Reliability of a Linear Combination of Tests (Scales).- Dimensionality and Item-Level Factoring.- Looking for Difficulty Factors.- Limitations on the Utility of Test Reliability.- Numerical Example I: A Unifactor Scale.- Numerical Example II: A Two-Factor Scale.- Test Validity.- Attenuation.- Appendix A―Tables of the Normal Curve.- Appendix D―Tables of Orthogonal Polynomial Coefficients.- Problems.- References.- Author Index.