M&A L8 Factor Analysis: Understanding Statistical Grouping of Variables

Class Information

PSY 400

• Item
  Lecture

Status
Done
Σ Days Until:
-30
Location
Lecture Theatre 2, Building K
Date
@May 3, 2025 6:30 -> 7:30
Amount of Time
1 hr 0 mins

Factor Analysis Overview

Overview of Multivariate Methods

Decision
point
What type of
relationship is
being examined?

Dependence
Interdependence

Multivariate
technique
How many
variables are
being
predicted?

Is the structure
of the
relationships
among:

Multiple relationships of
dependent and independent
variables

Several dependent
variables in a single
relationship

One dependent variable
in a single relationship

Variables
Cases/respondents
Objects

Structural
Equation
Modelling

What is the
measurement
scale of the
DV?

What is the
measurement
scale of the
DV?

Factor analysis
Confirmatory
factor analysis
Cluster analysis

How are the
attributes
measured?

Metric
Nonmetric
Metric
Nonmetric
Metric
Nonmetric
+
+
+
+
1
1

What is the
measurement
of the
predictor
variable?

Canonical
correlation
analysis with
dummy variables

Conjoint analysis

Linear
Probability
Models

Metric
Nonmetric
+
+

Canonical
correlational
analysis

MANOVA

Nonmetric

Multiple
Regression

Multiple
Discriminant
Analysis

Multidimensional
scaling

Correspondence
analysis

1Factor analysis (FA) is a statistical technique applied to a single set of variables
when the researcher is interested in discovering which variables in the set form
coherent subsets that are relatively independent of one another. Variables that are
correlated with one another but largely independent of other subsets of variables
are combined into factors.
Factors are thought to reflect underlying processes that have created the
correlations among variables. For instance, several individual variables from
personality measures combine with some variables from a motivation scale form a
called the reproduced correlation matrix.
factor measuring the degree to which a person prefers to work independently (an
independence factor). A major use of FA in psychology is in development of
objective tests for measurement of personality and intelligence etc. The
researcher starts out with a very large number of items reflecting a first guess
about the items that may eventually prove useful. The items are given to randomly
selected research participants, and factors are derived. As a result of the first
factor analysis, items are added and deleted, a second test is devised, and that
test is given to other randomly selected participants. The process continues until
the researcher has a test with numerous items forming several factors that
represent the area to be measured. The validity of the factors is then tested in
research where predictions are made regarding differences in the behaviour of
persons who score high or low on a factor.
The specific goals of FA are to summarize patterns of correlations among
observed variables, to reduce a large number of observed variables to a smaller
number of factors, to provide an operational definition (a regression equation) for
an underlying process by using observed variables, or to test a theory about the
nature of underlying processes.
FA has considerable utility in reducing numerous variables down to a few factors.
Mathematically, FA produces several linear combinations of observed variables,
where each linear combination is a factor. The factors summarize the patterns of
correlations in the observed correlation matrix and can be used, with varying
degrees of success, to reproduce the observed correlation matrix. But since the
number of factors is usually far fewer than the number of observed variables,
there is considerable parsimony in using the factor analysis. Further, when scores
on factors are estimated for each participant, they are often more reliable than
scores on individual observed variables.

Steps in Factor Analysis

2Steps in FA include:

• selecting and measuring a set of variables,
• preparing the correlation matrix (to perform a FA),
• extracting a set of factors from the correlation matrix,
• determining the number of factors,
• (probably) rotating the factors to increase interpretability, and,
• interpreting the results.
  Although there are relevant statistical considerations to most of these steps, an
  important test of the analysis is its interpretability. A good FA "makes sense"; a
  bad one does not. A factor is more easily interpreted when several observed
  variables correlate highly with it and those variables do not correlate with other
  factors. Once interpretability is adequate, the last, and very large, step is to verify
  the factor structure by establishing the construct validity of the factors. The
  researcher seeks to demonstrate that scores on the latent variables (factors)
  covary with scores on other variables, or that scores on latent variables change
  with experimental conditions as predicted by theory.

Problems with Factor Analysis

Some of the problems FA are:

1. there are no readily available criteria against which to test the solution. In
   regression analysis, for instance, the DV is a criterion and the correlation
   between observed and predicted DV scores serves as a test of the solution. In
   multiple discriminant analysis, logistic regression, and MANOVA, the solution
   is judged by how well it predicts group membership. But in FA, there is no
   external criterion such as group membership against which to test the
   solution.
2. after extraction, there is an infinite number of rotations available, all
   accounting for the same amount of variance in the original data, but with the
   factors defined slightly differently. The final choice among alternatives
   depends on the researcher's assessment of its interpretability and scientific
   utility. In the presence of an infinite number of mathematically identical
   solutions, researchers are bound to differ regarding which is best - hence,
   results are not replicated exactly, if different decisions are made at one, or
   more, of the steps in performing FA.

33. FA is frequently used in an attempt to "save" poorly conceived research. If no
other statistical procedure is applicable, at least data can usually be factor
analysed. Thus, in the minds of many, the various forms of FA are associated
with sloppy research. The very power of FA to create apparent order from real
chaos contributes to their somewhat tarnished reputations as scientific tools.

Types of Factor Analysis

There are two major types of FA: exploratory and confirmatory.

• In exploratory FA, one seeks to describe and summarize data by grouping
  together variables that are correlated. The variables themselves may or may
  not have been chosen with potential underlying processes in mind.
  Exploratory FA is usually performed in the early stages of research, when it
  provides a tool for consolidating variables and for generating hypotheses
  about underlying processes.
• Confirmatory FA is a much more sophisticated technique used in the
  advanced stages of the research process to test a theory about latent
  processes. Variables are carefully and specifically chosen to reveal underlying
  processes. Usually, confirmatory FA is performed through structural equation
  modelling (see next week), but can be undertaken as a standalone process
  (which we explore later in this module).

Basic Terms and Definitions in Factor Analysis

Some basic terms and definitions in FA:

• The correlation matrix produced by the observed variables is called
  the observed correlation matrix.
• The correlation matrix produced from factors, that is, correlation matrix
  implied by the factor solution, is called the reproduced correlation matrix.
• The difference between observed and reproduced correlation matrices is
  the residual correlation matrix.
  • In a good FA, correlations in the residual matrix are small, indicating a
    close fit between the observed and reproduced matrices.
• Rotation of factors is a process by which the solution is made more
  interpretable without changing its underlying mathematical properties. There
  are two general classes of rotation:
  • orthogonal - If rotation is orthogonal (so that all the factors are
    uncorrelated with each other), a loading matrix is produced. The loading
    4matrix is a matrix of correlations between observed variables and factors.
    The sizes of the loadings reflect the extent of relationship between each
    observed variable and each factor. Orthogonal FA is interpreted from the
    loading matrix by looking at which observed variables correlate with each
    factor.
  • oblique - if rotation is oblique (so that the factors themselves are
    correlated), several additional matrices are produced. The factor
    correlation matrix contains the correlations among the factors. The loading
    matrix from orthogonal rotation splits into two matrices for oblique
    rotation:
    • a structure matrix of correlations between factors and variables and
    • a pattern matrix of unique relationships (uncontaminated by overlap
      among factors) between each factor and each observed variable.
  • Following oblique rotation, the meaning of factors is ascertained from
    the pattern matrix.
  • for both types of rotations, there is a factor-score coefficients matrix -
    this is a matrix of coefficients used in several regression-like equations to
    predict scores on factors from scores on observed variables for each
    individual. FA produces factors - in FA, only shared variance is analysed;
    attempts are made to estimate and eliminate variance due to error and
    variance that is unique to each variable.

Theoretically, factors are thought to "cause" variables. The underlying construct
(the factor) is what produces scores on the variables. Thus, exploratory FA is
associated with theory development and confirmatory FA is associated with
theory testing.

• The question in exploratory FA is: "What are the underlying processes that
  could have produced correlations among these variables?"
• The question in confirmatory FA is: "Are the correlations among variables
  consistent with a hypothesized factor structure?"

Exploratory Factor Analysis (EFA)

5Factor Analysis decision tree.jpg

6