Research Methods and Analysis 4: Multivariate DV Designs

PSY400 Research Methods and Analysis 4

Dr Joshua Adie
University of the Sunshine Coast | CRICOS Provider Number: 01595DUniSC

Week 4: Multivariate DV Designs

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Reading - covers undertaking these techniques in SPSS

• Field, A. (2018) Discovering statistics using IBS SPSS statistics
• Chapters 17

Workshop content

1. MANOVA
2. Research problems suited to MANOVA
3. Single IV, 2 levels
4. Single IV, more than 2 levels
5. 2 IVs and 1 DV
6. Special case - covariates

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Overview of Multivariate Methods

UniSC

Decision
point

What type of
relationship is
being examined?

Dependence

Interdependence

Multivariate
technique

How many
variables are
being
predicted?

Is the structure
of the
relationships
among:

Multiple relationships of
dependent and independent
variables

Several dependent
variables in a single
relationship

One dependent variable
in a single relationship

Variables
Cases/respondents
Objects

Structural
Equation
Modelling

What is the
measurement
scale of the
DV?

What is the
measurement
scale of the
DV

Factor analysis
Confirmatory
factor analysis
Cluster analysis

How are the
attributes
measured?

Metric
Nonmetric
Nonmetric
Metric
Nonmetric
Nonmetric

Multiple
Regression

Multiple
Discriminant
Analysis

What is the
measurement
of the
predictor
variable?

Canonical
correlation
analysis with
dummy variables

Conjoint analysis

Linear
Probability
Models

Metric
+

Nonmetric

Canonical
correlational
analysis

MANOVA

University of the Sunshine Coast | CRICOS Provider Number: 01595D

UniSC
Multidimensional
scaling

Correspondence
analysis

Metric
+
1. MANOVA

MANOVA is classed as multivariate because it involves the analysis of multiple DVs.

Number of DVs

Number of groups in IV (levels of IV)

1
(univariate)

2 or more
(multivariate)

2 groups (specialised case)
t test
Hotelling's T2

2 or more groups (generalised case)
ANOVA
MANOVA

t test and Hotelling's T2 are portrayed as specialized cases as are limited to assessing only two groups (categories) for an
independent variable.
Both ANOVA and MANOVA can also handle the two group situations as well as address analyses where the independent
variables have more than two groups.

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Multivariate procedures for assessing group differences

UniSC
. Univariate techniques (t test and ANOVA) and multivariate extensions (Hotelling's T2 and MANOVA) are used to assess
the statistical significance of differences between groups. . In the multivariate techniques, the null hypothesis tested is
the equality of vectors of means on multiple DVs across groups.

ANOVA hypothesis:

Η0: 11 = 12= .... ΜΕ
[i.e. all group means are equal]

MANOVA hypothesis:

Ho :
μ11
μ21
=
=
[i.e. all group mean vectors are equal]
Hpk = means of variable p, group k

μ12
μ22
H1k
μ2k
.
.
F
. In the univariate case, a single dependent measure is tested for equality across the groups. In the multivariate case, a
variate is tested for equality.
. In MANOVA, the researcher actually has two variates:
1.
one for the DVs and
2.
another for the IVs

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Multivariate procedures for assessing group differences

UniSC
. The unique aspect of MANOVA is that the variate optimally combines the multiple dependent measures into a single
value that maximizes the differences across groups.
. ANOVA tests whether mean differences among groups on a single DV are likely to have occurred by chance.
. MANOVA tests whether mean differences among groups on a combination of DVs are likely to have occurred by chance.
. In MANOVA, the new DV is a composite score (a linear combination of measured DVs) so as to separate the groups
as much as possible. ANOVA is then performed on the newly created DV.

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Advantages of MANOVA over ANOVA

UniSC
1. by measuring several DVs instead of only one, improves the chance of discovering what it is that changes as a result of
different treatments and their interactions.
2. MANOVA is superior to a series of ANOVAs when there are several DVs as MANOVA protects against inflated Type I
error due to multiple tests of (likely) correlated DVs. That is, MANOVA corrects for the number of multiple comparisons
made for the same data set, thereby protecting against inflated Type I error arising from repeated comparisons.
3. under certain, probably rare conditions, it may reveal differences not shown in separate ANOVAs. That is, MANOVA,
which considers DVs in combination, may occasionally be more powerful than separate ANOVAs.
But
....
MANOVA is a substantially more complicated analysis than ANOVA, because:
. there are several important assumptions to consider
. there is often some ambiguity in interpretation of the effects of IVs on any single DV.
. the situations in which MANOVA is more powerful than ANOVA are quite limited - often MANOVA is considerably less
powerful than ANOVA, particularly in finding significant group differences for a specific DV.

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Summary and refresher - ANOVA to MANOVA

UniSC

One-way ANOVA model

Level 1
Level 2
Level 3
IV1
DV1
.
·
Level n
e.g., group differences on DV1

One-way repeated measures ANOVA model

Level 1
e.g., group differences on DV1 over time
Level 2
Level 3
IV1
.
.
Level n
DV1 (time 1)
DV1 (time 2)
DV1 (time k)

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Summary and refresher - ANOVA to MANOVA

UniSC

2-factor ANOVA model

Level 1
Level 2
IV1
Level n
DV1
Level 1
Level 2
IV2
.
Level n
e.g., treatment and age group
differences & interactions on DV1

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Summary and refresher - ANOVA to MANOVA

UniSC

Simple MANOVA model

Level 1
DV1
Level 2
Level 3
Variate
DV2
IV1
DVk
Level n
e.g., group differences on variate of DV1-DVK

University of the Sunshine Coast | CRICOS Provider Number: 01595D

MANOVA decision diagram

UniSC

Research problem

Specify type of research problem:
Multiple univariate
Structural multivariate
Intrinsically multivariate

Selection of dependent variables

Research design issues

Adequate sample size per group
Use of covariates
Selecting treatments (IVs)
Number of IVs
ONE
Simple MANOVA
2 or more
Develop factorial design
Interpret interactions

Assumptions

Independence
Homogeneity of variance/covariance matrices
Normality
Linearity/multicollinearity of DVs
Sensitivity to outliers

Estimating the significance of group differences

Selecting criteria for significance tests
-
Assessing statistical power
Increasing power
Use in planning and analysis
Effects of DV multicollinearity

Interpreting the effects of variables

.
Evaluating covariates
Assessing impact of IVs
Post-hoc vs a priori tests
Stepdown analysis

Identifying differences between groups

Post hoc methods
A priori or planned comparison methods

Validating the results

Replication
Split-sample analysis
MANOVA Decision tree
Copy available in Week 4 online materials

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Research problems suited to MANOVA

UniSC
1. Control of experiment-wide error rate
. Multiple ANOVAs on same data set = inflated Type I experiment-wide (family-wide) error rate
· MANOVA controls for experiment-wide error rate.
2. Difference among combination of DVs.
. Individual ANOVAs cannot identify a composite (linear combination) of DVS
· MANOVA may detect combined differences not found in the univariate tests.
. If multiple variates are formed, then they may provide dimensions of differences that can distinguish among the groups better
than single variables.
. If the number of DVs is <6 the statistical power of the MANOVA tests equals or exceeds that obtained with a single ANOVA. If the
hypotheses predict an interaction between a between-subjects IV (group) and a within-subjects IV (age) = a 2 factor ANOVA ->
as it evaluates the between-subjects and within-subjects changes separately before examining the interaction between the
within- and between-subjects factors
3.
Types of questions suitable for MANOVA
a)
Multiple univariate questions
b)
Structured multivariate questions
c)
Intrinsically multivariate questions

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Research problems suited to MANOVA

UniSC
4. Selecting the DV
· A common problem encountered with MANOVA is the tendency of researchers to misuse one of its strengths ( the ability to
handle multiple dependent measures) by including variables without a sound conceptual or theoretical basis.
. If some of the DVs that have the strong differences are not really appropriate for the research question, then "false" differences
may lead the researcher to draw incorrect conclusions about the set as a whole,
. Thus, the researcher should always scrutinize the DVs and form a solid rationale for including them. Any ordering of the
variables, such as possible sequential effects, should also be noted.
. MANOVA provides a special test, stepdown analysis, to assess the statistical differences in a sequential manner, much like the
addition of variables to a regression analysis.

University of the Sunshine Coast | CRICOS Provider Number: 01595D

MANOVA design issues

UniSC

Sample Size Requirements - Overall and by Group

. What differs most for MANOVA (and the other techniques assessing group differences such as the t test and ANOVA) is that the
sample size requirements relate to individual group sizes and not the total sample per se.
. A number of basic issues arise concerning the sample sizes needed in MANOVA:
· At a bare minimum, the sample in each cell (group) must be greater than the number of DVs. This problem is particularly prevalent in field
experimentation or survey research, where the researcher has less control over the achieved sample.
. As a practical guide, a recommended minimum cell size is 20 observations (i.e., per group) - which means you will require fairly large overall
samples even for fairly simple analyses. If you have only two factors, each with two levels, you would require 80 observations (cases) for an
adequate analysis.
. As the number of DVs increases, the sample size required to maintain statistical power increases as well. For example, required samples sizes
increase by almost 50% as the number of DVs goes from 2 to 6.
. Researchers should strive to maintain equal or approximately equal sample sizes per group.
. While MANOVA can easily accommodate unequal group sizes, the effectiveness of the analysis is dictated by the smallest group sizes, thus always
making sample size considerations a primary concern.

University of the Sunshine Coast | CRICOS Provider Number: 01595D