PSY400 Research Methods and Analysis 4: Univariate Designs and ANOVA

PSY400 Research Methods and Analysis 4

Dr Joshua Adie University of the Sunshine Coast | CRICOS Provider Number: 01595DUniSC

Week 3

Univariate designs - refresher course

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Reading - covers undertaking these techniques in SPSS

• Field, A. (2018) Discovering statistics using IBS SPSS statistics
• Chapters 10, 12, 13, 15

Workshop content

1. Univariate designs - the basics
2. From 2 group to multi-group, multi-time designs
3. Single IV, 2 levels
4. Single IV, more than 2 levels
5. 2 IVs and 1 DV
6. Special case - covariates

University of the Sunshine Coast | CRICOS Provider Number: 01595D

1. Univariate Designs - the basics

2 general research designs that can be used to obtain the sets of data to be compared:

1. sets of data from completely separate groups of participants. (e.g. compare grades for one group of undergraduates who are given laptop computers with grades for a second group who are not given computers). . This research strategy, using completely separate groups, is called an independent measures research design or a between-subjects design.
2. The sets of data could come from the same group of participants (e.g., compare depression scores for a sample of patients before they begin therapy against second set of depression scores from same individuals after 6 weeks of therapy). . This research strategy, in which the sets of data being compared are obtained from the same group of participants, is called a repeated-measures research design or a within-subjects design.

University of the Sunshine Coast | CRICOS Provider Number: 01595D

2. From 2-group to multi-group, multi-time designs

Hypotheses

Design decisions

When your hypotheses involve a variation to the levels of a single IV - then you will be using a univariate design

1. Where the IV is a group, the decision then is to how many levels of the group IV are required in the design.
   • If 2 groups (e.g. control vs treatment) = 2 level single IV design = only suitable analysis is an independent samples t-test.
   • If > 2 groups (e.g. placebo, drug 1, drug 2) = a one-way ANOVA is the only suitable analysis.
2. Where the IV is instead a repeated measure - i.e., same participants are tested on multiple occasions - then a within- subjects design is required -> the levels of the IV (e.g. number of test points) determines the analysis.
   • If 2 levels of the within-subjects IV = paired-samples t-test is the only suitable analysis.
   • If >3 levels of the within-subjects IV = a repeated measures ANOVA is the suitable analysis.
3. If the hypotheses predict an interaction between a between-subjects IV (group) and a within-subjects IV (age) = a 2 factor ANOVA -> as it evaluates the between-subjects and within-subjects changes separately before examining the interaction between the within- and between-subjects factors
   • e.g., comparing 2 groups (treatment vs control) over time (pre-treatment vs post-treatment) - Interaction effect would determine that the treatment group changed from pre- to post-assessment, but no change observed in the control group).

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Univariate Designs Summary

Between-subjects designs

Different groups of participants

• 2 groups Independent samples t-test
• >2 groups One-way ANOVA

Within-subjects designs

Same group, multiple time points

• 1 group, 2 times Paired samples t-test
• 1 group, >2 times Repeated measures ANOVA

Within-and Between-subjects designs

Different groups & multiple timepoints

• 2+ groups, 2+ times 2 Factor ANOVA

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Important (critical) lesson

1. Hypotheses determine design
2. Design determines analysis
3. Therefore, hypotheses are the critical component of research methodology

University of the Sunshine Coast | CRICOS Provider Number: 01595D

3. Single IV, 2 level designs

3.1. Between subjects - independent samples t-test

The hypothesis for an independent samples t-test

The goal is to evaluate the mean difference between two populations (or between two treatment conditions).

u1 = mean for the first population 12 = mean for the second population The difference between means is simply u1 - u2

As always, the null hypothesis states that there is no change, no effect, or, in this case, no difference. Thus, in symbols, the null hypothesis for the independent-measures test is

HO : 11 - 12 = 0 (No difference between the population means),

The alternative hypothesis states that there is a mean difference between the two populations,

H1 : [1 - 12 # 0 (There is a mean difference.)

Equivalently, the alternative hypothesis can simply state that the two population means are not equal: u1 # 2 .

University of the Sunshine Coast | CRICOS Provider Number: 01595D

See mini-lectorial 1 for worked example

3.1. Between subjects - independent samples t-test

The formulae for an independent samples hypothesis test

The overall t formula uses the difference between two sample means to evaluate a hypothesis about the difference between two population means. Thus, the independent-measures t formula is:

t = sample mean difference - population mean difference estimated standard error

= (M1 - Μ2) - (μ1 - 2) S(M1-M2)

The estimate standard error (S(M1 -M2)) measures the amount of error that is expected when you use a sample mean difference (M1 - M2) to represent a population mean difference (u1 - u2). The estimated standard error of M1 - M2 (S(M1- M2)) is how much difference is reasonable to expect between two sample means if the null hypothesis is true:

M -M2) = 5 + 5 n1 n2

S2 S2 In this formula, the value of M1 - M2 is obtained from the sample data and the value for u1 - 12 comes from the null hypothesis. As the null hypothesis is that HO : u1 - 12 = 0; then then expression (u1 So substituting the standard error formula into the t formula: - [2) = 0. So the resulting formula for a t-test is:

t = (M1- M2) S(M1-M2)

t = (M1- M2) n2 62 V S2 + 5% n1

The key thing to understand about the t formula is that it has included in it:

1. The difference in the means between 2 groups (M)
2. A computation of the variance in each group (s2),
3. Takes into account the sample size (n) for each group.

University of the Sunshine Coast | CRICOS Provider Number: 01595D

See mini-lectorial 1 for worked example

3.2. Within subjects - paired-samples t-test

The formulae for a paired samples hypothesis test

Only source of variance is intra-individual change (within-subjects).

As a result, these designs result in data that is less "noisy" - meaning that variability in scores is not attributable to some underlying difference between participants in the 2 groups.

The paired t formula Because each participant is tested on 2 occasions, and the participants are the same, we need to compute the mean difference (MD) in performance across the 2 testing sessions. Then we divide that by the estimate standard error of the mean difference score (SMD). So the formula for deriving t is:

t = MD SMD

Mean difference (MD) is simply the average of the differences in the performance of each participant in the study:

MD = Σ (Χ2-Χ1) n

Standard error of the mean difference To calculate the standard error of the mean difference. First need to calculate the Sum of Scores (SS) which is derived from using the total of the difference scores (ED) as well as the variance of the difference scores [(ED)2 / n] in the following formula:

SS = ED2. (ED)2 n

The next step is to calculate the sample using the SS computed in the following formula:

SS n -1

Now compute the standard error of the mean differences, using the following formula:

s2 n SMD V or S Vn

University of the Sunshine Coast | CRICOS Provider Number: 01595D

DON'T PANIC AND CARRY A TOWEL

USES FOR YOUR TOWEL:

• 1-Wave as a distress signal
• 2 - Soak it and use as a weapon
• 3- Cover your face against fumes
• 4 - Hide from bugblatters
• 5 - Use as a sail
• 6 - Small blanket
• 7- Dry off

4. Single IV, >2 level designs

When are more than 2 levels of the IV (e.g., 3 or more groups) where the data collected from each participant occurs only in one of the levels (i.e., it is an IV), then we need a technique for comparing differences for more than 2 groups.

• Where there are 2 levels of an IV (i.e., 2 groups) we use an independent samples t-test; which calculated the mean difference between the 2 groups and dived that by the mean standard error.

Where we have 3 or more groups to compare, the solution is a simple extension to this basic formula, but instead of a t statistic, we calculate the F ratio statistic.

F = MSwithin MS between

• There are series of computations required to reach the F statistic, which we will look over now (don't panic). The focus of this discussion is to encourage you to understand the concept and process of ANOVA models - you are not going to have to calculate anything.
• Let us start with a basic data set: In this example, we have a single IV and a single DV - the IV (group) has 3 independent levels and the DV (recall score) is a continuous variable. We therefore meet the requirements for a parametric test. As we have more than 2 groups, a t-test is not appropriate, so we reach for the most appropriate test of the hypothesis that there are group differences in recall .... A one-way ANOVA.

University of the Sunshine Coast | CRICOS Provider Number: 01595D

Basic Data Set Example

Let us start with a basic data set:

In this example, we have:

• 1 IV and a 1 DV
• the IV (group) has 3 independent levels
• the DV (recall score) is a continuous variable.
• As we have more than 2 groups, a t-test is not appropriate, so we reach for the most appropriate test of the hypothesis that there are group differences in recall .... A one-way ANOVA.

Recall score

Group 1 Group 2 Group 3 4 0 1 ΣΧ2 = 106 3 1 2 6 3 2 3 1 0 4 0 0 Total 20 5 5 Overall total = 30 SS 6 6 4 n 5 5 5 Total sample (N) = 15 Mean 4 1 1

Recall score Data with Variance

Group 1 Group 2 Group 3 4 0 1 ΣΧ2 = 106 3 1 2 6 3 2

Between-group variance

3 1 0 4 0 0 Total 20 5 5 Overall total = 30 SS 6 6 4 n 5 5 5 Total sample (N) = 15 Mean 4 1 1

Within-group variance

The basic premise of the ANOVA model is that there are 2 sources of variance that need to be accounted for:

• variability that occurs between members of different groups - between-group variance.
• variability that occurs between participants inside each group - within-group variance (also called "error variance").

University of the Sunshine Coast | CRICOS Provider Number: 01595D