PSY400 Research Methods and Analysis 4: Multiple Regression Strategies

PSY400 Research Methods and Analysis 4

Dr Joshua Adie University of the Sunshine Coast | CRICOS Provider Number: 01595DUniSC

Week 5 Multivariate Predictor Designs

University of the Sunshine Coast | CRICOS Provider Number: 01595DReading - covers undertaking these techniques in SPSS

• Field, A. (2018) Discovering statistics using IBS SPSS statistics
• Chapters 9 & 20

Workshop content

1. Regression techniques (Multiple vs Logistic)
2. Type of multiple regression
3. Interpreting regression models
4. Logistic Regression (minlectorial - online)

University of the Sunshine Coast | CRICOS Provider Number: 01595D UniSC

Overview of Multivariate Methods

What type of relationship is being examined?

Dependence Multivariate technique

How many variables are being predicted?

Is the structure of the relationships among:

Multiple relationships of dependent and independent variables

Several dependent variables in a single relationship

One dependent variable in a single relationship

Variables Cases/respondents Objects

+

+

Structural Equation Modelling

What is the measurement scale of the DV?

What is the measurement scale of the DV?

Factor analysis Confirmatory factor analysis Cluster analysis

How are the attributes measured?

Metric Nonmetric Metric Nonmetric Metric Nonmetric

+

What is the measurement of the predictor variable?

Canonical correlation analysis with dummy variables

Multiple Regression

Multiple Discriminant Analysis

Conjoint analysis

Linear Probability Models

Metric Nonmetric

+

Canonical correlational analysis

MANOVA

University of the Sunshine Coast | CRICOS Provider Number: 01595D UniSC Nonmetric

Multidimensional scaling

Correspondence analysis

Interdependence Decision point

Regression Techniques

• Regression analyses - statistical techniques to assess the relationship between one DV and several IVs
• term regression is often used when the intent of the analysis is prediction,
• term correlation is used when the intent is simply to assess the relationship between the DV and the IVs.
• Regression techniques can be applied to a data where IVs are correlated with each another and with the DV - because the shared variance between IVs can be partialled out.

University of the Sunshine Coast | CRICOS Provider Number: 01595D UniSC

Multiple Regression Analysis

• Multiple regression - used to analyse the relationship between a single DV (criterion) and several IVs (predictors).
• Each IV is weighted by the regression analysis procedure to ensure maximal prediction from the set of IVs. The weights denote the relative contribution of the IVs to the overall prediction and allow interpretation as to the influence of each variable in making the prediction, although correlation among the independent variables complicates the interpretative process.
• regression variate = the set of weighted IVs, a linear combination of the IVs that best predicts the DV.

Basic structure of a multiple regression

IV1 a IV1 unique variance IV2 IV2 unique variance b Variate IV3 unique variance DV IV3 IVn unique variance IVn

University of the Sunshine Coast | CRICOS Provider Number: 01595D UniSC

• Multiple regression analysis is a dependence technique.
• must be able to divide the variables into DVs and IVs.
• Regression analysis is also a statistical tool that should be used only when both the DV and IVs are metric.
• under certain circumstances it is possible to include nonmetric data either as IVs (by transforming either ordinal or nominal data with dummy-variable coding) or the DV (by the use of a binary measure in the specialized technique of logistic regression).
• In summary, to apply multiple regression analysis:

1. the data must be metric or appropriately transformed, and
2. before deriving the regression equation, the researcher must decide which variable is to be the DV and which remaining variables will be IVs.

University of the Sunshine Coast | CRICOS Provider Number: 01595D UniSC

Logistic Regression

• Logistic regression - used to predict a discrete outcome (e.g., group membership) from a set of variables that may be continuous, discrete, dichotomous, or a mix.

Basic structure of a logistic regression

IV1 IV1 unique variance a IV2 IV2 unique variance Group 1 (DV level 1) Variate b IV3 unique variance IV3 IVn unique variance Group 2 (DV level 2) IVn

• Logistic regression ~ discriminant analysis (next module).
• But, logistic regression is more flexible than the other techniques
• has no assumptions about the distributions of the predictor variables (i.e. the predictors do not have to be normally distributed, linearly related to the DV, or of equal variance within each group, do not need to be discrete, and can be any mix of continuous, discrete, and dichotomous variables).

University of the Sunshine Coast | CRICOS Provider Number: 01595D UniSC

Multiple Regression: Extension of Bivariate Regression

Multiple regression is an extension of bivariate regression (e.g., Pearson's correlation) in which several IVs are combined to predict a value on a DV for each case.

The result of regression represents the best prediction of a DV from several continuous (or dichotomous) IVs:

Y' = A + B1X1 + BzX2 + ... + BkXk

Where:

• Y' = the predicted value on the DV
• A = the Y intercept (the value of Y when all the X values are 0)
• X = the various IVs (of which there are k)
• B = the coefficients assigned to each X during regression
• A different Y' value can be predicted for each participant in a sample by substituting the participants own X values into the equations (with the A and B values being determined by the regression for the sample).
• The goal of regression is to arrive at the set of B values (regression coefficients) for the IVs that bring the Y values predicted from the equation as close as possible to the Y values obtained by measurement. The regression coefficients:

a) minimize (the sum of the squared) deviations between predicted and obtained Y values, and b) optimize the correlation between the predicted and obtained Y values for the data set.

University of the Sunshine Coast | CRICOS Provider Number: 01595D UniSC

• one of the important statistics is the multiple-correlation coefficient = the Pearson product-moment correlation coefficient between the obtained and predicted Y values: R = ryy '.
• Multiple regression analysis is used to examine the relationship between a single DV and a set of IVs. A necessary starting point in multiple regression, as with all multivariate statistical techniques is the research problem
• In selecting suitable applications of multiple regression, the researcher must consider three primary issues :

1. the appropriateness of the research problem,
2. specification of a statistical relationship, and
3. selection of the dependent and independent variables.

Multiple regression decision diagram

Research problem

Select objectives: Prediction Explanation

Select dependent and independent variables

Research design issues

Obtain adequate sample size to ensure: Statistical power Generalizability

Creating additional variables Transformations to meet assumptions Dummy variables for use of nonmetric variables Polynomials for curvilinear relationships Interaction terms for moderator effects

No

Assumptions in multiple regression

. Do the individual variables meet the assumption of: Normality Linearity Homoscedasticity Independence of error terms

Selecting an estimation technique

Does the researcher wish to: 1. Specify the regression model, or 2. Utilise a regression procedure to select the independent variables to optimise prediction?

Sequential search method + - forward/backward estimation Stepwise estimation - Combinational approach All possible subsets

No

Does the regression variate meet the assumptions of regression analysis?

Yes

Examine statistical and practical significance Coefficient of determination Adjusted coefficient of determination Standard error of the estimate Statistical significance of the regression coefficients Identifying influential observations

Yes

Are there any observations determined to be influential that require deletion from the analysis?

No

. Interpreting the regression variate Evaluate the prediction equation with the regression coefficients Evaluate the relative importance of the IVs with the @ coefficients Assessing multicollinearity and its effects Validating the results Split-sample analysis .. PRESS statistic

University of the Sunshine Coast | CRICOS Provider Number: 01595D UniSC

2. Procedure selects

1. Analyst specification Specification of regression model by researcher

Appropriateness of the Research Problem

Prediction with multiple regression

• One purpose of multiple regression is to predict the DV with a set of IVs

1. the variate is the combination of IVs formed to be the optimal predictor of the DV. Multiple regression provides an objective means of assessing the predictive power of a set of IVs. In all cases, regression analysis must achieve acceptable levels of predictive accuracy to justify its application.
2. To compare 2+ sets of IVs to ascertain the predictive power of each variate. The primary focus of this type of analysis is the relative predictive power among models.

University of the Sunshine Coast | CRICOS Provider Number: 01595D M

Explanation with Multiple Regression

• Interpretation of the variate may rely on any of three perspectives:

" the importance of each IVs in the prediction " the types of relationships found (i.e. linear or other) " the interrelationships among the IVs - when IVs are highly correlated some will become redundant to the prediction.

IV1 a IV1 unique variance IV2 IV2 unique variance DV Variate IV3 unique variance IV3 IV1 a IV1 unique variance IV2 IV2 unique variance DV Variate IV3 unique variance IV3

University of the Sunshine Coast | CRICOS Provider Number: 01595D M

Selecting Independent and Dependent Variables

Selection of IVs and DVs should be based principally on conceptual or theoretical grounds. If variables are selected indiscriminately or based solely on empirical bases, several basic assumptions of model development will be violated.

Selection of a DV

The selection of a DV is usually dictated by the research problem - but be aware of the measurement error, especially in the DV.

Measurement error = the degree that the variable is an accurate and consistent measure of the concept being studied.

Selection of IVs

The most problematic issue in IV selection is specification error - the inclusion of irrelevant variables or the omission of relevant variables from the set of IVs. Inclusion of irrelevant variables:

1) reduces model parsimony, which may be crucial to the interpretation of the results. 2) additional variables may mask or replace the effects of more useful variables. 3) additional variables may make the testing of statistical significance of the IVs less precise and reduce the statistical and practical significance of the analysis.

The omission of relevant variables means that the variables' effect cannot be assessed without their inclusion. Hence the need for theoretical and practical support for all variables included or excluded in a multiple regression analysis.

University of the Sunshine Coast | CRICOS Provider Number: 01595D