Mixed Effects Models - Simple and Main Effects

Week 7.2: Simple vs. Main Effects
! Follow-ups
! Fixed Effect Correlation
! ICC
! Categorical Variables with 2 Categories
! Intro & Terminology
! Treatment Coding
! What it Does
! How to Change Codes
! Interactions
! Effects Coding
! One Variable
! Two Variables
! Sign & Scale Changes
! Simple Effects vs. Main Effects
! Unbalanced Factors

Follow-up #1: Fixed Effect Correlations
• What does this mean?

• Remember how we found our model
results: Iterative estimation until
convergence
• verbose=2
Each of these
iterations
represents a
different set of
estimates for our
fixed effects
Intercept estimate #1 NumDots estimate #1

convergence
• verbose=2
Each of these
iterations
represents a
different set of
estimates for our
fixed effects
Sometimes each
estimate may be
bigger and
sometimes smaller

convergence
• verbose=2
Each of these
iterations
represents a
different set of
estimates for our
fixed effects
Sometimes each
estimate may be
bigger and
sometimes smaller
Fixed effect correlation is whether
these two columns are correlated
i.e., on iteration steps where we
estimate a larger intercept, do we
estimate a larger or smaller
NumDots effect?

• What can we do with this information?
• In most cases, very little
• Very high correlations (e.g., .90) could be a sign
of multicollinearity
• Estimator can’t separate the two effects
• Can be used to verify centering worked
• Correlations with intercept will be close to 0 IF you
centered
UNCENTERED CENTERED

Follow-up #2: ICC
• One way we can assess the importance
of our clustering variables is with the intra-
class correlation coefficient (ICC)
• Remember our math data?
School
1
School
2
Sampled SCHOOLS
Sampled
CLASSROOMS
Sampled STUDENTS
LEVEL 3
LEVEL 2
LEVEL 1
Student
1
Student
2
Student
3
Student
4
Mr.
Wagner’s
Class
Ms.
Fulton’s
Class
Ms.
Green’s
Class
Ms.
Cornell’s
Class

Follow-up #2: ICC
• 2 clustering variables here:
• Classrooms
• Schools
• How much of the total variance do these
clustering variables account for?
• We want to look the raw variance, not “explained”
by any fixed effect
• Empty Means model:
• empty.means.model <- lmer(FinalMathScore ~
1 + (1|Classroom) + (1|School), data=math)

Follow-up #2: ICC
• Empty Means results:
• library(performance)
• icc(empty.means.model, by_group=TRUE)
}
3 sources of variance
present in the data.
Which is most
important?
ICC =
Variance of Random Effect of Interest
Sum of All Random Effect Variances
=
Classroom Variance
Classroom Variance + School Variance +
Residual Variance
12% of the variance can be attributed to Classroom differences
2% of the variance can be attributed to School differences
The remaining 86% is at the student level (residual)

Follow-up #2: ICC
• library(performance) (must install the package first)
• icc(empty.means.model, by_group=FALSE)
}
Which is most
important?
ICC =
=
Classroom Variance
Residual Variance
Total explained by all random
effects added together
(Classroom + School) = 14.5%

Follow-up #2: ICC
• Likelihood-ratio tests tell us whether random
effects significantly contribute to the model
(yes/no)
• ICC tells us about the relative importance
(≈ “effect size”)
}
Which is most
important?
ICC =
=
Classroom Variance
Residual Variance

Today’s Dataset
! Task: Decide whether a picture matches a
sentence; measure RT

“The dog was chased by the man.”
Today’s Dataset

! Each Item: Unique sentence w/ a unique picture
! No picture or sentence repeats
“The dog was chased
by the man.”
“The bee stung the man.”
Today’s Dataset

! 16 people with aphasia and 16 healthy controls
(SubjectType)
! All participants see the same sentences, which
vary in SentenceType (Active or Passive)
! Active (more common): “Dog bites man.”
! Subject of the sentence is performing the action
! Passive: “The dog was bitten by the man.”
! Subject of the sentence is having an action performed to them
Today’s Dataset

(SubjectType)
! Which fixed effect(s) are between-subjects?
! Which fixed effect(s) are within-subjects?
! Hint: Imagine we had only 1 subject. If we could still
test the effect of a variable, it’s within-subjects.
Today’s Dataset

(SubjectType)
! Which fixed effect(s) are between-subjects?
! SubjectType
! Which fixed effect(s) are within-subjects?
! SentenceType
Today’s Dataset

(SubjectType)
! Which fixed effect(s) are between-items?
! Which fixed effect(s) are within-items?
! Hint: Imagine we had only 1 sentence. If we could
still test the effect of a variable, it’s within-items.
Today’s Dataset

(SubjectType)
! Which fixed effect(s) are between-items?
! SentenceType
! Which fixed effect(s) are within-items?
! SubjectType
Today’s Dataset

The Big Picture
! Both SentenceType and SubjectType are
categorical variables
! When categorical variables are independent /
predictor variables, need to make choices about
how to code them
! Allow us to answer different questions about the
data
! In most cases, multiple statistically valid ways to
code
! But, important that we actually perform the test that
corresponds to what we say we want to know

Terminology
• Factor: A variable with a fixed set
of categories
• as.factor() tells R to treat a
variable as categorical
• Displays frequency counts in summary()
aphasia %>%
mutate(SentenceType=as.factor(SentenceType),
SubjectType=as.factor(SubjectType)) -> aphasia

Terminology
• Factor: A variable with a fixed set
of categories
• as.factor() tells R to treat a
variable as categorical
• Displays frequency counts in summary()
• Levels: The individual categories
within a factor
• “Active” versus “Passive”
• “Aphasia” versus “Healthy control”
• …whether experimental or observational
• Today, we will look at cases where
the factor has 2 levels

Terminology
• Factorial Design: A design where each
combination of levels appears
• Common in experimental (& quasi-experimental)
contexts where we manipulate each variable
ACTIVE, APHASIA
240 observations
PASSIVE,
APHASIA
240 observations
ACTIVE,
CONTROL
240 observations
PASSIVE,
CONTROL
240 observations
SubjectType
SentenceType
Active Passive
Aphasia
Control

Terminology
• Factorial Design: A design where each
combination of levels appears
• Common in experimental (& quasi-experimental)
contexts where we manipulate each variable
• Cell: One individual combination
ACTIVE, APHASIA
240 observations
PASSIVE,
APHASIA
240 observations
ACTIVE,
CONTROL
240 observations
PASSIVE,
CONTROL
240 observations
SubjectType
SentenceType
Active Passive
Aphasia
Control

• So far, we’ve been writing regression
equations with numbers
• But what about active vs passive sentence?
Introduction to Contrast Coding
= Intercept + +
NumDots
RT γ100 γ200
5 4
= Intercept +
RT γ100
MathAnxiety

• R’s “secret decoder wheel”
assigns numerical coding
schemes:
• Variable with 2 categories (this week): Only one
comparison needed
• Variables with more categories: Multiple contrasts
= Intercept +
RT γ100
Active sentence: 0
Passive sentence: 1

• R’s “secret decoder wheel”
assigns numerical coding
schemes
• See the current codes:
• contrasts(aphasia$SentenceType)
= Intercept +
RT γ100
Active sentence: 0
Passive sentence: 1

Treatment Coding (Dummy Coding)
• R’s default system
• One baseline/reference level (category)
is coded as 0
• The other (the treatment) is coded as 1
• Remember, today we are just looking at
factors with 2 levels
• Default ordering is alphabetical: First
level is 0, second is 1
• We’ll see how to change this soon
• contrasts(aphasia$SentenceType)
Active coded as 0
Passive coded as 1

• Let’s do a model that just examines the effect of
sentence type in this task:
• SentenceTypeModel <- lmer(RT ~
1 + SentenceType +
(1 + SentenceType|Subject) +
(1|Item),
data = aphasia)
• Hint: SentenceType varies within-subjects, but only between
items
SUBJECT RANDOM EFFECTS
ITEM RANDOM EFFECTS

• Let’s do a model that just examines the effect of
sentence type in this task:
• SentenceTypeModel <- lmer(RT ~
1 + SentenceType +
(1|Item),
data = aphasia)

• Let’s think about what the model looks like for
each of our two conditions:
RT =γ000 + γ100 * SentenceType
Active
Sentences
Passive
Sentences

Active
Sentences
Passive
Sentences
?
?

Active
Sentences
Passive
Sentences
0
1

RT =γ000
Active
Sentences
Passive
Sentences 1
Intercept (γ000) is just the
mean RT for active
sentences

RT =γ000
RT =γ000 + γ100
Active
Sentences
Passive
Sentences
Intercept (γ000) is just the
mean RT for active
sentences
What is the difference between the equations
for the two sentence types?
SentenceType effect (γ100)
is the difference in RT
between passive & active
sentences

Treatment Coding Results
• Treatment coding makes one level the
baseline and compares everything to that
Intercept (γ000) : RT for active
sentences is 1758 ms
SentenceType (γ100): Passive
sentences take an additional
672 ms to read

Changing Codes
• We should think about adding SubjectType to
the model. Let’s check the codes:
• contrasts(aphasia$SubjectType)
• But, Control is really the baseline category here
• Assign new codes by using <- :
• contrasts(aphasia$SubjectType)
<- c(1,0)
• New codes are in the order you see
above & with summary() CONCATENATE

Changing Codes
• Need to set codes before you run the model!

Treatment Coding: Two Variables
• Now, we’d like SentenceType and SubjectType to
interact:
• Model.Maximal <- lmer(RT ~
1 + SentenceType * SubjectType +
(1 + SubjectType|Item),
data = aphasia)
• Hint #1: Remember that we can include a random slope by subjects for within-
subjects variables but not for between-subjects variables
• Hint #2: Does each subject see more than one SentenceType? Is each subject
more than one SubjectType?
SUBJECT RANDOM EFFECTS
ITEM RANDOM EFFECTS

interact:
data = aphasia)
• Hint #1: Remember that we can include a random slope by items for within-
items variables but not for between-items variables
• Hint #2: Is each item presented as more than one SentenceType? Is each item
presented to more than one SubjectType?
ITEM RANDOM EFFECTS

interact:
data = aphasia)

• Our design now has four cells:
RT =γ000 + γ100SentenceType +
γ200SubjectType + γ1200SentenceTypeSubjectType
Active,
Control Subj.
Passive,
Control Subj.
Active,
Aphasics
Passive,
Aphasics

Active,
Control Subj.
Passive,
Control Subj.
Active,
Aphasics
Passive,
Aphasics
0
0 0 0

RT =γ000
Active,
Control Subj.
Passive,
Control Subj.
Active,
Aphasics
Passive,
Aphasics
Intercept (γ000) is the RT when all
variables at their baseline: active
sentence type, healthy control subject
1
0 0
1

RT =γ000
Active,
Control Subj.
RT =γ000 + γ100
Passive,
Control Subj.
Active,
Aphasics
Passive,
Aphasics
SentenceType (γ100) : Passive
vs active difference for baseline
healthy controls
1
0
1
0

RT =γ000
Active,
Control Subj.
RT =γ000 + γ100
Passive,
Control Subj.
RT =γ000 +
γ200
Active,
Aphasics
Passive,
Aphasics
healthy controls
SubjectType (γ200): Aphasia vs control
difference for baseline active
sentences

RT =γ000
Active,
Control Subj.
RT =γ000 + γ100
Passive,
Control Subj.
RT =γ000 +
γ200
Active,
Aphasics
Passive,
Aphasics
healthy controls
sentences
If no special effect of passive sentence and
aphasia, we’d just have these two effects
1
1

RT =γ000
Active,
Control Subj.
RT =γ000 + γ100
Passive,
Control Subj.
RT =γ000 +
γ200
Active,
Aphasics
RT =γ000 + γ100 +
γ200
Passive,
Aphasics
healthy controls
sentences
If no special effect of passive sentence and
aphasia, we’d just have these two effects

RT =γ000
Active,
Control Subj.
RT =γ000 + γ100
Passive,
Control Subj.
RT =γ000 +
γ200
Active,
Aphasics
RT =γ000 + γ100 +
γ200 + γ1200SentenceTypeSubjectType
Passive,
Aphasics
healthy controls
sentences
1 1

RT =γ000
Active,
Control Subj.
RT =γ000 + γ100
Passive,
Control Subj.
RT =γ000 +
γ200
Active,
Aphasics
RT =γ000 + γ100 +
γ200 + γ1200
Passive,
Aphasics
healthy controls
sentences
Interaction: Special effect of
aphasia and passive sentence

Treatment Coding: Model Results
Intercept: RT for healthy
controls, active sentences
Significant RT difference for
passive sentences (among
healthy controls)
Not a significant RT
difference for aphasics
(among active
sentences)
Significant special effect of
aphasia + passive sentence

Treatment Coding: Model Results
Intercept: RT for healthy
controls, active sentences
passive sentences (among
healthy controls)
Not a significant RT
(among active
sentences)
Even though the SubjectType effect is not significant here,
we would not want to remove it from the model. It doesn’t
make sense to include the interaction without the lower-
order terms—the interaction is defined by what’s different
from the two simple effects alone.

Effects Coding (Sum Coding)
• So far, the intercept at 0 has referred to a
particular baseline level
• Remember centering?
• When we centered, we made the intercept at 0
correspond to the overall mean

• We can apply centering to a factor, too
• SentenceType has:
• 480 “Active” observations (currently 0)
• 480 “Passive”s (currently 1)
• Mean of 0.5
• Subtracting the mean from each code gives
us a new set of codes
Subtract 0.5
Subtract 0.5

• We can apply centering to a factor, too
• SentenceType has:
• 480 “Active” observations (currently 0)
• 480 “Passive”s (currently 1)
• Mean of 0.5
• Subtracting the mean from each code gives
us a new set of codes
• Effects coding (a/k/a sum coding): -0.5, 0.5
Subtract 0.5
Subtract 0.5

• Apply effects coding (-0.5, 0.5) to our two
sentence types:
Active
Sentences
Passive
Sentences
?
?

sentence types:
Active
Sentences
Passive
Sentences
-0.5
0.5
Imagine subtracting the equations.
The difference between the equations for
the two conditions is equal to what?

sentence types:
Active
Sentences
Passive
Sentences
-0.5
0.5
The equations differ by 1 γ100
SentenceType effect (γ100) is (still)
the difference between conditions

sentence types:
Active
Sentences
Passive
Sentences
-0.5
0.5
Intercept is what you expect when the
other variables are 0. Where does a
value of x=0 fall in this coding scheme?

sentence types:
Active
Sentences
Passive
Sentences
-0.5
0.5
Intercept is what you expect when the
other variables are 0. Where does a
value of x=0 fall in this coding scheme?
0
0 is midway between
Active and Passive;
i.e., the average of the
conditions

sentence types:
Active
Sentences
Passive
Sentences
-0.5
0.5
SentenceType effect (γ100) is (still)
the difference between conditions
Intercept (γ000) is now the mean RT
across all conditions.

• Let’s first go back to our model with only
SentenceType and try it with effects coding
• Old codes:
• New codes:
• contrasts(aphasia$SentenceType) <- c(-0.5,0.5)
SENTENCETYPE
SENTENCETYPE

Effects Coding Results
TREATMENT
CODING
EFFECTS
CODING
Same
SentenceType
effect
Different
Intercepts
(Active
baseline vs.
mean)

• Now let’s look at both variables together:
• Old codes:
• New codes:
• contrasts(aphasia$SubjectType) <- c(0.5,-0.5)
SENTENCETYPE SUBJECTTYPE
SENTENCETYPE SUBJECTTYPE

Effects Coding: Full Model Results
Intercept: Now mean RT overall
Significant overall RT
difference for passive vs
active sentences (across all
subject types)
Significant overall RT
(across all sentence types)
No correlation w/ intercept--we’ve successfully centered

Effects Coding: Sign Changes
• We picked one condition to be -0.5 and one to be 0.5
• Here, Active was -0.5 and Passive was 0.5
• Should we worry that this affects our results?
• Let’s try it the other way and see if we get something else
• contrasts(aphasia$SentenceType) <- c(0.5, -0.5)
• Then, re-run the model
Hang on! You coded SentenceType by making
active sentences -0.5 and passive sentences 0.5
How do I know that’s the right way to do it?

Effects Coding: Sign Changes
• Active is -0.5, Passive is 0.5:
• Active is 0.5, Passive is -0.5:
• Flipping the signs of the code just changes the sign of
the results
• Doesn’t affect absolute value or significance
• Choose whichever makes sense for your question:
• “Passive is slower than Active” vs “Active is faster than
Passive”
“RT 671 ms longer for
Passive than for Active”
“RT 671 ms shorter for
Active than for Passive”

Effects Coding: Why -0.5 & 0.5?
PASSIVE
ACTIVE
.5
-.5
CONTRAST
CODE
}1
PASSIVE
ACTIVE
1
-1
CONTRAST
CODE
1 unit change in contrast IS
the difference between
sentence types
1 unit change in contrast IS
only half the difference
between levels
}2

! What if we used (-1, 1) instead?
! Doesn't affect significance test
! Does make it harder to interpret the estimate
! Parameter estimate is only half of the actual
difference in means
Effects Coding: Why -0.5 & 0.5?
SENTENCE
TYPE:
c(-1, 1)
SENTENCE
TYPE:
c(-0.5, 0.5)

Simple vs. Main Effects
• Treatment coding and effects coding also change
our interpretation of the non-intercept effects:
• Treatment coding (of SentenceType):
• Effect of SubjectType within the baseline
level of SentenceType
• “Simple effect”– not a “real” main effect
• Effects coding (of SentenceType):
• Overall effect of SubjectType averaged across
sentence types
• “Main effect”
Non-significant RT difference
for people with aphasia
(among active sentences)
people with aphasia (across
all sentence types)

Simple vs. Main Effects
• Again, both of these are, in principle, reasonable
questions to ask…
• In factorial designs, traditional to talk about the
main effects averaged across other variables
• “Main effect of aphasia,” “Overall effect of priming,” “Overall
effect of study strategy,” “Main effect of ambiguity”…
• If you want to talk about main effects in this way,
use effects coding, not treatment / dummy
coding!
• In other designs, treatment coding
may be the most appropriate!

Unbalanced Factors
• Sometimes, we may have differing
numbers of observations per level
• Possible reasons:
• Some categories naturally more common
• e.g., college majors
• Categories may be equally common in the
population, but we have sampling error
• e.g., ended up 60% female participants, 40% male
• Study was designed so that some conditions are
more common
• e.g., more “control” subjects than “intervention” subjects
• We wanted equal numbers of observations, but lost
some because of errors or exclusion criteria
• e.g., data loss due to computer problems
• Dropping subjects below a minimum level of performance

Weighted Coding
• “For the average student, does course size
predict probability of graduation?”
• Random sample of 200 Pitt undergrads
• 5 are student athletes and 195 are not
• How can we make the intercept reflect the
“average student”?
• We could try to apply effects coding to the
StudentAthlete variable by centering around the
mean and getting (0.5, -0.5), but…

Weighted Coding
• An intercept at 0 would no longer correspond to the
overall mean
• As a scale, this would be totally
unbalanced
• To fix balance, we need to assign
a heavier weight to Athlete
ATHLETE
(5)
NOT ATHLETE
(195)
.5 -.5
0
-.475
Zero is here
But “not athlete” is actually far more common

Weighted Coding
• Change codes so the mean is 0
• c(.975, -.025)
• contr.helmert.weighted() function in my
psycholing package will calculate this
ATHLETE
(5)
NOT ATHLETE
(195)
.975 -.025
0

Weighted Coding
• Weighted coding: Change the codes so that
the mean is 0 again
• Used when the imbalance reflects something real
• Like Type II sums of squares
• “For the average student, does course size
predict graduation rates?”
• Average student is not a student athlete, and our
answer to the question about an “average student”
should reflect this!

Unweighted Coding
• Oops! Our experiment loaded up the wrong image
for one of our Passive sentences (“Groceries”)
• It may have been sabotaged
• UsableItem column is No for this item
• First, can we remove this from our data?
• Some possibilities:
• aphasia %>% filter(UsableItem == 'Yes') -> aphasia
• aphasia %>% filter(UsableItem != 'No') -> aphasia2
• etc.

Unweighted Coding
• Now, there’s an imbalance, but it’s an accident and
not meaningful
• In fact, we’d like to get rid of it!

Unweighted Coding
• Now, there’s an imbalance, but it’s an accident and
not meaningful
• In fact, we’d like to get rid of it!
• Retain the (-0.5, 0.5) codes
• Weights the two conditions equally—because the
imbalance isn’t meaningful
• Like Type III sums of squares
• Probably what you want for factorial experiments

Unbalanced Factors: Summary
• Weighted coding: Change the codes so that the
mean is 0
• Use when the imbalance reflects something real
• Can be done with contr.helmert.weighted()
• Unweighted coding: Keep the codes as -0.5 and
0.5
• Use when the imbalance is an accident that we
want to eliminate
• With balanced factors, these are identical
Mean across each individual:
Mean of the two levels:
Mean of the
active sentences
Mean of the
passive sentences

Mixed Effects Models - Simple and Main Effects

More Related Content

What's hot (20)

Similar to Mixed Effects Models - Simple and Main Effects (20)

More from Scott Fraundorf (7)

Recently uploaded (20)

Mixed Effects Models - Simple and Main Effects