Detecting local dependence in latent class models

Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependence in
latent class models
Daniel Oberski
Department of methodology and statistics
(Based on joint work with Jeroen Vermunt and Geert Van Kollenburg)
Detecting local dependence in latent class models Daniel Oberski

Latent class analysis
• Latent class analysis (LCA) used for: model-based
classification, clustering, latent structure analysis,
estimating false positives and false negatives, (specificity
and sensitivity) of error-prone variables;
• assumes local independence;

Latent class analysis
• Latent class analysis (LCA) used for: model-based
classification, clustering, latent structure analysis,
estimating false positives and false negatives, (specificity
and sensitivity) of error-prone variables;
• assumes local independence;
• If local dependence, severe bias can occur.

Detecting local dependence in LCA
• Local dependence in LCA leads to model misfit;
• Usual reaction: increase number of classes;
This talk:
• Do not increase number of classes;
• Model the local dependencies directly;

This talk:
• Use specifically tailored measures to detect which local
dependencies should be modeled:

This talk:
• Bivariate residuals (BVR),

This talk:
• Score test or ``modification index'' (MI),

This talk:
• Score test or ``modification index'' (MI),
• Expected parameter change (EPC).

..1 Example LCA
..2 Local dependence
..3 BVR and MI
..4 EPC
..5 Conclusions

Example LCA: dentists rating caries in xrays

Here is an xray of a person who possibly has caries:

y1 : Yes (1)
y2 : No (0)

y1 : Yes (1)
y2 : No (0)
y3 : No (0)

y1 : Yes (1)
y2 : No (0)
y3 : No (0)
y4 : Yes (1)

Actual dentists' ratings collected by Espeland &
Handelman (1989)
> dentists
Var1 Var2 Var3 Var4 Var5 Observed
1 0 0 0 0 0 1880
2 0 0 0 0 1 789
3 0 0 0 1 0 43
4 0 0 0 1 1 75
5 0 0 1 0 0 23
6 0 0 1 0 1 63
7 0 0 1 1 0 8
8 0 0 1 1 1 22
9 0 1 0 0 0 188
10 0 1 0 0 1 191
11 0 1 0 1 0 17
... etc.

2-class local independence model for dentists

Local independence latent class model
Patternwise likelihood:
Pr(Y = y) =
∑
t
Pr(Y = y|ξ = t)Pr(ξ = t), (1)
where the conditional probability of the response patterns is
Pr(Y = y|ξ = t) =
exp(Xθ)
1T
exp(Xθ)
(2)
With X a design matrix containing:
• Observed variables main effects (intercepts);
• Latent class × observed variables interaction.

Estimation of local independence latent class model
Log-likelihood
ℓ(θ) = nT
log Pr(Y = y),
with vector n the observed frequency of each response
pattern. Total sample size is N. Maximum likelihood estimates
ˆθ = arg max
θ∈Rq
ℓ(θ)
by expectation-maximization, quasi-Newton, or a combination.
This gives expected frequencies ˆµ := N · Pr(Y = y|θ = ˆθ)

2-class local independence model for dentists' ratings
Possible goals:
Goal: How LCA is useful:
* See how well dentists rate xrays ``sensitivity'' & ``specificity''
(conditional probabilities)
from intercepts and slopes
* Classify each photograph Pattern posteriors give
probabilistic classification
* Estimate ``true'' caries prevalence Latent class proportions

2-class local independence model for dentists' ratings
Possible goals:
Goal: How LCA is useful:
* See how well dentists rate xrays ``sensitivity'' & ``specificity''
(conditional probabilities)
from intercepts and slopes
* Classify each photograph Pattern posteriors give
probabilistic classification
* Estimate ``true'' caries prevalence Latent class proportions
Local dependence biases all of these results.

Questions for you
• What does the latent class variable represent?
• And if the number of classes is increased?

Questions for you
Model fit:
L2 X2 df BIC AIC AIC3 CAIC
129.85 132.00 20 -35.37 89.85 69.85 -35.37

Questions for you
Model fit:
L2 X2 df BIC AIC AIC3 CAIC
129.85 132.00 20 -35.37 89.85 69.85 -35.37
• The model does not fit. What could be a reason?
• What should be done?

Local dependence latent class models

2-class local dependence model for dentists

Local dependence latent class model
Conditional probability of the response patterns still
Pr(Y = y|ξ = t) =
exp(Xθ)
1T
exp(Xθ)
But now X is a design matrix containing:
• Observed variables main effects (intercepts);
• Latent class × observed variables interaction;
• (Some) observed variables × observed variables 2-way
interactions;

• Can we find a locally dependent two-class model that fits?
• Which local dependence parameters are needed?

• Can we find a locally dependent two-class model that fits?
• Which local dependence parameters are needed?
•• Goal: Examine fit of each pair of variables (dentist ratings)
to the local independence assumption without fitting all
possible alternative models.

Detecting local dependencies with the BVR and MI

Fit of two-way cross-table between dentists 1 and 3
Observed
No Yes
No 3250 280
Yes 123 216
Expected
No Yes
No 3217 313
Yes 156 183
Bivariate residuals
No Yes
No 32.6 -32.6
Yes -32.6 32.6

Observed
No Yes
No 3250 280
Yes 123 216
Expected
No Yes
No 3217 313
Yes 156 183
Bivariate residuals
No Yes
No 32.6 -32.6
Yes -32.6 32.6
BVR1,3 = r11
2
∑
k,l
ˆµ−1
kl
= (32.6)2
∑
k,l
ˆµ−1
kl
≈ 1063(0.0154) ≈ 16.3

Observed
No Yes
No 3250 280
Yes 123 216
Expected
No Yes
No 3217 313
Yes 156 183
Bivariate residuals
No Yes
No 32.6 -32.6
Yes -32.6 32.6
BVR1,3 = r11
2
∑
k,l
ˆµ−1
kl
= (32.6)2
∑
k,l
ˆµ−1
kl
≈ 1063(0.0154) ≈ 16.3
MI1,3 = r11
2
Var(r11)−1
= (32.6)2
/(31.3) ≈ 1063(0.0320) ≈ 34.0

Local dependencies between five dentists' x-ray
ratings for caries.
Dentist dependence MI BVR
1 ↔ 2 3.1 1.4
1 ↔ 3 34.0 ** 16.3
1 ↔ 4 13.1 ** 7.7
1 ↔ 5 2.7 0.8
2 ↔ 3 6.8 * 1.7
2 ↔ 4 1.8 0.6
2 ↔ 5 16.4 ** 4.7
3 ↔ 4 2.7 1.0
3 ↔ 5 5.1 * 0.8
4 ↔ 5 3.5 1.0
Note: ** Sig. (α = 0.05) with Bonferroni correction for multiple
testing, * without correction

BVR and MI
``Bivariate residual'' (BVR):
• Pearson "chi-square" residual in cross-table of a pair of
observed variables:
∑
(observed - expected)2/expected
• Surprise! Not a chi square statistic!

q
q
q
q
q
q
q qq
q qq
qqqqqqqqqqqqqqqqqq
q
q q
q qq
q
q
q
q
q
q
0
20
40
60
80
100
0
20
40
60
80
100
MI equals chi−square improvement...
χ2
MI
q
q
q
q
q
q
q qq
q qq
qqqqqqqqqqqqqqqqqq q q q
q qq
q q qq
q
q
0
20
40
60
80
100
0
20
40
60
80
100
... BVR does not.
χ2
BVR

BVR and MI
• Pearson residual in cross-table of a pair of observed
variables: (observed - expected)2/expected;
• Not a chi square statistic.
``Modification index'' (MI):
• ``Score test'' (``Lagrange multiplier'' test) for introducing a
local dependency between a pair of variables;
• Turns out to use the same residual as the BVR, but with
the correct variance! (thanks Jeroen)

BVR and MI
• Pearson residual in cross-table of a pair of observed
variables: (observed - expected)2/expected;
• Not a chi square statistic.
• ``Score test'' (``Lagrange multiplier'' test) for introducing a
local dependency between a pair of variables;
• Turns out to use the same residual as the BVR, but with
the correct variance! (thanks Jeroen)
• Under the null hypothesis, chi-square distributed (1 df)

BVR not even close to chi-square distributed, MI is.
Monte Carlo simulation under the null hypothesis
Condition α for nominal 5% Empirical distribution
BVR MI BVR
λ n Naive Boot MI Mean Var Mean Var
0.5 200 0.000 0.050 0.051 0.97 1.7 0.33 0.2
0.5 500 0.000 0.020 0.050 1.06 2.3 0.36 0.2
0.5 1000 0.000 0.060 0.065 0.96 1.9 0.33 0.2
0.5 5000 0.000 0.085 0.055 0.97 2.0 0.34 0.2
0.8 200 0.000 0.065 0.040 1.04 1.7 0.25 0.1
0.8 500 0.000 0.070 0.060 1.05 2.0 0.25 0.1
0.8 1000 0.000 0.060 0.090 1.22 2.6 0.30 0.2
0.8 5000 0.000 0.035 0.060 1.16 3.1 0.28 0.2
Should be: 0.050 0.050 0.050 1.00 2.0 1.00 2.0

BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
* See paper.

• BVR does not provide nominal α levels;
* See paper.

• BVR power to detect local dependencies miserably low*.
* See paper.

• Referring MI to chi-square:
* See paper.

• MI provides approximately nominal α levels;
* See paper.

• Power of MI is adequate except for very small
dependencies (±0.05)*.
* See paper.

• Power of MI is adequate except for very small
dependencies (±0.05)*.
•• Can bootstrap BVR values, leading to performance similar
to, though slightly below, MI*
* See paper.

Assessing substantive size of local dependencies with the EPC

Assessing substantive size besides statistical
siginificance
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;

siginificance
• Only statistical significance → MI depends on:

siginificance
• Sample size;

siginificance
• Sample size;
• Size of the loadings of the two variables;

siginificance
• Sample size;
• Size of the loadings of the other variables;

siginificance
• Sample size;
• Latent variable intercept(s);

siginificance
• Sample size;
• Observed variable intercepts;

siginificance
• Sample size;
• Observed variable intercepts;
• Regression coefficients of covariates if present.

siginificance
``Expected parameter change'' (EPC):
• Estimate of ψ, the strength of the local dependence
between two variables;
• No need to fit alternative model;

siginificance
``Expected parameter change'' (EPC):
• Estimate of ψ, the strength of the local dependence
between two variables;
• No need to fit alternative model;
• Assesses substantive rather than statistical significance.

Local dependencies between five dentists' x-ray
ratings for caries.
Dentist dependence EPC MI BVR
1 ↔ 2 -0.081 3.1 1.4
1 ↔ 3 -0.261 34.0 ** 16.3
1 ↔ 4 -0.146 13.1 ** 7.7
1 ↔ 5 -0.117 2.7 0.8
2 ↔ 3 -0.140 6.8 * 1.7
2 ↔ 4 -0.058 1.8 0.6
2 ↔ 5 -0.157 16.4 ** 4.7
3 ↔ 4 0.074 2.7 1.0
3 ↔ 5 -0.191 5.1 * 0.8
4 ↔ 5 -0.104 3.5 1.0
Note: ** Sig. (α = 0.05) with Bonferroni correction for multiple
testing, * without correction

Based on EPC and MI,
• keep two-classes, but...
• ...free five out of ten possible bivariate local
dependencies.

dependencies.
•• 15 degrees of freedom (was 20)

dependencies.
• L2
= 28.4 (p = 0.07) (was 129.9)

dependencies.
• L2
= 28.4 (p = 0.07) (was 129.9)
• BIC is −95.5 (was −35.37)

dependencies.
• L2
= 28.4 (p = 0.07) (was 129.9)
• BIC is −95.5 (was −35.37)
• Best two-class model in lit. (Qu et al. 1996): BIC −83.4

Conclusions

Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;

Conclusions
• Need to monitor local dependence.

Conclusions
•• Introduced MI and BVR, tests for local dependence:

Conclusions
• MI works well as chi-square statistic;

Conclusions
• BVR does not, but can bootstrap p-values.

Conclusions
• Introduced EPC: local dependence substantive size.

Conclusions
• Introduced EPC: local dependence substantive size.
•• Dentist example: obtained more parsimonious and
easy-to-interpret two-class model than current literature.

• Class-specific local dependence;
• Multivariate MI, EPC, (polytomous items);
• Effect of freeing local dependencies on params of interest;
• Problem of parameter dependence;
• Problem of multiple comparisons;
• (Post-hoc) power of MI (Satorra);
• MI for other params than locdeps (e.g. item bias).

Thank you for your attention!
Daniel Oberski
doberski@uvt.nl
Working papers on this topic:
Oberski, D., Van Kollenburg, G., and Vermunt, J. (submitted). A
Monte Carlo evaluation of three methods to detect local
dependence in binary data latent class models.
Oberski, D. and Vermunt, J. (submitted). The Expected Parameter
Change (EPC) for local dependence assessment in binary data
latent class models.
Oberski, D. (submitted). Change in SEM parameters of interest as
a criterion for partial measurement invariance: The
EPC-interest.

Appendix
Appendix

Appendix
0.5 0.8
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
−0.4−0.2−0.050.050.20.4
200
500
1000
5000
200
500
1000
5000
Log(sample size)
Power
p−value type
BVR
BVR_bootstrap
MI

Appendix
Convergence to noncentral chi-square in worst
condition
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qq
q
q
Sample size: 200
Q−Q plot, noncentral χ1
2
(4.7)
Theoretical quantiles
MI
0
10
20
30
40
0 5 10 15 20 25
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q q
Sample size: 500
2
(11.8)
MI
0
10
20
30
40
50
60
70
0 10 20 30 40
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
Sample size: 1000
2
(23.7)
MI
0
20
40
60
80
10 20 30 40 50 60
q
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqq
qq
q
Sample size: 5000
2
(118)
MI
50
100
150
200
80 120 160

Appendix
Local dependencies between indicators of Hispanic
ethnicity in the U.S. Census
Local dependence EPCL TL EPCgs TGS
Ancestry-re ↔ Language-in 0.92 5.0 1.45 7.9 0.
Ancestry-re ↔ Origin-in -0.76 2.5 -1.23 4.1 -0.
Ancestry-re ↔ Origin-re 2.94 45.6 1.32 20.5 1.
Language-in ↔ Origin-in 4.14 97.1 1.59 37.2 3.
Language-in ↔ Origin-re -1.08 7.9 -1.76 12.8 1.
Origin-in ↔ Origin-re 1.10 6.1 2.20 12.2 0.

Detecting local dependence in latent class models

More Related Content

Viewers also liked (9)

Recently uploaded (20)

Detecting local dependence in latent class models