Multidirectional survey measurement errors: the latent class MTMM model
1 like606 views
The document discusses the latent class multitrait-multimethod (MTMM) model, highlighting that stochastic errors in survey responses can bias estimates of relationships between variables. It emphasizes that while latent variables cannot be recovered, models can estimate their influence, thereby removing bias from estimates. The author proposes that traditional models assume unidirectional errors, and the latent class approach accommodates multidirectional errors to improve accuracy.
Multidirectional survey measurement errors: the latent class MTMM model
- 1. Multidirectional errors Latent class MTMM Conclusions Multidirectional survey measurement errors: the latent class MTMM model Daniel Oberski / doberski@uvt.nl Department of methodology & statistics AAPOR 2015 Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 2. Multidirectional errors Latent class MTMM Conclusions ..1 Multidirectional survey measurement errors ..2 Latent class multitrait-multimethod model ..3 Conclusions Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 3. Multidirectional errors Latent class MTMM Conclusions Stochastic errors Two respondents who: • Went to the doctor the same number of times • Have the same opinion about the role of women in society give different answers to these questions. This will bias estimates of relationships between the variables. Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 4. Multidirectional errors Latent class MTMM Conclusions Stochastic errors • Latent variables can never be recovered Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 5. Multidirectional errors Latent class MTMM Conclusions Stochastic errors • Latent variables can never be recovered • But latent variable models can recover the amount of influence they exert Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 6. Multidirectional errors Latent class MTMM Conclusions Stochastic errors • Latent variables can never be recovered • But latent variable models can recover the amount of influence they exert • This is useful to remove bias Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 7. Multidirectional errors Latent class MTMM Conclusions Stochastic errors • Latent variables can never be recovered • But latent variable models can recover the amount of influence they exert • This is useful to remove bias Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 8. Multidirectional errors Latent class MTMM Conclusions Stochastic errors • Latent variables can never be recovered • But latent variable models can recover the amount of influence they exert • This is useful to remove bias • Multitrait-multimethod (MTMM) approach to estimating this influence (Andrews 1984; Saris & Andrews 1991; Saris & Gallhofer 2007); Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 9. Multidirectional errors Latent class MTMM Conclusions Stochastic errors • Latent variables can never be recovered • But latent variable models can recover the amount of influence they exert • This is useful to remove bias • Multitrait-multimethod (MTMM) approach to estimating this influence (Andrews 1984; Saris & Andrews 1991; Saris & Gallhofer 2007); • Quasi-simplex approach (Wiley & Wiley 1970; Alwin 2007). Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 10. Multidirectional errors Latent class MTMM Conclusions Stochastic errors • Latent variables can never be recovered • But latent variable models can recover the amount of influence they exert • This is useful to remove bias • Multitrait-multimethod (MTMM) approach to estimating this influence (Andrews 1984; Saris & Andrews 1991; Saris & Gallhofer 2007); • Quasi-simplex approach (Wiley & Wiley 1970; Alwin 2007). Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 11. Multidirectional errors Latent class MTMM Conclusions Stochastic errors • Latent variables can never be recovered • But latent variable models can recover the amount of influence they exert • This is useful to remove bias • Multitrait-multimethod (MTMM) approach to estimating this influence (Andrews 1984; Saris & Andrews 1991; Saris & Gallhofer 2007); • Quasi-simplex approach (Wiley & Wiley 1970; Alwin 2007). ↑ Linear models Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 12. Multidirectional errors Latent class MTMM Conclusions Stochastic errors in the literature • Random mistakes • Acquiescence • Answering in the socially desirable direction • Tending to choose the first/last of several categories • Extreme response (outer categories) • Avoiding some particular category for whatever reason • Preferring the midpoint • Heaping (rounding) • ... Problem: Not all of these are linear Stochastic errors are multidirectional Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 13. Multidirectional errors Latent class MTMM Conclusions Example of the effect of a nonlinear error on estimate of relationships True relationship 1 2 3 4 1 136 112 91 75 2 112 75 50 34 3 91 50 28 15 4 75 34 15 7 Polychoric correlation = -0.25 Relationship with ERS 1 2 3 4 1 374 41 34 205 2 41 4 3 12 3 34 3 1 6 4 205 12 6 19 Polychoric correlation = -0.43 Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 14. Multidirectional errors Latent class MTMM Conclusions Problem: Stochastic errors can strongly bias relationship estimates; Problem: Linear latent variable models assume errors are all one-way, so bias is not appropriately removed. Solution: Latent class models to allow for multidirectional errors. Here MTMM but could also be quasi-simplex Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 15. Multidirectional errors Latent class MTMM Conclusions The latent class MTMM model Oberski, Hagenaars & Saris, to appear in Psychological Methods. Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 16. Multidirectional errors Latent class MTMM Conclusions M1 M2 M3 T1 T2 T3 y11 y21 y31 y12 y22 y32 y13 y23 y33 • Latent variables are discrete (categorical) • Observed may be continuous or discrete • Relationships (can be) nonparametric Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 17. Multidirectional errors Latent class MTMM Conclusions Experimental design: split-ballot MTMM Method 1 Method 2 Method 3 Random group 1 . . Random group 2 . . Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 18. Multidirectional errors Latent class MTMM Conclusions Opinion about the role of women experiment: Main questionnaire (first method) Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 19. Multidirectional errors Latent class MTMM Conclusions Opinion about the role of women experiment: Supplementary group 1 (second method) Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 20. Multidirectional errors Latent class MTMM Conclusions Opinion about the role of women experiment: Supplementary group 2 (third method) Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 21. Multidirectional errors Latent class MTMM Conclusions Effect of trait on item distribution, Greece Men more right, positive agree-disgree 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Latent class MTMM Daniel Oberski / doberski@uvt.nl 1 2 3 4 5 Factor score: 0.00 Category Proportion 0.00.20.40.60.81.0
- 22. Multidirectional errors Latent class MTMM Conclusions Effect of trait on item distribution, Slovenia Men more right, negative agree-disgree 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Latent class MTMM Daniel Oberski / doberski@uvt.nl 1 2 3 4 5 Factor score: 0.00 Category Proportion 0.00.20.40.60.81.0
- 23. Multidirectional errors Latent class MTMM Conclusions Method factors have a nonlinear influence on the items 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Cut down, method 1 Trait Method 1 2 34 5 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Cut down, method 2 Trait Method 1 2 3 4 5 0.0 0.2 0.4 0.6 0.8 1.0 0.00.20.40.60.81.0 Cut down, method 3 Trait Method 1 2 3 4 5 Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 24. Multidirectional errors Latent class MTMM Conclusions Prevalence of method behavior Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 25. Multidirectional errors Latent class MTMM Conclusions Conclusion Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 26. Multidirectional errors Latent class MTMM Conclusions Conclusion • Stochastic errors are important when you are interested in relationships; • Need latent variable models to estimate their extent so their effects can be removed statistically; • In traditional MTMM and quasi-simplex models, these effects are all one-way; • Latent class models allow for multidirectional errors. • Example: the latent class MTMM model (Oberski et al., to appear - see http://daob.nl/publications) Latent class MTMM Daniel Oberski / doberski@uvt.nl
- 27. Multidirectional errors Latent class MTMM Conclusions Thank you for your attention! doberski@uvt.nl @DanielOberski See http://daob.nl/publications for preprints Supported by Veni grant number 451-14-017 from the Netherlands Organization for Scientific Research (NWO). Latent class MTMM Daniel Oberski / doberski@uvt.nl