Investigating Self-Reporting Behavior in Long-Term Studies
1 like432 views
This study investigates self-reporting behavior in long-term studies, focusing on the accuracy and reliability of data collected through mobile devices. It found that self-reporting compliance decreases over time and participants tended to overestimate their app usage, influenced by reminders. The research provides empirical data and recommendations for maximizing reliability in self-reported information.
Investigating Self-Reporting Behavior in Long-Term Studies
- 1. INVESTIGATING SELF-REPORTING BEHAVIOR IN LONG-TERM STUDIES Andreas Möller ✽, Matthias Kranz ❖, Barbara Schmid ✽, Stefan Diewald ✽, Luis Roalter ✽ ✽ Technische Universität München, Germany ❖ Universität Passau, Germany
- 4. RESEARCH QUESTIONS Accuracy? Change over time? Influence of reporting frequency? Reliability maximization? A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 5. BACKGROUND ■ Electronic diaries show higher compliance (Hufford & Shields, 2002) ■ Mobile phone as survey tool (Consolvo et al., 2007) Consolvo et al., 2007 A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 6. CONTRIBUTIONS ■ Empirical quantitative long-term data on reliability of information collected with mobile devices ■ Recommendations for maximizing result reliability A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 7. A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris METHODOLOGY Evaluate reporting behavior Ground truth Can be gained in automated way Limited effort Smartphone usage GOAL REQUIRE- MENTS SOLUTION Frequently used apps Installed by everyone
- 8. Mail Facebook Frequently used apps Installed by everyone A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 9. A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris PROCEEDING Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 6 + 4 Pre-Questionnaire Reminder Emails Post-Questionnaire Post-Post- Questionnaire Requested Self-Reports & Logging
- 10. TASK ■ Answer questionnaire after Facebook or Mail usage ■ Report as accurate as possible 1. How long did you use the app? 2. How many times did you use the app without answering a questionnaire? 1 direct report n indirect reports A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 11. SELF-REPORTING AND EXPERIENCE SAMPLING ASSISTANT A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 12. SERENA ■ App usage logging ■ Server upload ■ Questionnaire triggers □ Event-based □ Time-based □ Manually A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 13. A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris 3 CONDITIONS Voluntary Interval Event No trigger Daily trigger Trigger after app usage 30 Participants 3,631 Mail usages 3,181 Facebook usages
- 14. SESSIONS Voluntary Interval Event Amount of reported Facebook usages 37.6% 63.8% 54.3% Indirect reports Direct reports A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 15. DURATIONS Voluntary Interval Event 1:29 1:29 1:22 2:52 3:02 3:35 Facebook sessions Real Self-Reported A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 16. OVER TIME ■ Self-report ratio decreases ■ Actual usage decreases „Answering the questionnaire changed my Facebook usage habits.“ Voluntary 2.3 Interval 2.2 Event 3.5 5 = strongly agree A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris
- 17. A. Möller, M. Kranz et al., Investigating Self-Reporting Behavior in Long-Term Studies, CHI 2013, Paris DISCUSSION Subjects reported max. 70% of usage Commitment decreased Reports may influence actual behavior Subjects overestimated session length Reminder emails pushed commitment in 2nd phase Behavior change with increasing burden Little control: forgetting High control: burden Trigger influence lower than hypothesized App usage decreased Most subjects would report max. 4 weeks
- 19. Please cite this work as follows: A. Möller, M. Kranz, B. Schmid, L. Roalter, S. Diewald Investigating Self-Reporting Behavior In Long-Term Studies In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), pp. 2931-2940, Paris, France, April-May 2013. If you use BibTex, please use the following entry: @inproceedings{chi2013selfreport, author = {Andreas M"{o}ller and Matthias Kranz and Barbara Schmid and Luis Roalter and Stefan Diewald}, title = {Investigating Self-Reporting Behavior In Long-Term Studies}, booktitle = {Proceedings of the 2013 ACM annual conference on Human Factors in Computing Systems}, pages = {2931--2940}, series = {CHI '13}, year = {2013}, isbn = {978-1-4503-1899-0}, location = {Paris, France}, numpages = {10}, publisher = {ACM}, address = {New York, NY, USA}, }