![Findings
Do students respond positively to the introduction of the
app?
The app “reinforced the content” & “kept [students]
involved and invested” by appealing to their competitive
nature
Leaderboards & badges: allowed students to see how they
placed compared to their peers. However: it is “dangerous
to start ranking people” & some students felt “exposed”
Immediate feedback was desired in addition to right/wrong
answer indicator](https://image.slidesharecdn.com/oatespechenkinaetaltlc2016-160919021519/85/Push-notifications-digital-badges-leaderboards-Evaluating-the-impact-of-Quitch-mobile-application-on-learning-14-320.jpg)
Push notifications, digital badges & leaderboards: Evaluating the impact of Quitch mobile application on learning
- 1. CRICOS 00111D TOID 3059 Push notifications, digital badges & leaderboards: Evaluating the impact of Quitch mobile application on learning Dr Grainne Oates FLB, Senior Lecturer, Accounting/Finance Dr Katya Pechenkina LTU, Research Fellow Dan Laurence LTU (formerly), Learning Designer Dr Daniel Eldridge FSET, Lecturer in Chemistry Prof Dan Hunter FBL, Foundation Dean, Swinburne Law Presented at Transforming Learning Conference Swinburne University of Technology 13-14 September 2016 Melbourne Australia http://transformconference.com/ #SwinTLC
- 2. Outline • Mobile apps for learning • Quitch: Rationale & elements • Quitch: Development process • Measuring impact • Findings • Further steps
- 3. Mobile apps for learning • Challenges: low retention & high failure rates • Students likely to access content via mobile phone (or a similar device) not desktop computer • Push notifications • Student expect immediate constructive feedback • Self-testing has capacity to improve outcomes • Multitude of mobile apps - rarely tailored to lecture content
- 4. Quitch: Rationale & elements • Engagement loops • Regular ‘push-notifications’ • Dynamic timed quiz • Points & badges • Leaderboards • Progression trees • Videos & forums A solution should: • Be easy for educators to use • Have real time analytics for early intervention & adaptive learning
- 8. Quitch: Analytics Possibilities: • Reengage ‘at risk’ students • Test if lecture content is comprehended • Adapt course material dynamically • Provide regular personalised feedback
- 9. Measuring impact • Is there a relationship between introducing a mobile app into the classroom and student retention rates? • Is there a relationship between the use of the app and academic performance? • Is student performance within the app related to academic performance? • Do students respond positively to the introduction of the app?
- 10. Measuring impact: Methods • Two cohorts taking part in the Quitch trial (S2, 2015) Accounting (462 enrolled, 265 app users) Science (249 enrolled, 129 app users) • Total sample: N=711, with 55% using the app (394) + prior cohorts for comparison • Data collected (with student consent): Final grades compared to the pre-app cohort App uptake & engagement analytics Small-scale qualitative online survey (N=8) Student feedback from end-of-year surveys
- 11. Findings Is there a relationship between introducing a mobile app into the classroom and student retention rates? • The app introduction was associated with a positive improvement in student retention (calculated at 12.23%) compared to the pre- app semester
- 12. Findings Is there a relationship between the use of the app and academic performance? • App users averaged a percentage mark of 65.19% compared to students who did not use the app (who averaged a percentage mark of 58.16%). • App users on average achieved marks that were 7.03% higher than students who did not use the app
- 13. Findings Is student performance within the app related to academic performance? • For app user there was a positive correlation of .40 between performing well on the app and achieving higher academic grades
- 14. Findings Do students respond positively to the introduction of the app? The app “reinforced the content” & “kept [students] involved and invested” by appealing to their competitive nature Leaderboards & badges: allowed students to see how they placed compared to their peers. However: it is “dangerous to start ranking people” & some students felt “exposed” Immediate feedback was desired in addition to right/wrong answer indicator
- 15. Further steps • Correlation ≠ causation • In-depth studies of student & educator experiences needed • Implications for universities, educators & students • Tailored content, timed content-aligned quizzes, immediate feedback, rankings, flexibility
- 16. Thank you Presented by Dr Katya Pechenkina Research Fellow The Office of the Senior DVC & Provost Learning Transformations Email: epechenkina@swin.edu.au Twitter: @katya_pechenk