Methodological Issues in Learning Analytics: Critical Insights and Reflections
1 like741 views
1. Professor Mark Brown gave a presentation on methodological issues in learning analytics based on his experience in distance education and digital learning. 2. He discussed how learning analytics can fall prey to oversimplifying education, basing decisions on weak theories or invalid variables, and confusing correlations with causality, leading to unintended consequences. 3. Brown also highlighted the risks of novelty bias in learning analytics research and the need to consider the broader literature from related fields.
Methodological Issues in Learning Analytics: Critical Insights and Reflections
- 1. Methodological Issues in Learning Analytics: Critical Insights and Reflections Professor Mark Brown ICDE Leadership Summit Nancy, France 22th May 2017
- 2. @mbrownz Chair, Digital Learning Director, National Institute for Digital Learning
- 3. • Long History of Distance Education • Hosted National Centre for Distance Education • OSCAIL Early Adopter of Online Learning
- 4. Excellence in Online Education
- 9. In short… A pilot learning analytics project using LMS data on learner activity to provide direct feedback to students and lecturers during Semester.
- 10. No significant difference in the entry profiles of participants vs. non-participants overall Participant Profile
- 11. Subject Description Non-Participant Participant BE101 Introduction to Cell Biology and Biochemistry 58.89 62.05 CA103 Computer Systems 70.28 71.34 CA168 Digital World 63.81 65.26 ES125 Social&Personal Dev with Communication Skills 67.00 66.46 HR101 Psychology in Organisations 59.43 63.32 LG101 Introduction to Law 53.33 54.85 LG116 Introduction to Politics 45.68 44.85 LG127 Business Law 60.57 61.82 MS136 Mathematics for Economics and Business 60.78 69.35 SS103 Physiology for Health Sciences 55.27 57.03 Overall Dff in all modules 58.36 61.22 Average scores for participants are higher in 8 of the 10 modules analysed. Academic Performance
- 12. Survey findings... Question Group 1 (more detailed email) Group 2 % of respondents who opted out of PredictED during the course of the semester 4.5% 4.5% % who changed their LMS usage as a result of the weekly emails 43.3% 28.9% % who would take part again/are offered and are taking part again 72.2% (45.6%/ 26.6% ) 76.6% (46% /30.6% )
- 14. Three comments…
- 15. “Not everything that can be counted counts, not everything that counts can be counted.” Albert Einstein
- 16. 1. The Hype Gap
- 19. (i) view education as a simple process that is easily modelled; (ii) base decisions and interventions on data rich but weak theory; (iii) inform decisions and interventions based on wrong or even invalid variables; (iv) make interpretations and arrive at conclusions that confuse correlations with causality; and (v) result in unintended and unwanted effects that pigeonhole and stereotype learners which may be counterproductive to learner success.
- 22. 2. Novelty and Impact Bias
- 23. Allen, K., Galvis, D., Katz R. (2004). Evaluation of CDs and chewing gum in teaching dental anatomy. Journal of Dental Research, 83. VS. Learning is like chewing gum (messy)
- 24. Survey findings... Question Group 1 (more detailed email) Group 2 % of respondents who opted out of PredictED during the course of the semester 4.5% 4.5% % who changed their LMS usage as a result of the weekly emails 43.3% 28.9% % who would take part again/are offered and are taking part again 72.2% (45.6%/ 26.6% ) 76.6% (46% /30.6% )
- 26. 3. Border Crossing the Literature
- 28. Performance takes place in the Network
- 31. Conclusion
- 32. Conclusion
- 35. “A prudent question is one-half of wisdom” Francis Bacon http://www.slideshare.net/mbrownz Questions…
- 36. Professor Mark Brown Director, National Institute for Digital Learning www.dcu.ie/nidlmark.brown@dcu.ie @mbrownz www.slideshare.net/mbrownz