Hidden learning spaces - What learning analytics cannot tell us
- 1. bournemouth.ac.uk Hidden learning spaces: What learning analytics cannot tell us Andrew Kitchenham and David Biggins 4 September 2019
- 2. bournemouth.ac.uk 2 Introduction Andrew Kitchenham David Biggins
- 3. bournemouth.ac.uk 3 Scene setting • Sector-wide interest in learning from data • BU invested in a new VLE in 2017 • Exploratory learning analytics • Piloting a project to use data to: • Understand student interaction and staff VLE use • Improve student experience, retention and outcomes • Develop the use of the VLE • Inform decision-making • Informed by student input (questionnaire n=26)
- 4. bournemouth.ac.uk 4 Questions 1. Is your institution collecting learning data? 2. What is the primary source of that data?
- 5. bournemouth.ac.uk 5 Terminology Core learning materials eg lecture slides, assignment briefs Supplementary learning materials eg External learning tools, ebooks Independent learning materials eg YouTube, Google search
- 6. bournemouth.ac.uk 6 Two student journeys Student A Interaction Outcome Student B VLE
- 7. bournemouth.ac.uk 7 Two student journeys Student A Interaction Outcome Student B VLE Visible ? Hidden
- 8. bournemouth.ac.uk 8 Hidden learning environment • Use of the VLE (frequently or very frequently) • Core materials 80% • Assignment briefs 90% • Supplementary material 27% VLE Visible HLE Hidden
- 9. bournemouth.ac.uk 9 Hidden learning environment VLE Visible HLE Hidden • Use of the HLE (frequently or very frequently) • Core materials from peers 34% • Assignment briefs 15% • Supplementary material 15% • Independent learning materials 80%
- 10. bournemouth.ac.uk 10 Hidden learning environment VLE Visible HLE Hidden • What is in the HLE? Social media Internet search iTunesU YouTube • Why the HLE? • Has always existed • Connectivist learning and generational characteristics • Oversight aversion • Lack of confidence in using the VLE Internet: Expert: 66% VLE: Expert: 13%
- 11. bournemouth.ac.uk 11 Hidden learning environment ‘we must understand the underlying structure of the phenomenon we seek to explore with analytics prior to digging into the data’ (Knight 2014)
- 12. bournemouth.ac.uk 12 Hidden learning environment VLE Visible • Taking the HLE into account • A twin-track approach • Accept the limitations on what VLE data tells us • Only part of student learning comes from the VLE • Avoid reliance/decision-making on visible data • Move beyond VLE data ..
- 13. bournemouth.ac.uk 13 Beyond VLE engagement • Alternate indicators • Attendance • Formative assessment • Demographics • Awareness can lead to changes in behaviour • Changes in levels, not absolutes • Issues • Institutional acceptance • Policy and guidance • GDPR • Copyright • Staff appreciation • Learning design
- 14. bournemouth.ac.uk 14 Back to the HLE: Next steps • Move from exploration to explanation • What HLEs are being used and why? • Why students don’t engage in all VLE components? • What is the optimal recipe? • Exploit the available data to • Inform curriculum design • Improve student outcomes • Develop staff confidence and capability • Mature appreciation • Nuances of data interpretation
- 15. bournemouth.ac.uk 15 Questions • akitchenham / dbiggins @ bournemouth.ac.uk
- 16. bournemouth.ac.uk 16 References • Educause Horizon Report 2019. https://library.educause.edu/- /media/files/library/2019/4/2019horizonreport.pdf?la=en&hash=C8E8D444AF372E705FA1BF9D4 FF0DD4CC6F0FDD1 • Educause Review. 2017. Learning Analytics: Avoiding failure. https://er.educause.edu/articles/2017/7/learning-analytics-avoiding-failure • Jayaprakash, S.M., Moody, E.W., Lauría, E.J., Regan, J.R. and Baron, J.D., 2014. Early alert of academically at-risk students: An open source analytics initiative. Journal of Learning Analytics, 1(1), pp.6-47. • Jisc briefing 2019. Learning analytics and student success – assessing the evidence. http://repository.jisc.ac.uk/6560/1/learning-analytics_and_student_success.pdf • Knight, S., A. F. Wise, G. Arastoopour, D. Williamson Shaffer, S. Buckingham Shum, and P. A. Kirschner. 2014. “Analytics for Learning and Becoming in Practice.” International Conference of the Learning Sciences (ICLS 2014), 23–27 June 2014, Boulder, Colorado, 1680–1683. • Shoufan, A., 2019. Estimating the cognitive value of YouTube's educational videos: A learning analytics approach. Computers in Human Behavior, 92, pp.450-458. • Society for Learning Analytics research (SOLAR). https://solaresearch.org/ • Wilson, A., Watson, C., Thompson, T.L., Drew, V. and Doyle, S., 2017. Learning analytics: challenges and limitations. Teaching in Higher Education, 22(8), pp.991-1007.