Learning Analytics for Student Support
1 like858 views
The document outlines the 'Analytics for Action' (A4A) project by The Open University, which aims to utilize data to drive evidence-based changes in teaching and support practices across its modules. It details the framework, toolkit, training, and various support mechanisms provided to staff, along with specific insights into student engagement and performance metrics. The project has demonstrated success in improving student outcomes and aims for further growth in support and analytics initiatives in the upcoming academic year.
Learning Analytics for Student Support
- 1. Innovate • Design • Enhance Analytics are everywhere
- 2. Innovate • Design • Enhance “Analytics for Action – enabling in-presentation evidence-based change on Open University modules” 15 September 2017 • Rafael Hidalgo - Senior TEL Designer LTI – The Open University
- 3. Innovate • Design • Enhance The Open University context • The UK's largest university (approximately 125,000 students) – 40% of part-time undergraduates in the UK study with us • 180 qualifications and 600 modules – most of our undergraduate courses have no formal entry requirements • 76% of students study alongside (full or part-time) work • 20,000+ people with one or more declared disabilities study with us each year – probably more than any other university in Europe
- 4. Innovate • Design • Enhance What is Analytics for Action (A4A)? • A4A is a project to enable evidence-based change in presentation – Active Presentation • It started as part of the Learning and Teaching Centre’s Analytics Project (2014-2016), encompassing: • Real-time Student Feedback, Student progression reporting, SAS Visual Analytics, Ethics, Predictive modelling (OU Analyse) and Analytics 4 Action • TEL Design provided a natural long-term home for Analytics for Action . • A4A is run by TEL LTI since 2016/2017- Data is business as usual
- 5. Innovate • Design • Enhance “Using the data as prompts, module teams can initiate a range of response actions, including pedagogic change and student support” - Bart Rienties, IET, The Open University
- 6. Innovate • Design • Enhance Analytics for Action Framework
- 7. Innovate • Design • Enhance The Active Presentation Toolkit • Based on the A4A framework • Available to all staff in the Open University intranet • Step-by-step instructions for running an active presentation
- 8. Innovate • Design • Enhance The Active Presentation Toolkit • Staged approach
- 9. Innovate • Design • Enhance What support is available? For all modules • Toolkit provides for a DIY approach • Training enables staff to get started with this, • Surgery support at Learning Systems drop-ins • Mailbox support: LTI-TEL-Design@open.ac.uk For selected modules: • Data support meetings: 3 per presentation
- 10. Innovate • Design • Enhance Which modules receive full support? • Agreed with Faculties • Modules identified as underperforming by the AQR – Z scores • High student population • Pedagogic challenges and/or innovations • Qualifications needs • Faculty strategic interest
- 11. Innovate • Design • Enhance Training • 2 hours hands-on training sessions run by Senior TEL Designers, using real and current data. • Priority places for MTC’s and CM’s of modules selected for formal support • Main focus: Active Presentation Toolkit and Student Progression Dashboards • Small groups to allow enough discussion and specific questions
- 12. Innovate • Design • Enhance Data support meetings • Senior TEL Designers will support the relevant faculty modules • Academics provide in-depth knowledge of the module, • STELDS provide wider view as they are exposed to other data, • Together we agree on the issues to be investigated, • STELDs suggest actions and help in the implementation (this may involve other units) • Comprehensive reports on discussions and agreed actions, • Regular updates to the faculty stakeholders about progress
- 13. Innovate • Design • Enhance What sort of data do we look at support meetings? Student progression dashboards • Student profile • Concurrency • Assessment submission rates • Withdrawer profiles • Online engagement • Online tools and resource usage • Retention • Pass and retention rates
- 14. Innovate • Design • Enhance Student profiles and concurrency • 267 students registered at FLP25 • Roughly half registered for Q72 and a quarter for Open Degree • 66% male, most common age group is 30-39 (37%) • 16.5% have declared a disability • 73.8% were studying at least another module, mostly T313, with 38 students studying 120 credits • 79% are employed part or full time T219 • 187 students registered at FLP25 • Q72 is the most popular qualification (30%) • 64% male, most common age group 30-39 (37%) • 11% have declared a disability • 55% were studying at least one other module with 20 students studying 120 credits • 87% are employed part or full time T319
- 15. Innovate • Design • Enhance Assessment Submission Rates 93,1 87,0 96,5 83,6 94,9 62,1 55,2 63,9 65,1 59,6 0,0 10,0 20,0 30,0 40,0 50,0 60,0 70,0 80,0 90,0 100,0 S104 16J SDK100 U116 S111 S10415J % of TMA Submissions - Level 1 Science TMA01 TMA02 TMA03 TMA04 TMA05 TMA06
- 16. Innovate • Design • Enhance Assessment Submission Rates 87,3 70,9 59,3 57,1 87,6 76 60,3 0 10 20 30 40 50 60 70 80 90 100 TMA01 TMA02 TMA03 Project % of TMA Submissions - T219 – by year 13J 14J 15J 16J
- 17. Innovate • Design • Enhance Retention 80 82 84 86 88 90 92 94 17 18 19 20 21 22 23 24 25 % Retention T219 T213 DST206 AVE
- 18. Innovate • Design • Enhance Online engagement VLE use for T219 in 16J
- 19. Innovate • Design • Enhance Use of online resources Use of Block 1 Forum in T319
- 20. Innovate • Design • Enhance Completion and Pass rates
- 21. Innovate • Design • Enhance Data insight: Conversations and actions • “Significant numbers of students are studying other modules concurrently. The module team don’t advise on multiple module study.” Student Support Teams need to be made aware of significant number of students taking 120 points at once. A faculty discussion is also needed at qualification level. • Are extensions any good? • Is the gap between TMA02 and Project too long? TMA02 weight and timing needs to be assessed • More support from the tutor at the start of collaborative work Module Team to consider an intro collaborative activity • Tutors should be trained in handling the forums, are they?
- 22. Innovate • Design • Enhance Data insight: Conversations and actions Are collaborative activities really an issue? Pass: Module v- Faculty Average Difference 12J av 13J av 14J av 15J av 16J av 12J 13J 14J 15J 16J T219 61% 64% 60% 63% 51% 62% -3 -3 -11 T319 67% 69% 69% 68% -2 1 T215 74% 64% 72% 63% 73% 62% 10 11 11 TD223 58% 65% 58% 67% 61% 68% -7 -9 -7 U101 71% 65% 63% 65% 63% 64% 57% 63% 6 -2 -1 -6 B205 n/a H817 (B)* 86% 79% 88% 79% 90% 79% 84% 76% 13 11 11 12 W302 n/a DD206 80% 67% 78% 65% 71% 67% 73% 68% 13 13 4 5 DD313 78% 75% 76% 74% 3 2 E207 80% 68% 82% 68% 78% 68% 75% 69% 12 14 100 6
- 23. Innovate • Design • Enhance Data Support Meetings Reports
- 24. Innovate • Design • Enhance What can we do about it? • Based on our findings we could decide: • A change to the materials • Some additional support to everyone • Some targeted support • Improvements upstream to get the right people onto the module
- 25. Innovate • Design • Enhance What can we do about it? http://article.iet.open.ac.uk/Q/QE%20report%20series/2015-QE- learning-analytics.pdf
- 26. Innovate • Design • Enhance Success examples KJY113 16B: Improved pass rates • Registered students dropped 12-13 % from start to 25FLP. Pass rates under faculty average for 3 years • Additional information via dedicated website and A5 postcard • AL’s contacted students at risk • Pass rate up 12%. Completion rate up 10%. TMA submissions improved. Drop between start and FLP25 was only 3%
- 27. Innovate • Design • Enhance Success examples S350 16J: Managing extensions • Concurrency levels very high: 73.4% • 75% of students were employed • Students unexpectedly taking Sxx390 • AL’s were given list of high workload students and their TMA dates for concurrent modules, to manage extensions. • No historical data to compare but early intervention by AL’s was enabled by data support meeting
- 28. Innovate • Design • Enhance Success examples T192 16J: design to evaluation • In the design, Module Team addressed a key mathematical concept – “rearranging equations”- • TEL Design helped to set up an adaptive learning package of iCMAs with support built-in via the website and module tutors. • Monitoring the data allowed to provide the module team chair with the information required to decide whether to take action on these individual quizzes. -10 0 10 20 30 40 50 60 70 80 90 100 0 2 4 6 8 10 12 14 16 18 20 Number of students with scores under 40% and 50% Scores < 40 Scores < 50
- 29. Innovate • Design • Enhance Analytics for action: Data is business as usual In 2016/17 • 29 OU modules fully supported – reaching almost 25,000 students • Over 100 staff trained in 23 training sessions • 75 data support meetings held • Coverage expected to grow in 2017/18
- 30. Innovate • Design • Enhance Analytics for action: The future For 2017/18: growing numbers and quality • Up to 49 OU modules fully supported • Improved tools and systems • More than 120 staff to be trained and over 100 data support meetings to be held • Increased scholarship
- 31. Innovate • Design • Enhance Analytics for action: What are they saying? About Data Training Tool sessions: “The initial question I had was answered and I found out other useful things also” “The trainers were able to support individual learners with their unique issues” “It was informative and with help easy to navigate” “Hands on appropriate support and guidance using real live data” About Data Support Meetings: “Being able to go “off grid” and actually – finally! – get some insights and answers into presentation questions we have had for ages” “Thank you that gives us something to think about for the future presentations” “Really useful, and I’ll follow things up here” “I was updating VB on the great work that has been happening with T219/T319”
- 32. Innovate • Design • Enhance The Open University LTI TEL Design team Level 7 Perry B lti-tel-design@open.ac.uk @TEL_Design_OU Get in touch!