ABLE - EMFD presentation - NTU student dashboard stream
1 like123 views
The document discusses the implementation and impact of the NTU student dashboard, which utilizes learning analytics to enhance student engagement and success. It highlights the dashboard's ability to provide insights into student behaviors and outcomes, noting that increased engagement is a stronger predictor of success than traditional entry qualifications. Challenges include cultural resistance and the need for confidence in data utility; however, positive student reactions indicate significant benefits for academic engagement and support.
ABLE - EMFD presentation - NTU student dashboard stream
- 1. Student Dashboard - lessons learned Ed Foster, Richard Gascoigne, Dr Rebecca Edwards, Melanie Currie
- 2. Today’s Structure • Welcome • Why NTU is interested in learning analytics - Ed • Solutionpath’s StREAM resource - Richard • Research from the Student Dashboard – Becky • NBS’s experience of embedding the Dashboard into working practice – Melanie • Discussion - All
- 3. Where are you? • Who is using learning analytics? • Who has run a pilot? • Who is thinking of using learning analytics? • Who thinks the whole thing is an appalling idea, an invasion of privacy and frankly an assault on freedom, personal responsibility and fundamentally undermines the purpose of higher education?
- 4. Ed Foster
- 6. NTU Student Dashboard Improving institutional data & systems • Cohort insight • Improve University systems • Informing future plans/ strategy Promoting student success • Progression • GPA/ degree attainment • Targeted information for students & staff Improving staff-student working relationships • Engagement information • Info for personalised tutorial discussions Supporting students to manage their own learning • Info to promote reflection • Benchmarking • Developing goal setting • Measures engagement, not risk of failure • Measures engagement using students’ electronic footprint • Do not measure socio-economic disadvantage
- 7. Core Dashboard Process Student data Engagement with learning Engagement score Risk alerts Referrals Notes, Sense checking Passive useful information Active engagement data ! Attendance E-resources Card swipes VLE log ins Dropbox submissions Library loans
- 8. Dashboard TutorStudent Metrics Raw data & engagement rating Student engagement with course Metrics & alerts Engagement with students presented to students students act presented to tutors more-informed interactions Fitting the Dashboard into the institutional ecosystem Two agents of change
- 9. Impact of Engagement Progression from 1st to 2nd year (2015-16) 80% Progress Withdraw Repeat Academic failure Transfer Other All Students 9% Low av. engagement 81% Partial av. engagement 92% Good av. engagement 95% High av. engagement
- 10. Implementation Challenges Technological Cultural Resource based Time based Data not neutral Data coded in a context Then there’s the plumbing Resistance to change Philosophical concerns Skepticism Whose problem is it? Knowing the problem not the same as solving the problem Time pressures on academics Seasonal nature of HE Urgency of responding to problems Time needed to respond to a crisis
- 11. This word cloud was drawn from repertory grid exercise with student interviewees. Students were asked to choose words categorised as positive or negative, and active or passive. As can be seen, most words selected were positive & active Student Reactions to Dashboard
- 12. Student Reactions to Dashboard • After logging in to the Dashboard: • 74% reported increasing the amount of time spent studying • 36% said they had sought out a tutor, 31% specialist support • 64% found the Dashboard to be ‘useful’ or ’very useful’ • Students who found it useful were: • More likely to be enjoying being a student (88% v 81%) • More engaged with their studies (73% v 66%) • More confident about coping with their studies (61% v 54%) • Less likely to have encountered an academic problem (64% v 69%) • Equally likely to have considered leaving (both 27%) • Students wanted to be told that they were at risk of dropping out (94%) or if we could improve their chances of progression (97%) N=753 1st year students, Feb/Mar 2017, (percentages in brackets indicate answered 4 or 5 out of 5 (positive or v. positive))
- 13. Student perceptions of Dashboard in tutorials • Students reported meeting tutors in a variety of ways: • 84% In a small group tutorial • 52% In a one to one tutorial • 50% Teaching in a lecture theatre / classroom • 29% said their tutors had used the Dashboard during one-to-one meetings with them: 80% found this useful • Moreover, where a tutor had used Dashboard • Students found the Dashboard to be more useful (75% v 56%) • Students felt more confident about coping with their studies (60% v 54%) Student Transition Survey Feb-Mar 2017, n=753
- 14. 0% 20% 40% 60% 80% 100% 0 (n=612) 1 (n=518) 2-3 (n=1,090) 4-6 (n=1,407) 7-9 (n=1,090) 10-19 (n=1,766) 20+ (n=1,393) Proportion of log-in category No.ofDashboardlog-ins Dashboard log-ins by engagement rating for 2015-16 (first year, FT, UGs) Low Partial Good High 46% 72% 78% 81% 84% 86% 90% 0% 20% 40% 60% 80% 100% 0 1 2-3 4-6 7-9 10-19 20+ Percentage of students progressing NoofDashboardLogins Proportion of students progressing to second year by number of Dashboard log-ins (1st year, FT UGs in 2015-16) Relationship between Dashboard use & success
- 15. • Taking an overview of student tutees • ”I use it to check the ‘academic health’ of students”* • Preparing, facilitating & reviewing tutorials* • 40% reported that using Dashboard led to changes in student engagement** • “Yes, I noticed a drop in the engagement level of one student. I arranged a 1- 1 meeting to discuss the various problems the student was experiencing. The student found this support very helpful and their engagement level picked up as a result.” • “Making students aware that they engage less than others has improved engagement for some, but not for the worst cases.”* Tutors using the Dashboard * = 2nd iPad survey, n=138, ** = 3rd iPad survey, n=107
- 16. Summary • Dashboard has become deeply embedded into institutional practices • Engagement is a stronger predictor of success than entry qualifications or demographic factors • Challenges are only primarily associated with cultural change
- 19. Using engagement data and beyond • Engagement data is displayed directly to staff and students for personal use • Engagement data can be combined with other University data to provide insight into student groups • Engagement data can be used to measure impact of University initiatives/interventions
- 20. Providing data to staff and students • Need confidence that the algorithm works so we can promote using it as a tool • The data needs to be useful on a time-scale that allows for change
- 21. Progression based on engagement
- 22. Progression based on engagement at different points in the year
- 24. Why use the Dashboard to measure impact of interventions/initiatives? • Daily engagement ratings allow the analysis of short term impact of a specific intervention. • Increasing a student’s engagement with the course is a success factor in its own right. • The data is automatically generated, it does not require additional work to gather
- 25. How can the Dashboard be used to measure impact? • Daily engagement data allows two types of analysis: 1. Change in engagement over timeframe 2. Rate of change in engagement over timeframe Date of intervention 7 days before14 days before 7 days after 14 days after Time • Important context for analysing impact of intervention: 1. Behaviour before intervention 2. Behaviour of others on same course
- 26. Case study: Library Learning and Teaching Team • The team offers guidance covering all aspects of academic skills. • Students can book 30 minute one-to-one sessions with team members between 9 am and 5 pm using an online booking system • In 2015-16, the eight team members inputted notes into the NTU Student Dashboard during/shortly after one-to-one sessions with 815 students
- 27. Impact of Library Learning and Teaching Team on student engagement • After the meeting the proportion of students with engagement higher than the course average increased
- 28. Impact of Library Learning and Teaching Team on student engagement • After the note was inputted the proportion of students whose rate of increase in engagement was higher than course average increased
- 29. • Engagement increases in the time leading up to the appointment with the library team for both the attending students and the course average Timing of the appointments with the Library Learning and Teaching Team
- 30. Progression differences for users of the 1:1 library service • Progression rates for students who visited the library team was 8.6 % higher than those who didn’t
- 31. Attainment differences for users of the 1:1 library service • 65.2 % of students who used the service had a GPA equivalent of a 2:1 or first compared to 54.1 % for students who didn’t • So 11.1 % more students who got a 2:1 or first for those who had an appointment (20.5 % higher)
- 32. • Students who visited the library team were more engaged with the University than their course average before attending their one-to-one appointment (as well as being more engaged afterwards) Engagement of students before meeting the Library Learning and Teaching Team One week before appointmentTwo weeks before appointment
- 33. Case study conclusions: • Students had higher engagement after visiting the Library Learning and Teaching Team • Students who used the 1:1 library service had better outcomes in terms of progression and attainment than those who didn’t • The 1:1 library service was primarily used by students who were more engaged than their peers
- 35. #EFMDBachelors17 A sense of the possible
- 37. Can Learning Analytics help us improve Student Engagement and deepen Personalisation? A Case Study
- 38. Personalised learning approach Our aim is… • To help you to achieve your full potential through understanding your experiences, exploring your aspirations and developing a personal development plan. • To provide you with a mentor who will help you to create a development plan to be the best you can, and who you can go to if you have any concerns. @NBSallaboutYOU
- 40. NBS Personalised Student Experience Personalised Knowledge Portfolio Personalised Experiential Portfolio Personalised Career Support Personal Learning Styles Alumni Fellowship Programme Student Dashboard Optional Choices Study Mode Choices Diagnostic Tools Adaptive Learning Tools Tailored Placements & Internships Simulation & Project Based Learning Community Projects Consultancy Project Study Tours & Missions Reflective Practice Specialist Support Tailored Inspirational Programmes Enterprise Development Programme Student Competitions Academic Mentors International Exchange Professional Memberships Virtual Learning Environment
- 41. Undergraduates: All NBS students undertook a minimum of 20 hours CPLD All students completed an assessed reflective portfolio All students had a 1-2-1 with their academic mentor each term All students access to relevant diagnostic tools 700+ Events by NBS & Library, Employers, Alumni Fellows and Professional bodies 100,000 hours of CPLD/CPD. @NBSAllaboutYOU
- 42. Providing innovative opportunities to apply your learning Lots of extracurricular opportunities to engage with to support your personal professional development, enabling a solid platform for your future
- 43. Where next?
- 44. Learning Analytics - References Gunn, C. (2014) 'Defining an agenda for learning analytics'. In Hegarty, B., McDonald, J. & Loke , S.K. (eds) Rhetoric and Reality: Critical perspectives on Educational Technology. Proceedings Eascilite, Dunedin, 2014: 683-687. Healey, M., Flint, A and Harrington, K (2014) Engagement through partnership: students as partners in learning and teaching in higher education. York: Higher Education Academy McPherson, N., Heggie, G., Faina, K., Kean., M. & McCarroll, J. (2015) Partners in Learning/Partners in Research: Developing a Culture of Research Mindedness in Social Science Students. Case study. Online: HEA. Neary, M and Winn, J (2009) The student as producer: reinventing the student experience in higher education., in Bell, L., Stevenson, H. & Neary, N. (2009) The future of higher education: policy, pedagogy and the student experience. pp 192–210 London: Continuum. Sclater, N (2014) Learning analytics The current state of play in UK higher and further education. Online. Jisc.
- 45. Engagement as a Bridge (Paul and Vicki Trowler, 2011) NBS Personalisation - @NBSAllaboutYOU Bridge as a metaphor – environment, climate terrain, similar journeys…. different destinations