ALT-C 2019 Jisc curriculum analytics - full set of slides
- 1. Curriculum analytics: A deep dive into student data to discover curriculum insights Presenters: Paul Bailey, Jisc and Niall Sclater, Sclater Digital, Consultant Michael Webb (Jisc), Alan Paull (Cetis) and Scott Wilson (Cetis) September 2019
- 2. Emerging types of analytics in education Educational analytics Learning Curriculum Welfare Intelligent campus Employability and apprenticeship Institutional Curriculum analytics2
- 3. What makes curriculum analytics different?
- 4. Potential applications of curriculum analytics Real-time adjustments to teaching by the lecturer during the lecture Identifying modules which appear to result in better learning and/or greater student satisfaction Understanding which aspects of the curriculum result in better learning and build this knowledge into future curriculum development Understanding how sequencing of exams affects student performance Curriculum analytics4
- 5. Curriculum analytics is the use of data to help understand and enhance the curriculum.
- 6. Potential users of curriculum analytics Lecturers to see which aspects of their modules are proving more effective than others and others responsible for overseeing and reporting on module and course performance and others who can identify and promote good practice in module development who wish to assess the relative success of different schools or faculties and develop policy to ensure that good practice is embedded Associate deans Learning technologists Senior management Curriculum analytics6
- 7. Challenges Lack of instrumentation Difficulties in describing curriculum Staff resistance Curriculum analytics7
- 8. We should only provide learning content and activities where we have ways of measuring their impact on student learning.
- 9. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 9
- 10. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 Attributes Core/supplementary Compulsory/optional Digital/physical Scheduled/in own time Formative/summative Group/individual % of final mark Assessment method Prerequisite Adaptive release 10
- 11. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 Attributes Core/supplementary Compulsory/optional Digital/physical Scheduled/in own time Formative/summative Group/individual % of final mark Assessment method Prerequisite Adaptive release Instrument Attendance monitoring system 11
- 12. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 Attributes Core/supplementary Compulsory/optional Digital/physical Scheduled/in own time Formative/summative Group/individual % of final mark Assessment method Prerequisite Adaptive release Instrument Measured data Attendance monitoring system Student ID Date/Time 12
- 13. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 Attributes Core/supplementary Compulsory/optional Digital/physical Scheduled/in own time Formative/summative Group/individual % of final mark Assessment method Prerequisite Adaptive release Instrument Measured data Calculated data Attendance monitoring system Student ID Date/Time Number of students attending Percentage of enrolled students attending 13
- 14. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 Attributes Core/supplementary Compulsory/optional Digital/physical Scheduled/in own time Formative/summative Group/individual % of final mark Assessment method Prerequisite Adaptive release Instrument Measured data Calculated data Expected range Attendance monitoring system Student ID Date/Time Number of students attending Percentage of enrolled students attending 40-90% 14
- 15. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 Attributes Core/supplementary Compulsory/optional Digital/physical Scheduled/in own time Formative/summative Group/individual % of final mark Assessment method Prerequisite Adaptive release Instrument Measured data Calculated data Expected range Attendance monitoring system Student ID Date/Time Number of students attending Percentage of enrolled students attending 40-90% Audience response system 15
- 16. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 Attributes Core/supplementary Compulsory/optional Digital/physical Scheduled/in own time Formative/summative Group/individual % of final mark Assessment method Prerequisite Adaptive release Instrument Measured data Calculated data Expected range Attendance monitoring system Student ID Date/Time Number of students attending Percentage of enrolled students attending 40-90% Audience response system Student ID Question ID Response 16
- 17. Curriculum objects Curriculum analytics Type: Lecture Descriptive data Course ID: Mathematics 101 Lecture number: 03/10 Learning objective: understand polynomial representations Lecturer: Joe Simpson Location: LT203 Attributes Core/supplementary Compulsory/optional Digital/physical Scheduled/in own time Formative/summative Group/individual % of final mark Assessment method Prerequisite Adaptive release Instrument Measured data Calculated data Expected range Attendance monitoring system Student ID Date/Time Number of students attending Percentage of enrolled students attending 40-90% Audience response system Student ID Question ID Response Percentage of attendees correctly responding to each question Average score over all questions 70-100% 70-90% 17
- 18. A curriculum object describes an aspect of the curriculum, the data and the analytics that can be used to enhance it
- 19. • Lecture, including - Presentation - Individual activity (eg self-check) - Group discussion - Audience response • Physical textbook • Online textbook • Lab • Seminar/tutorial • Reading material • Journal article • LMS content page Curriculum analytics • Assessed activities - Essay - Exercise - Artwork - Code - Project - Presentation - Exam - Reflective journal - Goal setting - Forum - Survey • Audio • Video - Primary content - Supplementary - Interactive - Lecture capture • Interactive • Online content (different ways to present it) • Self-declared data • Online meeting • Group meeting (without tutor) • Student feedback • Office hours 19
- 21. Data about the student Curriculum analytics Student information system VLE Library Self-declared data Learning records warehouse About the student Activity data TinCan (xAPI) ETL 21
- 22. Building on the LA Architecture Curriculum analytics Engagement dataStudent records Learning Data Hub Presentation and action Data storage and analysis Data collection Course and module information Lecture activity Learning outcomes Student feedback Aggregated Data HubData processing tools Institutional BI Tools Staff dashboards in Data Explore/Curriculum Design tools 22
- 25. Curriculum analytics Course Course instance Module Module instance Assessment Assessment instance Student Activity Data - VLE, attendance, … Existing UDD New Data Additional course information Additional module information Sessions ie lectures, labs etc Learning design activities Learning outcomes Calculations Derived course and module data Student activity data metrics Lecture capture data Feedback data 25
- 28. User stories Outputs from workshop Nov 2018 • 63 user stories • Institutional goals - Improving benchmarks (eg NSS, TEF) - Quality assurance - Resource usage • Course/module review - Data-informed course and module design - Course reporting dashboard - Improving assessment - Online learning optimisation - Feedback driven course design Curriculum analytics28
- 29. Discussion activity Choose one area and discuss… 1. Course and module insights for teachers 2. Testing learning designs 3. Effective learning technologies Curriculum analytics Consider the following: • What additional questions would you want to ask in this area? • Are there any issue you see with collecting the data? • How can colleagues be encouraged to use the analytics to change their teaching practices? 29 https://tinyurl.com/y3yf48kz
- 30. customerservices@jisc.ac.uk jisc.ac.uk Paul Bailey Senior co-design manager Paul.bailey@jisc.ac.uk
Editor's Notes
- #27: The first one shows a programme (Foundation CS) with four core metrics at module level (attendance, digital engagement, pass on first attempt, and student satisfaction), and changes from semester 1 to semester 2. Then on the right we have some correlation plots showing attendance is strongly correlated with passing on first attempt, and that cohort size is negatively correlated with student satisfaction.
- #28: The second dashboard is looking at the programme in more depth, with scoring of metrics for each module in the programme for the previous academic year on the left with problem areas being highlighted in shades of red. On the right are the trends over the past 5 years for each metric. In the actual dashboard, clicking any module id filters the trends to show for the selected module.