Learning Analytics @ The Open University

Editor's Notes

• #3: Belinda This encapsulates our strategy which is moving forward on all fronts. Kevin will now demonstrate an operation tool available at scale and one of our latest experimental prototypes.
• #4: Belinda This encapsulates our strategy which is moving forward on all fronts. Kevin will now demonstrate an operation tool available at scale and one of our latest experimental prototypes.
• #6: Belinda Analytics is at the heart of the university’s strategic priority to deliver an outstanding student experience. We’ve developed this vision that drives our development of the use of analytics for both short term action and long term strategic decision making.
• #8: Belinda Our strategy is based around the 3 key underpinning strengths we need to develop as an institution. Each equally important.
• #9: Belinda This encapsulates our strategy which is moving forward on all fronts. Kevin will now demonstrate an operation tool available at scale and one of our latest experimental prototypes.
• #10: Belinda This encapsulates our strategy which is moving forward on all fronts. Kevin will now demonstrate an operation tool available at scale and one of our latest experimental prototypes.
• #12: Needing to improve our longer-term forecasting of student numbers we are now able to predict a students chances of success by mapping their situation/characteristics to 30 of the most signification factors associated with student success: These include student characteristics or factors such as previous educational attainment occupational status previous study or reasons for study the number of credits already achieved the total number of previous passes or withdrawals the number of credits being studied in the year latest assignment scores what modules or qualification they are studying. We then map this data to historical records. Essentially we model one cohort of students based on the activates and outcomes of previous cohorts. This enables us to correlate those student characteristics to likely student outcome. This aggregated data is very accurate – providing us with a 2% tolerance. This data can be used to predict student progression and therefore income. As well as being applied to a cohort these predictions can also be used to trigger interventions with individual students who may have a low probability of success.  
• #15: Mention Zdenek Zdrahal and how session… In the second approach we have developed a tool called OU Analyse  which enables us to undertake a week by week analysis of student engagement including study engagement with their learning activities. From this data we can analyse previous student cohorts and correlate learning activities against outcome. This enables us to identify, on a module by module basis, which activities carry the greatest value in terms of student success. To note: the predictive model assesses a student using 4 different algorithms, each of the algorithms takes into account different factors and the historic data we hold about previously successful students. The model will then give a student a vote for each of the algorithms that shows the student is at risk, therefore the more votes you have the higher the likelihood you are at risk. On this slide, the diagram represents one of the algorithms that takes into account the learning activities, which, are represented by the circles. Blue circles are activities that combine in different ways, actions such as forum use, interaction or downloading of resources, or online content. The pink circle shows a week of non-engagement with the VLE. Here we see a stylised pattern of weekly activity of a student who successfully achieved a pass for this module. Contrast this with the activity of a student who didn’t submit. You can see that the pattern is very different. In this example, the student who didn’t submit spend periods not engaging with the VLE. By modelling the pattern of behaviour of students who pass, fail or don’t submit, we are able to get a predict what successful and not successful VLE engagment looks like, and get a ‘finger-print’ for each module. Armed with this knowledge we are then able to monitor activity within the current cohort and quickly identify individual students who are not engaging with those high value activities. These students are flagged as ‘at risk’ and can be offered targeted interventions. Finally, we have also applied analytics to the exploration of the impact of learning design on learning performance.   Takes demographic data + presentation-related data (aggregated VLE data available weekly) For each module, identifying important VLE actions, e.g., access forum, access or download a resource, access OU content. Learning activities will include a combination of these actions Identify students at risk of failing the module as early as possible so that OU intervention is efficient and meaningful. Finally, we have also applied analytics to the exploration of the impact of learning design on learning performance.   Takes demographic data + presentation-related data (aggregated VLE data available weekly) For each module, identifying important VLE actions, e.g., access forum, access or download a resource, access OU content. Learning activities will include a combination of these actions Identify students at risk of failing the module as early as possible so that OU intervention is efficient and meaningful.
• #16: Based on demographic data Based on VLE activities
• #20: Belinda This encapsulates our strategy which is moving forward on all fronts. Kevin will now demonstrate an operation tool available at scale and one of our latest experimental prototypes.
• #23: Belinda This encapsulates our strategy which is moving forward on all fronts. Kevin will now demonstrate an operation tool available at scale and one of our latest experimental prototypes.
• #30: Belinda This encapsulates our strategy which is moving forward on all fronts. Kevin will now demonstrate an operation tool available at scale and one of our latest experimental prototypes.
• #33: Cluster analysis of 40 modules (>19k students) indicate that module teams design four different types of modules: constructivist, assessment driven, balanced, or socio-constructivist. The LAK paper by Rienties and colleagues indicates that learning design and learning design activities in particular strongly influence how students are engaging in our LMS. VLE engagement is higher in modules with socio-constructivist or balanced variety learning designs, and lower for constructivist designs. In terms of learning outcomes, learning design seems to have an impact on learning performance. Students rate constructivist modules higher, and socio-constructivist modules lower. However, in terms of student retention (% of students passed) constructivist modules have lower retention, while socio-constructivist have higher. In particular, modules with a heavy reliance on content and cognition (assimilative activities) seemed to lead to lower completion and pass rates. Thus, learning design strongly influences behaviour, experience and performance. (and we believe we are the first to have mapped this with such a large cohort). Red lines represent –ve effect Green lines represent +ve effect Blue lines represent no significant effect. Eventually with the availability of yet more data, we may soon be better able to understand the complex relationship between learning design and learning processes and outcomes with resultant sharing of best practice evidenced by such analyses. If this could then be mapped to student characteristics, there is the potential to personalise learning designs. (Learning performance was calculated by the number of learners who completed and passed the module relative to the number of learners who registered for each module.)
• #34: Cluster analysis of 40 modules (>19k students) indicate that module teams design four different types of modules: constructivist, assessment driven, balanced, or socio-constructivist. The LAK paper by Rienties and colleagues indicates that learning design and learning design activities in particular strongly influence how students are engaging in our LMS. VLE engagement is higher in modules with socio-constructivist or balanced variety learning designs, and lower for constructivist designs. In terms of learning outcomes, learning design seems to have an impact on learning performance. Students rate constructivist modules higher, and socio-constructivist modules lower. However, in terms of student retention (% of students passed) constructivist modules have lower retention, while socio-constructivist have higher. In particular, modules with a heavy reliance on content and cognition (assimilative activities) seemed to lead to lower completion and pass rates. Thus, learning design strongly influences behaviour, experience and performance. (and we believe we are the first to have mapped this with such a large cohort). Red lines represent –ve effect Green lines represent +ve effect Blue lines represent no significant effect. Eventually with the availability of yet more data, we may soon be better able to understand the complex relationship between learning design and learning processes and outcomes with resultant sharing of best practice evidenced by such analyses. If this could then be mapped to student characteristics, there is the potential to personalise learning designs. (Learning performance was calculated by the number of learners who completed and passed the module relative to the number of learners who registered for each module.)
• #35: Belinda This encapsulates our strategy which is moving forward on all fronts. Kevin will now demonstrate an operation tool available at scale and one of our latest experimental prototypes.