OLT conference Learning analytics
- 1. Student retention and learning analytics:A snapshot of current Australian practices and a framework for advancement. OLT commissioned research project
- 2. UTS: a data intensive university Overview 1. A case study of an institutional approach 2. The commissioned project
- 3. “We haven’t fully figured out how to put analytics to work pervasively throughout higher education to make a difference and resolve our most pressing issues.” Susan Grajek Vice President of EDUCAUSE in lamenting the lack of broad scale impact that analytics has made to date:
- 5. a university that knows about data, regardless of its volume and diversity. It knows about the use and reuse of data to better inform teaching, learning and research, as well as understand business, society and the university itself, how to learn about and research data, how to store and curate it, and how to apply and develop analytical tools.
- 6. UTS: a data intensive university Learning UTS: a data intensive university 1. Solve problems • attrition • preparation • ‘killer’ subjects 2. Promote student engagement 3. Personalised learning 4. Allocate resources
- 7. Event lifecycle analytics Past Present Future Information What happened? (Reporting) What is happening now? (Alerts) What will happen? (Extrapolation) Insight How and why did it happen? (Modelling, experimental design) What’s the next best action? (Recommendations) What’s the best/ worst that can happen? (Prediction, optimisatioin) Understanding Intervention Source: Davenport et al (2010) Analytics at work
- 9. i-Educator- introductionHi – I’m calling from Student Services. We’re calling all first years just to see how you’re going First year student list (7000+) UTS students Student Systems Outreach program Outreach workflow
- 10. Decision tree model for attrition after two years of engineering degree study for the 2003 Domestic entry cohort at Institution D Source: http://www.altc.edu.au/resource-engineering-qualification-curriculum-uts-2011
- 11. The UTS Model of Learning Why are pass rates for subject X < 20%% Killer subjects
- 12. Basic Analysis Failure rate of 5 killer subjects 48510 is the most stable subject in terms of failure rate 48530 is the most volatile 48540 was no longer offered in recent years 48521 just starts for 3 years 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Basic Analysis • Failure rate in 5 “killer” subjects
- 13. Learning informed by data analytics
- 14. Findings • The following factors relate to failing: – Interval between two adjacent subjects, – the semester of the last year of student enrollment, – average grade, – student pathway.
- 16. UTS: a data intensive university Student dashboard
- 20. UTS: a data intensive university Policies and processes - a roadmapEnvironmental scanning and mapping Policies Data management The role of information technology Staff and organisational structure Communication Numeracy levels – staff and students
- 21. UTS: a data intensive university Data literacy for staff and students
- 23. Student retention and learning analytics:A snapshot of current Australian practices and a framework for advancement. OLT commissioned research project
- 25. Assoc. Prof Shane Dawson Project leader Dr Tim Rogers co-leader Project Partners Associ Prof Dragan Gašević Professor Lori Lockyer Professor Shirley Alexander Gabrielle Gardiner Professor Gregor Kennedy Ms Linda Corrin Professor Karen Nelson
- 26. OLT Commission Project • OLT commissioned project to identify learning analytics technologies, processes and policies Australian universities are employing to address student retention • 2 funded teams (working collaboratively) - – Target policy, future direction and international comparisons (UniSA, UTS, Macquarie, UMelb, USC, UNE) – Identify current technologies and practices (CDU, Griffith, BIITE, UNewcastle, Murdoch)
- 27. Questions Q1. What is the research evidence, the strengths and limitations of learning analytics? Q2. How are Australian universities planning and utilising analytics to support their learning and teaching goals; retention strategies; and identification of at-risk students? Q3. How are international peer universities using and developing analytics to support learning and teaching practice? Q4 What are the future trends for learning analytics and what implications do they have for student retention and informing student learning outcomes?
- 28. Project aims Specifically: - map current (and proposed) LA activity in Australian institutions - identify the drivers shaping, and factors affording and constraining, its adoption - Comparatively situate Australian LA activity within the international context Outcomes: - produce a ‘roadmap’ of best practice, principles and resources that can inform and assist institutions in their LA activity - develop resources that provide an overview of LA including definitions, applications and tools within Australian universities, the overarching barriers and potential opportunities surrounding its deployment, future trends and ethical issues - A Good Practice Guide that will provide case studies of best practice in LA activity that will assist institutions in future LA activity.
- 29. Project methods 1. Semi-structured one-on-one interviews with: - internationally-recognised research experts around current and future trends and developments in LA research (completed) - higher education leaders in North America, UK and Europe with responsibility for LA around their institution’s LA strategy, progress and goals (completed) - DVC’s in Australia around their institution’s LA strategy, progress and goals 2. Delphi-method exercise - - involving panel of international and national experts to explore and provide consensus on themes arising from interviews and survey 3. Survey data - Snapshot of current use and planning for LA at all Australian universities. The survey will be forwarded to DVC’s for input and coordination - Investigation of academic understanding and use of LA in Australian institutions