Learning analytics: developing an action plan ... developing a vision

Learning
analytics:
developing an
action plan… …developing
a vision
Rebecca Ferguson
The Open University, UK
SLATE
September 2017

Innovating in technology-enhanced learning (TEL)
The TEL Complex
2
The many elements of
the ‘TEL Complex’ must
all be taken into account
as an innovation is
designed, developed
and embedded
Scanlon, E., Sharples, M., Fenton-O'Creevy, M., Fleck, J., Cooban, C.,
Ferguson, R., Cross, S. & Waterhouse, P. 2013. Beyond Prototypes. London: TEL Programme.

Priority areas for education and training
3
• Bringing together different sectors: higher education, schools & workplace learning
• Building enduring networks
• Helping to develop learning analytics capability
• Creating and sharing resources
• Developing visions of the future and agreeing how to work towards them
www.laceproject.eu

LAEP: learning analytics for
European educational policy
4
• What is the current state of
the art?
• What are the prospects for the
implementation of learning
analytics?
• What is the potential for
European policy to be used to
guide and support the take-up
and adaptation of learning
analytics to enhance
education in Europe? http://bit.ly/2jLfx9p

5
The Open University
open.ac.uk

Developing institutional strengths
The OU is developing its capabilities in 10 key areas
6
The university
needs world class
capability in data
science to
continually mine
the data and build
rapid prototypes of
simple tools, and a
clear pipeline for
the outputs to be
mainstreamed into
operations
We need to ensure we have the right architecture and processes
for collecting the right data and making it accessible for analytics
– we need a ‘big data’ mind-set
Benefits will be realised through
existing business processes
impacting on students directly
and through enhancement of
the student learning experience
– we will develop an ‘analytics
mind-set’ in
these areas
The strategic roadmap
will build these
capabilities prioritised
using the indicators and
drivers of student success

Relating design and outcomes
Learning design and analytics at the OU
7

Easily accessible OU data
Learning design and analytics at the OU
8

Learning analytics help us to identify
and make sense of patterns in the data
to improve our teaching, our learning
and our learning environments

Educators use analytics to…
• Monitor the learning process
• Explore student data
• Identify problems
• Discover patterns
• Find early indicators for success
• Find early indicators for poor marks or drop-out
• Assess usefulness of learning materials
• Increase awareness, reflect and self reflect
• Increase understanding of learning environments
• Intervene, supervise, advise and assist
• Improve teaching, resources and the environment
10
Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti, M. A., & Schroeder, U. (2013).
Supporting Action Research with Learning Analytics. Paper presented at LAK13.

Learners use analytics to…
• Monitor their own activities and interactions
• Monitor the learning process
• Compare their activity with that of others
• Increase awareness, reflect and self reflect
• Improve discussion participation
• Improve learning behaviour
• Improve performance
• Become better learners
• Learn!
11
Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti, M. A., & Schroeder, U. (2013).
Supporting Action Research with Learning Analytics. Paper presented at LAK13.

Rapid Outcomes Modelling Approach (ROMA)
The ROMA Framework
12
Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S., & Dawson, S.. (2015). Setting learning analytics in
context: overcoming the barriers to large-scale adoption. Journal of Learning Analytics, 1(3), 120-144.
Adapted from: Young, J., & Mendizabal, E. (2009). Helping researchers become policy entrepreneurs: How to
develop engagement strategies for evidence‐based policy‐making. ODI Briefing Papers. London, UK: ODI.
Define (and
redefine)
your policy
objectives

What does success look like?
13
Academic analytics can guide future change
Student perspectives
● Overall, I am satisfied with the quality of this module
● Overall, I am satisfied with my study experience
● I would recommend this module to other students
● I was satisfied with the support provided by my tutor on this module
● I enjoyed studying this module
● This module met my expectations
Academic perspectives
● The students were well prepared
● The students met specified learning outcomes
● The students defined and achieved their own learning goals
University perspectives
● The module enhanced the university’s reputation
● The module aligned well with others
● The module generated income

What does success look like?
● Students demonstrate the skills necessary to network, collaborate,
browse and reflect
● Students show progress towards defined learning outcomes
● Students communicate well… when asked to collaborate
● Students access and share links… when encouraged to browse
● Students return to materials... when encouraged to reflect
● Students engage with course content
● Students seek out new challenges
● Students persist when the work is challenging
● Students persist in the face of failure
● Students ask for help… when they are stuck
after several attempts
● Students compare their learning strategies with those of experts
● Students adapt their learning strategies to resemble those of experts
14
Learning analytics help to identify appropriate interventions

Policy objectives
OU Strategic Analytics Investment Programme
15
Vision
To use and apply information
strategically to retain students and
enable them to progress and
achieve their study goals.
This vision requires
• Discursive changes
to the communication of data
and analytics
• Procedural changes
in how learners are supported
• Behavioural changes
associated with sustainable
change in learner support.
Define (and
redefine)
your policy
objectives

Political context
Mapping people and processes
16
Tynan, B. & Buckingham Shum, S. (2013). Designing systemic learning analytics at the Open University.
http://www.slideshare.net/sbs/designing‐systemic‐learning‐analytics‐at‐the‐open‐university

Key stakeholders
17
Define
(and
redefine)
your policy
objectives
A community of stakeholders
working in different areas:
• Intervention and Evaluation
• Data Usability
• Ethics Framework
• Predictive Modelling
• Learning Experience Data
• Professional Data
• Student Tools
Key stakeholders are
• University administrators
• Students
• Educators

Desired behaviour changes
18
Define
(and
redefine)
your policy
objectives
Vision
To use and apply information
strategically to retain students and
enable them to progress and
achieve their study goals.
Desired behaviour changes
• Staff will use and apply
information strategically
• Students will extend their
learning journeys
• Students will complete their
learning journeys
• Students will set learning goals
• Students will work effectively
towards study goals

Engagement strategy
19
Define
(and
redefine)
your policy
objectives
• Data in action is provided to
stakeholders through a live portal,
enabling them to understand learner
behaviour and make adjustments
and interventions that will have an
immediate positive impact.
• Data on action is a more reflective
process that takes place after an
adjustment or intervention.
• Data for action takes advantage of
predictive modelling and innovation
in order to isolate particular
variables and make changes based
on a variety of analysis tools.

Internal capacity to effect change
20
Define
(and
redefine)
your policy
objectives
Includes
• Recruitment
• Capacity building
• Developing an ethical
framework for the use of
learning analytics.

Monitoring
21
Tynan, B. & Buckingham Shum, S. (2013). Designing systemic learning analytics at the Open University.
http://www.slideshare.net/sbs/designing‐systemic‐learning‐analytics‐at‐the‐open‐university

Pedagogy We have a social duty to
facilitate and provide
opportunities for learners
to achieve their full
potential
• Why do we educate people?
• How do people learn?
• What pedagogic outcomes are we
trying to achieve?
• How can we measure those outcomes?
Learning is not only about
success – it is about
learning from failure
there is a time for
learners to be confronted
in order for transformation
and growth to occur
We need to nurture
rich, reflective
communities in both
teaching and learning

Smart houses, wearable
technology, the Internet of
Things and face
recognition are
increasingly part of
everyday life
Hard to believe that there
will be enough processing
power to do this, but I
guess people always say
that when something is
ten years away
A new government
authority that acts as a
trusted clearing house for
data and analytics
Complexity
• How can we understand the
internal process of learning by
measuring external actions?
• How do we engage a wide
range of stakeholders?
• How do we process huge
amounts of data from diverse
sources?

Ethics
• We need some form of
regulation in this area
• Control of data has
ethical implications
• Encourage awareness of
how data are used and
how analytics function
• Focusing on data as a
valuable commodity can
lead to unethical
practices
The key is to establish
the notion that each of us
own our own data: the
companies do not
institutional rules and
regulations must exist
and should meet
certain criteria
As long as the
data is
anonymous
data should be
allowed to be
used in these
kinds of
applications
without any
consent

One of the purposes of LA is to
empower the teachers to
provide better learning for the
individual learners
It is even worse to put
that control in the hands
of system designers and
programmers, thus
embedding their
assumptions and beliefs
• Who should control the data?
• Who should control the learning
and teaching process?
• Who sets goals for learners
and teachers?
• Who needs to understand
the analytic process?
Power
if tracking and monitoring
are used to foster and
support education and
learning, it might be
desirable. If it is used to
monitor and control and
to enforce power it is not
desirable

drawing on previous
legislation in the areas of
privacy, child protection,
data protection,
consumer protection, and
the use of personal data
in medical research
It must be handled as a human
right in the 21st century that
every single person should have
the power to decide, when + how
+ for what purpose + for which
timeframe + ... his/her personal
data can/cannot be used
• Need to regulate protection,
ownership and storage of data
• Need new policies on
education, ethics, privacy and
assessment
• Need to decide how this
regulation is developed and
enforced
Regulation

Very little credible
research has
demonstrated any real
large-scale benefits to
learners or institutions
The use of LA
applications in real
practice has be
conscious of the
limitations of any
analysis, and apply
them in a way that is
coherent with the
limitations of the
approach
we MUST be willing to
unpack the algorithms.
Academics are extremely
unlikely to accept 'black box'
predictive tools - it goes
against the very principles of
critical thought
Validity
How can we be sure that the
results generated by learning
analytics are valid, reliable
and generalisable?

Affect
• Bear in mind what
engages and motivates
teachers and learners
• Be aware that there is
discomfort and unease
about various aspects of
learning analytics
the real fuel of
Learning is
motivation and
volition, which you
cannot capture with
external sensors
I might be an alarmist, but there
is too much at stake: from
developing an underclass of
limited-dimension robiticized
learners, to propaganda-fed
righteous fanatics, an
automated, corrupted learning
environment puts us on a path
to an Orwellian future
autonomy begets
engagement, motivation,
persistence, relevance

Visions
of the
future
bit.ly/28X5tq7
Which vision are you working towards?

31
Slides online at www.slideshare.net/R3beccaF
Rebecca Ferguson @R3beccaF
http://r3beccaf.wordpress.com/

Learning analytics: developing an action plan ... developing a vision

More Related Content

What's hot (20)

Similar to Learning analytics: developing an action plan ... developing a vision (20)

More from Rebecca Ferguson (20)

Recently uploaded (20)

Learning analytics: developing an action plan ... developing a vision

Editor's Notes