Scaling personalised feedback
- 1. Scaling personalised feedback Yi-Shan Tsai University of Edinburgh yi-shan.tsai@ed.ac.uk @yi_shan_tsai LALA symposium—Learning Analytics for Feedback at Scale July 1, 2019, KU Leuven, Belgium
- 3. EVALUATION
- 4. EVALUATION FEEDBACK A product and an evaluation process of the relationship between a set goal and the existing state of learning or performance (Hattie & Timperley, 2007). Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112.
- 5. EVALUATION FEEDBACK DIALOGUE Clarify expectations, monitoring the current progress of learners, and moving towards desired learning goals (Hattie & Timperley, 2007). Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112.
- 6. Self-regulated learning Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of educational research, 65(3), 245-281.
- 7. 7 principles to support SRL • Clarify what good performance is • Facilitate self-assessment • Deliver high quality feedback information • Encourage teacher and peer dialogue • Encourage positive motivation and self-esteem • Provide opportunities to close the gap • Use feedback to improve teaching Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2), 199-218.
- 8. Effective feedback • Feed up – Where am I going? • Feed back – How am I going? • Feed forward – Where to next? Task level Process level Self- regulation level Self level Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112.
- 10. Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using Learning Analytics to Scale the Provision of Personalised Feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/bjet.12592 Image by Eak K. from Pixabay. Massiveness Inclusiveness Personalisation
- 11. “It doesn’t seem consistent with the overall strategy and direction of the university, which is ever-increasing student numbers… We are offering a high quality industrial scale education…. We say somebody is watching you as an individual, there’s an individual member of staff who is looking out for you, but that member of staff doesn’t have time to actually do it, I think that’s really the worst of both worlds.” http://sheilaproject.eu/
- 13. A data-supported feedback model Pardo, A. (2017). A feedback model for data-rich learning experiences. Assessment & Evaluation in Higher Education, 1-11. doi:10.1080/02602938.2017.1356905
- 17. • Influence — Personalised dialogue • Data driven — Situated Sense-making • At scale — Automatic Conceptual exoskeleton
- 22. Helpful feedback Effect size (Cohen’s d) = 0.49 Medium positive effect Midterm Scores Effect size (Cohen’s d) = 0.21 Small positive effect Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using Learning Analytics to Scale the Provision of Personalised Feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/bjet.12592
- 23. Personalised dialogue Image by Andrew Martin from Pixabay https://www.ontasklearning.org/
- 24. Scaling personalised feedback Yi-Shan Tsai University of Edinburgh yi-shan.tsai@ed.ac.uk @yi_shan_tsai LALA symposia July 1, 2019, KU Leuven, Belgium
Editor's Notes
- #3: What is the problem?
- #4: We didn’t see any form of evaluation.
- #5: There was no evaluation that allows any form of feedback on the learner’s progress (the dog in this comic strip). Feedback is understood both as a product and an evaluation process of the relationship between a set goal and the existing state of learning or performance.
- #6: Feedback is a crucial part of communication between students and teachers in terms of clarifying expectations, monitoring the current progress of learners, and moving towards desired learning goals (Hattie & Timperley, 2007).
- #7: Butler and Winne (1995, p. 246) highlight feedback as an evaluation process that can prompt self-regulated activities.
- #10: However, there is substantial evidence showing that higher education struggles to deliver consistent, timely, and constructive feedback to meet the needs and expectations of students
- #11: The inadequacy in delivering effective feedback to students is partly due to conflicts between an increasing focus on ‘massiveness’, ‘inclusiveness’, and ‘personalisation’ in higher education and yet unmatched capacity of staff to produce feedback that speaks to the needs of individual students (Ryan, Gaśević, & Henderson, working paper).
- #13: Learning analytics emerged to address some of the issues that HE encounters today, e.g., providing timely, data-informed feedback to trigger appropriate interventions that may improve learning experience and success.
- #14: Pardo (2018) propose a data-supported feedback model where learning analytics collects and integrates multiple sources of evidence showing learning engagement or achievements. Such evidence is subsequently measured according to the set standards of a task or a learning goal, and analysed alone or with additional sources of data (e.g., student characteristics) to produce feedback, which may further shape a learner’s knowledge, beliefs, and attitudes if used. The feedback tool, OnTask, was developed based on this framework (Pardo et al., 2019).
- #15: OnTask is one of the tools that have been developed to facilitate the process (a LA cycle), particularly in leveraging teaching staff in the provision of feedback. OnTask is meant to improve the efficiency of feedback provision, so that teaching staff may have more time to focus on the content of feedback, as mentioned previously, to feed up, feed back, and feed forwards.
- #17: 0.51-2.32
- #24: If interested in using OnTask to personalize feedback dialogue with students, visit the website. OnTask is OER.