Stuck in the middle? Making sense of the impact of micro, meso and macro institutional, structural and organisational factors on implementing learning analytics
Download as PPTX, PDF3 likes715 views
The presentation discusses the complexities and challenges of implementing learning analytics across different institutional levels, highlighting the impact of macro, meso, and micro factors. It emphasizes the importance of understanding the complexities of student data and the need for collaboration between students and institutions to effectively utilize learning analytics. Key takeaways include recognizing the limitations of demographic and behavioral data and acknowledging the interconnectedness of student agency and institutional practices.
Stuck in the middle? Making sense of the impact of micro, meso and macro institutional, structural and organisational factors on implementing learning analytics
- 1. Stuck in the middle? Making sense of the impact of micro, meso and macro institutional, structural and organisational factors on implementing learning analytics Presentation at the European Distance Education and E-Learning Network (EDEN) Conference, Genoa, Italy, 17-20 June 2018 Imagecredit:https://www.flickr.com/photos/furiousgeorge81/95137658 Paul Prinsloo University of South Africa (Unisa) @14prinsp Sharon Slade Open University (UK) @SharonSlade Mohammad Khalil Delft University of Technology, Netherlands @TUMohdKhalil
- 2. ACKNOWLEDGEMENTS We do not own the copyright of any of the images in this presentation. We therefore acknowledge the original copyright and licensing regime of every image used. This presentation (excluding the images) is licensed under a Creative Commons Attribution 4.0 International License.
- 3. Overview of the presentation • In the beginning… • Why do we collect what data for what purpose? • Data as broken and often, stuck in the middle • Sticky factors: macro, meso and micro levels • Getting (un)stuck • Two key takeaways • Contact details of authors
- 4. Source credit: https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education In the beginning…
- 5. Source credit: https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education Object of analysis • Institutional • Regional/national • International Who benefits • Administrators/funders/marketing • Funders/administrators • National governments/educational authorities Academic analytics Learning analytics Object of analysis • Course level - discourse analysis • Departmental level – predictive modeling/ patterns of success Who benefits • Learners • Faculty
- 6. Source credit: Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64-71.
- 7. We know, take into account and we measure: age, gender, race, street address and zip code, occupation, pre-enrolment educational data, registration data, engagement data, academic data, library data, financial aid data, behavioural data, location data, who-are-in-their-networks- data, their chances of failing, dropping out, stopping out… Image credit: https://pixabay.com/en/side-profile-black-male-student-1440176/ And we use this data to… Image credit: https://pixabay.com/en/girl-library-education-student-1721436/
- 9. We know, take into account and we measure: age, gender, race, street address and zip code, occupation, pre-enrolment educational data, registration data, engagement data, academic data, library data, financial aid data, behavioural data, location data, who-are-in-their-networks-data, their chances of failing, dropping out, stopping out… Image credit: https://pixabay.com/en/side-profile-black-male-student-1440176/ We forget these data are broken… and often, stuck… Image credit: https://pixabay.com/en/girl-library-education-student-1721436/
- 10. Our data is broken and we often work with proxies and zombie categories Our understanding of learning is incomplete and provisional Agency (institutional and student) is constrained Imagecredit:https://www.publicdomainpictures.net/en/view- image.php?image=172020&picture=brown-knot 1 3 2
- 11. Citation: Pink, S., Ruckenstein, M., Willim, R., & Duque, M. (2018). Broken data: Conceptualising data in an emerging world. Big Data & Society, 5(1), 2053951717753228. 1
- 12. Image credit: https://pixabay.com/en/zombie-horror-creepy-night-531116/ Zombie categories “categories from the past that we continue to use even though they have outlived their usefulness and even though they mask a different reality” (Plummer, 2011, p. 195)
- 13. Image credit: https://pixabay.com/en/side-profile-black-male-student-1440176/ Image credit: https://pixabay.com/en/girl-library-education-student-1721436/ Macro-societal factors, e.g. economic, political, social, technological, environmental and legal factors. Institutional/lecturer/student inactions, inefficiencies, or lack of control impacting and shaping students’ behaviour, chances of failing, dropping out, stopping out… 2 3
- 14. Operational budgets/ resource allocation Identified student needs Institutional understanding Student understanding Institutional locus of control Student locus of control Insufficient & broken data Students are excluded as partners in learning analytics Stuck in the middle…
- 15. Subotzky, G., & Prinsloo, P. (2011). Turning the tide: A socio-critical model and framework for improving student success in open distance learning at the University of South Africa. Distance Education, 32(2), 177-193.
- 16. Sticky factors: Macro level • Institutional understanding and responses shaped by changes at national/regulatory/ funding levels • National imperatives and understanding of the function of higher education • Institutional senior management’s understanding (or lack of) the interdependencies • Beliefs about data/evidence • Lack of political will (and resources) to engage students in the meaning of their data, on what is collected, when it is collected and what other data may provide both the institution and learners with a more comprehensive view of students’ habitus and capital in the “student walk”
- 17. Sticky factors: Meso level Students have a clear responsibility to contribute to their own student walk, but there is less attention directed to other partners in that walk, namely faculty, course teams, support staff, and the institutions themselves.
- 18. Sticky factors: Micro level • The nature, scope and (mis)understanding of student data, the proxies we use and the the role and noise of zombie categories • Our understanding and expectations of causal links and correlations • What students, course teams, faculty and the institution can do is often constrained and dependent on what the other role players do in a particular context
- 19. Important pointers for getting (un)stuck: 1. The student and institution as situated agents – constraint agency/loci of control 2. The student walk – learning as the result of multiple, often mutually constitutive and intersecting variables at the intersections of macro, meso and micro levels 3. The possibility and scope of agency depends on student/institutional habitus, capital and the state of the field. On the student side – the role of inter- and intra-personal connections/relationships. On the institution’s side – academic, social, operational
- 20. Source credit: http://oro.open.ac.uk/48173/ “The evidence shows that the use of learning analytics to improve and to innovate learning and teaching in Europe is still in its infancy. The high expectations have not yet been realised.” Where is the evidence?
- 21. Key Takeaway 1 To only consider student demographic, historical and current academic data combined with their behavioural data underestimates and may misrepresent the complexity of learning and the real potential of learning analytics.
- 22. Unsticking learning analytics depends on acknowledging what students do, can do, and want to do is connected to, and often dependent on what we do, and have the resources to do, in a particular context Key Takeaway 2 Image credit: https://pixabay.com/en/bubble-gum-shoes-glue-dirt-438404/
- 23. Paul Prinsloo (Prof) Research Professor in Open Distance Learning (ODL) College of Economic and Management Sciences, University of South Africa (Unisa), South Africa prinsp@unisa.ac.za Skype: paul.prinsloo59 Personal blog: http://opendistanceteachinga ndlearning.wordpress.com Twitter profile: @14prinsp Sharon Slade (Dr) Senior Lecturer l Student Experience Manager Faculty of Business and Law, Open University Business and Law Schools, Open University, A1, Michael Young Building, Walton Hall, Milton Keynes, MK6 6AA, United Kingdom sharon.slade@open.ac.za Twitter profile: @SharonSlade Mohammad Khalil (Dr) Centre for Education and Learning, Delft University of Technology, Netherlands M.F.D.Khalil@tudelft.nl Twitter profile: @TUMohdKhalil