UTS Case Study - Ascilite2015 Learning Analytics Workshop
4 likes2,003 views
The document discusses the introduction of an academic writing analytics tool at UTS, led by Simon Buckingham Shum and collaborators. It highlights the development process, including user requirements and interface design, as well as the importance of validating the tool through student feedback and ensuring trust among stakeholders. Key findings emphasize the tool's benefits in providing constructive feedback and fostering academic growth without the emotional strain of traditional assessment.
UTS Case Study - Ascilite2015 Learning Analytics Workshop
- 1. Introducing Analy0cs at UTS Example: Academic Wri0ng Analy0cs Simon Buckingham Shum Director, Connected Intelligence Centre Professor of Learning Informatics @sbuckshum #LearningAnalytics utscic.edu.au
- 2. Stakeholders to build writing analytics tool 2 Ágnes Sándor Xerox Research Centre Europe Grenoble (linguist: modelling academic writing) Xerox Incremental Parser Rosalie Goldsmith Institute for Multimedia Learning (researcher in reflective writing in engineering) Philippa Ryan Faculty of Law (academic teaching civil law) Reflective writing rubrics Civil Law student writing Simon Buckingham Shum Shawn Wang, Simon Knight Connected Intelligence Centre (learning analytics researchers and web developer) User requirements, interface design and evaluation, data collection & analysis
- 3. AWA: Analytical Academic Writing 3 Roll over sentences with Fkeys for a popup reminding you of their meaning
- 4. Analytical academic writing (Civil Law) 4
- 5. Engaging Law academic in tuning AWA 5 1. Trigger curiosity through CIC briefings 2. Submit collabora0on proposal to CIC 3. Agree 0melines 4. Think aloud user interface walkthrough 5. Preliminary text analyses 6. Systema0c text analyses 7. Refine parser 8. Design student pilot 9. Gather student feedback
- 6. 6 From informal rubric to formal rhetorical pattern
- 7. AWA: reflective academic writing 7
- 8. Validating the tool: metrics 8 human highlighting automated highlighting
- 9. Structured annotation from the civil law academic to tune the parser for her discipline 9
- 10. Validating the tool: student feedback 10 “takes the emo+on out of having your work scru+nized” “it was not embarrassing in the way that it can be when a tutor or marker gives feedback” “I definitely found it useful. It also made me realise that I tend to use bold, certain language in making my point towards the end of each paragraph rather than up front at the beginning (when introducing my point).” “I realise now what descrip+ve wri+ng is -‐ the soCware had quite a bit to say about my lack of jus+fica+on -‐ also true -‐ pressed for +me and difficult circumstances have caused this for me in this instance -‐ good to see it sampled.”
- 11. Validating the tool: academic trust 11 An issue of “Algorithmic Accountability” Critical to forge trust between all the stakeholders: Educators, Analysts, Students We aim to ensure that academics feel that: their expertise is respected their feedback has an impact they understand the tool sufficiently to trust it ideally they get esteem and research outputs from engaging with analytics