Give them what they want: Participatory approaches to developing anonymous assignment provision
Download as PPTX, PDF2 likes1,078 views
The document discusses the development of a flexible anonymous assignment workflow at the University of York, emphasizing the importance of electronic management of assessment (EMA) in enhancing efficiency and student experience. It outlines the iterative development process from 2007 to 2016, noting the feedback received from students and markers, which revealed mixed reactions and highlighted the need for improved workflow and technology integration. The evaluation suggests an ongoing need for policy guidance and central strategies to balance diverse departmental requirements.
Give them what they want: Participatory approaches to developing anonymous assignment provision
- 1. Give them what they want: developing a flexible anonymous assignment workflow to meet diverse needs Simon.davis@york.ac.uk Wayne.britcliffe@york.ac.uk University of York
- 2. Outline • EMA in the sector and York • Requirement gathering &“Agile” development • Evaluation and ways forward
- 3. Electronic Management of Assessment (EMA) in the UK • Growing adoption and interest across the sector (UCISA 2014) • Strategic priority for JISC / UCISA / HELF identified benefits for – Students (convenience) – Admin (efficiencies) – Academics (pedagogic) • Turnitin Grademark leading e-submission solution, 63% • End to end solution remains elusive – JISC, Southampton, Northumbria, BB/SITS integration?
- 4. Generic EMA lifecycle 1 - Specifying 2 - Setting 3 - Supporting 4 - Submitting 5 - Harvesting the work 6 - Marking and feedback 7 - Returning marks and feedback 8 Reflecting / evaluation Lifecycle model adapted from Manchester University TRAFFIC Project
- 5. Context of EMA @ York • ~14,000 students • Anonymity mandatory “except where unfeasible” – Assessment principles: Equity, Openness, Clarity, Consistency • No current institutional EMA policy /mandate • Widespread opt in to e-assessment workflows – Iterative development delivering the big things first – Flexibility for local adaptations – Active user group and network driving local adoption – 20,558 files submitted, 11,267 feedback files returned online last academic year
- 6. Anonymous assignment submission: iterative development 2007: Online submission • Submitted files anonymised using exam numbers 2013: Feedback return • Feedback forms / marksheets generated and returned to students through SITS 2014: Feedback flexibility • Support for annotated work • Marks and feedback for exams 2015: User experience • Policy & guidance • Markers • Students 2016? Feedback hub • Supervisor access • Prototype in development Increasing departmental uptake
- 7. Lifecycle stage Common requirements Flex? Support Location Specifying Summative anonymity Depts specify assessment X Setting Deadlines, file types, late submissions, fb type Assessment details Y Supporting Student guidance MIT circs Central Y Submitting Anonymised with exam number Text files Multiple files in zips Assessment details Central Y Harvesting Student work Feedback (forms and annotations) Marksheets Tracking tools Dept approach Y Marking and feedback File sharing Feedback forms / templates / annotation Platform Printing F’b approach X Returning marks / fb 6 week anonymous fb turnaround Marks registered in SITS Central Y Reflecting / evaluation Student / supervisor access to anonymous feedback Dept / central FBH
- 8. York Anonymous Assignment: Evaluation Highlights • Institution: Need to fill gaps in policy and guidance • Admin (n=18): Easy to use and saves time • Students (n=261): Generally +ive, FB quality, confidence issues • Markers (n=48): Mixed reaction to all aspects of workflow – Most negative response to reading work on screen – Broadly positive response to feedback production – Areas for development: • Improved workflow and speed / More managed system • Annotations – awareness, workflow, reusable, workload implications • Resistance to greater use of technology 8
- 9. Polarised response from markers “I can see no merit in this system. I have other experience of on- line marking from my external examining and have found it very difficult, time consuming and problematic. There are some parts of the pedagogic process that are not amenable to on-line systems and marking essays is one of these. The students get a worse experience, worse feedback and and I have found it tiresome, unnecessarily complicated and much much slower. I conclude that it is inefficient use of time, makes turning the essays round longer, and pedagogically worse outcome.” “This was my first year using this system and it was revolutionary. I could mark anywhere... Also there was no printing out of forms at the Dept Office. The whole thing was slick and easy.” 9
- 10. Continued incremental improvements • Marker experience – Annotation; explore use of Google drive / docs / forms – Other forms of feedback; portfolios, media etc • Supervisor access – Feedback hub prototype • Central e-assignment strategy • Balance tension between universal / “niche” requirements for central IT developments