Alt c 2016presentation-hattersley
- 1. Enhancing assessment opportunities? Reflections on a co-created assessment matrix for e-portfolios Sara Hattersley Learning Development Centre, University of Warwick S.Hattersley@warwick.ac.uk @sarahattersley Resources available at: http://bit.ly/2caoVxA
- 2. How would you assess your engagement at ALT-C this week? a) Passive observer (attended, no notes, no contributions) b) Active observer (attended, made notes, had own thoughts) c) Passive participant (engaged in workshop activities; didn’t record/publicise or meet people otherwise) d) Active participant (attended, presented, engaged in groups and ‘play’, used social media, conversations with others etc.) Warm-up question!
- 4. Integrated learning through e- portfolios
- 5. About the project The co-creation and testing of an assessment matrix for e-portfolios, as part of the Transforming Technologies PGA Some pedagogical themes of interest: alternative assessment methods in HE, specifically e-portfolios. the notion of 'student as producer' curriculum as a process student agency in curriculum design decisions
- 6. Others are talking about it… JISC (2015) claim there is a “growing body of evidence that highlights the active engagement of learners in assessment and feedback as the critical factor in enhancing learning.” Higher Education Academy (2015) in their recent publication, set out a framework for re-considering assessment and one of the key tenets they describe as ‘integrating assessment literacy into course design’ with the aim of increasing student understanding of standards.
- 7. Summative assessment: digital poster Students were asked to create a ‘digital poster’ in MyPortfolio to describe a technology tool. This poster is to be included in the TAP repository (first student contributions!). As part of the assessment, students helped to design, test and evaluate their experience of the marking matrix. Students produced their own poster, but also peer assessed another student’s poster.
- 8. Working together to create the matrix
- 9. Where do digital skills ‘sit’? HE traditionally assesses: Knowledge and understanding Analysis, synthesis and critique Academic and communication skills Are digital literacies a ‘sub-section’ of the latter, or something more integral across the assessment criteria range?
- 10. Examples from the matrix: ‘knowledge’ Distinction “Solid understanding of the tool used for the digital poster, including the context of the software/platform.” Merit “Good evidence of understanding of the tool used for the digital poster.” Pass “Understanding of the tool used although at a basic user level and may have made more sophisticated choices (layout etc) with more knowledge.” Refer “Limited command of the tool used to create digital poster.”
- 11. Examples from the matrix: ‘presentation skills’ Distinction “Indicative digital content might include: text, hyperlinks, external multimedia, podcasts/screencasts, RSS feeds, infographics, embedded content, polls etc” Merit “Indicative digital content might include: text, hyperlinks, external multimedia, podcasts/screencasts, infographics, and embedded content.” Pass “Indicative digital content might include: text, hyperlinks, infographics, and photographs.” Refer “Digital content might be limited to text and hyperlinks.”
- 12. The ‘new column’: what digital skills are assessed? Use of authentic versus appropriated materials Design skills, layout, readability, visual coherence Connectedness of artefacts, tagging etc. Consideration for reader (scrolling, accessibility, navigation, linkage etc). Currency of materials and links Copyright and security of materials and links Interoperability/multi-platform approaches
- 13. Student poster example 1
- 14. Student poster example 2
- 15. Tutor and peer marking
- 16. Feedback from students “I learned how to import images and You Tube clips and embed links etc.” “It is nice to think that should my digital poster be deemed to be good enough to appear on the TAP, it will be posted there for others to see rather than filed away somewhere (or destroyed) in the way that a typical exam paper would be.” “Marking someone else's poster makes you really look properly at the marking criteria and brings it to life..”
- 17. Successes and issues • Enhancement of student digital skills • Peer assessment exercise valued • Digital poster valued as assessment piece • Raised awareness of assessment processes and criteria • Assessing digital skills enhanced grades • ‘Time-consuming’ nature of true student-tutor collaboration • Ownership of assessment criteria remains with the tutor? Student skills? • How far students’ own submissions are enhanced by co-authoring of assessment criteria? • Assessing digital skills enhanced grades
- 18. Critical questions How far do students want to be ‘producers’ and how do we prepare them to be ‘equal partners’? Does writing your own assessment criteria make you use it more effectively? Can we equate/account for digital literacy skills alongside ‘traditional’ academic criteria in HE assessment?
- 19. Thanks for listening! Sara Hattersley Learning Development Centre, University of Warwick S.Hattersley@warwick.ac.uk @sarahattersley http://bit.ly/2caoVxA
Editor's Notes
- #2: Thanks for the welcome. A short session, so there are additional materials on my Mahara page, which you can follow through the bit-ly link. This is ALT-C ‘Part 2’ for me – make reference to last year’s presentation – the new course Transforming Technologies (10 week, 20 credit PGA Masters level). Small cohort of ‘returning adult learners’ (most of whom worked). – ways in which I was going to try to ‘draw out’ a ‘meta-cognitive understanding of curriculum design and technology, getting students to make curriculum-related decisions, be ‘explicit’ about why I chose certain technologies in the sessions and enable students to build their own marking rubric.
- #3: How might I assess your contributions? 1) How easy or hard was it to assess yourself against this? What extra detail would you need to make a sound assessment (number of tweets, percentage of conference attended, food consumed?!) 2) Would you be interested in something that measured and/or compared your contributions to others? We assume we might be ‘enhancing’ student experience by involving them in ‘meta-curricula’ decisions….but are we?
- #4: I love e-portfolios…..a bit of a saddo! Nothing less ‘academic’ about them; but something more personal and creative – and (above all else) learner-centric. They are not just handy for finding stuff all in one place, but for presenting materials and sharing and (at the ‘highest level’) as a tool for exposing and measuring the transformation of learning. (Springfield 2015 – changes in identity, confidence, attitude etc).
- #5: E-Portfolios have the potential to expand our view of assessment – not just the traditional cognitivist domains, but affective and physical domains. Where is this valued in HE? Today I want to look at the Cognitive and Physical domains particularly.
- #6: The principle objective was to get students to co-write an assessment matrix for the marking of their own work, and then test it/use it in peer assessment of each others’ work. The themes at the bottom are areas of interest for me (but are they to students?). All this was made explicit to students at the start of the programme.
- #7: This theme is very current – the notion of ‘assessment literacy’. I have worked with many students who have not fully understood the criteria by which they are being assessed (some courses do not even share it with them!). Would students understand it better and use it more effectively of they had had an input into its production? My group were all teachers, trainers or instructors in some way, so explicit reference to assessment design is a ‘no-brainer’!
- #8: Technologies for Academic Practice – we presented on this at ALT last year – a repository of ‘experts’ from across the Uni who write about tools they have used and the pedagogical significance.
- #9: This shows how the work on the matrix was threaded through the course curriculum – different points of relevance, but the need to not make this an ‘onerous’ task (after all, it was not the principle reasons students were attending the course!). Hand out the matrix to attendees (or else it can be found on the bit-ly link).
- #10: There was some thought about where digital skills ‘sit’ in a traditional assessment framework. They are non-existent in traditional Warwick frameworks (other than some mention in the ‘communication skills’ sections related mostly to presentation skills). In the end, digital skills were threaded through the matrix, as well as having their own ‘status’ in the final column.
- #11: Here is some sample text from the matrix in the knowledge section – this is about knowledge of the tool used for assessment.
- #12: Here is some sample text from the matrix in the knowledge section – this is about indicative digital content and how the digital poster is presented.
- #13: These are some of the ‘sub-areas’ for assessment in the final column: digital skills.
- #14: Example of a digital poster
- #15: Example of a digital poster
- #16: Students submitted a poster and then peer-marked another one, anonymously. The tutor also provided feedback. Marking students and tutors highlighted in the matrix where they thought the student had met certain competencies. The image above shows (blue circles) where the student and tutors’ marking ‘agreed’. At best there was a 75% correlation between marks.
- #17: The poster creation and peer marking exercise were positively received. The assessment ‘type’ appeared to be valued too.
- #18: The exercise raised critical questions. There was a general valuing of the exercise, but how far students took ownership of these standards may be questionable (these were students on a short course, for one thing). Some students showed that they had used the criteria effectively for marking but NOT for their own work – a disconnect? The inclusion of ‘digital skills’ in its own right did seem to ‘bolster’ students’ overall grade (those with a merit would perhaps only have achieved a pass without this column to consider) – how far is that relevant/right in HE? And if the course were to run again….would we ‘recreate’ these assessment criteria or try to refine them?