Academics' Understanding of Authentic Assessment
Download as PPT, PDF1 like921 views
The document explores the concept of authentic assessment among academics at the Open University, aiming to comprehend their understanding and application of it through a questionnaire. Key findings reveal varying perceptions of authenticity in assessment tasks, with significant importance placed on realistic and meaningful tasks that align with learning outcomes. The document discusses different dimensions, issues, and the types of assessments being utilized across various disciplines.
![Academics’ Understanding of Authentic Assessment Denise Whitelock & Simon Cross Institute of Educational Technology The Open University [email_address] [email_address]](https://image.slidesharecdn.com/caa-2011-whitelockcrossfv-110708050339-phpapp02/85/Academics-Understanding-of-Authentic-Assessment-1-320.jpg)
![Authentic assessment Wiggins, 1993 – ‘[the extent to which] a student experiences questions and tasks under constraints as they typically and “naturally” occur, with access to the tools that are usually available for solving such problems.’ Synonym for performance assessment (Hart, 1994; Torrance, 1995) Realistic value of the task and context (Herrington & Herrington, 1998) Authentic to what? Criterion situation (Gulikers, Bastiaens & Kirschner, 2008) 1](https://image.slidesharecdn.com/caa-2011-whitelockcrossfv-110708050339-phpapp02/85/Academics-Understanding-of-Authentic-Assessment-5-320.jpg)
![Discussion 1 Achieving something ‘meaningful’ Real world assessment importance ‘ Real experiences’ of collaboration Awareness and understanding [email_address] [email_address]](https://image.slidesharecdn.com/caa-2011-whitelockcrossfv-110708050339-phpapp02/85/Academics-Understanding-of-Authentic-Assessment-19-320.jpg)
Academics' Understanding of Authentic Assessment
- 1. Academics’ Understanding of Authentic Assessment Denise Whitelock & Simon Cross Institute of Educational Technology The Open University [email_address] [email_address]
- 2. Outline 1 Introducing Authentic Assessment Results Response rates Defining the survey items Discussion Project overview
- 3. Project objectives To understand what is meant by authentic assessment in the literature by examining a set of examples of authentic assessments. To construct a questionnaire which could be used by Open University academics to explore their understanding of authentic assessment. To investigate through means of a questionnaire, the types of assessment academics were currently undertaking and whether they fitted into, through means of a questionnaire, a broad definition of authentic assessment. 1
- 4. Familiar terms? 1 Terms probed by questionnaire % responses Alternative assessment 37 Authentic assessment 24 Feed-forward 59 Authentic learning 26 Learning Design 55
- 5. Authentic assessment Wiggins, 1993 – ‘[the extent to which] a student experiences questions and tasks under constraints as they typically and “naturally” occur, with access to the tools that are usually available for solving such problems.’ Synonym for performance assessment (Hart, 1994; Torrance, 1995) Realistic value of the task and context (Herrington & Herrington, 1998) Authentic to what? Criterion situation (Gulikers, Bastiaens & Kirschner, 2008) 1
- 6. 5 dimensions of authenticity Task Physical context Social context The form containing multiple context of learning Criteria based on criteria used in professional practice 1 (Gulikers et al)
- 7. Issues Assessment is itself inauthentic Reliability Student perceptions Student treat it as authentic Cost/time Empirical research Favours certain disciplines 1
- 8. Developing survey items Authentic intellectual work Authentic situations Authentic measurement and criteria Authentic competence or performance Authentic learning experience Authentic personal trajectories Authentic audiences Authentic design structures 1
- 9. Items 1
- 10. Items 1
- 11. Response rate 1 CAU Number of Responses Social Sciences 7 Maths, Computing & Technology 39 Science 24 Faculty of Education & Language Studies 6 Health & Social Care 7 OU Business School 10 Institute of Educational Technology 8 Total 102
- 12. Results Key research questions What aspects of authenticity in assessment are important to academics? How successful do academics feel they have been in delivering on factors associated with authentic assessment? What types of technologies and pedagogies are being used? 1
- 13. 1. Perceived importance 18 items ‘ It is probable that your subject area adopts an approach to assessment that may differ from other disciplines. Some considerations or expectations about assessment may be more, or less, important than others. When assessing students in your subject area, how important is it that the following should be included?’ 1
- 14. Results 1 16 17 28 33 Marking criteria that relate specifically to competences and practice 1 16 46 33 Assessment tasks that students enjoy 2 8 44 39 Complex assessment tasks that require use of multiple skills and knowledge 3 16 34 42 Problem tasks that are like those encountered by practitioners or experts in the field 4 18 34 44 Demonstration and use of judgement 1 13 33 46 A range of assessment tasks rather than just the traditional ones 7 6 30 52 Resources taken specifically from real-world case studies or research 5 3 19 74 Tasks that are fully aligned with learning outcomes or objectives 4 3 19 75 Tasks that students find meaningful Not at all important Slightly important Quite important Very important % responding
- 15. Results 1 46 33 14 2 Simulations of role-play or scenarios 36 31 14 10 Grading of assessment by those who, in a relevant real-world situation, would do so 37 28 17 10 Student involvement in the negotiation of the assessment task 43 17 18 10 Examination takes place in real world settings/places 19 33 29 13 Collaboration that is similar to that experienced by practitioners or experts in the field 20 28 23 23 Course work or reflective logs 9 27 31 25 A test of how well the student thinks like a practitioner (is ‘in-tune’ with the disciplinary mind) 8 24 33 30 Processes and methods that are similar to those used by practitioners or experts in the field 5 17 40 31 A sustainable life-long approach to learning Not at all important Slightly important Quite important Very important % responding
- 16. 2. Successful use 1
- 17. 3. Types used 1
- 18. Familiar terms 1 Terms probed by questionnaire % responses Alternative assessment 37 Authentic assessment 24 Feed-forward 59 Authentic learning 26 Learning Design 55
- 19. Discussion 1 Achieving something ‘meaningful’ Real world assessment importance ‘ Real experiences’ of collaboration Awareness and understanding [email_address] [email_address]
Editor's Notes
- #8: Note in passing that we’re aware of the criticisms and concerns levelled at authentic assessment
- #9: Authentic assessment is more about how assessment is done rather that what assessment task takes place.
- #10: Illustrative only
- #11: Illustrative only
- #13: Describe results to 3 of the key questions in the survey. Remind them that there were only questions and about demographics and subject
- #17: The second part of the questionnaire probed how successful the academics were with the following factors. There were only two factors where they declared they were fully successful and these were: Use of resources taken specifically from real-world case studies or research (41% said fully successful) Use of a range of assessment tasks rather than just traditional ones (29% said fully successful) The factors where the academics felt their achievements were mostly successful were: Students consider assessment activities meaningful (49%) Answer problems that are like those encountered by practitioners or experts in the field (39%) Use methods and procedures similar to those used by real practitioners or experts in the field (28%) Meanwhile the areas where there was less success included: Demonstrate how well they think like a practitioner Adopt a sustainable life-long approach to learning Experience collaboration similar to the real experience of relevant practitioners or experts
- #18: Don’t say much about this but hold until the discussion
- #19: Return to slide shown near the beginning
- #20: One of the features of authentic assessment described in the literature is that the assessment tasks are meaningful to the students. Only 26% of the academics surveyed believed they had ‘fully succeeded’ in producing a course where the students considered the assessment tasks meaningful. A further 49% felt they had ‘mostly succeeded’, yet in the first part of the survey 75% of the participants declared that meaningful tasks were very important for assessment in their subject area. This is not such a surprising finding as designing probing, insightful and meaningful assessments is a difficult undertaking but one can see from the responses that the academics in question are working towards this goal. About a third of courses were using fieldwork or work-based learning in their assessment portfolios. However, ‘simulations of role-play’ and ‘examinations taking place in real world settings’ were regarded as of little importance in these subject areas. This is a surprising finding as one would expect those subject domains which make use of fieldwork or work-based assessment in their courses would also consider the examinations taking place in real world settings as important. This finding deserves further investigation and will be followed-up in a set of semi-structured interviews. Another interesting finding was that although 43% of courses are using electronic tutor forms for assessment, the course chairs declared that they had only partially succeeded in designing a course that gives students ‘experience of collaborations that are similar to the real experiences of relevant practitioners or experts’. In fact a ‘real experience’ of collaboration was regarded as ‘not at all important’ to most participants despite almost half of them making use of the tutor forums for assessment purposes. Although many of the courses were employing assessment tasks that could be considered as ‘authentic’, only 25% of the academics had heard of the terms ‘authentic learning’ and ‘authentic assessment’, which is a low response compared with ‘learning design’. However, there has been a well-publicised Learning Design initiative taking place across the University. This finding suggests that Authentic Assessment needs to be given priority in future Assessment projects at the Open University. Particular emphasis needs to be placed on meaningful assessment especially since student negotiation around assessment was not considered important across the different subject domains but has been shown to impact on how meaningful students find their assessments. More negotiation can be seen to take place with open-book, open-web examinations. Williams & Wong (2009) used this approach when assessing final year business students as they believed this approach mirrored real life problem solving scenarios. They also found that authenticity “engages students and inculcates deeper and enriched learning”. Although Cummings & Maxwell (1999) argued that authenticity is the way to go, they found that a lack of understanding what makes an assessment really authentic resulted in a shortfall in assessment practice. This questionnaire has revealed academics’ lack of comprehension and points the way towards increasing understanding in order to avoid making assessment appear on the surface to be more like real-life but the students perceive them as more artificial and contrived. Looking towards frameworks for designing authentic assessment and drawing upon Gulikers et al’s (2008) five dimensions of authenticity will prompt future work in this demanding arena in order to promote the “Assessment for Learning” agenda throughout the University.