Implications of TESTA for curriculum design
Download as PPTX, PDF0 likes105 views
This document discusses the implications of TESTA (Thinking about Education, Students, Teaching and Assessment) for curriculum design. It addresses some common problems with assessment and feedback such as an over-reliance on summative assessments, lack of formative feedback, and confusion about learning goals and standards. The document presents case studies of programmes that have successfully implemented more formative assessment and feedback. It also provides principles and tactics for using formative assessment, improving feedback dialogues between students and lecturers, and helping students better understand expectations and criteria. Overall, the document argues that applying TESTA concepts can help rebalance assessment, strengthen connections across modules, and ultimately enhance student learning outcomes.
Implications of TESTA for curriculum design
- 1. Implications of TESTA for curriculum design @solentlearning @tansyjtweets Tansy Jessop SLTI 17 January 2018
- 3. What you know about TESTA
- 4. The challenges of A&F
- 5. Why assessment & feedback? 1) Assessment drives what students pay attention to, and defines the actual curriculum (Ramsden 2003). 2) Feedback is the single most important factor in learning (Hattie 2009; Black and Wiliam 1998).
- 8. TESTA…. “…is a way of thinking about assessment and feedback” Graham Gibbs
- 9. Curriculum design problems TESTA addresses
- 10. Curriculum paradigms PRODUCT: structuring and managing content (WHAT) PROCESS: the lived experience (HOW) PRAXIS: wider purpose of HE, social justice, equality (WHY)
- 11. Curriculum (n) running, chariot, course
- 13. IKEA 101: great for flat-pack furniture but..
- 14. …not so good for ‘slow learning’ • Too much summative • Too little formative • Connections not clear • Promotes disposable learning • Reinforces content-heavy curricula
- 15. TESTA seems to improve students’ perceptions of A&F… 60% 65% 70% 75% 80% 85% 90% 95% Q5 Q6 Q7 Q8 Q9 OS AVERAGENSSSCORES COMPARISON OF 32 PROGS IN 13 UNIVERSITIES WITH SECTOR SCORES NSS 2015 SCORES TESTA SCORES
- 16. The brief TESTA 2017/18 Courses TESTA Audit AEQ Focus Group Team meeting SOMAT Computer Games Development Film and Television Performance Media Technology Media Production February SHSS Social Work Psychology Fitness February SADF Fashion Styling Architectural Technology Construction Management February WSMSE Engineering Maritime Business February SBLC Business Management Marketing Finance and Accounting
- 17. TESTA tactics for problems with 1. High summative and low formative diets 2. Disconnected feedback 3. Confusion about goals and standards
- 18. TESTA definitions Summative: graded assessment which counts towards the degree Formative: Does not count: ungraded, required task with feedback
- 19. 1. High summative: low formative • High summative on UK, Irish, NZ and Indian degrees • Low formative: ratio of 1:8 formative to summative • Formative weakly practised and understood
- 21. Your experience Which of these quotations resonates, and why? Any ideas to address the problem?
- 22. A lot of people don’t do wider reading. You just focus on your essay question. In Weeks 9 to 12 there is hardly anyone in our lectures. I'd rather use those two hours of lectures to get the assignment done. It’s been non-stop assignments, and I’m now free of assignments until the exams – I’ve had to rush every piece of work I’ve done. CONSEQUENCES OF HIGH SUMMATIVE
- 23. It was really useful. We were assessed on it but we weren’t officially given a grade, but they did give us feedback on how we did. It didn’t actually count so that helped quite a lot because it was just a practice and didn’t really matter what we did and we could learn from mistakes so that was quite useful. WHAT ABOUT FORMATIVE?
- 24. If there weren’t loads of other assessments, I’d do it. It’s good to know you’re being graded because you take it more seriously. BUT… If there are no actual consequences of not doing it, most students are going to sit in the bar. The lecturers do formative assessment but we don’t get any feedback on it.
- 25. 1) Low-risk opportunities for students to learn from feedback (Sadler, 1989) 2) Helps students to fine-tune and understand requirements and standards (Boud 2000, Nicol, 2006) 3) Feedback to lecturers from formative tasks helps to adapt teaching (Hattie, 2009) 4) Engages students in cycles of reflection and collaboration (Biggs 2003; Nicol & McFarlane Dick 2006) 5) Encourages and distributes student effort (Gibbs 2004). Why formative matters
- 26. So, how do we do formative? Three case studies of successful formative What made them work?
- 27. Case Study 1 • Systematic reduction of summative across whole business school • Systematic ramping up of formative • All working to similar script • Systematic shift, experimentation, less risky together
- 28. Case Study 2 • Problem: silent seminar, students not reading • Public platform blogging • Current academic texts • In-class • Threads and live discussion • Linked to summative
- 31. How to encourage formative assessment Go to www.menti.com and use the code 49 19 74 Choose the three which you think work best
- 34. System-wide problems • Modular structures impede feedback • Students don’t feel known • There is very little formative feedback • Lecturers do too much judging; students too little
- 35. The feedback is generally focused on the module Because it’s at the end of the module, it doesn’t feed into our future work. If It’s difficult because your assignments are so detached from the next one you do for that subject. They don’t relate to each other. I read it and think “Well, that’s fine but I’ve already handed it in now and got the mark. It’s too late”. STRUCTURAL
- 36. It was like ‘Who’s Holly?’ It’s that relationship where you’re just a student. Because they have to mark so many that our essay becomes lost in the sea that they have to mark. Here they say ‘Oh yes, I don’t know who you are. Got too many to remember, don’t really care, I’ll mark you on your assignment’. RELATIONAL
- 37. An impoverished dialogue The many diverse expressions of dissatisfaction with feedback can be taken as symptoms of an impoverished and fractured dialogue David Nicol 2010
- 39. Irretrievable breakdown… Your essay lacked structure and your referencing is problematic Your classes are boring and I don’t really like you
- 40. Ways to be dialogic • Conversation: who starts the dialogue? • Cycles of reflection across modules • Quick generic feedback • Feedback synthesis tasks • Peer feedback (especially on formative) • Technology: audio, screencast and blogging • From feedback as ‘telling’… • … to feedback as asking questions
- 41. Students feedback to us
- 42. Students to lecturers: Critical Incident Questionnaire Stephen Brookfield’s Critical Incident Questionnaire http://bit.ly/1loUzq0
- 43. Theme 3: Confusion about goals and standards • Consistently low scores on the AEQ for clear goals and standards • Alienation from the tools, especially criteria and guidelines • Symptoms: perceptions of marker variation, unfair standards and inconsistencies in practice
- 44. We’ve got two tutors- one marks completely differently to the other and it’s pot luck which one you get. They read the essay and then they get a general impression, then they pluck a mark from the air. It’s like Russian roulette – you may shoot yourself and then get an A1. They have different criteria, they build up their own criteria.
- 45. There are criteria, but I find them really strange. There’s “writing coherently, making sure the argument that you present is backed up with evidence”.
- 47. Taking action: internalising goals and standards • Regular calibration exercises • Discussion and dialogue • Discipline specific criteria (no cut and paste) Lecturers • Rewrite/co-create criteria • Marking exercises • Discussing exemplars Lecturers and students • Enter secret garden - peer review • Engage in drafting processes • Self-reflection Students
- 48. The TESTA Effect • Rebalancing formative and summative • Greater connections across modules • Better sequencing and progression of assessment across the programme • New approaches to formative, including more authentic assessment • Revives and refreshes curriculum design • Improved student learning and NSS scores.
- 49. From this educational paradigm…
- 52. References Arum R. and Roksa J. 2011. Academically Adrift: Limited Learning on College Campuses. Uni of Chicago Press. Barlow, A. and Jessop, T. 2016. “You can’t write a load of rubbish”: Why blogging works as formative assessment. Educational Development. 17(3), 12-15. SEDA. Boud, D. and Molloy, E. 2013. ‘Rethinking models of feedback for learning: The challenge of design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712. Gibbs, G. & Simpson, C. 2004. Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education. 1(1): 3-31. Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. 2014. ‘An assessment arms race and its fallout: High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Jessop, T. and Tomas, C. 2016 The implications of programme assessment on student learning. Assessment and Evaluation in Higher Education. Published online 2 August 2016. Jessop, T. and Maleckar, B. 2016. The Influence of disciplinary assessment patterns on student learning: a comparative study. Studies in Higher Education. Jessop, T. , El Hakim, Y. and Gibbs, G. 2014. The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88. Nicol, D. 2010. From monologue to dialogue: improving written feedback processes in mass higher education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517. O'Donovan, B , Price, M. and Rust, C. 2008. 'Developing student understanding of assessment standards: a nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 — 217 Sadler, D. R. 1989. ‘Formative assessment and the design of instructional systems’, Instructional Science, 18(2), pp. 119–144. doi: 10.1007/bf00117714. Wu, Q. and Jessop, T. 2018. Formative assessment: missing in action in both research-intensive and teaching focused universities. Assessment and Evaluation in Higher Education.
Editor's Notes
- #2: Tansy
- #9: What started as a research methodology has become a way of thinking. David Nicol – changing the discourse, the way we think about assessment and feedback; not only technical, research, mapping, also shaping our thinking. Evidence, assessment principles. Habermas framework.
- #10: Data – persistent problem A&F scores. NSS not a good enough diagnostic tool. Green, amber, red - more luck than judgement. Traffic light systems – green for good. DVC find the people wo are doing well so we can share best practice. Three programmes. We don’t actually know why.
- #13: Disconnected seeing the whole degree in silos – my module, lecturer perspective (Elephant, trunk, ears, tusks etc) compared to student perspective of the whole huge beast. I realise that what we were saying is two per module
- #14: Not so good for complex learning, integrating knowledge, lends itself to disposable curriculum fragmented learning. Amplified summative, less time for formative. Hard to make connections, difficult to see the joins between assessments, much more assessment, much more assessment to accredit each little box. Multiplier effect. Less challenge, less integration. Lots of little neo-liberal tasks. The Assessment Arms Race.
- #15: The TESTA report back of programme findings was by far the most significant meeting I have attended in ten years of sitting through many meetings at this university. For the first time, I felt as though I was a player on the pitch, rather than someone watching from the side-lines. We were discussing real issues. (Senior Lecturer, Education
- #17: No longer relevant
- #21: Summative as a ‘pedagogy of control’ Teach Less, learn more. Assess less, learn more.
- #43: Is anyone listening?
- #48: Students can increase their understanding of the language of assessment through their active engagement in: ‘observation, imitation, dialogue and practice’ (Rust, Price, and O’Donovan 2003, 152), Dialogue, clever strategies, social practice, relationship building, relinquishing power.