'Stepping into the unknown' - Assessment practices in a digital age
1 like252 views
The document discusses implementing electronic assessment in higher education. It notes both benefits and challenges of e-assessment, as well as lessons learned from a case study at the University of Liverpool School of Life Sciences. Key points made include: 1) E-assessment can reduce staff time spent on marking while allowing for self-paced learning, but lacks visual cues and requires new policies; 2) Interoperability, staff engagement, technology issues, and existing policies presented challenges; 3) Communication, learning from others, and understanding e-assessment is not simply adding technology to traditional practices are important lessons.
![The role of e-assessment
What does the literature say?
• Many studies have reported on the positive effect of electronic
assessment (e-assessment) (cf. Bond, et al., 2013; Orsmond & Merry, 2013; Smith,
et al., 2005; Voelkel, 2013; Walker, Voce, & Ahmed, 2012)
– Not least the savings in time and money for paper copy (Bridge & Appleyard,
2008)
• Offers the opportunity for self-paced assessment and a personal
learning experience
– “.. the potential […] to contribute a desirable shift from teacher to student
centred learning” (Gunn, 2006, p. 255)
• Innovative assessment practices can help to increase student
engagement & motivation
– Alternatives to the exam/essay/MCQ format are available and have the
potential to “significantly reduce demand on staff time” (Gunn, 2006, p. 269)](https://image.slidesharecdn.com/assessmentpracticesinadigitalageslideshare-160308104639/85/Stepping-into-the-unknown-Assessment-practices-in-a-digital-age-5-320.jpg)
![Implementing continuous online assessment
The ‘tutor’ perspective (findings from previous study)
Assessment
Time consuming and problematic
“The thought of doing any marking that is not in the exam period sends
shivers down people’s spines. It’s not as awful as it sounds, as people have
grant deadlines and they come in the middle of a semester. It’s not just
laziness which made us do this [the terminal exam].”](https://image.slidesharecdn.com/assessmentpracticesinadigitalageslideshare-160308104639/85/Stepping-into-the-unknown-Assessment-practices-in-a-digital-age-9-320.jpg)
![Implementing continuous online assessment
The ‘tutor’ perspective (findings from previous study)
Scholarship AND research
engaged in pedagogy; over-reliance on assessment; how do students learn?
“We have no culture which forces people to engage. I’m not saying
assessment here is bad, but it can be localised and influenced by ‘corridor
mythology’ […] and staff need up to date information on how you can set
objective tests and see how they achieve different things [because]
everyone has done the simple ‘how many legs has a dog’ multiple choice
questions, but not everyone here in my estimation has done much more
complex demanding questions.”](https://image.slidesharecdn.com/assessmentpracticesinadigitalageslideshare-160308104639/85/Stepping-into-the-unknown-Assessment-practices-in-a-digital-age-10-320.jpg)
![Implementing continuous online assessment
The ‘tutor’ perspective (findings from previous study)
Professional development encompasses technology
Plagiarism; accessibility & reliability
“The obvious problem with online assessment is you can never test to see
if the individual did the work […] or the extent to which they contributed
to the work, unless there’s some magic way that I don’t know of. There’s
no way that you can’t tell that two people sat down and discussed or
someone gave somebody else all the answers.”
“From a staff viewpoint, [we need] training to set up the tests, making
sure all the boxes are correct and feedback is given back at the right time.
I think we also need some guidelines so that everyone does it
consistently.”](https://image.slidesharecdn.com/assessmentpracticesinadigitalageslideshare-160308104639/85/Stepping-into-the-unknown-Assessment-practices-in-a-digital-age-11-320.jpg)
'Stepping into the unknown' - Assessment practices in a digital age
- 1. ‘Stepping into the unknown’ – Assessment Practices in a Digital Age Pete Alston – Lecturer (Learning Technology) School of Life Sciences @PeteAlston palston1 1 LIFECHANGING WorldShaping Research Seminar (March 2016) – Department of Computing, Edge Hill University
- 2. Overview What’s coming up … • Assessment in Higher Education – The role of electronic assessment • Case Study - Life Sciences – The rationale – Findings from previous study – Implementation • Challenges faced … – … lessons learned
- 3. Before we get going … ‘Electronic Assessment’ – What does that term mean to you? – Don’t discuss(!) – … will see why later on
- 4. Assessment in Higher Education What does the literature say? • Plethora of research into the importance of assessment & the integral part it plays in the student experience – Yet lowest scores in NSS are reserved for assessment and feedback practices (Beaumont, O'Doherty, & Shannon, 2011; Nicol, 2009; Williams & Kane, 2009) • Why? – Formative vs. Summative? Difficulty in striking a balance? – Lack of empirical research regarding tutor attitudes and experiences? (Hernández, 2012) • Continuous assessment embodies both formative and summative functions – Important to find the balance (Gibbs, 2006; McDowell, et al., 2007; QAA, 2007)
- 5. The role of e-assessment What does the literature say? • Many studies have reported on the positive effect of electronic assessment (e-assessment) (cf. Bond, et al., 2013; Orsmond & Merry, 2013; Smith, et al., 2005; Voelkel, 2013; Walker, Voce, & Ahmed, 2012) – Not least the savings in time and money for paper copy (Bridge & Appleyard, 2008) • Offers the opportunity for self-paced assessment and a personal learning experience – “.. the potential […] to contribute a desirable shift from teacher to student centred learning” (Gunn, 2006, p. 255) • Innovative assessment practices can help to increase student engagement & motivation – Alternatives to the exam/essay/MCQ format are available and have the potential to “significantly reduce demand on staff time” (Gunn, 2006, p. 269)
- 6. E-assessment – a caveat What does the literature say? • As students become more self-regulated, we, as educators, must also evolve – Consider our own role and assessment practices to meet the challenge (Beebe, Vonderwell, & Boboc, 2010) • E-assessment has the potential to engage tutors and learners in a meaningful educational experience (Gikandi, Morrow, & Davis, 2010) – Need to consider fundamental issues of validity, reliability and dishonesty • We can use traditional methods to inform ‘new’ pedagogy, but … – The lack of visual cues and asynchronous communication mean that e- assessment cannot be conducted in the same way as face-to-face assessment (Reeves, 2000) – Lack of empirical research which guides tutors on the development of policies concerning e-assessment (Milam, Voorhees, & Bedard-Vorhees, 2004)
- 7. http://images.liv.ac.uk Case Study – Life Sciences A little about us!
- 8. • Wide range of programmes – Anatomy & Human Biology, Biochemistry, Genetics, Microbiology, Tropical Disease Biology, Zoology, and many more … – Practical (Lab) and Theoretical content • One of the largest departments in the University – Over 1600 students (undergraduate/postgraduate) – More than 100 academics contributing to teaching • Curriculum Review - 2013 – Continuous Assessment forms a key component of all new modules – Online assessment a substantive component of Year 1 • Move towards electronic submission, marking & feedback University of Liverpool (2015) “ http://images.liv.ac.uk
- 9. Implementing continuous online assessment The ‘tutor’ perspective (findings from previous study) Assessment Time consuming and problematic “The thought of doing any marking that is not in the exam period sends shivers down people’s spines. It’s not as awful as it sounds, as people have grant deadlines and they come in the middle of a semester. It’s not just laziness which made us do this [the terminal exam].”
- 10. Implementing continuous online assessment The ‘tutor’ perspective (findings from previous study) Scholarship AND research engaged in pedagogy; over-reliance on assessment; how do students learn? “We have no culture which forces people to engage. I’m not saying assessment here is bad, but it can be localised and influenced by ‘corridor mythology’ […] and staff need up to date information on how you can set objective tests and see how they achieve different things [because] everyone has done the simple ‘how many legs has a dog’ multiple choice questions, but not everyone here in my estimation has done much more complex demanding questions.”
- 11. Implementing continuous online assessment The ‘tutor’ perspective (findings from previous study) Professional development encompasses technology Plagiarism; accessibility & reliability “The obvious problem with online assessment is you can never test to see if the individual did the work […] or the extent to which they contributed to the work, unless there’s some magic way that I don’t know of. There’s no way that you can’t tell that two people sat down and discussed or someone gave somebody else all the answers.” “From a staff viewpoint, [we need] training to set up the tests, making sure all the boxes are correct and feedback is given back at the right time. I think we also need some guidelines so that everyone does it consistently.”
- 14. Challenges faced What we were up against … Interoperability maintaining different assessment platforms at the same time Staff engagement deciding on questions and making them available; resorting to ‘traditional’ practices Facilities 400+ students taking a test will cause problems; smaller modules are no different Technology ‘force completion’; downtime (systems/WiFi); devices
- 15. Challenges faced My personal favourites … • Student: “Why have I got that question wrong?” • How many ways to write an answer? – Let me show you an example … • Existing University Policy – “Where is it?” – FAQSC • “What is an online test” coursework? exam? something else? – Financial Suspension/Late submissions
- 16. Lessons learned So you don’t have to go through what we have! • Previous experience can guide you – But be careful - formative ≠ summative! • ‘Framework’ for assessment – ‘What do you understand by e-assessment?’ • Communication is essential – How will you ‘deal’ with existing policy? – And the ‘smaller’ issues … • What happens if the VLE goes down? • What if a student is ill/mitigating circumstances? • What about a student with a Learning Support Plan?
- 17. Student Announcement about the upcoming tests …
- 18. Lessons learned So you don’t have to go through what we have! • Previous experience can guide you – But be careful - formative ≠ summative! • ‘Framework’ for assessment – ‘What do you understand by e-assessment?’ • Communication is essential – How will you ‘deal’ with existing policy? – And the ‘smaller’ issues … • Learn from each other – What is going on in other departments?
- 19. Summary What’s the take home message? • Impetus on staff to use a variety of instructional and assessment methods – “Natural affinity with technology” (Waycott, Bennett, Kennedy, Dalgarno, & Gray, 2010, p. 1202) But … • Lack of adequate support is one of the major factors for non-engagement in online teaching and learning (Hiltz, Kim, & Shea, 2007) • Electronic assessment ≠ add on to traditional assessment! – (Unreliable) technology demands a rethink as to how students are assessed • Hard to plan for all scenarios at the ‘start’ – Keeps you on your toes and will be a work in progress! • An awareness of innovative practice in other areas is ‘invaluable’ – Let someone else suffer the pain, anxiety and stress!
- 20. Questions? Angelo, T., & Cross, K. (1993). Classroom Assessment Techniques: A Handbook for College Teachers. San Francisco, CA: Jossey-Bass. Beaumont, C., O'Doherty, M., & Shannon, L. (2011). Reconceptualising assessment feedback: a key to improving student learning? Studies in Higher Education, 36(6), 671-687. doi:10.1080/03075071003731135 Beebe, R., Vonderwell, S., & Boboc, M. (2010). Emerging Patterns in Transferring Assessment Practices from F2f to Online Environments. Electronic Journal of e-Learning, 8(1), 1-12. Retrieved from http://www.ejel.org/issue/download.html?idArticle=157 Beecher, T., & Trowler, P. (2001). Academic Tribes and Territories: Intellectual Enquiry and the Culture of Disciplines. Buckingham: SRHE with Open University Press. Bond, E., Bodger, O., Skibinski, D., Jones, D., Restall, C., Dudley, E., & van Keulen, G. (2013). Negatively-Marked MCQ Assessments That Reward Partial Knowledge Do Not Introduce Gender Bias Yet Increase Student Performance and Satisfaction and Reduce Anxiety. PLoS One, 8(2). doi:10.1371/journal.pone.0055956 Bridge, P., & Appleyard, R. (2008). A comparison of electronic and paper-based assignment submission and feedback. British Journal of Educational Technology, 39(4), 644-650. doi:10.1111/j.1467- 8535.2007.00753.x Gikandi, J., Morrow, D., & Davis, N. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333-2351. doi:10.1016/j.compedu.2011.06.004 Gunn, C. (2006). Engaging Learners Through Continuous Online Assessment. In D. Hung, & M. Khine, Engaged Learning with Emerging Technologies (pp. 255-273). Netherlands: Springer. Hernández, R. (2012). Does continuous assessment in higher education support student learning? Higher Education, 64(4), 489-502. doi:10.1007/s10734-012-9506-7 Hiltz, S., Kim, E., & Shea, P. (2007). Faculty motivators and de-motivators for teaching online: Results of focus group interviews at One University. 40th Annual Hawaii International Conference on System Sciences. Hawaii. Retrieved from http://web.njit.edu/~hiltz/publications/HICSS_HiltzKimShea.pdf Laurillard, D. (2005). E-Learning in Higher Education. In P. Ashwin, Changing Higher Education: The Development of Learning and Teaching (pp. 71-84). London: Routledge. Milam, J., Voorhees, R., & Bedard-Vorhees, A. (2004). Assessment of Online Education: Policies, Practices and Recommendations. New Directions for Community Colleges, 2004(126), 73-85. doi:10.1002/cc.156 Miller, J., Martineau, L., & Clark, R. (2000). Technology Infusion and Higher Education: Changing Teaching and Learning. Innovative Higher Education, 24(3), 227-241. doi:10.1023/B:IHIE.0000047412.64840.1c Nicol, D. (2009). Quality Enhancement Themes: The First Year Experience. Transforming assessment and feedback: enhancing integration and empowerment in the first year. Mansfield: Quality Assurance Agency for Higher Education. Norton, P., & Wiburg, K. (2003). Teaching with technology: Designing opportunities to lean. Belmont, CA: Thompson-Wadsworth. Oncu, S., & Cakir, H. (2011). Research in online learning environments: Priorities and methodologies. Computers & Education, 57(1), 1098-1108. doi:10.1016/j.compedu.2010.12.009 Orsmond, P., & Merry, S. (2013). The importance of self-assessment in student's use of tutors' feedback: a qualitative study of high and non-high achieving biology undergraduates. Assessment & Evaluation in Higher Education, 38(6), 737-753. doi:10.1080/02602938.2012.697868 Prensky, M. (2001). Digital Natives, Digital Immigrants. On the Horizon, 9(5), 1-6. doi:10.1108/10748120110424816 Reeves, T. (2000). Alternative approaches for online learning environments in higher education. Journal of Educational Computing Research, 23(1), 101-111. doi:10.2190/GYMQ-78FA-WMTX-j06C Rowley, D., Lujan, H., & Dolence, M. (1998). Strategic choices for the academy: How demand for lifelong learning will re-create higher education. San Francisco, CA: Jossey-Bass. Smith, A., Stewart, R., Shields, P., Hayes-Klosteridis, J., Robinson, P., & Yuan, R. (2005). Introductory Biology Courses: A Framework To Support Active Learning in Large Enrollment Introductory Science Courses. Cell Biology Education, 4(2), 143-156. doi:10.1187/cbe.04-08-0048 Sujo de Montes, L., & Gonzales, C. (2000). Been there, done that: reaching teachers through distance education. Journal of Technology and Teacher Education, 8(4), 351-371. Trowler, P., & Knight, P. (2002). Exploring the Implementation Gap: Theory and Practices in Change Interventions. In P. Trowler, Higher Education Policy and Institutional Change (pp. 142-163). Buckingham: SRHE and Open University Press. Ulmer, G.L. (2003). Internet Invention: From Literacy to Electracy. New York: Longman. Walker, R., Voce, J., & Ahmed, J. (2012). UCISA 2012 Survey of Technology Enhanced Learning for higher education in the UK. Retrieved from UCISA: http://www.ucisa.ac.uk/~/media/groups/ssg/surveys/TEL_survey_2012_with%20Apps_final.ashx Voelkel, S. (2013). Combining the formative with the summative: the development of a two-stage online test to encourage engagement and provide personal feedback in large classes. Research in Learning Technology, 21(19153). doi:10.3402/rlt.v21i0.19153 Waycott, J., Bennett, S., Kennedy, G., Dalgarno, B., & Gray, K. (2010). Digital divides? Student and staff perceptions of information and communication technologies. Computers and Education, 54(4), 1202-1211. doi:10.1016/j.compedu.2009.11.006 Whitelock, D., & Brasher, A. (2006). Roadmap for e-assessment. Retrieved from http://www.jisc.ac.uk/whatwedo/programmes/elearningpedagogy/assessment.aspx Williams, J., & Kane, D. (2009). Assessment and Feedback: Institutional Experiences of Student Feedback, 1996 to 2007. Higher Education Quarterly, 63(3), 264-286. doi:10.1111/j.1468- 2273.2009.00430.x