eAssessment in practice
- 2. Why assess? • To support and improve student learning • To test learning outcomes
- 3. Assessment – supporting student learning (1) • Quantity and distribution of student effort • Condition 1 Assessed tasks capture sufficient study time and effort • Condition 2 These tasks distribute student effort evenly across topics and weeks • Quality and level of student effort • Condition 3 These tasks engage students in productive learning activity • Condition 4 Assessment communicates clear and high expectations to students • Quantity and timing of feedback • Condition 5 Sufficient feedback is provided, both often enough and in enough detail • Condition 6 The feedback is provided quickly enough to be useful to students
- 4. Assessment – supporting student learning (2) • Quality of feedback • Condition 7 Feedback focuses on learning rather than on marks or students themselves • Condition 8 Feedback is linked to the purpose of the assignment and to criteria • Condition 9 Feedback is understandable to students, given their sophistication • Student response to feedback • Condition 10 Feedback is received by students and attended to • Condition 11 Feedback is acted upon by students to improve their work or their learning Source: Gibbs, G and Simpson, C. (2004). Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education vol.1 pp.3-31.
- 5. eAssessment – overview Key players 4 3 Key issues Types of eAssessment Question & Test Design eAssessment at PolyU Online Tests 1 2 Pros & Cons
- 6. eAssessment – definitions (1) • The Joint Information Systems Committee (JISC) defines eAssessment as: the end-to-end electronic assessment processes where ICT is used for the presentation for assessment activity, and the recording of responses. • The Scottish Centre for Research into On-Line Learning and Assessment (SCROLLA) (2006) defines eAssessment as: an entirely automated process of delivering and marking assessments using web or intranet resources.
- 7. eAssessment – definitions (2) • The University of Oregon defines eAssessment as: the use of digital information and communication technologies to gather and analyse information from multiple and diverse sources to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences
- 8. Questions to think about… • Do you recognise eAssessment as a separate category of assessment? • Is eAssessment just about multiple choice? • Can eAssessment assess learning levels appropriate to study in HE? • Can eAssessment be used to assess relevant knowledge and skills? • Can I use (more) eAssessment in my work?
- 9. Activity 1 What do you think are some of the main eAssessment issues for the key players?
- 10. Key Issues for Organisations, 1 Qualification Boards & Governments • What should be assessed online – high stake? Should it be institution/country-wide? • National frameworks and guidelines • JISC (E-Learning Framework, Good E-Assessment Practice guide), Higher Education Academy, EDNA, Australian Learning & Teaching Council Fellowship e-Assessment, European - Leonardo da Vinci project • Institutional projects, policies and codes of practice • Dundee University, Open University (UK), Deakin University, University of Adelaide • Can the institution handle the necessary academic cultural change?
- 11. Key Issues for Organisations, 1 Qualification Boards & Governments Source: www.scotland.gov.uk
- 12. Key Issues for Organisations, 1 Qualification Boards & Governments • 7 million adults lack functional numeracy and 5 million lack functional literacy. The proportion of adults aged 25 - 64 with low or no qualifications is more than double that in countries like Sweden, Japan and Canada. Source: Leitch Report, 2007 • ….up to half of the 12 000 recruits entering the Army each year have literacy or numeracy skills at levels at or below those expected of a primary school leaver.
- 13. Key Issues for Learning Designers, 2 Publishers & Developers • Enhance the validity & quality of assessment • Promote good practice in assessment • Marking & Grading assessments (in LMS) • (Intelligent) Feedback • Engage learners and teachers in assessment • Equity and diversity of assessment resources (e.g. gender, language, computer anxiety) • Improve cost-effectiveness of assessment • Keep up-to-date with latest technology and pedagogical developments in assessment • Design challenges
- 14. eAssessment – design challenges • symbolic systems • free text processing • computer-assisted validation of human marked scripts • mathematical input • diagrammatic reasoning • free-hand graphical input
- 15. 3 Key Issues for teachers, authors & subject matter experts • What to assess – type? problem-based? project-based? • Administration within LMS (+ Security) • Marking & Grading assessments • Question content – paper-base to electronic • Will it help to meet learning outcomes? • Integration into curriculum • Lack of familiarity with the medium • Perceptions – just multiple choice? • Will it ease the workload? • Will it help the weaker learners (remedial)?
- 16. 4 Key Issues for learners & other stakeholders • Why am I being assessed? Will it help me get my degree? • Are the right skills being tested? • Is it good preparation for the “real” exam? • Will it help to meet the learning outcomes? • Quick, accurate & detailed feedback • Security • Remedial work • Self-assessment • Graduate attributes
- 17. eAssessment – a rationale? • Many assessment strategies focus on what is easier to measure rather than what is important - measuring lower level cognition • But if we want students to have 21century skills we must focus on higher order outcomes. Who decides? The institution? Individual departments? Teachers? Learners?
- 18. eAssessment - types diagnostic tests student’s prior knowledge formative assessment to promote learning by (self- providing feedback. Usually not assessment; counted towards final grade. peer assessment) summative assessment which counts towards final grade. adaptive students have more control over the authored assessment. personal response quick way to assess understanding system
- 19. Assessment – part of the learning Journey
- 20. Activity 2 • How many kinds of activities can be eAssessed? • How do you assess? • What criterion do you use when assessing your activities?
- 21. Forms of eAssessment Form of assessment Examples Traditional assessment submitted online Essays, reviews, reports, case studies Automated assessment Multiple choice, short answer, matching, calculations Automated assessment – advanced options Multiple choice, short answer, matching/label matching Calculations/randomly generated answers, drag & drop Invigilated online exams – (mid/final semester) Range of formats, multiple choice/ short answer, automated Longer essay type Online discussions Forums, online debates/role plays, invited online guests Group projects PowerPoint presentations, CD-ROMs, multimedia projects, group online projects laboratory reports, networked collaborative learning, wikis Authentic assessment Simulations, critical incident analysis, case studies, access to external databases Critical reflection and meta-cognition Electronic portfolios, online journals, blogs, embedded reflective activities Advanced problem-solving Problem-based learning scenarios, learning contracts Table: Forms of eAssessment (adapted from a study by Peter Donnan, University of Canberra: Conducting assessment online. (2004)
- 22. Assessment Methods & Learning Outcomes • Matching learning outcomes with assessment methods Learning Outcomes Assessment methods Higher level •Reflect •Portfolios Thinking •Evaluate •Reflective writing •Predict •Practicum/workplace assessment •Argue •Group projects •Apply •Problem or case-based assessment •Relate •Extended essay •Explain •Essay exam •Describe •Objective test (e.g. MCQ) •Compute •Identify •Recall
- 23. Question & Test Design – more than just multiple choice! • Objective Tests • Question types
- 24. Online tests • PolyU examples • Examples • eAssessment resources
- 25. eAssessment @ PolyU • Moodle – English Language • Clinical Education • Anatomy
- 26. eScholars Group Statistics Simulators Self-test Mobile- Program based Glossary Tool
- 27. Examples • PRINCE 2 • English Language Exams • OpenMark – formative • OpenMark – diagnostic
- 28. Online testing – planning process Source: Sheffield Hallam University, UK
- 31. eAssessment – other resources • QuestionMark Perception • Pearson • Maple TA
- 32. Activity 3 • Can you think of any advantages and disadvantages of eAssessment?
- 33. eAssessment - pros • Immediate feedback can be given to students and to staff • Assessments can be linked to other online course materials being used • Ease of marking/consistency/accuracy • On demand delivery • Geographic flexibility • Time flexibility/convenience • Cost effectiveness • Ease of administration
- 34. eAssessment – cons • Staff need (more) time and extra skills to create eAssessment resources • Testing higher order skills requires greater effort • More difficult to assess written expression or creativity • Restrictions of multiple choice • Accessibility • Authentication/candidate validity • Security/confidentiality • Cost • Technical failure/issues & lack of support
- 35. Questions to think about • Do you recognise eAssessment as a separate category of assessment? • Is eAssessment just about multiple choice? • Can eAssessment assess learning levels appropriate to study in HE? • Can eAssessment be used to assess relevant knowledge and skills? • Can I use (more) eAssessment in my work?
- 36. Goodbye!!! Contact us: Ian etian@inet.polyu.edu.hk Jenny etjeni03@inet.polyu.edu.hk