![Opening up Multiple Choice: Assessing with confidence CALRG Conference, June 2011 Jon Rosewell [email_address]](https://image.slidesharecdn.com/openingupmultiplechoice-assessingwithconfidence-110614080022-phpapp01/85/Opening-up-multiple-choice-assessing-with-confidence-1-320.jpg)
![Jon Rosewell [email_address] www.open.ac.uk](https://image.slidesharecdn.com/openingupmultiplechoice-assessingwithconfidence-110614080022-phpapp01/85/Opening-up-multiple-choice-assessing-with-confidence-17-320.jpg)
Opening up multiple choice - assessing with confidence
- 1. Opening up Multiple Choice: Assessing with confidence CALRG Conference, June 2011 Jon Rosewell [email_address]
- 2. Multiple Choice Questions (MCQ) Basic fare of e-assessment: robust easy to implement but pedagogically not ideal Open questions pedagogically preferable but difficult to provide robust and valid marking
- 3. Demo Standard MCQ Standard CBM Open CBM screen dump
- 5. Confidence-Based Marking (CBM) Student selects: answer and level of confidence/certainty Incentive is scoring: Confidently correct: Full marks A tentative correct answer: Some credit Cocksure & wrong: Penalty Pays to be honest about own knowledge
- 6. CBM scoring Tentative & correct Confidently correct Cocksure – and wrong! Score -6 3 High -2 2 Medium 0 1 Low Wrong Correct Confidence
- 7. CBM motivations Rewards care and effort Greater engagement Encourages reflective learning Encourages self-assessment
- 8. Demo Standard MCQ Standard CBM Open CBM screen dump
- 10. Open CBM Two stage question: Initially: Question (but no MCQ options) Student sets confidence level Next: MCQ options revealed Finally: Marking and feedback as for normal MCQ / CBM
- 11. Demo Standard MCQ Standard CBM Open CBM screen dump
- 13. Open CBM advantages Mechanically, question remains simple MCQ answer matching is trivial easy to implement possible to reuse existing question banks But transformed closed to open question student must formulate answer to decide confidence must decide answer without +ve or –ve clues. cannot work backward will not be led into misconceptions. must answer the question as set .
- 14. MCQ Open CBM Pros objective marking reliable marking easy to implement Cons distractors trivial distractors engender misconceptions working backwards Pros open question reflection ...as MCQ Cons not always applicable intimidating? personality dependent?
- 15. Possible benefits Students better engagement improved learning more reflective learners Staff easier question setting
- 16. eSTEeM project plan (revised…) Implement open CBM question type in OU Moodle Quiz and/or OpenMark Trial in course assessment controlled experimental design pilot (formative) on T216 CISCO networking Survey/interview to probe attitudinal aspects User-lab trials Log measures Assignment scores Time spent on task
- 17. Jon Rosewell [email_address] www.open.ac.uk