Objective tests
- 1. Objective Tests Description Objective tests measure the learners’ ability to remember facts and figures as well as their comprehension of course materials. Common variations include multiple-choice (MCQ), true-false, and matching items. A typical MCQ test (Example 1) measures only the acquisition of factual or declarative knowledge. It is possible, but very difficult, to write objective questions (example 2) to measure higher order thinking. Example 1 Typical MCQ Which e-Business model best describes an online auction company like eBay? (A) B2B (B) B2C (C) C2C (D) Portal (E) Infrastructure Example 2 High Order Thinking MCQ After installing Windows 2000 Server on a new computer, you try to boot the computer but it fails. You receive an error message stating that the Boot.ini file is missing. The computer’s system partition has been formatted with NTFS. Which of the following describes the simplest solution to the problem in this scenario? You should (A) Use the latest System State backup to restore the missing file. (B) Boot the computer using Safe mode and then copy the missing file from a Windows 2000 CD. (C) Use the ERD to boot the computer and then restore the missing file to the Windows 2000 Server computer. (D) Use the Recovery Console to copy the missing file from the Windows 2000 CD. (Source: http://www.microsoft-cisco-certification.com) What Outcomes are Assessed? • Example 1 type questions can be used to assess students’ ability to recall, relate, or explain some factual knowledge which, as explained in Chapter 2, are lower-level learning outcomes. • Example 2 type questions can be used to assess simple knowledge application, which means being able to use factual knowledge and information to interpret data, diagnose and solve well-defined problems. Only lower end problem-solving skills can be assessed by MCQs. Real world problems are often too complex to describe in a MCQ. Often, they have no one right answer, whereas the fundamental design of MCQs aims at soliciting a definite right answer. How Authentic is the Task? • Although some professionals are required to recall facts and data, MCQs cannot simulate this kind of recall situation adequately. For example, an operating theatre nurse will have to memorise the names of hundreds of surgical instruments and to be able to retrieve the correct item in a split-second with dead accuracy. However, the response is not prompted by a question of choice. The nurse has to observe the operation, anticipate what is needed, and respond to the
- 2. surgeon’s command instantly. Furthermore, getting the right instrument into the surgeon’s hand involves correct eye-hand-brain coordination. A MCQ assesses only the correct recall of terms. • Example 2 is obviously more authentic than Example 1. Customer support engineers, and even computer users, face this kind of question on a regular basis. However, constructing this kind of MCQ requires an in-depth understanding of performance measurement theories – something best left to professional test makers. What Kind of Learning is Promoted? • When we over-use MCQs in quizzes and examinations for grading purposes, we send a message to students that learning is always about getting the right answers, rather than raising the right questions. Hence, we inadvertently perpetuate a culture of memorisation and rote learning. • Typical MCQs assess knowledge bit by bit, item by item, with little or no reference to any real world application. Such knowledge is quickly and easily forgotten after the exam. • External licensing and certification exams often involve a high proportion of objective questions. If one of your intended outcomes is successful professional entry, students will need drills in writing this type of exam. However, you cannot let these exams dictate or control your curriculum. In other words, your curriculum is concerned with something much more than performing well in any MCQs, it is about becoming an excellent person on the job and in life. Hence, academics should influence the professional bodies to change their assessment practices instead. • Even when MCQs are written to assess higher order learning, as in Example 2, they still encourage rote learning. Since these questions are very expensive to create, they are banked and re- used many times in professional exams. The constant leakage reduces their overall reliability. The typical preparation for this type of exam is memorising answers from hundreds upon hundreds of past exam questions. • Objective questions, like those in the examples, are best used as a tool to diagnose learning problems rather than a grading instrument. You can store banks of objective questions on the computer where students can check their mastery of basic subject knowledge on a regular basis. Good courseware can pinpoint concepts that a student has difficulty grasping and can provide remedial instruction on the spot. For example, an online diagnostic quiz after a lecture on ‘Internet Security’ may reveal that a number of students are still confusing the use of ‘Public Key’ with ‘Private Key’ in data encryption. Base on this information, the courseware can launch an e-learning module on ‘Public Key Infrastructure’ as many times as it takes for the student to grasp the basics. On the other hand, the instructor can take this information and revise his or her lesson plan for the next class.