Designing Test Questions
- 1. Designing Test Questions Descriptions follow with uses, advantages, disadvantages, and tips for writing test questions in the following formats. True/False Good for: • Knowledge level content • Evaluating student understanding of popular misconceptions • Concepts with two logical responses Advantages: • Can test large amounts of content • Students can answer 3-4 questions per minute Disadvantages: • They are easy • It is difficult to discriminate between students that know the material and students who don't • Students have a 50-50 chance of getting the right answer by guessing • Need a large number of items for high reliability Tips for Writing Good True/False items: • Avoid double negatives. • Avoid long/complex sentences. • Use specific determinants with caution: never, only, all, none, always, could, might, can, may, sometimes, generally, some, few. • Use only one central idea in each item. • Don't emphasize the trivial. • Use exact quantitative language • Don't lift items straight from the book. • Make more false than true (60/40). (Students are more likely to answer true.) Matching Good for: • knowledge level • some comprehension level, if appropriately constructed Types:
- 2. • terms with definitions • phrases with other phrases • causes with effects • parts with larger units • problems with solutions Advantages: • Maximum coverage at knowledge level in a minimum amount of space/prep time • Valuable in content areas that have a lot of facts Disadvantages: • Time consuming for students • Not good for higher levels of learning Tips for Writing Good Matching items: • Need 15 items or less. • Give good directions on basis for matching. • Use items in response column more than once (reduces the effects of guessing). • Use homogenous material in each exercise. • Make all responses plausible. • Put all items on a single page. • Put response in some logical order (chronological, alphabetical, etc.). • Responses should be short. Multiple Choice Good for: • application, synthesis, analysis, and evaluation levels Types: • Question/Right answer • Incomplete statement • Best answer Advantages: • Very effective • Versatile at all levels • Minimum of writing for student • Guessing reduced • Can cover broad range of content
- 3. Disadvantages: • Difficult to construct good test items. • Difficult to come up with plausible distracters/alternative responses. Tips for Writing Good Multiple Choice items: • Stem should present single, clearly formulated problem. • Stem should be in simple, understood language; delete extraneous words. • Avoid "all of the above"--can answer based on partial knowledge (if one is incorrect or two are correct, but unsure of the third...). • Avoid "none of the above." • Make all distracters plausible/homogenous. • Don't overlap response alternatives (decreases discrimination between students who know the material and those who don't). • Don't use double negatives. • Present alternatives in logical or numerical order. • Place correct answer at random (An answer is most often). • Make each item independent of others on test. • Way to judge a good stem: student's who know the content should be able to answer before reading the alternatives • List alternatives on separate lines, indent, separate by blank line, and use letters vs. numbers for alternative answers. • Need more than 3 alternatives, 4 is best. Short Answer Good for: • application, synthesis, analysis, and evaluation levels Advantages: • easy to construct • good for "who," what," where," "when" content • minimizes guessing • Encourages more intensive study-student must know the answer vs. recognizing the answer. Disadvantages: • May overemphasize memorization of facts • Take care - questions may have more than one correct answer • Scoring is laborious Tips for Writing Good Short Answer Items: • When using with definitions: supply term, not the definition-for a better judge of student knowledge.
- 4. • For numbers, indicate the degree of precision/units expected. • Use direct questions, not an incomplete statement. • If you do use incomplete statements, don't use more than 2 blanks within an item. • Arrange blanks to make scoring easy. • Try to phrase question so there is only one answer possible. Essay Good for: • application, synthesis and evaluation levels Types: • Extended response: synthesis and evaluation levels; a lot of freedom in answers • Restricted response: more consistent scoring, outlines parameters of responses Advantages: • Students less likely to guess • Easy to construct • Stimulates more study • Allows students to demonstrate ability to organize knowledge, express opinions, show originality. Disadvantages: • Can limit amount of material tested, therefore has decreased validity. • Subjective, potentially unreliable scoring. • Time consuming to score. Tips for Writing Good Essay Items: • Provide reasonable time limits for thinking and writing. • Avoid letting them to answer a choice of questions (You won't get a good idea of the broadness of student achievement when they only answer a set of questions.) • Give definitive task to student-compare, analyze, evaluate, etc. • Use checklist point system to score with a model answer: write outline, determine how many points to assign to each part • Score one question at a time-all at the same time. Oral Exams
- 5. Good for: • knowledge, synthesis, evaluation levels Advantages: • Useful as an instructional tool-allows students to learn at the same time as testing. • Allows teacher to give clues to facilitate learning. • Useful to test speech and foreign language competencies. Disadvantages: • Time consuming to give and take. • Could have poor student performance because they haven't had much practice with it. • Provides no written record without checklists. Student Portfolios Good for: • knowledge, application, synthesis, evaluation levels Advantages: • Can assess compatible skills: writing, documentation, critical thinking, problem solving • Can allow student to present totality of learning. • Students become active participants in the evaluation process. Disadvantages: • Can be difficult and time consuming to grade. Performance Good for: • application of knowledge, skills, abilities Advantages: • measures some skills and abilities not possible to measure in other ways Disadvantages:
- 6. • Can not be used in some fields of study • Difficult to construct • Difficult to grade • Time-consuming to give and take