Creating (more) effective tests and quizzes.spr.15
Download as PPTX, PDF1 like1,105 views
The document discusses designing effective tests and quizzes. It provides tips for creating different types of assessment items, including supply items, true/false questions, multiple choice, matching and essay prompts. The document emphasizes writing clear, unambiguous questions that accurately measure learning objectives. It also stresses the importance of culturally responsive assessment practices to ensure equitable evaluation of all students.
![Inference Question
Stems….
Which of the following conclusions can be most
properly drawn from the data above?
Which of the following is [implied, must be true,
implicit, most reasonably drawn] in the passage
above?
Which of the following conclusions can most
properly be drawn if the statements above are
true?](https://image.slidesharecdn.com/creatingmoreeffectivetestsandquizzes-150227144519-conversion-gate02/85/Creating-more-effective-tests-and-quizzes-spr-15-52-320.jpg)
Creating (more) effective tests and quizzes.spr.15
- 1. Designing (More) Effective Tests & Quizzes @kristobolsloan @StEds_CTE Thursday, February 12, 3:30-4:45 Fleck 314
- 2. Assumptions Effective assessments can drive, propel even student learning. Effective assessment and evaluation leads to more clear course outcomes. Effective assessment and evaluation can better help a professor determine their own teaching effectiveness
- 3. Assumptions Most importantly: To better ensure greater access to academic success of a diverse group of learners, I must offer my students an array of assessments and evaluations to better understand the fuller range of what my students know and can do vis-à-vis my course outcomes
- 4. Time Management Electronic management of scoring/grading Provides immediate feedback/correct misconceptions Can offer students opportunities for input on test creation
- 5. Socrative Pre-assessments example Using student questions as a source of student- to-student talk and input into test creation.
- 6. (Educational) Literacy The questions that will be graded on the following three criteria: Explicit connection to course content—not reflective of outside/internet sources Specificity (question could only be answered by the term below) Relevance to (contemporary educational context). Provide the correct answer (with citation of where correct answer can be found) Limit of 20 words!
- 7. What’s “wrong” with these ?s 1. A ______ is used in a house to help prevent the ________of the electrical wiring. 2. _____is the largest export product in Brazil? 3. When did the English first arrive in North America? 4. Discuss what you learned about the British parliament and the U.S. congress.
- 8. Instructor-created assessments What sorts of assessments items do you offer on a typical instructor-created tests & quizzes? Which types of items are most common? Which are easiest/most difficult to create? What are some possibilities/limitations of each type of item?
- 9. Instructor-Created Quizzes/Tests Validity/Reliability concerns: What is the difference between reliability and validity?
- 10. Instructor-Created Quizzes/Tests Validity: Does the quiz/test item actually measure what it is intended/supposed to measure relative to the content objectives? Reliability: Does the quiz/test item consistently estimate what the learner knows and can do relative to the content objectives?
- 11. Two general categories of items Supply (“Completion”) items sentence completion fill-ins short answer essay-response items “Objective items” true/false multiple choice matching
- 12. Avoiding Traps…. Taking care not to get what you ask for….
- 13. Supply items What qualities do you believe makes a good supply item, or “fill-in-the-blank?”
- 14. Supply items Tip #1: Use only one blank per item Poor: A ______ is used in a house to help prevent the ________of the electrical wiring.
- 15. Supply Items Tip #1: Use only one blank per item The overheating of electrical wiring in a house may be prevented if _____ are installed.
- 16. Supply Items Tip #2: Blanks should appear either at the end or near the end of an item. Poor: _____is the largest export product in Brazil?
- 17. Supply Items Tip #2: Blanks should appear either at the end or near the end of an item. The largest export product in Brazil is ______?
- 18. Supply Items Tip #3: Use precise language, or language/terms that cannot be construed in multiple and correct ways The largest export product in Brazil is ______?
- 19. Supply Items Tip #3: Use precise language The largest export product in Brazil is ______? By “largest” do you mean is size (e.g., jackfruit, palm trees), biggest revenue producer (e.g. $?), or by “units” (e.g., number of bananas, pounds of coffee)
- 20. Supply Items The highest revenue-producing export of Brazil is ______? Instead of… The largest export product in Brazil is ______?
- 21. Supply Items Tip #4: Include a word or phrase that indicates the category to which the answer must belong. Poor: When did the English first arrive in North America?
- 22. Supply Items Tip #4: Include a word or phrase that indicates the category to which the answer must belong. What year did the English first arrive in North America?
- 23. True/False Items What qualities do you believe makes a good “true/false” question?
- 24. True/False Items Tip #1: Write more false than true items. Researchers have concluded that teachers tend to do the reverse and students tend to mark items they are unsure about as “true”
- 25. True/False Items Tip #2: Avoid words like “most” (which usually indicates a “true” statement) and “never” (which usually suggests a false statement).
- 26. True/False Items Tip #3: Keep wording clear and specific. Avoid broad, general statements such as always, never, all, may seldom, usually, etc Poor: __ Birds eat more than mammals.
- 27. True/False Items Tip #3: Keep wording clear and specific. ____Relative to their body weights, birds eat more than mammals.
- 28. True/False Items Tip #4: Avoid simply taking statements directly from text (even if modified slightly) because it may encourage rote memorization with little understanding.
- 29. True/False Tip #5: Generate binary-choice items in pairs not singly. Because True/False questions need to be mutually exclusive when reversed, check it.
- 30. True/False Items Tip #6: Restrict T/F items to ONE, or a single concept!
- 31. Multiple Choice Items What qualities do you believe makes a good Multiple Choice Item?
- 32. Multiple Choice Which of these is the most valid question? How old is Bob? How many years ago was Bob born? In what year was Bob born?
- 33. Multiple Choice Which of these is the most valid question? How old is Bob? How many years ago was Bob born? In what year was Bob born? Time sensitive questions are tricky because the answers come in many different forms and can change
- 34. Multiple Choice Items Tip #1: Stem should present enough information so that answer choices can be relatively short. Tip #2: Avoid negatives (e.g., not) as well as words like all, some, often, usually, etc If you do use word not in stem, italicize or underline it.
- 35. Multiple Choice Items: Stems and Choices Tip #3: All choices should be feasible Tip #4: Don’t expect students to make narrow distinctions (e.g., “Freezing”: 30’F, 31’F, 32’F, 33’F)
- 36. Multiple Choice Items Tip #5: All choices should be grammatically consistent The largest tree-borne fruit is a: A. Banana B. Apple C. Jackfruit D. Strawberries
- 37. Multiple Choice Items Poor: An example of: A. An animal that is a mollusk is a whale. B. An animal that is a mollusk is a clam. C. An animal that is a mollusk is a crab. D. An animal that is a mollusk is a lobster.
- 38. Multiple Choice Items An example of a mollusk is a: A. Whale B. Clam C. Crab D. lobster
- 39. Multiple Choice Items Tip #6: Do not overuse “none of the above”/ “All of the above” & variations of “A and B” or “C and D, but not A.”
- 40. Multiple Choice Items Strategies to find the elusive“higher- level”* multiple choice question: Analogy based items: Bandura is to social cognitive theory as _____________ is to social constructivist theory: a. Piaget b. Siegler c. Vygotsky d. Skinner *Bloom’s Taxonomy
- 41. Knowledge-Level Items Identify meaning of term: Reliability is the same as: A. consistency. B. relevancy. C. representativeness. D. usefulness.
- 42. Comprehension-Level Item Identifies an example of a concept or principle . Which of the following is an example of a criterion-referenced interpretation? A. Derik earned the highest score in science. B. Erik completed his experiment faster than his classmates. C. Edna’s test score was higher than 50 percent of the class. D. Tricia set up her laboratory equipment in five minutes.
- 43. Application-Level Items Distinguish between properly and improperly stated outcomes Which one of the following learning outcomes is properly stated in terms of student performance? A. Develops an appreciation of the importance of testing. B. Writes valid test questions. C. Realizes the importance of validity.
- 44. Analysis-level Items Analyze whether a word fits with the accepted definition of pragmatism. Directions: Read carefully through the paragraph below, and decide which of the options A-D is correct. “The basic premise of pragmatism is that questions posed by speculative metaphysical propositions can often be answered by determining what the practical consequences of the acceptance of a particular metaphysical proposition are in this life. Practical consequences are taken as the criterion for assessing the relevance of all statements or ideas about truth, norm and hope.”
- 45. A. The word “acceptance” should be replaced by “rejection.” B. The word “often” should be replaced by “only.” C. The word “speculative” should be replaced by hypothetical.” D. The word “criterion” should be replaced by “measure.”
- 46. Synthesis-level Items Directions: Read the following comments a teacher made about testing. Then answer the questions that follow by circling the letter of the best answer. “Students go to school to learn, not to take tests. In addition, tests cannot be used to indicate a student’s absolute level of learning. All tests can do is rank students in order of achievement, and this relative ranking is influenced by guessing, bluffing, and the subjective opinions of the teacher doing the scoring. The teacher-learning process would benefit if we did away with tests and depended on student self-evaluation.”
- 47. Identifies relationships: Which one of the following propositions is most essential to the final conclusion? A. Effective self-evaluation does not require the use of tests. B. Tests place students in rank order only. C. Test scores are influenced by factors other than achievement. D. Students do not go to school to take tests.
- 48. Multiple Choice Items Advantages: Efficient way of testing students’ ability to recall facts Demonstrates understanding Disadvantages Hard to create, time consuming
- 49. Inference Questions Ask students draw conclusions Ask you to find assumptions made in the passage. Assesses your ability to go beyond what is on the page.
- 50. Inference Question Stems…. The author suggests which of the following… It can be inferred from the passage that… The author assumes that…
- 51. Inference Question Stems…. The main point of the passage is that... Which of the following statements about... is best supported by the statements above? Which of the following best states the author's conclusion in the passage above?
- 52. Inference Question Stems…. Which of the following conclusions can be most properly drawn from the data above? Which of the following is [implied, must be true, implicit, most reasonably drawn] in the passage above? Which of the following conclusions can most properly be drawn if the statements above are true?
- 53. Making Inferences/ Drawing Conclusions Which conclusion is best supported by the sentence: “The Golden Gate Bridge spans the San Francisco Bay, connecting northern California to the peninsula of San Francisco.” a. The bridge enables many people to go from SF to northern California b. The bridge has the longest span of any other suspension bridge c. The cables are best d. The bridge is the most popular bridge in the bay area
- 54. Making Inferences/ Drawing Conclusions Joe woke up and could hear the roosters crowing. We can infer the time of the day is__________. a. At noon b. Early in the morning c. In the afternoon d. At night
- 55. Making Inferences/ Drawing Conclusions Which of the following statements would make the most accurate conclusion to the passage a. Bacteria are often an indication that a person’s health has been seriously compromised. b. Both measures would help protect the health of countless people throughout the country. c. Researchers are always looking at ways to provide people with better dental health care. d. Sensors are so incredibly small that they can easily be placed on teeth or even tinier spots.
- 56. Matching 1. ___The Seine river divides this city into two famous banks. 2. ___The capital city of the Island of Taiwan. 3. ___Home city to the Statue of Liberty. 4. ___The Kremlin is located in this city. ……. A. Paris B. Moscow C. Taipei D. New York City E. Hawaii F. Newark ……………….
- 57. Matching Advantages: especially suitable for who, what, when, and where questions. Can ask students to discriminate among, and to apply concepts. efficient use of space, easy to score Disadvantages: Poor for diagnosing strengths and weaknesses
- 58. Matching Tip: Include more responses than premises! This increases the difficulty and decreases the likelihood that correct answers are guesses.
- 59. Essay Items/Prompts What qualities do you believe makes a good essay question, or essay prompt?
- 60. Essay Items/Prompts Tip #1:Clearly indicate a task. Be precise about the information you expect, and how much information you expect Poor: Discuss what you learned about the British parliament and the U.S. congress.
- 61. Essay Items/Prompts Tip #1:Clearly indicate a task. Be precise about the information you expect, and how much information you expect Identify and describe two major differences and one similarity between the BP and U.S. congress. Compare and contrast….??
- 62. Essay Items/Prompts Identify and describe at least three assumptions about B.F. Skinner’s version of behaviorism as it relates to educational psychology. Define Vygotsky’s notion of the Zone of Proximal Development AND provide a clear and convincing example that one might see in a typical classroom. Your example should include specific content or specific examples of a teaching strategy
- 63. Essay Items/Prompts Tip #2: Try to avoid using “why” questions* Poor: If you were a Catholic priest in Texas in the early 1800’s, why would you be Texas? *unless your purpose is to have your students produce a teleological response.
- 64. Essay Items/Prompts Tip #2: Try to avoid using “Why” questions Identify and describe at least three activities a typical Catholic priest would have been engaged in during the early 1800’s in the Spanish Texas.
- 65. Essay Items/Prompts Using Rubrics to direct students toward desired content/quality
- 66. Culturally Response Assessment Literacy Tests are culturally loaded Tests require language
- 67. Culturally Response Assessment Literacy More equitable, culturally responsive assessment practices requires teachers to……
- 68. Culturally Response Assessment Literacy The use of an array of assessments/evaluations Evaluation of students against their own records (vs. comparisons with other students) Inclusion of socioemotional measures as well as measures of academic content (look for both verbal/nonverbal information)
- 69. Culturally Response Assessment Literacy Items should measure only the content of interest, not one’s knowledge of the item context. Write items to measure what students know, not what they do not know. Never couch correct answers around the notion of “intelligence” (See Claude Steele’s notion of “Stereotype Threat”)