Stages of test development and common test techniques (1)
Download as PPTX, PDF22 likes15,094 views
This document outlines the key stages in developing language tests, including specifying the problem, writing test specifications, developing test items, validating the test on native and non-native speakers, analyzing results, and training staff. It discusses establishing criteria and scoring procedures. Common test techniques are also presented such as multiple choice, true/false, short answer, and gap filling items. The goal is to provide a framework to help ensure tests are objective, valid, and meet their intended purpose.
Stages of test development and common test techniques (1)
- 1. STAGES OF TEST DEVELOPMENT & COMMON TEST TECHNIQUES DANIEL F. CALI. O.
- 3. OUTLINE STAGES OF TEST DEVELOPMENT SPECIFICATIONS WRITING AND MODERATING ITEMS ITEMS ON NATIVE SPEAKERS VALIDATION CALIBRATION OF SCALES ANALYSIS OF RESULTS ON NON-NATIVE SPEAKERS HANDBOOKS FOR TEST TAKERS TRAINING STAFF STATING THE PROBLEM
- 4. • SHOULD BE A TEST DESIGNED BY A SINGLE PERSON OR A TEAM? TEAM! • IT IS DIFFICULT TO DESIGN A TEST BECAUSE OF… …OBJECTIVITY …CRITICISMS …NATIVE COMMAND OF THE LANGUAGE
- 5. STEPS…
- 7. STATING THE PROBLEM • ONCE THE PROBLEM IS CLEAR, IT IS ALSO IMPORTANT TO: GATHER INFORMATION ON ALREADY EXISITING TEST DESIGNED FOR SIMILAR SITUATIONS!
- 8. WRITING SPECIFICATIONS FOR THE TEST • A) CONTENT • B) STRUCTURE, TIMING, MEDIUM/CHANNEL AND TECHNIQUES • C) CRITERIAL LEVELS OF PERFORMANCE • D) SCORING PROCEDURES
- 9. A) CONTENT • INCLUDE CLEAR SPECIFICATIONS REGARDING: • • • • • • • • • • SKILLS (SUB-SKILLS) TYPE OF TEXTS ADDRESSEES LENGTH OF TEXTS TOPICS (READABILITY) STRUCTURAL RANGE VOCABULARY RANGE DIALECT, ACCENT AND STYLE SPEED OF PROCESSING
- 10. B) STRUCTURE, TIMING, MEDIUM/CHANNE L AND TECHNIQUES • TEST STRUCTURE • NUMBER OF ITEMS • NUMBER OF PASSAGES • TIMING • MEDIUM • KIND OF TEST TECHNIQUE(S)
- 11. C) CRITERIAL LEVELS OF PERFORMANCE • ACCURACY • APPROPRIACY • RANGE • FLEXIBILITY • SIZE
- 12. D) SCORING PROCEDURES • WHAT RATING SCALE WILL BE USED? • HOW MANY PEOPLE WILL RATE EACH PEACE OF WORK? • WHAT HAPPENS IF TWO OR MORE RATERS DISAGREE ABOUT A PIECE OF WORK?
- 13. WRITING AND MODERATING ITEMS • A) SAMPLING • B) WRITING ITEMS • C) MODERATING ITEMS
- 14. A) SAMPLING HOW THE TEXTS ARE GOING TO BE CHOSEN?
- 15. B) WRITING ITEMS • TRY TO LOOK AT THE TEST THROUGH THE EYES OF TEST TAKERS! • AN ITEM WITHOUT A KEY IS INCOMPLETE! • “THE BEST WAY TO IDENTIFY ITEMS THAT HAVE TO BE IMPROVED OR ABANDONED IS THROUGH THE PROCESS OF MODERATION”.
- 16. C) MODERATING ITEMS • INTERVATION OF TWO COLLEAGUES:
- 17. INFORMAL TRIALLING OF ITEMS ON NATIVE SPEAKERS • 20 OR MORE… • THEY SHOULD BE SIMILAR TO THE GROUP BEING TESTED IN TERMS OF: AGE EDUCATION GENERAL BACKGROUND
- 18. TRIALLING OF THE TEST ON A GROUP OF NON-NATIVE SPEAKERS SIMILAR TO THOSE FOR WHOM THE TEST IS INTENDED • PROBLEMS IN ADMINISTRATION AND SCORING CAN BE NOTED.
- 19. ANALYSIS OF RESULTS OF THE TRIAL; MAKING OF ANY NECESSARY CHANGES • STATISTICAL AND QUALITATIVE ANALYSIS. HOW DIFFICULT ARE THE ITEMS? DISCOVER MISINTERPRETATIONS (TO BE MODIFIED OR DROPPED)
- 20. CALIBRATION OF SCALES • IT IS IMPORTANT TO COLLECT SAMPLES OF PERFORMANCE AND ASSIGN EACH OF THEM TO A POINT ON THE RELEVANT SCALE.
- 22. WRITING HANDBOOKS FOR TEST TAKERS, TEST USERS AND STAFF
- 25. OUTLINE DEFINITION MULTIPLE CHOICE GAP FILLING COMMON TEST TECHNIQUES TRUE OR FALSE SHORTANSWERS
- 26. WHAT ARE TEST TECHNIQUES? • MEANS OF ELICITING BEHAVIOR
- 27. MULTIPLE CHOICE ITEMS • STEM • DISTRACTOR
- 28. MULTIPLE CHOICE ITEMS ADVANTAGES DISADVANTAGES DIFFICULT TO WRITE ITEMS GREAT DEMAND ON TIME AND EXPERTISE
- 29. YES/NO AND TRUE/FALSE ITEMS Reliability and validity
- 31. GAP FILLING ITEMS • IT IS A VALUABLE TECHNIQUE AS LONG AS THE CONTEXT IS PROVIDED! • A: WHAT WILL HE DO? • B: I THINK HE ______ RESIGN. BUT: A: I WONDER WHO THAT IS. B: IT ______ BE THE DOCTOR. A: HOW CAN YOU BE SO CERTAIN? WILL, MIGHT, COULD MAY, ETC.
- 32. • Now you know what a teacher should do when preparing tests, what advices would you give? • (Each one should only mention only 1 advice) Names Advices Carolina Think of the applicability to real situations ar real life situations. Paola Put in your students´ shoes. Mauricio Consider multiple intelligences.
- 34. REFERENCES • Huges, Arthur. Testing for Language Teachers. New York: Cambridge UP, 2003.