Test Design, construction and administration
- 1. TEST DESIGN, CONSTRUCTION AND ADMINISTRATION Presented by: • Noureddine Abada • Hicham El Mhali Training year: 24/25 CRMEF SOUSS MASSA • Module: Testing & Assessment • Trainer: Prof Ayad Chraa
- 2. Table Of Content Defining the Test Purpose and Scope Designing the Test Structure and Items Administering, Scoring, and Analyzing the Test Quiz Test Blueprint/ Item/Instructions/Timing Test's Purpose/ Target/Language Skills/ Content Coverage Plan Test - Scoring - Analyze Test Results
- 3. • This initial phase is crucial for establishing the foundation of the test. It answers the fundamental "why" and "what" of the assessment. 1.Defining the Test Purpose and Scope Why is it important? Brown, H. Douglas. (2004). Language Assessment: Principles and Classroom Practices. Pearson Education. Read More >> Brown (2004): "Perhaps the most important question to ask in beginning to design a test is 'What is the purpose of the test?'" (p. 41)
- 4. Determine the Test Purpose: 1.Why is this test being created? (e.g., Placement, Achievement, Diagnostic, Proficiency, Selection) 2.What specific decision or action will be taken based on the results?
- 6. Define the Language Skills and Components to be Tested 1.Which of the four skills (Reading, Writing, Listening, Speaking) are included? 2.Which language components are relevant? (Grammar, Vocabulary, Pronunciation, Discourse Markers, etc.) Alderson, J. Charles, Clapham, Caroline, & Wall, Dianne. (1995). Language Test Construction and Evaluation. Cambridge University Press. Read More >> Alderson, Clapham, & Wall (1995): "Before we even start to think about the items that will be on the test, we need to know what skills or knowledge we want to test, and for what purpose." (p. 10)
- 7. • What specific linguistic structures, vocabulary sets, topics, or functions from a curriculum or domain should be included? • What kind of texts or situations will be used? Specify the Content Coverage: Bachman & Palmer (1996): "Once we have determined the purpose(s) of a test, we must consider the context in which it will be used... This involves describing the characteristics of the test takers and the specific uses that will be made of the information from the test." (p. 30) Read More >> Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford University Press.
- 8. Outline the Learning Objectives Being Assessed 50% 60% What specific abilities or knowledge points should successful test-takers demonstrate? (Align with curriculum goals or external standards like CEFR).
- 9. • How many sections will the test have? • What skills/components will each section cover? • What is the order of the sections? • How much weight (points or time) will be assigned to each section or skill? • Determine the overall length of the test. Outline the Learning Objectives Being Assessed:
- 10. This phase focuses on the blueprint of the test and the creation of the actual test questions and tasks. 2. Designing the Test Structure and Items Read More >> Brown, H. Douglas. (2004). Language Assessment: Principles and Classroom Practices. Pearson Education. Brown (2004): "A test blueprint is a detailed plan for the contents of a test, specifying the skills, topics, and item types to be included, and the proportion of the test devoted to each." (p. 45)
- 11. Develop the Test Blueprint/Structure • How many sections will the test have? What skills/components will each section cover? • What is the order of the sections? • How much weight (points or time) will be assigned to each section or skill? • Determine the overall length of the test. Multiple Choice & Matching True/False/Gap Fill&Short Answer Essay, Role-play, Interview
- 12. Develop the Test Blueprint/Structure Choose question formats (e.g.,Multiple Choice , True/False, Matching, Gap Fill, Short Answer, Essay, Role-play, Interview) that are suitable for the skills/components being tested and the target population. Multiple Choice & Matching True/False/Gap Fill&Short Answer Essay, Role-play, Interview
- 13. Write and Refine Test Items/Tasks • Draft clear, unambiguous questions and tasks. • Ensure that items accurately measure the intended skill/objective. • Develop authentic and relevant stimuli (texts, audio recordings, images). Draft clear, unambiguous questions and tasks. Ensure that items accurately measure the intended skill/objective. Develop authentic and relevant stimuli (texts, audio recordings, images).
- 14. • Provide explicit and easy to understand instructions for each section and item type. • Include examples if necessary. Write Clear Instructions: Read More >> Alderson, J. Charles, Clapham, Caroline, & Wall, Dianne. (1995). Language Test Construction and Evaluation. Cambridge University Hughes (2003): "Writing good test items is perhaps the most crucial and demanding part of language test construction." (p. 41)
- 15. Allocate appropriate time for each section and the overall test, considering item difficulty and the nature of the task (e.g., reading comprehension versus writing an essay) Determine Timing:
- 16. Choosing Item Types Objective Items: • Multiple Choice Questions (MCQs): Easy to score; tests recognition. • True/False Items: Suitable for factual knowledge but often too simplistic. Subjective Items: • Short Answer Questions: Require recall, not just recognition. • Essay Questions: Assess ability to organize and express ideas. • Performance Tasks: Speaking tasks, presentations, role- plays. Each type has advantages and limitations. A good test often combines several types to assess multiple skills.
- 17. Using a Test Blueprint (Table of Specifications) Objective Content Area Item Type Number of Items Weight (%) Reading Comprehension Environmental Issues MCQ 10 30% Writing Skills Opinion Essay Essay Question 1 30% Grammar Tense Usage Gap-fill 5 20% Vocabulary Word Formation Matching 5 20% Benefits of a blueprint: • Prevents overemphasis on certain topics or skills • Increases fairness and transparency • Facilitates analysis after the test
- 18. Test Administration Pre-Administration: • Preparation: Ensure all materials are ready (papers, audio recordings, technology). Confirm physical conditions: proper seating, lighting, and minimal distractions. • Informing Students: Explain the format, length, and rules of the test in advance. Discuss how the test will be graded.
- 19. Administration Day: Starting the Test: Give clear instructions verbally and/or in writing. Clarify how students should fill in answer sheets, especially for MCQs. Monitoring: Supervise students discreetly. Be attentive to signs of confusion or cheating. Handle technical issues (in case of online or listening tests) professionally. Managing Timing: Give time warnings (e.g., "15 minutes left").
- 20. Post-Administration: Scoring and Grading: Use clear, consistent rubrics especially for subjective items. Double-check results for accuracy. Providing Feedback: Return tests promptly. Highlight strengths and explain errors. Allow students to review their performance and ask questions.
- 21. Post-Administration: Analyzing Test Results: Was there an unusually high failure rate? (May indicate poor question design.) Were some questions answered incorrectly by most students? (May suggest ambiguity.) Test Reflection: Identify areas to improve in future tests.
- 22. 2. Example of a rubric
- 23. References • Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge, University Press. • Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press. • Brown, H. D. (2004). Language assessment: Principles and classroom practices. Pearson Education. • Hughes, A. (2003). Testing for language teachers. Cambridge University Press.