1.2 principles of test design: plenary CTS-Academic
1 like1,663 views
The document discusses the principles of test design, focusing on assessment objectives, measurement qualities, and challenges in testing. It highlights the importance of defining test objectives, ensuring reliability and validity, and understanding the purposes of different types of tests in educational contexts. It also addresses misconceptions and outside factors affecting testing effectiveness.
1.2 principles of test design: plenary CTS-Academic
- 1. Principles of test design Objectives, measurement qualities, challenges KAYSERI, 29-31 JANUARY 2014
- 3. Aims ! Definining our assessment objectives ! Measurement qualities: ‚test usefulness ! Testing challenges ! Considering impact
- 4. Testing: a definition ! “the systematic gathering of information for the purpose of making decisions” (Weiss 1972) ! “the collection of reliable and relevant information” (Bachman 1990) ! What information? ! What decisions?
- 5. Defining our test objectives 1) What kind of test is it going to be? 2) What decisions do we need information for? 3) What abilities are tested? 4) How much detail do we need? 5) How accurate should our results be? 6) Is there any washback effect? 7) What are the practical constraints?
- 6. Assessment in the classroom ! examinations ! formal ! oral tests tests ! home ! term assignments reports ! continuous assessment ! projectwork, ! student research work self-evaluation
- 7. Purposes of tests ! for placement or level testing: to decide on the level of learners ! for diagnostic testing: to identify individual strengths or weaknesses ! to evaluate progress ! to evaluate skills for specific needs ! to give marks for performance ! to evaluate and update syllabus and objectives ! also to prepare/train for exams
- 8. Types of tests (Madsen, 1983) Contrasting categories of ESL tests Knowledge Performance (or skills) Subjective Objective Productive Receptive Language sub-skills Norm-referenced Discrete-point Proficiency Communication skills Criterion-referenced Integrative Achievement (or progress)
- 9. Key principles of testing " A correspondence between language test performance and language use ◦ " “In order for a particular language test to be useful for its intended purposes, test performance must correspond in demonstrable ways to language use in non-test situations.” A clear and explicit definition of qualities of test usefulness ◦ “Usefulness = Reliability + Construct validity + Authenticity + Interactiveness + Impact + Practicality” (Bachman & Palmer, 1996)
- 10. Qualities of test usefulness: reliability " consistent across populations " consistent within the same test irrespective of: ! setting ! marker ! item set When conditions of the test remain unchanged, it always produces essentially the same results.
- 11. Qualities of test usefulness: validity ! Types of validity (Alderson et al, 1995) ! internal ! face validity validity ! content validity ! response ! external validity validity ! concurrent ! predictive ! construct ! validity validity validity meaningful? appropriate? representative?
- 12. Construct validity CHECKLIST FOR VALID TESTS Tests what it claims to test? Best format for what you want to test? Content relevant to student’s real-life needs? Items typical/representative of learner use? Items don’t accidentally test other things? Test skill, NOT knowledge, creativity or logic? Clear and appropriate marking criteria?
- 13. Testing competence from Language Testing in Practice by Bachman & Palmer (1996, OUP) Misconceptions ! one best test for a given situation ! misunderstanding the nature of testing and test development ! unreasonable expectations ! blind faith in measurement Resulting problems ! inappropriate tests ! failure to meet specific needs ! uninformed use of popular testing methods ! frustration about not finding the perfect test ! loss of faith in self ! testing can only be done by experts ! defending the indefensible (stakeholders s expectations)
- 14. Challenges in testing " " " " " " testing = “real life”? “authentic”? “communicative language testing”? defining abilities in a test selecting tasks and items ! competence? measurement principles: can “unidimensional scores for locally independent test items reflect authentic language use”? (Bachman) outside factors ! performance?