Principles of Lang Assessment_Recently RvsdRe.ppt
- 2. What Are the Principles of Language Assessment? 1) Practicality 2) Reliability 3) Validity 4) Authenticity 5) Washback
- 3. 1) Practicality An effective test is practical. This means that it … • is not excessively expensive. • stays within appropriate time constraints. • is relatively easy to administer. • has a scoring/evaluation procedure that is specific and time-efficient.
- 4. 2) Reliability A reliable test is consistent and dependable. If you give the same test to the same students on two different occasions, the test should yield similar results. 2a) Student-related reliability 2b) Rater reliability 2c) Test administration reliability 2d) Test reliability
- 5. 2a) Student-Related Reliability • The most common issue in student-related reliability is caused by temporary illness, fatigue, a bad day, anxiety, and other physical and psychological factors that may make an “observed” score deviate from a “true” score.
- 6. 2b) Rater Reliability • Human error, subjectivity, and bias may enter into the scoring process. • Inter-rater reliability occurs when two or more scorers yield inconsistent scores on the same test, possibly for lack of attention to scoring criteria, inexperience, inattention, or even preconceived bias toward a particular “good” and “bad” student.
- 7. 2c) Test Administration Reliability • Test administration reliability deals with the conditions in which the test is administered. • Street noise outside the building • bad equipment • room temperature • the conditions of chairs and tables, photocopying variation
- 8. 2d) Test Reliability A test is said to lack test reliability when it … • is too long. • is poorly written or contains ambiguous test items.
- 9. 3) Validity A test is valid if it assesses the objectives and what has been taught. 3a) Content validity 3b) Criterion validity (test objectives) 3c) Construct validity 3d) Consequential validity 3d) Face validity
- 10. 3a) Content Validity • A test is valid if the teacher can clearly define the achievement that he or she is measuring • A test of tennis competency that asks someone to run a 100-yard dash lacks content validity • If a teacher uses the communicative approach to teach speaking and then uses the audiolingual method to design test items, it is going to lack content validity
- 11. 3b) Criterion-Related Validity • The extent to which the objectives of the test have been measured or assessed. For instance, if you are assessing reading skills such as scanning and skimming information, how are the exercises designed to test these objectives? • In other words, the test is valid if the objectives taught are the objectives tested and the items are actually testing these objectives.
- 12. 3c) Construct Validity • A construct is an explanation or theory that attempts to explain observed phenomena. • If you are testing vocabulary and the lexical objective is to use the lexical items for communication, writing the definitions of the test will not match with the construct of communicative language use.
- 13. 3d) Consequential Validity A test is said to have consequential validity if it has … • accuracy in measuring intended criteria. • its impact on the preparation of test-takers. • its effect on the learner. • social consequences of a test interpretation (exit exam for pre-basic students at El Colegio, the College Board).
- 14. 3e) Face Validity Face validity refers to the degree to which a test looks right and appears to measure the knowledge or ability it claims to measure. • It is well-constructed and has the expected format with familiar tasks. • It is clearly doable within the allotted time limit. • The directions are crystal clear. • The tasks are related to the course (content validity). • The difficulty level of the test presents a reasonable challenge.
- 15. 4) Authenticity A test is said to have authenticity when … • the language in the test is as natural as possible. • its items are contextualized rather than isolated. • its topics are relevant and meaningful for learners. • some thematic organization of items is provided. • the tasks represent or are closely approximate to real- world tasks.
- 16. 5) Washback • Washback refers to the effects the tests have on instruction in terms of how students prepare for the test “Cram” courses and “teaching to the test” are examples of such washback. • In some cases, the students may learn when working on a test or assessment. • Washback can be both positive and negative.
- 18. Alternative Assessment Options Self- and Peer-Assessments • Oral production: student self-checklist, peer checklist, offering and receiving a holistic rating of an oral presentation • Listening comprehension: listening to TV or radio broadcasts and checking comprehension with a partner • Writing: revising work on your own, peer-editing • Reading: reading textbook passages followed by self-check comprehension questions, self-assessment of reading habits (page 416, Brown, 2001)
- 19. Authentic Assessment • Performance assessment: any form of assessment in which the student constructs a response orally or in writing. It requires the learner to accomplish a complex and significant task while bringing to bear prior knowledge, recent learning, and relevant skills to solve realistic or authentic problems (O’Malley & Valdez, 1996; Herman, et. al., 1992).
- 20. Examples of Authentic Assessment • Portfolio assessment • Student self-assessment • Peer assessment • Student-teacher conferences • Oral interviews • Writing samples • Projects or exhibitions • Experiments or demonstrations
- 21. Characteristics of Performance Assessment • Constructed response • Higher-order thinking • Authenticity • Integrative • Process and product • Depth versus breadth
- 22. Journals • Specify to students the purpose of the journal. • Give clear directions to students on how to get started (prompts for instance “I was very happy when…) • Give guidelines on the length of each entry. • Be clear on the principal purpose of the journal. • Help students to process your feedback and show them how to respond to your responses.
- 23. Conferences • Commonly used when teaching writing • One-on-one interaction between teacher and student • Conferences are formative assessment as opposed to offering a final grade or a summative assessment. In other words, they are meant to provide guidance and feedback.
- 24. Portfolios • Commonly used with the communicative language teaching approach (CLT) • It is a collection of students’ work that demonstrates to students and others the efforts, progress, and achievements in a given area. You can have a reading portfolio or a writing portfolio, for instance • You can also have a reflective or assessment portfolio as opposed to collecting every piece of evidence for each objective achieved in the course
- 25. Portfolio Guidelines • Specify the purpose of the portfolio. • Give clear directions to students on how to get started. • Give guidelines for acceptable materials or artifacts. • Collect portfolios on pre-announced dates and return them promptly. • Help students to process your feedback. • Establish a rubric to evaluate the portfolio and discuss it with your students.
- 26. Cooperative Test Construction • Cooperative test construction involves the students’ contribution to the design of test items. It is based on the concept of collaborative and cooperative learning in which students are involved in the process (Brown, 2001, p. 420)