Chapter 3 teachback
Download as PPTX, PDF0 likes219 views
1) There are three main types of reliability evidence: stability reliability, which measures consistency over time; alternate-form reliability, which uses equivalent tests; and internal consistency reliability, which ensures all items measure the same construct. 2) Reliability is important for standardized tests to ensure consistency and accuracy of results. It helps determine how reliable assessments and test scores are. 3) The standard error of measurement provides information about consistency for individual students and helps account for variability in scores, reminding teachers not to consider scores as exact measures of understanding.
Chapter 3 teachback
- 1. By: Lauren Chadwick ED 450 Monday September 9, 2012
- 2. Bellwork O Please use a pen or pencil and draw 2 appointment slots Once you are finished on the plate I gave open the folded paper you when you and fill in the blank entered the with the type of word classroom: written on the outside. 1 2 Put paper in box.
- 3. SLO’s O Students will … O Understand reliability in assessment. O Describe the 3 types of reliability evidence. O Explain the importance of standard error of measurement.
- 5. Let’s have some fun! O Everyone get up and stand in a circle!
- 7. Reliability = Consistency O Reliability is an O 2 important ways important criteria reliability may be useful: used when judging O How reliable are standardized tests. your assessments for you and your students? O Parents may ask how reliable standardized tests may be.
- 8. 3 Types of Reliability Evidence O Stability O Alternate Form O Internal Consistency
- 9. Stability Reliability O Consistency of test O Example: Midterm on results over time Tuesday O The goal is to see O A masked man grabs your briefcase close scores in containing the students’ tests! O The next day you tell the students tests administered that their tests were stolen and that even if they are they need to re-take the test! given on different O Because there has been no new occasions. information taught since the last test, you expect that the test scores would be similar. O So, if you were able to see that the scores were not similar, then your exam would be judged to have little stability reliability.
- 10. Test-Retest O To see how stable an assessment’s results are over time, we usually test students, then in a week or two retest with the same form of assessment. O It is important though that there is no significant event taking place in between the two assessments. Example: field trip, movie, speaker- exposing student to test-relevant information.
- 11. So, how are scores computed? Correlation Coefficient (r) So….. O Reflects the degree of O …if the test–retest r similarity between for two sets of test scores on two tests/ scores were .84 this O Ranges from: would show that the +1.0 to -1.0 students’ performance on both tests were similar. (close to 1.0 = strong relationship) (closer to 0 = weak relationship) O If the r was instead .23 this would show that the results were not as similar.
- 12. Classification- Consistency O To decide which students Student Test 1 Test 2 would be exempted from further assessment. Jill 84% 99% O Example: If you decide on Jack 65% 82% an 80% answers correct required for exemption, then on a test- retest approach, the teacher would find the % of students who scored ≥ 80% on both assessments.
- 13. How to determine Classification-Consistency A % of students exempt 42% B % of students not exempt 46% C % of students different classification 12% D % of classification – consistency ( A + B) ----------------------------------------------------------------------- ----------------------- 88% Classification- Consistency of that test
- 14. Alternate-Form Reliability O Simply put: O Usually found in settings where a student is testing to O Providing an receive his/her diploma. equivalent retest O BUT keep in mind when you are reviewing a for students in state-developed test another form other which claims to have than the first and equivalent forms, that failed assessment. those tests assess the same information as the original.
- 15. Internal Consistency Reliability O Reflects the degree O Example: of consistency of the items on a given 20 item test on problem test. solving: O Homogeneous Students who are strong O Each of the test’s problem solvers should get most of the 20 items items should be correct assessing the same variable.
- 16. FYI O Tests containing items O Tests containing items which a students can where students can either be right or wrong be given different such as multiple- grades such as in choice, the most commonly used essay items, the most approaches are the: common approach is: Kuder-Richardson Cronbach’s procedures coefficient alpha
- 17. Just a thought… O The more times you taste from a pot of soup, the more accurate will be your estimate of what the ingredients are. O One taste might fool you, twenty tastes will give you a much better idea of what’s in the pot. O More items on a test will yield more reliable estimates of understanding.
- 18. Choose one… Stability Reliability ≠ Alternate-Form Reliability ≠ Internal Consistency Reliability These three forms of reliability evidence should not be used together. Instead, keep in mind the kind of educational decision linked to the results of the assessment.
- 19. Standard Error of Measurement O Supplies information of test consistency of an individual student by estimating their score consistency. O Problem: Teacher see’s student results from a pervious IQ test. They expect that the student will perform on that exact level O SEM helps to remind teachers that scores earned on commercial or classroom tests are not always exact on student’s understanding. O Another area where SEM is used is in reporting of student performance on state tests.
- 20. 60 item Math test O Jack answers 53 Classificatio # correct n out of the 60 Advanced 54 – 60 questions correctly, Proficient 44 – 53 so what would he Basic 37 – 43 be classified as? Below Basic 36 and below a proficient student.
- 21. Appointment Time! Everyone find a classmate for your 1st appointment. You have 3 minutes to decide whether the following is stability, alternate-form, or internal consistency reliability. Suppose you were trying to show that the items on an educational assessment all measured the same variable. What type of reliability evidence would you use? Internal consistency reliability
- 22. Appointment Time! Everyone find a classmate for your 2nd appointment. You have 3 minutes to decide whether the following is stability, alternate-form, or internal consistency reliability. A new nationally standardized reading test for junior high students has been created, but the test developers want to make sure that the three different forms of tests are performing in essentially the same way. What sort of reliability evidence should they rely on? Alternate-form reliability
- 23. Why do you need to know? O You may be called on O Understanding to explain to parents standard error of the meaning of their measurement will child’s test scores. be helpful when you need to explain to O At least be parents and knowledgeable about students how to the meaning of make sense of their reliability in educational scores on assessments. accountability tests.
- 24. Check your understanding.. O Do you understand assessment reliability? O Do you know the 3 types of reliability evidence? O Can you explain the importance of standard error of measurement?