vertical Moderate Standard setting
- 1. 1 Presented By: Tariq Mehmood Presented To : Dr Muhammad Iqbal University of Education Lahore
- 2. After completion of this unit Students will be able to : Define vertically Moderated Standard setting. Describe the history of VMSS. Elaborate Approaches of VMSS. Describe Application of VMSS.. 2
- 3. vertically-moderated standard setting(VMSS) sometimes referred to as a process of vertical articulation of standards—is a new and developing concept. 3
- 4. Simply VMSS is a procedure or set of procedures, typically carried out after individual standards have been set. Reasonable expectations are typically stated in terms of percentages of students at or above a important performance level, such as Proficient. To illustrate VMSS, consider the following scenario for a typical state testing program. Six groups of standard setters have gathered to set standards for the state mathematics tests in Grades 3–8. 4
- 5. Table 14-1 Hypothetical Results of Independent Standard Settings Across Six Grade Levels 5 Grade Percentage of Students Classified as At or Above Proficient Performance Level 3 37 4 41 5 34 6 43 7 29 8 42
- 6. Given the placement of the cut scores for the Proficient category, percentages of students that would be classified as Proficient or above are shown, by grade level, in Table 14-1. In this example, we see that the percentages of students considered Proficient or better (e.g., Proficient or Advanced), goes up from Grade 3 to Grade 4, drops back down at Grade 5, goes up again at Grade 6, drops again at Grade 7, and rises again at Grade 8. 6
- 7. Each of the methods of setting performance standards we described is routinely applied to contexts in which the need is for a single cut score or set of performance levels for a single test. These contexts have included setting a single cut score on a licensing examination, deriving three performance levels (e.g., Basic, Proficient, Advanced) on a single end-of-grade subject area test, and obtaining multiple cut scores on a set of four or five different subject tests used to determine eligibility for high school graduation. In each instance, there was no apparent need to link cut scores on one test to those on another. 7
- 8. The challenges to creating meaningful systems of performance standards across grades and subject areas. One approach has been proposed by Lissitz and Huynh (2003), who introduced the concept of VMSS in a background paper prepared for the Arkansas Department of Education. The paper, titled “Vertical Equating for the Arkansas ACTAAP Assessments: Issues and Solutions in Determination of Adequate Yearly Progress and School Accountability,”. 8
- 9. As an alternative to vertical scaling (equating), Lissitz and Huynh (2003b) recommended VMSS. Specifically, they recommended that “new cut scores for each test be set for all grades such that each achievement level has the same (generic) meaning across all grades, and the proportion of students in each achievement level follow a growth curve trend across these grades.” They then offered a list of specific recommendations to carry out the vertical moderation of a set of performance standards across Grades 3– 8, including an annual validation of the process. 9
- 10. It remains to be seen if VMSS will provide satisfactory solutions to the current contexts to which it is being applied, as well as how adaptable VMSS will be to other contexts. 10
- 11. Simply VMSS is a procedure or set of procedures, typically carried out after individual standards have been set. Reasonable expectations are typically stated in terms of percentages of students at or above a important performance level, such as Proficient. To illustrate VMSS, consider the following scenario for a typical state testing program. Six groups of standard setters have gathered to set standards for the state mathematics tests in Grades 3–8. 11
- 12. VMSS typically focuses on percentages of students at various proficiency levels. A simple solution to the problem of different percentages of students reaching a given performance level—say, the Proficient cut score— at different grades would simply be to set all standards at the same score point or such that equal percentages of students would be classified as Proficient at each grade level, by command. 12
- 13. An alternative would be to set standards only for the lowest and highest grades and then align the percentages of Proficient students in the intermediate grades accordingly. In the preceding example, we would simply take the 37% figure for Grade 3 and the 42% figure for Grade 8 and set cut scores for Grades 4–7 so that their resulting percentages of students at or above Proficient would fall on a straight line between 37% and 42% as shown in Table 14-2 13
- 14. . If complete standard-setting procedures were used for establishing cut scores for Grades 3 and 8, then there is some reason to have confidence in at least the 37% and 42% values for the anchor grades. 14
- 15. Table 14-2 Results of Smoothing Standard- Setting Results Across Six Grade Levels 15 Grade Percentage of Students Classified as At or Above Proficient Performance Level 3 37 4 38 5 39 6 40 7 41 8 42
- 16. According to these authors, assumptions for standard setting are based on the intersection of three growth models and four expected growth amounts as depicted in Table 14-3. The three growth models may be summarized as follows: (1) Linear growth, which assumes that the proficiency of all examinees increases by a fixed amount, and examinees retain their positions relative to one another; 16
- 17. (2) Remediation, which assumes that the proficiency of examinees at the lower end of the score distribution increases more than those of examinees at the upper end; and (3) Acceleration, which assumes that the proficiency of examinees in the upper portion of the score distribution increases at a greater rate than that of examinees at the lower end of the score distribution. 17
- 18. 18
- 19. Because of the newness of VMSS, only recently have results of VMSS approaches been presented in the psychometric literature. As we noted 19 Growth Trajectori es Growth Model Types Linear Remediation Acceleration Negative growth All groups show decline over time. Overall group shows negative growth, but remedial group fares better than non remedial group. Overall group shows negative growth, but non remedial group fares better than remedial group.
- 20. 20 Growth Trajectori es Growth Model Types Linear Remediation Acceleration No growth All groups show no growth over time. Overall group shows no growth, but remedial group fares better than non remedial group. Overall group shows no growth, but non remedial group fares better than remedial group.
- 21. 21 Growth Trajectori es Growth Model Types Linear Remediation Acceleration Low growth All groups show low growth over time. Overall group shows low growth, with most gain coming from remedial group. Overall group shows low growth, with most gain coming from non remedial group.
- 22. 22 Growth Trajectorie s Growth Model Types Linear Remediation Acceleration Moderate growth All groups show moderate growth over time. Overall group shows moderate growth, with most gain coming from remedial group. Overall group shows moderate growth, with most gain coming from nonremedial group.