Sc linking study august2010 revised final
- 1. SOUTH CAROLINA LINKING STUDY A Study of the Alignment of the NWEA RIT Scale with South Carolina’s Palmetto Assessment of State Standards (PASS) and with the High School Assessment Program (HSAP) August 2010 Revised April 2011 The Kingsbury Center at Northwest Evaluation Association
- 2. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 2 | P a g e COPYRIGHT © 2010 NORTHWEST EVALUATION ASSOCIATION All rights reserved. No part of this document may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without written permission from NWEA.
- 3. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 3 | P a g e A STUDY OF THE ALIGNMENT OF THE NWEA RIT SCALE WITH SOUTH CAROLINA’S PALMETTO ASSESSMENT OF STATE STANDARDS (PASS) AND HIGH SCHOOL ASSESSMENT PROGRAM (HSAP) KINGSBURY CENTER AT NWEA AUGUST 2010 In March 2010, NWEA completed a project to connect the scale of Palmetto Assessment of State Standards (PASS) used for South Carolina mathematics and reading assessments with NWEA’s RIT scale. Information from the PASS assessments was used in a study to establish performance-level scores on the RIT scale that would indicate a good chance of success on these tests. In August 2010, this study was updated to include the relationship between the NWEA RIT scale and South Carolina’s High School Assessment Program (HSAP) scale. This report is the combined results for both South Carolina assessments. To perform the PASS analysis for grades 3-8, we linked together state test and NWEA test results for a sample of 37,000 South Carolina students from 126 schools who completed both exams in the spring of 2009. An equipercentile method was used to estimate the RIT score equivalent to each state performance level by determining the percentage of the population within the selected study group that performed at each level on the state test, and finding the equivalent percentile ranges within the NWEA dataset to estimate the cut scores. For example, if 40% of the study group population in grade 3 mathematics performed below the proficient level on the state test, we would find the RIT score that would be equivalent to the 40th percentile for the study population (this would not be the same as the 40th percentile in the NWEA norms). This RIT score would be the estimated point on the NWEA RIT scale that would be equivalent to the minimum score for proficiency on the state test. 2nd grade results were extrapolated using the distribution of scores from the 3rd grade students. For instance, if 40% of the study group population in grade 3 mathematics performed below the proficient level on the state test, we would find the RIT score that would be equivalent to the 40th percentile for the 2nd grade study population. The original PASS Linking Study published in March 2010 also extrapolated 9th and 10th grade scores based on 8th grade distributions. However, now that HSAP data is available, those extrapolations are not included. The methodology for the HSAP analysis for grades 9-12 was the approximately same as the methodology for PASS. However, because South Carolina students can take the HSAP test during any high school year, all 3266 student records from 34 schools in either the spring of 2009 or the spring of 2008 were aligned to a single grade level which was repeated for 9th through 12th grades. The majority of students took the test in the 10th grade. More complete documentation about our linking study methodology can be found on our website.
- 4. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 4 | P a g e In the following pages, Tables 1 through 4 show the best estimate of the minimum RIT equivalent to each PASS or HSAP performance level for same-season (spring) and prior-season (fall) RIT scores. These tables can be used to identify students who may need additional help to perform well on these tests. Tables 5 through 8 show the estimated probability of achieving “Meets Standard” or better on PASS/HSAP, based on that student’s RIT score on MAP. These tables can be used to assist in identifying students who are not likely to pass these assessments, thereby increasing the probability that intervention strategies will be planned and implemented. These tables can also be useful for identifying target RIT-score objectives likely to correspond to successful performance on PASS. Table 9 shows the correlation coefficients between MAP and PASS, and for MAP and HSAP, for reading and mathematics at each of the grades 3 through 8 and high school. These statistics show the degree to which MAP and PASS, and MAP and HSAP, are linearly related. Values at or near 1.0 suggest a perfect linear relationship, and values near 0 indicate no linear relationship. Table 10 shows the percentages of students at each grade and within each subject whose status on PASS/HSAP (i.e., whether or not the student “met standards”) was accurately predicted by their MAP performance and using the estimated cut scores within the current study. This table can be used to understand the predictive validity of MAP with respect to PASS and HSAP. NOTE: This study was revised in April 2011 to correct an error on page 10—the table showing the probability of a student passing the state Reading test based on Spring MAP scores was offset by one row. The other tables, as well as the information for all tables in the NWEA reporting system, are correct and have not been changed. In addition to this correction, all of the probability tables (pages 9-12) have been revised to conform to current practices of showing the minimum probability as 1% (instead of 0%) and the maximum probability as 99% (instead of 100%).
- 5. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 5 | P a g e TABLE 1 – MINIMUM ESTIMATED SAME-SEASON (SPRING) RIT CUT SCORES CORRESPONDING TO SOUTH CAROLINA PERFORMANCE LEVELS – MATHEMATICS MATH-Current Season Cut Scores and Percentiles for each State Performance Level Grade Level 1 Level 2 Level 3 Level 4 Cut Score Cut Score Percen- tile Cut Score Percen- tile Cut Score Percen- tile 2 <186 186 35 197 69 3 <198 198 35 208 69 4 <203 203 27 218 69 5 <212 212 32 231 78 6 <218 218 34 235 75 7 <223 223 36 241 77 8 <231 231 43 247 79 High <223 223 21 238 47 250 74 * Note: the cut scores shown in this table are the minimum estimated scores. Meeting the minimum MAP cut score corresponds to a 50% probability of achieving that performance level. Use the probabilities in Tables 5-8 to determine the appropriate “target” scores for a desired level of certainty. Note: bolded, italicized text denotes extrapolated cut score
- 6. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 6 | P a g e TABLE 2 – MINIMUM ESTIMATED SAME-SEASON (SPRING) RIT CUT SCORES CORRESPONDING TO SOUTH CAROLINA PERFORMANCE LEVELS – READING READING-Current Season Cut Scores and Percentiles for each State Performance Level Grade Level 1 Level 2 Level 3 Level 4 Cut Score Cut Score Percen- tile Cut Score Percen- tile Cut Score Percen- tile 2 <180 180 23 192 54 3 <189 189 23 201 54 4 <198 198 26 212 67 5 <200 200 19 217 66 6 <209 209 30 222 69 7 <212 212 30 226 70 8 <216 216 32 230 72 High <208 208 12 225 44 236 74 * Note: the cut scores shown in this table are the minimum estimated scores. Meeting the minimum MAP cut score corresponds to a 50% probability of achieving that performance level. Use the probabilities in Tables 5-8 to determine the appropriate “target” scores for a desired level of certainty. Note: bolded, italicized text denotes extrapolated cut score
- 7. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 7 | P a g e TABLE 3 – MINIMUM ESTIMATED PRIOR-SEASON (FALL) RIT CUT SCORES CORRESPONDING TO SOUTH CAROLINA PERFORMANCE LEVELS – MATHEMATICS MATH-Prior Season Cut Scores and Percentiles for each State Performance Level Grade Level 1 Level 2 Level 3 Level 4 Cut Score Cut Score Percen- tile Cut Score Percen- tile Cut Score Percen- tile 2 <175 175 35 184 69 3 <188 188 35 198 69 4 <196 196 27 209 69 5 <206 206 32 222 78 6 <213 213 34 228 75 7 <219 219 36 236 77 8 <227 227 43 243 79 High <221 221 20 235 46 247 73 * Note: the cut scores shown in this table are the minimum estimated scores. Meeting the minimum MAP cut score corresponds to a 50% probability of achieving that performance level. Use the probabilities in Tables 5-8 to determine the appropriate “target” scores for a desired level of certainty. Note: bolded, italicized text denotes extrapolated cut score
- 8. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 8 | P a g e TABLE 4 – MINIMUM ESTIMATED PRIOR-SEASON (FALL) RIT CUT SCORES CORRESPONDING TO SOUTH CAROLINA PERFORMANCE LEVELS – READING READING-Prior Season Cut Scores and Percentiles for each State Performance Level Grade Level 1 Level 2 Level 3 Level 4 Cut Score Cut Score Percen- tile Cut Score Percen- tile Cut Score Percen- tile 2 <169 169 23 181 54 3 <181 181 23 193 54 4 <192 192 26 207 67 5 <196 196 19 213 66 6 <206 206 30 219 69 7 <210 210 30 223 70 8 <214 214 32 228 72 High <207 207 12 224 44 234 74 * Note: the cut scores shown in this table are the minimum estimated scores. Meeting the minimum MAP cut score corresponds to a 50% probability of achieving that performance level. Use the probabilities in Tables 5-8 to determine the appropriate “target” scores for a desired level of certainty. Note: bolded, italicized text denotes extrapolated cut score
- 9. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 9 | P a g e TABLE 5 –ESTIMATED PROBABILITY OF “MEETING STANDARDS” OR BETTER ON THE MATHEMATICS TEST IN SAME SEASON (SPRING), BY STUDENT GRADE AND RIT RANGE * Note: This table provides the estimated probability of passing the state test based on a MAP test score taken during that same (spring) season. Example: if a third grade student scored 170 on a MAP test taken during the spring season, her/his estimated probability of passing the state test is 7%. RIT Range 2 3 4 5 6 7 8 High 130 1% 1% 1% 1% 1% 1% 1% 1% 135 1% 1% 1% 1% 1% 1% 1% 1% 140 1% 1% 1% 1% 1% 1% 1% 1% 145 2% 1% 1% 1% 1% 1% 1% 1% 150 3% 1% 1% 1% 1% 1% 1% 1% 155 5% 2% 1% 1% 1% 1% 1% 1% 160 8% 3% 2% 1% 1% 1% 1% 1% 165 13% 4% 3% 1% 1% 1% 1% 1% 170 20% 7% 4% 2% 1% 1% 1% 1% 175 29% 11% 7% 3% 2% 1% 1% 1% 180 40% 17% 11% 5% 3% 2% 1% 1% 185 52% 25% 17% 8% 4% 3% 1% 2% 190 64% 36% 25% 12% 7% 4% 2% 4% 195 75% 48% 36% 18% 11% 7% 3% 6% 200 83% 60% 48% 27% 17% 11% 5% 9% 205 89% 71% 60% 38% 25% 17% 8% 14% 210 93% 80% 71% 50% 36% 25% 13% 21% 215 96% 87% 80% 62% 48% 36% 20% 31% 220 97% 92% 87% 73% 60% 48% 29% 43% 225 98% 95% 92% 82% 71% 60% 40% 55% 230 99% 97% 95% 88% 80% 71% 52% 67% 235 99% 98% 97% 92% 87% 80% 64% 77% 240 99% 99% 98% 95% 92% 87% 75% 85% 245 99% 99% 99% 97% 95% 92% 83% 90% 250 99% 99% 99% 98% 97% 95% 89% 94% 255 99% 99% 99% 99% 98% 97% 93% 96% 260 99% 99% 99% 99% 99% 98% 96% 98% 265 99% 99% 99% 99% 99% 99% 97% 99% 270 99% 99% 99% 99% 99% 99% 98% 99% 275 99% 99% 99% 99% 99% 99% 99% 99% 280 99% 99% 99% 99% 99% 99% 99% 99% 285 99% 99% 99% 99% 99% 99% 99% 99% 290 99% 99% 99% 99% 99% 99% 99% 99% 295 99% 99% 99% 99% 99% 99% 99% 99% 300 99% 99% 99% 99% 99% 99% 99% 99% MATH-Current Season Estimated Probability of Passing State Test Based on Observed MAP Score
- 10. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 10 | P a g e TABLE 6 –ESTIMATED PROBABILITY OF “MEETING STANDARDS” OR BETTER ON THE READING TEST IN SAME SEASON (SPRING), BY STUDENT GRADE AND RIT SCORE * Note: This table provides the estimated probability of passing the state test based on a MAP test score taken during that same (spring) season. Example: if a third grade student scored 190 on a MAP test taken during the spring season, her/his estimated probability of passing the state test is 57%. RIT Range 2 3 4 5 6 7 8 High 130 1% 1% 1% 1% 1% 1% 1% 1% 135 1% 1% 1% 1% 1% 1% 1% 1% 140 2% 1% 1% 1% 1% 1% 1% 1% 145 4% 1% 1% 1% 1% 1% 1% 1% 150 6% 2% 1% 1% 1% 1% 1% 1% 155 9% 4% 2% 1% 1% 1% 1% 1% 160 14% 6% 3% 2% 1% 1% 1% 1% 165 22% 10% 4% 4% 1% 1% 1% 1% 170 31% 16% 7% 6% 2% 2% 1% 2% 175 43% 23% 11% 9% 4% 3% 2% 4% 180 55% 33% 17% 14% 6% 5% 3% 6% 185 67% 45% 25% 22% 10% 8% 5% 9% 190 77% 57% 36% 31% 16% 12% 8% 14% 195 84% 69% 48% 43% 23% 18% 13% 21% 200 90% 78% 60% 55% 33% 27% 20% 31% 205 94% 86% 71% 67% 45% 38% 29% 43% 210 96% 91% 80% 77% 57% 50% 40% 55% 215 98% 94% 87% 84% 69% 62% 52% 67% 220 99% 96% 92% 90% 78% 73% 64% 77% 225 99% 98% 95% 94% 86% 82% 75% 85% 230 99% 99% 97% 96% 91% 88% 83% 90% 235 99% 99% 98% 98% 94% 92% 89% 94% 240 99% 99% 99% 99% 96% 95% 93% 96% 245 99% 99% 99% 99% 98% 97% 96% 98% 250 99% 99% 99% 99% 99% 98% 97% 99% 255 99% 99% 99% 99% 99% 99% 98% 99% 260 99% 99% 99% 99% 99% 99% 99% 99% 265 99% 99% 99% 99% 99% 99% 99% 99% 270 99% 99% 99% 99% 99% 99% 99% 99% 275 99% 99% 99% 99% 99% 99% 99% 99% 280 99% 99% 99% 99% 99% 99% 99% 99% 285 99% 99% 99% 99% 99% 99% 99% 99% 290 99% 99% 99% 99% 99% 99% 99% 99% 295 99% 99% 99% 99% 99% 99% 99% 99% 300 99% 99% 99% 99% 99% 99% 99% 99% READING-Current Season Estimated Probability of Passing State Test Based on Observed MAP Score
- 11. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 11 | P a g e TABLE 7 – ESTIMATED PROBABILITY OF “MEETING STANDARDS” OR BETTER ON THE MATHEMATICS TEST IN PRIOR SEASON (FALL), BY STUDENT GRADE AND RIT RANGE * Note: This table provides the estimated probability of passing the state test in spring, based on a MAP test score taken during the previous (fall) season. Example: if a third grade student scored 170 on a MAP test taken during the fall season, her/his estimated probability of passing the state test in spring is 17%. RIT Range 2 3 4 5 6 7 8 High 130 1% 1% 1% 1% 1% 1% 1% 1% 135 2% 1% 1% 1% 1% 1% 1% 1% 140 4% 1% 1% 1% 1% 1% 1% 1% 145 6% 2% 1% 1% 1% 1% 1% 1% 150 9% 3% 1% 1% 1% 1% 1% 1% 155 14% 4% 2% 1% 1% 1% 1% 1% 160 22% 7% 3% 1% 1% 1% 1% 1% 165 31% 11% 5% 2% 1% 1% 1% 1% 170 43% 17% 8% 3% 2% 1% 1% 1% 175 55% 25% 13% 5% 3% 1% 1% 1% 180 67% 36% 20% 8% 4% 2% 1% 2% 185 77% 48% 29% 13% 7% 4% 2% 3% 190 84% 60% 40% 20% 11% 6% 3% 4% 195 90% 71% 52% 29% 17% 10% 5% 7% 200 94% 80% 64% 40% 25% 16% 8% 11% 205 96% 87% 75% 52% 36% 23% 12% 17% 210 98% 92% 83% 64% 48% 33% 18% 25% 215 99% 95% 89% 75% 60% 45% 27% 35% 220 99% 97% 93% 83% 71% 57% 38% 48% 225 99% 98% 96% 89% 80% 69% 50% 60% 230 99% 99% 97% 93% 87% 78% 62% 71% 235 99% 99% 98% 96% 92% 86% 73% 80% 240 99% 99% 99% 97% 95% 91% 82% 87% 245 99% 99% 99% 98% 97% 94% 88% 92% 250 99% 99% 99% 99% 98% 96% 92% 95% 255 99% 99% 99% 99% 99% 98% 95% 97% 260 99% 99% 99% 99% 99% 99% 97% 98% 265 99% 99% 99% 99% 99% 99% 98% 99% 270 99% 99% 99% 99% 99% 99% 99% 99% 275 99% 99% 99% 99% 99% 99% 99% 99% 280 99% 99% 99% 99% 99% 99% 99% 99% 285 99% 99% 99% 99% 99% 99% 99% 99% 290 99% 99% 99% 99% 99% 99% 99% 99% 295 99% 99% 99% 99% 99% 99% 99% 99% 300 99% 99% 99% 99% 99% 99% 99% 99% MATH-Prior Season Estimated Probability of Passing State Test Based on Observed MAP Score
- 12. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 12 | P a g e TABLE 8 – ESTIMATED PROBABILITY OF “MEETING STANDARDS” OR BETTER ON THE READING TEST IN PRIOR SEASON (FALL), BY STUDENT GRADE AND RIT SCORE * Note: This table provides the estimated probability of passing the state test in spring, based on a MAP test score taken during the previous (fall) season. Example: if a third grade student scored a 190 on a MAP test taken during the fall season, her/his estimated probability of passing the state test in spring is 75%. RIT Range 2 3 4 5 6 7 8 High 130 2% 1% 1% 1% 1% 1% 1% 1% 135 4% 1% 1% 1% 1% 1% 1% 1% 140 6% 2% 1% 1% 1% 1% 1% 1% 145 10% 3% 1% 1% 1% 1% 1% 1% 150 16% 5% 2% 1% 1% 1% 1% 1% 155 23% 8% 3% 2% 1% 1% 1% 1% 160 33% 13% 5% 3% 1% 1% 1% 1% 165 45% 20% 8% 5% 2% 1% 1% 1% 170 57% 29% 12% 8% 3% 2% 1% 2% 175 69% 40% 18% 13% 5% 4% 2% 4% 180 78% 52% 27% 20% 8% 6% 4% 6% 185 86% 64% 38% 29% 13% 9% 6% 10% 190 91% 75% 50% 40% 20% 14% 10% 15% 195 94% 83% 62% 52% 29% 22% 16% 23% 200 96% 89% 73% 64% 40% 31% 23% 33% 205 98% 93% 82% 75% 52% 43% 33% 45% 210 99% 96% 88% 83% 64% 55% 45% 57% 215 99% 97% 92% 89% 75% 67% 57% 69% 220 99% 98% 95% 93% 83% 77% 69% 79% 225 99% 99% 97% 96% 89% 84% 78% 86% 230 99% 99% 98% 97% 93% 90% 86% 91% 235 99% 99% 99% 98% 96% 94% 91% 94% 240 99% 99% 99% 99% 97% 96% 94% 96% 245 99% 99% 99% 99% 98% 98% 96% 98% 250 99% 99% 99% 99% 99% 99% 98% 99% 255 99% 99% 99% 99% 99% 99% 99% 99% 260 99% 99% 99% 99% 99% 99% 99% 99% 265 99% 99% 99% 99% 99% 99% 99% 99% 270 99% 99% 99% 99% 99% 99% 99% 99% 275 99% 99% 99% 99% 99% 99% 99% 99% 280 99% 99% 99% 99% 99% 99% 99% 99% 285 99% 99% 99% 99% 99% 99% 99% 99% 290 99% 99% 99% 99% 99% 99% 99% 99% 295 99% 99% 99% 99% 99% 99% 99% 99% 300 99% 99% 99% 99% 99% 99% 99% 99% READING-Prior Season Estimated Probability of Passing State Test Based on Observed MAP Score
- 13. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 13 | P a g e TABLE 9 – CORRELATION COEFFICIENTS BETWEEN MAP AND SOUTH CAROLINA TEST FOR EACH GRADE AND TEST SUBJECT Grade Math Correlation Pearson's r Reading Correlation Pearson's r 3 0.807 0.787 4 0.837 0.778 5 0.858 0.778 6 0.850 0.786 7 0.844 0.780 8 0.845 0.781 High 0.866 0.815 TABLE 10 – PERCENTAGE OF STUDENTS WHOSE STATUS WAS ACCURATELY PREDICTED BY THEIR MAP PERFORMANCE USING REPORTED CUT SCORES Grade Sample Size MAP Accurately Predicted State Performance MAP Underestimated State Performance MAP Overestimated State Performance Mathematics 3 6475 86.04% 6.39% 7.34% 4 6363 86.64% 5.72% 7.51% 5 6103 88.33% 4.54% 6.90% 6 6004 80.63% 6.26% 7.88% 7 5992 84.61% 5.81% 8.76% 8 5686 81.23% 7.49% 9.34% High 3266 88.17% 5.46% 6.37% Reading 3 6475 85.36% 6.33% 8.31% 4 6363 88.54% 5.03% 6.43% 5 6103 88.28% 5.54% 6.18% 6 6004 86.73% 6.26% 7.01% 7 5992 85.58% 6.93% 7.49% 8 5686 86.26% 6.44% 7.30% High 3231 89.27% 4.90% 5.83%
- 14. S C ( P A S S & H S A P ) N W E A A l i g n m e n t 2 0 1 0 14 | P a g e The Kingsbury Center at NWEA 5885 SW Meadows Road, Suite 200 Lake Oswego, OR 97035-3526 www.kingsburycenter.org Tel 503-624-1951 Fax 503-639-7873