Using Response to Intervention with English Language Learners

Best Practices in Assessment and Intervention for ELL Mike Vanderwood, Ph.D. University of California-Riverside [email_address]

ELL Growth 9.6% of students in the US public schools are ELLs 25.2% of students in California schools are ELL, and 85.3% of these students speak Spanish ELLs’ literacy skills wane in comparison to non-ELL students (NCES, 2005) ELLs have a higher risk of being placed into special education across elementary grades, and are 40-50% more likely than their White peers to qualify for SLD. (Artiles, et al, 2005) Schools often delay examination for Special Ed because they want to eliminate language interference

Current Practice Very few teachers and psychologists are trained to work with students who have diverse language backgrounds Some suggest that we have a national shortage of qualified staff and the shortage will continue to grow.

Foundation Of Practice Growing understanding that educational practices should be guided by high quality research and standards of practice New standards for determining whether a practice is “evidence based” IES implementation of “What Works Clearinghouse” AERA/APA/NCME Test Standards

Foundations: Test Standards and ELL

LEP Assessment (English language learners) Students are considered Limited English Proficient when: they were not born in the U.S. and native language is not English, or Come from environments where English is not dominant language Bilingual individuals do not necessarily have equal proficiency in both languages

Standards Must have validity evidence for the purpose used Pg. 95. If student is not from the culture or linguistic that produced norms, scores may not provide a valid comparison Standard 9.3, bilingual students should be assessed to determine the proficiency in both languages

Standards (cont) Standard 9.10 determining language proficiency needs to be based on a range of language features Standard 9.11, interpreters need to be fluent in both languages, and have some knowledge of assessment

Test Bias for LEP (ELL) Significant flaws in research in this area Primary issue is determination of English proficiency Studies should account for: acculturation, cultural background, quality of instruction and educational history Current evidence does not indicate bias, yet results are questioned by many

Foundations: Research Standards

Scientifically Supported Interventions Dept. of Education was reorganized to put a focus on high quality research Institute of Education Sciences (IES) Created What Works Clearinghouse http://www.whatworks.ed.gov/ WWC evaluates educational interventions Focused on applying randomized controlled experiments as methodology to determine causation

Prevention is Critical Several authors (Gersten, 2005; Vaughn, 2005; What works, 2007; Vanderwood, 2008) suggest RtI is an exceptionally appropriate service delivery approach for ELL

RTI Defined RtI is the practice of (1) providing high-quality instruction/intervention matched to student needs and (2) using learning rate over time and level of performance to (3) make important educational decisions. Source: Response to Intervention: Policy considerations and implementation (National Association of State Directors of Special Education-2005).

Quick Summary of What We Know for NS Early screening and Intervention impacts performance and can reduce those who need special education Reading Performance is best described by performance in: Phonological Awareness Phonics Comprehension Fluency Progress monitoring with instruments that have strong reliability and validity can improve outcomes for students with academic problems

Does this literacy knowledge apply for ELLs? Additional Focus on oral language proficiency Influence of culture? ELL Students are Often not included in literacy screens. Why? Research suggests, we can accurately assess English early literacy skills as early as Kindergarten for ELLs.

Language Proficiency Degree of control one has over the language in question Basic Interpersonal communication skill (BICS) Communicating in socially related situations First area to develop in new language Cognitive academic language proficiency skills (CALPS) Developed through academic activities

Language of Instruction Center for Research on the Education of Students Placed At Risk (CRESPAR) Very few studies that meet current research standards Native language instruction appears to be most beneficial, and does not harm English Language Dev. Early exit appears to work as well as late exit (research is not exceptioinally clear). http://www.csos.jhu.edu/crespar/techReports/Report66.pdf

Critical Point It is not necessary to wait to deliver English Language literacy support/interventions until the native language is established In fact, all ELL students’ literacy skills should be assessed as soon as possible (i.e., kindergarten).

Previous ELL Reading Research Chiappe, Siegel, & Wade-Woolley (2002) 858 Kindergarteners, 23 languages A significant relationship between PA and improved reading was reported in this study for monolingual and ELL students Linan-Thompson & Hickman-Davis (2002) Supplementary reading instruction with at-risk, low SES second grade monolingual and ELL students 30-minute supplementary instruction: phonological awareness, vocabulary, decoding, comprehension, and word analysis strategies Positive outcomes for reading for monolingual and ELL students

Lesaux & Siegel (2003) Suggest that PA may be a better predictor of reading development than oral language proficiency Development of reading for ELL students was not predicted by English Language Proficiency PA instruction was effective for ELL students EL group developed quickly and by grade 2 were performing as well or better on most tasks as EO students Metal-linguistic advantage for ELL may explain rapid improvement PA is an important skill for EL. For poor performers, similar profile exists to EO poor performers

Literacy Research with typical RtI measures What type of evidence to we need Ability to identify students at risk Sensitivity and specificity Sensitive to growth during intervention Use across curricula and interventions Minimally impacted by language proficiency?

Nonsense Word Fluency (NWF) Standardized, individually administered test of alphabetic principles Intended for children from mid to end of kindergarten through the beginning of second grade Children are given 1 minute to read as many nonsense words as possible Example: Read whole word, such as “lut” or say the individual sounds, such as /l/ /u/ /t/ Psychometrics One month alternate form reliability is .83 Predictive validity in first grade with CBM ORF first grade is .82, and CBM ORF second grade is .60

NWF Study # 1: Purpose of Study To examine the appropriateness of using the NWF in assessing students’ reading readiness skills for an EL, urban population

Site Demographics 1 school site 100% Free/ Reduced Lunch Ethnicity: 83% Hispanic (n=165) 11% African American (n=21) 6% other (n=13) Home language: 82% Spanish (n=186) 14% English (n=32) 5% other (n=12)

EL Status (Determined by CELDT): Beginning LEP N = 49 (21%) Intensive/structured-English classes Early Intermediate Range N = 58 (25%) Structured English class Intermediate Range N = 111 (47%) structured-English/English only classes Early Advanced/Advanced range N = 16 (7%) Excelled class

Mean Differences (All Signficant)

Results of Study NWF was able to significantly predict reading scores on a state-mandated achievement test above and beyond EL level There were significant mean differences on all NWF scores by EL Level. NWF alone significantly predicted SAT9 reading composite at 11 of 12 opportunities Growth was not significantly different by EL group

NWF Study # 2: S AMPLE M ETHODS 3-year longitudinal study Spanish-speaking ELL population 90% ELL in Grade 1 58% ELL in Grade 3 Assessed first-grade students: Nonsense Word Fluency SAT9 Assessed third-grade students: ORF MAZE CAT6

Correlation of Grade 1 Measures to Grade 3 Measures CAT6 ORF MAZE .36 .38 .34 .17 .50 .57 .64 .25 .26 .24 .14 .23

Specificity and Sensitivity Analysis VP = Valid positive; FN = False negative; FP = False positive; VN = Valid negative; Pos. PV = Positive predictive value (VP/(VP + FP); Neg. PV = Negative predictive value (VN/(VN + FN); Sensitivity = VP/(VP + FN); Specificity = VN/(VN + FP); Hit rate = (VP + VN)/(VP + FN + VN + FP).

Specificity and Sensitivity of NWF to ORF Hit Rate = 75% Neg. PV = 84% Pos. PV = 51% Specificity = 82% VN =82 FP =18 CBM ORF > 25 th percentile Sensitivity = 55% FN = 15 VP = 19 CBM ORF < 25 th percentile Indices NWF Not at Risk NWF At Risk Outcome Measure

Specificity and Sensitivity of NWF to Maze Hit Rate = 65% Neg. PV = 59% Pos. PV = 81% Specificity = 89% VN = 57 FP = 7 Maze > 25 th percentile Sensitivity = 43% FN = 40 VP = 30 Maze < 25 th percentile Indices NWF Not at Risk NWF At Risk Outcome Measure

Specificity and Sensitivity of NWF to CAT6 Hit Rate = 71% Neg. PV = 81% Pos. PV = 43% Specificity = 79% VN = 79 FP = 21 CAT6 > 25 th percentile Sensitivity = 47% FN = 18 VP = 16 CAT6 < 25 th percentile Indices NWF Not at Risk NWF At Risk Outcome Measure

Progress Monitoring - Intervention Study #1: Research Question What is the impact of a phonological awareness intervention implemented with at-risk ELL students? How effective is progress monitoring in English with ELL. Healy, Vanderwood, & Edelston. (2005)

Methods Participants 15 Low SES ELL first grade students (7 male, 8 female) with PSF and NWF baseline scores 30 and below Materials Sounds and Letters for Readers and Spellers (Greene, 1997) Progress Monitoring PSF and NWF weekly

Methods (cont.) Procedures Small group (max. 5) ½ Hour Sessions x2 per week Exited after PSF > 50 and NWF > 45 12 – 25 Sessions

Results 6 students exited at first exit point after 12 sessions 12 students (80%) exited by conclusion 2 students (13.3%) met exit criteria for PSF, but not NWF 1 student (6.7%) met exit criteria for NWF, but not PSF The group as a whole went from mean PSF and NWF scores that were considered to be in the at risk range to mean PSF and NWF scores that were in the mastery level range

Example of PSF scores of a participant who was exited from the intervention

A participant’s NWF scores who met the exit criteria for PSF, but not NWF

A participant’s PSF scores who met the exit criteria for NWF, but not PSF

Progress Monitoring/Intervention Study #2: To examine the extent to which a direct and explicit PA intervention impacts at-risk ELLs’ early literacy skills. To replicate findings by Healy, et al., 2005 To examine the impact of a bilingual, direct and explicit PA intervention with at-risk ELLs. To assess the extent to which progress monitoring is effective with ELLs.

Method 10 Participants (8 males, 2 females) CELDT Levels ( beginner, early intermediate, & intermediate levels ) Three Pronged Screening Method Bottom 25% of students on Developmental Reading Assessment DIBELS PSF & NWF (at-risk levels) Intervention Curriculum Sounds and Letters for Readers and Spellers (Greene, 1997) with modifications Progress Monitoring One time per week using PSF and NWF tests

Procedures 2 groups of 5 students 30 minutes intervention 2x per week Reinforcement: Token Economy Treatment Integrity A 10-item checklist was developed Assessed 30% of intervals with 90% accuracy Curriculum manual was used 100% of observations Added Bilingual support and Spanish PA intervention

Results Baseline PSF: 0 – 6 (M = 3, SD = 2.75) NWF: 0 – 32 (M = 10, SD = 9.75) Final 3 Monitoring Data Points PSF: 19 – 72 (M = 56.8, SD = 20.6) NWF: 49 - 66 (M = 56.8, SD = 23.4) Effect Size [Cohen’s d (1988); Pooled (Swanson & Saches-Lee (2000)] PSF: 19.8; 1.9 (Large) NWF: 4.8; 1.3 (Large)

Conclusion In 20 sessions of supplemental English PA instruction, 8 of 10 students met or exceeded PSF and NWF goals This study provides further support for the use of RTI to determine LD status among ELL students at-risk for reading failure. 3 of 3 students met PSF goals using the bilingual PA intervention Trendline analysis indicates that the Bilingual PA intervention was as and more effective than English PA alone for this very small sample, but only for PA skills.

Oral Reading Fluency and Middle School Students, including EL Learners

CBM per Week Gains by English Fluency

Assessment Conclusions Initial evidence that typical measures used for monitoring progress for EO students can be used with ELL Initial evidence that measures can pick up growth caused by an intervention Initial evidence that rate of growth for ELL is similar to EO

Instructional Practices for ELL (Gersten & Geva, 2003) Explicit teaching Promotion of English Language Learning Phonemic awareness and decoding Vocabulary development Interactive teaching that maximizes student engagement Opportunities for accurate responses with immediate feedback

Vanderwood et al. English Intervention studies Used Dibels to select group Small group instruction (3 to 5) Explicit instruction in phonemic awareness and phonics Frequent use of Choral responding Behavioral reinforcement techniques to maintain high levels of engagement 30 minutes of intervention 2X per week Between 87-93% of students moved established level on Dibels measures within 16 weeks

Experimental ELL PA Intervention All 12 Sessions include a Vocabulary Section, and 5 Phonological Awareness Activities in the following order: Phoneme Production/Replication; Phoneme Segmentation and counting, Phoneme Blending, Phoneme Isolation, and Rhyming.

Sessions 1-3 Session 1: / t /; / s /; / m /; / b /; / k /; / f /, Short / a /. Session 2: Review: / t /; / s /; / m /; / b /; / k /; / f /, Short / a /. New : / r /; / h /; / j /; / n /; / l /; / p /. Session 3: Review: / t /; / s /; / m /; / b /; / k /; / f /; / r /; / h /; / j /; / n /; / l /; / p /, Short / a /. New : / g /; / d /; / v /.

Spanish Intervention (Vaughn et al., 2006a) 69 ELL (Spanish) 1 st graders at risk for reading difficulty Screening was conducted in Spanish (Woodcock Language Proficiency Battery-Spanish—LWID) and Experimental measure of Spanish word reading ability) Randomly assigned by school (7 schools, 20 classrooms) to treatment or comparison.

Treatment: Systematic, explicit instruction in oral language and reading (Lectura Proactiva, Mathes, et al., 2003) for groups of 3 to 5 for 50 minutes a day. In Spanish to match core literacy instruction Comparison: School’s standard reading intervention (e.g., guided reading, reading recovery, tutoring)

Vaughn et al., 2006a Results Pre-Post design No group differences in pre-test scores Assessed Spanish & English literacy & oral language Treatment group scored higher on most Spanish outcome measures No group differences on English outcome measures

Intervention Starting Points (Vaughn, et al., 2006b) Started with assumption that ELL learn to read like monolingual students: through phonological recoding and spelling sound patterns Assumed that those students have problems learning how to read are struggling because they have not mastered the alphabetic principle. Decodable text used throughout instruction Sight words were taught (words that are less phonetically regular in English)

Instructional Design (Vaughn, et al., 2006b) Systematic and explicit instruction for Phonemic awareness Phonemic decoding skills Word recognition fluency Construction of meaning Vocabulary Spelling Writing Used the same intervention as others had used with EO students, but interspersed language support activities

Population 2 districts in Texas 4 schools English intervention to at least 2 classes of ELL first graders (48 to 99% Spanish speaking) At least 60% of school population was Latino 80% or more of 3 rd grade students passed state reading tests

Screening Assessment 14 first grade classrooms 2 tests in English, 2 in Spanish Letter word Identification from Woodcock Language Proficiency Battery in English and Spanish Word lists in English and Spanish Note: this is not currently an empirically supported approach to screening 56 students met criteria in both languages, 48 participated in the study 24 treatment, 24 comparison

Intervention Structure 48 student Groups of 3 to 5 50 minutes per day from October to May Supplemental to core reading instruction (not as replacement!) Teachers received 12 hours of professional development before intervention and 6 hours after first 6 weeks Staff development throughout the year occurred and lessons were videotaped Conducted intervention validity checks

Intervention implementation Fast paced, all students responding, followed by individual turns Provided immediate feedback

Results Intervention students outperformed comparison group on English letter naming, phonological awareness, other language skills and reading achievement Differences were less significant on Spanish measures

Evaluating Interventions for ELL Structure: Direct Instruction is critical Size: Small group is essential Feedback process: immediate and often Content: similar to monolingual students with enhanced vocabulary

Special Ed Eligibility Issues for ELL

Special Ed Assessment Issues Eligibility determination issues: Non-verbal IQ tests

IQ/ Processing Assessment Techniques bilingual examiner Translator Translated version developed with norms translated version with English norms Translated version with foreign country norms non-verbal assessment Problem-solving

Testing Options English developed Test translated to foreign language May not measure same characteristic in the foreign language Questions may have changed in meaning Cultural groups may differ in conceptualization of construct Violates Test Standards Foreign norms with American translated test Different culture, issue of equivalence

Non-verbal for ELL Many lack appropriate psychometric evidence Predictive power is limited and not as strong as traditional IQ tests Not normed with ELL population

Issues related to LD identification for ELL We need to improve our system at identifying and intervening with ELL students who have academic problems. We need to identify students who are most at risk of academic difficulties earlier than 3 rd grade, preferably kindergarten. We need to use resources in general and special education more efficiently. We need to provide an integrated continuum of services that uses high quality data for entry to succeeding levels. We need to re-conceptualize the construct of LD to integrate language proficiency.

We need to better connect our initial eligibility assessments to instruction and intervention (assessment to intervention link) and address issues of literacy development in L1 and L2 We need to integrate recent assessment science that suggests that progress monitoring can have substantial positive impact on academic outcomes.

RTI special education decision making Disability and Need Concept of Dual Discrepancy Disability: Lack of response to high quality intervention Torgesen’s work shows about 6% Need: Convergent Data District determined Norm referenced Individualized achievement test Large scale assessment Alternative assessment local norms Percentage for cutoff ranges from 6 to 10%

What about culture? MAMBI Acculturation surveys?

Conclusions Initial evidence early literacy measures can be used to identify EL students at risk of literacy problems. Initial evidence tools can be used assess growth during intervention

What we don’t know How does language proficiency affect the ability of the measures to achieve their purposes? How do we integrate culture into the decision making process? How do we integrate native language knowledge into the process?

Using Response to Intervention with English Language Learners

More Related Content

What's hot (20)

Viewers also liked (12)

Similar to Using Response to Intervention with English Language Learners (20)

Recently uploaded (20)

Using Response to Intervention with English Language Learners