Constructing Effective Rubrics for L2 Classrooms
- 1. Constructing Effective Rubrics for L2 Classrooms Workshop at the BC TEAL Spring Island Sessions, Victoria, February 19, 2011 Michael Burri, M.A. TESOL Program Coordinator BC Institute of Technology Vancouver, BC
- 2. “The goal of assessment has to be, above all, to support the improvement of learning and teaching” (Frederiksen & Collins, 1989, p. 32).
- 3. Program Background - Established in September 2008 - Intensive EAP program - 7 levels: DTC and BBY campus - 4 skills - 7 week terms - 25 hrs of instruction/week - 50% pass - Approx. 300 students + 25 instructors - Ongoing curriculum project
- 4. Assessment Summative - Assessment OF learning: achievement-based Formative (e.g. Black, 2009) - Assessment FOR learning: facilitates students’ learning - Frequent assessment of student progress informs teaching - Assessment to meet student needs - Use of varied assessment approaches - Learning action must follow feedback (Black et al., cited in Black, 2004)
- 5. “Assessment for learning is any assessment for which the first priority in its design and practice is to serve the purpose of promoting pupils’ learning” (Black et al., 2002, cited in Black, 2004)
- 6. Rubrics? “At its most basic, a rubric is a scoring tool that lays out the specific expectations for an assignment. Rubrics divide an assignment into its component parts and provide a detailed description of what constitutes acceptable or unacceptable levels of performance for each of those parts” (Stevens & Levi, 2005, p.3).
- 7. “Rubrics are used to provide feedback on and to grade an array of student products, including concepts maps, literature reviews, reflective writings, bibliographies, oral presentations, critical thinking, citation analyses, portfolios, projects and oral and written communication skills” (Reddy & Andrade, 2010, p.437).
- 8. Issues in Rubric Design - Course objectives + curriculum - Consistency in scoring (reliability) - Measuring what rubric is intended to measure (validity) - Analytic or holistic? - Establishing assessment criteria - Simple/clear language - Time-consuming to design? - Instructional guides, not just grading tools
- 9. Holistic vs. Analytic Rubrics (Mertler, 2001) Holistic: - score of overall process, product, quality - quick scoring process - summative in nature - limited feedback
- 10. 5 Displays excellent paragraph 4 Displays good paragraph 3 Displays adequate paragraph 2 Displays inadequate paragraph 1 Displays weak paragraph Comments:
- 11. Analytic: - Focused response - Several scores and summed total score - Scoring process potentially slower than holistic - Effective, detailed, formative feedback - Encourage critical thinking (Stevens & Levi, 2005) - Time-consuming to construct, but save time in grading
- 12. Evaluation Criteria Levels of quality/scale Descriptors Final Presentation Poor (12 pts) Good (18 pts) Exceeds Expectations (25 pts) Student is not fully informed about the topic. Presentation does not offer well-rounded discussion of the topic. Slides may be unorganized or difficult to follow. Student demonstrates knowledge of the subject, but doesn’t elaborate much. Presentation is logical and presents both sides of the issue. Student demonstrates full knowledge of the subject. Presentation is logical, interesting, and presents both sides of the issue in detail & without bias. Uses fewer than 3 sources and/or Works Cited page has several formatting errors. Used 3 credible sources. Works Cited page has some errors in formatting or spelling. 3 credible sources. Works Cited Page follows correct MLA formatting. Correct information and sources are doublespaced, alphabetically listed, & correctly indented. Slides Slides contain sparse, incorrect, or unrelated information. Little attention to polishing final product. Slides are organized and contain sufficient information. Some blending of appropriate graphics and text. Creative; eye-catching; seamlessly blends graphics and text. Slides in logical, easy-to-follow sequence. Oral Presentation Speaker seems uninterested in the topic. Speaker simply reads slides with little attempt to capture/keep audience’s attention. Speaker seems interested in the topic. Speaker may read some information off slides. Presentation is interesting to audience. Speaker is animated, captivating, informative. Speaker does not read slides; rather supplements information on slides with other facts/examples. Subject Knowledge Organization Sources Works Cited Rubric taken from http://www.rcampus.com/index.cfm
- 13. Designing Analytic Rubrics: Guidelines (adapted: Mertler, 2001; Stevens & Levi, 2005) 1. 2. 3. 4. 5. 6. 7. 8. 9. Examine learning objectives Examine task Identify evaluation criteria Decide levels of quality/scale (e.g. 4-1, excellent poor) Write clear (simple?) descriptors Pilot Implement Revise Collect samples (benchmark)
- 14. …simply handing out a rubric cannot be expected to have an impact on student work: students must be taught to actively use a rubric……in order to reap its benefits” (Reddy & Andrade, 2010, p.445).
- 15. Finally……. ……….“Rubrics are not cast in cement. They are flexible, adaptable grading tools that become better and better the more times we use them. Their strength, reliability and validity increase as we use rubrics, discover limitations, and make revisions” (Stevens & Levi, 2005, p.93).
- 17. Bibliography Black, P. (2009). Formative assessment issues across the curriculum: the theory and the practice. TESOL Quarterly, 43(3), 519-524. Black, P. (2004). The nature and value of formative assessment for learning. Retrieved January 28, 2011 from www.kcl.ac.uk/content/1/c4/73/57/formative.pdf. Frederiksen, J. R., & Collins, A. (1989). A systems approach to educational testing. Educational Researcher, 18(9), 27-32. Mertler, C. A. (2001). Designing scoring rubrics for your classroom. Practical Assessment, Research & Evaluation, 7(25). Retrieved August 10, 2010 from http://PAREonline.net/getvn.asp?v=7&n=25. Rcampus. (2011). Retrieved January 31, 2011 from http://www.rcampus.com/index.cfm. Reddy, M. Y., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435-448. Stevens, D. & Levi, A. (2005). Introduction to rubrics. Sterling, VI: Stylus Publishing, LLC. Underhill, N. (2002). Testing spoken language. Cambridge: Cambridge University Press.