Csun presentation 2015 final v2
- 1. NWEA: Bryan Goodwin, Regan Daniels, & Elizabeth Barker NCAM: Bryan Gould & Geoff Freed Text Alternatives in Computer-Adaptive Assessments
- 3. • Bryan Goodwin: Item Publishing Manager • Regan Daniels: Director of Test Publishing • Elizabeth Barker: Early Learning and Accessibility Specialist NWEA Presenters
- 4. …NCAM brings accessible media and technology into homes, schools, workplaces, and communities… • Research, development and solutions • Technology • Standards • Content • Compatibility • Training and outreach
- 5. National Center for Accessible Media (NCAM) at WGBH • Presenters – Bryan Gould, Director of Accessible Learning and Assessment Technologies – Geoff Freed, Director of Technology Projects and Web Media Standards NCAM Presenters
- 6. • Computer adaptive assessments, unique to each student • Powered by more than 30 years of academic research • Efficient, more data with fewer questions • Immediate results
- 7. Our Steps to Remove Barriers CAST HTML/QTI WCAG 2.0 Proof of Concept NCAM
- 8. • Education, education, education! – Presentations – A voice at meetings, discussions, etc. Crash if you have too! – Lunch and Learns • Fostering a workplace culture where an accessibility mindset is the norm. – Identify others in your organization who might want to work with you on this topic. – New product development with accessibility in mind. – Be an Accessibility Advocate in your organization. A Challenge: Getting Buy-In We’re not going away…
- 9. • Top leadership buy-in is critical. – Having a influential individual back and/or drive your accessible technology initiative sets the tone for your organization's accessibility mindset. • Education, education, education! – Mission drivers – Market/Commercial Drivers – Legal drivers – Cost drivers A Challenge: Getting Buy-In This is the business case…
- 10. Conception to Execution • Step 1: Image descriptions for item bank
- 11. Immediate Work • Technical Development – Item Construction • Conversion to QTI Format • Support for alternative text – Test Delivery • Exposing alternative text to screen readers • Enabling other accessible behaviors, e.g. keyboard navigation
- 12. Challenges/Examples • When you can’t provide alternate text – Decorative/visual bias/excessive complexity – Impact on psychometric validity
- 15. External Support • Text Authoring for Existing Bank – 9533 test items containing 1-5 images each • Over 11000 images to review • Guidelines/Style – Formal standards for content within image descriptions are lacking – Consistency required for test validity
- 17. NCAM • Hired editors with experience in instruction, publishing, and item writing. We trained them in accessibility. – Variety of subjects: science, math, ELA • Multiple rounds of review for description and copy accuracy. Editors worked collaboratively on small batches of similar items. – Helped to improve descriptions and reduce burnout
- 18. NCAM • Open communication with NWEA SMEs to ensure the description was appropriate for grade and construct. • Documented description approaches • Reviewed for Visual Bias • Descriptions were tested with Screen Readers
- 19. NCAM • NCAM is working with NWEA developers to create accessible interactive items, also called Technology Enhanced Items (TEI) – choice multiple – drag/drop – click/pop
- 20. NCAM • Choice multiple – choose more than one answer – text or images • All users must be able to… – identify each answer choice – determine how to select one or more answers – select/unselect each answer – receive confirmation – identify state when reviewing
- 21. NCAM
- 22. NCAM • identify each answer choice • alt • determine how to select one or more answers • role=“checkbox”
- 23. NCAM • select/unselect each answer • spacebar/key combination • receive confirmation • “truck, checkbox checked”
- 24. NCAM • identify state when reviewing • “car, checkbox checked; school bus, checkbox not checked; truck, checkbox checked”
- 25. Maintaining Process • Step 2: Develop an operational process to continue development internally – Internal training/skills – Documentation • Role of Content Specialists – Subject-matter experts on hand for consultation – Across disciplines: Reading, Language Usage, Mathematics, General Science
- 26. • Keeping accessibility visible. • What to do when access- ibility isn’t top priority? – Tackle “low-hanging fruit.” • What to do when it IS a priority? – Formal team – opportunities for education – Funding – where does the money come from? Keeping the Momentum
- 27. • We’ve got alternative text… • Now what?! • Delivering on our platform • New frontiers for our TEIs • How do we alt-tag interactions? • Readiness for Braille and large print • Availability of technologies in schools • Early learning • Non-readers Current Challenges
- 28. Our Steps to Towards the Future ARIA MathML 508 Tools Beyond Assessment
- 29. • Contact information: Elizabeth.Barker@nwea.org Regan.Daniels@nwea.org Bryan.Goodwin@nwea.org Bryan_Gould@wgbh.org Geoff_Freed@wgbh.org • Twitter handles: @NWEA and @AccessWGBH • Find our presentation at www.slideshare.net Questions?
Editor's Notes
- #11: After determining A) we’re going to pursue this and B) establishing organizational support, we have to make the project a reality. Image: An NWEA test item with an illustration of a crayon and a ruler, asking the student to find the length of the crayon. Alt tag for this image would be: “A crayon is next to a centimeter ruler. The base of the crayon is at 2 centimeters, and the tip is at 9 centimeters”
- #12: It’s not as simple as “add alt tags to test items”; all surrounding systems must support alternative text properly QTI = Question and Test Interoperability specification, produced by IMS Global. Designed to provide a standardized format for storing items & tests.
- #13: Though students don’t all see the same items in an adaptive test, we must ensure that there is a suitable pool of items for the adaptive selection engine to get its best results. i.e., we want to make sure that all relevant goal areas are provided to students using accessibility features.
- #14: Item 1: A test item with images of four fish of varying size, asking students to identify which fish is shortest. Item 2: A test item with a "net" diagram of a three-dimensional figure, asking students which figure could be made from that net. Item 3: A test item with an overly complicated table/chart combination image
- #15: Similar images aren’t always described in the same manner. Must take into account the intent of the item, the expected instructional level of students seeing the item, whether the description gives away the answer, and whether subject-matter-specific vocabulary is required for effective description. Image 1: A test item with an image of an analog clock asking the student to identify what time the clock shows. The alternate text for the image would read “The short hour hand points just after 5. The long minute hand points to 3” Image 2: A test item with an image of an analog clock asking the student about what activity occurs at the displayed time. The alternate text for the image would read “The analog clock shows eleven thirty a.m.”
- #16: Emphasize – content and delivery
- #17: Through a series of grants funded by the National Science Foundation and The US Department of Education, NCAM has been researching and developing methods for creating effective and efficient text alternatives to images so that kids and adults who are blind and visually impaired could have equal access to learning materials. NSF funded research focused on refining description approaches for STEM images. Reviewed hundreds of descriptions and dozens of images types, then narrowed focus to the most commonly used images in STEM for the highest impact, for example: line graphs and bar charts. This research included rounds of surveys with b/vi STEM professionals (scientists, mathematicians, etc) and user-testing with higher ed students who were b/vi. The recommended approach encourages: Brevity, focus on Data, Clarity and Consistency in language, Navigation Control via HTML lists and tables. Further research projects funded by the DoED focused on description for K-12 students, for images used in assessments. This included images found in a widening set of disciplines including ELA and social studies. In addition, NCAM has collaborated with content authors and publishers in describing tens of thousands of images for clients in K-12 and higher ed. both in instruction and assessment.
- #26: You can train all of your subject matter experts, or you can train a group dedicated to producing alternative text. In our case, the task fit in well for my team to provide the text while finalizing the items. Writing alternative text is not a common skill; best to train internal staff rather than try to hire someone just for this purpose. Copy Editor background well suited to the task, they are accustomed to using brief, precise language wherever possible and are able to adapt to style requirements of different kinds of writing/items Keep expert guidance close at hand, it’s very nice to have an arrangement where you can consult with experts as-needed
- #29: Longdesc W3C finally approved and in standards