Machine Learning applications in accessibility
0 likes94 views
The document discusses the development and capabilities of va11y, an automated accessibility testing tool created by VMware. It highlights the importance of accessibility for the 1.3 billion people with disabilities globally and outlines the challenges of current automated testing methods. Planned enhancements for va11y in 2021 include features for detecting captions, transcripts, keyboard focus indicators, and additional accessibility checks.
Machine Learning applications in accessibility
- 1. ©2020 VMware, Inc. Automated Accessibility Machine Learning SHERI BYRNE-HABER Accessibility Architect, Staff II
- 2. ©2020 VMware, Inc. Agenda 2 Introduction AGENDA Next StepsvA11yCurrent State
- 4. ©2020 VMware, Inc. 4 Machine Learning - Definition Definition by Tom Mitchell (1998): Machine Learning is the study of algorithms that • improve their performance P • at some task T • with experience E.
- 5. ©2020 VMware, Inc. 5 Natural Language Processing Dog Breeds Dachshund Pass Great Dane Pass Samoyed Pass Petunia Fail NLP is the automatic manipulation like speech and text by software.
- 6. ©2020 VMware, Inc. 6 Image Recognition The ability of software to identify objects, places, people, writing and actions in images Source: Imgur Dog Bagel Dog Bagel
- 7. ©2020 VMware, Inc. 7 Pattern Recognition The automated recognition of patterns and regularities in data Bagels don’t have eyes, a tail, or noses Dogs don’t have sesame seeds
- 8. ©2020 VMware, Inc. In the beginning, all the data is unstructured … Data Data Data Data Data Data Data Data Data Data Data Data Data Data Supervised Machine Learning – You start with unstructured data
- 9. ©2020 VMware, Inc. Data Data DataData Good Data Model Bad Data Model Data Data A human manually sorts the data, which builds a model
- 10. ©2020 VMware, Inc. Pass Then you test unknown data against the models you have built Fail Good Model Bad Model Unk. Data Unk. Data Unk. Data Unk. Data Unk. Data Unk. Data
- 11. ©2020 VMware, Inc. 11 Pivoting to Disability Disability is the largest minority group impacting more than 1.3 billion people globally. • Mobility & dexterity • Hearing • Other • Vision • Neurological & cognitive
- 12. ©2020 VMware, Inc. 12 25% of disabilities are visible. 75% of disabilities are visible. • ADHD • Asthma • Cancer • Chronic Fatigue Syndrome • Chronic Pain • Cystic Fibrosis • Diabetes • Epilepsy • HIV/AIDS • Learning Disabilities • Mental Health • Traumatic Brain Injury • Autism • Down Syndrome • Fetal Alcohol Syndrome • Tourette Syndrome • Amputations • Paralysis • Cerebral Palsy • Muscular Dystrophy (MD) • Multiple Sclerosis (MS) Resource: World Health Organization
- 13. 13 ©2020 VMware, Inc. Automated Accessibility Testing – Current State
- 14. ©2020 VMware, Inc. 14 Automated Accessibility Testing – Current State 1. Code Analysis 2. Framework Dependent 3. Open Source for unlimited commercial use – unavailable • No live data • Only 30 % of tests can be executed automatically
- 16. ©2020 VMware, Inc. 16 How vA11y came into being • Started as an idea when I realized I had way more testing to do than I had time for • Went into a contest at work where it was chosen as the winning idea • Was accepted into our experimental development program as a proof of concept • When debugging the code we went from 1 site to 30 sites to 300 sites – and found a serious bug in the home page of a computer company known for its public commitment to accessibility. • VMware partners are clamoring for it • Has been promoted to experimental labs full track – 3 updates scheduled in 2021
- 17. ©2020 VMware, Inc. 17 What’s in vA11y Proof of Concept • Automatic detection of captions and subtitles in videos • Automatic detection of transcripts for podcasts • Automatic detection of the presence of keyboard focus indicators • Automatic measurement if a keyboard focus indicator meets the thickness and contrast requirements
- 18. ©2020 VMware, Inc. 18 Planned 2021 vA11y additions 1. 2.4.4 - Link consider ARIA overrides 2. 4.1.2 - name role value look for announcements associated with state changes 3. 1.3.1 - Required fields look for legend and marker 4. 2.4.1 - Check skip link behavior not just for presence 5. 1.4.4 / 1.4.10 - Is the web page responsive 6. 1.3.5 - If the page contains a form, is autocomplete available? 7. 1.4.13 - hover cancelation 8. 2.1.5 - Form and session timeout 9. 2.1.1 - are there mouse only events or ARIA hidden objects
- 19. ©2020 VMware, Inc. 19 Planned 2021 vA11y additions (Continued) 10. 2.4.5 - multiple ways 11. 3.2.1 / 3.2.2 - Is there a live region when an action triggers changes? 12. 4.1.3 - is there a live region for message announcements when message doesn't get focus? 13. 1.4.5 - warning when there is text embedded in picture 14. 1.4.11 - KFI approach except as applied to image buttons, 15. 2.2.2 - automatically moving carousel 16. 3.2.3 - Take inventory of menus, compare to menu organization on subsequent pages, focus on horizonal and vertical nav
- 20. ©2020 VMware, Inc. 20 Planned 2021 vA11y additions (Continued) 17. If a keyboard shortcut is used, test to make sure only one non-alpha key is used 18. 1.4.12 - Check to see if works are cut off or obscured after spacing changes 19. 1.1.1 - Everything that is not “Google I am not a robot v3” is a fail 20. 1.3.2 / 2.4.3 - focus order 21. 1.2.3/1.2.5 DA/AD button
- 21. ©2020 VMware, Inc. 21 How did VMware Test vA11y Supervised learning at every level UK.gov 31 Sites, 8 industries Top 300 “safe for work” sites Branch into building custom data models
- 22. 22 ©2020 VMware, Inc. Next Steps
- 23. ©2020 VMware, Inc. 23 vA11y Demo
- 24. ©2020 VMware, Inc. 24 How do you use vA11y • Go to Git, download the package • Install the package • Use the default model or build your own • Make vA11y API calls • Review results • Feedback corrections into your model (optional)
- 25. ©2020 VMware, Inc. 25 Interested in contributing to vA11y? We need help with the following • Early testers • Documentation translators • Larger, more evolved, generic data models for other vA11y users to work from • People to monitor slack channel and answer basic questions • Participate in coding one of the 26 planned additions in 2021
- 26. Confidential │ ©2020 VMware, Inc. Thank You THANK YOU SHERI BYRNE-HABER sbyrnehaber@vmware.com https://medium.com/@sheribyrnehaber https://www.linkedin.com/in/sheribyrnehaber/