The value of measuring your accessibility maturity

Editor's Notes

• #3: Neil
• #4: Andrew Define what we’re taking about
• #5: Andrew A measure of how well an organisation is doing with respect to ensuring all their digital products and services are Various models can be used to measure this Diagram is from Icon in the UK - It has 5 levels: Initial – process unpredictable, poorly controlled and reactive Repeated – processes are defined and documented Defined – processes are standardised Managed – Processes are monitored and controlled Optimised – processes are continuously improved Sources: Diagram: https://www.icon-uk.net/Fraud_Masterclass/Fraud_Risk_Digital_Maturity_Assessment.html Definitions: https://www.techtarget.com/searchsoftwarequality/definition/Capability-Maturity-Model
• #6: Neil Why might we want to know what our maturity level is? To get a measurement of the current effectiveness of a group or an organisation’s accessibility practices To assess gaps between the current capabilities and the next level of accessibility maturity Knowing where you are on a given scale and what the gaps in culture, knowledge, process, etc should encourage and enable improving overall accessibility performance over time Mitigate risk – legal, reputational, financial A couple of organisations we’re working with at present hope it will give them ‘permission’ to work on accessibility as BAU.
• #7: Andrew There are many good models out there which can be used to measure accessibility maturity. Lots of commonality and all of them have their pros and cons. For the purposes of demonstrating the value of maturity assessments, we’re going to take a look at the draft model currently being designed by the World Wide Web Consortium (W3C). You should find a model which is right for you. The W3C model is composed of dimensions and maturity stages. Each dimension has associated proof-points and deliverables by which progress is measured. You can set goals and outcomes for each stage which are relevant to the dimension being assessed. We will first provide an overview of the dimensions, and then we will pick a couple of dimensions and take a closer look at examples of goals, outcomes and proof points relevant to those specific dimensions.
• #8: Neil The W3C Accessibility Maturity Model is a tool that assist you to: assess the current effectiveness and capabilities of an entire organization or just teams within the organization identify gaps between the current capabilities and the next level of accessibility maturity Plan for improvements to your organisation's accessibility performance over time
• #9: Neil 7 Dimensions to measure in W3C’s model: Communications: Information as it relates to an organization’s accessibility, as well as accessibility of all internal/external communications. Knowledge and Skills: Ongoing education, and outsourcing practices to fill gaps for accessibility operations. Support: Accessibility assistance provided to internal employees and external customers with disabilities. Information and Communication Technology Development Life Cycle: Incorporation of web, software and hardware accessibility considerations in development processes - from idea conception, to design, development, testing, ACR production, user research, maintenance and obsolescence. Personnel: Job descriptions, recruiting, disability-related employee resource groups necessary to provide lived-experience to accessibility efforts. Procurement: A strategic process that concentrates on finding and acquiring accessible products required by an organization. Activities may include: sourcing, negotiation, and selecting goods and services. Culture: The attitudes, sensitivity, and behaviors around accessibility, including internal interaction, perception, and decision-making. Each dimension can have a goal and outcome associated with each stage, though strictly speaking, the first stage (Inactive) has neither since it is inactive).
• #10: Andrew How is each dimension measured? To assess the maturity stage for each dimension, a series of Proof Points have been suggested which can be used as evidence. These are: evidence-based, organizational deliverables which are used to measure the maturity stage. specific to the dimension being focused on. For example, if only procurement maturity is being measured, only procurement proof points should be evaluated. likely to span several stages. They can be partially completed at the Launch and Integrate stages, but must be fully completed for the optimize stage. The proof points demonstrate the completed outcomes, and should be relevant to the dimension and to the organisation. If an outcome is not relevant, it should be marked as Not Applicable. for instance, it would be pointless requiring an organisation to meet outcomes relating to native apps if they don't have them.
• #11: Neil Using the evidence or proof points, you can measure an organisation’s progress on each dimension which leads to a determination of the maturity level of the organisation or department within it. The four Stages of maturity in the W3C models are: Inactive - No awareness and recognition of need. Launch - Recognized need organization-wide. Planning initiated, but activities not well organized. Integrate - Roadmap in place, overall organizational approach defined and well organized. Optimize- Incorporated into the whole organization, consistently evaluated, and actions taken on assessment outcomes. The W3C is producing an assessment template which can be used against this model, with worksheets corresponding to the dimensions, each organised to reflect the four stages above, with a list of relevant proof points. Each can then be scored with a final overall score helping to determine the level of maturity. Time does not permit us to take you through each of the seven dimensions in detail but we will use two as an example. This will enable us to demonstrate likely dimension-related goals and outcomes for each stage of maturity, and the proof points which might be used as evidence. It is assumed that there are no goals and outcomes at the Inactive stage of maturity.
• #12: Andrew Time does not permit us to take you through each of the seven dimensions in detail but we will use two as an example. This will enable us to demonstrate likely dimension-related goals and outcomes for each stage of maturity, and the proof points which might be used as evidence. It is assumed that there are no goals and outcomes at the Inactive stage of maturity. We’ll consider Communications Knowledge and skills
• #13: Andrew Goals and outcomes relevant to communications dimension might include: Launch Stage Goal: Recognized need for accessibility of internal and external communications. Plans begun or in place to identify communications where accessibility criteria should be integrated. Outcomes: Plans in place for making all internal / external communications accessible (and compliant with accessibility regulations, where applicable). Integrate Stage Goal: Communications roadmap in place, integration of accessibility criteria into most internal and external communications is being implemented. Non-accessible tools replaced, or procurement of accessible tools in process. Outcomes: Internal and external delivery of accessible communications across most media and platforms An accessibility policy that includes requirements for a feedback mechanism for users, including a formalized process for handling accessibility complaints. Optimise Stage Goal: Full accessibility processes and tools implemented across all internal and external media resulting in fully accessible communications. Outcomes: All processes, procedures, and tools are in place, used consistently, and are regularly reviewed and refined to ensure that all internal and external communications are fully accessible.   Proof points used to measure progress might include but by no means be limited to: Accessible corporate document templates (word, ppt, etc) Processes and procedures for creating communications are documented, and infrastructure is in place to ensure accessibility Websites, e-mail marketing and social media are accessible per regulatory requirements (e.g. conforms to WCAG) Websites may have an accessibility statement or contain a statement of commitment to accessibility There is a clear feedback mechanism for handling questions and accessibility complaints
• #14: Neil Goals and outcomes relevant to this dimension might include: Launch Stage Goal: Recognized need for organization-wide accessibility and disability inclusion, as well as accessibility expertise. Planning initiated, but activities not well organized Outcomes: Skill areas identified, plans for organization wide surveys to identify gaps initiated, but not implemented Mostly Ad hoc training (professional development is not required or monitored.) Some training courses established, e.g. for onboarding or annual refresher Integrate Stage Goal: Workforce ICT accessibility skills and training roadmap in place. Integration of process to assess and increase accessibility knowledge and skills is in progress. Outcomes: Role-based training available for people’s role plus current information on new technologies, platforms, and tools Training metrics are established Optimise Stage Goal: Organization-wide, role-based expertise in accessibility and disability inclusion is well-defined, evaluated, remediated, and continuously enhanced. Accessibility knowledge and skills are consistently implemented across the organization Outcomes: Required and preferred knowledge and skills are consistently communicated to all personnel Workforce is periodically evaluated to ensure knowledge and skills are current with the most up to date standards and accessibility practices Periodic analysis used to identify gaps in knowledge as well as training materials Annual training (conferences, events, online, etc.) is provided to maintain skills current with ICT accessibility requirements and industry best practices. Workforce inclusion training incorporates accessibility for persons with disabilities. Tracking systems in place and consistently used to maintain training inventory, measure skills, and track completion   Proof points used to measure progress might include but by no means be limited to: Assessing Current Skills to Identify and Address Gaps such as: Organizational surveys that identify current skill levels and gaps Internal database(s) to track employee training for ICT accessibility skills Keeping skills up-to-date with current requirements Accessibility criteria integrated into individual performance measurements Building and Maintaining Organizational Capacity such as Implementation of role-based training plans and curriculums Accessibility training when onboarding all new employees Accessibility requirements included in position descriptions Subject matter experts (SMEs) appointed within the organization to provide training and support
• #15: Andrew Undertaking a gap analysis approach to assessing maturity – four aspects: Interviews Documentation reviews Process reviews Output reviews
• #16: Neil Interviews, conversations really, with: Product owners Designers Developers Content creators Testers / QA Leadership Staff with disabilities Marketing and Comms People and Culture (incl D&I) Call centre support Looking for different things from different teams and individuals understanding across the organisation Commitment Leadership Internal support Client support Recruitment communication What is happening at the coal face Understanding of policies Level of knowledge Role responsibility Stage considered Tools used
• #17: Andrew Processes considered - as appropriate Design and development Publishing Procurement Events Marketing Recruitment Support, including feedback and complaints
• #18: Neil Documents reviewed - depends what’s documented Looking for commitment Looking for measurement Looking for plans Looking for process Documentation might include: Policies and strategies Annual reports, including disability action plans Risk register Procurement documentation Accessibility support materials Job descriptions Professional development
• #19: Andrew Sample output to look at – might include: Client / customer communications Website content Any audits if conducted Especially looking for critical and high impact failures also common failures
• #20: Neil Bring it all together What is their maturity by dimension considered What do we recommend, next slide, but: We want people to know what’s working well and to keep doing it What did we find that is holding them back
• #21: Neil Recommendations Strategic and tactical Leadership messages Support groups Implementers skills uplift Portal for accessibility support Recruitment Outsourcing /procurement Priorities Timelines Depends on the organisation
• #22: Andrew Suggestions for policy and strategy improvements Eg in D&I policy Suggestions for improving specific documents we’ve reviewed Eg branding guidelines Practical suggestions around software and platforms in use Basis for an a11y support portal Eg MS Office or AEM etc List of different disability days to celebrate and increase awareness, eg Basis for an a11y support portal Eg MS Office or AEM etc
• #23: Neil Remember it’s much more than just ICT – it’s all things that touch a customer or employee Rinse and repeat
• #24: Andrew So, why do we care about measuring digital accessibility maturity? “Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.” ― H. James Harrington
• #25: Chair - 10 mins