2020 was the year when governments across Europe had to deliver accessibility – so what can we learn from what went wrong? The goal of accessibility isn’t to find issues, it’s to fix them

A month ago, in our survey about what people wanted us to present on in our popular 2021 Digital Accessibility Experts Live webinars, we received one pointed cry for help:

“Is there any tangible evidence that progress is being made?”

I think I know what the person was worried about…

You can have great standards, you can have great laws; but if people don’t do the right things in response, then there’s a real problem. The key thing is organisations’ attitudes and commitment to actually delivering accessibility.

There was a big example of this in 2020, in the results of the Public Sector Bodies Accessibility Requirements. PSBAR, as it’s commonly called, is the UK’s response to the EU Web Accessibility Directive, and it required all government websites to get to WCAG 2.1 AA by 23 September 2020.

A few weeks after the deadline, Scope shared some research that said that 9 out of 10 websites that they tested had a multitude of accessibility errors.

That’s not sounding good.

To be fair, testing 10 public sector websites is not really representative. If you compare that with the UK government’s accessibility monitoring plans for 2021-22, you’ll see they’re going to test over 1,000 sites in the coming year to try and identify any bodies who are struggling. That will certainly help.

Some in the disability community don’t think it’s enough. They have suggested a “naming and shaming” approach to motivate organisations’ commitment to accessibility if their site’s don’t test well.

This has been tried before. The problem is: historically, it doesn’t work. Not on its own.

At Hassell Inclusion we care about accessible results as much as the next person, but our viewpoint is that to get good results, you need to look at the how and why behind those results. It’s more about why things happen than about what happens. And not asking “why?” can give you the wrong strategy.

I believe that what happened in the UK in 2020 was that, in all the noise and challenge around the pandemic, web teams mistook what the goal was. They spent all their time arranging testing. That’s useful. But that’s not the goal.

The goal is fixing the problems that the tests find, and documenting workarounds to help affected users get around any issues that can’t be fixed.

Loads of tests were done last year – ask any accessibility company in the UK (and quite a few non-accessibility companies who jumped on the bandwagon, increasing the supply of audits but unfortunately also lowering their quality), and they’ll tell you how busy they were. Then ask them how many of the people who bought an audit came back and asked for clarification of the issues found, for help in fixing them, or to check their fixes were done right. You’ll find a lot less…

Chances are the owners of those 9 out of 10 websites knew all of the accessibility problems they had. They just hadn’t got over the hurdle of fixing stuff.

And they are certainly not alone in having difficulties getting to the end of the race. Because it’s not easy.

So how can we help?

It’s all about what you do when you’ve got that audit report in front of you, and if the number of pages in it is already starting to depress you…

Prioritisation: the essential way of getting from an audit report to an accessible site

Whatever accessibility testing methodology you use, it is essential to enable your product team to quickly assess the accessibility issues (defects) it finds and to prioritise fixes on a cost-benefits basis.

That’s why at Hassell Inclusion we provide test results in two ways that enable:

1. Product & Project management staff to quickly get a strategic overview of the number of issues, and their relative importance, based on an analysis of the benefits of fixing each issue to the different disabled audiences it affects, and an estimate of the cost of implementing its proposed fix.
2. Implementation staff to understand each issue and its impact in more detail and apply the suggested fix (or workaround).

My team’s experience of writing and reading countless accessibility test reports is that the second of these – the technical information to help the people who’ll actually fix issues – is usually well covered in audits by reputable accessibility companies.

But the essential overview information to enable the results of testing to be quickly and strategically understood, and the resources prioritised for fixing, is often poorly done or completely missing. This then requires time-poor project management staff to read through tens or hundreds of pages of detail to work out what the test results mean for their project planning.

To rectify this, back in 2014 I created the Accessibility Issue Prioritisation Matrix. This provides a way for testers to communicate the cost-benefits of fixing each issue found to product and project managers, and facilitate their discussion of prioritisation of fixes with all stakeholders.

Summarising the cost-benefits of all those accessibility issues

The Accessibility Issue Prioritisation Matrix is a spreadsheet that provides an overview of all issues found in testing, with each line describing an issue, a unique reference number so it can be easily and precisely referred to in discussions, the suggested solution for fixing it, and finally, values for each of the four key factors that are most useful in prioritising fixes.

The four key factors are:

• The extent/frequency of occurrence of the issue and the importance of the parts of the product in which the issue occurs:
  • In the site or app’s structure, whether the issue occurs in an element which appears on every page across the whole site or app (e.g. in the navigation); on a section navigation page; or on a single leaf page
  • Whether the page(s) the issue occurs on are part of product’s core user journeys or not
    (note for accessibility experts: this prioritisation of errors that appear in key user journeys has now been adopted in the proposed scoring mechanism of WCAG 3.0)
  • How frequently the issue occurs
• The size of the audiences that would experience difficulty in using the product because of the issue (note for accessibility experts: the AIPM assumes that most products will be used by the general public, so uses figures of the number of people with each disability from national census; however, it also allows the relative importance of different groups to be tweaked for products being created for particular groups – e.g. older people)
• The impact of the issue on those audiences’ use of the product:
  • High = a total block to them completing a user journey
  • Medium = they can complete the user journey, but the issue will slow them down
  • Low = the issue is unlikely to impact them, in practice
• The estimated cost of fixing the issue

Each line in the Matrix also includes three different cost-benefits measures for fixing the issue, calculated from the four key factors using business intelligence embedded in the spreadsheet:

1. Value for money – the number of people affected by the issue, multiplied by the impact of how much it affects them (benefits), compared with the estimated costs of fixing it
2. Political risk aversion – the political weight (in terms of adverse PR and litigiousness) of the disabled groups affected by the issue, multiplied by the impact of how much it affects them (benefits), compared with the estimated costs of fixing it
3. A blend of both of these two measures

How the situation you find yourself in dictates how to prioritise fixes

Which of these cost-benefits measures is most use to you is based on your organisation’s motivation for accessibility, and where you are in your product development when the audit happens:

• If you’re just about to launch, it’s all about the quick wins that you can action right now.
• If it’s earlier in the process, then you can look at value for money – what are the greatest benefits for the lowest costs?
• If your site has been live for a while, and your lack of accessibility has been found out (by central government or, worse, a law-firm), it’s all about how to make the problem go away by complying with whatever they are contesting – whether it’s a specific level of WCAG compliance that you need to get to, or whether it’s just fixes needed for people who are deaf or blind (who are often those who complain about lack of accessibility).

The spreadsheet’s tabs allows you to choose which of these makes most sense to you and places the issues in the prioritisation order that best suits your needs. Then you can work through them from the most important to the least important, deciding which fixes you can budget for and which you cannot (and so many need workarounds).

How you get yourself an AIPM

Our Accessibility Issue Prioritisation Matrix has been so useful for Hassell Inclusion clients that all of the Standard WCAG audits we do come with an AIPM as standard. As one product manager client, at a Prioritisation Workshop we ran with his team, recently said:

“You’ve done all my work for me”

But you don’t need to have us audit your site to benefit from what an AIPM can give you…

We’ve helped lots of organisations who’ve had an audit done by another accessibility company understand their audit results, by creating them an AIPM and supporting them through a Prioritisation Workshop to create a sensible, budgeted plan for their fixing or remediation.

Or, if you want to have a go yourself, I’ve included a version of the AIPM that you can fill in yourself in the free support materials for my Inclusive Design for Products book.

I’d recommend that you review it, together with its documentation and case study, to see whether it would help you to quickly understand and action the prioritisation of accessibility fixes in your project process. Then, if you need further help or training in using it, contact us at Hassell Inclusion.

And, once you’ve got that done, take inspiration from these folks who’ve got it right

One last thing. The best thing about last year is that some organisations really got it. They tested and fixed their sites, worked out that they never ever want to go through that costly, difficult remediation process again, and asked us for help:

“I want to make sure this never happens again…”

That’s where our optimism is in 2021. For increasing numbers of organisations, they’ve worked out prevention is better than cure. They’ve learned from their experiences of 2020, and they want to get good at accessibility in 2021, right from the start of projects.

If, like them, you’d like us to help you with that, please get in touch.