The future of WCAG – maximising its strengths not its weaknesses

2012 was a year of real ups and downs for the de-facto Standards for web accessibility, WCAG 2.0:

• It’s finally become ratified as an international Standard (ISO/IEC 40500:2012)
• It’s been directly included in legislation such as the EU proposed directive on accessibility of public websites, Canada’s AODA, and work going into the refresh of US Section 508
• At the same time it’s had its value questioned by academic research, and the achievability of its AAA level questioned by a number of voices in the accessibility community (building on articles from as far back as 2006 that accuse WCAG 2.0 of being ‘unrealistic and unproven’)
• Calls for it to be updated are also becoming louder and louder (from WebAIM and the PDF/UA community, amongst others)
• And frameworks like BS 8878, in which WCAG 2.0 can be more successfully integrated with the practicalities of real-world web product development, are gaining support in the commercial world and in the academic world

So, for 2013, I thought it was time to summarise WCAG 2.0’s strengths and weaknesses, what strengths other national standards have that it may lack, and what will be needed to make it a much better ‘harmonised Standard’ for the future.

Here’s a menu of the aspects of WCAG 2.0 that we’re going to consider:

• the strengths and weaknesses of having harmonised accessibility Standards
• the strengths and weaknesses of WCAG 2.0’s success criteria
• the strengths and weaknesses of WCAG 2.0’s conformance levels
• the strengths and weaknesses of WCAG 2.0’s stability and static-ness
• where WCAG should go next, and what needs to support it to become a full solution rather than the partial solution it is now

Strengths and weaknesses of having harmonised accessibility Standards

I’d agree with the W3C that there are many benefits that can be achieved by harmonization of accessibility Standards globally:

“Formal approval by JTC 1 of WCAG 2.0 will increase deployment, reduce fragmentation, and provide all users with greater interoperability on the web.” (W3C)

“Such recognition is expected to increase internationally harmonized uptake of WCAG 2.0 by governments, business, and the broader Web community.” (Jeff Jaffe, W3C CEO)

“We also expect that ISO/IEC recognition will encourage greater convergence around WCAG 2.0, further driving development of supporting tools and software.” (Karen Higginbottom, Chair of ISO/IEC JTC 1)

The claims that harmonizing around WCAG 2.0 is likely to increase WCAG 2.0’s current poor uptake – addressing one of the critiques in Power et al’s research paper earlier this year – are well founded. And this will allow WAI to make more progress in their ‘carrot and stick’ work to encourage adoption:

• governments globally will be more able easily to place WCAG 2.0 in legislation;
• more uptake may allow WAI to provide more business cases of the benefits of accessibility to organisations– so far they have only managed to persuade a few to quote any ROI figures (although the OneVoice report in the UK is another useful source of case studies)

But the most important question that seems to have been glossed over is: what affect will that have on the accessibility of websites worldwide?

Because none of their quotes claim that making WCAG 2.0 an International Standard will actually improve disabled & elderly people’s experience of using websites worldwide, that WCAG 2.0 are the right (or only) Standards that should be harmonised around, or that wiping out existing non-WCAG Standards won’t also involve losing useful insights.

So is WCAG 2.0 a great set of guidelines to harmonise around?

Strengths and weaknesses of WCAG 2.0 success criteria

In general, each of WCAG 2.0’s individual success criterion is very useful.

From my experience of using them with website creation teams, there’s a couple of obvious improvements that can be made:

• each success criterion would be more useful if it gave an idea of which disabled audiences would benefit from conforming with the criterion, and a rough idea of how much it might cost to conform to it
• the criteria could also be made easier to read, and it would be useful to structure them via job-roles so it’s easy to know which member of a website creation team needs to deal with each criterion

The main weaknesses, however, are WCAG 2.0’s incompleteness and lack of future-proofing.

The Uni of York paper found that numerous problems that disabled people experience with web pages are not covered by WCAG 2.0. This is certainly true, especially for the needs of people with cognitive, learning and literacy difficulties – disabled groups that were under-represented in the creation of WCAG 2.0.

So, covering more problems than WCAG 2.0 currently addresses could potentially improve WCAG, if it’s to be used on its own to improve accessibility.

However the paper’s assertion that ‘we must address all the problems that disabled people find on web pages’ is over-stated, as the variety of disabilities that people have, and the interaction of those disabilities with those people’s length of experience in using the web, means that it may be impossible to address all problems.

Moreover, it’s debatable whether many of the missing success criteria to address those missing problems are accessibility or usability issues.

I agree with WAI that including all potential website usability problems, whether shared by disabled and non-disabled people or not, in a notional future WCAG may not be the answer.

But the distinction between ‘issues that block access or interfere with access to the Web more severely for people with disabilities’ and ‘general usability issues’ can be rather murky, especially for WCAG 2.0’s very useful success criteria on error conditions for forms and headings (see Roger Hudson’s great blog on ‘the absurd distinctions that are sometimes made about the usability and accessibility of web content’).

So, if it’s not totally clear which issues should be allowed in WCAG 2.0 and which shouldn’t, I think it’s important to be clear that conformance with WCAG isn’t the only thing website creators need to do to make sure their sites work for disabled and elderly people.

And it’s important to be clear that WCAG is never going to be complete or future-proof. The web is too fast-moving for web guidelines to ever be complete. WAI did a great job attempting to make WCAG 2.0 technology agnostic, but they didn’t future-proof it with regard to mobile/tablet sites.

But it’s also important for WCAG to at least try and keep up with technology. As Richard Morton contributed on linkedin:

“I would also say that unless the WCAG 2.0 guidelines are updated to remove some of the anomalies, and make it fit for purpose with the changes to the web scene over the past few years since it was released (e.g. much more mobile/touch based access, responsive websites, apps), then it could create more problems, and result in a backwards step.”

If the aim of accessibility is to give a great web experience to all people, including those who are older or who have disabilities – which is how WCAG 2.0’s proponents are selling it – WCAG 2.0’s success criteria will need to be frequently updated, because currently it just isn’t enough to do that.

And it needs to balance how often it gets updated to become more relevant, true and complete, with ensuring its slow-moving enough to give website creators a target they can hit.

Strengths and weaknesses of WCAG 2.0 conformance levels

While I’m a great fan of WCAG 2.0’s success criteria, I can’t say the same for its conformance levels.

The idea of conformance levels is definitely attractive:

• it gives website creators an idea of the importance of complying with each individual success criteria;
• it gives websites a means of ‘badging’ their level of conformance, which is also very useful for regulators and legislators

However, the implementation is fundamentally flawed.

Assignment of the level to each success criteria

Let’s start with the assignment of levels A, AA and AAA to each success criteria.

This is currently flawed as a practical tool for helping website creators prioritise their accessibility work, as the levels don’t take into account two important aspects of modern website creation:

• the vast variety of purposes of a website – especially for media-rich and user generated/social content
• the cost-benefits of each success criterion

To give the simplest example of the problem of purpose:

• Criterion 1.2.5 requires all pre-recorded video content to be enriched with audio description. What this means is that for a site that includes video to achieve AA it needs to include audio description on all its pre-recorded video. So, for example, a user-generated video sharing site like YouTube could never achieve AA without requiring all of its users to include that audio description themselves. So YouTube’s owners would need to change the site’s purpose, or decide that WCAG 2.0 AA isn’t worth bothering with. To deprive websites of using video and social media because of unreasonable accessibility constraints is the type of bad guidance that enables web developers to claim ‘accessibility is anti-creative‘.

And here’s a simple example of the problem of cost-benefits:

• The number of people who benefit from captions being provided for video content is huge, and the cost is reasonable. Comparatively, the number of people who benefit from audio description (or an ‘alternative’ to video) is very small, and the cost is large. And yet none of this essential information is mentioned in the success criteria, and both are set at level A in WCAG 2.0.

WAI’s assignment of levels to WCAG 2.0 success criteria only really considers the needs of individual disabled users (specifically the users whose needs where championed in WCAG 2.0’s creation – so, not so good for cognitive, dyslexic, learning difficulties), and doesn’t include an idea of the number of people who will benefit from following each criterion (e.g. how many people there are in those disability groups), or any idea of the cost of implementing each criterion.

While it is sometimes hard to pin down how much each different group of disabled people would be helped by each success criterion, as criteria regularly help more than one group, WAI don’t make it obvious that any cost-benefits thinking was involved in their thinking, despite publishing useful statistics of prevalence of different disabilities.

This means that individual organisations trying to make their sites accessible often have two options:

• Stick dogmatically to the level of each individual WCAG 2.0 success criterion, however much they think it does or doesn’t make pragmatic sense (this is generally done by organisations that have WCAG 2.0 compliance required of them – let’s call them ‘Category 1 organisations’); or
• Do their own cost-benefits analysis of each success criterion, to decide whether or not they are going to comply (this is generally done by every other organisation that is allowed to come up with their own accessibility aims – let’s call them ‘Category 2 organisations’)

Conformance levels and logos

And, unfortunately, WCAG 2.0’s conformance rules makes things even worse, as conformance is marked against perfection in achieving all success criteria at a given level.

This means that, for WCAG conformance purposes, it’s no better to have a site that obeys all but one of the many Level A success criteria than have a site that doesn’t even know that WCAG exists.

So, to use my two types of organisation, from above:

• Organisations in category 1 tend to try and conform, then ‘wriggle’ and try and come up with ways of getting around problems using exception processes like the “comply or explain” principle in Holland’s national accessibility guidelines.
• Organisations in category 2 may make the hard task of deciding which of the success criteria is sensible themselves, hoping no-one’s going to argue too strongly with their decisions; or, more likely, take WCAG 2.0’s lack of commercial pragmatism as the perfect excuse to not comply with any of WCAG.

WCAG 2.0’s emphasis on needing to score perfection on all the success criteria on each level to get the level conformance logo completely mis-understands the realities of web product development – in website development perfection is not something you strive for; you aim for continuous, pragmatic improvement over versions.

It also places huge strain on the committee making sure that the success criteria are perfectly categorised into levels, as one mistake considering the reasonableness of assigning a success criterion the right level can (and, as we’ve seen, does) make it impossible for some websites to achieve any level of conformance.

The fact is that while idea behind conformance levels is attractive to anyone creating a Standard, and to those wanting simple metrics for governance, I’ve never seen a categorisation of success criteria into levels that created more positives than negatives.

Detlev Fischer nicely summarises the positives in his comment on my EU accessibility law blog:

“The advantage of WCAG is that it is out there as a recognised international standard, ready to assess a site’s status regarding many vital a11y requirements, and ready to inform many significant improvements. Good usability is much more context-dependent and harder pin down, so compliance to an augmented set of guidelines would be much harder to impose and measure in the way that WCAG 2.0 level AA conformance might be imposed and checked…”

But here are the negatives from my response:

“The worry is that WCAG 2.0′s flaws and inflexibility are going to make each case where WCAG 2.0 AA compliance is unreasonable into a shouting match that may give accessibility a bad name.

I’d also agree that compliance to an augmented set of guidelines would be harder to impose and check. But I think imposing and checking against any set of site creation guidelines slightly misses the point. It’s the results of following the guidelines that are important, not the following of them itself. Put it this way – you wouldn’t assess the usability of a website by assessing the guidelines the design and build team used to create the site; you’d measure the usability of the site itself, in terms of effectiveness, efficiency and satisfaction defined by ISO 9421-12. So the only reason that assessing usability for disabled people should be any different is if the cost of doing such usability measurement is unreasonable itself… different types of testing/measurement methodologies have different cost-benefit dynamics, and so the site owner themselves should define the amount of money they think is reasonable to give them the level of confidence they need that their site has achieved its inclusion goals.”

Strengths and weaknesses of WCAG 2.0 stability and static-ness

So, if WCAG is useful, but needs improvement, is that improvement likely soon?

One of WCAG 2.0 main strengths is its stability. Certainly WAI promote its stability in this quote from their press release on its approval as an ISO/IEC standard:

“As an ISO/IEC JTC 1 Standard, WCAG 2.0 is now also available from ISO/IEC, while it remains a stable international W3C standard with extensive supporting resources. JTC 1 recognition neither changes nor supercedes the existing standard, which remains freely available from the W3C website along with multiple W3C authorized translations of WCAG 2.0.”

Reliability and dependability are very useful aspects of WCAG 2.0’s stability. And it’s questionable if WCAG 2.0 would have become an ISO/IEC Standard if it wasn’t stable.

However, another aspect of stability is ‘resistance to change’ and, for want of a better term, static-ness. And it’s this static-ness that is WCAG 2.0’s main weakness, as it tends to divide accessibility experts into two extreme camps:

• defenders of WCAG, who tend to see any questioning of the Standard – including any critique or research that points out any flaws in it – as an attack to be rubbished
• critics of WCAG, who tend to look at WCAG as the orthodoxy of accessibility to be critiqued and questioned, sometimes with an unfortunate undercurrent of ‘we weren’t invited to feed into WCAG before it’s adoption as a standard, so we’re going to throw stones now’

It’s a reflection of WCAG 2.0’s success that, like any successful, influential thing in history – whether it’s a document, a movement or an ideology – it attracts both devotees and detractors.

But, as much as I dislike the under-appreciative tone of some critiques of WCAG, I really hate the attack on new research in accessibility, if that research doesn’t ‘fit in’ with the orthodoxy of WCAG.

Like any good idea, if WCAG is a great set of guidelines it should be able to stand up to rigorous critique.

And like the best ideas, if that critique finds WCAG wanting, it should be big enough to flex to accommodate that new thinking.

So the whole accessibility community should welcome the publishing of new research, like that from York and Norway in 2010, especially as much of the best research done on accessibility is commissioned for companies that are reluctant to publish the results for commercial reasons (at the BBC, I had similar research done into how well WCAG audits could predict problems identified by user-research, and found it similarly wanting).

All such published research adds to our understanding of the complex relationships between disabled users, accessibility guidelines, assistive technologies and usability outcomes.

As someone who has gone through the no-sleep-for-months hell of getting a Standard out there, it’s obviously frustrating to the drafters of WCAG 2.0 for the its value to be questioned.

However, WAI shouldn’t push criticism away. No Standard is perfect. And having your baby’s imperfections pointed out is only threatening if you don’t have any possibility or intention of fixing those imperfections.

W3C’s current way of ‘fixing WCAG 2.0’ seems to create supporting documents like Understanding WCAG 2.0 that can be updated more regularly than the Standard itself.

But more than this is needed.

A better way forwards

So can we imagine a better way? How about this?

Imagine a future where, instead of the crude A, AA and AAA, you could:

• Score a site’s accessibility against how many accessibility success criteria you’ve achieved (even including a multiplier for their importance based on the number of people affected), rather than whether you’ve reached a certain level so you can get a badge. Your step-by-step advances in accessibility through a site’s versions can now be reflected in its increasing accessibility score.
• Score the site’s accessibility against the needs of different disabled groups – e.g. a video site which doesn’t include audio description may be perfectly accessible to most disabled and non-disabled people; it’s deficiency is just a problem for blind and VI people. You can now let disabled people know these specific deficiencies in the site’s accessibility statement (the ‘limitations’ recommended by BS 8878), which will be much more useful to them than placing a WCAG conformance logo on the site.

Imagine a future where this release from crude conformance levels and notions of completeness allows WCAG to cease to be the static battering ram that it is now, but allows it to flex over time (possibly moderated by the proposed International Society of Accessibility Professionals):

• New success criteria can be added, when they are proven to have an impact on users, and when sufficient techniques are proven to lessen that impact.
• Existing success criteria can be promoted, demoted or removed as they are found – by the best research we can find – to be more or less important than we first thought.

Moreover, imagine a future where WCAG 2.0’s technical approach to accessibility is placed within a framework that allows accessibility to be more effectively considered from a product perspective, from an enterprise programme perspective, from a technology strategy perspective, and from the high-level national and global perspective of how governments can legislate and encourage accessibility.

That is what BS 8878 provides, and I would suggest that all legislators take a good look at it, because it:

• provides a more nuanced model for measuring accessibility, and for sensibly handling situations where the website development team can reasonably manage all of WCAG 2.0’s AA success criteria except one or two;
• provides a way of enabling organisations to embed accessibility considerations in their working practices so they can consistently produce accessible products.

How do we make this happen?

The problem with WCAG 2.0 isn’t that it’s not perfect; the problem is that it’s static and WAI are resistant to updating it, so its imperfections are argued over, rather than improved upon.

Having led the creation of a Standard – BS 8878 – myself, I know it’s not a trivial thing at all to try and create an updated Standard. And it’s not guaranteed that a WCAG 3.0 would automatically be better than WCAG 2.0, even with the insights we now have into WCAG 2.0’s deficiencies.

But I think it’s worth an open discussion to see whether there is consensus for a new direction for WCAG 3.0.

And, in the meantime, we have to go forwards with what we have – warts and all.

We can’t afford to wait for the perfect accessibility Standard to come along – it will never exist.

The problem isn’t that WCAG 2.0 isn’t any good; the problem is that it is being used for things that it’s not designed to do well. It’s a mistake for anyone to claim more than it can do, especially when advocates start using huge numbers of disabled people to motivate organisations to follow WCAG, when WCAG 2.0 doesn’t sufficiently make clear how its success criteria guarantee to help each of those audiences, or the cost-benefits of doing this.

What we need to do is to be very careful about how we apply the best accessibility standards that we have at any one time – making sure we use them for what they’re good at, making sure we don’t use them for what they’re not good at, and using the best insights we have to know the difference, with the courage to have the humility to say when we just don’t know yet…

So, what do you think?

Is WCAG 2.0 fit for legislation? Or do you also have concerns?

Have you experienced occasions where WCAG 2.0 has clarified exactly what you need to do to make your website accessible?

Or have you found it less useful?

Let us know by commenting below.

Want more?

If this article has been useful, please sign-up for the Hassell Inclusion newsletter to get more insights like this in your email every other week.