Gian Sampson-Wild on WCAG 2.0’s concept of testability
In the latest A List Apart article, Testability Costs Too Much, Gian
Sampson-Wild points out why she believes WCAG 2.0’s tenet of testability to be fundamentally flawed.
The article’s example of how Guideline 1.1.1. (relating to text alternatives to non-text content) was watered down
for testability purposes is interesting. However - and maybe I’m missing the point here - I don’t quite follow why the form that’s now in the latest working draft is supposed to be bad:
All non-text content has a text alternative that presents equivalent information, except for the situations listed below.
Gian points out the apparent problem of consistently applying testability throughout the entirety of the guidelines:
There are many instances in WCAG2 where success criteria are actually not testable—and the Working Group knows it. In Bugzilla, the Working Group’s issue tracking system, there is a tracked issue lodged by three Working Group members that reads: “In particular, the current wording [of WCAG2] does not seem testable. Words such as, “key,†“consistent,†“predictable,†“inconsistent,†and “unpredictable†are subjective.†Yet these terms have been used throughout WCAG2—there’s even an entire guideline that rests one of these subjective, non-testable terms:
Make Web pages appear and operate in predictable ways.
Now, despite the use of subjective terms, I’d still maintain that those success criteria are actually testable in the context of human testing:
Reliably Human Testable: The technique can be tested by human inspection and it is believed that at least 80% of knowledgeable human evaluators would agree on the conclusion.
The WCAG Samurai Errata are mentioned as an example of a set of guidelines that don’t rely on testability but do give developers clear instructions on how to comply with relevant requirements
. Let’s look at two examples from the Samurai:
Guideline 2. Don’t rely on colour alone [...] For actual content, do not set one confusable colour on top of another.
The Samurai mention confusable colour
, but then just give an example of red type on a green or black background
. Who says what is and isn’t a confusable colour combination? And, without a clause on testability, are the Samurai errata not more ambiguous and open to interpretation?
Guideline 3. Use markup and stylesheets and do so properly [...] Requirement to use HTML semantics [...] For ambiguous cases, you must use the elements and attributes that most closely fit your content even if other authors would disagree. [...] Other authors’ disagreement with your choices is not relevant to these errata. (emphasis added)
This almost sounds like “anti-testability”…but how is that a clear instruction
?
Don’t get me wrong, Gian’s point that, because of testability, many valid guidelines and techniques - particularly for users with cognitive disabilities - did not make it into, or were removed from, WCAG 2.0 is indeed valid. The fact that, at least in part, testability was probably pushed through in the WCAG Working Group by those members (stakeholders?) involved in producing authoring/testing tools is deplorable, and certainly shows where some of the members’ priorities and (not so hidden) agendas lie. However, I personally can’t see how that leads to Gian’s conclusion:
With the publication of the WCAG Samurai Errata, the web community finally has a choice—and if WCAG 2.0 continues to be unworkable, developers will simply turn to another set of guidelines
…particularly since even the Samurai document (just like the current WCAG 2.0 Working Draft) acknowledges that:
Compliance with WCAG+Samurai cannot be a claim of full accessibility to people with cognitive disabilities.
Maybe it’s just me, but somehow I don’t see how guidelines can realistically be implemented and enforced (say in the context of a large university with multiple authors), unless there are actually testable sets of guidelines and success criteria / checkpoints?
