After offering my own thoughts on the Better Connected 2007 thing, where I mentioned some of the problems in trying to assess accessibility, particularly when it comes to measuring it across different websites. I’ve since volunteered to give someone else’s site a good going-over and do an accessibility audit for them — and provide them with a score.

I am very much aware this gives people the opportunity to shoot me down in flames. Fair enough. I shouldn’t be in the kitchen if I can’t take the heat, kind of thing. However, I thought it might help if I’m open about what I’m doing, it’s limitations and so on first…

Firstly, I’m not a professional accessibility auditing company. Secondly, I don’t have a disability. Thirdly, I don’t have access to the complete spectrum of assistive technologies. Finally, I don’t have access to a wide variety of test users. It’s just little old me.

Nonetheless, I’m confident that I can still do a pretty darn good job of auditing the accessibility of someone’s website. Oh, I’m not saying I’m in the same league as the Shaw Trust with their pan-disability checking, or in the same league as the RNIB with their See it Right audits, or even that I can compare to that excellent new body the Usability Exchange which connects website owners directly to disabled users for feedback. Now there’s an idea, huh?

So if I’m not this, and I’m not that, and I’m not the other, how come I think I can cut the mustard when it comes to accessibility auditing/testing?

Precisely the same way you get to Carnegie Hall. Practice.

I’ve been evaluating the accessibility and semantic construction of websites for some time now. I’m “the standards guy” at work (not to be confused with that standards guy) and have been for some time. I’ve one of the members of Accessites, and I’ve evaluated the accessibility of a good number of sites for that. I’ve also been a member of AccessifyForum for some time, where I’ve not only offered accessibility-related advice to others, but I’ve listened to what other people have had to say and have learned a lot. I’ve even tried to contribute to accessibility standards (but we’ll not talk about that).

Accessibility is something I understand, and accessibility testing is something I do, and I do well. That’s not being arrogant about it — many other people might be better — but I do have significant experience in the area.

So that’s my justification for why I feel I can do this. I understand the concepts, and I’ve had plenty of experience at applying ‘em. Hope everyone is happy with that.

What am I looking for?

I’ve already publicly — and frequently — made the point that WCAG compliance does not equate to “accessibility”, and that scoring and measuring sites according to some kind of accessibility metric is somewhere between difficult and pointless. So, purely in order to spite myself, I’m therefore going to score my accessibility auditing along some kind of metric.

Hah! That’ll teach me.

So what am I actually looking for?

Firstly, I’ll explain that I am only going to look at ten different pages of the site. From ten pages, I’m fairly confident — assuming that they are specifically chosen, and chosen carefully — that I can get a representative sample. I’m doing this because I’m going to look through the pages manually, and given that my testers compose of me and my two cats (and allowing them anywhere near anything called a “mouse” is not a good idea), I simply don’t have the resources to look at more pages than that.

WCAG Compliance

The first part of the actual audit will be to evaluate the sites against WCAG 1.0. Yes, I am aware that I have said WCAG isn’t great for measuring accessibility like this, but it is so far the only internationally recognised standard we have, and it is the one that people will normally reference when they are claiming that a site is inaccessible.

So a necessary part of the audit is to compare the site against WCAG 1.0 to see how it stands up. I’ll also offer my comments on the various checks to indicate how important I think they are, but — and this is important — the WCAG rating I give to the site will not reflect my personal weighting. It will simply reflect the WCAG conformance level that I believe the site has achieved. I wouldn’t be surprised to see a number of fairly accessible sites not even making it to single-A conformance, to be honest.

So that’s part 1 — the WCAG conformance level and breakdown of failures.

ThePickards Criteria

Part two of the audit will look more at what I believe to be important. In many ways this can be considered to be an edited version of the Accessites Criteria, but with some significant changes.

I’m then going to break these down into what I’m terming standard and important criteria. Fairly self-explanatory so far?

Important Criteria

1. Text must be resizable in modern browsers

2. Site renders logically and is usable without CSS, images or both being supported

3. Site is navigable and operable using just the keyboard

4. Links should be obvious and easy to use

5. All information available visually must be provided in the content (includes use of images and colour to convey meaning)

6. Language used is appropriate to the site’s content

7. Luminosity contrast ratio of foreground and background of at least 5:1 where information is being conveyed, or the luminosity can be altered to such a ratio by means of site customisations

8. Labels are explicitly associated with their controls

9. Site ‘works’ in the latest version on multiple modern browsers

10. New windows are not opened without somehow warning the user in advance

11. If data tables are used, the row or column headers are associated with the appropriate cells

12. Site ‘works’ whether or not javascript is enabled

Standard Criteria

1. The pages on the site validate

2. Deprecated elements and attributes are not used

3. The CSS used on the site validates

4. Tables are not used for layout

5. Links and form controls offer hover styling

6. Links and form controls offer active/focus styling for keyboard users

7. The main language used on the page is identified

8. Significant passages in alternate languages are marked up appropriately. Words or phrases commonly in use such as c’est la vie do not need to be, and neither do real names

9. Lists are used sensibly and where appropriate

10. Headers are used sensibly and where appropriate

11. Text is not right-justified

12. At least one of a sitemap, a search facility or a table of contents is provided where the site contains more than 20 pages

13. The first occurrence of an acronym or abbreviation on a given page is expanded by some means, including use of the acronym and abbr elements, expanding within the text, linking to a definition, or providing a glossary with an appropriate entry.

You’ll notice that there are a mixture of things in here. Some relate to usability; some relate to accessibility; some relate to how the site operates with different equipment and software (some people would call this ‘universality’, others make it a part of accessibility).

These criteria are what I judge to be important. If I’ve missed anything out, or you can think of some reason why I’ve prioritised something incorrectly, let me know. I’m certainly open to suggestion. On the other hand, this ain’t a democracy. It’s about how I am going to evaluate sites. I’ll certainly open to suggestions, but I reserve the right to ignore the ones where I’m not swayed by the awesome power of your arguments.

In some cases it will obviously be subjective, too. Do I feel lists have been used properly? Well what if I feel that on the whole, list markup is pretty goog but there are one or two things which maybe should have been marked up as a list but weren’t?

Well under WCAG, it’s a binary state. It either passes — you do everything right; or you make one or more mistakes anywhere and it fails. Under my marking scheme however, I am allocating points to the different critiera — up to 5 points for something I consider to be important, and up to 2 points for something I consider a “standard” criterion.

I’ll then tot these up to give an overall figure and work out the percentage of the maximum that was achieved. I’m going to express this as a percentage rather than a score, because this gives me a better comparison over time if I later revise the criteria I use.

And then we come to the third and final leg of the audit…

Simulated User Testing

As I’ve mentioned before, the user pool I have to select from is …um, me, so that does set some limits as to how useful this simulated user testing will be. Nevertheless, I’ll do my best. I plan to have a look at the site “as myself”, and decide on five realistic tasks that I’d like to complete using the site (including just finding out particular pieces of information).

I’ll then assess, on an admittedly subjective scale, how easy it was to complete each of those tasks — as well as detailing any problems I encounter — when using various tools to simulate accessing the site as though I were:

• Myself
• A user requiring the use of screen magnification tools
• A user requiring the use of a screen reader
• A user unable to use the mouse
• A user with colour blindness (looking particularly at Deuteranopia, as the Deuteranopia/Deuteranomaly types are the most common forms)

This will obviously not cover all of the possible situations — I’ve missed out the whole category of hearing problems and of cognitive problems, for a start.

Hearing problems I have deliberately missed: I don’t find that many websites rely on auditory information. Having said that, if I do test a website where I find auditory information to be a requirement, I will note this down and comment on it appropriately in the report, as that would be something pretty major.

Cognitive problems — ranging from mild dyslexia through to severe cognitive disability — are much more difficult to mimic and so I won’t even try. I have incorporated criteria in the second stage that relate to dyslexia and will help with milder cognitive disabilities, and so this will already be picked up in the scoring to some extent.

Nonetheless, I am aware that this is maybe a weak spot within my audit procedure and is somewhere where genuine user testing with disabled users would be of benefit. Not that that’s an option for me, being the one-man band with no resources that I am, but it’s something I could recommend to anyone I do an audit for…

And that’s it.. That’s what I’m planning to do. Does anyone have any constructive advice, comments or suggestions?