Accessibility: More than WCAG compliance – CSUN2013 Talk

On 27 February 2013, I gave the presentation “Accessibility: More than WCAG compliance” at the 28th Annual International Technology and Persons with Disabilities Conference (CSUN 2013).

Many countries now use the Web Content Accessibility Guidelines, Version 2 (WCAG 2) as the benchmark for determining the accessibility of web content, and the regulators in these countries require government agencies and corporations to comply with the WCAG 2 Success Criteria. Since the release of WCAG 2, there has been considerable debate about how to determine the accessibility of websites: either using a checklist to determine the level of compliance/conformance with the specific Success Criteria; or, undertake some form of user testing by people who have different disabilities and/or who rely on different assistive technologies. Both approaches have their strengths and limitations, and neither can provide a reliable declaration about the accessibility of a site on its own.

Unfortunately web content accessibility is being increasingly viewed solely from the perspective of how well a site complies with WCAG 2 rather than how well people with disabilities can access the content.

During this session, I outline a new Accessibility Priority Tool (APT) that was developed in early 2013. This tool (Excel Worksheet) aims to help organizations and developers identify likely access barriers to web content, and prioritise their efforts to correct them.

I would like to thank Andrew Downie for his help in preparing the APT Excel worksheet, also the many people who attended this session for their interest, comments and questions.

(NB: In this Accessibility Priority Tool Excel worksheet, Column I is hidden. This column is used to calculate the worksheet ‘Access Barrier Advice’ for each item that is presented in Column F

The slides from the presentation and the speaker’s notes follow:

Speakers notes

Slide 1

For those who aren’t familiar with the Web Content Accessibility Guidelines, when I use the acronym WCAG during this talk this what I am referring to.

Slide 2

On the screen we have a heading, “Tale of two sites”, followed by the opening of Charles Dickens book with a similar tile:

“It’s the best of time, it’s the worst of time. We have everything before use, we have nothing before us.”

Okay, say we have two sites:

  • One with a few images in the content area that are missing alt attributes
  • The other, where the developer has gone for the subtle look, so contrast ratio for all the text is 3.2:1

For most of us neither of these sites is likely to present any accessibility problems.

  • So which is the least accessible?
  • Which is the most likely to cause users problems?
  • Or to continue to take liberties with Charles Dickens’s Tale of Two Cities, which developers should go to the guillotine and which to the Bastille?

Slide 3

Now this is the people’s court, you have to choose.

Bear in mind however, that under WCAG 2.0 failure to include alternatives for non-text content is a Level A issue – an absolute no-no, but only a few alts are missing.  On the other hand, colour contrast is at Level AA and a contrast ratio of 3.2:1 would be fine for text that is considered “large”, but all the text on the site is too low.

Okay, hands up those who want to give the thumbs down, to mix historical metaphors, to the missing alts.

And now, those who feel our subtle colour scheme should go to the chopping block.

And, the rest of you are just going to sit there knitting, watching the action.

Slide 4

Now, let’s consider our two examples from the perspective of some potential site users:

For people who rely on a screen reader, the missing alts would be a serious problem if the images were important, but only a minor irritation if they don’t convey any significant content or functionality.

But of course, the lack of adequate text alternatives can affect other people such as those with slow internet connection who turn off images in order to reduce the time it takes for pages to load. They might be cross if they have to turn on images just to see a picture of your aunt Mary or Uncle Bob.

For most screen magnifier users, the failure to include a couple of alt attributes is not likely to be a problem, but inadequate colour contrast could well be.

And, those with an impaired ability to distinguish colours may not worry about missing alts at all, but the low contrast ratio could render all the text totally unreadable.

Slide 5

For the individual web user, the accessibility of a site depends on many factors:

1. For a start there are the personal barriers to web access that they might face, for example:

  • technological or environmental limitations,
  • a physical disability that might necessitate the use of an assist device, or
  • a cognitive, learning or language problems that make the content of a page hard to understand.

2. Then we need to consider the actual quality of the website code. Has the site been prepared in a way that will allow the content to be accessible.

3. Then there is the ability of the user-agents, such as browsers and screen readers, to present the content of the page in a way that the user can perceive.

4. And finally, how skilful is the user in using the browser and/or assistive technology they rely on to access the web. It is wrong to assume that most people know how to use the accessibility features of a browser or computer operating system, or that all assistive technology users are expert users of their technology.

Slide 6

Although web accessibility should be primarily about how well people can access content, over the last few years the conversations have been increasingly about complying with rules, in Australia at least, and maybe many other countries as well.

For example, at the beginning of 2010 the Australian Government adopted WCAG 2, requiring the sites of all government agencies to move to double A compliance by the end of 2014 as part of a National Transition Strategy.

But compliance with a set of predetermined guidelines, no matter how comprehensive they might be, is no guarantee of accessibility, a fact well recognised by the W3C in the introduction to WCAG 2.0.

Slide 7

How often have you had clients ask does my site comply WCAG? They often have no real understanding of what it might mean, just that it something they are supposed to ask about.

 Slide 8

And even within the web community WCAG 2.0 is often misunderstood.

You still hear people with many years experience producing high quality web content or tools saying things like “you can’t use JavaScript if you want to comply with WCAG”.  The whole move to technological neutrality in WCAG 2 seems to have washed over them without raising so much as ripple of understanding.

Same thing with the difference between ‘normative’ Guidelines and Success Criteria andinformative’ Techniques. Many people interpret the WCAG techniques as Rules, with a capital R; Even in discussions of topics on the WAI interest group mail list.

I sometimes come across very conscientious and well meaning developers, who become pre-occupied with the idea of complying with as many techniques as possible for each Success Criteria. Often this is unnecessary and sometimes counter-productive when it comes to improving overall accessibility of a page element.

Slide 9

Working out if a page is likely to cause any accessibility problems is not rocket science, but then again, it is not a piece of cake either.

With a bit of knowledge and a few tools it is pretty easy to get a good idea if a site does or does not fully comply with WCAG 2. However, determining the accessibility implications of non-compliance, and what can be done to overcome it, does take a bit more work and experience.


Slide 10

When it comes to determining the accessibility of websites, the discussion is often polarised around two simplistic choices:

  • A compliance or conformance- based approach that usually involves a checklist of criteria;
  • or, some form of user testing by people who have different disabilities and/or who rely on different assistive technologies.

Both approaches have their strength and limitations, and neither can provide a reliable declaration about the accessibility of a site on its own.

Ideally, any thorough accessibility evaluation should involve both approaches, but the constraints of budgets and time often mean that this is not possible.

Slide 11

In order to help illustrate some of the complexities involved in determining if some web content is accessible or not, I have prepared this very simple and innocuous looking form. The form has just five text inputs, imaginatively labelled text one, text two etc.

The accessibility of this form, and all other web content, can depend on a variety of factors, but fundamentally it comes down to a question of how well all web users are able perceive and interact with the component.

Screen reader users, for example, need to be able to easily identify the purpose of each form input, and people who are unable to use a mouse need to be able to access all elements of the form with the keyboard.

But, for the purpose of this talk I am not concerned about anything other than how these five form inputs are identified, or labelled.  They all look the same visually, but the ways they have been coded is different, and are they all accessible?

What sort of result do we get when they are tested with screen readers?  And, what happens when we test for strict compliance the Web Content Accessibility Guidelines?

But first, a quick reminder of the relevant WCAG 2 Sufficient Techniques for identifying form inputs.

Slide 12

It seems to me, that there are three Level A Success Criteria that relate directly to the identification of form inputs:

  • 1.3.1 Info and relationships
  • 3.3.2 Labels or instructions
  • 4.1.2 Name, role, value

There is also the Level double A Criterion 2.4.6 Headings and labels, which canvasses much the same a 3.3.2 when it comes to identifying form inputs.


Slide 13

At this time, the W3C suggest two Sufficient Techniques for identifying form inputs:

  • H44: Using label elements to associate text labels with form controls (HTML)
  • H65: Using the title attribute to identify form controls when the label element cannot be used (HTML)  

Slide 14

And, one Advisory Technique which uses WAI-ARIA

  • ARIA1: Using the aria-describedby property to provide a descriptive label for input controls (ARIA)

So, how does our five input form perform with screen readers.

I will run through the form using NVDA, a free screen reader which I am sure all of you are very familiar with and hopefully support with the occasional donation. If not, please do because they whole thing is run by a couple of very bright young Australians with very limited resources.

I use NVDA with an Australian voice, Nicole, from Ivona. I use the Australian voice because to my ear it is easier to understand than the standard e-speak voice.

Slide 15

A checklist conformance review usually involves someone manually checking a site or selected pages from a site often using a variety of tools.

You can run automated tools right through a site to see how well it conforms, but these just provide a snapshot or overview of likely issues, many of which will require later manual checking to verify.

Tools are just that, tools!  They are something to help you do what you need to do. Accessibility tools can only measure things that can be measured programmatically, like, does an image has an alt attribute. But, they can’t tell you if the alt attribute is appropriate.

These are just some of the tools I use when checking the accessibility of sites. But today I am going to use my old favourite the WAT toolbar developed and maintained by Steve Faulkner, who now works for the Paciello Group.

Slide 16

Well with NVDA, it seems that all of the inputs can be identified when just reading down the form. And,  all apart from number 4 can be identified with ease when filling the form in.

When we check the page with the WAT toolbar it indicates:

  •  the first input has an explicitly associated label,
  •  inputs two and five have label elements that don’t contain the “for” attribute,
  • And, input 4 appears to have an “id”, but no label.

Slide 17

The WAT tool bar, or an examination of the page code, quickly tells us that Input three is identified with a title attribute and, as we saw earlier, title can be used when it is not possible to use a label.

However in the form we do have a visible label for text three, but if we leave aside the question of whether or not we should in fact be using the label element in this case, we know that title attribute can be used to identify form inputs and still comply.

So Input 1 has an explicitly associated label and Input 3 uses the title attribute, but  what about the other three inputs.

Slide 18

Text two was clearly identified by NVDA since both the label and the input are inside the label element. Although, this is quite acceptable from a HTML specifications point of view, it is not provided as one of the technique examples in WCAG 2.0.

As far as I can see, the reason it is not cited as a Sufficient Technique is because some years back screen reader support for this was patchy.  But with modern screen readers today, it seems to be pretty good – so let’s call that a MAYBE.

Slide 19

With Text Four, the answer is simple: Since there is no label and no input title attribute, there is no accessibility API for the screen reader to hang on to. So that is clearly a NO.

Number five is more interesting because here we move into the realm of WAI-ARIA and the advisory technique about using ARIA-DESCRIBEDBY to provide descriptive labels. Even though this is an advisory technique only, it appears to be well supported by recent versions of commonly used screen readers and is consider an acceptable way of complying by the WAVE tool, so I think we can call that a YES.

Slide 20

So back to the question: Are all these form inputs accessible?

Well, they can all be identified by modern screen readers at least. And all, with the exception of number 4, have identifying information that can be exposed to screen readers via the API.

But when it comes to WCAG 2, in my opinion only text input one, which uses an explicitly associated label, unambiguously complies with WCAG 2.0 Success Criteria, however I think we can say YES to Text One and Text Five.

At the other extreme, text input four unambiguously doesn’t comply. So that is clearly a NO.

The remaining two are open to interpretation and discussion. For me, things like this are the exciting challenge with web accessibility today.

Slide 21

When WCAG 1.0 was released in 1999, Ajax was just a cleaning product and Prince told us it was a great time to party.  Back then Web content technologies and assistive technologies were much more primitive than what they are today and it was relatively easy to set criteria for determining what was accessible.

A good starting point was to just say if it ain’t basic HTML forget it, or provide an alternative. And, none of this fancy JavaScript stuff thank you very much!

Oh how the world has changed, and it’s time to stop pretending its 1999.

Now, web developers and clients are pushing the accessibility envelope everyday by wanting to include more sexy elements on the page and greater interactivity.  For example, during the last year or so, there seems to have been a growing fascination with providing sliders for users to input information and then displaying the results in a graph.  I can’t for the life of me see what is wrong with text inputs and a data table; but then I guess I am just a bit old-fashioned.

I now want to talk about another way of identifying accessibility issues that I have been playing with for the last year or so.

Slide 22

While checklists of criteria may provide an indication of whether or not a site complies, they don’t usually provide much information about the likely impact non-compliance may have for web users with disabilities.

During 2011, I began looking at ways we might consider the accessibility of web content from the perspective of the likely access barriers it might present to user.  One of the main motivations for doing this was to find a way of helping clients understand the significance of accessibility issues on their sites and prioritise their efforts in correcting them.

In part, this was a reaction to the look of bamboozled panic the acronym WCAG caused on the faces of some clients. Sadly, sometimes as a result accessibility experts who seem more interested in generating fear, and perhaps work, rather than helping clients make the content of their sites more accessible to as many people as possible.

At the end of 2011, I suggested the Access Barrier Score system which aims to provide a measure of how often barriers to accessibility occur in a site, and the likely seriousness of those barriers. This was the precursor of the Accessibility Priority Tool which I will outline in a few minutes.

I would like to stress, neither the 2011 Access Barrier Score or the 2013 Accessibility Priority Tool are intended to replace WCAG 2.0, but to be used in conjunction with it.

Slide 23

Before discussing the recent Accessibility Priority Tool, I think it might be worth looking briefly at the original Barrier Score system.

The Access Barrier Score system is basically an excel worksheet containing 26 common accessibility barriers.

The worksheet records the incidence (or frequency), and severity of each potential access barrier as determined by an experienced accessibility evaluator. These scores then generate a “Remediation Priority” ranging from ‘Critical’ to ‘None’ which can be used by developers and site owners to prioritize those issues that need to be addressed. I am most grateful for the help I got from Andrew Downie in making the Excel sheet work.

Over the last 12 months I have been using the Access Barrier Score when reviewing the accessibility of websites and I have provided a completed ABS sheet in conjunction with my reports on the accessibility of sites.

The feedback from clients and web developers to the ABS sheet has been generally very positive. Most clients reported the worksheet made it easier for them to see the extent and severity of problems and they also found the Remediation Priority generated by the system useful.  Over the year, I have also received suggestions from clients, developers and accessibility practitioners about how the system might be improved.

Slide 24

Here’s a brief overview of the changes that I made in preparing the Accessibility Priority tool earlier this year:

First,  the labels of a few columns has changed, most importantly the old “Remediation Priority” has been replaced with  “Access Barrier Advice”. This was mainly done to  stop people being too literal when interpreting the information in this column

I have also introduced a new column, “Importance Ranking” which I will talk about in a few minutes.

When preparing the original Accessibility Barrier Score worksheet, I was very keen to keep the number of barriers as low as possible in order to reduce the risk of information overload. This meant that some issues got combined and others were overlooked. For example issues relating to keyboard operability were contained within one barrier.

The 26 barriers identified in the 2011 worksheet have been expanded to 33 with the Accessibility Priority Tool.

Slide 25

Also, as a result of feedback, the priority tool introduces two new categories of access barriers.

  • Use of Other Technologies: This has been included because many sites now use third-party components such as social networking tools or ticketing systems, that are often inaccessible.  And, sometimes the major, or only, cause of accessibility problems on a site relate to inaccessibility of these tools, so I thought it would be useful to provide a way of highlighting them.
  • Device Usability: When reviewing the accessibility of sites I am being increasingly asked to comment on how well they will work on an iphone or tablet, so I have introduced this new category as a way of providing some basic feedback on the usability of sites with different devices.

Slide 26

I now want to talk about the new Importance Ranking column.

I work with many different organizations, ranging from small not-for-profits with perhaps one part-time web worker, to government agencies and corporation with large web teams, numbering in the hundreds. And, while the easy to understand information provided by the original Access s Barrier Score system was generally useful, among the feedback I got from the larger organizations was the desire for a tool that could reflect their requirements and capabilities, as well as the needs of their site users.

As such, the focus of the Accessibility Priority Tool has shifted from being something for individual accessibility evaluators, to a tool that can be used by large organizations where many different people may be responsible for determining and maintaining the accessibility of the web content.

2013 Accessibility Priority Tool allows the organization, and not just the person doing the accessibility evaluation, to have an input into the perceived importance of different accessibility barriers in the worksheet. Thereby, bringing greater consistency in the prioritization of issues from the results obtained by the different people who evaluate the accessibility of web content.

Importance Ranking should be determined well in advance by the organisation, depending on their circumstances and resources, e.g:

  • Regulator want to set national of industry agenda
  • Organizations meeting the specific requirements, and the needs of their users and site audience
  • Organisations highlighting the issues that they have the in-house capabilities to address immediately.

Slide 27

Determining the Importance Ranking for each access barrier, will require an awareness of the needs of the organisation and primary audience for the site, as well an understanding of the general principles of web accessibility.

I also believe that people using the Accessibility Priority Tool worksheet when reviewing the accessibility of a site should have experience in evaluating web content accessibility, and I imagine they will probably use a range of accessibility testing tools and a screen reader, for example NVDA.

I am going to now walk through the Accessibility Priority Tool Excel worksheet. (The  following slides outline the issues discussed during the demonstration).

Slide 28

Starting, with the Importance ranking (column H).

As I just said, this should be done in advance. With large organisations and government agencies this will probably be done by the in-house team responsible for determining and monitoring the accessibility of sites.

For small organisations it should be undertaken by an experienced accessibility evaluator, but it should still be done in advance of the actual evaluation and in close consultation with the client organisations.

(NB: The Importance Ranking values that are already entered into the worksheet are my perceptions of the relative importance of issues. These can, and should, be changed as required to meet the specific needs of different organisations.)

The Importance Ranking values are used by the Excel formula when calculating the advice generated about each access barrier.

I envisage that once the Importance Rankings have been decided and entered, this column will usually not be displayed when the Excel worksheet is being used during the evaluation of web content.

Slide 29

A person using the worksheet when reviewing a site will mainly be concerned with the Incidence score, column D, and the Severity Score, Column E.

Incidence score is a measure of how frequently the person using the tool believes that a particular barrier occurs on the site, or how frequently the way a site component is used does not meet the relevant accessibility requirements. The score is not a raw measure of how often a problem occurs, but rather an indication of how often, in percentage terms, the use of particular site component is likely to cause problems.

While evaluating the site, the person doing the evaluation enters an Incidence Score in Column D for each access barrier item using a five point scoring system:

0 – There is no incidence or occurrence of a failure to make the component accessible.

1 – The use of the page component or element causes access problems up to 25% of the time.

2 – The use of the page component or element causes access problems between 25% and 50% of the time.

3 – The use of the page component or element causes access problems between 50% and 75% of the time.

4 – The use of the page component or element causes access problems more than 75% of the time.

A few examples:

  • If there are 10 images, and 4 of those images have no alt text, the lack of a text alternative could cause a problem 40% of the time images are used, so the incidence score would be 2.
  • If a site has just one CAPTCHA and it is inaccessible; then 100% of the times CAPTCHA is used could cause a problem, so the incidence score would be 4.
  • If a site contains a number of pages with good content headings and sub-headings that use the correct mark-up, but on one page there are a few sub-headings that don’t use heading elements, then incidence score for the use of heading elements would be 1.

Slide 30

The Severity score (Column E) is used to record the likely impact the accessibility evaluator believes each barrier will have for someone with a disability.

The impact is rated with a Severity score of 1 to 5, where 1 is a minor inconvenience, and 5 indicates an extreme problem that will totally prevent some people from accessing the site content or functionality. For example:

1. Minor inconvenience: Not likely to prevent anyone from accessing content, but could be a minor irritant.

2. Minor difficulty: Not likely to prevent anyone from accessing content, but could affect the ability of some people to use a page.

3. Average difficulty: Could make it difficult for some people to access content and use a page.

4. Major problem: Could prevent some people from accessing or using page content.

5. Extreme problem: Will prevent access to sections of the site or the ability to perform required functions.

Allocation of the severity rating will of course be subjective, and as such, is always liable to be influenced by the experiences and knowledge of the person making the decision.  This is one of the reasons why I believe it is important for anyone using the suggested Accessibility Priority Tool to have some knowledge of accessibility and assistive technologies.

Slide 31

The Access Barrier Advice is determined by the APT tool from an Excel formula which uses:

  • The Incidence and Severity scores entered by the accessibility evaluator, and
  • The Importance Ranking values entered by the site regulator or owner in advance.

The Access Barrier Advice for each item is then presented in Column F using the following terms: None, Low, Medium, High, Very High, and Critical.

NB: The Access Barrier Advice generated by the APT worksheet is just a suggestion and as such should not be solely relied upon. Since, the advice may not accurately reflect the significance of a serious or important issue when the incidence is low.  For example, a critical image with a missing text alternative among many (less important) images with good text alternatives.

Slide 32

The Accessibility Priority Tool is not intended to be an automated process that can tell you what to fix on your site and when. Rather, it is a tool to help site managers make these decisions.

The comments section (Column G) is an important part of the tool since it allows the evaluator to provide their subjective impressions of the likely impact of potential access barriers, and highlight those that they believe will be most important.

For example, as I just mentioned, the Access Barrier Advice generated by the Excel formula may under report the importance of critical issues that occur infrequently: If there is one critical image with a missing text alternative among many less important images with good text alternatives, the APT will generate a maximum Access Barrier Advice of “High”.

However the image might be absolutely essential to the ability of people to use the site, say a login button, and this fact would be recorded in the comment section so that it is available when decisions are being made about prioritising efforts to correct accessibility issues on the site.

Slide 33

I would like to finish with a few comments about some of the lessons I have learnt and some points for discussion.

  • First, to pick-up on something I have already mentioned, I believe it will be important for the setting of the Importance Rankings to be done by someone with knowledge of accessibility. And, the person using the worksheet when reviewing a site should also know about accessibility.
  • Second, given the complex nature of websites and diversity of user needs, I don’t think it is possible to make an automated system that can reliably priortize issues for remediation. The outlier issues, like low incidences of critical failings or high incidences of minor irritants are always likely to present anomalies in the results.
  • Third, finding the balance between providing enough likely barrier items to represent the needs of people with disabilities, and the desire to have a tool that is not too daunting to use or difficult to understand. I am not sure I have it right, so if you want to try out the tool feel free to add, remove or edit the access barrier items as you wish.
  • Fourth, don’t use decimal points when preparing Excel formulas! Many thanks to Detlev for identifying this problem in the worksheet and it will be fixed soon, once again I hope with Andrew’s considerable assistance.
  • And finally, I would like to stress once again, this should not be seen as an alternative to WCAG when it comes to evaluating the accessibility compliance of sites. It is just a tool to help in the process of identifying and priortising issues that should be addressed.

In my experience, just telling a client or web developer that something is a problem they must fix can often lead to greater resistance or hostility towards the whole concept of web accessibility.  Most site owners and developers want to obtain some indication of the severity of the various problems you might see on their site so that they can prioritize their remediation efforts, for not all accessibility failings are equal.

Slide 34

I think this tool may be particularly useful to organisations that have a number of people monitoring the accessibility of sites. It considers both the needs of the organisation and the audience for the site, as well as input from experienced accessibility evaluators, when determining advice relating to each of the potential accessibility barriers.

Organisations can then combine this advice with knowledge of their own in-house accessibility capabilities when determining those issues they might be able to address quickly (the so called low-hanging fruit) and those that might take longer and require greater resources.

If you are interested in trying the Accessibility Priority Tool please do so.  Also feel free to modify it as you like.

All I ask is that you give me some feedback about how well it works and any suggestions for how it might be improved.

Slide 35

Thank you,

And, I think we have few minutes it there is anything you would like to comment on or discuss about both the Accessibility Priority Tool or my general concerns relating to the tendency of clients and regulators to view accessibility as just a series of items on checklist that have to be ticked-off.


  1. Pingback: The Great Big List from the 2013 CSUN International Technology & Persons with Disabilities Conference | Curb Cut

  2. You could definitely see your skills in the article you write.

    The sector hopes for more passionate writers such as you who aren’t afraid to mention how they believe. At all times go after your heart.

    my web-site :: Bradly

  3. That is a very good tip particularly to those new to the blogosphere. Brief but very precise info… Thank you for sharing this one. A must read post!

Leave a Reply

Your email address will not be published. Required fields are marked *