Presented at OZeWAI Conference, 9 December 2005

• Summary
• Introduction
• Stage 1: Source order expectations survey
• Stage 2: Task-based user testing
• Stage 3: Preferences survey
• Discussion
• Conclusion

Summary

Is page source order important to screen reader users? Recently, the idea of placing the informational content of a web page before the navigation has gained some currency. This paper reports on our research into the relevance and importance of page source order, skip links and structural labels for screen reader users.

Our results suggest that:

• Most screen reader users expect at least the main site navigation to be presented before the informational content of the page.
• The source order of a web page is likely to be of little relevance to the majority of screen reader users.
• About half of the screen reader users we either tested or surveyed found the use of skip links on the test sites helpful.
• All the participants indicated the inclusion of structural labels identifying the different levels of navigation on a web page was useful.

This paper proposes that when it comes to accessibility, the quality of the actual code on a web page is much more important than the ordering of the page content. Meaningful and appropriately marked up headings, descriptive link text and the clear identification of different levels of navigation, allow screen reader users to most effectively use their technologies when visiting a website.

Introduction

In general, web users come to a site with preconceived notions of how the site is likely to work based on their past experiences with other sites. As they move around the site, these notions are confirmed, refined or replaced. They learn the design and layout of the main site elements (e.g. navigation) and develop a procedural model for operating within the site.

For sighted users, developing this procedural model is usually relatively easy. An unusual, but well made site is often not difficult for a sighted person to learn, due in no small part to the visual feedback they receive.

For screen reader users, it might not be so simple. For about four years we have been observing screen reader users use websites. During this time, we have noticed that when many of these users find it necessary to develop a mental picture of the site they are using – they appear to rely heavily on preconceived notions of ‘what is a website’.

When a site does not fit these expectations, the inability to perceive visual cues such as colour differences and visual positioning can make it more difficult for screen reader users to learn the structure of a new site.

The appropriate use of Cascading Style Sheets (CSS) can greatly improve the accessibility of websites. Over the last few years, there has been a rapid increase in the awareness and use of CSS. Sometimes, proposed enhancements in accessibility through the advanced use of CSS are based on assumptions about what screen reader users want: assumptions that are not derived from research or tested.

CSS means that the position of page components on the screen is no longer dependent on the order of those components in the source code. This source order freedom has contributed to an emerging view, that for screen reader users it is better to present the informational content of the page before the navigation elements. This topic is picked up in the draft for the Web Content Accessibility Guidelines 2.0, released on 23 November 2005.

WCAG 2.0, includes Success Criterion 2.4.3, which states:

2.4.3 – Blocks of content that are repeated on multiple perceivable units are implemented so that they can be bypassed. (Level 2)

WCAG 2.0 – Guideline 2.4.3

The document, “Understanding WCAG 2.0 (Working Draft 23 November 2005)”, includes the following as one of the techniques that can be used to meet Success Criterion 2.4.3:

“Structuring the content so the main content comes first (in structure – but the default presentation may be a different order), and adding links to the blocks of repeated content.”

While this might seem logical from one perspective, we are concerned that it maybe based on an unproven assumption that ‘this is what blind web users want’. So, rather than help this assumption grow into some form of mythical fact by virtue of repetition, we decided to test it.

In addition to testing the appropriateness of presenting page content first, we decided to adopt a process that would allow us to also assess the usefulness of skip links and structural labels for screen reader users.

Skip links, which are traditionally used to allow screen reader users to by-pass site navigation, have been around for a while and are required in the Section 508 Web Standards. In recent times however, the value of skip links has been called into question.

For the last two years, we have been using structural labels to indicate the main components of a web page, such as global site navigation, sub-site (local) navigation and footer information. CSS can be used to prevent the structural labels, which are contained in heading elements, from being displayed on the screen, while still allowing screen readers and text browsers to present them.

Outline of testing process

A three-stage process was adopted:

• Stage 1: Survey screen reader users and text browser users to gain an insight into their preconceived ideas of how a site will be presented.
• Stage 2: Qualitative task-based user testing with a small sample of screen reader users using two very similar sites, but with different ordering of the content and navigation.
• Stage 3: Follow up survey with the original survey participants, asking them to examine and use two test sites in their own time and identify which they found the easiest to learn and use.

Stage 1: Source order expectations survey

As a first step, we decided to survey screen reader and text browser users to see how they expect the components of web pages to be organized. We restricted the survey to these groups of users since we felt that they both probably depend on a page’s source order rather than its visual appearance.

Our aim was to recruit as many participants as possible. Information about the project and requests for people to help were posted in a variety of forums. Unfortunately, we only received responses from two screen reader and five text browser users.

Eventually, with the help of Steve Faulkner from Vision Australia, we were able to recruit eighteen screen reader and five text browser users for the survey.

Each participant was sent an email outlining the aims of the project. They were then asked to draw on their experience of how websites are presented when providing a ‘yes’ or ‘no’ response to the following six statements:

1. I would expect the site navigation to the main site areas to be presented first.
2. I would expect the information content of the page to be presented first.
3. I would expect the site navigation to the main areas and the local navigation to pages within an area to be separate and easy to identify.
4. I would expect the site navigation to the main areas to be presented first, followed by the local navigation within that area, and then the information content.
5. I would expect the site navigation to be presented first, followed by the information content and then the local navigation.
6. I would expect the information content to be presented first, followed by the site navigation and then the local navigation.

Source order expectations survey results

The prime aim of the Source order expectations survey was to determine the extent to which screen reader and text browser users might have preconceived notions relating to the ordering of web page content. In particular, we were keen to discover if they expected the informational content of a page to be presented before or after the navigation elements.

The 18 screen reader and 5 text browser respondents all appear to have very similar expectations about how the content of web pages will be ordered. All 23 participants believe that when they visit a site, the main site navigation will be presented before the informational content of the page.

Furthermore, it appears that 20 of the participants expect both the main site and the local navigation to be presented before the content. A table presenting the combined results for the six survey statements on the ordering of web page content follows:

	18 Screen reader users		5 Text Browser users
	Yes	No	Yes	No
1. I would expect the site navigation to the main site areas to be presented first.	18	0	5	0
2. I would expect the information content of the page to be presented first:	0	18	0	5
3. I would expect the site navigation to the main areas and the local navigation to pages within an area to be separate and easy to identify:	4	14	3	2
4. I would expect the site navigation to the main areas to be presented first, followed by the local navigation within that area, and then the information content:	15	3	5	0
5. I would expect the site navigation to be presented first, followed by the information content and then the local navigation:	4	14	2	3
6. I would expect the information content to be presented first, followed by the site navigation and then the local navigation:	1	17	0	5

The results for Statement 3 suggest many of the screen reader users have experienced difficulties differentiating between site-navigation and local-navigation of sites they have visited. These difficulties probably contributed to the apparent inconsistencies in the screen reader user results for questions 4 to 6.

The problems some screen reader users have in identifying the different navigation menus on web pages were confirmed during follow up discussions with nine of the screen reader participants. It appears that when a screen reader presents the local navigation directly after the site navigation, some users find it difficult separating the two menus. The participants’ comments suggest the links in the footer section of the page are sometimes identified as the second level navigation menu on the page.

Interestingly, during these follow up discussions, four screen reader users volunteered the opinion that although navigation is usually presented before the page content, they felt that they would probably find that pages which present the content first easier to use.

Stage 2: Task-based user testing

The second stage of the project involved the direct observation of screen reader users performing tasks on sites with different source ordering. We felt that this would provide further insight into the potential importance of source order for screen reader users.

Four test sites were prepared, two versions of a site relating to Australian frogs and two relating to Australian birds. The ‘Birds’ test sites present the navigation menus first followed by the information on the page, and the ‘Frogs’ test sites present the page information before the navigation.

While the actual content of the ‘Frogs’ and ‘Birds’ test sites is obviously different, the nature of the site pages in terms of their length and the number and use of headings and subheadings is very similar. Also, each page of the sites has two levels of navigation, global (top level) navigation to the main site areas, and local (second level) navigation to pages within a site area.

• Birds (version 1)
• Birds (version 2)
• Frogs (version 1)
• Frogs (version 2)

The second versions of the ‘Frogs’ and ‘Birds’ test sites (Birds 2 and Frogs 2) were prepared with the aim of assessing whether the participants found skip links and structural labels useful. The inclusion of these elements is the only difference between the two versions of the sites. The second versions contain skip links to either content or navigation (depending on which is presented first) and headings identifying the different levels of navigation.

A standard task-based testing process was adopted, involving four representative users of screen readers. Each participant used the ‘Birds’ and ‘Frogs’ test sites for four different tasks that required them to visit different areas of each site.

Screen readers provide a wide range of facilities that allow users to easily identify and locate the headings, links and information on a page. Different screen reader users seem to employ different combinations of these facilities when using a site. One of the most common is to have the screen reader present all the links on a page in a list. While this is an effective technique for quickly navigating a site, it does tend to mean that the ordering of navigation in relation to page content may not be particularly relevant. For this reason, we asked the participants not to use this facility for some of the test tasks.

Four of the screen reader users, who had participated in the Stage 1 Source order expectations survey, were asked to participate in the Task-based user testing stage of the project. The selected participants are users of different technologies and/or have different levels of skill in using the technology:

• Very experienced screen reader (Windows Eyes) user who relies on audio output.
• Very experienced screen reader (JAWS) user who uses Braille output.
• A moderately experienced screen reader (JAWS) user.
• A screen reader (JAWS) user who has been using the technology for only a few months.

We observed each of the participants as they used the sites for the same tasks. They were encouraged to comment on how easy or difficult they found the sites to use. Following the tasks, the participants were asked to rate the difficultly of the navigation systems on the sites and how easily they were able to find the information they needed for the tasks.

At the end of the evaluation sessions, the participants were asked to nominate which site they found the easiest to use and rate the usefulness of the labels identifying the different levels of navigation.

Task-based user testing results

The four participants used the two ‘Frogs’ and two ‘Birds’ test sites for a range of predetermined tasks. The three participants with most experience using screen readers had no difficulties completing all the tasks in less than 30 minutes. The person with the least screen reader experience took a lot longer and did encounter a number of problems.

Each task-based user testing session was facilitated by one of the researchers and observed by the other two researchers.

While the survey sample is smaller than we had hoped for, analysis of the performance and comments of the screen reader users revealed significant differences in the strategies they adopt when using assistive technologies to access the test sites:

• The participant with the least experience in using a screen reader tended to rely on ‘reading’ the entire page and so the order of the content was important to her.
• The moderately experienced screen reader user would scan the first couple of pages and then use the technique of displaying a list of links from the page to move around the website. This participant would also often use the keyboard to skip down the headings on the page.
• The two most experienced screen reader users relied almost entirely on a variety of assistive technology facilities:
  • The Window Eyes user made extensive use of the screen reader’s ability to present a list of links and headings on the page. He also often used a screen reader facility that would jump him to the next line of plain text.
  • The Braille user mainly used the Braille Note to skip from the start of one line to the start of the next. He didn’t use the screen reader shortcuts as much, since they required him to move his hands from the Braille Note to the keyboard.

It appears from these observations that the more skilful a person is at using an assistive technology the more likely they are to primarily use the facilities provided by the technology when undertaking a task.

The main results of the Task-based user testing by the four screen reader users are:

• The major determinant in how well a participant was able to do the tasks was their level of skill (and experience) in using the assistive technology.
• For all but the least experienced screen reader user, the ordering of the page content was not an important issue.
• Two participants said the ‘Birds’ test sites, which present the navigation menus before the information on the page, were easier to use, one participant said the ‘Frogs’ test sites were easier to use, and the fourth said they were the same.
• The least experienced screen reader user found the ‘Frogs’ test sites, which present the content before the navigation, significantly more confusing and difficult to use than the ‘Birds’ test sites.
• The participant with moderate screen reader experience made extensive use of the skip links, commenting favourably on them. The participant with the least experience did not seem to understand the purpose of these links.
• The two participants with the least screen reader experience appeared to have difficulty distinguishing between the main site and local navigation on version 1 of the ‘Birds’ and ‘Frogs’ test sites, which do not have headings identifying them. However, the participant with moderate experience found the navigation headings very useful on version 2 of the sites.
• The four participants were asked to rate the usefulness of labels identifying the different levels of navigation on a seven-point scale (1 indicating “no use at all” to 7 for “very useful”). The averaged score for the four participants is 6, which suggests this may be an effective way of overcoming the difficulty screen reader users report in identifying different navigation.

Stage 3: Preferences survey

Following the task-based user testing, we felt it would be useful to obtain feedback on the different ordering of page content and navigation from a larger number of screen reader users. We were also interested to see if they found the skip links and navigation headings useful.

The 14 original Source order expectations survey participants, who hadn’t been involved in the task-based user testing, were contacted again and asked to participate in a second survey. Each participant was sent an email containing links to the second versions of the Frogs and Birds test sites, which contain the skip to navigation (or content) links and headings identifying the different levels of navigation. The email also contained the following test task for each site.

• Site 1: BirdsTask: What similarity is there in the feeding habits of the Purple Swamphen and the Green Catbird?
• Site 2: FrogsTask: What similarity is there in the habitat of the Cascade Tree Frog and the Common Mist Frog?

The participants were asked to look at the test sites and use each site for the specified task. To complete the tasks, it was necessary for the participant to locate and use two pages in different areas of each site.

The participants then answered the following four questions:

1. Were you able to complete the tasks on both sites? (If no, on which sites were you not able to complete the task.)
2. Which site did you find the easiest to use? (Frogs, Birds or both the same.)
3. Did you use the skip to content or skip to navigation links?
4. Did you find the headings identifying the different levels of navigation on the sites useful?

The participants were also asked to provide comments, which they might wish to make, relating to the questions.

Preferences survey results

Eight of the original fourteen screen reader participants used the two test sites before answering the survey questions. The collated results are presented in the following table:

Question	Result (n=8)
1. Were you able to complete the tasks on both sites? (Yes or No)	Yes: 8No: 0
2. Which site did you find the easiest to use? (Frogs, Birds or both the same)	Frogs: 3Birds: 2The same: 3
3. Did you use the skip to content or skip to navigation links? (Yes or No)	Yes: 5No: 3
4. Did you find the headings identifying the different levels of navigation on the sites useful? (Yes or No)	Yes: 8No: 0

The results for Question 2 do not indicate any overall preference by the survey participants for the informational content of a web page to be presented before the navigation elements. Some of the participant comments relating to this question follow:

Bird site slightly easier.

I found the frogs site better as I like to get an intro to a site before finding links and navigation.

Both the same

Both as easy as each other

Five of the eight survey participants indicated they found the skip to content or skip to navigation links on the site useful (Question 3). Some of the participant comments follow:

Yes, I always read through a new site to get a feel for it’s’ layout and then use any available navigation links.

Yes, after accessing the initial web pages of the sites.

I used them on the frogs site. Didn’t find I needed to on the birds

NO I didn’t use these links I just went into each of the categories to find what I was looking for.

All eight participants found the headings identifying the different levels of navigation useful. Some of the participant comments relating to Question 4 follow:

Yes, headings are always useful.

Yes, extremely!

Yes. I found these very useful.

Discussion

The growing awareness that CSS can be used to break the nexus between the position of page components on the screen and the order of those components in the source code has lead to increasing advocacy for this technique it recent times. In part this appears to be based on a belief, that the accessibility of websites for screen reader users is improved when the informational content of the page is presented before the navigation in the page source code. Indeed, there are suggestions that this is what screen reader users would prefer, and in some case this is what they need.

As new techniques become available, allowing impossibilities of the past to become possible, the desire to promote these techniques and push the boundaries even further can result in a tendency to equate what is possible with what is desirable.

In our view, consideration of whether any particular web-coding technique is likely to improve the usability of websites for people who rely on screen readers, needs to involve four fundamental, but interrelated questions:

• How do screen reader users expect a web page to be presented?
• How important are these expectations?
• How are screen-reading technologies used?
• Do the proposed techniques help screen reader users use a website or find information on a web page?

Given that the vast majority of web pages present the informational content of the page after the navigation, it is not surprising that this was the expectation of the 23 participants (18 screen reader and 5 text browser users) who completed the (Stage 1) Source order expectations survey.

Since screen reader users appear to expect the navigation to be presented before the content of the web page, how important is this expectation?

For people who are able to perceive the graphical presentation of a web page, the position (or order) of material on the screen is an important usability consideration. However, for three of the four screen reader users we observed, the order the material in the test sites was presented by the screen reader did not seem to be important. The participant with the least screen reader experience, whose loss of vision was relatively recent, did appear to rely more on her preconceived notions of how a site should look, including the presentation of navigation before content. She had considerable difficulties using the ‘Frogs’ test sites, where the content is presented before the navigation.

The four screen reader users who used the test sites (Stage 2) and the eight users who responded to the Preferences survey (Stage 3) were asked to nominate which site they found the easiest to use:

• Four nominated the ‘Birds’ site which had the navigation before the content.
• Four nominated the ‘Frogs’ site which had the content before the navigation.
• Four said they were both equally easy to use.

These results do not indicate any clear preferences relating to the order of navigation and content by the participants in this project. And, we did not find much evidence to support the notion that, ‘blind web users want to have page content presented first’!

Following our research, we feel that the order of the material on a web page is likely to be of little importance to most screen reader users. However, for the inexperienced screen reader user, presenting the informational content before the navigation is more likely to be a source of confusion rather than a benefit.

Screen reading technologies provide the user with many different ways to use a web page. It appears to us, that when most screen reader users embark upon navigating through a site or finding specific information on a page, they are likely to use the screen reader facilities (shortcuts) they are familiar with, rather than scanning through the content of the page starting at the beginning. It is worth noting however, most of these screen reader shortcuts, such as obtaining lists of links or headings depend to a large extent on the use of well structured code, which conforms to the Web Content Accessibility Guidelines.

The results of the Task-based user testing (Stage 2) and the Preferences survey (Stage 3) allow us to throw a little light on to the value of skip links, a subject which has roused some discussion in recent times, and the usefulness of including structural labels to identify the different levels of navigation on a page.

Of the four people we observed during the Task-based user testing (Stage 2), one person relied very heavily on the skip links and found it frustrating that they weren’t available on the alternative version of the sites. Two of the participants did not use them at all, relying instead on different screen reader facilities. The person with the least screen reader experience did not seem to understand the purpose of the skip links and did not use them.

Of the eight Preferences survey (Stage 3) respondents, five indicated they found the skip links useful.

When our observations of the site testers are combined with the results from the Preferences survey, 50% of the screen reader users who participated found the inclusion of skip links useful. While we do not think this means that half of all screen reader users are likely to use skip links when they are provided, it does suggest that a significant number of people are likely to find them beneficial.

One of the most striking results from this project concerns the use of structural labels for the different levels of navigation. When web pages are displayed graphically, it is usually easy for a sighted person to clearly identify the different navigation menus, for example main navigation across the top and second level navigation down the side.

But this is not always case for people who are unable to perceive the visual appearance of web pages. Fourteen (77%) of the screen reader users who participated in the Source order expectations survey (Stage 1) indicated they sometimes find it difficult differentiating the site navigation from the local navigation on web pages. Observation of the four participants, who used the test sites prepared for this project (Stage 2), highlighted this issue. We noticed that when they were using the pages without structural labels for the navigation, the presentation of the local navigation items immediately after the site navigation caused some problems for the two least experienced screen reader users.

Although it is likely many of the screen reader users who participated in the Task-based user testing and Preferences survey (Stages 2 and 3) were not use to the different levels of navigation on a web page being labelled, all said they found inclusion of these structural labels on the test sites very useful.

Conclusion

Source order

It appears that when visiting a web page, most, if not all, screen reader users expect at least the main site navigation to be presented before the content of the page. There appears to be little evidence to support the view that screen reader users would prefer to have the content presented first, or find sites easier to use when this occurs.

It is our view, that a continuation of the practice of placing navigation before the content of the page will benefit some screen reader users, in particular those users who are still developing their skills with the technology.

It is probably desirable however, to present the content of the page before extraneous information, such as advertisements and related links, as well as the page footer.

Skip links

The wide range of screen reader options for accessing page content means that many experienced users of these technologies do not need to use skip links. But for less experienced screen reader users, it seems clear that many are likely to find skip links a useful device for moving directly to specific sections of the page. In our opinion, websites should continue to provide skip links at the top of pages.

Structural labels

All of the research participants who used the test sites found the structural labels for the different levels of navigation useful. The inclusion of structural labels seems to be an effective and relatively simple solution to the significant problem some screen reader users have in identifying the different elements on a page. Also, through the use of CSS it is possible to include these labels without affecting the visual appearance of the page. In our opinion, structural labels should be used to help describe different components of the page to screen reader users.

Finally, the importance of adhering to the Web Content Accessibility Guidelines cannot be overstated. The use of semantically correct and valid code, meaningful and appropriately marked up headings and descriptive link text will enable screen reader users to make the most effective use of their technologies when visiting a website.