Alternative Text

All non-text content, images, and actionable components are labeled with descriptive alternative text unless they are purely decorative. Screen readers use the alternative text provided by the website or application to read to the user when the user focuses on a component. Following is how this looks on a website (Figure 3-1), on Android (Figure 3-2), and on iOS (Figure 3-3), along with code samples for each:

Web:

 <img src="bedroom.png" alt="A bedroom in a remote cabin with a bed, table, and two windows">

Or

<ImageView

 ...

 android:id="@+id/detail1" />

 android:contentDescription="A bedroom in a remote cabin with a bed, table, and two windows" />

Adding a label is not enough to make the experience usable. It should describe what the component is, if it is in a particular state (if the element represents a state of the application), and what action a user can take (if applicable). For example, if a checkbox or toggle component is selected or unselected, the state should be read with the description. For a volume slider on an audio component, the percentage or value of where the user is should be read with the description.

What about user-generated content?

Twitter is a great example of this. As shown in Figure 3-4, as of 2020,⁶ the platform has an “+Alt” button always visible when posting a tweet with an image that lets the uploader add a label.

1. 2.

   Use image recognition to generate captions/alternate text for visual assets (Figure 3-5).

    

Image Processing and Machine Learning in Action

If the images are baked into the application, adding alternative text is pretty straightforward. If the content is user generated, for example on Facebook or Instagram, where users are posting their own content, the product can use image recognition to provide labels for images that don’t have captions as a fallback to (or to enhance) captions supplied by the user uploading.

In 2016, Facebook (now Meta) released a feature for users with visual impairments that uses neural networks to recognize faces and objects in images. In 2017, the company launched a feature for everyone which automatically tags people in photos using facial recognition. This technology detects people, objects, scenes, actions, places of interest, and whether an image or video contains objectionable content.⁷ With over 250 billion photos on the platform,⁸ you can imagine the enormous amounts of data such a feature generates, as well as its endless applications in search, content moderation, and ads personalization while providing a richer, more powerful experience for users with visual impairments.

Meaningful Sequence, Grouping, and Hierarchy

Screen readers allow navigation by headings, links, paragraphs, and other structural attributes. In order for them to work as expected, these elements must be programmatically identifiable, and in a focus order that is consistent with the intent of the content.

In the following example (Figure 3-6), reading the image description of the sofa, followed by the title and price might be confusing and repetitive. Even though visually the title appears after the image, it should be the first thing read to the user. To make the experience even better, the title, image description, and price should be grouped together so the user does not have to focus on three items to get information that is part of one item.

In terms of hierarchy, the page title is first, followed by the section headings “Shop Sale Items” and “Browse by Department,” followed by department cards such as “Office,” “Living,” “Dining,” and “Bath,” and finally the individual items under the sale tab.

Not convinced? No worries. Search engines use text, headings,⁹ titles, alternative text, and page structure to determine search ranking. If your site isn’t taking meaningful hierarchy and headings into account, you won’t be getting many visitors anyway.

Dynamic Sizing

Users can adjust the size of text and other components such as images. The WCAG guideline for text resizing allows up to a 200% increase in size without the use of assistive technology. This guideline also helps people with motor disabilities such as tremors, who might have trouble reliably tapping a small target area.

If the viewports are not restricted in both directions, that is, horizontally and vertically, such that overflowing content wraps in one direction, most dynamic sizing issues can be resolved. Restricting the flow of content to either vertical or horizontal scrolling (also known as reflow¹⁰) makes it easier for people with vision loss (who need enlarged text) and with motor impairments to track and easily read the content.

In Figures 3-7 and 3-8, we have two examples of potential issues that can arise in apps when the user increases their font size and magnification. The device will take all the text and enlarge it, but if the surrounding UI elements don’t adapt to this change, we might end up with an indecipherable experience such as the one in Figure 3-7.

Colors Alone Are Not Used to Convey Meaning

Content and instructions if conveyed through color, shape, size, or other solely visual means should have text equivalents or another marker.

For example, on a Spotify playlist, a dot below the shuffle button, in addition to the changed color represents the current status of the setting as shown in Figure 3-9.

Another example is in Figure 3-10, a user with red-green color blindness (the most common strain of color blindness) will not be able to tell which stocks have had a positive or negative movement.

However, in Figure 3-11, the simple addition of the plus and minus characters provides a way of understanding whether a stock has risen or fallen without relying on color to convey that meaning.

Note that the same colors might have different meanings in different cultures. For example, the color red has a positive connotation in Chinese culture, and that is reflected in how stock price movements are presented on the Apple stocks app in China (Figure 3-12).

Most mobile devices also have dark mode and settings such as color inversion in their accessibility menu, that transform all colors specified in the code of an app or website. We have the following two images: one where images and videos are excluded from color inversion (Figure 3-13), and one where they are not (Figure 3-14).

Links and Other Actions That Lead to Context Change Have Clear Descriptions

Links or components that redirect a user to a page other than the one they are currently on, or that change the context must have a clear purpose and a way to revert the action if accidentally activated, such as with a back button. Links should also be visually distinguishable from regular text, either with an underline or another marker so it is clear that they are actionable elements, and with text that describes what the link is, instead of the commonly used “Read More” or “Here” links. The reason for this is that screen reader users can navigate by links, and nondescript links read out of context are not helpful. Figure 3-15 shows an example of a link within a block of text.

An Auto-playing Video or Audio Should Not Interfere with the Screen Reader

This guideline requires that you provide user controls for auto-playing files if the content is played for more than three seconds since the sound from the file can interfere with the screen reader.¹³

Synchronized Video Content Has Audio Descriptions

If there are movements or context changes in a movie scene while there is no sound, audio descriptions are the only way for blind users to follow along with their sighted counterparts.

Beyond Compliance

Aside from the benefits to people with permanent visual impairments, here are a few other use cases and opportunities unlocked by the above guidelines:

Defaulting to no volume regardless of video duration and letting the user decide whether a file should be played with the sound on is a better experience for everyone. Overall, auto-playing videos are quickly going out of style as people are realizing their accessibility pitfalls, as well as their adverse business repercussions.¹⁴ Auto-play also increases page load times, adds cognitive load, and introduces unnecessary data usage.

If an application supports voice interaction and audio descriptions, it is safer and more accessible while driving. The same goes for use of voice assistants such as home speakers without screens.

People might just prefer larger or smaller text sizes, regardless of disability. We have all at some point, zoomed into a block of text on our phones or changed the font size on our Kindle devices.¹⁵ Some people prefer smaller text sizes so they can fit more content on a given screen, especially on mobile devices.

Accounting for dynamic views depending on magnification also allows for expansion to other languages that might require different screen space compared to the default.

Additionally, descriptions for images and audio descriptions for videos make content more discoverable given that search engines use alt text along with other metadata to rank and show images in search results.¹⁶

Audio and Video Content Include Captions

Note Audio descriptions (covered in the section on visual impairments), closed captions, and subtitles are sometimes used interchangeably, even though they are meant for different use cases. Audio descriptions include (in addition to the original soundtrack), a description of what is happening visually.¹⁹ Closed captions assume that the user cannot hear, and include background sounds along with speaker changes and text. Subtitles assume that the user is hearing and only include spoken dialogue. A transcript is a text file with the spoken dialogue in the entire video or audio file, but not part of the video/audio file. It may or may not have speaker labels, time stamps, and audio descriptions.

The Telecommunications Act of 1996 in the United States requires broadcasters, cable companies, and satellite television service providers to provide closed captioning for 100% of all new, nonexempt, English-language video programming.²⁰

Captions or transcripts can be manually generated, for instance, by creating a WebVTT file²¹ or with ASR (Automatic Speech Recognition). The trade-offs between manual and ASR options are price, speed (turnaround time), and most importantly accuracy. Depending on the quality of the audio file (background noise, number of speakers, recording equipment, etc.), the accuracy of ASR captions can drop significantly.²² The accuracy of speech-to-text systems is typically measured by WER (Word Error Rate), that is, the percentage of transcription errors a system makes. A word error rate of 5-10% is considered good.²³ Acceptable error tolerance will of course depend on the application and the brand using ASR models.

Apart from availability of captions, giving users the option to adjust the size of captions ensures that people with overlapping hearing impairments and low vision can access the content. If the design and implementation account for different text size preferences, presenting captions in other languages that might take more screen space comes at no additional cost.

In the following, we see an example of captions presented with regular text size (Figure 3-16), followed by a screen with captions in large font (Figure 3-17). Lastly, we have a screen showing captions in German (Figure 3-18).

Sound Cues Alone Are Not Used to Convey Meaning

Visual or text alternatives for sound-based information should be present. For example, while playing an online game, if a user enters an erroneous command, a beep lets the user know they made an error. In this case, the game should also present a visual or text cue. On phones, haptic feedback is sometimes used to reinforce error messaging.

Beyond Compliance

The use cases for captions go well beyond accessibility. One video publisher with over 150 million users on Facebook reported that 85% of videos were watched without sound.²⁴ Captions also allow for search, intelligent segmentation, translation (and therefore broader reach), and smart features on top of media files that would not otherwise be possible. Potential use cases of transcripts and captions are discussed in further detail in the case study below.

Case Study: Transcripts and Captions

A few years ago, when I worked at Yahoo Finance, our team collaborated with the accessibility team on a project called “Live tickers.” Closed captions on all prerecorded financial markets news already met the compliance requirement. We decided to go one step further.

In this project, we combined captions, machine learning, and front-end design to show the live stock price of companies that a reporter was talking about in real time as part of the video experience.

News providers in the financial markets sector already have this feature as you might have seen on tv, but it requires someone to manually identify and post this information. The automated solution is far more scalable and customizable. For example, it can pull in stock prices from when the news came out, and show its current price for comparison if the video is being watched later.

This was an example of assistive technology powering a richer experience for everyone. Another benefit of transcripts and captions is SEO or Search Engine Optimization. Discovery Digital Networks (DDN) performed an experiment on their YouTube channel, comparing videos with and without closed captions. They found that captioned videos enjoyed 7.​32% more views on average.²⁵ Captions and transcripts also give way to creating subtitles that can open up content to entirely new demographics.

Similar to the live ticker feature, captions enable search and navigation to specific parts of a media file. It can also power automatic semantic segmentation of videos into individual sections. For example, YouTube allows creators to add chapters to their videos so watchers can skip to segment-wise timestamps, as shown in Figure 3-19. Viewers can also share specific sections of the video.

Currently, the uploader decides where these breakpoints are. Captions combined with natural language processing might enable this feature by default, without manual segmentation, and even personalize sections for individual watchers.

Another amazing feature powered by live captions is live transcription and auto-generated meeting notes so people can focus on communication during meetings.

Content Is Organized, Digestible, and Consistent

Adding features to an application is the easiest thing for teams to do. The less intuitive thing to do is to constantly evaluate the cognitive complexity of functionality on a given app or screen. This not only makes applications more accessible for people with cognitive impairments but also more intuitive for everyone. Especially on mobile, where the number of UI components and actions a user can take on a given screen is limited to begin with.

Figure 3-20 is an example of a mobile interface with too many UI components including a search bar, filters, and ads. Figure 3-21 on the other hand shows a simpler interface with a focus on search results.

Timeouts or Limits on Interactions Are Adjustable

If an interaction requires to be completed within a certain time frame, a user is given enough warning (20 seconds according to WCAG guidelines) before the expiration, or the ability to stop or adjust the timing. This especially applies to forms and other input activities where users can lose data if they are unable to complete a time-based task. These controls give users with reading or learning disabilities, as well as those with less experience with technology products sufficient time to complete tasks.

The exception to this rule is situations where real-time interaction is required, for example in real-time auctions.

Animations, Complex Language, and Auto-updating Content Can Be Turned Off or Paused

For mobile developers, using the operating system’s standard APIs for animations is the easiest way to ensure that user preferences on disabling or slow animation are honored.

Instructions and Errors on Forms Are Presented as Text, and in Context

For example, if a user needs to input a date in MM-DD-YYYY format, that is part of the instructions next to the field. The user is then less likely to fill an entire form, and at the submission stage come back to a list of errors that could have been avoided with clear instructions.

Below we have two sets of images that show common pitfalls in online forms and side-by-side comparisons with remediation for those pitfalls.

The first set of images (Figure 3-22) shows a payment form where the field name, such as card number, disappears as soon as a user starts typing, which can be confusing and disorienting for people with short memory or attention spans. Next to this image is an example of the field remaining visible at all times.

In the next set (Figure 3-23), we have an example screenshot where all errors related to a form are reported above the form. This is even more concerning in longer forms, which might require users to scroll through multiple pages in order to correct their input and verify the new input. For people with anxiety, memory, attention, and other cognitive impairments, it can be overwhelming to fill out a simple form if error messaging and remediation instructions are not in context.

Next to the first image is an example with the error message appearing below the field in question, which updates dynamically as the user enters new information, instead of waiting for form submission.

Captcha and Other Authentication Methods Have Alternatives

This overlaps with user needs for people with visual impairments. Audio Captcha, two-factor authentication, or alternate ways of verifying identity as a human must be available so users are not obstructed from access to their online accounts without compromising security.²⁸ The first iteration called reCAPTCHA was rated among the most inaccessible components on the web. It now supports audio alternatives.²⁹ The W3C releases a detailed report on the accessibility challenges and proposed solutions on this topic, which includes third-party authenticators such as password managers to noninteractively authenticate humanity.³⁰

Focus Indicator Is Visible

This coincides with user needs for people with low vision and motor impairments who are using screen readers and keyboard navigation. A focus indicator helps the user know the component (a button, text, etc.) on the screen they are interacting with at all times. Browsers and mobile operating systems have built-in focus indicators that meet WCAG requirements if standardized UI components are used. The requirements center around two aspects – size and color contrast so that the item in focus is distinguishable enough from items not in focus. The current guideline for the WCAG AA standard is to have a 3:1 contrast ratio between the focused and unfocused state, between the component and its adjacent UI elements, as well as a one-pixel outline or four-pixel shape around it.³¹

It is usually best not to tamper with default focus indicators but if you are using custom views or focus indicators, make sure to review these requirements. This can be particularly challenging with dynamic or user-generated content, where for example, the colors of uploaded images are unknown. One way to tackle this is to leave some space between the UI component and the focus indicator, so you are working with a known background color.

Beyond Compliance

Making content easily readable, less chaotic, and more understandable is simply good design and engineering practice. Simplifying user flows, and removing features is not only good for engagement and reach (creating a smaller application or web page size), but also better for teams to maintain.

Primarily Voice Input Applications Provide Alternative Ways of Interaction

The voice use case is not usually covered in guidelines due to its recent popularity and the fact that most actions that can be accomplished with voice also have alternate equivalents (keyboard or touch through visual interfaces).

Google’s project Euphonia³⁶ is a great example of an artificial intelligence application in making speech technology more accessible to people with conditions such as ALS (amyotrophic lateral sclerosis) and Down syndrome. The technology trains on people’s natural intonations and speech patterns during the early stages and helps them communicate with their own voice after they lose speech. It has also been used to trigger smart home devices using nonspeech sounds, as well as to cheer during a sports game using facial gestures on one user with ALS.³⁷ According to a 2021 study,³⁸ the technology is already outperforming human transcribers, especially for the most severely affected people.

Beyond Compliance

All Content Is Accessible Through a Keyboard

This especially applies to mobile and touchscreen devices, with which users attach external keyboards or custom assistive devices. The most rudimentary way to check whether an application is largely keyboard accessible is to pair a keyboard with the device, and navigate through the content (this is done by using the “Tab” key). This also ties in with the guideline on meaningful grouping and focus order we covered in the section on visual impairments and screen readers, where users can navigate by headings instead of tabbing through every element on a page. Additionally, for speech input users, as well as keyboard users who might not be able to type accurately, a user should be able to turn off or remap character key shortcuts.

All actions should be available without requiring timing or individual keystrokes. This is important for users with specialized or adapted input devices such as a head pointer, eye-gaze system, or speech-controlled mouse emulator, which makes certain gestures cumbersome, error-prone, or outright impossible.⁴⁰ One example of this is a sortable list that requires a user to drag and drop items precisely in the position they want.

While drag and drop is a great interactive way to place elements where a user wants them, developers must also provide a way for users to reorder elements individually. For example, by entering the item’s place in the list, or moving an item up and down, one at a time.

Motion Actuation, Pointer Focus, or Activation Is Reversible

A user is able to remove focus from an element that can receive keyboard focus. The same applies to the activation of functions upon clicking or by certain movements of the device. For example, some apps allow activation of functions when a user shakes the device. This might happen unintentionally for users with motor impairments, or for anyone, and the user should be able to either turn off the functionality or be able to reverse it when it happens. Equally, these applications can’t rely on the idea that a user will be able to shake the device at all, as the device may be mounted to a wheelchair or other assistive device.

Beyond Compliance

Providing keyboard access goes hand in hand with screen-reader access and focus order. Elements that are accessible by keyboard are also accessible to screen readers.

Avoid Using Jargon or Unnecessarily Complicated Language

There are tests such as the Flesch-Kincaid reading test,⁴¹ that you can run through text to ensure that the content isn’t too difficult to read for your target audience. This is particularly important for marketing material or user onboarding messaging.

Adhere to Global Settings

We have touched on this in the previous sections on text sizes, animation settings, color inversion, and dark mode. If users have accessibility settings enabled on their device, developers must honor those settings within the application’s context. Similarly, if the user has chosen an in-app custom accessibility setting, it is a better user experience to persist (save) those settings between sessions, so the user does not have to reconfigure them everytime.

Allow Users to Provide Feedback

User feedback is a goldmine of possible improvements to their existing offering, as well as new ways to serve their customers. The best product teams make it as easy and intuitive as possible for users to provide their feedback. Having multiple channels such as email, phone, and live chat that meet a variety of functional needs is crucial for people with disabilities. This also allows all users to engage with customer support teams in the way they prefer. An example of this is shown in Figure 3-24.