A good testing process, including accurate and comprehensive reporting on testing results, can improve communication with employees, customers, and other end users about your company's commitment to accessibility and foster a culture of continuous improvement.
As a technology provider, or a company's internal technology developer, you should understand how accessible your products are. The key is a good testing process. Such a process, including accurate and comprehensive reporting on testing results, can improve communication with employees, customers, and other end users about your company's commitment to accessibility and foster a culture of continuous improvement.
Find Out Where Testing "Lives" in Your Organization — Accessibility testing belongs wherever your organization already performs product testing. The two most common "homes" are with usability testing and compliance testing. Either type of testing group can perform accessibility testing, although you may need to invest in equipment (such as assistive technology) and staff training.
Usability testing often involves real people going through a list of tasks with the product in order to assess how well the design matches human performance behaviors. Thus, accessibility testing fits right in, because people with disabilities are among any product's potential users.
Compliance testing, on the other hand, involves lab bench-type testing to meet the requirements of a specific law or regulation, and may never involve actual testers.
Decide Which Standards Apply — Depending on the product you are testing, you need to determine if there are federal and/or state laws that apply and then, based on that information, decide which technical standards you need to meet. For example, a videoconferencing product may fall under the 21st Century Communication and Video Accessibility Act (CVAA) and its guidelines. Many websites are subject to the Section 508 guidelines, which generally means you must use the U.S. Access Board's 508 standards or the World Wide Web Consortium's Web Content Accessibility Guidelines (WCAG 2.0).
PEAT TIP: If you chose the WCAG standard, we recommend using the WCAG-EM Report Tool to generate a customized, structured report to help you outline and organize the testing process.
Do Comprehensive Testing — There is general consensus that while automated testing is a good start and can find a majority of accessibility issues, manual testing is essential. For example, an automated test can tell you whether all images have alternative text, but only a manual test can indicate whether an image actually requires a text alternative (it might just be used for formatting a screen, for example) or whether the alternative text makes sense, given the context of the image. Given that many work-related websites are designed to allow users to accomplish a goal and not just read text or watch videos, a purely guideline-driven testing approach will often be weaker than one based on a list of tasks users actually have to perform.
Procedurally, it makes sense to perform all automated tests and then have a skilled analyst go through the results to confirm the errors and perform the necessary manual tests. It is important to note that exact testing protocols are still more art than science and sometimes viewed as proprietary. However, as the domain of accessible information and communications technology (ICT) matures, there is more consensus and sharing.
PEAT TIP: Check out the automated tools available for testing software and web content (note that the same cautions mentioned above apply):
- World Wide Web Consortium (W3C) Web Accessibility Initiative's List of Web Accessibility Evaluation Tools
- U.S. General Services Administration's list of various technology testing tools used by private industry and government agencies: Section508.gov's Technology Tools
Incorporate Scenario-Based Testing — Effective accessibility testing can also be performed by creating persona-based test cases using a scenario model. In these cases, you gather relevant information about who your primary users are and add the possibility of differing abilities to the dimension of that persona.
For instance, if you are developing an employee travel system, including tests like searching for flights, booking hotel rooms, and comparing car rental prices can all be scenarios to include in the testing phase. Taking that idea to the next level, creating personas allows your coverage to include sometimes overlooked populations. For example, someone who uses a screen-reading application, either built into certain devices or installed separately, could be one persona. Then, you would run all the scenarios listed above using available screen-reading applications. Other possible personas could include people who use hearing aids, persons with limited dexterity, and others who may require large or high-contrast fonts and displays.
The more your testing can mimic potential users, including users with disabilities, the better your final product will be. Thorough testing early on could potentially reduce time-to-market for subsequent updated or new releases as you establish a mechanism for testing, reporting, and addressing the findings that this type of testing methodology can reveal.
Check Compatibility of Screen Readers and Other AT — If possible, test with all the assistive technology (AT) devices commonly used by the people who will use your product to ensure interoperability. Particularly in the case of screen readers, your testing should be comprehensive. Use the same task list you use for users without disabilities, and thoroughly exercise the interface, using several different methods to accomplish each task.
One thing to keep in mind is your testers' level of expertise compared to your expected users. Not every AT user knows how to use the most advanced features of their AT products. So while it's okay to have more advanced AT users test highly sophisticated or complex products, you may want to have users also test the more basic functions.
Remember Documentation and Support — While the product's own accessibility is the most important thing to test, everything the user encounters should be accessible as well. So be sure to test the user manuals and online help features. If you have live support for the product, provide appropriate details to your help desk personnel so they can answer common questions and understand when to forward more complex situations. You can also go even further with accessibility testing. For example, several companies test their ICT product packaging, which can often be difficult to open for people with dexterity limitations.
Report Test Results — Test results should be reported internally so they can be shared with the rest of the product team and any others. That way, lessons learned can be incorporated into current and future product development. They should also be reported externally as a way to keep customers informed and convey your company's ongoing commitment to accessibility. External reporting can take place a number of ways. One common method is a Voluntary Product Accessibility Template (VPAT), a Section 508 compliance reporting tool. Although this tool was developed to assist federal contracting officials in assessing accessibility of products from potential vendors, it can be used by any technology providers to allow all potential customers to learn about the accessibility of their products. However, customers may need much more detail than a VPAT typically provides. For example, if the customer has employees who use certain AT products, it may be necessary to reveal detailed testing results, or even perform follow-up compatibility testing. Alternative methods of access—often called "workarounds"—and AT application notes can be shared with customers and users. Some companies consider this detailed information to be privileged intellectual property, and worry about disclosing known accessibility flaws. That said, many vendors have successfully found acceptable ways of communicating this useful information during the procurement process without publishing it broadly.
Plan for Any Needed Fixes — Once testing is complete, you should create a plan for repairing or remediating accessibility errors. Such plans will vary based on the nature of the product and the barriers or issues you've identified. Not everything can be fixed at once. But it's important to keep track of the list and rank them by importance. Doing so is essential to keeping accessibility as a priority and developing products that are usable by the widest number of people possible—a very smart business strategy.
PEAT TIP: Check out this guidance on developing remediation plans —
Electronic Information Resources Accessibility Implementation and Remediation Plan (Texas Department of Information Resources)
508 Web Compliance and Remediation Framework (U.S. Department of Health and Human Services)
Join the Conversation
How does your company approach accessibility testing? Please contact us to share your company's strategies for ensuring your products and services are accessible to all potential users, including those with disabilities.