Transcript of PEAT Talks: The Importance of User Testing for Accessibility webinar with Sharron Rush of Knowbility held in March 2016.
Introduction
Hello and welcome to PEAT Talks, the virtual speaker series from the Partnership on Employment and Accessible Technology. On every third Thursday of the month, PEAT Talks showcases various organizations on individuals whose work and innovations are advancing accessible technology in the workplace. My name is Christa Beal. I'm a member of the PEAT Team and I'll be hosting today's talk.
Before we get started, I'm going to quickly review a few logistics. We'll have time at the end for Q&A, so please enter your questions in the chat window or by emailing info@peatworks.org. You can also email info@peatworks.org if you're having any technical difficulties. You can download the presentation from today's event on peatworks.org and an archived recording will be posted online following the event. We will be live tweeting today's event from @peatworks, so please feel free to join us and follow along using the hashtag #peattalks with two "Ts."
PEAT is honored to welcome Sharron Rush, cofounder and executive director of Knowbility, and a recognized leader in raising awareness and skills around the issue of access to technology for people with disabilities. Since 2004, she has served as an invited expert at the World Wide Web Consortium, and is now a co-chair of their Education and Outreach Working Group. Sharron has performed policy review, performance analysis, technical consultation, and training development for hundreds of private and public companies, government agencies, and schools.
Today, Sharron will be talking about the importance of user testing to ensure that workplace technologies are truly accessible. At PEAT, we believe that user testing is a critical tool, as many systems we use in the workplace today may not necessarily meet the needs of all users, even if they comply with existing policy requirements. Now, without further delay, I'll turn things over to Sharron.
Presentation
Thank you, Christa. And thanks everybody. Thanks for coming out this afternoon. And I'm really pleased to be able to talk about this topic because it's one that's quite, I think, at the very heart of accessibility. You know, we tend to talk a lot about the standards and the checklists and the technical requirements, and I think it's always really a good thing and a happy thing for us to remember that at the heart of accessibility are people, people with disabilities, people with different ways of processing information, and just start that conversation and talk to the people who use our technology so we get a little bit more understanding of how they use it and what problems they might have.
So, today, we're going to — I expect to hear a lot about these two terms, accessibility, usability, how do they relate to each other, is one of them a subset of the other one. And so I just wanted to kind of start with the standards because my expectation is that that's a place where all of us are generally pretty familiar. If you work in accessibility, you define a standard. I know one of the previous talks was about policy-driven accessibility and that you adopt the standards, develop your policy to support hose, and develop your checklists and that.
And so, of course, you know, when we think about the standards for accessibility, we think about Section 508 or the WCAG's Global Standard. And, you know, many people don't really realize it but the International Standards Organization, or ISO, also adopted the WCAG's standard as its way to measure accessibility as well in 2012. ISO also has a standard on usability. And I think this is a very interesting standard and a very interesting way to define something because they talk about — their definition of the standards is that it's the effectiveness, efficiency, and satisfaction with which specified users achieve specified goals in particular environments. So you think, okay, so that's what usability is defined as.
Well, how do you unpack that? What does that actually mean? And then if you unpack it entirely, you look at those words — "efficiently," "effectively, and "satisfaction." So what we're saying — and this phrase, by the way, "usable accessibility," is one that I first came across in the book by Whitney Quesenbery called "A Web for All" where she introduced this notion of "usable accessibility." And so, in our field, when we talk about accessibility and user testing and user feedback, I think the notion of usable accessibility is one that we can really use very effectively.
So what we're saying then is that we expect, if we design our systems properly, that people with disabilities can efficiently and effectively to the same things as any other user; right? They can get the same information, perform the same interactions, actively produce — not just passively consume content — but be producers and participants, and that they can do these things with satisfaction. So that's an important concept, particularly in the workplace. If you think about how we are now using electronic information communications in the workplace, I mean, I'm sure any of you could start telling me right now, typing in, different systems that are electronic-based, that are web-based, that are used in the workplace.
And one of the examples that really brought that home to me was when we started Knowbility in 1999. And the idea was we starting doing contents, accessible design contents. And the idea was engage people, designers, in a sort of a fun way. And we did this content in Denver. One of the companies who participated was driven by — they were mainframe programmers in this big software company, and one of them was blind. And he said, "You know, I'm really struggling at my job now because we've introduced these calendaring things, these messaging things internally that really I can't use," because they weren't designed with him in mind. Now, previously, as a text-based mainframe for-train COBOL programmer, he was fine. But as these interfaces were introduced and his needs weren't considered, he was increasingly marginalized at work.
So, as we think about that, I think we really need to consider we've got these notions now of effectiveness, efficiency, and satisfaction. Well, those are pretty vague terms. How do you measure that? And, really, the only way that I know of is to ask users. So you have to find this diverse group of people who may use these tools and ask them what their experience is. And, often, I think people who are screen reader users — my blind programmer, for example — they kind of have to do all the heavy lifting when it comes to accessibility, because everybody figures, "Oh, I'll just ask the guy with the screen reader." It's really important, as you start thinking about engaging with users and thinking about including users with disabilities in this kind of endeavor, that you think about the diversity of abilities. And while certainly the input that you're going to get from screen reader users is extremely important and very, very valuable, you want to also consider that there are other types of disabilities and be sure to think about and include them.
So I wanted to take a minute to ask the group here who's taken the PEAT Tech Check. And the reason I ask about that is because I think some of these questions were really, really relevant to this topic today. Like, there was a question — a category, "Understanding Accessibility," and one of the choices you could have made is "We have a good understanding of how people with disabilities experience technology and how it affects their work life." Now, if you were able to answer that question with this particular response, I think your company is in great shape. Some of the other answers to that question were kind of along the lines of, "Oh, we've never even thought about that," and I think that's where many of us are.
Another one of the questions was, "When you are evaluating your workplace technologies, do you rely on employees with disabilities to identify problems and do you involve people with disabilities, both employees and non-employees in testing?" So those, again — and there were other choices, of course, along the lines of, "Well, we know we should, but we don't really know how," or, "No, we haven't even thought about that."
So you might take a visit through that Tech Check. In addition to this topic, I think it has some really good sort of general parameters for how you can start to think about how your own accessibility program stacks up. But on this topic, I thought it was really great that this is included in the Tech Check because often — and, you know, I got back to my Denver-based blind programmer, he said, "And so every time now that I've complained about it, every time they want to buy something new, they ask me to test it." Well, that takes away from his time at work and isn't really fair for him to have to be the only conduit to be able to tell them, and is one person's experience enough to inform your purchasing decisions or your design decisions? So the idea of involving people with disabilities, employees and non-employees alike, in testing before you buy, as you design your technology resources is really key to succeeding in this whole endeavor.
So I know this is supposed to be a short talk so we have plenty of time for conversation. I wanted to just leave a couple of things here to the idea that you want to integrate people, people with disabilities, in any usability testing that your company or your agency does. And a lot of times people have trouble understanding, well, gosh, where do I find those people, how do I even know? And I wanted to give you these resources from the W3C. One of them is here on the slide, how people with disabilities use the Web. I'm going to put a couple more of them in chat. These are also from the W3C, and one of them is how do you involve people in the design process and involve people with disabilities. That's there. And the other one is how do you involve people with disabilities in evaluation? So the first one is more about the design, if you're designing products from the ground up, and the other is evaluation of products that maybe you're thinking of purchasing or that you just want to evaluate things that are already in place.
And this — you know, it's a real problem for people if you are — think about that word with satisfaction. Not too long ago there was a human resource software used here where I am in Texas by several of the big agencies. And even though they said that they had a VPAT, they claimed conformance to standards, but when it came to usability it was a miserable experience for so many of the people with disabilities who work at these agencies. So they're proactive in hiring people, but then they put them through a miserable human resource experience. And one woman said, you know, "I don't necessarily want to have to go get sighted assistance when I'm filling in my request for a raise or request for vacation or any of those kinds of personal details." So these are the kinds of — okay, you clicked on it and it says "Document not found." Which one, Gilbert, did you click on, the one in the window there or the one on the slide?
It looks like the backslash involving, it's giving us a 404 — yeah, both links are giving us a 404 error. We can always follow up with the links in the —
I think Denis found — Denis, did you find the right link? So I don't know what the difference was, but so there they are.
That one works.
And I have no idea what the difference was there, but there you go. And then the other thing — and, you know, this is a shameless self-promotion part of the talk — is that we have started, a couple years now, we have panels of people with disabilities who work from home on their own assistive technology. They're paid to participate so that you don't have to ask for volunteers all the time. I think, a lot of times, people with disabilities get tired of always being asked to volunteer their time. But this is another place where you can find testers across a number of categories of ability. And if you have a disability and you want to join as a tester, by all means, please sign up. And there's a promo code there that will let people know that you heard about it here at the PEAT Talks.
All right. Great. Thanks, Sharron. And I think we'll have plenty of time for questions, which is great. So, for those of you on the phone, please start entering your questions in the chat window. And I am going to get started with one of mine which is when is the best time to conduct that user testing?
Well, if you are privileged enough to be able to design your own system, then the time to include people with disabilities is right from the start. So, as you are doing your design mock-ups, for example, you would be able to get some feedback about the colors you've chosen or the fonts you've chosen or some of the layout choices you've made. And so getting feedback all along, throughout the process, of course, is the best, and starting right from the start is optimal. Gilbert asked if I'm going to send the slides. I think the slides are archived; aren't they, Christa?
They are. And I will pull up that link and put it in the chat window. You can download them right now if you want. The next question I have for you is, you know, kind of high level or dig into the ones you want to, but what are the key steps involved in that user testing process?
Okay, I think we touched very lightly on some of them, and I would say the number one thing when you think about involving people with disabilities is to try to get a diverse panel of people. So you don't want to make all of your accessibility decisions to be based solely on one type of disability. So — and, you know, you want to think about your target audience. Naturally, if you're the National Federation of the Blind it's not going to be unreasonable to depend on a particular set. But, in general, your goal, as an organization, is to be able to hire a diverse staff as possible. So you want to test for as many disability conditions and across abilities as possible.
And then the other thing that you said was the fact of when do you involve people and making sure that you really try, to the best of your ability and resources, to be able to include people all the way through. Now, in the procurement process, when you're getting ready to purchase a piece of software or contract with a web-based business, then, in that case, you definitely want to — you know, you're not involved in the design process, clearly, but somehow, in your procurement process, if you include some user testing before you make that purchasing decision, building that and integrating your inclusionary practices into your general procurement design, all of your practices. I was thinking about that.
And then when you find — like, when you get the feedback, getting the feedback from making sure that you document it well, whatever feedback you get, and discuss it and make sure you really understand it, because maybe you have a form or, you know, some real brief way for people to provide you with feedback. But just to be — to make sure you have full clarity on it, to promote discussion and give people the opportunity to really explain what was the barrier that they encountered and what ideas would they have for how it might be designed differently that would allow them to use it more effectively.
All right. Great. Thanks, Sharron. We have a couple questions from the audience. The first one you spoke to a little bit, but I'll let you expand on it. It says, "Do you have recommendations to help us put together a great group of users so that there's a broad representation of abilities and needs?"
Well, you know, I think that's going to — that is a great question, and it's really going to depend on your, again, your — what do you do, what is your business, what is your target communication goal? But, in general, I think you want to look across disability categories. So you want to think about speech, hearing, vision, mobility impairment, cognitive disabilities, and that's one that's really emerging these days as a disability category that many feel have been overlooked in the creation of the standards so that people who have processing differences, who process information differently across dyslexia, ADD, autism, those kinds of cognitive situations, if you find a way to be sure that you're including those categories in your testing, I think you'll get results that sometimes really surprise you.
All right. The next question we have is "How do you test the potential testers to ensure that they know their assistive technology well enough?"
That's a good question, too, because what we find is that you have novice users, people who have, you know, just begun. Maybe they were blinded because of an accident, and they're adults, and they're just learning how to use their screen reader or they've acquired a disability over time and are new users. So that really is a good question, and it's one that is relevant in the testing situation. And I think when you set up your user groups, it's a perfectly reasonable and — I don't know that necessarily you need to test them to be sure that they are in one category or another, but usually they can — people who use assistive technology know if they are experienced or, you know, you can ask for that information of do you consider yourself a novice, an intermediate, or an advanced user of whatever the assistive technology is that they use.
All right. Great. So the next question we've got is "Can you talk a little about the perils of involving users that may be too proficient with their tools to be good representatives of a typical user?" So this is kind of the other side of the coin.
Yeah, that is the other side. That is the other side of it, and it's a tricky one. I mean, it really is because often — and we actually have a person like that on our staff. I can't use Desiree for testing. I can use for — not usability testing. I can — you know, she can work on other things. She can certainly advise and do — advise on programming and fixing and repairing, but as far as usability testing, Desiree is so far outside the norm in terms of her ability not only to use the inherent qualities of the screen reader but to reprogram or to write programs that make this technology perform better than it is out of the box. And so someone like that, I think, is outside of where you would want to find your — and I don't know — so those are definitely the perils. How you identify them, which was Peter's question — how you identify those users in a general way is something that might be a little trickier to do. And I think, again, you're just going to have to rely on how they self-identify. If they self-identify as "I’m an expert user," then you might just give that consideration as you put your panel of users together to do the testing.
All right. The next question we had is "Have you had any experience looking at usability for state job search databases?"
Yes, and, overall, they are not very good. So I don't know what more to say about that. I think for people with disabilities who are trying to use those job search databases, to continue to give feedback. And, again, you know, I go back to the W3C Web Accessibility Initiative, they try to provide guidance to people. There's one resource there that's called — I forget what the formal name of it is, but it's something like "How to complain about a really rotten website." And so I think you file your complaint and you say, "Look, I'm a job searcher with a disability." And, you know, I think there's a lot of concern as well about things like that that are really basic human rights; voting websites, some of the states' voting registration websites as well, just miserable in terms of accessibility and usability. And so I think the only recourse you have really is to write to them, complain to them, and if it doesn't get better, you know, there are legal — you can file a civil rights complaint.
All right. Well, we may talk about state websites in another PEAT Talk. I'm going to wrap it up with one last question. And this kind of — I'm going to reframe it a little bit, but it's how does cost play a factor in deciding about the technology? Are the good technologies similar in price or is there a range? And I think, you know, if you could speak to that and sort of is there a premium on getting a really truly accessible product?
Well, of course, in my opinion, there is, of course, that premium. But I think we — and this is really — I'm pretty excited about this because I think we have finally reached the tipping point or the watershed or whatever you call that place where enough companies have realized the benefits of that. So, in the month of May, every year, here in Austin, Knowbility hosts a three-day web accessibility training conference. We've been trying for years to get an app that we can use to build it ourselves. We're a small nonprofit. It's really prohibitive. So we've been working to try to buy one. And, you know, we've been in these conversations now for a couple of years. Well, this year, I think we finally got them to pay attention.
One of them put out a VPAT, and the fact that they put out a VPAT was encouraging, but then it turned out their VPAT was not accurate. But at least we knew they were interested. And we started talking to them, saying, you know, "Your VPAT isn't quite accurate, but if you're interested we'll give you this guidance." And they were interested because they know that many, many state and federal agencies also do conferences, and if they can build an accessible conference app, that's an advantage for them. And so if they hear from enough users that they have a conference app that's usable, findable, searchable by people with different disabilities. I think they know that that's a real competitive advantage. So they've been eagerly listening to us and making those changes.
So the fact is the cost on their — they are assuming the cost. You give them the feedback. They assume the cost. And then the cost to us to subscribe is no different than for any other comparable app. In fact, this one is, in some cases, less expensive than some of the others.
Conclusion
All right. Thanks, Sharron. We had some really good questions today. And I appreciate your time, but I think we're going to wrap up and let you all go right at 2:30. But don't forget to join us for next month's PEAT Talk. It will be on Thursday, April 21st at 2:00 PM Eastern Time. We will be welcoming Denis Boudreau, who you saw chime in a couple times today. He is a senior web accessibility consultant and strategy who will be talking about "Building Accessible Online Recruiting and Hiring Systems." You can find that registration link on peatworks.org or look for an email from PEAT in the coming days with more information. I'd like to give a huge thank you to Sharron for speaking with us today, and for all of you who took the time to join us. Have a great afternoon everyone. All right. Bye-bye.
