Podcast: Machine Learning and Ability-Based Design

Intro: [00:00:01] Welcome to the Workology podcast, a podcast for the disruptive workplace leader. Join host Jessica Miller-Merrell founder of Workology.com as she sits down and gets to the bottom of trends tools and case studies for the business leader. HR and recruiting professional who is tired of the status quo. Now, here’s Jessica with this episode of Workology.

Jessica: [00:00:25] Everywhere I turn there seems to be conversations in the human resources and recruiting industry centered around artificial intelligence. It is truly permeating our landscape and myself along with PEAT found some pretty amazing things that researchers are doing in the fields of accessibility and technology. One of those emerging technology fields is in the area of touch screen technology allowing those with motor impairments to be able to use and access technology. Today I’m joined with Martez Mott. Mr. Mott is a six year Ph.D. candidate at the Information School at the University of Washington where he is a member of the mobile accessible design lab. His focus is designing touch interaction techniques for people with motor impairments and also for people under the effects of situational impairments. Martez. Welcome to the Workology podcast.

Martez: [00:01:20] Hi Jessica. It’s great to be here.
Jessica: [00:01:21] Talk a little bit about your background because this is really a sense of time digging into what you’re working on is really interesting.

Martez: [00:01:30] Yeah. Yes so yes my background I have a computer science background so for my undergraduate master’s degrees I received and computer science at Bowling Green State University in Bowling Green Ohio. And while I was at Bowling Green I started working for a research group there and they were working on how to make video games more accessible to people with different type of visual impairments and that just got me really interested in the accessibility space. Overall, it kind of opened my eyes to different design problems and challenges that I’ll worry that I wasn’t aware of beforehand. So after my experience at Bowling Green I knew I wanted to go to a Ph.D. because I wanted to continue to do research and build on these technologies. So that led me to the University of Washington at information school where I’m advised by Professor Jacob Walbrook and I’m a member of his mobile access Design Lab.

Jessica : [00:02:20] Can you talk to our podcast listeners more about your work in improving artificial intelligence with those with motor impairments.

Martez : [00:02:29] Yeah. So what my work looks from my dissertation what I’m really trying to do is I want to be able to improve to access touch enabled devices for people with motor impairments so touch enable devices could be things like your smartphone or tablet. It could be the public kiosks at the grocery store or at the airport. And what we really want to do is we want to be able to create experiences for people that master abilities. So when we approach technologies we approached them with a certain assumptions I guess and what happens is that people may not always match the assumptions that technology makes about them. So we will we want to be able to do is we want to be able to train systems to be able to react to people as they are. So in that case a touchscreen technologies people with Parkinson’s there are multiple sclerosis or cerebral palsy may touch in different ways that the system isn’t expecting. So what we want to do is instead of telling the user hey this system is inaccessible to you you need to go find some work around to make it accessible. We kind of want to flip that we want to kind of place the burden of adaptation on the system so we can use machine learning to be able to to do that.

Martez: [00:03:42] So machine learning kind of and more general purposes might be used by businesses of corporations to let’s say figure out something about the browsing behavior of their customers. So if you’re Amazon and you really want to understand what book preferences I have you might use some data about which books I’ve bought in the past or what my browsing history might look like it and that allows you to make some predictions that say hey Martez based on your past browsing history in your past purchasing history these are some books that might be of interest to you. So what we want to do is we want to train the systems to react to people not browsing behaviors or purchasing behaviors but their touch behaviors. So some of the ways that people might take their smartphone and try to use like speech like speech recognition if you ever use the systems they might actually say like hey say a few words before you start using a system so that we can calibrate it so that we can recognize your unique vocal patterns for example. We want to to say but we want to look at people’s unique touch patterns. And once we can do that we can build models of their touch behavior and allow people to have more pleasant accessible experiences with their touchscreens.

Jessica: [00:04:53] I like that and you think about how much touch screens have become a part of our everyday life it’s not just your mobile device or your tablet. I have an Apple Watch. There’s also touchscreens on your computer and also television. So this is an area of design that is becoming what this technology is becoming more part of our everyday lives.

Martez: [00:05:16] Yeah, yeah exactly. Yeah I mean just think about all the things you might use a touch you might use a touchscreen device for in a given day. So yes so if you’re on your smartphone you might use it to send messages to people to take photographs you know browse social media play video games. There’s all these different things that we do in our day to day lives that are now mobile base. So in terms of thinking about technology and the accessibility of technology especially in the workplace you have to we have to kind of shift this mindset from thinking of people only sitting at their desk working at their workstation to people doing small bits of work on the bus or doing you know replying to a quick e-mail while they’re waiting in line for their coffee or things like that. So as these technologies become more and more pervasive more and more ubiquitous they’re going to seamlessly integrate with what we do in our day to day lives. And if we can improve the accessibility of these devices we can’t just allow for people more freedom more flexibility in how they work where they work and the type of work that they want to do.

Jessica: [00:06:18] A lot of this podcast series has been focused on individuals with disabilities focused on accessibility and also the freelancer the gig economy where your work is more fluid and I think that having your technology embedded into phones tablets kiosks at airports or whatever is going to allow these individuals like you said to be able to do their work on the go whether it’s a full time permanent job or something that they’re kind of doing you know on the side or maybe it’s a full time job that they’re just picking up freelance work for.

Martez: [00:06:56] Yeah yeah I think that’s exactly right. I had a situation where one of my study participants was telling me. So he’s a deejay and he was telling me about some software that he wanted to use on the iPad. Some music I think is mixing software or something like that and he was telling me about it and he was like oh I would love to use it. But I just can’t use an iPad. He has cerebral palsy and I pass just don’t work for him. So I think what we really need to look at is you know it’s not just the access to these devices it’s all of the things that these devices allow us to do.

Martez: [00:07:29] All of the features and applications that they have it is how we use these features and applications in an increasingly social and connected world. So yeah if you want people to be able to have the flexibility art is musicians other different types of creatives people who want to create their own businesses or work where and how they were like. We really do need to think about improving the accessibility of mobile technologies which in our current stage are predominately touchscreen based right. So we have smartwatches smartphones are tablets all of these touch enable technologies that could allow people to be able to be more productive to do more things to be able to create more connections. But we really need to get the technology there to the point where people feel confident comfortable using these technologies in a variety of different situations.

Jessica: [00:08:19] I like the example that you provided and I think a lot of creatives you know that the iPad or those tablets are so important. I think in design and also as you’re drawing or writing notes or using a stylus. It’s becoming more touch based but these are things that are such an important part of our everyday life and certainly our work and we go into these workplace situations like maybe we’re getting a PowerPoint presentation and the polling or interaction for the individuals who are participating in the presentation is done over their mobile device through touch. Having these kind of smarts machine learning ways to enhance and make the technology accessible to all is important for everyone.

Martez: [00:09:06] Yeah yeah. Yeah that’s right. And I think when you look at too like it has a lot of workplace things that you know that might require you know access to a smartphone or tablet so even things like you know no two factor authentication. So it might be something that you know your employer requires you to download something for your phone so that when you log in you know you have to approve the request on your mobile device for the two factor authentication. I mean just imagine how much of a struggle that might be if you accidentally hit deny a set of approved when you want to approve actually approve or request that you just have difficulties in general interacting with these really ubiquitous you know systems that we’re creating around security or our productivity in all these different things.

Jessica: [00:09:49] So interesting about the two factor because I’ve been using Google authenticator and so you have to log in. Like every time you log into your e-mail or your Twitter or whatever you have to get to your smartphone open the authenticator app and then there’s a code that you have to plug in. But you have to slide through with your touch screen to be able to get all those places and something like that’s easy for me. But for others it is probably a really difficult task that might be almost impossible.

Martez: [00:10:18] So yeah exactly. For some people it might be impossible for some people it might take one person a colleague you know ten seconds and it might take someone else five minutes or a few minutes. So yeah I think that’s true for like a lot of these different situations really looking at connectivity in terms of how much time does it take a person to be able to complete a task. And what’s the kind of bottleneck between that. So yeah if you’re waiting in line for coffee and you have a few minutes because it’s a long line that you want to send an e-mail you’re not as great if you have two minutes let me send this e-mail. But if you know hey I know this e-mail is going to take me 10 minutes because every time I try to type on a phone it always Hister all but in my experience tremor. So I have difficulty being able to type to e-mail or send it or do all these different things so yeah if we can look at these technologies in terms of what did they allow people to do and how also is it accessibility these technologies holding people back from being able to accomplish the things they want to do. I think we can.

Jessica: [00:11:20] That will give us a better perspective of kind of looking at how these technologies influenced not just our work laws our social lives and now those things are very interconnected a lot of different ways too.

Jessica: [00:11:31] Well let’s take a bit of a reset here. This is Jessica Miller Merrell and you’re listening to the work Algy podcast in partnership with Pete. Today we’re talking about machine learning and touch screen accessibility technology with Martez maat. You can connect with him on Twitter at Martez underscore Mottt or @martez_mott.

Sponsor Message: [00:11:52] The Workology Podcast Future of Work series is supported by PEAT. The partnership on employment and accessible technology program is designed to foster collaboration and action around accessible technology in the workplace. PEAT funded by the U.S. Department of Labor’s Office of Disability Employment Policy. Learn more about PEAT and PEATworks.org that’s PEATWorks.org.

Jessica: [00:12:23] Let’s get back to this. Can you talk about a little bit more about maybe some other examples of technology in terms of accessibility as it relates to work. We had the example which I think is a great one. Any other ideas in your work.

Martez: [00:12:40] Yeah. So yeah. So there might be certain cases so there might be examples let’s say at a factory or warehouse things like that where people might have to do inspections or take inventory and maybe you duty’s things now using an iPad or some type of some type of tablet.

Martez: [00:12:58] You know you walk around or you navigate to these different locations take notes or take inventory of things like that. So if there are situations in which kind of the core tenet of your work is mobile based you know you’re going out into the world doing things so you know that may be somebody who’s an electrician for example who needs to take this step of wanting to take tons and tons of manual physical documents with you. You might prefer to take an iPad with you with all these documents on it stored with you.

Martez: [00:13:31] But if those technologies once again aren’t accessible to you in those might be careers or things that you may want to shy away from because you don’t believe that you could do the job adequately or you might feel self-conscious because you feel that you can’t do the job as adequately as some of your colleagues. So I really think it is this this notion of there might be.

Martez: [00:13:52] Might be the situations in which there are certain jobs that require just by their nature uses a mobile devices mobile technologies, and we have to look at and say hey are these jobs that are inherently mobile based out in the world are out doing things not just sitting in front of a desk all day. All those type of jobs accessible to people because you know the underlying technologies might not be.

Jessica: [00:14:16] I think that’s an important point to make. It’s not just people who are using tablets right? It’s people who are out there maybe safety or they’re in a warehouse or they have a QR and they’re responsible for making sure that products are running down the assembly line the right way. Or I mean working with pets, I mean whatever it is like you need to be able to use the tools to be successful in your job. One thing I was thinking about as you were talking I think about like my world in human resources and recruiting so much of our work for the employee populations contractors freelancers or permanent employees is done over a mobile device email.

Also these mobile apps where you can input your time access your employee directory log into your company portal put in your employee hours for the week. They’re all done over a mobile device so having your technology as part of the accessibility tools for your employees is really critical I think to these individuals with motor impairments success and the ability to be able to do their job as they should be able to.

Martez: [00:15:30] Yeah yeah yeah yeah definitely. And they like do it. You know how they want to wear it they want to yeah like you know if you have all these work-based as you know an inherently mobile base or that you could do it you know using mobile technologies. It’s just great to have that flexibility right to be able to say hey, I know I don’t have to go to my desk to do this. I can do this right now. At the coffee shop or I can handle this later. You know when I have a minute or something so yeah like having the flexibility and the freedom to be able to decide which technology do you want to use and when and how you want to use them. You know that’s just going to make the workplace more enjoyable and more expressive for people. What are your thoughts on

Jessica: [00:16:12] Having or like the future plans for your technology particularly for maybe people who are freelance or gig worker. So let’s say that you are trying to go into fiber and they have a mobile process for you to be able to as as an individual like select work and sign up and you know even completing your profile. How would a company like Fiverr engage you to be able to add the technology that you’re working on and kind of embed that into their current tech stack.

Martez: [00:16:46] Yes. So kind of the work that I’m looking at right now is is more what I would call application agnostic. So what we’re hoping for is a look at the underlying technology and improve it so that if we can.

Martez: [00:17:02] So let’s get someone I want to think about this as if we could retrain systems to respond accurately to you and how you want to interact with them that will then allow any application that runs on top of that technology to be inherently more accessible. So kind of the view the vision that I have at this research is kind of like an accessibility feature or a setting that will be available on touch enabled devices. So you go out and you buy a new Android tablet or a Microsoft Surface or an iPad or something and you turn on this accessibility feature the smartwatch accessibility feature a step you do some type of calibration process where you know you tap on the screen a few times to give us some examples of which are touch behaviors or like.

Martez: [00:17:48] So this is the analogy again to use in the speech recognition software you say in a few of kind of preset phrases so that the system can train to be attuned to your voice. We’re doing a similar thing. We want to tune to your touch behavior and once we do that. Now any application whether it be something specific from a specific corporation that’s worn it on the web or a specific app that you need to download to be able to do your work securely. All of these things now will become more accessible because the underlining technology at the system level not necessarily the application level but at the system level is now more accurate. So that’s kind of where we want to be able to go. We want to be able to take this technology be able to put it on these devices. Have it just as a feature that people could turn on and use it when they need to and how they need to edit and now we’ll just improved the accessibility of all of the applications and services running on top, on top of the device.

Jessica: [00:18:44] I love it. And I think about like I have an Alexa at my house and it is voice recognized but it does go through training right. And I can work with it and for not just me but different members of my family within the house they can talk to Alexa and she gets smarter and then can understand the differences between dialects and how we speak so that she can provide a better experience for us when we ask her for certain things. I feel like you’re you’re talking about the same thing but for these touch like devices which I think is amazing.

Martez: [00:19:19] Yeah is actually yes. For the devices we need that we need a similar type of afford it. So yeah you want your you know we might not think of it right now. Right. So if you know if you use a smartphone right now it’s like hey I could use it pretty accurately Yes sometimes when I’m typing a message I might hit the wrong key and I have to backspace but generally it works well for me right. So you might have debt perspective in front of people it’s like oh yeah. You know it takes me five minutes to try to send one text message.

Jessica: [00:19:46] So how do we get your work in front of the mobile phone and the tablet makers because is that is that the solution like to be able to add this to the iPhone or the androids or your technology , your touch technology at the airport.

Martez: [00:20:04] Yes. I guess my my primary way been able to have that influence is to really light my publication. So I view my role as a one as a doctoral student but after I graduate as you know as a member of the academic community in general as to be able to push ideas forward and then hopefully be able to disseminate those ideas to the broader community for the community to take those ideas refine them and improved it and hopefully you know.

Martez: [00:20:32] The hope is yeah you know I go to a conference and I’m up there giving the presentation about this work. And yes somebody at Google is in the audience and I like oh yeah that’s a great idea and they take that back to their team and you know they actually have the engineering power and prowess to be able to do these things. You know one graduate student is trying to work on these things so I could create kind of like proof of concept I can create testers and algorithms and show and kind of these more limited situations like hey this seems to be promising this may be something that we want to investigate in the future.

But I don’t have that engineering powerhouse behind me that will allow me to go win business ad change and make huge changes to software and to be able to update that throughout time. So you know imagine me as one person trying to keep this up to date. And every time there’s a new Android update I have to go back and update the software again. It’s kind of it’s not. It’s not sustainable for a single person. So that’s really whereas you know academics we need to rely on industry to kind of do the right thing to kind of say hey yeah you know you do have some accessibility features and those are great and those are changing people’s lives.

But you know there’s more work we can do. There’s more improvements we can make and good now talking to people you know trying to be like an evangelist for accessibility and to tell people like you know this is important and this you know this impacts more people and you know they may think so.

Jessica: [00:21:58] Google, Apple, Anybody listening that’s looking at accessibility options for mobile devices monetizes is gone all the research and he’s been doing some really great work in this area. So it’s something that I think they should consider. I mean it makes sense to me and it provides a better experience for everyone.

Martez: [00:22:20] Yeah. Yeah. One of the things I think I forgot to mention for Yeah. So one of the great things about these technologies is that you know when you when you create technologies for people with disabilities or certain types of impairments you actually do end up improving the technology for everyone. So we talk about things also these things called situational impairments. You might think that if you know let’s say if you’re trying to send a text message while you’re holding grocery bags for example you don’t have access to your hands as much as you did before. So we would call you like your situation and we impaired at this moment but if we could create technology that’s more accurate for people with Parkinson’s disease then we can also create that make sure cell phone or your smartphone more accurate while you’re walking or jogging or you’re riding a bumpy bus or something like that. So there are situations in which we could look at it like oh yeah.

You know we live in as you know this mobile world. We’re not just sitting at the desk anymore we have to think about these different context and situations in which we use all these mobile devices and some of them in Paris in these ways and if we can create technology to helps people with disabilities that can help a lot of people in general. And that’s kind of the one of the main hopes of our research is that we can show that we can have this kind of effect that you know this is a pretty big return on investment if you focus on accessibility.

Jessica: [00:23:38] So you’re at you’re a 6th year Ph.D. candidate. So what what is next for you and your research.

Martez: [00:23:46] I’m so finishing up my dissertation so I have a few more projects I’m finishing up over this upcoming academic year. I hope to graduate this year so that’s my focus is on writing the dissertation and yes so yeah we’re just continuing to work on these problems. So I mean there’s so many problems in this space. Just to give you an example in terms of machine learning machine learning typically requires you know lots of data points. So the more more books I buy on Amazon the more confident Amazon

might be and try to predict what my next book might be. You bought one book you may not know much about you but if you buy 200 books it may get a pretty accurate representation of the type of book buyer you are.

Martez: [00:24:33] So if we look at that in terms of machine learning for touch interaction we need to have to collect a lot of data and people’s abilities change dropped the day some people take medication to that are a touch behavior’s might be a certain way in the morning but at night after it’s been a long day you’re tired your fatigue your touch behaviors and your medicines worn off your touch behaviors might be quite different. So how can we collect enough data on the day the times that she/he might be experiencing different effects so that we can create these more accurate models of what your touch behaviors like. So there’s so many different research questions research directions that this work can go into and just kind of we’re we’re working on is just trying to keep on improving and and making the systems better and more accessible.

Jessica: [00:25:16] I didn’t even think about like your medication schedule or your energy levels throughout the day. You know how that might impact your touches and it is an individual with your mobile device or this technology.

Martez: [00:25:31] Yeah yeah. Like some people with maybe Parkinson’s or multiple sclerosis that you know experience tremor yeah they might say yeah I’m not taking my medicine. You know I have less tremor but as the day goes on or if I’m feeling stressed or things like that on my head episodes where I have might might have more tremor. So yeah we really need to be able to to think about people’s abilities not just as a single snapshot of this it’s who they are but as a person whose abilities change drought the day and change with the different contacts. And if we can create systems that can be aware of these things it will just create better experiences.

Jessica: [00:26:05] Martez thank you so much for joining us today. Where can people go to learn more about you and the research and the work that you’re doing.

Martez: [00:26:12] Thank you for having me. This is great. Been able to chat with you. So yes I’m a Ph.D. student at the University of Washington. So if people want to look me up you could just kind of look up my google scholar Martez Ma to see some of my publications about working on working on a new website that’s going to be Martez dot dot com and that should be coming up live probably within the next few weeks or so. So people could definitely check me out there and see what I’m working on.

Jessica: [00:26:38] We’ll have some links to some of Martez this publications and different research and interesting work that’s being done in this space. So thank you so much for taking the time of joining us.

Martez: [00:26:49] Thank you so much Jessica has been great.

Jessica: [00:26:51] I love this conversation and I think that it’s interesting research and programs like these that are really going to push this technology and this accessible inclusive technology forward. I love how we are looking at technology from different angles and not just consumer focus but workplace productivity focus which is where I challenge you to sit back and think about. How Technology Nameer current stock that you have in your workplace or the tools that you have. Can be used to help make the workplace better for all different types of employees. Thank you for joining the work Algy podcast today. It’s a podcast for the disruptive workplace leader who’s tired of the status quo. This is Jessica Miller-Merrell. Until next time you can visit Workology.com to listen to all our previous Workology Podcast episodes.