Beginner’s Checklist for Inclusive AI
Guidance to help your organization procure and implement inclusive artificial intelligence (AI) hiring technologies.
Many employees from across your organization can influence how technology factors into the hiring process. This process can include staff who work in hiring, procurement, information technology, leadership, and other roles. Depending on the size of your organization, a single employee might work across multiple domains. Regardless, as your organization considers adopting new hiring technologies, relevant staff members should make sure accessibility and equity are top of mind.
The Partnership on Employment & Accessible Technology (PEAT) created this general checklist to support staff in working together to ensure that your organization uses artificial intelligence (AI) tools in inclusive ways. The questions in this checklist were created after speaking to innovators with disabilities who have worked extensively with AI. PEAT developed a report based on these insights and created this checklist as a companion to that report. While not meant to be a comprehensive guide, this checklist will help your organization get a solid start and begin thinking about how to use AI in inclusive ways.
To find definitions for the technical terms in this checklist, view the Glossary in our AI & Disability Inclusion Toolkit.
Ask Before You Buy
Your vendor should be able to provide an AI transparency statement for any new technology your organization procures. AI systems use algorithms, which are sets of rules or processes that computers can run on their own (e.g., looking for trends in data). Before your organization procures an AI-enabled technology, ask the vendor the following questions:
What datasets does the platform use to train the algorithms to match candidates to jobs? How do the vendors test or validate the algorithms’ effectiveness?
Did the vendor incorporate diverse perspectives in designing and testing the platform? Is there any evidence or documentation of this?
Does the platform incorporate bias mitigation, including for disability bias?
Does the vendor have an AI transparency statement? If not, how will they address that fact?
Does the product conform to accessibility standards? Can the vendor provide an Accessibility Conformance Report (known as an ACR) showing how the product conforms to standards?
Ensure Mindful Implementation
After procuring an AI-enabled hiring technology, your organization needs to use the technology in inclusive ways and continually check that it is making fair decisions. As your organization implements the technology, ask the following questions:
Does your organization involve people with disabilities in the decision-making process regarding the technology’s rollout?
Does your organization ensure that humans are continuously involved in auditing AI processes as they evolve?
Has your organization defined how this tool will be used in coordination with disability-focused hiring programs?
Plan to be Transparent
Your organization should inform applicants about how AI systems factor into the recruiting and hiring processes. AI systems analyze data to make predictions and recommend decisions (e.g., reviewing resumes to determine candidate fit for a job). These AI models offer results that hiring managers can use to inform their decisions.
To be transparent, hiring managers should ensure that applicants receive plain language explanations about how their organization uses AI tools in hiring. These explainability statements give applicants context for how their qualifications are assessed and for how the organization fosters equity in the process. As your organization begins to use the technology, ask the following questions:
Does the platform provide a plain language explainability statement that hiring managers and applicants can understand?
Are applicants able to opt out or request accommodations once they understand how AI is used in the hiring process?
Can applicants provide feedback to alert your organization to problems they encounter?
Mitigate Bias & Monitor for Disability Inclusion
Your organization should work to mitigate bias and monitor for disability inclusion. As your organization begins to use the technology, ask the following questions:
Prior to adopting AI hiring technology, what methods did your organization use to sort essential versus preferred qualifications in job postings and job matching? Are they weighted differently in the algorithm?
How does the platform support candidate disability status self-disclosure while also respecting individual privacy?
Does the platform allow candidates to request reasonable accommodations for other parts of the hiring or onboarding process that do not involve AI (e.g., captioning for interviews)?
How does the platform address the needs of multiply marginalized candidates?
Remember, this is just the beginning. AI tools must be continually monitored to ensure they produce fair outcomes for job candidates. To learn more about AI bias in hiring tools, please visit PEAT’s AI & Disability Inclusion Toolkit.
