Your organizational culture often determines what gets done, so the organizational vision for Equitable AI must be apparent in everyone’s day-to-day work. Once your initial Equitable AI implementations are completed and your governance structure has been put in place, it is important to make continuous efforts to ensure that the change is seen in every aspect of your organization. This will help give ingrain Equitable AI principles into your organization’s culture.

Evaluating your success is an essential step to visit throughout each Equitable AI implementation, creating a cycle of continuous improvement. You can use what you learn in this play to measure the progress you’ve made, celebrate what your community has achieved, and identify where you can course-correct to ensure you accomplish your vision for Equitable AI.

Tips to Get Started

• Continue to measure principle-based outcomes across the initiative. Use the metrics you developed in Play 8 to ensure an ongoing quality management program. Metrics paint a picture of the status of your initiative, show progress, and create accountability across the Equitable AI Principles you adopted. Organizations like the AI Ethics Impact Group offer a practical, principle-based measurement framework.¹ As outcomes are measured across AI implementations, your organization can better determine where challenges exist, what’s working well, and where to focus resources to improve outcomes.
• Tie your measured outcomes to larger business goals. As you measure adherence to the Equitable AI Principles across your AI implementations, link the results back to your organization’s short and long-term objectives. Doing this can help you understand the impact of your AI program on business growth, market reputation, and social responsibility. For instance, you may decide to measure the number of persons with disabilities who you have hired into the organization and measure the retention of these individuals over time.  Those measurements could enable your organization to achieve its goal of being an employer of choice for people with disabilities.
• Regularly review and check progress. Your Equitable AI Hub should regularly review progress with senior leadership and with your Board, in addition to key internal stakeholders engaged in your organization’s DEIA efforts and participating in ERGs.  Reviews can help you celebrate successes, discuss challenges, and ensure sufficient resources are in place for continued success.
• Continuously build organizational knowledge and competency. Successful Equitable AI initiatives are built off an ongoing and sincere commitment and efforts to continuously learn and improve. As you monitor progress, identify opportunities to streamline what you’re doing, make tweaks to your process when things aren’t quite right, and stop doing the things that aren’t working. Continuously build organizational knowledge about Equitable AI by partnering with vendors, researchers, and consultants. You should also make sure your organization is on top of the latest trends in the AI industry to ensure your policies and practices are current. Consider external trade publications, media, and organizations you should follow such as:
  • Global Partnership on AI – Responsible AI Working Group
  • RAI Global/World Economic Forum- Responsible AI Certification
  • IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (Ethically Aligned Design)
• Create new positions within the organization as needed: According to the Harvard Business Review, HR may soon transform to include new roles related to AI such as: Human Bias Officer, Chatbot and Human Facilitator, HR Data Detective, Algorithm Bias Auditor, VR Immersion Officer, and more. As your initiative matures, consider hiring staff for these new roles and involving them in the Equitable AI initiative.

Guiding Questions

1. How does your senior leadership and Board review ongoing progress with other strategic IT and business initiatives?
2. How do strategic IT and business initiatives influence and shape your organization’s overall business strategy?
3. Does your organization already have a strong change management culture?
4. What are your markers of success and how can you measure success?
5. How will you monitor progress over time and share progress within your organization?
6. How will you know when you’ve achieved your vision?

Resources

1. AI Ethics Impact Group, led by VDE / BertelsmannStiftung, From Principles to Practice: An interdisciplinary framework to operationalise AI ethics, 2020. Licensed under a CC Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.
2. Stephen Ibaraki, Responsible AI Programs to Follow and Implement – Breakout Year 2021, Forbes.
3. IEEE, IEEE P7000 – IEEE Draft Model Process for Addressing Ethical Concerns During System Design.