AI Gender Bias - Project

Abstract

Gender bias is rampant in AI systems, causing bad user experience, injustices, and mental harm to women. School curricula fail to educate AI creators on this topic, leaving them unprepared to mitigate gender bias in AI. In this paper, we designed hands-on tutorials to raise AI creators' awareness of gender bias in AI and enhance their knowledge of sources of gender bias and debiasing techniques. The tutorials were evaluated with 18 AI creators, including AI researchers, AI industrial practitioners (i.e., developers and product managers), and students who had learned AI. Their improved awareness and knowledge demonstrated the effectiveness of our tutorials, which have the potential to complement the insufficient AI gender bias education in CS/AI courses. Based on the findings, we synthesize design implications and a rubric to guide future research, education, and design efforts.

Study Motivation

Gender bias broadly exists in AI tools for recruitment, recommendation, and many other scenarios that people regularly encounter. As a result, procedural and substantive injustices frequently arise against women. For example: women receive fewer job posting advertisements on social media; medical AI is less effective for women; and so on.

Gender bias in AI thus remains a pressing issue, and AI creators are the ones who are responsible for building bias-free AI systems. Yet, AI creators often lack knowledge of preventive measures to avoid bias in AI applications. Male AI creators, who account for a significant portion of this workforce, have less awareness of gender bias than female creators. Equipping AI creators with both awareness and knowledge of AI gender bias is timely and important. However, little research has been devoted to understanding how and how well AI gender bias is taught in CS/AI courses. To our knowledge, there is only one AI gender bias education tool devoted to teaching this topic to children and youth, effectively raising their awareness of gender bias in AI. However, this tool is not suitable for AI creators who need more technical knowledge such as debiasing techniques. The best way to introduce the topic of AI gender bias to AI creators remains under-investigated.

Design

To bridge the above mentioned research gap, we designed hands-on tutorials to teach AI gender bias to AI creators and equip them with awareness and practical knowledge. At the awareness level, we intended to help them better recognize gender bias in AI and motivate them to solve this issue. At the knowledge level, we intended to convey technical knowledge, e.g., how gender bias is introduced into AI (sources of gender bias) and how gender bias can be mitigated from AI (debiasing techniques). To engage the learners, we designed the tutorials in a hands-on manner and used real-world scenarios, i.e., AI-based recruitment, which helps organizations effectively source and screen candidates, and AI-based autocomplete, which search engines use to complete searches that users start to type, to facilitate the learning activities. To make the education more meaningful for real-world AI development, we embedded technical knowledge (e.g., debiasing techniques) and components (e.g., code, dataset) into the tutorials.

Tutorial Link

Click to Access AI Gender Bias Education Pragram.

Evaluation

Eighteen AI creators evaluated our tutorials, including four AI researchers who focused on the technical aspects of AI, four AI/HCI researchers who focused on the user/society aspects of AI, three AI developers, two AI product managers, and five students who had learned AI.

Results

Status quo of AI gender bias education

In the pre-study interview, the participants commonly expressed their lived experience of gender bias when using or creating AI, and indicated insufficient education in CS/AI courses and no education tools on this topic. The lack of education made them unable to identify gender bias in AI and mitigate gender bias from AI.

Education effect of our tutorials

After completing our tutorials, the participants showed an improvement in terms of both awareness and knowledge of AI gender bias. Awareness-wise, they expressed a strengthened ability to identify gender bias in AI and an enhanced intent to address the bias issue. Knowledge-wise, they perceived a heightened level of technical knowledge, evidenced by a higher accuracy rate in knowledge question (KQ) surveys, and felt more confident in debiasing AI.

Usability of our tutorials

The participants expressed a generally good user experience of our tutorials and also suggested areas for further improvement.

Takeaways

Main Findings

1. AI gender bias education is urgently needed yet insufficient.

2. Our tutorials improved AI creators’ awareness and knowledge of AI gender bias.

3. Hands-on activities are highly effective in engaging learners.

Design Implications

1. Equipping Tech People with Gender Awareness.

2. Accommodating Different Levels of Tech Literacy.

3. Promoting Workforce Diversity.

"I'm Not Confident in Debiasing AI Systems Since I Know Too Little": Teaching AI Creators About Gender Bias Through Hands-on Tutorials