Amazon Fires Secret Sexist Recruiting AI
When it comes to giving AI control over recruiting and hiring, there's a potential drawback that employers need to be aware of: racism, sexism and bigotry are all learned attributes, and machines can learn them too.
Amazon recently learned this lesson the hard way. It basically had to abandon a project that involved using AI to find ideal candidates when it found out that the AI had learned to be a sexist. What's worse is that the AI learned it from Amazon's own past hiring practices.
Garbage in, Garbage Out
The problem for Amazon's AI was that the team that created it relied on Amazon's own prior successful applicants, which have been heavily dominated by men, like much of the tech industry. In addition to objective criteria, the AI reviews the type of language applicants use, and many other factors to score the applicants. However, because the AI learned from a pool of successful candidates dominated by men, the type of language it started to value higher became the more masculine words like "executed" or "captured."
In tech, this phenomenon of a machine providing bad results due to bad data is known as "garbage in, garbage out."
Teaching AI to Not Discriminate
The big problem with AI and machine learning, at the moment, is the fact that systemic discrimination, both along race and gender lines, has been going on for so long and is so ingrained in society, that there's almost no way for an AI to not pick up on it. What we all now understand as implicit bias gets translated into actual bias by AI.
However, AI can still be helpful in recruiting and hiring, just not when it comes to choosing what criteria to value. Notably, it can simply be used to anonymize applicants until it is actually interview time.
Related Resources:
- Age Discrimination Claims for Facebook's Targeted Ads (FindLaw's Technologist)
- Why Asking Applicants Riddles Is a Bad Idea (FindLaw's Technologist)
- You Can Train AI to Spot Legal Issues -- for Fun (FindLaw's Technologist)
