Computer Vision API Prejudice Audit

Inspiration

• Major tech companies are rolling out enterprise solutions that leverage their computer vision APIs
• These tools are being applied to decision making situations that can have significant impact on the individuals and society at large

What it does

• Addressed the research question: Will image classifiers give different predictions about stock images of doctors split by race and gender?

How we built it

• Collected 247 images of doctors in two gender and four race categories from ShutterStock
• Ran images through 4 APIs (Google Vision API, Clarifai API, Amazon Rekognition, Microsoft Azure Computer Vision API) and extracted the predictions of physician/doctor
• Evaluated differences in prediction confidence levels for doctor/physician across the eight separate groups
• Manually downloaded and labelled images
• Used python and jupyter networks for data wrangling, api access,
• Used R for statistical analysis

Challenges we ran into

• APIs do not provide their classes for proprietary reasons - presented an issue with respect to missing data

Accomplishments that we're proud of

• Local Hack Day University of Guelph First Place
• Great feedback

What we learned

Results of our case study:

Google Vision API

• Men significantly higher than women
• Black significantly lower than other three groups
• Black Female lowest predicted class

Amazon Rekognition API

• Males significantly higher
• No bias at race level
• White Female lowest predicted class

Clarifai API

• Females significantly higher
• No bias at race level
• Black Male lowest predicted class

What's next for Computer Vision API Prejudice Audit

• Continued investigation into bias in CV APIs
• More images for a larger sample size

Built With

• amazon-rekognition
• api
• clarifai
• google-cloud-vision
• python
• r