Facial Recognition Racial Bias

Inspiration

The inspiration for this project originated from a common experience of mine. Oftentimes, my teachers or peer would mistake me for a friend of mine who is also east Asian or vice versa. This prompted me to question whether this phenomenon is distinct to humans or are other things, such as software also prone to such tendencies. This led me to investigate how much bias there is in facial recognition software.

What it does

my visualizations point out specific biases in DeepFace facial recognition software, and my data is newly generated from datasets of images so no one has seen them before.

How we built it

Challenges we ran into

I had originally planned to compare the accuracy of one-to-one image matching for the different races using an open-source facial recognition software -- deepFace. However, I could not find a dataset readily available that would allow me to test that (i.e. both have race labeled for each image and have multiple images per individual).

I was unable to find a downloadable dataset of their study results so I had to generate my own. In order to attack this problem, I had to go about it in a roundabout way. I was able to find a set of data created by the LFW organization that consists of 9162 images of approximately 1,600 individuals. Unfortunately, this data set does not include labels about the person’s race. Thus, I decided to run all the images through a race detection in DeepFace with the intention of then matching the images - now identified with race, and find the accuracy. However, due to time constraints, I was only able to finish the racial recognition portion. Even so, I had plenty of data to analyze.

Accomplishments that we're proud of

I am really proud of how hard I’ve worked and the sheer amount of hours I’ve put into this project.

What we learned

In terms of technical skills, I learned to navigate Panda, Matplotlib, as well as how to approach a problem I am interested in.

In terms of my findings learned a lot about the biases built within the facial recognition system. For example, looking at the “Misidentification Probabilities for Each Race” graph, you can see that it is quite likely for an Asian or Middle Eastern person to be misclassified as white yet white subjects are not likely to be classified as Middle Eastern or Asian. This suggests that the data this program was trained on had many more white subjects than Asians Middle Easterns, and etc.

What's next for Facial Recognition Racial Bias

Moving forward, I would of course finish the original plan of testing the one-to-one matching and then compare the results of each race for disparities. Assuming there are disparities, I would then crop all of the images so that it only includes facial features. Putting these cropped images into the program could tell us what features the algorithm is looking for when identifying humans and perhaps see if it focuses on different things for different races. Next, I would turn the skin color of all the images to an artificial color such as blue and pass it into the program. This will allow me to assess how much skin color matters in recognition and again, how the different races compare to each other on this score.

Built With

• deepface
• matplotlib
• panda
• python
• wfldataset