Cory was browsing the web and came across a photo of a veterinary assistant holding a dog. He noticed that her ID tag was blurred out, and realized that someone had to manually do that. Having to manually edit out sensitive information sucks, because what if you accidentally missed something? What if you forgot to edit just one photo, and now your address or something is out there on the internet?
People in the public light like celebrities, influencers, and journalists could be at risk if they accidentally release personally identifying information, so what if something could automatically censor their photos for them?
What it does
- Scans an image for faces
- Identifies the main face the photo focuses on
- Marks out all other faces, either with a box or emoji matching the person's mood
How we built it
- Amazon Rekognition: Facial recognition and sentiment analysis
- Python: service backend
- Flask: web server
- Pillow: image manipulation
Challenges we ran into
AWS uses a relative coordinate system where 0 is the left/top edge and 1 is the right/bottom edge. Translating these coordinates to actual coordinates in the image was confusing at first. We figured out how to convert the two by comparing AWS' coordinates to the image coordinates we expected.
Originally we stored images to an S3 bucket and then made Rekognition API calls to analyze the images in the bucket. We quickly realized that storing images somewhere went against our main focus of privacy. To counter this we switched to a stateless API call that sent the image bytes directly to Amazon Rekognition without storing the image anywhere.
While testing the application we were getting unexpected results where images didn't seem to be updating. Eventually we remembered that browsers tend to cache images, and discouraging caching on the images gave us the results we expected.
Accomplishments that we're proud of
Getting facial recognition working seamlessly, sentiment analysis, webpage integration
What we learned
- Facial recognition and sentiment analysis with Amazon Rekognition
- GCP Compute Engine
- Python + Flask for web backends
- Image manipulation
- Don't doubt ur vibe
What's next for Face Off
- Specialized text filtering (such as addresses, credit card info, phone numbers)
- Context-aware fill for more natural looking image removals
- Optional alert system if a user posts a photo of themselves with personally identifying information