We saw a picture on Reddit who looked very similar to a person in a painting. We believed that we could implement a web app to automate this.
What it does
DoppelGallery is a facial feature recognition web app that attempts to find faces that look similar to yours from open source art databases online.
How we built it
The facial recognition software is based on an open source python library called Openfaces. In order to successfully build and use it, we ran the software in a virtual machine called docker. The web interface backend interacts with Python through Tornado.
We also used CockroachDB to store metadata for each painting, and how similar each painting was (for reinforcement learning).
Challenges we ran into
Learning how to use docker was relatively challenging, and while the Openfaces software does the majority of the facial recognition work, effectively implementing its python interface was very time-consuming. Furthermore, since we are attempting to find your Doppelganger, we needed a large image database. Associated with this were long download times and image processing.
Accomplishments that we're proud of
Figuring out how to get 1/5 of the world's largest database on art in less than an hour (Jack) Writing a web app with HTML+CSS+JS in like an hour (Jeff) Processing 100k faces in three hours (Alex)
What we learned
ML is pretty hard. Getting data can be even harder.
What's next for Doppel Gallery
Though the app is a novelty, the underlying technology has many useful applications in real life as a reverse face search (like Google's reverse image search).