This image is currently processing. Please be patient...
Inspiration Inspired by our intrepid classmate, we all wondered how fun it would be to get roasted. However, getting roasted is often a lengthy and fear-arousing process. Hence, we built RoastMe!
What it does RoastMe uses Artificial Intelligence processes to match your face to someone who's already been "roasted" on Reddit, hence providing laughs for everyone around at none of the cost. It uses Deep Learning to query a large corpus of images with another image based on rich visual facial features. An initial face tracker is used on the image that is captured on the webcam. The facetracker aligns the image, and crops it out of the frame. It then goes through several pre-processing steps before being fed into a very deep convolutional neural network. We use the VGG Faces pretrained network and extract the feature vector from FC7 layer. The layer enters a Markov matrix based distance match to find out the most similar matrices.
How we built it Over twenty-four tribulating hours, we scraped over 4,000 images from Reddit, and then built and trained a visual recognition program for it to be fed into based on VGG. It was then tested on a subset of 500 images by comparing our team's faces with those from the dataset, and we tuned the parameters to best match these fits. We created a ranking of images based off the L2 norm. Lastly, we created a well-designed website which would deliver its web camera feed to our visual recognition program, ultimately providing laughs for all.
Challenges we ran into Due to the nature of the project, we ran into obstacles at every turn: first, Reddit would not allow us to download images off of their servers, and we had to cleverly design around their bot-preventative obstacles. Next, we weren't sure on what norm to use to measure how similar two images were, because using the L2 norm (on the subset of 500 images) gave pictures of people who were not very similar to the user.
Accomplishments that we're proud of At every turn of the project, we were unsure if this would be feasible, and everyone told us otherwise. We're proud of having put together an application of Deep Learning technologies that we taught ourselves within twenty-four hours into a polished application. In particular, we are pretty pleased with how we aggregated the data and how we constructed a distance function for the neural network!
Finishing the project. After fixing the problem of finding the appropriate norm to measure the relationship between the user's face and the images scraped from Reddit, we were incredibly excited. At 5AM, we dashed to build a web app which would demonstrate our features, and added novel features such as live-tracking to enhance our application..
What's next for RoastMe There are still some problems in this implementation. For example, a few comments on r/roastme were highly dependent on the title of the specific Reddit post, and not on the image itself, or that were dependent on something in the image's background, which our algorithm does not incorporate. This meant that for certain images, we would get back the top Reddit comments that were not directly related to our user's face. In the future, we'd aim at using something like Machine Learning to optimize for the best "insult" to display.