We wanted to help ease the awkwardness of when you forget people's names in networking situations. You meet many people but it's really hard to remember their names. It would be really nice to see their name and other information (such as interests) alongside their faces.
What it does
Displays the name of a KPop star that is most similar to the person out of all 80 stars in our database next to the face of the person in augmented reality. Additionally, we expose a web interface for people to upload photos to see which KPop star it is. You could imagine this feature being a framework for Buzzfeed-like quizzes that compare the face of a user against all of the characters from a certain show.
How we built it
We explored ARkit to grab the faces from the phone and display text near their face. We use the face_recognition package along with data we collected for recognizing faces. Numpy/Scipy are used for vector manipulation of the encodings.
Challenges we ran into
Face tracking was hard. Anchoring scene elements to the AR Scene View based on relative camera position was especially difficult. Doing the face tracking and rendering in the same frames was challenging because of the immense amount of computation needed each frame. It was also hard to find adequate ways to aggregate data about users, which caused us to try to have another way to show the power of augmented reality in the context of useful information near a person's head.
Accomplishments that we're proud of
Scene elements are anchored to the faces! The API we created for uploading faces for analysis is fairly easy to use.
What we learned
Swift, ARkit, more numerical computation experience
What's next for KPop Queen
We want to expand more into the information space so we show user information by the face. Given more time we can integrate with Facebook to get more of that information.