FaceME

CV-based face similarity scoring flow
"Join multiplayer session" landing page
Annotated UI Flow
Annotated backend
End page top scores podium
Initial testing of face similarity scoring testing

What it does 🙋❓

FaceME is a competitive multiplayer scavenger hunt game inspired by classic searching games like "I Spy" and "Where's Waldo?" with a modern twist. Prompted with an AI-generated face, players race around the room to find real-life people who most closely resemble the doppelgänger. Using a session code, multiple players can connect to a shared lobby and compete to find the closest match, with an ML encoder system automatically judging the similarity between the user's submissions and the AI-generated face, awarding points. The player with the most points wins!

How we built it 🔨👷

We built FaceME by splitting up the project into manageable components. The main things we had to deal with were:

Creating a user-friendly UI.
- Landing page to connect players to a multiplayer session.
- Allow the user to take pictures while viewing target.
- Display winners!
Generating realistic faces using StableDiffusionXL, and writing scripts to generate prompts for the model.
Recognize and bound faces from StableDiffusion output and user input using MTCNN.
Crop and resize faces to meet encoder input dimension requirements using PIL
Generate embedding vectors from cropped face images using ArcFace
Calculate player scores by computing embedding distance with numpy

Each of these was accomplished by using our witty programming skillz.

Challenges we ran into 😱🚨

Some challenges we had were that we couldn't run our face generator locally and hosting our webapp was proven to be difficult. However, the most difficult challenge was integrating our backend with our frontend. On Windows, our flask app did not reproduce all the intended features as MacOS. A major issue was the improper handling of exceptions, which led to a few hard to spot bugs.

Accomplishments that we're proud of 😺🐶

We're mostly proud of the work we put into our front-end. None of us are very experienced with front-end programming, so it was a big struggle integrating our game into a flask app.

We are also proud of the back-end of our game: the central premise of our project required a robust pipeline for generating AI faces and ingesting user images, and performing face recognition, cropping, encoding, and comparison.