FaceYourself

Inspiration

We wanted to create a game where you play hide and seek within a city, but everywhere you pass, your face will be remembered (as in, your face stays on the buildings). The face would be a live feed from the webcam.

How we built it

• We made use of public StreetView datasets for raw images with location coordinates to bypass API call limits
• We relayed webcam data over to a server along with user keystrokes, so that the player's face can be implanted onto the streetview
• We made use of fast neural style transfer, contour analysis of scenes, semantic segmentation (detectron2) to decompose scenes into components such as they sky, road, cars, etc, and transfer weighted/interpolated masks of the users' warped faces in real-time
• We built a front-end with javascript, and connected it to flask python server to run all the image processing and load balancing

Challenges we ran into

A game on a website interacting with a server is difficult to implement, and having multiplayer was also very difficult. The name of the project changed from Face Your Stalker, which would have been the name if multiplayer was a thing, to Face Yourself, because we only got one player working.

What we learned

Don't try doing too many features, especially if you don't know what you're doing.

What's next for FaceYourself

Sleepless nights crying for all four team members.

Built With

• flask
• google-streetview
• javascript
• opencv
• python
• pytorch
• webgl