During our annual read-through-stupidhackathon-during-the-opening-ceremony-brainstorming-for-ideas-session, David came up with using eye tracking to count the the number of glances advertisements. Meanwhile Marek came up with using eye tracking to move around and fire lasers by yelling.

What it does

An online multiplayer game controlled with your head. Tilt your head to fly across the screen. Open your mouth to shoot bubbles. Your face will be displayed in real time to everyone playing.

How we built it

The main application is written in javascript and runs in your browser. Face tracking is based on clmtracker. The web server and game server use node.js and are running on linode servers.

Challenges we ran into

We started off with the ambitious goal of supporting multiple faces per computer. This provided a number of difficulties. At one point we were using three different computer vision algorithms on a single client. However, this had a huge impact on performance, and since javascript is single-threaded we had few options for improving performance. Therefore, we were forced to stick to one face per client in order to achieve good enough performance for gameplay, and for simplicity of the code.

In addition to performance, we faced issues with game network programming, and tracking down obscure bugs.

What we learned

We learned that client side javascript is surprisingly capable. Prior to this project, none of us ever anticipated that any computer vision applications could run purely in javascript. However, as it turns out, javascript is more than capable of handling computer vision, game rendering, and video streaming at the same time.

What's next for Face Off

We will be working on creating more maps, adding a player queue and more servers, and a ranking system. Giant lasers and firebreathing.

Built With

Share this project: