GIF
Real-time eye tracking in action!
The game modes
View when first launching a game

TRY IT NOW

Inspiration

The idea for this project came to me after spending countless hours playing higherlowergame.com, where players guess which of the two options is more frequently searched on Google. For this hackathon, I explored TensorFlow.js's facial landmark detection model, which recently introduced 10 new highly accurate iris-related landmarks. This sparked an idea: since the game's controls are so straightforward, why not use real-time eye tracking in the browser to create a more accessible version for people with limited hand mobility? My hope is that this will serve as an example of how open-source ML learning models can be used to make game controls more accessible.

What It Does

Eye Tracking System:
- Uses TensorFlow.js and MediaPipe's face mesh detection to track the iris position
- Calculates which direction the player is looking
- Requires calibration to establish a baseline position
Game Mechanics:
- Supports multiple game modes:
  - Spotify (monthly listeners)
  - Country population
  - Country size
- Shows two options (left and right) for players to choose between
- Players make selections by looking at their chosen option
- Scores are tracked, and high scores are saved to a database
Visual Feedback:
- Progress indicators show how close the player is to making a selection
- Active color gradients provide visual feedback during the selection process
- Results are displayed with green/red indicators for correct/incorrect answers

How It Was Built It

This project was built using Gadget, a serverless full-stack web app platform, and TensorFlow.js. Gadget provided infrastructure for user authentication, data storage, and frontend hosting, while TensorFlow.js enabled the real-time face tracking functionality. The game interface was developed using React with Tailwind CSS for styling.

For data sources, datasets for monthly Spotify listeners, country populations, and country sizes were generated.