GitFit

Inspiration

I wanted to make movement part of the fun of gaming. I also wanted to try a hardware hack for the first time and build something that connected physical movement with software.

What it does

GitFit turns your right arm into a motion controller using an ESP32 and two motion sensors, one on the upper arm and one on the forearm.

Players connect through Bluetooth and calibrate with two poses: arm down and arm extended sideways. A live avatar then follows their movements across three games:

Zombie Boxing: Fight approaching zombies with jabs, hooks, and uppercuts. Faster estimated wrist movement deals more damage.
Bird Flight: Use downward flaps to gain altitude while flying through obstacle gaps.
Orbit Guard: Move a shield to intercept incoming projectiles, earning bonuses for accurate blocks.

Players can set time and calorie targets, view estimated movement speed, and track estimated calories during live play. I also added practice controls so people can try the games without the wearable.

How I built it

I programmed the ESP32 in C++ with the Arduino framework, reading two MPU6050 sensors over I²C. My firmware integrates gyroscope readings into normalized quaternions and uses accelerometer gravity for tilt correction. Two-pose calibration calculates each sensor’s mounting orientation and gyroscope bias.

I encoded both quaternions as signed 16-bit components inside compact 20-byte Bluetooth Low Energy packets, using NimBLE-Arduino and the browser’s Web Bluetooth API.

I built the frontend using HTML, CSS, and JavaScript, with Three.js and WebGL for procedural characters, environments, and animation.

I reconstruct the arm using forward kinematics with a fixed shoulder position. The upper-arm and forearm quaternions rotate their respective segment vectors to determine elbow and wrist positions. Both orientations use the same world reference frame.

I use wrist-velocity trajectory analysis to distinguish jabs, hooks, and uppercuts through path straightness, axis travel, and directional change. Swept collision tests detect strikes between sampled positions.

I used the Web Audio API to generate music and sound effects, Node.js for automated JavaScript tests, Python for documentation and calibration tooling, and GitHub Actions to verify changes before deployment to GitHub Pages.

Challenges I ran into

Calibration and natural movement: Early calibration rejected small, natural shakes. I improved it by averaging samples, using numerically stable variance calculations, and allowing brief movement spikes without restarting the entire capture.

IMU drift: I encountered yaw drift because the MPU6050 sensors do not have magnetometers. Gravity can help correct tilt, but it cannot provide a heading reference to correct yaw drift.

Accomplishments that I'm proud of

I built a complete path from physical movement to interactive gameplay: sensor readings, calibration, wireless transmission, motion interpretation, and visual feedback.

I’m especially proud of:

A shared tracking system powering three distinct games.
Two-pose calibration integrated directly into the lobby. I struggled with this because the sensors initially had no reference for their mounting orientation relative to the user’s body.
Procedural characters, music, and impact effects.
Wireless communication over Bluetooth, which helped make the experience feel immersive.

What I learned

I learned a lot about hardware, especially how inertial measurement units work and how gyroscope and accelerometer data can be combined to estimate rotation.

I also learned how I²C and Bluetooth work and how to use them to connect sensors, a microcontroller, and a browser.

Working across embedded firmware and browser graphics taught me to treat sensing, communication, motion processing, and rendering as one connected system.