WeFit

Inspiration

Fitness has always been gruelling work and can be hard to start. Fitness, however, is important as it can decrease our chances of chronic diseases. Making fitness entertaining solves this problem and was, in fact, experimented with in the 2000s with Wii Sports. Unfortunately, it did not become popular due to the execution. We were inspired by Wii Sports and decided to tune their idea to the modern era.

What it does

WeFit is a motion-controlled Subway Surfer game. The actions you can take are jumping, sneaking, sliding left, and sliding right. The goal of the game is to dodge any obstacles and collect coins. The further you progress, the more difficult the game gets. For example, you might see moving trains, speed increases, and more obstacles.

How we built it

Architecture Diagram ESP32 microcontroller transmits motion data via IMU sensor to Web Interface. Using 3.js as our game engine, we implemented a randomization system for obstacles. Our backend stores any data collected (i.e. movement stream, high score, user data) to the Postgres database.

Challenges we ran into

We had to heavily tune the thresholds for what motion was read and what motion was not. We ended up making a variety of thresholds, in this case different acceleration values, for the x and z directions in order to account for small human movement. This made it so that the game motions only triggered when the player moved at a high acceleration, thus promoting high intensity exercise. Using wireless connections like Bluetooth introduces some level of latency, which the gaming consumer space is highly sensitive to. We actively log and report latency results to keep track of potential weaknesses in the product. The IMU sensor accuracy can degrade over time due to drift. Because the hardware handles recalibration internally, it can cause inaccuracies, delays, and degradation in sensor quality during the recalibration periods.

Accomplishments that we're proud of

Bridging the gap between accessible consumer hardware and entertainment to deliver novel gaming experiences to players, while promoting fitness and health goals. Specifically, we are proud that we accurately captured motion to translate movements consistently with minimal errors while encouraging high intensity exercise. We are also proud that we were able to establish analytics to report data such as latency statistics, input data, and calories burned.

What we learned

We learned about hardware software interactions and the importance of latency and throughput. Additionally, we learned about how to optimize our sensors for human motion which is oftentime irregular. As for the software side, we learned about randomization algorithms such as Perlin noise to develop a never-ending game that is engaging.

What's next for WeFit

Next is POSITION, specifically pitch and yaw. Currently we only measure vector acceleration which is why we are facing some errors with movement. The body parts of a human do not always accelerate in the same direction, thus, just measuring acceleration and direction is not enough to fully translate human movements into the game. We must introduce a position to accurately capture the complexities of human movement and output not just acceleration data, but also intent of where the player wants to move in game. This is the next step that WeFit will be pursuing.