Petr Jump

Inspiration

Petr Jump was inspired by the notion of combining healthcare and entertainment, similar to the spirit of Nintendo's Ring Fit Adventure and the mechanics of Doodle Jump.

What it does

Petr Jump reads acceleration data from a player's jump and translates that input into character movement. When a player jumps, their associated character jumps as well. The goal is to ascend as many platforms as possible and avoid falling off!

How we built it

In terms of the hardware, we used the ESP32 S3 DevKitC-1 board as the microcontroller responsible for sending data up to our pygame implementation of Petr Jump. We also utilized Adafruit's LSM6DSOX accelerator to read a player's acceleration from their jumps, alongside a joystick to allow players to move left or right in the game.

Regarding software, we programmed our microcontroller using C++ on PlatformIO. We used Python to program our game server which was responsible for connecting to and receiving data from the ESP32. We used pygame to develop our interactive video game.

Challenges we ran into

FastAPI and moving to UDP: We initially wanted to have a clean backend game server with FastAPI, but we ran into a number of problems when combining FastAPI protocols with ESP32 code. We opted for UDP for faster real time response.

Data overload: We also hit the issue of the ESP32 overwhelming the game server by sending too much information into the server. This led to game issues such as freezing due to an alleged lack of input (because the server couldn’t process ESP32 data fast enough), but we were able to delay the output from the ESP32 enough for the server to process the information and pass it on accordingly to the game.

Acceleration and calibrating based on its raw data + super sensitivity: We had some issues with calibrating the accelerator accordingly to appropriately set reasonable jump heights for the player, since we were working with raw data from said accelerator. We also had issues with the joystick being incredibly sensitive to player input. However, we were able to resolve the problem by scaling input values accordingly.

A side challenge we ran into was not being able to procure a pulse sensor, but if we got another chance to do this project, we’d definitely incorporate it for reading player data.

Accomplishments that we're proud of

We’re proud to have gotten a minimum viable product going within the timeframe allotted, learning a new game library (pygame), and learning a new microcontroller board.

What we learned

How to integrate an accelerometer, joystick, and ESP32 How to develop a video game using pygame How to send data from an ESP32 over WiFi using socket programming How to develop better PlatformIO practices

What's next for Petr Jump

We would like to improve graphics if we had more time and perhaps game mechanics as well. We’d also like to include multiplayer with multiple ESP32 player units.