InkJump

close-up of the game projecting what lines it "sees"
Andrew working on creating a more accurate mapping

Inspiration

We were heavily inspired by this projection-mapped pop-up book, but we decided to take it one step further and allow for the user's creativity in designing their own levels. We loved the concept of combining a physical drawing with a virtual environment that the user can interact with, and our product really shows that.

What it does

InkJump is a game where the user draws a platformer level on our paper that they can then play. First, we have our backend scan the lines drawn and the corners of the paper to account for distortion. Then, that image is sent over websocket to the frontend game engine where the player is then projected back onto the paper and the user can play their level.

How we built it

Our project is split into two discrete parts, a Python3 backend running OpenCV and a Websocket HTTP server and a frontend using P5JS to render the game, handle the physics, and normalize for projection screen space. The backend and the frontend talk over a high-performance websocket so that we can quickly get new levels from the camera once OpenCV detects them. Due to the many moving pieces involved, we took full advantage of our team's experience with Git and Github so that we could split the project into independent parts and functions and then later Git merge them into a cohesive final project.

Challenges we ran into

We ran into challenges with OpenCV and trying to normalize the camera footage to take into account lens distortion, our next major problem was properly pulling out the contours from the image. This was a balancing act between not missing any lines, but also not having it accidentally connect lines that should not be connected.

Accomplishments that we're proud of

We are especially proud of how everything came together. The most stressful part of a group project like this is combining all the different moving parts and hoping it works, but thanks to our collaboration and lots of communication, we were able to near-seamlessly combine the frontend and backend.

What we learned

Due to our varying skill levels, we all started at different places. One thing we did all learn and work on, though, is collaboration and how to effectively work as a group on a software project. We divided up tasks throughout the hackathon, but would frequently talk aloud to each other and brainstorm ways to most effectively accomplish tasks and figure out ways to solve problems we were encountering.

What's next for InkJump

Some expansion ideas we had include adding preset start and end points on the level paper so that there is a win state in the game. Another idea we had that we didn't have time to implement was creating a website for our program that would upload the level scans so that people could replay their levels and other people's levels online.