HacMan

Our logo
Our team

Inspiration

Nathan was cold while walking back from Howey Physics, so he wanted to put his hands into his pockets. Unfortunately, you can't play Pacman if your hands are in your pockets! We were inspired to prototype a handless game, whose core idea has the potential to become an accessibility feature.

What it does

Play Pacman without lifting a finger!!!!! Activate power ups by blinking your eyes!!

How we built it

The Hacman uses the dlib facial feature recognition package to track key points of the face (edges of the eyes, vertical length of face) using the computer's webcam. Using the ratio of distances between various features, we can determine the pitch, yaw, and roll of the head. In this way, the human head operates as a joystick, enabling touchless Pacman gameplay. We run a Flask server in parallel with the computer vision algorithm to serve the Pacman webapp, which receives "joystick" information from the Python server. The webapp uses Create.js and has a modified Pacman game we built from the ground up. We made sure to play sad Joji when you die, in order to create an enhanced Hacman experience.

Challenges we ran into

Everything. Nothing didn't go wrong. We want y'all to know that we all pulled an all-nighter since we're all so committed. We couldn't figure out collision with the walls for a very long time, so the HacMan ran around eating a lot of wall and also colliding with seemingly random invisible objects :( Eventually it was resolved by realizing our array list was indexed incorrectly so our collision map was rotated 90 degrees from the actual visible map. In itself, designing a control scheme using only one's head took significant deliberation and went through many revisions, from increasing the number of rotational axes (twice!) to constantly tuning the multitude of threshold limits and scaling factors Also we had a billion merge conflicts.

Accomplishments that we're proud of

Different members of our group explored different parts of CS that we've never tried before! It was our first time doing UI/UX for some of us, while it was the first time trying backend for others! Going in, we were looking to strike a balance between working on complex systems and making a fun and memorable experience. Interfacing a rotation-based control system with a game that suits both the theme and the occasion, we pushed the limits of what is possible in such a short period of time and created a project that will inspire us (and hopefully others) in our future coding endeavors.

What we learned

We learned that we should not all push onto the main branch in Github. This caused some people's code to block other's progress, since the main branch was not guarenteed to be always functional. We all became comfortable using Anaconda environments for Python, a widely applicable tool, and we learned about the communication between frontend and backend systems using get and post requests.

What's next for HacMan

We want to all take a nap first. Next, we want to enable HacMan to utilize the browser's webcam instead of using the webcam from the python server. This would allow HacMan to run from any device with a browser and a webcam, and all processing would be done on the Python server. Additionally, we want to implement more features and powerups in the game, and make the face-joystick more robust.