AIR WHACK

Overall
When circle is right above, you want to move your yellow object to the corner where it has a circle with the same color.
Contouring
Align yellow object with correct color

Inspiration

Hello! I don’t know about you, but man do I miss playing games. Music games especially. When was the last time I danced my heart out on a dance dance revolution machine? When was the last time I played Just Dance with my friends? With COVID-19 still at large, I may have to kiss my only form of exercise goodbye… wait did someone say Air Whack?

What it does

Air Whack is a game served on the web that allows you to play a fun and interesting take on a dance genre game with just the help of a laptop. So how does it work? To play the game, you must hold a yellow object to the camera so our program can detect your hands’ positions in the frame. When you see the circle right above the box, move your hand to the corner with the corresponding color of the circle.

How we built it

For the server that's hosting the application, we built it with flask, leveraging its streaming functionality.
The image processing and manipulation that renders game objects and performs object detection and object collision detection is built with OpenCV-python.

Challenges we ran into

Now let’s talk about the challenges we faced. First was the implementation of object detection and object collision. Due to our inexperience with OpenCV and Computer Vision, we had trouble understanding and finding different implementations that suits our needs the best. We tackled this challenge by attempting different methods and choosing the best after comparing the results. Second was the implementation of a rhythm game with opencv. We had no idea rhythm games were this hard to implement. From misalignment of the beat with the circles due to frame rate differences, to using OpenCV as the canvas for our game. We’ve come to appreciate the well-developed music games out there.

Accomplishments that we're proud of

Stuck till the end of the Hackathon with something to demo, especially when it's so easy to give up and just flop on the bed that is right next to us.
Learn, experiment and utilize OpenCV without having any prior experiences with it

What I learned

Learned how to detect objects using computer vision.
Learned different ways of manipulating images with OpenCV-python.
Learned of a concept in python called generators.
Learned of the complexity in rhythm games.
Teamwork makes the dream work

What's next for Air Whack

Make the game look prettier
Implement generic mechanism that takes in an array of certain time points in a song, and generate the corresponding falling orbs
Implement functionality that upon receiving a song, generates an array of time points in which an orb should be spawned
Improve or redesign object detection functionality such that object is detected consistently