Web Cam Controller for Piano Tiles

• 

  Usage of controller

Inspiration

Inspired by my passion for Computer Vision and the game Piano Tiles implemented using python by pyGuru123.

What it does

It allows a user to play Piano Tiles using their webcam.

How I built it

I took inspiration from pyGuru123's piano tile project, which was created in Python using pygame. I completed an end-to-end Computer Vision project, starting with data collection, image labeling, and image augmentation, then model training and fine-tuning. I then deployed the model to my webcam and used the results from the image classification and object detection to track the user's hands. I then integrated this with the PyGame application.

Challenges we ran into

It took some time to pre-process the data and train the model. It was also challenging as it was the first time I extended the use of the output of YOLOv8 for purposes other than classification and object detection.

Accomplishments that I proud of

Proud to have been able to complete the project I imagined exactly how I pieced it together, despite doing it in a day and solo.

What we learned

I learned how to extend the use of YOLOv8 other than just loading and using the model at face value.

What's next for Web Cam Controller for Piano Tiles

I will try to build the game into the webcam straight, using cv2, so that it doesn't have to be 2 separate windows.

Built With

• cv2
• pyautogui
• pygame
• python
• ultralytics