Getting pictures to train the cup detection algorithm
Pong is as old as college, and as college students, God only knows how many games of pong we play a week. How much time is spent arguing about who won which game and wasted trying to figure out whose shot is actually the best? Clearly, this is an objective game, and we can track it so that the champion is always undisputed.
So, how do we determine who truly is the best at getting balls into cups? Obviously, the solution is a system that will make our difficult, difficult pong-playing lives just a little bit easier by constantly watching over our games and keeping track of our scores over time, all in the name of making the world a better place.
What it does
Sinkr uses computer vision to track when a ping pong ball hits a cup in pong. This data is then sent to iOS clients, allowing them to follow the game live. We also use our database to keep a leaderboard of top players
How we built it
The ball/cup tracking is handled using OpenCV, a computer vision library for Python. Our backend, also written in Python, stores information in a Google Firebase database in order to communicate with our front-facing iOS app written in Swift.
Challenges we ran into
One of our first big problems was setting up the camera to track the cups. We didn't have an external webcam, so setting up the computer physically in a good position to track was hard. We ended up modifying the algorithm to compensate for the challenges that came with having a physical laptop as the deadweight for our camera
Additionally, we ran into issues using machine learning (Haar Classification) to track the cups, as it took around 8 hours to finish creating the algorithm, and did not work at first. So, we shifted towards a color-tracking based approach.
Accomplishments that we're proud of
This is our first time working with computer vision, and so we're glad to see that working pretty well. Additionally, we got our entire front-end app and database working in pretty much the last third of the time allotted after scrapping an earlier plan to have a GUI on the same computer as our image processing.
What we learned
We definitely learned a lot about how to train Machine Learning algorithms, and just overall a good amount about OpenCV and computer vision.
Additionally, we learned to use Firebase to communicate in real time between our computer and the clients. Specifically, we had to figure out which data was crucial to the success of our program and the best ways to organize the information in a way to quickly be processed.
What's next for Sinkr
In addition to some polish to the machine learning/training and overall ball/cup tracking, we'd like to polish up our app some more. Additionally, we'd like to expand to Android and/or the web.
Other features we'd like to see are facial recognition to automatically attach a face to an account and record your stats to the cloud, and more advanced pong statistics to record.