Once I was looking at someone's video game footage on a tiktok video and wanted to know what game it was. The post didn't mention the title nor did the comments. So we used machine learning to solve this problem.
What it does
The user takes an image with their camera of a video game and the machine learning algorithm processes it to try and guess which game it is. The user is then brought to a page with details about the video game obtained from the IGDB database. From there, you can choose to buy the video game and are brought to a checkout screen.
How we built it
After brainstorming the idea, we decided to plan out the design for a few hours. Once we were happy with the design, we began researching how to implement machine learning and object detection into iOS through the camera. When the machine learning algorithm was designed, we began training it with several iterations of images of popular video games and tested it. After hours of testing, we were satisfied with how precise the ML evaluation became, and began implementing it into our iOS application.
Challenges we ran into
- iOS XCode environment does not allow for more than 1 user to work on the same part of a project effectively
- Gathering data to train machine learning algorithm requires a lot of human oversight
- UI/UX is very meticulous to implement and make it look right
Accomplishments that we're proud of
- Implemented machine learning object detection to detect video games from an image
- Learning about and implementing some of the IGDB video game database
What we learned
- Machine learning requires a diverse set of _ training _ images to accurately recall test objects
- UI/UX requires plenty of planning to create a professional/good looking interface
What's next for Game Sense
- Create a better machine learning algorithm to detect games better
- Add sound detection for in-game sound effects or music to help the image object-detection guess the video game
- Integrate marketplaces so users can purchase the games they find from respective merchants
- Add more games and integrate a video game database to display more information about titles