HeyFind - A Computer Vision Powered Game

Find common things all around you, and show them to your webcam

Comment

Inspiration

I have always found computer vision extremely cool and I really wanted to create something within that field. I also wanted to make something that could be useful and fun at the same time. Being extremely bored during quarantine I decided I wanted to make an app that could make quarantine a lot more fun! This game will be even more fun when multiplayer functionalities are added.

Even though the app is a game, I believe parents could use the app with young children to help them understand common objects they would see in their daily lives.

What it does

HeyFind is an online machine learning-powered virtual game where you find objects that you can find around your house and go on a virtual scavenger hunt! The ML computer vision algorithm is able to detect objects rapidly for immediate feedback.

How I built it

I used the p5.js library to easily get video and also to make a built-in bot that speaks to you as you are playing. I was originally using mobilenet for objects but I found that the tensorflow.js COCO-SSD model was much better and accurate.

Challenges I ran into

One of the biggest challenges I ran into was the models detecting background noise over the actual object. This would result in some really weird things being detected.

I also found designing the website to be pretty difficult as I was a beginner in HTML and CSS. I would spend a long time trying to do the smallest things in CSS and it got frustrating at some points.

Accomplishments that I'm proud of

I am proud that I was able to build the entire project by myself without much Frontend knowledge.

What I learned

I learned a lot about p5.js and tensorflow.js during this hackathon. I feel much more confident about making websites, AI, and websites involving AI.

What's next for HeyFind - Computer Vision Powered Game

In the future, I believe that multiplayer functionalities could really enhance this game on a whole new level! I also plan to add more classes of objects that can be recognized to add more depth to the game!

Built With

Share this project:

Updates