Find it!

Home Screen
Game Screen
End Screen

Inspiration

The inspiration for this project came from the realization of the potential behind Google’s Cloud Vision API. Not knowing what to do or where to get started for our project, we attended the Google Cloud workshop in hopes of learning a thing or two. It was there we discovered the Cloud Vision API and after leaving the workshop, realized all we could do with it. We brainstormed many different ideas that could incorporate that technology, but it wasn't until we got out of the room for some fresh air that we thought of scavenger hunts. A quick bit of research online let us determine there weren't any real apps that used object detection technology to build a scavenger hunt and provided us with the spark we needed to get started.

What it does

How we built it

Our team consists of both Android and iOS users. We wanted our app to run on all of our phones so we could play together which lead us to go with React Native for the client. We were able to build a single codebase and instantly run the app on both mobile platforms.

For the backend, we sided with Node.js paired with socket.io for real-time communications. None of us have worked with cameras before so we had to research how to quickly send our images to the backend. We ended up base64 encoding each picture and sending it via socket.io to our backend server. We then sent the image to the Google Cloud Vision API to retrieve object labeling. This allowed us to loop through each object label and check if it matched up with the user’s currently assigned item.

Challenges we ran into

Working with the device camera was a struggle. We found it challenging to find the right level of quality our images should be to allow accurate machine learning analyzing while keeping image upload times to a minimum.

Gamifying the app was also fairly difficult. We wanted the app to be genuinely fun to play. After long sessions of play testing and asking others for feedback, we think we ended up with something unique and enjoyable.

Accomplishments that we're proud of

We are proud that we were able to build a fully working project within the timeframe. We came into the hackathon with absolutely no clue on what we would build and spent the majority of the first day trying to come up with ideas. During the production phase, we were satisfied with our progress with the pairing of the front end and back end. We did not run into as many bugs or issues as we thought we were going to be. Overall, through collaborative effort, we all learned new skills and are proud of our working demo!

What we learned

We learned a lot about React Native. Being able to run the app on both iOS and Android was huge for us. We also got to see just how easy it is to incorporate several of Google Cloud api’s through the Node.js and Cloud SDK.

From a non-coding perspective, I think we improved on being able to take test user’s feedback and incorporate it into our app.

What's next for Find it!

If we were to continue the app, we could incorporate augmented reality to introduce additional gameplay mechanics. One idea was to only allow an object to be scannable by one player only; so we would render a red box around items already claimed.

We could also introduce sponsored scavenger hunts where name brand products and locations are items in the hunt. For example, finding a Monster Energy Drink or a Starbucks.