Soylent Go

Inspiration

Minutes after meeting each other, we were delighted to discover our shared love for Soylent and Pokemon Go. One suggestion led to the other, and here we are.

It identifies Soylent bottles from a live camera feed and rewards the user with points. It allows a user to verify themselves through Snapchat OAuth and complete with other Snapchat users near them.

SnapKit for user authentication
Unity and Vuforia for simulating AR game environment.
BitmojiKit to highlight faces and Soylent bottles in the live camera feed.
Google Cloud Vision's Product Recognition engine for identifying Soylent bottles.
Google Cloud Dataflow for streaming data on points and location in real time.
NodeJS/Express to create REST service that ranks all users within a certain radius.

We considered several techniques for detecting Soylent bottles in the live feed. A custom CNN built with Tensorflow worked accurately on individual frames but was too slow. OpenCV code was hard to debug because of poor documentation. Vuforia seemed like a good option but we struggled to create a sufficiently accurate 3D model of a Soylent bottle and we were not sure how we'd deploy the solution.
Almost exceeded our Google Cloud credit limit because one of us forgot to stop an unused instance.

We were delighted when we were able to get the bottle recognition step working seamlessly.
After we finished, we were actually excited to compete against each other on our own game.