- Minutes after meeting each other, we were delighted to discover our shared love for Soylent and Pokemon Go. One suggestion led to the other, and here we are.
What it does
- It identifies Soylent bottles from a live camera feed and rewards the user with points. It allows a user to verify themselves through Snapchat OAuth and complete with other Snapchat users near them.
How we built it
- SnapKit for user authentication
- Unity and Vuforia for simulating AR game environment.
- BitmojiKit to highlight faces and Soylent bottles in the live camera feed.
- Google Cloud Vision's Product Recognition engine for identifying Soylent bottles.
- Google Cloud Dataflow for streaming data on points and location in real time.
- NodeJS/Express to create REST service that ranks all users within a certain radius.
Challenges we ran into
- We considered several techniques for detecting Soylent bottles in the live feed. A custom CNN built with Tensorflow worked accurately on individual frames but was too slow. OpenCV code was hard to debug because of poor documentation. Vuforia seemed like a good option but we struggled to create a sufficiently accurate 3D model of a Soylent bottle and we were not sure how we'd deploy the solution.
- Almost exceeded our Google Cloud credit limit because one of us forgot to stop an unused instance.
Accomplishments that we're proud of
- We were delighted when we were able to get the bottle recognition step working seamlessly.
- After we finished, we were actually excited to compete against each other on our own game.
What we learned
- Working with Google Cloud Platform and Snapkit was a new experience for us.
What's next for Soylent Go
- Generalize to more products. Come up with more creative rules.