How many times have you seen somebody's new watch or that classy hat and thought: "Hey, that's so cool, I should definitely buy one for myself!" or "Oh, the queue at the groceries store is so long, I wish I could just take a picture of the product and purchase it immediately!" Now, with the Spectify App, it's possible! Spectify is a camera-based shopping assistant that allows you to purchase your desired product by just taking a picture of it - easy and fast!

What it does

Spectify takes a picture of your desired product and analyzes it with the help of Google Cloud Vision API. The recognized object is looked up on eBay and items similar to your desired product are displayed on your smartphone, giving you the possibility to purchase or to bid on the product conveniently.

How we built it

We used the Ionic Framework using AngularJS to build our mobile application. The Ionic Framework enables us to deploy Spectify for both Android smartphones and iPhones. We used the Google Cloud Vision API to identify the desired product. We integrated our app with a Node.js web app to make a RESTful call (findItemsByProduct) to the eBay Finding API.

Challenges we ran into

Due to a bug in rendering from Javascript to Objective-C in Xcode, we had to switch the mobile framework from React Native to another rendering framework; AngularJS/Ionic. Another hassle, was using Amazon Product API. We consistently had issues with calling the API and the API Keys changing. We had a lot of hope in getting it working, but it did not work out in the end, which was disappointing because we spent 50% of our time working on this problem.

Accomplishments that we're proud of

Successfully dealing with Google Cloud Vision API was a very fruitful experience. Even though we did not get the Amazon Product API working, we came very far.

What we learned

We leaned how to interact with the Google Cloud Vision API and the eBay Finding API.

What's next for Spectify

Currently, we have the app deployed on Android. We would like to deploy it on iPhones as well (minor changes required thanks to Ionic Framework). Furthermore, we would like to solve the issues with the Amazon Web Services API and scaling the app up by involving other providers. User authentication should also be added. Finally, we would like to use the Microsoft Azure Image Moderation API to filter out inappropriate images captured by the camera.

Share this project: