Vision Shopping is an e-commerce website for users to search Amazon and Walmart products by simply taking photos of objects they want. We realized that people from different countries have different cultures and speak different languages, so we build Vision Shopping to help users buy products they want even if they don't know how to describe the products in their languages.

What Vision Shopping does?

**Vision Shopping **helps users to search the right products by simply taking photos of the object they are interested in. Users don't have to necessarily type in any keywords in order to search the products they desire to purchase. In general, Vision Shopping is providing a new shopping experience to our users.

How we built it

  • We developed the front-end of Vision Shopping using Angular.js to organize the JavaScript code.
  • The photos uploaded by users will be sent to AWS S3 and return the image URL as String.
  • We used IBM Watson Visual recognition to recognize the photos from users using the URL we obtained from the AWS S3. From the Visual recognition, we obtain a list of keywords/labels related to the photos.
  • Using these keywords/labels, the searching engine will search their meanings and images from our MongoDB via LoopBack on IBM Bluemix, and the searching engine will also simultaneously obtain a list of related products from Walmart and Amazon open API.
  • Finally, the app will display a list of products related to the photos uploaded by the user.

Challenges we ran into

Using APIs and and open sources from different companies is hard, especially we need to build the application in one day.There were some conflicts when using the AWS S3 files upload API with PHP on IBM Bluemix Cloud Foundry, and one of the issues was continuously receiving a HTTP 500 error during the HTTP request. After we spent almost a hour and a half, we finally found that the issue was caused by the request command, and now the Application is working properly.

Accomplishments that we're proud of

We are proud to say that the application is fully functioning. Users are able to search product by their input photos (Not hard-coded). However, because we didn't have time to train our own classifiers, some results may not be accurate. In addition, the interface of this app is really clean thanks to the hard work of our designer.

What we learned

From this challenge, we learned that LoopBack is a clean and easy-to-use open source project for building the API using Node. It saves us a lot of time during the product development. In addition, the use of Visual Recognition technology got us started on IBM Watson, and we would love to try out more AI features that IBM Watson provides in our future learning path.

What's next for Vision Shopping

Improve the accuracy of the Visual Recognition by creating appropriate classifiers.

Share this project: