Smart E-Commerce Search

Inspiration

Searching for a product online is not always easy. A lot of times, we might have a picture of what we want, but not the exact query string to leverage the search engines of many e-commerce websites. So why not use an image based search engine?

What it does

By indexing products across various e-commerce websites, the application can provide both text and image based searching. Furthermore, it also supports image segmentation to segment pieces of clothes, and perform image search on those.

How I built it

1. Built crawlers to get products metadata from Amazon, Ebay, and Etsy.
2. Built a Flask service to serve API endpoints to support text and image based searching. Also serves a middleware to Samasource API for image segmentation.
3. Built a React frontend application that talks to the Flask service.

Challenges I ran into

A lot of time was spent in reading and learning about text and image search algorithms.

Accomplishments that I'm proud of

Building the whole system by myself.

What's next for Smart E-Commerce Search

• Index more products over more e-commerce websites.
• Support more classes in the segmentation part, possibly fine-tuning a pre-trained model with dedicated data.
• Try out machine learning based methods for image similarity, and compare to the current hashing based strategy.
• Use a better and dedicated text search engine.

Built With

• flask
• image-segmentation
• natural-language-processing
• python
• react
• typescript