We are imagining a whole new way to find products by building a hierarchical visual map of a catalog based on style and other visual features.

The Problem

When customers have a very specific style or visual feature in mind, they either -

  1. Try to describe it in search bar (Good luck.. long descriptions dont work.. tada..)
  2. Try to browse thousands of products page by page (Too many random visual styles and takes too long.. yawn..)
  3. Use Visual Search (That's smart but not enough as you need to have the item in front of you or have its photo.. ouch..)

The Solution

We created a visual style graph using our own AI capabilities. It intuitively guides users to the style they have in mind.

How Did We Do It?

We built a visual hierarchical map of Chandeliers and Dining Chairs using multiple level clustering. We began by putting together visually similar products into small clusters. We then rolled small clusters up into larger clusters and repeated this process at multiple levels creating a tree like structure (See slides for details). So if users start browsing it from the top they would eventually be able to walk down the 'path' to the right SKU by following the clues from visual features of the product.

App & Python Web Service

We then exported this map into searchable data structures and created a Python web service to navigate the map. We also built a PHP end point that works in tandem with the web service. It features a clean user interface that hides usual product meta data to help focus on product image. See it for yourself!

Video Demo - Chandeliers

Video Demo - Chairs

Clustering Approach

Our approach works at the class level. We start by creating many small clusters within a class. We then repeat the process of merging smaller clusters into larger ones. In this way we organize the catalog into hierarchy of clusters and logically connected like a tree. We then pick representative SKUs for each cluster at each level in tree which acts as a preview of the cluster. Clicking on the preview or representative SKU we would take them deeper into that cluster. A cluster can contain other clusters having preview SKUs of their own. We found that at lower levels products automatically organized themselves into similar styles. At upper levels similar styles ended up in same clusters creating a beautiful style hierarchy. This automatic arrangement is quite useful in many ways. Here we use it to help customers intuitively explore a catalog as they look for a very specific style.

Challenges we ran into

Hierarchical Clustering - We had to try out multiple clustering methods as well as different ways to merge clusters before choosing the right one that worked. There are still many parameters we can fine tune to make it suitable for specific class.

Building browsing experience - We had to smartly arrange the clustering information in different data structures so that we can create a smooth browsing experience which is aware of which cluster user is currently into.

Cluster Preview SKUs - It was difficult to choose single SKU to represent larger cluster as the style varies as the clusters grow larger.

Accomplishments that we're proud of

We found reasonably workable solutions to all the challenges described above by choosing speed vs. accuracy. We were able to show that we can build a style hierarchy within a class using our visual search model.

What we learned

There is no one size fits all solution. We will need to fine tune clustering for each class as the distribution of products differs per class.

Other Use Cases

  • Learn customer taste by tracking how deep they walk down a path in the style graph
  • Add style tags to clusters so that we can correlate them across classes
  • This will help in providing cross class recommendations

What's next for Visual Catalog Map

  • Improve clustering by exploring other clustering methods
  • Find a better way to organize cluster hierarchy within a class
  • Extend the use case beyond browse

Built With

+ 1 more
Share this project: