Theme & Topics:

Generative and multimodal AI for E-commerce stores. 🛍️

Objective/Goal 🔍

Reinvent the purpose of chatbots, moving them away from costumer support and on to improving the product search experience.

Inspiration 🌟

My first encounter with visual multimodal AI was with CLIP, a model developed by OpenAI. Despite not getting as much recognition as the company's other works, I found the concept to be extremely useful when implemented with a clear use case in mind.

Role of generative AI and mobile usage 📱

By far the biggest challenge of implementing generative AI in mobile phones is the cost of running instances in real time. This project works around this problem by creating an extensive one-time database using generative AI for the given products, and then making each upcoming instance significantly reduced computationally.

Features 🤖

Sighted chatbots offer a unique experience to users, by allowing them to submit descriptions or images of what they would want to buy, as well as provide a summarizing 'menu' of the products offered. The 3 modes proposed all serve the purpose of improving the user experience and boosting the online stores' sales.

How it works 🛠

This project was developed with Flask. The project utilizes Python, JavaScript, HTML, and CSS. The data comes from an online CSV dataset of a Shopify store. However, there's an extensive preprocessing step where product images are processed through the VQA model, BLIP2, to automatically determine attributes like color, material, design, and more, extending the database greatly.

Achievements 🏆

Regardless of the competition, I'm confident that these chatbots can be integrated into numerous online shops. Discussions are already underway to introduce it to a major apparel brand with an annual revenue of $6.5 million. Moreover, a marketing firm with a combined following of over 1.7 million on social media has reached out to promote the idea. It is clear that despite its very recent creation people find value behind this product.

NOTE:

For security reasons the chatbot will not have the replicate API registered, hence the visual search demo is currently unfeasible. Feel free to contact me privately for the API. Time has been by far the biggest challenge of this hackathon so I could not find a better workaround to the hosting and responsiveness within the time limit.

Built With

Share this project:

Updates

Submission history