AskTheShelf

Hero
Chat Interface
Link Feature

Inspiration

While browsing Amazon, I frequently find myself sifting through product pages to find important details such as weight, size, and specifications. I would often read customer reviews to gain insights. Amazon provides a brief summary of customer sentiments, but it was never sufficient. So I set out to build an app that would scrape Amazon reviews and generate a comprehensive summary. Once that was working, I aimed for something even bigger.

What it does

AskTheShelf enables users to search for almost anything. Instead of using keywords in a search box, users can leverage natural language to ask for what they want. They can ask follow-up questions to compare, find similar products, or get detailed product information. There's even an option to supply an Amazon link and chat about the product after it's been scraped. 🛒

The link feature is particularly powerful because it scrapes data from multiple pages:

Product Page
Critical Reviews Page
Positive Reviews Page
Q&A Page

In contrast, a standard search in the chat only retrieves data from the product page. Why? 💸.

With this much context, users can engage in detailed and complex conversations about the product.

How we built it

The app comprises four main components:

Frontend: Built with Nuxt.js
API: Developed using FastAPI
Scraping Service: Built with Scrapy and BrightData proxies
Database: MongoDB

The front end maintains a socket connection to the API. When the user submits a query, it hits the Manager Agent, which analyzes the conversation history and the query, and then returns an action. An action could be search, compare, get_product_details, or more_info. We query the MongoDB database based on the manager's action and any additional details provided. If no results are returned, a web scrape of the query on Amazon is initiated. An agent validates the results and initiates a scrape for the product details. An embedding containing important information about the product is created and stored alongside the product in MongoDB. By this point, the user would have received their results and can ask follow-up questions

Challenges we ran into

Initially, the Manager Agent didn't exist. Instead, all agents processed the conversation history and the query, but the LLM often hallucinated product data I hadn't provided. To address this, I created the Manager Agent, which outputs only an action and is the only agent with conversation context. This reduced hallucination, but the conversational aspect of the chat diminished.

My application heavily relies on the LLM returning responses in a specific JSON format, while the LLM mostly returned the right structure, occasionally it returned invalid JSON. To tackle this, I implemented a retry system to self-correct the LLM, which helped but wasn't enough. I then created a custom parsing function to resolve common JSON issues.

Accomplishments that we're proud of

I'm proud to have achieved much more than I initially envisioned. Although the LLM sometimes struggles with simple tasks like price comparisons and occasionally does random gibberish, I'm thrilled that I embraced unfamiliar technologies like Scrapy, FastAPI, BrightData, and MongoDB and got the project working so quickly. Quite a few late nights after work, almost stopped halfway but I am proud to have brought the app to a usable state. 🌟

What we learned

A lot about web scraping 🕷️
Managing LLM agents effectively 🧠
Creating embeddings and searching for similar ones using MongoDB vector search 🔍
Having played around a lot with web scraping for this project I see a huge market for combining it with LLM

What's next for AskTheShelf

Find ways to enhance the conversational aspect without LLM hallucination 💬
Depending on user interest, expand to multiple online stores for price comparisons and enhanced features 🛍️
Make the UI more appealing 🎨