alab-mart

AI-powered e-commerce with RAG product assistance and real-time voice shopping.

Comment

Inspiration

I built alab-mart to explore how AI can make a traditional e-commerce experience more natural and interactive. Instead of limiting users to clicking through products and manually managing their cart, I wanted to create a shopping experience where users could ask questions about products, get answers from their store knowledge base, and control their cart using natural language and voice commands.

The project was inspired by the growing use of AI assistants and the idea of bringing those capabilities directly into an e-commerce platform.

What I Learned

While building alab-mart, I gained practical experience in several areas of AI application development:

  • Building a Retrieval-Augmented Generation (RAG) pipeline for product and store information.
  • Using Chroma as a vector store for semantic retrieval.
  • Connecting a local LLM through Ollama to generate answers from retrieved knowledge.
  • Building APIs and backend services with FastAPI.
  • Integrating Whisper.cpp for local speech-to-text processing.
  • Using gTTS for text-to-speech workflows.
  • Connecting AI capabilities with real e-commerce functionality such as product search and cart management.
  • Designing a system where AI responses are grounded in the application's indexed knowledge rather than relying only on the language model's general knowledge.

How I Built It

alab-mart combines a browser-based storefront with an AI-powered backend.

The FastAPI backend handles the product catalog, authentication, cart operations, orders, and assistant logic. SQLite is used for local persistence.

For the RAG product assistant, I created a knowledge base containing product and store information, generated embeddings, and stored them in a Chroma-backed vector store. When a user asks a question, the system retrieves relevant information and provides it to the local Qwen 2.5 3B model running through Ollama to generate a grounded response.

The project also includes a voice shopping assistant. Users can type or speak commands such as adding a product, removing an item, changing quantities, or checking out. When voice input is enabled, Whisper.cpp performs local speech-to-text processing before the assistant interprets the command and updates the shopping cart.

This creates two complementary AI experiences:

RAG Assistant → Ask questions about products and store policies.

Voice Assistant → Control the shopping experience using natural language.

Challenges I Faced

One of the biggest challenges was integrating multiple technologies into a single application while keeping the different components working together reliably.

Building the RAG pipeline required careful handling of document processing, embeddings, vector storage, retrieval, and local LLM generation. I also needed to make sure the assistant could gracefully handle situations where the RAG components were unavailable.

The voice assistant introduced another set of challenges, particularly around local speech recognition, Whisper.cpp configuration, model files, and connecting transcribed speech to real-time cart operations.

Another challenge was designing the system so that AI was not just a chatbot added to an e-commerce website, but actually interacted with the application's underlying functionality.

Outcome

The final result is alab-mart, an AI-focused e-commerce platform that demonstrates how RAG and voice AI can be integrated into a practical shopping workflow.

The project helped me understand that building useful AI applications is not only about choosing an LLM. It also involves retrieval, backend architecture, data handling, speech processing, APIs, user experience, and reliable integration between AI and traditional software systems.

Built With

  • chromadb
  • embeddings
  • fastapi
  • html
  • python
  • rag
  • voiceai
  • whisper.cpp
Share this project:

Updates