Orbit AI: AI-Powered Voice Shopping Agent

Speak. Navigate. Shop. Done.

Inspiration

Current chatbots are boring and unhelpful. They answer questions but don't take action. We wanted to build something different - an AI agent that actually does things for you. Orbit AI integrates directly into e-commerce websites and handles the entire shopping process through voice commands. Instead of a chatbot that just talks, we built an agent that clicks, searches, filters, and shops autonomously.

What it does

Orbit AI is a voice-controlled AI agent for e-commerce websites that automates the entire shopping experience:

Speak what you want to buy
AI searches products automatically
Applies filters based on your preferences
Compares products and prices
Adds items to cart
Guides you through checkout

Works on any e-commerce website. Reduces shopping time by 70%.

How we built it

System Architecture

$$ \text{Frontend (React)} \leftrightarrow \text{Backend (FastAPI)} \rightarrow \text{Google Gemini 1.5 Flash} \rightarrow \text{ChromaDB} $$

Frontend

React + TypeScript
Web Speech API for voice
DOM scanner finds products and buttons
Executes AI decisions automatically

Backend

FastAPI server
Google Gemini 1.5 Flash for decisions
ChromaDB stores shopping patterns
RAG system for pattern retrieval

Challenges we ran into

AI Selecting Wrong Products Enhanced DOM context with product names, prices, and descriptions for accurate selection.

Managing Multi-Step Shopping Built action history system to track search → filter → select → checkout workflows.

Voice Recognition Errors Added confidence scoring - asks for clarification when unsure.

Cost Management Switched to Gemini 1.5 Flash (50% cheaper), implemented caching, optimized prompts.

Accomplishments that we're proud of

Built complete voice-to-checkout automation
95% intent recognition accuracy
70% reduction in shopping time
AI learns from successful patterns
Works universally on any e-commerce site

What we learned

AI prompt engineering for e-commerce tasks
Vector databases for semantic pattern search
Voice interface design for shopping
Full-stack integration with AI agents
Managing API costs at scale

What's next for Orbit AI

Short-term

Multi-language support
Image-based product search
Mobile apps

Long-term

White-label solution for retailers ($499-$2,999/month)
API for developers
Browser extensions
Integration with major e-commerce platforms

Business Model

Freemium: Free basic, $9.99/month premium
B2B white-label licensing
Affiliate commissions

Team Bandana Boys: Jaskirat Singh & Gurpreet Singh

Tech Stack: React, TypeScript, Vite, Tailwind CSS, Web Speech API, Python, FastAPI, Google Gemini 1.5 Flash, ChromaDB, Sentence Transformers, shadcn/ui

Built With

chromadb
fastapi
gemini
lucide
numpy
pydantic
python
react
sentence-transformers
shadcn/ui
tailwind-css
typescript
uvicorn
vite
web-speech-api

Submitted to

HackUTA 7: Enter the Hacker-Verse

Created by

I focused entirely on the AI intelligence and agent system. I integrated Google Gemini 1.5 Flash as the decision-making brain, engineered prompts to achieve 95% intent recognition accuracy, and built the ChromaDB vector database with Sentence Transformers for semantic pattern storage and retrieval. I implemented the RAG (Retrieval-Augmented Generation) system that allows the AI to learn from successful shopping patterns and improve over time. I also developed the action history tracking system that manages complex multi-step workflows from search → filter → product selection → checkout. A major challenge was optimizing AI costs while maintaining high performance, which led me to implement caching strategies and refine prompts. This was my first time building an autonomous AI agent system with memory and learning capabilities, and I learned a lot about prompt engineering, vector databases, and production AI optimization.

Gurpreet Singh
I handled the complete full-stack development for Orbit AI. On the frontend, I built the React + TypeScript application with Vite and Tailwind CSS, implementing the Web Speech API for voice recognition and designing the UI with shadcn/ui components. I developed the DOM scanner that intelligently identifies products, prices, buttons, and interactive elements across different e-commerce websites. On the backend, I built the FastAPI server architecture, set up the API endpoints, and integrated the communication layer between the frontend voice interface and the AI decision engine. I also implemented the action execution system that translates AI commands into actual website interactions. This was challenging because I had to ensure seamless coordination between voice input, backend processing, and real-time DOM manipulation across various website structures, but I learned a lot about building full-stack AI-integrated applications.

Jaskirat Singh

Updates

Gurpreet Singh started this project — Oct 04, 2025 11:35 PM EDT

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.