VShop AI

A voice-first AI e-commerce agent bridging the digital divide for newcomers and seniors, while empowering global crypto holders for true financial inclusion.

Comment

Inspiration

Imagine having your very own voice-activated personal shopper that not only finds the perfect items for you but also lets you buy them instantly—whether you want to pay in USD or ETH.

You speak your shopping wish—like “I need tennis balls” or “Find me a new laptop”—and the AI assistant transcribes your voice, interprets what you’re asking for, and automatically searches with Exa.ai for the best matching products. You get a curated list of recommendations with images, prices, and descriptions.

Then, with a single click, you can buy your chosen item using either traditional payments (USD) or crypto (ETH). The project seamlessly handles the purchase flow, calculates how much ETH you need (pulling real-time price data from an oracle), and even interacts with your web3 wallet (via Wagmi hooks and RainbowKit).

It’s built on Next.js 13 for the modern UI, plus RainbowKit and Wagmi for a clean crypto wallet experience. The AI side uses custom endpoints to parse user intent, fetch product data, and manage the cart logic. You get a smooth, voice-driven, multi-currency shopping experience all in one place.

What it does

VShop AI is a voice-first, AI shopping agent. You talk to it, and it:

  1. Understands your request (like “Buy my mother something for Valentine’s Day”).
  2. Retrieves relevant products using a custom RAG (Retrieval Augmented Generation) flow.
  3. Validates the product details, ensuring correct Amazon Standard Identification Numbers (ASINs—unique IDs for Amazon products).
  4. Displays a curated list of items, letting you buy in USD or crypto (ETH).
  5. Checks out seamlessly, so you never have to fumble with complex online forms.

How we built it

  • Voice & AI: I capture spoken input, then use AI (OpenAI & RAG pipeline w Exa AI) to parse user intent.
  • Audio Responses: I used ElevenLabs to audibly and naturally respond to shopping queries, so users can hear confirmations and item details.
  • ASIN Validation: Once the AI returns potential ASINs, I call a dedicated API endpoint to verify each ASIN is real and retrieve final product details.
  • Decentralized Price Feeds: I integrated an EigenLayer AVS (eoracle.io) to reliably fetch the USD–ETH price in a non-centralized manner, with multiple oracles acting as operators to confirm data integrity.
  • Multi-Currency Checkout: I integrated RainbowKit + Wagmi to let users pay with ETH, while also supporting traditional USD purchases via Rye.
  • UI/UX: Built with Next.js 13 for a modern, voice-first interface, bridging the gap for seniors and crypto adopters alike.
  • Search: I used perplexity as an official search partner. They helped me plan out my agentic architecture by providing me with tradeoffs between different APIs and also allowed me to discover Rye.

Challenges we ran into

  • Hallucinated ASINs: I tried using Perplexity Sonar to find ASINs but found that it sometimes hallucinated, forcing me to build a robust ASIN validation step. I also made several iterations of the system prompt so that the model’s accuracy in identifying real and appropriate ASINs would be as high as possible.
  • Decentralized Price Data: Ensuring I had a non-centralized price feed required hooking into EigenLayer AVS and verifying multiple oracles. The AVS architecture was a little confusing at first, so integrating it was a learning curve.
  • Voice Complexity: Adapting the interface so older or less tech-savvy users can comfortably interact with a voice agent required frequent modifications and adjustments.

Accomplishments that we're proud of

  • Decentralized Price Feeds: Tapping into Eoracle's AVS to get stable, non-centralized ETH price data.
  • Audio Responses: Using ElevenLabs for natural speech, enhancing accessibility for the visually impaired or those uncomfortable reading screens.
  • Multi-Currency: Bridging USD and ETH in a single shopping flow, broadening financial inclusion.

What we learned

  • Prompt Engineering & Validation: Combining RAG with open endpoints requires careful prompt design and external checks.
  • Accessible Voice Design: Realizing how essential straightforward voice interactions are for seniors and people with disabilities.
  • UX for Crypto: A frictionless web3 checkout requires balancing user familiarity with wallet security.

What's next for VShop AI

  • Wider Marketplace Support and MVP Validation: Testing the waters with the local community by integrating with local e-commerce retailers and small businesses back in Berkeley.
  • More Payment Assets: Adding stablecoins or other tokens for broader international usage.
  • Extended Voice Commands: Letting users say, “Compare the second and fourth items” or “Add two of the tennis ball packs to my cart and pay for it in USD.”
  • Deeper Accessibility Testing: Ensuring the product serves the full range of seniors, disabled individuals, and beyond—no one is left behind.

Built With

  • eleven-labs
  • eoracle.io-avs
  • exa.ai
  • gpt-4o
  • json
  • next.js
  • rainbow-kit
  • ryepay
  • typescript
  • wagmi
Share this project:

Updates