AgentPay: Autonomous AI Shopping & Checkout Agent

An autonomous AI shopping assistant with real-time product discovery, voice interaction, spec comparisons, and server-verified checkout powered by PayPal Orders v2.

Comment

Inspiration

Online shopping has become overwhelming. When someone searches for a laptop or smartphone, they are bombarded by sponsored listings, confusing spec sheets, and dozens of browser tabs. While standard AI chatbots can give advice, they stop right where the shopper needs them most—they cannot check real-time stock or prices, and they cannot execute payments safely.

We built AgentPay to bridge this gap: an autonomous, conversational shopping agent that understands natural language constraints (e.g. "laptop for programming under $700 with at least 16GB RAM"), searches real-time web products across retailers, and completes a verified, secure checkout using the official PayPal Orders v2 API.

What it does

AgentPay is an end-to-end AI e-commerce agent featuring:

  • Conversational & Voice Discovery: Talk or type naturally using speech-to-text (microphone) and text-to-speech (audio playback).
  • Infinite Live Product Search: Integrates real-time Google Shopping data via Serper to discover any buyable product on the web with live merchant names, prices, and ratings.
  • Deterministic AI Constraints: Strictly enforces budget ceilings, minimum hardware specifications (e.g., RAM/CPU), and negative filtering (e.g. automatically filtering out phone cases when a phone is requested).
  • In-Depth Hardware Breakdowns: Provides granular architectural analysis (processor, RAM, storage, battery life, pros & cons) for any item on demand.
  • Side-by-Side Spec Comparison: Compares multiple products side-by-side to assist in rapid decision-making.
  • Server-Verified PayPal Checkout: Bypasses client-side price tampering by computing verified prices, taxes, and order proposals on the backend before authorizing and capturing transactions via PayPal Smart Buttons.
  • Verified Transaction Receipts: Generates verified post-capture receipts with authentic PayPal Order IDs and Capture IDs.

How we built it

  • Backend: Node.js & Express powering a modular service-oriented architecture:
    • paypalService.js: Handles OAuth2 Bearer token lifecycle, Orders v2 creation (intent: CAPTURE), idempotent payment capture, and currency/amount integrity checks.
    • catalogService.js: Multi-tiered discovery pipeline (5-minute TTL cache ➔ Live Serper Google Shopping ➔ Grounded local fallback) with server-side tax and order breakdown calculation.
    • aiService.js: Intent extraction engine and tool registry (search_products, get_product_details, compare_products, prepare_order_summary).
  • Frontend: React 18, Vite, Tailwind CSS, Lucide icons, and the official @paypal/react-paypal-js SDK.
  • Voice Engine: Native browser Web Speech API for low-latency hands-free audio interaction.
  • Testing: Automated test suite (12/12 passing) covering AI intent extraction, catalog constraints, and PayPal Sandbox OAuth & Orders v2 workflows.

Challenges we ran into

  1. Preventing AI Price Hallucination & Tampering: An LLM or client should never dictate monetary values. We resolved this by requiring the backend to independently calculate all line items, subtotals, and taxes before generating the PayPal order payload.
  2. Accessory Noise in Live Web Search: Searching for devices like "iPhone 15" on real-time shopping feeds often returns phone cases and screen protectors. We designed a regex-based negative filter (isAccidentalAccessory) to eliminate accessories unless explicitly requested.
  3. Idempotent Payment Captures: Ensuring that double-clicks or repeated network requests never trigger duplicate PayPal captures by tracking internal order states and verifying capture units against expected values.

Accomplishments that we're proud of

  • 100% Verified Sandbox Integration: Successfully creating, approving, and capturing real PayPal Orders v2 transactions with genuine PayPal Order IDs and Capture IDs.
  • Robust 12/12 Test Suite: Full automated test coverage across our AI tool pipeline, constraint filters, and live PayPal Sandbox authentication.
  • Seamless User Experience: Combining voice recognition, rich product specification cards, and interactive PayPal checkout without clutter or developer debug traces.

What we learned

  • The Golden Rule of Agentic Commerce: AI is great for recommendation and intent extraction, but financial transactions must remain strictly deterministic and server-verified. Never let an AI model invent numbers in a payment payload.
  • Working with the PayPal Orders v2 API taught us the importance of explicit breakdown matching (item_total, tax_total, shipping) and idempotent capture handling.

What's next for AgentPay

  • Multi-Currency Auto-Conversion: Dynamic currency detection using PayPal's supported global currencies based on buyer locale.
  • PayPal Subscription & Recurring Billing: Enabling automated recurring orders for consumables (e.g. coffee beans, printer ink) using PayPal Subscriptions.
  • Multi-Merchant Cart Aggregation: Consolidating products across different retailers into a single unified PayPal checkout.

Built With

Share this project:

Updates

Submission history