Inspiration

Online shopping still feels like work: search, filter, compare, scroll, fill in forms. In a shop, you just tell someone what you need. We wanted that experience online: describe what you want in plain words, and get it in your cart. Payments had to stay as trustworthy as ever, so we paired a conversational AI with PayPal checkout.

What it does

Ask & Buy is an AI shopping assistant you can chat with.

  • Ask things like "I need a gift for my dad under $50, he likes coffee" and it recommends matching products with images and prices.
  • It adds and removes items in your cart for you, and shows the real subtotal, shipping and tax.
  • You pay securely with PayPal (sandbox), and the order is confirmed by PayPal webhooks.
  • It remembers your orders: ask "what did I buy last time?" or "order the same again".
  • Accounts keep your order history across devices, and an admin dashboard shows all orders.
  • If the AI is unavailable, a built-in rule-based assistant takes over, so the store never stops working.

How we built it

  • Backend: Node.js and Express 5, with SQLite (better-sqlite3) storing orders, carts, chats and accounts.
  • AI: Google Gemini (gemini-3.5-flash-lite) with function calling. The model uses 6 tools: search products, add to cart, remove from cart, view cart, order history and reorder. Replies stream live to the browser (Server-Sent Events).
  • Payments: PayPal Orders v2 API and the PayPal JS SDK. Orders are created and captured on the server, and webhooks are checked with PayPal's signature verification API.
  • Pricing: all amounts are integer cents, calculated only on the server:

$$\text{total} = \text{subtotal} + \text{shipping} + \text{tax}, \qquad \text{shipping} = \begin{cases} \$5.99 & \text{subtotal} < \$50 \ \$0 & \text{otherwise} \end{cases}$$

  • Frontend: vanilla JavaScript modules, accessible (keyboard navigation, focus-trapped dialogs, screen-reader labels) and mobile-friendly.
  • Security: scrypt password hashing, HttpOnly session cookies, CSRF protection, a strict Content Security Policy and per-IP rate limiting.
  • Quality: 121 automated tests, 9 browser tests (Playwright, desktop and mobile), a 15-scenario AI quality evaluation, GitHub Actions CI, and a live deployment on Render.

Challenges we ran into

  • The AI silently stopped working. Gemini 3 models reject the old way of turning off "thinking" (thinkingBudget: 0), so every chat quietly fell back to the simple assistant. We traced it, switched to thinkingLevel, and added a startup self-check so this kind of failure is reported loudly.
  • Keeping AI conversations valid: each tool call must be paired with its result. Trimming long histories, two messages arriving at once, or too many tool rounds could break that pairing, so we handle all three.
  • Free API quota limits: we added a cooldown so that when Gemini is rate-limited, shoppers get an instant fallback answer instead of waiting.
  • Payment race conditions: the payment capture and PayPal's webhook can arrive at the same time. We made settlement happen exactly once, so paid items are never removed from the cart twice.
  • Local webhook testing: receiving PayPal webhooks on a laptop meant setting up tunnels and debugging signature verification.

Accomplishments that we're proud of

  • The AI never handles money. It can't invent prices or totals; it only quotes what the server calculates, and only PayPal can take a payment.
  • Payments are reliable: webhooks are verified, duplicates are ignored, and a failed webhook is retried safely.
  • It keeps working when the AI doesn't, thanks to the fallback assistant.
  • It is live, tested and deployed, not just a demo on a laptop.

What we learned

  • Treat an LLM like any other untrusted input: give it narrow tools and keep the important rules (prices, payments, ownership) on the server.
  • Payment systems need idempotency and protection against race conditions from day one.
  • AI features need their own tests: we built an evaluation that checks the assistant's answers, not just the code.
  • Money should always be integer cents, never floating point.

What's next for Ask & Buy — AI Checkout Assistant

  • Live PayPal payments, Venmo and Pay Later options
  • A real product catalog with stock levels, shipping addresses and email receipts
  • Persistent hosting so data survives restarts
  • Personalized recommendations from past purchases
  • Multiple languages and voice shopping

Built With

Share this project:

Updates

Submission history