Voice Checkout

Shop for groceries by talking. Before any money moves, every payment gets a plain-language scam check, and risky ones wait for a trusted family member to approve them.

Comment

Inspiration

Phone scams aimed at older people tend to end the same way: someone is talked into paying fast, often with gift cards, crypto or a wire "to keep your account safe". Voice is also the easiest way for many older people to shop. I wanted a checkout that is easy to use by talking, and that slows down at exactly the moment a scammer needs things to go fast.

What it does

  • Shop by talking. The shopper says things like "add a dozen eggs" or "what's in my order?". A Claude agent searches the catalog, fixes misheard items and reads the order back before anything is paid.
  • Pay with a PIN, not a voice. When she says yes, a big on-screen PIN pad opens with a "Pay $X" button. The PIN goes straight to the server and never passes through the model. Five wrong PINs pause paying for 15 minutes.
  • A scam check on every payment. It looks for gift cards, crypto, wire transfers, scam-call words in the conversation, repeat orders, and totals over 3 times her usual shop.
  • Low risk is paid, risky is held. Low-risk orders are charged to her saved PayPal account. Anything else waits. She hears one calm sentence on why, and who to call. The family gets a phone alert.
  • A family view. Every order is listed in a grid with its risk level and scam signals. Held orders sit at the top with Approve and pay and Reject, and paid orders can be refunded from the same page. The shopper's page says the family's answer out loud when it comes.

How I built it

  • Claude (Anthropic API) runs the shopping conversation through four tools: search_catalog, update_cart, read_back_order and start_payment. start_payment only opens the PIN pad. The model never sees the PIN and never moves money. Claude also writes the one plain sentence the shopper hears when an order is held.
  • PayPal REST APIs (sandbox): the family connects the shopper's PayPal account once with Vault v3 setup and payment tokens. Orders v2 then charges that saved account with no buyer present. Refunds use Payments v2, and webhooks are checked with PayPal's signature verification.
  • The scam check (scamcheck.py) is plain rules with a test for each signal, so the decision about money is predictable and testable.
  • Frontend: React and Vite, with the browser's Web Speech API for listening and speaking. The family view uses AG Grid Community for sorting, column filters, search and a red-flags-only switch.
  • Backend: FastAPI and SQLite.
  • Render runs both from one Blueprint: the API and a static site that forwards /api/* to it.
  • ntfy sends the family a phone alert when an order is held or the PIN is locked.

Designed for older adults

The shopper's page follows research on voice interfaces and design for older adults: 24px base text that still follows the browser's text-size setting, 7:1 contrast for every text pair, the Atkinson Hyperlegible Next typeface, large buttons, short tones when the mic opens and closes, a live "You said" line, a slower reading speed, a dark theme and a screen-reader mode. An axe-core check found no WCAG 2.2 AA violations on either page.

Challenges

  • The model was too protective. Asked for a gift card right after a "bank" call, Claude refused on its own. That sounds safe, but nothing was held and the family never heard about it. The fix was to tell it never to refuse a catalog item: warn gently and let the scam check decide.
  • Keeping the PIN away from the model. A spoken PIN can be overheard and ends up in transcripts, so paying moved to an on-screen pad. If she says digits anyway, they show as •••• and the agent asks her to type them instead.
  • Charging with no buyer present. The family approves once in PayPal, then later payments use the saved token. Render's free disk is wiped on redeploy, so the vault ID can also be set as an environment variable.
  • Alerts. Render's free tier blocks outgoing mail, so family alerts go through ntfy push notifications instead of email.

What I learned

Let the model handle the conversation and the explaining, and let plain code decide about money. PayPal's vault plus Orders v2 made "pay later, with no buyer present" much simpler than I expected.

What's next

Testing with older adults themselves, per-family rules such as "always ask me above $50", and more languages.

Built With

Share this project:

Updates

Submission history