SpeedMart

Shopping so quick, feels like shoplifting😂

Comment

Inspiration

I needed a energy drink, I went to retail store, I picked up the energy drink in 1 min and waited 10 minutes in line just to checkout. We don't need that anymore, technology now enables to fully automate a store so we can just pick and leave.

What it does

SpeedMart is a tiny grab and go store you can actually walk into.

1) Join once by scanning a code. Your phone's own Face ID, fingerprint, or PIN becomes your key. Your face never leaves your phone.

2) Walk in with a tap on an NFC sticker. The kiosk greets you by name, tells you your budget, and asks what you're in the mood for.

3) Talk to the store. Say "I just finished a run" and the voice agent picks items that fit your budget and diet, and the right bays glow on the shelf map.

4) Grab stuff. An overhead camera recognizes products and puts them in your phone cart in about a second. Put it back and it vanishes. No scanning, no bagging area drama.

5) Walk out and approve at the exit. You see exactly what the payment network sees: a single use agent token capped at your budget. Nothing is charged until you say yes.

6) After the purchase: return an item by putting it back on the shelf (the camera verifies it), or tap "Not mine?" and an AI reviews the shelf clips while a human makes the final call.

How I built it

  • Vision: OpenCV plus a YOLO model I trained on our own products, with a motion freeze so a hand hovering over a bay isn't mistaken for a purchase. Early versions used ArUco tags on every item; YOLO let us peel them off.

  • Backend: FastAPI, SQLite, and WebSockets. The cart is never counted, it's computed (what was on the shelf when you entered minus what's there now), so it can't drift or double count.

  • Trust: WebAuthn passkeys with a PIN fallback, a Visa Intelligent Commerce style authorization object, Stripe test mode charges and refunds, and a demo bank balance that moves like a real account.

  • AI: an ElevenLabs voice agent that can only act through the store's own tools (so it can't invent products or prices), Grok for the store's short suggestions, and a dispute reviewer that can speed up small refunds but can never deny one.

  • Hardware: a LilyGO T Display S3 as the gate terminal, a tablet as the talking kiosk, NFC stickers as doors, and a camera staring at five snacks with great intensity.

  • Process: I wrote a detailed spec first, then orchestrated several Claude Code agents, each owning its own files, with every step checked by acceptance tests (over 400 of them) and physical tests on the real shelf.

Challenges I ran into

  • Hands. Humans reach over things. The camera thought every hover was a heist until I taught it to wait for the bay to settle.

  • Windows expanded a * into our entire file list and fed it to the web server. The server was confused. So were I.

  • A secure cookie that was too secure to work on localhost.

  • Zips with backslashes turned our training data into one very long file name on Linux.

  • I couldn't solder, so the planned shelf LEDs became a glowing on screen shelf map. Honestly, it looks better.

  • A blue mug had to be convinced it was a hydration drink.

Accomplishments that I'm proud of

  • A judge can walk up and shop with zero explanation from us.

  • The shelf cart updates live and fixes itself when you put things back.

  • Trust is visible, not promised: the permissions card spells out what the AI may do and where only the human decides.

  • Returns and disputes backed by camera evidence, which almost no checkout demo bothers with.

What I learned

  • Computing state beats counting events.

  • The best AI features have guardrails you can explain in one sentence.

  • "Human in the loop" isn't a limitation; it's the product.

  • Framing the problem and verifying the output mattered more than generating code fast.

What's next for SpeedMart

  • More than one shopper at a time, which means knowing whose hand is whose, politely.

  • Weight sensors as a second opinion for the camera.

  • Real issuer integration through Visa's agentic commerce APIs.

  • A campus pilot on retail market.

Built With

Share this project:

Updates

Submission history