StoreYog: Talk to Your Store

An Alexa+ voice assistant for small stores, built on a real self-hosted MCP server. Ask about stock, approve restocks by voice, help shoppers find items, and verify every answer.

Comment

Inspiration

Raj runs a small grocery on his own. He is the cashier, the stock clerk and the cleaner. At closing time there is still one more job: counting what sold and deciding what to reorder. He shouldn't need a clipboard or a dashboard for that. He should be able to just ask, while his hands are busy. Raj is fictional, but that evening is real for a lot of small store owners.

What it does

StoreYog gives one small store a single, shared picture of its shelves, with three ways in:

  • Owner, by voice. In our web voice simulator, the owner asks "Where is the jaggery?", "How's my store doing?" or "Draft a restock for only rice". Answers come from the store's real records, with the shelf photo, the demo video and short audio tips on screen. Nothing is ordered until he says yes, and orders are simulated. A Mode selector lets anyone compare three tool-calling approaches on the same store.
  • On a real Echo Show. The same store also runs as a classic Alexa custom skill ("Alexa, open store yog"): shelf photos with an arrow, audio guides, restock approval and the demo video, every turn a real MCP call.
  • Store app. Inventory, point-of-sale checkout, receiving, orders and shelf photos. All of it uses the same records the voice assistant reads.
  • Customers. A read-only shelf finder: "Do you have toor dal?" returns the aisle, the shelf and a photo with an arrow. It never shows counts, costs or sales.

Every answer can be checked. A password-protected audit shows each MCP tool call, with its request ID, result and the exact SQL it ran, plus the exact prompt and response of every LLM call and the mode that made it.

How we built it

  • MCP server: a self-hosted MCP server (spec 2025-11-25, Streamable HTTP) built with the official TypeScript SDK. Its tools include get_inventory, get_todays_transactions, reconcile_inventory, update_inventory, place_reorder and report tools. A separate customer server exposes only find_product.
  • Identity: every credential is bound to one store with scopes; a model can never pick the tenant. Public /mcp uses OAuth 2.1 with PKCE, exact redirect URIs, resource-bound tokens and rotating refresh tokens.
  • Data: a Node.js/Express backend over SQLite: a daily inventory ledger, idempotent checkout, stale-count protection and a fresh explicit approval for every order.
  • Voice simulator: a browser front end with hands-free duplex voice. It detects your pause, lets you interrupt, and treats any sentence containing "stop" as "turn off the microphone". Speech goes through Whisper and Kokoro, and language through Gemma 4 E2B. A short spoken phrase or soft background music fills the wait while the store is checked.
  • Our own small model: OPT-350M, fine-tuned on an NVIDIA DGX Spark on 19,408 conversations that we generated by running the real MCP server. It is published for non-commercial research on Hugging Face.
  • Safety: a responsible-AI prompt guard checks every request and every model-written reply. The server enforces what a prompt cannot: store identity, writes only when the owner asked, and approval only from the owner's fresh "yes".
  • Judge setup: the public repo runs with python3 setup.py. With no API keys it executes real MCP tools and SQLite; three base URLs (LLM, STT, TTS) turn on live voice.

Challenges we ran into

  • No access to the official Alexa+ tools. The Alexa+ preview tools (MCP Toolkit, AI CLI, official simulator) are limited to select partners. So we built our own web voice simulator, kept the MCP server standards-based for a later native connection, and tested on a real Echo Show through a classic custom skill.
  • Getting a small model to call MCP tools reliably in the simulator. This was our biggest challenge, and we turned it into a study. We first gave Gemma 4 E2B every tool schema and let it plan the calls. It skipped tools, mangled arguments, and worse, told the owner it had drafted restocks and updated stock when it never had. We then built two alternatives and ran all three side by side on the same live store, behind the same guard:
    • An intelligent router. Gemma only picks one of eleven numbered business actions from in-context examples and names the product. Code runs the MCP calls and builds the reply from real results.
    • Our own fine-tuned 350M model. It learned this store's tools from synthetic conversations generated by the real MCP server. Its first version memorised the demo store's numbers; randomising stock, prices and shelf locations in the training data fixed that (correct first call on unseen products rose from 30% to 54%).
    • The result. On a 30-turn benchmark run three times on the live site, the router was right 96.7% of the time, the small model 90.0% (and fastest, 0.74 s per turn), and native tool calling 48.9%. We wrote this up as a paper with all prompts, scored runs and audited calls: What Our Experience Says About MCP Tool Calling for Small Businesses: An Intelligent Router with a Small Model Is All You Need.
  • Real speech in a real store. Testing by voice showed Whisper hearing "dal" as "doll" and "dull", "toor dal" as "Turdal" and "atta" as "otta". It also showed a reaction like "Oh great." being treated as a stock change. We added a mishearing fix in front of every mode, and a server rule that refuses any write the owner did not clearly ask for.
  • A safety guard that does not block the owner. Our first guard prompt, based on our earlier moderation paper, blocked the owner's own "send me my inventory report". Telling it that the user is the signed-in owner kept moderation at F1 99.67 and stopped the false blocks.
  • Natural hands-free voice in a browser: pause detection, barge-in while the assistant is speaking, and a reliable "stop".
  • OAuth discovery through a reverse proxy. A blanket /.well-known/ exclusion for certificates hid the metadata endpoints until it was narrowed.

Accomplishments that we're proud of

  • Every number the assistant speaks can be traced to one MCP call and the SQL behind it.
  • A measured answer to "which way of tool calling works for a small business", published as a paper with its data, and our own small model published on Hugging Face.
  • One store, three experiences, with strict separation. Customers stay read-only, and saying "I'm the owner" is not authentication.
  • Judges can clone the repo and run the real MCP demo with one command and no keys.

What we learned

MCP is a good boundary between a conversation and a business. The model decides what to ask for, and the server decides what is allowed and what is true. For a small business, the less the model has to decide, the better: one classification by a small model, with grounding, writes and approvals in code, beat native tool calling by a wide margin.

What's next for StoreYog

Native Alexa+ account linking and device testing once preview access is available; giving the router conversation context and training the small model on many kinds of stores; MCP Apps visuals for Echo Show; real supplier ordering behind the same approval step; onboarding for more small retailers.

Built With

Share this project:

Updates

Submission history