-
-
Mission Control dashboard — live claim stats, voice intake presets, and the sub-millisecond verification trace.
-
Verification deck — every claim audited through REPORTED → VERIFYING → DECISION → EXECUTION, with REFUSED stamps on false claims.
Tagline Every agent acts. HANDOFF knows when NOT to. ⛔
Inspiration Every AI agent demo ends with the agent saying yes. We kept asking: who says no? On a night shift at a small store, acting on a wrong voice report is worse than doing nothing. A phantom "the delivery arrived" becomes missing stock on the shelf. A missed low-stock alert becomes an empty shelf at 9 AM. Shift workers already speak their problems out loud — what they lack is an agent that verifies before it acts, and refuses when it can't. So we built the gatekeeper: an agent whose most valuable output is a well-explained "no."
What it does HANDOFF is a voice-driven operations gatekeeper for shift work, built for the Alexa+ track as a simulated web experience (rules-sanctioned path). The demo — Kumar Stores, night shift: Ravi reports by voice: "the delivery never arrived, cups are almost empty." The engine splits this into two independent claims, each verified and audited separately. Cups are at 4% (threshold 10%) → VERIFIED → a replenishment order is prepared. The freezer delivery was scheduled — but PO-103 has no receipt scan → REFUSED: "I will not act on this. A schedule is not evidence." The supervisor approves by voice → the cups order executes → CONFIRMED. At morning handoff, Priya asks "shift start" and inherits only the unresolved freezer claim — nothing else carries forward. Every claim moves through an auditable state machine: report → verify → prepare → approve → execute → confirm. Reported is not verified. Refusal is a terminal state, and every single transition is written to an audit log with the evidence (or the missing evidence) cited by name.
How we built it FastAPI backend: POST /api/voice takes an utterance, splits it into fragments, runs a per-fragment verification trace, and returns a verdict per claim. SQLite (WAL mode) as the system of record: inventory ledger, purchase orders with receipt flags, claims with full state history, thinking traces, audit log. Self-seeding — a fresh clone is demo-ready with no migrations. Streamlit "Mission Control" night-ops console as the simulated Alexa+ experience: chat-style claim intake, demo preset buttons, live verification-trace panel, animated verdict stamps, inventory gauge, morning-handoff queue. FastMCP server (mcp_server.py, 4 tools: report_issue, check_inventory, check_delivery_receipt, get_pending_handoff) so an agent runtime like Alexa+ can drive the same gatekeeper directly. Zero cloud, zero API keys, fully local. Verification traces measured at sub-2ms live.
Challenges we ran into The honest ones, straight from our friction log: Multi-claim utterances silently dropped the second claim — fixed by splitting on commas into separate audited claims. Our trace claimed "acoustic signature extraction" on plain-text input — a judge reading the code would catch the bluff, so we renamed it to honest TEXT_PARSE. We committed a binary handoff.db with stale demo data to git — removed it, git-ignored *.db, and made the app self-seed. A latency reading that looked like a 35x regression turned out to be environment noise (proved with a controlled A/B). The demo video mixed Tanglish and English for US judges — rebuilt fully in clean English with every input verified live against the API.
Accomplishments that we're proud of A refusal engine as a feature, not a failure: unverified claims stop cold, with the missing evidence named. One backend, two clients: the same FastAPI engine drives the web UI and the MCP agent tools unchanged. Sub-2ms verification, measured live — the trace is the demo. A 1:27 demo video built from real product footage, not slides — the product on screen is the product in the repo.
What we learned Claim state is the product. Once every utterance became a row in an auditable state machine, handoffs, approvals, and refusals all fell out of the same model — and "no" became the system's most valuable output.
What's next for HANDOFF Wire the same FastMCP tools to real Alexa+ voice when the device path opens to us, add Tamil/Hindi voice intake for the store floors that actually run on them, and a multi-store rollup so a regional manager sees every refused claim across locations.
Log in or sign up for Devpost to join the conversation.