Secret Grounds Shift Assistant, Amazon Developer Hackathon

A voice assistant for gaming café staff. Ask about sales, PCs and members, or message PCs and add time hands-free, built on a real café system with MCP and human-confirmed actions.

Comment

Inspiration

I built Secret Grounds, the system that runs a PC gaming café: it starts and times every PC session, locks screens when time runs out, tracks member balances and reports sales. It was field-tested at the shop.

The person on duty there is never just on duty. The owner runs the shop at night while working as a graphic designer and building content for Ran Online, on their own PC right next to the cashier PC. Whoever is on duty is usually mid-match. Every customer question means turning to the cashier PC and clicking through the dashboard. My teammate Er suggested giving it a voice, so whoever is on duty could just ask. I built it; Er also produced the demo video.

What it does

Shift Assistant is a voice assistant for café staff. Hold the mic button (or the Spacebar) and talk:

  • Ask: "Which PCs are free?", "Which sessions are running out soon?", "Who's on PC 7?", "How did we do today?", "How much time does Rogene have left?"
  • Act, always with confirmation: message one PC or every PC ("tell everyone we close in fifteen minutes"), add time to a running session (always billed), and shut down idle PCs at closing (Admin only).
  • Hand over: "Tell the next shift PC 7's mouse is broken." The next person on duty hears it when they start.
  • Check the drawer: "Check my drawer." It asks for the counted cash first, then compares it with the starting float plus the cash taken while on duty, and flags any difference.

Every action shows a confirm card first. Nothing happens until a person says "confirm" or presses the button.

How we built it

  • MCP server (Python, MCP spec 2025-11-25, Streamable HTTP): read-only tools for sales, sessions, PCs, members and who's on duty, plus "propose" tools that check and prepare an action without running it. This is the piece that would plug into Alexa+.
  • Agent (Python, Strands Agents SDK, Claude Haiku 4.5 on Amazon Bedrock): connects the model to the MCP server, keeps conversation memory and shift handover notes.
  • Voice page (Next.js, inside the existing dashboard): push-to-talk speech recognition, spoken replies, and confirm and result cards. A simulated Alexa+ experience.
  • Existing backend (FastAPI + SQLite), unchanged in its logic: the assistant goes through the same API and permission checks as the dashboard.

Safety by design

  • The AI can only propose. The server prepares the exact request, a person confirms, and it then runs with that person's own login and permissions. The AI can never do anything the person couldn't do themselves.
  • Spoken confirmation is matched by the page, never by the model. Each prepared action is single-use and expires after 2 minutes.
  • Adding time is always billed. The assistant can't give free time.
  • Results are honest: "PC-01 received the shutdown command", never a claim a PC is off without proof. A PC only reports a message as shown once it is actually on screen.
  • Everything stays local. The agent is never exposed to the network without the dashboard login, and the only outbound call is to Amazon Bedrock.

Challenges we ran into

  • The model claimed actions were ready without preparing them. In long conversations it sometimes said "I've got that message ready, confirm to send it" without calling the tool, so no card appeared. Conversation history holds only the text of earlier replies, so the model copied the pattern. I added a guard that detects a "ready to confirm" reply with no prepared action, retries once cleanly, and otherwise answers honestly. In long test conversations this went from 7 of 10 action requests getting a card to 10 of 10, twice in a row.
  • A PC said it showed a message it never showed. While filming, an announcement never appeared on the real test PC, yet the PC reported "message displayed". The client acknowledged as soon as the pop-up was scheduled. Now a PC confirms only once the pop-up is on screen; otherwise it reports the failure, and the assistant says "PC-01 got it but couldn't show it".
  • Adding AI to a live cash system without breaking it. I built in small, reversible steps: opt-in token access that started read-only, new code in its own folders, and the existing backend tests run before and after every step. Along the way I found that adding time by minutes alone gave it for free, so the assistant only uses a paid package or a peso amount.
  • Real speech is messy. Chrome heard "Rogene" as "raw jin" and wrote "PC one" instead of "PC 1". Member search now suggests sound-alike matches ("Did you mean Rogene?") and PC names accept number words.
  • Choosing a model that doesn't invent facts. Amazon Nova 2 Lite was three times faster, but in testing it made up a session list and claimed a note was resolved when it wasn't. Claude Haiku 4.5 got every checked fact right, so I chose correctness over speed.
  • Building for how the shop actually works. My first close-shift check used cashier shifts, then I learned the owner had stopped using them in favour of "on duty". I rebuilt it as a drawer check against the cash taken while on duty.
  • Getting model access. On a new AWS account, every Bedrock model returned "Operation not allowed" for about two days. I built the agent with a configurable model provider so work could continue, and AWS Support enabled access after I contacted them.

Accomplishments that we're proud of

  • Added AI to a real system without breaking it: all existing tests passed after every step, and the new code has its own tests (87 for the MCP server, 103 for the agent).
  • A confirmation design where the AI physically can't skip the person.
  • Every result tells the truth, down to "received the shutdown command (test mode: not powered off)" and "got it but couldn't show it".

What we learned

  • Testing with a real model finds problems a fake one never shows: invented numbers, misread time zones, copied reply patterns.
  • Voice changes everything about wording: replies have to be short, spoken naturally and never in tables.
  • Build for the real workflow, and ask the real users before assuming how they work.

What's next for Secret Grounds Shift Assistant

  • A push-to-talk key on the owner's own PC, so they never have to turn to the cashier PC.
  • Connecting the MCP server to real Alexa+ once outside developers can register one.
  • Starting sessions and taking payments by voice, with the same confirmation design.
  • Rolling it out at the shop after a proper test on the real setup.

About the demo

The video uses a demo copy of the system with made-up members and sales. PC-01 is a real client on a second machine; the other 14 PCs are simulated so the demo shows a busy shop. The assistant, the MCP server, the agent and Amazon Bedrock are all real.

Built With

Share this project:

Updates

Submission history