Checklist Assistant: Pre-Incision Safety Simulation

A voice-first MCP simulation that flags checklist omissions, contradictions, and unclear speech—using synthetic data only.

Comment

Inspiration Surgical safety checklists exist because small omissions in the final moments before incision can have serious consequences. In practice, the time-out is a rushed, high-pressure moment, and a paper or on-screen checklist only records what someone ticks. It doesn't notice a skipped item or a spoken answer that contradicts the patient's chart. We wanted to explore whether a voice-first assistant, built as an MCP server for Alexa+, could support the team in that moment without taking decisions away from them.

What it does Checklist Assistant is a simulation-only voice assistant for one workflow: the pre-incision safety checklist. Using synthetic patients, it walks a simulated team through eight items adapted from the public WHO Surgical Safety Checklist. It: retrieves approved protocol text for each item tracks the state of every item flags omissions and contradictions against the synthetic patient record (for example, "no allergies" spoken for a patient whose chart lists penicillin) asks for a repeat when speech recognition confidence falls below 0.6 gives short spoken prompts, and stops talking when interrupted (barge-in) logs every action to an audit trail It never diagnoses or recommends treatment, and it can never mark an item complete. Only the designated clinician's confirmation can. We compared it against a static digital checklist across four scripted scenarios: a clean run, a skipped item, a contradiction, and noise with interruptions.

How we built it MCP server: a self-hosted server at /api/mcp on Next.js, deployed on Vercel, using Streamable HTTP (protocol version [version you saw in Inspector]). It exposes seven tools: start_session, get_protocol_item, record_response, check_consistency, get_status, confirm_item, and get_audit_log. Deterministic safety logic: state transitions, consistency rules, the confidence threshold, and the clinician-confirmation gate are plain code, not prompts, so the safety rules can be tested and don't depend on model behavior. Supabase: stores synthetic patients, protocol content, sessions, responses, confirmations, and the audit log. Serverless functions don't keep memory between calls, so session state lives here. Groq: interprets spoken utterances, selects tools, and writes concise prompts under a system prompt that forbids diagnosis and treatment advice. Console and voice: a simulated Alexa+ style web console with browser speech recognition and synthesis, a live tool-call panel, and a speech simulator for controlled confidence levels. Evaluation: a scenario runner replays four scripted transcripts through both the assistant and a static checklist and exports the metrics as CSV. Tooling: we used Google Antigravity to generate code from a written SPEC.md, and reviewed the safety-critical parts ourselves.

Challenges we ran into Connecting an AI-generated voice interface to a stateful MCP workflow required careful handling of session IDs, tool results, authentication, and frontend state. We also had to distinguish successful tool discovery from successful execution of a real tool call against the deployed server.

Another challenge was evaluating the assistant fairly against a static checklist. We modelled the static approach as a user ticking every displayed item without automated contradiction detection or confidence checks. This is an explicit benchmark assumption, not a claim that all real-world static checklists behave that way.

Finally, browser audio capture and speech-recognition behavior varied across the recording and testing workflow, making short end-to-end checks important before preparing a demo.

Accomplishments that we're proud of A working, deployed MCP server over Streamable HTTP that the console calls for every action A hard guarantee in code that the assistant cannot complete a checklist item on its own Contradiction and omission detection against a synthetic record, with every decision in an audit log A reproducible scenario evaluator with exportable results. Built and shipped within the hackathon window

What we learned For safety-sensitive workflows, put the rules in code and use the LLM for understanding and wording, not authority. MCP tool results need careful parsing on the client, and small state-handling mistakes (like stale closures in speech callbacks) can break a whole voice flow. A good simulation is mostly about being honest regarding what it can and can't show. Writing a clear spec first made AI-assisted coding much faster, but the safety-critical code still needed human review.

What's next for Checklist Assistant: Pre-Incision Safety Simulation Production-grade authentication using the MCP authorization flow instead of a bearer token Testing on real Alexa+ devices and with real speech streams, including theatre noise and multiple speakers Evaluation with clinicians and human-factors experts, with a proper study design Support for other checklists and languages, and integration with synthetic EHR data standards Richer timing and sequencing checks (for example, antibiotic timing against the incision time) Open-sourcing the checklist MCP server as a reusable template for other safety-critical workflows

Built With

Share this project:

Updates

Submission history