SurgiGuard AI v2.0

• 

  SurgiGuard AI Cockpit: Real-time intra-operative telemetry and surgical case tracking dashboard.

• 

  Multimodal AI visual object localization detecting lap sponges, needles, and clamps on the surgical tray.

• 

  Scenario A: Deterministic mathematical count verification confirming zero cavity delta (CLEARED [GO]).

• 

  Scenario B: Hazard interlock triggers DISCREPANCY [HOLD] and audible voice alert on missing sponge.

• 

  Gravimetric Telemetry: Dynamic estimated patient blood loss (EBL) calculated at 1.06 g/mL sponge density.

• 

  One-click HL7 FHIR R4 procedure export with standardized LOINC and SNOMED-CT clinical ontologies.

• 

  Cryptographic Audit Blackbox: SHA-256 Merkle ledger detecting tamper attempts for Part 11 compliance.

• 

  Structured Gemini 2.5 Flash schema integration with deterministic validation to eliminate hallucinations.

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

Inspiration

Every year, surgical teams perform millions of life-saving procedures. Yet, Retained Foreign Objects (RFOs)—surgical sponges, needles, and instruments unintentionally left inside patients—remain a persistent \$1.3 billion catastrophe in healthcare. Traditional manual whiteboard counts are vulnerable to human fatigue, cognitive overload, and emergency conversion chaos. We set out to solve this safety crisis by combining multimodal AI vision with deterministic verification under a strict clinical axiom: Rules Engine Decides, AI Explains.

What it does

SurgiGuard AI v2.0 is an intra-operative reconciliation and telemetry engine designed to guarantee zero retained foreign objects before surgical incision closure.

• Deterministic Closure Gate: Bridges multimodal computer vision with a strict mathematical balance checker. The closure gate only authorizes clearance (GATE STATUS: CLEARED [GO]) when the calculated discrepancy delta equals exactly zero ($\Delta = 0$).
• Multimodal Visual Object Localization: Uses Google Gemini 2.5 Flash with structured schema validation to detect, count, and classify surgical trays, sponges, needles, and clamps.
• Hazard Interlock & Audible Web Speech Alerts: Detects missing items instantly, engages a crimson interlock (DISCREPANCY [HOLD]), and broadcasts touchless, synthesized voice alerts across the operating room.
• Gravimetric Blood Loss (EBL) Telemetry: Dynamically calculates estimated blood loss in milliliters using sponge wet-mass density calibrated at $1.06\text{ g/mL}$.
• Interoperable FHIR R4 Export: Generates HL7 FHIR R4 Procedure and Observation bundles with native LOINC and SNOMED-CT clinical ontologies with one click.
• FDA 21 CFR Part 11 Cryptographic Blackbox: Implements a SHA-256 Merkle ledger recording every count event to ensure tamper-evident medical-legal compliance.

How we built it

• Multimodal Engine: Google Gemini 2.5 Flash API configured with strict Zod JSON schemas to enforce structured object detection without LLM hallucination.
• Frontend Cockpit: Built with Next.js, React, and Tailwind CSS as a low-cognitive-load, high-contrast Obsidian Dark cockpit for the operating room.
• Intra-Operative Audio: Implemented via the browser-native Web Speech API for hands-free audio announcements to the surgical team.
• Clinical Informatics Layer: Mapped data structures to HL7 FHIR R4 standards using standard medical coding (LOINC for gravimetric blood loss, SNOMED-CT for surgical counts and RFO identification).
• Cryptographic Ledger: Custom client-side SHA-256 Merkle tree implementation for tamper-evident data logging.

Challenges we ran into

• Eliminating AI Hallucinations in Safety-Critical Systems: LLMs cannot be trusted to perform raw arithmetic in surgical environments. We decoupled the architecture: Gemini handles visual extraction and localization, while an isolated, deterministic state kernel performs the count balance and gate interlock logic.
• Standardizing Health Informatics Data: Mapping live telemetry into structured HL7 FHIR R4 resources required rigorous schema alignment with LOINC and SNOMED-CT code sets.
• Real-time UX Under Pressure: Designing a high-stress cockpit UI that communicates vital information instantly without causing visual alarm fatigue.

Accomplishments that we're proud of

• Engineering a zero-trust, deterministic safety architecture that guarantees the AI cannot override hard mathematical safety rules.
• Building a complete end-to-end gravimetric telemetry system calculating patient blood loss with medical precision.
• Creating an immutable cryptographic ledger demonstrating Part 11 audit compliance right in the browser.

What we learned

• Multi-modal models excel at semantic scene parsing, but clinical safety requires strict sandwiching between deterministic validation layers.
• Building for healthcare means meeting hospital infrastructure halfway by prioritizing interoperability standards (FHIR R4) from day one.

What's next for SurgiGuard AI

• Direct integration with hospital Electronic Health Record (EHR) systems like Epic and Cerner via SMART on FHIR.
• Edge deployment on sterilizable OR camera hardware for ultra-low-latency real-time video feed inference.
• Expanding computer vision models to track multi-quadrant abdominal packs and micro-vascular surgical needles. ## What's next for SurgiGuard AI v2.0

Built With

• cryptography
• fhir-r4
• gemini-2.5-flash
• google-gemini-api
• healthcare
• hl7-fhir
• loinc
• next.js
• react
• sha-256
• snomed-ct
• tailwind-css
• typescript
• web-speech-api
• zod