VigilOR

VigilOR

VigilOR - AI-Powered Surgical Safety Monitor

Inspiration

VigilOR was inspired by the critical problem of retained surgical items (RSI), which occur in 1 in 5,000 surgeries and can lead to severe complications or death. Traditional manual counting is error-prone in high-stress surgical environments. We envisioned an AI system that never blinks or gets tired - providing constant, vigilant monitoring to prevent these preventable medical errors.

What it does

VigilOR is a real-time surgical tracking system that uses computer vision AI to monitor surgical instruments across two zones: the Tray Zone (where instruments should be stored) and the Incision Zone (the surgical site). It tracks items as they move between zones, provides critical warnings when attempting to close with items still in the patient, and performs baseline and post-surgery scans to detect discrepancies. The system maintains comprehensive session metrics with exportable compliance reports and supports both live camera feeds and uploaded video files.

How we built it We built VigilOR using React 18 with Tailwind CSS for the frontend, integrating two AI vision systems: Overshoot SDK for real-time streaming analysis and Roboflow for periodic validation. OpenCV.js handles image enhancement and preprocessing. We developed a sophisticated tracking algorithm that maintains item identity across frames using distance-based matching and multi-frame zone stability confirmation. The system uses React hooks for state management, with LocalStorage providing session persistence and comprehensive event logging for compliance. We leveraged TRAE Dancing Byte's platform to establish our development infrastructure and streamline our hackathon workflow.

Challenges we ran into

We struggled with balancing real-time performance against accuracy, solving this with a dual-frequency approach combining continuous streaming with periodic validation. Body part filtering was challenging - hands and gloves were initially detected as items, requiring multi-layer filtering through prompt engineering and keyword exclusion. Maintaining item identity across frames with occlusions required implementing sophisticated matching algorithms with fallback strategies and stale item retention. Zone transition stability was noisy, necessitating a multi-frame confirmation system to prevent false alerts.

Accomplishments that we're proud of

We successfully integrated two AI providers (Overshoot + Roboflow) for complementary real-time and validation capabilities. Our advanced tracking algorithm maintains identity across frames while handling occlusions with minimal false positives. The medical-grade UI features CCTV-style overlays, color-coded feedback, and intuitive zone calibration. We implemented comprehensive session management with full audit trails and exportable compliance reports, plus multiple safety layers including lock mechanisms and persistent warnings that prevent accidental session closure with items still in patients.

What we learned

We learned that medical computer vision requires multiple validation layers - a single AI model isn't sufficient for reliable results. Prompt engineering is critical for AI quality, and real-time systems need smart state management using refs for frequently-updating data to avoid render bottlenecks. Medical UIs must be unambiguous with clear visual hierarchy and explicit confirmations since mistakes could harm patients. Video processing has hidden complexity requiring careful normalization across different sources, aspect ratios, and frame rates. We also discovered that type-based filtering prevents tracking errors when different instruments overlap spatially.

What's next for VigilOR

Surgical Complications Monitoring: We'll expand beyond retained items to detect bleeding patterns, tissue damage, and anatomical landmarks in real-time. Integration with patient vitals will enable early complication detection and predictive analytics based on surgery patterns. Enterprise Integration: EHR/PACS integration for automatic logging, multi-camera systems with fusion algorithms for complete OR coverage, and hospital-wide analytics dashboards for safety metrics and trend analysis across all operating rooms.

Clinical Validation & FDA Approval: Partner with hospitals for validation studies and pursue 510(k) clearance. Develop comprehensive complication prediction models using historical data, patient history, and real-time monitoring to prevent adverse events before they occur.Claude is AI and can make mistakes. Please double-check responses.