Inspiration

The inspiration for DriftGuard comes directly from the critical need to address the life-or-death risks associated with artificial intelligence degradation in healthcare, specifically highlighted in the research paper "Risks of Model Drift in AI-Guided Robotic Surgery Over Time". While AI has revolutionized minimally invasive robotic surgery by improving precision and reducing operative times, these models are highly susceptible to "model drift". This drift—whether it is a covariate shift from changing patient demographics or a concept shift from evolving surgical techniques—can cause a 4.0% to 6.0% degradation in AI performance across specialties like orthopedics and laparoscopic surgery. Recognizing that even subtle inaccuracies in surgical guidance can lead to compromised patient safety and increased error rates, we wanted to translate this complex theoretical research into a tangible, proactive monitoring solution.

What it does

DriftGuard is a comprehensive, interactive frontend proof-of-concept dashboard designed to monitor AI model health across multiple surgical specialties (such as cardiac, neurosurgery, and orthopedics).

Key features include:

Drift Detection & Visualization: It tracks 24-month performance degradation and visualizes specific drift events, such as covariate shifts in patient age and BMI distributions.
Live Equation Engine: It actively computes complex statistical drift detection algorithms—including Maximum Mean Discrepancy (MMD), Kolmogorov-Smirnov (K-S) Test, and Population Stability Index (PSI)—live within the application.
Human-in-the-Loop (HITL) Validation: An expert annotation workflow where surgeons can review, validate, or escalate flagged AI predictions.
Clinical Workflows: It features an automated 5-point Pre-Op Health Check to provide a clear "Go/Warn/Hold" verdict before surgery, a Model Registry with version control, and a Shadow Mode that compares old and new models side-by-side.

How we built it

We built DriftGuard as a robust, 9-page frontend application. We designed a clinical, high-tech visual identity using a dark navy/slate palette accented with neon-cyan and emerald for the data visualizations.

We started by mapping out the core entities required (Drift Events, Model Performance, Recalibration Actions) and built the UI components in parallel. To make the prototype realistic, we seeded the application with comprehensive clinical data across various surgical domains. We also engineered the core mathematical logic from scratch in JavaScript so the application could actively calculate and render statistical drift formulas instead of just displaying static mockups.

Challenges we ran into

One of our biggest challenges was moving from a descriptive dashboard to a fully functional mathematical prototype. Initially, the app simply described the detection methods (MMD, K-S, PSI) and used randomized data for its charts. We had to write a live "Equation Engine" from scratch in JavaScript to ensure the app could actively calculate metrics like the MMD RBF kernel and the K-S test critical values against sample distributions.

Additionally, tailoring the user experience to a highly regulated medical environment was challenging. We had to ensure the workflows mirrored reality by building a Human-in-the-Loop (HITL) module with agree/disagree/escalate options for surgeons, and an FDA/EU MDR compliance Audit Trail.

Accomplishments that we're proud of

We are incredibly proud of the live Equation Engine. Implementing algorithms like Maximum Mean Discrepancy (MMD) and the Population Stability Index (PSI) to compute live in JavaScript allows users to tune parameters with sliders and watch the statistical charts update in real time.

We are also proud of successfully translating a highly academic, complex problem into a polished, 9-page intuitive interface. Features like the Shadow Mode, which evaluates the accuracy delta between two models before switching, and the Pre-Op Health Check successfully demonstrate how this highly theoretical math applies to a practical, pre-surgery safety workflow.

What we learned

Building DriftGuard taught us a tremendous amount about the specific vulnerabilities of AI in clinical settings. We learned the critical distinctions between data drift (like a covariate shift where the patient population ages or average BMI increases) and concept drift (where the relationship between inputs and outputs changes, such as when new surgical best practices are adopted).

We also learned that traditional performance metrics (like accuracy) are often delayed "lagging indicators" in healthcare, and that statistical data distribution monitoring using the Maximum Mean Discrepancy (MMD) method is superior due to its high sensitivity (0.89) and faster 7-day detection latency.

What's next for DriftGaurd

To transition DriftGuard from an advanced hackathon prototype to a fully deployable medical device for hospitals, we need to focus heavily on backend infrastructure and regulatory compliance:

Real Data Integration: We need to replace our simulated data with DICOM and HL7 FHIR connectors to pull live electronic health records (EHR) and imaging, as well as build live API bridges to surgical robots like the da Vinci.
Security & Access Control: We must implement Role-Based Access Control (RBAC) to differentiate the views and permissions of surgeons, ML engineers, and hospital admins, alongside MFA enforcement.
Multi-Site Capabilities: Because drift is relative (one hospital's "normal" baseline differs from another's), we need to build federated learning coordination and per-hospital baseline profiles.
Clinical Workflows: Integrating the app directly into operating room scheduling systems and requiring formal surgeon consent or override flows before a surgery begins.

[Please note that we do declare our use of AI to help us make this project ]