Surgical Tool Detection

A complete agentic AI system for detecting surgical tools in laparoscopic surgery videos using dendritic neural networks, deployed with Docker and orchestrated via MCP servers.

Comment

This hackathon project introduces a Multi-Scale Agentic AI System designed to detect surgical tools in real-time using Dendritic Neural Networks. By mimicking the complex, segment-wise computations of biological neurons, the system achieves higher accuracy and parameter efficiency than traditional models.

## Core Innovation: Dendritic Computation

Traditional AI uses simplified "neurons." This project implements Dendritic Layers that process information in segments, much like real biological cells.

  • Dendritic Gating: Learns to prioritize specific features (e.g., fine textures for graspers vs. broad context for irrigators).
  • Multi-Scale Detection: Processes images at three different resolutions () to capture both tiny tool details and the overall surgical scene.
  • Efficiency: Achieved a 7% improvement in accuracy ( mAP) with only a 14% increase in parameters.

## System Architecture: The Agentic Approach

The project isn't just a model; it’s a production-ready ecosystem of 13 microservices coordinated via the Model Context Protocol (MCP).

Component Technology Role
Orchestration FastAPI Gateway Routes traffic and manages multi-step workflows.
Data Engine PySpark & MinIO Distributed processing of surgical video frames.
Real-time Layer Redis & WebSockets Provides sub-100ms dashboard updates.
Inference PyTorch (CPU-optimized) Delivers predictions in 15ms (approx. 66 FPS).

## Challenges & Lessons Learned

  • Hardware Constraints: Switched to CPU-only Docker images to ensure the system could be deployed in hospitals without expensive GPU clusters.
  • Gradient Flow: Biological models don't naturally "backpropagate" well. Adding residual connections was essential to make the dendritic layers trainable.
  • Async Performance: Using asynchronous database drivers (Motor for MongoDB) resulted in a 10x speed improvement for the API.

## Future Impact

This work bridges the gap between biological research and clinical application. By providing a "vertical slice" of a working system, it sets the stage for:

  1. Safety Monitoring: Alerting surgeons to misplaced tools in real-time.
  2. Autonomous Robotics: Providing "tool awareness" for surgical robots.
  3. Edge Deployment: Running high-accuracy AI on low-power medical devices.

Next Step: Would you like me to generate a simplified code snippet demonstrating how the Dendritic Gating mechanism is implemented in PyTorch?

Built With

Share this project:

Updates