NeuroSim

NeuroSim turns T1 MRI scans into interactive 3D surgical sims with anatomy visualization, real-time stress analysis, AI-generated risk reports, and uncertainty bounds to aid neurosurgical planning.

Comment

NeuroSim : AI-Powered Neurosurgical Simulation

Inspiration

Modern neurosurgery demands millimeter-level precision. While MRI provides detailed imaging, surgeons must mentally reconstruct 3D structure from 2D slices.

We asked:

What if surgeons could rehearse procedures in a physics-aware digital twin before making an incision?

NeuroSim was built to bridge medical imaging, interactive 3D visualization, and AI-driven risk analysis into one unified surgical planning system.

What We Built

NeuroSim transforms a T1-weighted MRI (.nii.gz) into:

  • Automated segmentation of ~61 anatomical regions
  • 3D mesh reconstruction
  • Interactive browser-based visualization
  • Real-time stress estimation on selected tissue
  • AI-generated surgical risk report with uncertainty bounds

The system effectively creates a _patient-specific digital twin.

How We Built It

1. MRI Processing & Segmentation

We represent the volumetric scan as:

$$ V(x,y,z) \in \mathbb{R} $$

Segmentation assigns labels:

$$ S(x,y,z) \in {0,1,2,\dots,61} $$

We extract surfaces using marching cubes to generate triangular meshes suitable for WebGL rendering.

2. 3D Visualization

We used Three.js for real-time browser rendering:

  • GPU-accelerated WebGL
  • Interactive orbit controls
  • Click-based tissue selection
  • Transparency and isolation tools

This enables surgeons to rotate and inspect structures intuitively.

3. Real-Time Stress Modeling

When a region is selected, we estimate mechanical stress using simplified continuum mechanics:

$$ \sigma = E \cdot \epsilon $$

Where:

  • ( \sigma ) = stress
  • ( E ) = elastic modulus
  • ( \epsilon ) = strain

A full finite element solution would require solving:

$$ \nabla \cdot \sigma + f = 0 $$

But we implemented a lightweight approximation to maintain real-time responsiveness.

4. AI Risk Assessment

We synthesize anatomical proximity, deformation risk, and structural density into a quantitative risk estimate:

$$ \text{Risk Score} = \mu \pm \sigma $$

The system generates:

  • Surgical approach recommendations
  • Risk classification
  • Confidence bounds
  • Explanation of contributing anatomical factors

Challenges We Faced

Medical Data → Web Rendering

MRI volumes are large and computationally heavy. Converting them into optimized meshes required decimation and preprocessing.

Accuracy vs Speed

Full biomechanical simulations are computationally expensive. We had to balance realism with real-time interactivity.

Communicating Uncertainty

Presenting AI confidence clearly without overwhelming clinicians required thoughtful design.

End-to-End Integration

The full pipeline included:

  • MRI ingestion
  • Segmentation
  • Mesh generation
  • Browser rendering
  • AI inference
  • Report streaming

Integrating these systems seamlessly was one of the biggest engineering challenges.

What We Learned

  • Interpretability matters more than complexity in medical AI.
  • Real-time systems require principled trade-offs.
  • Visualization dramatically improves surgical intuition.
  • Cross-domain engineering (ML + physics + graphics) is essential.

Built With

Share this project:

Updates