RadScan AI

RadScan AI: Autonomous 2.5D volumetric MRI triage and explainable Grad-CAM report generator powered by Vertex AI Gemini.

Comment

RadScan AI: Autonomous Radiology Triage & Multimodal Report Copilot

The Story Behind RadScan AI

Radiologists are facing massive burnout. On average, a radiologist evaluates 50+ volumetric MRI studies every single day. Each study contains 24 to 40 high-resolution DICOM slices. Manually inspecting every plane, checking ligament continuity across slices, and dictating repetitive clinical reports takes anywhere from 10 to 15 minutes per patient.

I built RadScan AI to fix this workflow bottleneck.

Instead of forcing doctors to scroll back and forth through dozens of raw DICOM slices, RadScan AI acts as an autonomous co-pilot: it runs a 2.5D Volumetric CNN-BiGRU model across all 24 slices, flags 12 different knee pathologies, generates a Grad-CAM heatmap over the exact tear location, and uses Vertex AI Gemini 1.5/3.5 Pro to draft a structured clinical report in under 3 seconds.

How It Works

  1. DICOM & Sample Selection: You can pick one of the pre-loaded 1-click test cases (ACL Tear, Meniscus Tear, Normal Knee) or drop a custom .dcm file.
  2. 2.5D Volumetric Classification: The backend model doesn't just look at isolated 2D images. It processes 24 parallel slices across Sagittal, Coronal, and Axial views using a spatial CNN combined with a depth-wise BiGRU temporal model.
  3. Grad-CAM Visual Explainability: Radiologists don't trust black-box AI scores. The app overlays a glowing red/yellow Grad-CAM heatmap directly on the target slice, with an opacity slider so doctors can verify the exact anatomical coordinates.
  4. Gemini 1.5/3.5 Pro Report Drafting: The model tensor features feed into Vertex AI Gemini, which drafts two views:
    • Radiologist Clinical View: Official DICOM findings, impression, and orthopaedic recommendations.
    • Patient Portal View: A simple, jargon-free summary that patients can actually understand.

How I Built It

  • Frontend: Built using Next.js 14 (App Router), React 18, and Tailwind CSS. Features a custom 24-slice DICOM canvas viewer with interactive Grad-CAM opacity sliders and an interactive 6-step guided tour for judges.
  • Backend API: Built with FastAPI and PyTorch. The microservice is containerized with Docker and configured for GCP Cloud Run (with NVIDIA L4 GPU scale-to-zero support).
  • LLM Integration: Vertex AI Gemini 1.5/3.5 Pro SDK (google-cloud-aiplatform) for clinical impression drafting.

The Math Behind the 2.5D Model

Depth-Wise BiGRU Slice Attention

For an MRI volume stack S = {s1, s2, ..., s24}, spatial CNN features f_t are passed to a Bidirectional GRU to track ligament continuity across consecutive depth slices:

h_t = [ GRU_fw(f_t, h_{t-1})  ||  GRU_bw(f_t, h_{t+1}) ]
P(Pathology_i | S) = Sigmoid( W_i * h_t )

Built With

Share this project:

Updates

Submission history