Inspiration

Medical imaging plays a critical role in modern healthcare, but interpreting scans, managing patient information, prioritizing urgent studies, and preparing radiology reports can create a demanding workflow for radiologists.

We were inspired by a simple question:

What if medical image analysis, explainable AI, patient management, and radiology reporting could work together in one intelligent clinical workstation?

Many AI systems focus on producing a prediction. However, in healthcare, a prediction alone is not enough. Clinicians need visual explanations, relevant patient context, and the ability to review and document findings.

This inspired us to develop MedVision AI (RADMAX-AI) — an AI-assisted radiology platform designed around the needs of the clinical imaging workflow.

What it does

MedVision AI combines medical image analysis, Explainable AI (XAI), patient management, and structured radiology reporting in one platform.

Its key capabilities include:

1. AI-Assisted Medical Image Analysis

The platform incorporates deep learning architectures such as ResNet18 and DenseNet121 for medical image classification workflows. The project explores multiple imaging applications, including brain MRI, chest imaging, ultrasound, and other clinical datasets.

2. Explainable AI with Grad-CAM

Instead of displaying only a classification result, MedVision AI provides Grad-CAM visualizations that highlight image regions influencing the model's prediction. This helps users inspect model attention alongside the original scan.

3. Clinical Imaging Workstation

The workstation provides an imaging upload interface, patient selection, modality and protocol selection, scan visualization, and comparison between original images and AI-generated heatmaps.

4. Patient Management and EHR

The platform includes patient registration, demographic information, medical history, allergies, clinical notes, patient directories, and study history.

5. Structured Radiology Reporting

MedVision AI provides a structured reporting workflow with sections for clinical history, radiographic findings, diagnostic impression, and recommendations. Reports can be saved, finalized, and exported as PDF documents.

6. Intelligent Worklist and Triage

The radiologist dashboard organizes studies, displays AI processing statuses, highlights priority cases, and provides recent activity information.

7. MedVision Copilot

A conversational assistant interface helps users retrieve patient and study information through natural-language queries.

How we built it

We designed MedVision AI using a modular architecture that separates the clinical interface, AI inference services, and data persistence.

Frontend

  • React 19
  • Vite
  • Lucide Icons
  • Recharts
  • HTML5 Canvas
  • jsPDF

AI and Machine Learning

  • Python
  • PyTorch
  • Torchvision
  • TorchXRayVision
  • OpenCV
  • Scikit-learn
  • ResNet18 and DenseNet121
  • Random Forest

Explainability

  • Gradient-weighted Class Activation Mapping (Grad-CAM)
  • Feature-map visualization
  • Heatmap generation and image blending

Backend and Data

  • FastAPI
  • Uvicorn
  • Pydantic
  • Node.js and Express
  • SQLite with PostgreSQL compatibility

The workflow begins with patient and imaging information entering the clinical workstation. The backend handles image preprocessing and model inference. AI outputs and visual explanations are then presented in the workstation for human review. Patient information and reporting workflows connect the analysis to clinical documentation.

MedVision AI (RADMAX-AI): Explainable Deep Learning for Radiology Inspiration

Radiology workflows face three bottlenecks: interpreting high-dimensional scans, prioritizing urgent studies, and producing structured reports under time pressure. Deep learning can address the first two, but a bare class label is not clinically actionable. A radiologist needs to know where in the image the evidence lies and how confident the model is, so they can verify the output against their own reading.

This led us to a central question:

Can we combine the representational power of deep convolutional networks with spatial explainability to build a transparent, human-supervised radiology workflow?

MedVision AI (RADMAX-AI) is our answer. It pairs two CNN backbones (ResNet18 and DenseNet121) with Grad-CAM attribution, patient context, study management, and structured reporting, all designed to keep the radiologist in control.

  1. Convolutional Feature Extraction

CNNs learn hierarchical spatial representations: low-level edges and intensity gradients in early layers, textures and local structures in middle layers, and anatomical or pathological patterns in deeper layers. A convolutional layer computes:

$$Y_{i,j,k} = \sum_{m,n,c} X_{i+m,\,j+n,\,c}\,K_{m,n,c,k} + b_k$$

where X is the input feature map, K the learnable filter bank, b_k the bias of filter k, and Y:,:,k the resulting k-th activation channel. Because filter weights are shared across spatial positions, the layer is translation-equivariant and has far fewer parameters than a fully connected layer. Both properties suit medical images, where a lesion’s appearance matters more than its exact pixel location.

  1. ResNet18: Residual Learning

As networks grow deeper, optimization becomes harder, and accuracy can degrade even on training data. ResNet reformulates each block to learn a residual mapping instead of a direct one:

$$y = \mathcal{F}(x, {W_i}) + x$$ $x$: input feature map to the block $\mathcal{F}(x,{W_i})$: learned residual function (stacked convolution, batch normalization, and ReLU) $y$: output feature map

The identity shortcut gives gradients a direct path during backpropagation:

$$\frac{\partial \mathcal{L}}{\partial x} = \frac{\partial \mathcal{L}}{\partial y}\left(1 + \frac{\partial \mathcal{F}}{\partial x}\right)$$

The additive term $1$ prevents the gradient from vanishing through the block, which stabilizes training. ResNet18 (about 11.7M parameters) is a lightweight, well-validated backbone, which makes it a good fit for transfer learning on limited labeled medical data.

  1. DenseNet121: Dense Feature Reuse

DenseNet connects every layer to all preceding layers within a dense block, using concatenation rather than summation:

$$x_l = H_l\big([x_0, x_1, \ldots, x_{l-1}]\big)$$

This gives DenseNet121 three advantages:

Feature reuse: fine-grained textures from early layers stay accessible to deeper layers, which matters for subtle radiographic findings. Implicit deep supervision: each layer has a short path to the loss, which improves gradient flow. Parameter efficiency: about 8M parameters, fewer than ResNet18’s 11.7M, despite 121 layers.

  1. Softmax: Logits to Probabilities

The final layer produces a logit vector $z \in \mathbb{R}^C$. Softmax maps it to a categorical distribution:

$$P(y=c \mid x) = \frac{e^{z_c}}{\sum_{j=1}^{C} e^{z_j}}$$

Softmax outputs are not guaranteed to be calibrated probabilities. Modern CNNs tend to be overconfident, so confidence scores should be treated as a decision-support signal, not a clinical risk estimate, unless calibration (for example temperature scaling) is applied and validated.

  1. Cross-Entropy: Training Objective

We train with categorical cross-entropy, which measures the divergence between the ground-truth distribution and the model’s predicted distribution:

$$\mathcal{L} = -\sum_{c=1}^{C} y_c \log(\hat{y}_c)$$

$$\frac{\partial \mathcal{L}}{\partial z_c} = \hat{y}_c - y_c$$

This gives stable, well-scaled updates. In medical datasets, where class imbalance is common, the loss can be extended with class weights or sampling strategies so that rare but critical findings are not drowned out by majority classes.

  1. Grad-CAM: Making Predictions Inspectable

Grad-CAM (Gradient-weighted Class Activation Mapping) answers the question which regions of the image drove this class score? It needs no architectural changes and works with any CNN.

Step 1: For a target class c and a chosen convolutional layer with feature maps A^k, the importance of channel k is the global-average-pooled gradient of the class score:

$$\alpha_k^c = \frac{1}{Z}\sum_{i}\sum_{j}\frac{\partial y^c}{\partial A^k_{ij}}$$

where Z is the number of spatial positions and y^c is the pre-softmax score for class c. Using the pre-softmax logit avoids the suppression of gradients that softmax saturation can cause.

Step 2: Localization map. The weighted combination of feature maps is passed through ReLU:

$$L^c_{\text{Grad-CAM}} = \operatorname{ReLU}!\left(\sum_k \alpha_k^c A^k\right)$$

ReLU keeps only features with a positive influence on class c, which are the regions that support the prediction.

Step 3: Visualization. The map is upsampled (bilinear interpolation) to the input resolution, min-max normalized to $[0,1]$, color-mapped, and alpha-blended onto the original image.

Target layer. Grad-CAM is computed at the last convolutional stage (layer4 in ResNet18 and the final dense block in DenseNet121), where features are semantically rich but still spatially organized. The trade-off is resolution: deeper layers give a coarse map, so the heatmap indicates the region of interest, not exact lesion boundaries.

Limitations. Grad-CAM shows what the model attended to, not necessarily clinical causality, and a model can attend to spurious cues such as markers, devices, or acquisition artifacts. This is why we use heatmaps as a verification aid for the radiologist and not as a diagnostic output.

System Perspective

MedVision AI combines these components into a single workflow:

Layer Role Feature extraction ResNet18 / DenseNet121 CNN backbones Prediction Softmax class probabilities with confidence Explainability Grad-CAM overlays on the original study Context Patient information and study management Reporting Structured radiology report generation Oversight Human-in-the-loop review and sign-off Our Vision

Accuracy alone is not enough in clinical AI. Models must be inspectable, contextual, and accountable. By grounding predictions in sound mathematics, exposing the evidence behind them through Grad-CAM, and embedding them in a complete study-to-report workflow, MedVision AI aims to make AI assistance something a clinician can interrogate and trust.

Our goal is not to replace the radiologist, but to make AI-assisted medical imaging more transparent, connected, and useful, with every clinical decision remaining under human supervision.

Challenges we faced

One of our biggest challenges was integrating different components into a unified application.

Medical images require careful preprocessing, and different imaging modalities have different characteristics. Building a consistent workflow while accommodating different model architectures and input requirements required modular design.

Another challenge was making AI predictions understandable. A numerical confidence score alone does not explain which image regions influenced a prediction. We integrated Grad-CAM visualization to provide an additional interpretability layer.

We also worked on connecting patient records, image analysis, report generation, and dashboard navigation without making the interface difficult to use.

What we learned

Through this project, we learned that building an AI healthcare application requires more than training a machine learning model.

A useful clinical AI platform must consider:

  • Model performance and evaluation.
  • Explainability and uncertainty.
  • Patient data consistency and privacy.
  • Human supervision and clinical workflows.
  • Usability and accessibility.
  • Reliable integration between frontend, backend, and AI services.

Most importantly, we learned that AI should support clinical professionals rather than replace their judgment.

What's next for MedVision AI?

Our future development goals include:

  • More robust multi-slice DICOM and volumetric imaging support.
  • Expanded modality-specific AI models.
  • Improved model validation across independent datasets.
  • More comprehensive clinical audit trails and access controls.
  • FHIR-based healthcare interoperability.
  • Federated learning for privacy-conscious model development.
  • Enhanced reporting and clinical workflow automation.

Our vision is to make medical imaging workflows more explainable, connected, and efficient — while keeping the radiologist at the center of every clinical decision.

Built With

Share this project:

Updates

Submission history