CortexView

Generative where it helps, Deterministic where it matters

Comment 1

CortexView: Making Brain MRI Understandable

1. Inspiration & Brainstorming

Brain MRIs are often difficult for patients to interpret. Raw DICOM scans present complex black-and-white cross-sections that offer little intuitive understanding without specialized medical training. Patients are frequently left navigating unfamiliar clinical terminology, while many AI healthcare prototypes rely on conversational interfaces that risk oversimplifying medical information or autonomous diagnostic systems that may produce unreliable conclusions.

We set out to bridge this gap by building CortexView, an educational medical visualization platform designed to improve patient health literacy.

Our central engineering principle was simple:

“Generative where it helps, deterministic where it matters.”

Generative diffusion models enhance visual clarity by transforming anatomical segmentations into accessible illustrations. Meanwhile, classical computer vision and spatial mathematics independently calculate measurements, volumes, and geometric boundaries. This separation ensures that generative rendering cannot modify or invent the medical metrics presented to users.

CortexView is designed to make complex anatomical information more approachable without replacing clinical interpretation or diagnosis.

2. What It Does: The Final Product

CortexView transforms dense DICOM imaging data into an interactive, layered anatomical explorer, targeting an end-to-end processing time of approximately 5–6 seconds.

DICOM Ingestion & Privacy Protection

Users or clinicians can upload individual .dcm slices or supported archives. The FastAPI backend sanitizes applicable Protected Health Information (PHI) and normalizes spatial coordinates and acquisition metadata before downstream processing.

2.5D Layered Medical Visualization

CortexView converts anatomical segmentation masks into clean, visually distinct illustrations representing structures such as the cerebrum, ventricles, and annotated lesions. These layers provide a more intuitive representation of anatomical relationships than isolated grayscale slices.

Anatomical Focus Slider & Interactive Pins

An interactive focus slider allows users to isolate individual anatomical layers. Coordinate-anchored measurement pins and contextual cards let users inspect selected structures and explore their associated spatial information.

Deterministic Metrics & Grounded Explanations

Selecting a layer reveals mathematically derived physical volumes, maximum caliper diameters, and relevant anatomical context. Educational explanations use predefined templates rather than unrestricted generative medical advice.

The result is an interactive visualization experience that prioritizes clarity, traceability, and separation between visual enhancement and quantitative analysis.

3. Design Process & Technical Architecture

CortexView follows a modular architecture that separates privacy preprocessing, deterministic geometric computation, generative illustration, and frontend presentation.

The FastAPI backend coordinates the pipeline, while two independent engines process the sanitized imaging data. Their outputs are reconciled through a typed Pydantic v2 data contract before reaching the React client.

System Architecture

The following diagram illustrates the primary data flow and the separation between deterministic analysis and generative rendering.

\begin{figure}[htbp]
\centering
\resizebox{\textwidth}{!}{%
\begin{tikzpicture}[
    font=\sffamily,
    >=Latex,
    flow/.style={
        -{Latex[length=2.5mm]},
        thick,
        draw=gray!75
    },
    main/.style={
        draw=blue!55!black,
        fill=blue!5,
        rounded corners=3mm,
        thick,
        align=center,
        text width=10cm,
        minimum height=10mm,
        inner sep=5pt
    },
    engine/.style={
        rounded corners=3mm,
        thick,
        align=left,
        text width=6.5cm,
        minimum height=32mm,
        inner sep=7pt
    },
    geometry/.style={
        engine,
        draw=teal!65!black,
        fill=teal!5
    },
    generative/.style={
        engine,
        draw=violet!65!black,
        fill=violet!5
    },
    contract/.style={
        main,
        draw=orange!75!black,
        fill=orange!8
    },
    client/.style={
        main,
        draw=blue!65!black,
        fill=blue!9
    }
]

\node[main] (input) {
    \textbf{Raw DICOM Upload}
};

\node[main, below=7mm of input] (backend) {
    \textbf{FastAPI Backend}\\
    PHI Sanitization \& Coordinate Normalization
};

\node[geometry, below left=12mm and 3mm of backend] (geo) {
    \textbf{\large Deterministic Geometry Engine}\\[2mm]
    \textbullet\ PixelSpacing \& SliceThickness\\
    \textbullet\ SciPy EDT-Based Anchor Placement\\
    \textbullet\ Convex Hull \& Caliper Diameter\\
    \textbullet\ Voxel-Based Volume Computation
};

\node[generative, below right=12mm and 3mm of backend] (gen) {
    \textbf{\large Generative Illustration Engine}\\[2mm]
    \textbullet\ FLUX.2 Klein 4B + LoRA\\
    \textbullet\ Segmentation-Conditioned Rendering\\
    \textbullet\ Cloudflare-Based Inference
};

\node[contract, below=15mm of geo.south west,
      anchor=north west, text width=13.5cm] (schema) {
    \textbf{Typed JSON Schema Contract}\\
    Pydantic v2: Metrics, Coordinates \& Illustration References
};

\node[client, below=7mm of schema] (frontend) {
    \textbf{React 19 / Tailwind CSS v4 Client}\\
    Interactive 2.5D Layers, Focus Slider \& Measurement Cards
};

\draw[flow] (input) -- (backend);

\coordinate (split) at ($(backend.south)+(0,-4mm)$);
\draw[thick,draw=gray!75] (backend.south) -- (split);
\draw[thick,draw=gray!75] (split) -| (geo.north);
\draw[thick,draw=gray!75] (split) -| (gen.north);

\coordinate (merge) at ($(schema.north)+(0,5mm)$);
\draw[thick,draw=gray!75] (geo.south) -- ++(0,-5mm) -| (merge);
\draw[thick,draw=gray!75] (gen.south) -- ++(0,-5mm) -| (merge);
\draw[flow] (merge) -- (schema.north);
\draw[flow] (schema) -- (frontend);

\end{tikzpicture}%
}
\caption{CortexView's modular architecture, separating deterministic
geometric analysis from generative anatomical illustration.}
\label{fig:cortexview-architecture}
\end{figure}

3.1 Fine-Tuned Generative Rendering

Foundation model: FLUX.2 Klein 4B, adapted to generate publication-quality anatomical illustrations conditioned on color-coded segmentation masks.

Parameter-efficient fine-tuning: We used Low-Rank Adaptation (LoRA) with the Hugging Face Diffusers and PEFT ecosystems, alongside memory-optimization techniques intended to make fine-tuning feasible on consumer hardware.

Inference pipeline: The rendering service is integrated with the backend through a cloud-hosted inference workflow. Cloudflare infrastructure and tunneling are used where applicable to expose and connect the inference endpoint.

The generative engine is responsible for visual presentation—not measurement, diagnosis, or authoritative anatomical boundary computation.

3.2 Deterministic Metric Engine

The deterministic engine derives its outputs from segmentation data and acquisition metadata rather than asking a language or diffusion model to estimate physical measurements.

Spatial geometry — scipy.spatial

Convex hull vertex extraction and pairwise Euclidean distances are used to determine maximum caliper diameters for supported segmented structures.

For a set of points (P={p_1,p_2,\ldots,p_n}), the maximum pairwise distance is

[ D_{\max}=\max_{i,j}|p_i-p_j|_2. ]

Distances are converted into physical units using the appropriate image-coordinate spacing.

Deterministic pin placement — scipy.ndimage

Euclidean Distance Transforms (EDTs) identify suitable interior anchor locations within segmentation masks. These locations provide stable measurement-pin placement without relying on arbitrary bounding-box centers.

Physical volume derivation — NumPy

For a binary segmentation mask containing (N) voxels, the voxel-based volume is computed as

[ V=N\,s_xs_ys_z, ]

where (s_x) and (s_y) represent in-plane pixel spacing and (s_z) represents the effective slice spacing or thickness appropriate to the acquisition.

For nonuniform slice spacing, the implementation must account for individual slice geometry rather than assume a constant thickness.

Typed data contracts — Pydantic v2

The geometry engine's physical measurements, coordinates, and predefined explanatory content are serialized into typed schemas. Illustration outputs remain separate from authoritative numeric fields, preventing generative rendering from silently changing reported measurements.

4. Resources & Technology Stack

Layer Technologies
Frontend React 19, TypeScript, Vite, Tailwind CSS v4, Lucide Icons, React Markdown
Backend API Python 3.10+, FastAPI, Uvicorn, Pydantic v2
Medical imaging & mathematics pydicom, NumPy, SciPy (ndimage, spatial), scikit-image, Pillow
Generative AI & fine-tuning FLUX.2 Klein 4B, PyTorch, LoRA, PEFT, Diffusers, bitsandbytes, Accelerate
Cloud & deployment Cloudflare Workers AI, Cloudflare Tunnel (cloudflared), as applicable to the deployed inference configuration
Research datasets TCIA ReMIND, Kaggle LGG dataset
Development assistance Claude 3.5 Sonnet and Gemini for architectural planning and pair programming; no runtime dependence on these development assistants

5. Challenges We Encountered

Fine-Tuning a 4B Diffusion Model on Consumer Hardware

Adapting a four-billion-parameter diffusion model under an 8 GB VRAM constraint on an NVIDIA RTX 5060 required careful memory management. We explored techniques including quantization, mixed-precision computation, gradient checkpointing, and selective LoRA target-module training to reduce memory requirements.

These optimizations involved trade-offs between training speed, numerical stability, and the range of model components that could be adapted.

Medical Data Heterogeneity

Public medical-imaging datasets introduce substantial preprocessing challenges. We had to account for differences in DICOM acquisition planes, slice thickness, spatial orientation, segmentation formats, and multi-page TIFF masks.

Harmonizing these inputs required custom preprocessing to align segmentation masks with the source images and normalize coordinate systems before calculating physical measurements.

Decoupling Visual Quality from Mathematical Accuracy

A generative model can produce visually convincing illustrations without preserving every anatomical boundary exactly. Allowing generated pixels to become the source of measurement would therefore undermine the reliability of the system.

We addressed this architectural risk by keeping geometric masks and calculated metrics independent of the generative rendering pathway. The frontend receives the illustration and its associated authoritative spatial data as separate outputs, preserving the distinction between visual interpretation and quantitative computation.

6. Accomplishments We're Proud Of

Functional separation of concerns. CortexView demonstrates an architecture in which generative AI enhances accessibility and visual presentation while a separate mathematical pipeline remains responsible for quantitative measurements.

Rapid visualization. We targeted an end-to-end processing lifecycle of approximately 5–6 seconds, from image upload to interactive 2.5D presentation.

Testable, reproducible geometry. We implemented automated pytest test suites to validate geometric calculations and data handling. This establishes a foundation for reproducibility and regression testing of the analytical pipeline.

Patient-centered design. By combining layered visualization, interactive measurement pins, and fixed educational explanations, CortexView aims to make complex imaging data easier to explore without presenting itself as an autonomous diagnostic system.

7. What's Next for CortexView

Expanded Anatomical and Pathology Support

We plan to investigate extending the platform beyond its initial neuro-oncology use cases to additional clinical imaging scenarios, including ischemic stroke and structural ventricular abnormalities. Such extensions would require pathology-specific segmentation, validation, and clinical review.

A Privacy-First Medical Companion

Our longer-term vision is to package CortexView as a secure web application that helps patients explore their imaging before or after consultations with radiologists, neurologists, and oncologists.

Future work will focus on stronger privacy safeguards, robust handling of diverse DICOM inputs, clinically reviewed educational content, and validation of the visualization and measurement pipeline against appropriate reference standards.

Our Vision

CortexView is built around a distinction that we believe matters in AI-assisted healthcare: the model may improve how information is seen, but it must not invent what the measurements say.

By combining generative illustration with deterministic geometry, we aim to make medical imaging more understandable while keeping quantitative outputs grounded in the underlying data.

Built With

Share this project:

Updates

Submission history