CardioLens

Flowchart

CardioLens

Research prototype for education and decision support only. Not a diagnostic device and not a substitute for formal diagnostic imaging or clinical judgement.

Multimodal AI Hackathon 2026 · Track A: Cardiovascular Risk Visualization & Prediction

Inspiration

Cardiovascular risk is usually handed over as one bare percentage. That tells a clinician or a patient how much risk, but not where it sits or why. We wanted a tool where the risk lives on an anatomical heart you can turn around, and where every number can be questioned.

What it does

CardioLens 3D predicts overall CAD and LAD / LCX / RCA stenosis probability from 43 routine clinical inputs, then shows the result on an interactive 3D heart.

Prediction: four models on the Z-Alizadeh Sani data set (UCI #411, n = 303). Pooled out-of-fold ROC-AUC: CAD 0.93, LAD 0.83, RCA 0.73, LCX 0.70.
3D heart: a real BodyParts3D atlas (12 peelable layers, 54,487 triangles). The three coronary arteries are coloured by probability (teal → amber → red, always with text and an icon). Rotate, zoom, click a vessel to fly to it.
Explanation: exact SHAP values grouped by clinical domain, what-if curves with out-of-range hatching, "n of 100 patients like this", and an in-app model card.
Scan reading room: open a CT (NIfTI or DICOM) or a cardiac cine MRI, decoded in the browser tab and never uploaded. From the MRI it measures left-ventricular volumes and ejection fraction and shows the beating left ventricle inside the heart.
Agent-native: 13 WebMCP tools let an AI agent drive the page. It only ever receives numbers. export_findings is the only tool that can move anything off the page, and it waits for a human to approve.
Runs without a GPU, in one self-contained file, including an offline build.

flowchart LR
    A[Clinical inputs<br/>43 features] --> B[Logistic-regression models<br/>CAD · LAD · LCX · RCA]
    B --> C[Cap: P vessel ≤ P CAD]
    C --> D[Dashboard<br/>probabilities + exact SHAP]
    C --> E[3D heart<br/>vessels coloured by probability]
    F[CT / cine MRI<br/>decoded in the tab] --> G[Measurements<br/>calcium candidates · EF · volumes]
    G --> E
    G -. MRI EF as echo-EF input .-> A
    H[AI agent<br/>WebMCP, 13 tools] -->|numbers only| D
    H -->|numbers only| G
    G --> I{Human approval}
    I -->|approved| J[JSON report]
    I -->|denied| K[Nothing leaves the page]

Plain-text version of the flowchart

How we built it

Models (Python): 7 model families compared with 5×5 repeated stratified nested cross-validation, preprocessing inside the folds, outcome columns (Cath, LAD, LCX, RCA) never used as inputs. We exported logistic regressions, so SHAP is exact and the model runs in the browser.
Engine: a dependency-free JavaScript engine that reproduces the scikit-learn models (60 parity cases, max difference 5.6e-16).
3D: three.js with a BodyParts3D heart, vessels driven directly by the model outputs, adaptive resolution for machines without a GPU.
Reading room: our own NIfTI and DICOM readers, a calcium-candidate finder, and a classical contour tracer for the cine MRI (JavaScript, checked against a Python reference to 1e-6).
Agent layer: WebMCP tools with a payload guard on every result, a live privacy meter and a human-approval dialog.
Optional backend: FastAPI serving the same maths, cross-checked against the sklearn pipelines at start-up.
Quality: 462 automated checks across seven suites, including real-Chromium tests.

Challenges we ran into

Keeping the story consistent. Separate vessel models can say a vessel is more likely than CAD itself (it happened for 19% of patients). We cap each vessel at P(CAD) and flag it on screen.
Honest 3D. Clinical features are not anatomical coordinates, so SHAP is never drawn onto the heart, and the heart is labelled a generic atlas.
Weak targets. LCX (0.70) and RCA (0.73) are only modestly predictable. We kept that visible rather than hiding it.
A privacy boundary for agents. Tool results had to be numbers only, so we built a guard and a meter instead of trusting good intentions.
MRI without a deep model. Tracing the left ventricle with classical methods meant dealing with papillary muscles, unreliable slice locations and varying heart-rate tags.
Smooth 3D with no GPU, on 54k triangles.

Accomplishments that we're proud of

Cross-validated CAD ROC-AUC of 0.93, with a validation protocol we can defend and a model comparison that shows where linear models are good enough.
A heart whose colours are the model outputs, so the picture and the numbers cannot disagree.
A scan-to-model path: MRI ejection fraction (41.0% on the built-in sample) measured in the browser and shown as a beating ventricle in the 3D heart.
An agent interface that cannot leak a scan, with a human approval gate.
462 / 462 automated checks passing, and limits stated on screen and in the video.

What we learned

With 303 patients, a well-tuned linear model plus exact explanations beats a fancier model you cannot explain.
Calibration, leakage guards and coherence between outputs matter as much as AUC.
Saying what the system cannot do (generic atlas, no external validation, unvalidated MRI estimate) makes it more credible, not less.
Privacy is easiest to keep as an architectural rule, not a policy.

What's next for CardioLens

External validation on other centres and cohorts.
A patient-specific heart from whole-heart segmentation instead of a generic atlas.
Validated MRI measurements against manual contours, and tests on real DICOM CT series.
Wire scan findings into the risk model, with proper validation.
Test on Safari, Firefox and real GPUs.
More clinical features, models and anatomical structures through the existing extension points.

Built with

Python · scikit-learn · SHAP · JavaScript · three.js · WebGL · WebMCP · FastAPI · BodyParts3D · Sunnybrook Cardiac Data · UCI Z-Alizadeh Sani

Credits and licences

Data: Extension of Z-Alizadeh Sani Dataset, UCI #411 (CC BY 4.0), Alizadehsani, Roshanzamir & Sani (2013), https://doi.org/10.24432/C5461K
Cardiac MRI sample: Sunnybrook Cardiac Data, CC0 (Radau et al., 2009)
WebMCP (MIT), three.js (MIT), Faraday (MIT) as design inspiration