CardioViz3D

From Clinical Biomarkers to 3D Hemodynamics: AI-Powered Coronary Risk Visualization.

Comment

Inspiration

Cardiovascular diseases (CVDs) remain the world’s leading cause of mortality, claiming nearly 18 million lives each year. Among these, Coronary Artery Disease (CAD)—particularly acute stenosis of the Left Anterior Descending (LAD) artery, clinically known as the "Widowmaker"—requires rapid, decisive clinical triaging. Yet, when we spoke with clinicians and examined contemporary computer-aided diagnostic tools, we discovered a glaring "black-box" disconnect: Most AI models output isolated, abstract risk percentages (e.g., "78% CAD Risk") without anatomical context. Interventional cardiologists think spatially in terms of 3D vascular anatomy, angiographic catheterization views (AP, RAO, LAO), and luminal occlusion percentages. Patients leaving cardiology consultations are often overwhelmed by abstract lab numbers without understanding how actionable lifestyle or therapeutic interventions directly affect their heart. We built CardioViz3D to bridge the gap between high-precision predictive machine learning, game-theoretic explainable AI (XAI), and real-time 3D spatial anatomy. Our vision was to give clinicians and patients a living, interactive cardiac digital twin that explains what the risk is, where the lesion is located, why the algorithm flagged it, and how targeted interventions can reverse it.

What it does

CardioViz3D is a medical-grade 3D cardiovascular risk visualization and clinical decision support workstation: Multi-Territory Stenosis Prediction: Ingests 54 patient biomarkers (demographics, history, vitals, ECG, echocardiography, and laboratory panels) to simultaneously predict gold-standard catheterization-verified stenosis probabilities across four targets: Overall CAD ($\ge 50%$ stenosis in $\ge 1$ epicardial vessel) LAD (Left Anterior Descending): High-acuity anterior wall & septal ischemia LCX (Left Circumflex): Lateral and posterior ventricular territory RCA (Right Coronary Artery): Inferior wall and conduction node perfusion Interactive 3D Anatomical Cardiac Digital Twin: Features a high-fidelity 3D human heart compiled from 129 Foundational Model of Anatomy (FMA) meshes. The heart beats dynamically in real time synchronized to the patient's actual resting pulse rate (BPM), with interactive myocardial opacity controls. Hemodynamic Stenosis Heatmapping: Vessel branches dynamically illuminate in real time based on localized predicted risk—transitioning from emerald green (patent) to amber warning ($>40%$) and arterial crimson glow ($>70%$ critical stenosis). Isolated Vessel & Luminal Cutaway (IVUS Simulation): Clinicians can isolate any coronary branch with 360° studio lighting, rotate between standard angiographic fluoroscopy presets (Anterior AP, RAO, LAO), and inspect an interactive intravascular cross-section demonstrating atheroma plaque buildup against the clinical 50% ischemic threshold. Game-Theoretic Explainable AI (TreeSHAP): Unpacks the model's reasoning through real-time local feature attributions, categorizing the top 15 patient-specific biomarkers into risk-promoting factors (e.g., Regional RWMA, Typical Angina, elevated LDL) and protective factors (e.g., preserved Ejection Fraction, elevated HDL) relative to the population baseline. Counterfactual "What-If" Simulator: Enables live intervention modeling (e.g., reducing systolic blood pressure, smoking cessation, statin therapy, improving ejection fraction) to compute real-time risk reduction deltas before making treatment decisions. Clinical Persona Presets & Dossier Export: Offers one-click loading of archetypal diagnostic personas (High-Risk Multi-Vessel, Isolated LAD Widowmaker, Borderline RCA, Young Low-Risk) and generates downloadable, print-ready clinical diagnostic PDF dossiers.

How we built it

CardioViz3D was architected from the ground up as a decoupled, high-performance full-stack medical intelligence workstation:

  1. Data Engineering & Clinical Cohort Trained on the Extended Z-Alizadeh Sani Cardiovascular Dataset (303 consecutive catheterization-verified angiography patients across 54 clinical attributes). Implemented robust clinical preprocessing pipelines with scikit-learn and imbalanced-learn: median imputation for laboratory values, mode imputation for categorical signs, ordinal encoding, and standard scaling. Prevented target leakage by isolating vessel-specific catheterization markers and performed SMOTE minority oversampling strictly within cross-validation training folds.
  2. Machine Learning & Calibration Engine Conducted hyperparameter optimization across 100 Optuna trials using RepeatedStratifiedKFold(n_splits=5, n_repeats=3). Evaluated Regularized Logistic Regression, Random Forests, XGBoost, and LightGBM, constructing heterogeneous Stacking Ensembles with L2 meta-learners. Applied Isotonic Probability Calibration (CalibratedClassifierCV) to ensure output probabilities represent true empirical incidence, minimizing Brier loss. Built a downstream Multi-Output Classifier Chain ($LAD \rightarrow LCX \rightarrow RCA$) to model true anatomical hemodynamic inter-vessel dependencies.
  3. Explainable AI (XAI) Integrated shap.TreeExplainer and analytical linear explainers to compute exact polynomial-time Shapley values ($\phi_i(x)$), delivering instant local biomarker attributions.
  4. 3D Anatomical Pipeline & WebGL Rendering Normalized and merged 129 separate anatomical Foundational Model of Anatomy (FMA) polygonal .obj meshes into a compact, optimized 8.2MB binary GLTF/GLB asset (heart.glb) using trimesh. Rendered using Three.js, React Three Fiber (@react-three/fiber), and @react-three/drei with custom vertex normal calculations, double-sided PBR MeshStandardMaterial, emissive pulse shaders, and a 6-point studio lighting rig.
  5. Frontend Workstation & API Backend Frontend: Built with React 18, Vite, Tailwind CSS, Framer Motion, Recharts, and Lucide Icons for a clinical-grade dark/light dashboard experience. Backend: Built with FastAPI and Uvicorn, featuring typed pydantic schemas and asynchronous REST endpoints for /predict, /explain, and /whatif.

Challenges we ran into

Compiling 129 FMA Anatomical Meshes: The raw Foundational Model of Anatomy datasets contained dozens of individual OBJ files with divergent scales, non-manifold geometry, inverted surface normals, and uncentered origin points. We wrote a custom Python geometry pipeline (merge_objs.py) using trimesh to normalize coordinates, recompute smooth vertex normals, and preserve distinct anatomical node hierarchies for selective shader illumination. Preventing Dark / Pitch-Black Shading in WebGL: Early Three.js renders suffered from severe black shading when viewing complex coronary crevices from oblique angles. We solved this by developing a specialized multi-directional studio lighting rig (Ambient + Hemisphere + Front-Key + Fill + Rim + Center Point) and enforcing THREE.DoubleSide PBR rendering with calibrated roughness and emissive baseline glow. Calibrating Tree Ensembles: Tree boosting algorithms like XGBoost and LightGBM naturally push predictions toward 0 and 1, distorting posterior probabilities. Because clinical decisions require dependable risk estimates, we integrated 5-fold Isotonic Calibration (CalibratedClassifierCV), successfully driving Brier scores down to 0.0812 for CAD. Sub-second SHAP Extraction from Stacking Pipelines: Unwrapping complex stacking ensembles and calibrated classifier pipelines on the fly to calculate SHAP values initially incurred noticeable latency. We optimized our model extraction to isolate the primary tree estimators and pre-cache background feature structures, achieving near-instantaneous XAI waterfall rendering.

Accomplishments that we're proud of

High Diagnostic Accuracy: Achieved an ROC-AUC of 0.9401 (89.2% Accuracy, 91.0% Precision, 0.0812 Brier Score) on an independent $20%$ holdout test set for overall CAD, and 0.8793 ROC-AUC for the high-mortality LAD territory. Living 3D Digital Twin: Successfully brought medical anatomy to life in the browser at 60 FPS, syncing the cardiac beating rhythm to the patient's real pulse rate while color-mapping multi-vessel stenosis risk. Angiographic Fluoroscopy Views: Recreated authentic clinical catheterization lab views (Anterior AP, RAO, LAO) paired with simulated IVUS luminal cross-sections. True Counterfactual Interactivity: Built an intuitive "What-If" simulator that provides clear, visual reinforcement of how medical and lifestyle interventions directly mitigate coronary disease risk.

What we learned

Calibration Trumps Raw Accuracy in Medicine: In clinical decision support, an uncalibrated 95% accuracy model that outputs extreme, distorted probabilities can be dangerous. Proper calibration (minimizing Brier score) is paramount for clinical adoption. Spatial Context Builds Trust: Presenting predictive machine learning through interactive 3D anatomy combined with SHAP explanations transforms AI from a suspicious "black box" into an intuitive, trusted clinical ally. WebGL Performance Optimization: Efficiently rendering detailed biological structures in real time requires careful normal recomputation, material memoization, and selective branch pattern matching.

What's next for CardioViz3D

Patient-Specific DICOM / CCTA Mesh Reconstruction: Expanding the 3D pipeline to automatically convert a patient's own Coronary CT Angiography (CCTA) DICOM scans into personalized 3D meshes using neural surface reconstruction. Computational Fluid Dynamics (Virtual FFR-CT): Integrating lattice Boltzmann or Navier-Stokes CFD simulations to compute virtual Fractional Flow Reserve ($FFR_{CT}$) across stenotic lesions. EHR & FHIR / HL7 Interoperability: Enabling automated biomarker population via SMART-on-FHIR integrations with hospital EHR systems (Epic, Cerner). Clinical Validation Studies: Initiating multi-center prospective validation partnerships with university cardiology departments to benchmark CardioViz3D's utility in pre-catheterization triage.

Built With

Share this project:

Updates

Submission history