CardioVision 3D

AI-powered 3D cardiovascular twin predicting multi-vessel stenosis (LAD/LCX/RCA) with explainable SHAP insights, real-time What-If simulations, and automated clinical report analysis.

Comment

💡 Inspiration

Cardiovascular disease (CVD) remains the leading cause of mortality worldwide. In clinical cardiology, traditional statistical algorithms output abstract risk percentages (e.g., "78% risk"), but numbers alone fail to provide cardiologists and clinical teams with spatial anatomical intuition: which specific coronary artery is occluded, where is the ischemia, and how severe is the stenosis?

Furthermore, clinical intake is bogged down by manual transcription of dense laboratory reports, lipid profiles, and ECG records. We built CardioVision 3D to revolutionize cardiovascular triage by bridging multimodal clinical document extraction, calibrated machine learning, and an interactive 3D anatomical digital twin.

🫀 What It Does

CardioVision 3D is an interactive, clinical-grade cardiovascular decision support system that predicts, explains, and visualizes multi-vessel coronary artery disease:

  1. Multimodal Document Triage: Uses Google Gemini Multimodal AI to instantly extract physiological biomarkers from uploaded patient lab reports (lipid panels, blood chemistry, ECG observations) with zero manual data entry.
  2. Multi-Vessel Stenosis Classification: Rather than a simple binary risk score, our engine predicts overall Coronary Artery Disease (CAD) and vessel-specific stenosis ($\ge 50\%$) across the three primary coronary arteries:
    • LAD (Left Anterior Descending — the "widow-maker")
    • LCX (Left Circumflex Artery)
    • RCA (Right Coronary Artery)
  3. Interactive 3D Anatomical Twin: Maps predicted risk directly onto a photorealistic 3D human heart and coronary tree in real time (WebGL/Three.js), using dynamic emissive pulse shaders ($Green = Normal$, $Amber = Moderate\ Risk$, $Pulsing\ Red = Critical\ Stenosis$).
  4. SHAP Explainability Engine: Eliminates the "black-box" dilemma using TreeExplainer feature attributions to show clinicians exactly why a risk was assigned (e.g., ST-depression, RWMA on echocardiogram, LDL cholesterol levels).
  5. Real-Time "What-If" Clinical Simulator: Allows cardiologists to simulate pharmacological and lifestyle interventions (e.g., statin therapy, BP reduction) with sub-30ms re-inference, dynamically cooling the 3D heart vessels from red to green before the patient's eyes.
  6. Hospital-Ready PDF Reports & Cloud Sync: Exports comprehensive cardiology clinical summaries with disclaimers and synchronizes patient records seamlessly with Supabase PostgreSQL.

🛠️ How We Built It

CardioVision 3D is architected across three synchronized tiers:

  • Frontend & 3D Visualization: Built with React 18, Vite, React Three Fiber (R3F), Three.js, and custom GLSL glow shaders. Styled with modern clinical glassmorphism and Lucide icons.
  • AI & Explainability Backend: Powered by Python 3, FastAPI, and Uvicorn. Employs XGBoost and Scikit-learn calibrated ensemble models alongside SHAP (SHapley Additive exPlanations) for instant local feature attribution.
  • Multimodal AI & Cloud Infrastructure: Google Gemini Multimodal API for clinical OCR and document parsing; Supabase (PostgreSQL & Storage) for relational patient triage, telemetry, and encrypted audit trails.

🔬 Machine Learning & Anti-Leakage Compliance

Our models are trained and rigorously validated on the Z-Alizadeh Sani Cardiovascular Dataset (UCI ML Repository ID: 411; 303 clinical patients, 55 non-invasive physiological biomarkers).

Strict Anti-Leakage Enforcement

To ensure scientific integrity and eliminate data leakage, target catheterization variables (Cath, LAD, LCX, RCA) were strictly excluded from model feature sets. Models rely strictly on pre-catheterization non-invasive clinical findings.

Model Performance Metrics (5-Fold Cross-Validation)

  • Overall CAD Diagnosis: 0.912 ROC-AUC | 85.2% Accuracy | 0.894 F1-Score
  • LAD Artery Stenosis: 0.844 ROC-AUC | 79.2% Accuracy | 0.817 F1-Score
  • LCX Artery Stenosis: 0.731 ROC-AUC | 70.6% Accuracy
  • RCA Artery Stenosis: 0.721 ROC-AUC | 67.3% Accuracy
  • Inference Latency: $< 30\text{ ms}$ for real-time slider responsiveness.

🧗 Challenges We Faced

  1. Mapping ML Probabilities to 3D Geometry: Translating tabular stenosis probabilities into smooth, anatomically faithful glowing artery shaders in Three.js required custom GLSL shader uniforms and vertex hierarchy management.
  2. Sub-30ms Explainability: SHAP computations are typically computationally heavy. We pre-computed base margins and optimized TreeExplainer caching to allow instant re-computation during live "What-If" slider drags.
  3. Medical Document Variety: Structuring unstructured lab reports into normalized medical parameters required precise prompt engineering and structured JSON output schema enforcement with Gemini.

🏆 Accomplishments That We're Proud Of

  • Creating a true spatial bridge between complex machine learning and intuitive 3D cardiology.
  • Achieving 0.912 ROC-AUC on clinical CAD prediction while maintaining full explainability.
  • Delivering a lightning-fast clinical workflow where a doctor can upload a PDF, inspect a 3D heart, run a treatment simulation, and print a signed summary in under 60 seconds.

📚 What We Learned

  • How to integrate React Three Fiber with high-frequency state updates from ML inference without dropping below 60 FPS.
  • The vital role of model explainability (SHAP) in clinical decision support—clinicians do not adopt models they cannot interpret.
  • Best practices in medical multimodal document parsing and strict anti-leakage data pipelines.

🚀 What's Next for CardioVision 3D

  • Direct DICOM / Coronary CT Angiography (CCTA) volumetric mesh reconstruction.
  • Longitudinal patient progression tracking across multi-year checkups.
  • Mobile companion application for patient adherence and medication tracking.

Built With

Share this project:

Updates

Submission history