SkinTriage — AI Dermoscopy Risk Screening

AI-powered dermoscopy triage that flags high-risk skin lesions with a Grad-CAM heatmap explaining why — 0.911 AUROC on HAM10000, built as a research prototype, not a diagnostic device.

Comment

Inspiration

Skin cancer is one of the most common cancers worldwide, and early detection is the single biggest factor in survival — but dermatological expertise isn't evenly distributed, and many people never get a lesion checked until it's advanced. We wanted to explore whether an accessible, explainable AI triage tool could help close that gap: not by replacing a dermatologist, but by giving people (and under-resourced clinics) a fast, transparent first read on whether a lesion looks concerning enough to seek care.

What it does

Skin Lesion Triage AI takes a dermoscopy image and classifies it as benign or malignant, then overlays a Grad-CAM heatmap showing exactly which regions of the lesion drove that prediction — so the output is never a black-box number, it's an explainable, inspectable decision. The tool runs through a simple Streamlit web app: upload an image, get a calibrated probability, a risk flag, and a visual explanation in seconds.

How we built it

We trained a binary classifier on the HAM10000 dermoscopy dataset (10,015 images across 7 diagnostic categories, mapped to a binary benign/malignant triage label), using an EfficientNet-B0 backbone via timm.

  • Patient-safe splitting: we grouped by patient/lesion ID (via StratifiedGroupKFold) before splitting into train/val/test (6,991 / 1,504 / 1,520 images) to prevent leakage between splits.
  • Training: PyTorch + AdamW, BCE loss with pos_weight for class imbalance, RandAugment, cosine LR schedule, and label smoothing, iterated across baseline → v2 → v3.
  • Calibration & robustness: test-time augmentation (original + horizontally-flipped logits averaged) and temperature scaling (T = 3.72) to produce well-calibrated probabilities rather than overconfident raw logits.
  • Interpretability: a Grad-CAM engine hooked into the final conv block to generate per-image heatmap overlays.
  • Multimodal experiment: a fusion variant that combines the image branch with tabular metadata (age, sex, lesion site) as a bonus track.
  • Deployment: a Streamlit app wrapping the calibrated model with a curated sample selector for quick demoing.

Results

  • Test AUROC: 0.911 (with TTA + temperature calibration)
  • Sensitivity: 88.7% at a 90%-specificity operating point
  • Multimodal (image + metadata) variant: 0.905 AUROC

Challenges we ran into

  • Avoiding data leakage: HAM10000 has multiple images per lesion/patient, so a naive random split silently leaks the same lesion into both train and test. We had to enforce patient-grouped splitting throughout.
  • Calibration: raw model logits were overconfident; temperature scaling was necessary to make the reported probabilities trustworthy rather than just a rank-ordering.
  • Class imbalance: malignant cases are a minority class in HAM10000, so we tuned pos_weight and the operating threshold (val-tuned, test-reported) rather than using a naive 0.5 cutoff.
  • Interpretability vs. performance trade-off: making sure Grad-CAM overlays stayed faithful to the actual decision region rather than just looking plausible.

What we learned

Calibration and explainability matter as much as raw accuracy for a tool like this — a high-AUROC model that outputs miscalibrated probabilities or an unfaithful heatmap could be actively misleading in a triage context. We also learned firsthand how much careful data-splitting discipline (patient-grouped, not random) matters for any medical imaging task, since it's an easy mistake that quietly inflates every downstream metric.

What's next for Skin Lesion Triage AI

  • Expand beyond binary triage to multi-class lesion typing across all 7 HAM10000 diagnostic categories
  • Validate against external datasets to test generalization beyond HAM10000's population and imaging conditions
  • Explore uncertainty quantification so the tool can flag "not confident enough to triage" cases
  • Formal clinical validation pathway, if pursued beyond the hackathon stage

Disclaimer: This is a hackathon research prototype for the GIBC V2 challenge. It is not a medical device, has not been clinically validated, and must not be used for diagnostic decisions or patient care.

Built With

Share this project:

Updates