DermaScan homepage
skin picture upload
Skin disease prediction
home remedies
skin health scores
DermaScan system architecture
AI ML architecture and flow

DermaScan

Executive Summary

DermaScan is a production-oriented, AI-driven dermatology platform that converts commodity camera inputs into clinically-informed skin health assessments and personalized care plans. The system fuses state-of-the-art computer-vision pipelines, efficient on-device inference, explainable deep learning, and large-language-model (LLM) reasoning to deliver low-latency, auditable skin-disease detection and long-term progress tracking. The architecture is designed for regulatory readiness, robustness to heterogeneous capture conditions, and incremental learning in production.

Problem Statement & Motivation

Access to timely, affordable dermatology triage is limited worldwide. Manual diagnosis requires specialist time, clinical infrastructure, and often multiple consultations. DermaScan addresses this by providing an automated, clinically-aware triage and monitoring stack that augments dermatologists' capacity while preserving a strict human-in-the-loop escalation pathway for higher-risk cases.

System Overview

DermaScan ingest pipeline → Preprocessing & Quality Filter → Ensemble CNN-based inference → Multimodal fusion (image + metadata) → Explainable report generation (LLM-assisted) → User dashboard + Clinician escalation.

Key design goals:

High precision recall trade-offs tuned for triage (minimize false negatives).
Explainability and traceability (per-prediction saliency maps + textual rationales).
Low-latency edge/near-edge inference with cloud fallback.
Continuous learning with privacy-preserving data collection.

Core Technical Components

1. Image Ingestion & Preprocessing

Face/region detection via a lightweight MTCNN-inspired module; ROI extraction with landmark alignment.
Photometric normalization: adaptive histogram equalization, color constancy correction, illumination-invariant preprocessing to mitigate capture variation.
Automated capture-quality checks (focus, illumination, occlusion) with heuristic+ML filters to reject poor-input frames.

2. Data & Labeling Strategy

Training on an aggregated, de-identified dataset composed of clinical images, crowd-sourced captures, and curated dermatology corpora — annotated with ICD-like labels and lesion-level masks where available.
Annotation workflows with multi-rater consensus, label harmonization, and hierarchical taxonomies (disease vs. symptom vs. artifact).
Class-imbalance mitigation using stratified sampling, focal loss, and synthetic augmentation pipelines (photometric transforms, GAN-based lesion augmentation).

3. Model Architecture & Training

Primary detectors: ensemble of CNN backbones (EfficientNet / ResNet variants) fine-tuned with transfer learning.
Multi-task heads for: (a) disease classification, (b) lesion segmentation, (c) severity regression (continuous score).
Regularization & robustness: label smoothing, mixup, CutMix, stochastic depth, and adversarial augmentation.
Uncertainty quantification using Monte Carlo dropout + deep ensembling; post-hoc calibration with isotonic regression.
Explainability: Grad-CAM / integrated gradients for visual saliency; concept activation testing for higher-level attributions.

4. Multimodal Reasoning & Personalization

LLM integration (Gemini 2.5 Flash) used as a reasoning and synthesis layer—not for primary diagnosis—with retrieval-augmented generation (RAG) over a curated knowledge base (evidence-backed references, ingredient lists, contraindications).
Inputs to LLM: structured model outputs, patient metadata (age, skin type, allergies), and historical trend embeddings.
Personalized regimen generation via template-conditioned prompting and constraint-checking (e.g., avoid ingredients conflicting with allergies).

5. Model Serving & Inference Optimization

Dual-mode inference:
- On-device: quantized TensorFlow Lite / ONNX runtime models for instant Glow Score and basic alerts.
- Cloud/edge: containerized TF-Serving / TorchServe instances for full ensemble inference and high-confidence reports.
Performance optimizations: dynamic batching, mixed-precision on GPUs, model pruning, and knowledge distillation to smaller student models for mobile deployment.

6. MLOps & CI/CD

Versioned datasets and model artifacts using DVC; training pipelines orchestrated by Airflow/Prefect.
Continuous evaluation on holdout and synthetic testbeds (lighting, angles, Fitzpatrick distribution).
Canary deployment with automated rollback on metric drift; telemetry collected to Prometheus and surfaced in Grafana.

7. Privacy, Compliance & Security

Data-at-rest encryption, zero-knowledge identifiers, role-based access control.
Pseudonymization and opt-in consent flows; plans for HIPAA/GDPR readiness (audit logs, data retention policies).
Differential-privacy-aware aggregation for any population-level analytics; optional federated learning to adapt models without centralizing raw images.

Product Features (Technical)

Automated Skin Disease Detection: Multi-label classification and segmentation for acne, eczema, dermatitis, fungal infections, pigmentary disorders, and suspicious lesions requiring escalation.
Glow Score (Composite Metric): Multi-factor index combining texture metrics, symmetry, pigmentation variance, and inflammation regression (normalized, trend-aware).
Personalized Regimen Engine: LLM-driven, constraint-aware output combining active ingredients, routine scheduling, evidence citations, and contraindication checks.
Progress Dashboard: Time-series analytics with change-point detection, per-metric trend decomposition, and cohort benchmarking.
Explainability Toolkit: Visual saliency overlays, textual justification, and per-prediction uncertainty/confidence bands.
Clinician Integration: Audit-ready reports, DICOM-like export, one-click escalation to teleconsultation queue with annotated images and LLM-synthesized clinical summary.

Evaluation & Validation

Validation protocol: stratified k-fold cross-validation with clinically-sourced holdout sets.
Evaluation metrics: AUC-ROC, F1 (per-class), sensitivity at fixed specificity thresholds, mean IoU for segmentation, calibration error for confidence estimates.
Human-in-the-loop validation: clinician review of edge cases and post-deployment sampling for bias detection across Fitzpatrick scales and demographic cohorts.

Deployment & Scalability

Frontend: React + Next.js, Tailwind, Framer Motion for production-grade UX; server-side rendering for SEO and first-paint performance.
Backend: Flask microservices for control-plane; gRPC for high-throughput model requests; MongoDB for user metadata + time-series storage for scan history.
Infra: Dockerized services orchestrated with Kubernetes; autoscaling model-serving clusters; object storage for compressed artifacts.
Observability: request tracing, latency SLOs, model-performance dashboards, and A/B experiments for continuous improvement.

Challenges & Solutions

Capture Variability: mitigated via adaptive preprocessing, synthetic augmentation, and capture-quality gating.
Class Imbalance & Rare Conditions: addressed with data-synthesis, focal loss, and active learning to prioritize clinician labeling.
Latency vs. Accuracy Trade-offs: solved using hybrid inference (on-device fast path + cloud fallback) and student-teacher distillation.
Fairness Across Skin Types: dataset balancing, per-Fitzpatrick validation, and model calibration to ensure equitable sensitivity and specificity.

Achievements & Impact

Demonstrated an integrated pipeline that produces clinically-actionable triage reports with explainable evidence traces.
Built end-to-end stack from capture to clinician escalation ready for hackathon demo and rapid prototyping toward telehealth integration.
Designed for safe deployment: auditability, consent-first data collection, and privacy-preserving update mechanisms.

Future Roadmap (Research & Product)

Federated fine-tuning across clinic partners to improve generalizability without centralizing PHI.
Continuous active learning loop: surface high-uncertainty cases to dermatologists for labeling via an operator console.
Regulatory pathway planning: dataset curation and controlled clinical validation to pursue medical-device classification where applicable.
AR-guided capture for standardized imaging, and wearable integration (hydration/UV telemetry) for multimodal health signals.

Why DermaScan Is Hackathon-Ready

Modular architecture: clear separation of perception, reasoning, and orchestration layers enables parallel team workstreams (frontend, ML, MLOps).
Demonstrable MVP surface: live demo (mobile capture → instant Glow Score → LLM-generated plan) with clinician escalation flow.
Strong technical differentiation: explainable ensemble models + LLM-enabled personalized reasoning + production-grade serving/monitoring pipeline.