DermaScan-AI

Inspiration

Skin cancer is among the most common cancers globally, yet early detection dramatically increases survival rates. However, access to specialized dermatologists can be constrained by geographic barriers and long appointment wait times. We built DermaScan AI to serve as an accessible, rapid triage and screening assistant that bridges this gap—empowering users and general practitioners with instant preliminary assessments while keeping the diagnostic rationale transparent through explainable AI.

What it does

DermaScan AI is a full-stack, cross-platform screening system that:

• Analyzes dermatoscopic skin lesion images and classifies them across 7 clinical diagnostic categories (aligned with the HAM10000 benchmark: Melanoma, Basal Cell Carcinoma, Actinic Keratosis, Melanocytic Nevi, Benign Keratosis, Dermatofibroma, and Vascular Lesions).
• Triages predictions into actionable Clinical Risk Tiers (High, Moderate, Low).
• Delivers real-time Grad-CAM visual explainability, overlaying activation heatmaps so users and clinicians can see the exact morphological features driving the model's prediction.
• Maintains local, privacy-first scan histories for convenient tracking without persisting sensitive health images to external databases.

How we built it

• Deep Learning Core: Fine-tuned an EfficientNet-B0 architecture on dermoscopic datasets and integrated a gradient-weighted class activation mapping (Grad-CAM) pipeline for saliency heatmaps.
• Inference Optimization: Converted the pipeline to ONNX Runtime, stripping away heavy PyTorch runtime overhead to compress the operational memory footprint to sub-120 MB for lightweight cloud hosting.
• Backend Service: Built a modular, asynchronous FastAPI service containerized with Docker and deployed on Render.
• Frontend & Client: Built a responsive cross-platform web and mobile interface using React Native / Expo Web, deployed seamlessly to Vercel with client-side blob normalization for smooth image transmission.

Challenges we ran into

• Cloud Memory Constraints: Standard PyTorch backends easily exceeded standard free/starter tier RAM limits (512 MB). We overcame this by migrating the inference pipeline to ONNX Runtime and stripping redundant pre-processing dependencies.
• Cross-Platform File Serialization: Handling file uploads consistently across mobile camera pickers and desktop browsers caused multipart boundary errors; we resolved this by implementing unified binary blob conversion on the client before network dispatch.
• Transparent Healthcare AI: Ensuring the tool doesn't act as an uninterpretable "black box" required fine-tuning our Grad-CAM layer to generate sharp, clinically meaningful feature attributions.

Accomplishments that we're proud of

• Achieving end-to-end inference and Grad-CAM generation in under 2 seconds.
• Building a truly unified codebase that performs consistently on mobile devices and desktop browsers.
• Delivering a functional, production-deployed system that balances high classification performance with visual transparency.

What we learned

• Best practices in ONNX graph optimization and operationalizing deep learning models on resource-constrained servers.
• The critical role of Explainable AI (XAI) in medical diagnostics to foster user trust and prevent misinterpretation.

What's next for DermaScan AI

• Integrating multi-spectral image analysis and ABCDE dermoscopy metric evaluation.
• Adding offline-first, on-device neural network inference directly inside the mobile app via ONNX Runtime Mobile / TensorFlow Lite.

Built With

• computer-vision
• deep-learning
• docker
• efficientnet
• expo.io
• fastapi
• onnx
• onnxruntime
• opencv
• pillow
• python
• react
• react-native
• render
• vercel