SkinGuardian

SkinGuardian: AI-powered app detects skin cancer early on-device with Qualcomm tech. Fast, private, fair across skin types—smart healthcare in your pocket!

Comment 2

SkinGuardian: AI-Powered Skin Cancer Detection

📌 Project Overview

SkinGuardian is an on-device AI application designed for early skin cancer detection, ensuring privacy and accessibility. It leverages Qualcomm AI Hub and Snapdragon processors to process images locally, classifying skin lesions as cancerous or non-cancerous. Our focus on fairness across diverse skin types makes it a groundbreaking healthcare tool.

💡 Inspiration

The project was inspired by the need for early skin cancer detection, which can significantly improve patient outcomes. SkinGuardian ensures accessibility and privacy by processing images directly on the user’s device, eliminating reliance on cloud-based solutions.

⚙️ What It Does

  • Users upload images of skin lesions via a user-friendly Windows app.
  • A fine-tuned BEIT model (from Qualcomm AI Hub) classifies lesions as cancerous or non-cancerous.
  • The app ensures fairness in AI by evaluating performance across different skin types to address potential biases.

🛠️ How We Built It

1️⃣ Dataset Preparation

✅ Used ISIC Archive, a large dermoscopic image dataset for skin cancer detection.
✅ Removed duplicate images to enhance data quality.
✅ Evaluated fairness using Fitzpatrick17k, which labels skin types to measure performance across diverse groups.

2️⃣ Model Selection & Training

✅ Chose BEIT (by Qualcomm AI Hub), a lightweight image classifier ideal for on-device inference.
✅ Fine-tuned it on ISIC for binary classification (cancerous vs. non-cancerous).
✅ Used TensorFlow for training and fine-tuning.

3️⃣ Optimization for On-Device Use

✅ Converted the model to ONNX format for efficient inference.
✅ Optimized using ONNX Runtime for fast execution on Snapdragon devices.

4️⃣ Application Development

✅ Built a Flask webserver to handle requests.
✅ Integrated ONNX Runtime for model inference.
✅ Implemented image preprocessing (resizing, normalization, and tensor conversion).
✅ Created routes for uploading images, running inference, and rendering results.
✅ Designed UI with HTML, CSS, and JavaScript, styled using Tailwind CSS.
✅ Used Jinja2 for dynamic content rendering.
✅ Added Drag & Drop Upload functionality for better UX.

🚧 Challenges We Faced

  1. Dataset Diversity

    • The ISIC dataset had limited demographic diversity (only 2.1% of images had Fitzpatrick skin type labels).
    • Underrepresentation of darker skin tones posed a risk of bias.

  2. Model Optimization

    • Ensuring the model ran efficiently on-device required careful compression and profiling for Snapdragon devices.
    • Iterative testing was necessary to balance accuracy and computational cost.

  3. Fairness in AI

    • Addressing AI biases was challenging due to limited diverse datasets.
    • While Fitzpatrick17k helped, its smaller dataset size limited generalization.
    • Future research could explore synthetic data generation for better representation.

🏆 Accomplishments We're Proud Of

✅ Successfully trained and deployed an AI model for on-device skin cancer detection.
✅ Implemented a fairness evaluation framework using Fitzpatrick17k to address biases in medical AI.
✅ Developed a user-friendly Windows application that ensures privacy-first AI inference.

📚 What We Learned

💡 Dataset quality & diversity play a crucial role in fair AI models.
💡 Model optimization techniques using Qualcomm AI Hub were key to on-device efficiency.
💡 Transparency in AI fairness is critical, especially in medical applications.

🚀 What's Next for SkinGuardian?

🔹 Enhancing Dataset Diversity

  • Integrate additional datasets like PAD-UFES-20 or DDI to improve fairness.
  • Explore federated learning for privacy-preserving model training.

🔹 Advanced AI Techniques

  • Implement style transfer-based data augmentation to synthesize diverse skin types.
  • Improve classification to identify specific skin cancer types beyond binary classification.

🔹 Feature Enhancements

  • Add real-time image capture for instant analysis.
  • Provide detailed AI explanations to help users understand results better.
  • Expand to mobile applications for broader accessibility.

🛠️ Built With

  • Python
  • Flask (Backend Framework)
  • ONNX Runtime (Model Inference)
  • TensorFlow (Model Training)
  • Jinja2 (Template Rendering)
  • Tailwind CSS (Styling)
  • JavaScript (Frontend Enhancements)
  • Qualcomm AI Hub (Model Optimization)
  • Windows (C# & WPF) (Desktop Application UI)

📌 Final Thoughts

SkinGuardian is a privacy-first, AI-powered skin cancer detection tool optimized for Snapdragon devices. By combining efficient AI, fairness in healthcare, and on-device processing, we aim to redefine accessible early skin cancer detection.

🔗 Built for the Qualcomm AI Hackathon 🎯

Built With

Share this project:

Updates