DermaAI

Instant AI-powered skin analysis​

Comment

DermaAI

Inspiration

Many people struggle to understand their skin conditions without seeing a doctor. Private healthcare is expensive, making professional diagnosis less accessible, particularly for underrepresented communities who face additional barriers to quality healthcare. We wanted to create a solution that provides instant, AI-powered skin analysis to help people get preliminary insights into their skin health, empowering everyone regardless of their financial situation or access to healthcare resources.

What it does

Derma AI provides:

  • Instant AI-powered skin analysis - Users can upload an image and describe their symptoms
  • Top 5 possible conditions - The system analyzes the image and provides a ranked list of potential skin conditions
  • Context-aware recommendations - Personalized follow-up suggestions based on the analysis
  • Dual AI processing - Combines vision AI (SwinV2.5) for image analysis with language AI (Gemini) for natural explanations

User Flow:

  1. Upload image + describe symptoms
  2. SwinV2.5 processes image
  3. Gemini generates explanation
  4. User receives analysis
  5. Context-aware Gemini generates follow-ups

How we built it

Tech Stack:

Frontend:

  • React 18
  • Vite
  • Tailwind CSS
  • Framer Motion

Backend:

  • Flask (Python)
  • REST API
  • File management

AI/ML:

  • SwinV2.5 (Ultralytics)
  • Google Gemini API
  • PyTorch

Architecture:

  • Frontend: React + Vite for fast, responsive UI with smooth animations (Framer Motion)
  • Backend: Flask API for clean integration
  • AI Pipeline: SwinV2.5 for image processing + Gemini for natural language generation
  • Integration: Clean, modular pipeline design

Key Features:

  • Dual AI (Vision + Language) system for accurate, understandable results
  • Context-aware responses that guide users through next steps
  • Top-5 detection ranking for comprehensive analysis
  • Intuitive, accessible UI design with Tailwind CSS for a polished user experience
  • Mock mode for development and testing

Challenges we ran into

  1. Model unavailable → Implemented mock mode for development and testing
  2. AI integration complexity → Adopted modular design for cleaner architecture
  3. Limited Machine Learning Datasets → Applied TTA (Test Time Augmentation) and Data Augmentation techniques
  4. Context integration → Enhanced prompts to maintain context between vision and language models
  5. Model accuracy limitations → Time constraints prevented optimal training; the model isn't as accurate as it could be, but could easily be improved within a week or month of dedicated training time

Accomplishments that we're proud of

  • Successfully integrated two different AI models (vision and language) into a seamless, functional pipeline that delivers real-time results
  • Created an intuitive, visually appealing interface designed for accessibility - making dermatology insights available to non-technical users and those without medical backgrounds
  • Implemented robust error handling with mock mode fallback, ensuring the system works reliably
  • Built a system that directly addresses healthcare equity by democratizing access to skin condition information, empowering underrepresented communities who face barriers to dermatological care
  • Developed an innovative dual-AI approach that combines cutting-edge computer vision with natural language processing for an original solution to skin analysis

What we learned

  • How to effectively integrate multiple AI models in a production pipeline while maintaining functionality and user experience
  • The importance of modular design when working with complex AI systems
  • Techniques for handling limited datasets through augmentation - crucial for developing inclusive AI that works across diverse skin types and conditions
  • The value of mock modes and fallbacks in AI development for robust, reliable applications
  • How to create context-aware AI responses that feel natural and helpful, prioritizing user empowerment over technical jargon
  • Designing with accessibility in mind from the start creates better products for everyone

What's next for DermaAI

  • Self-Supervised Learning with Pseudo-Labelling - Improve model accuracy with advanced training techniques
  • Mobile app - Make the service accessible on-the-go
  • Telemedicine integration - Connect users with healthcare professionals for confirmed diagnoses
  • Multi-language support - Expand accessibility to non-English speaking users

Built With

Share this project:

Updates