DermAssist: the Future of Skinformatics

• 

  DermAssist in action.

Inspiration

The journey of DermAssist began with a clear mission: to harness the latest advancements in AI for the betterment of skin care. Our inspiration was drawn from the diverse needs of individuals across different skin types and colors, recognizing the gap in accessible and accurate skin diagnostics. The collaboration with Google's recently open-sourced SCIN dermatology dataset was a catalyst, providing us with a rich foundation to build upon.

What it does

DermAssist is a cutting-edge dermatology chatbot that utilizes the multimodal AI capabilities of LLaVA-Next and Mistral-7B for comprehensive skin analysis and diagnostics. It's designed to be inclusive, catering to a wide array of skin conditions across diverse skin types. By analyzing images and descriptions of skin conditions, DermAssist offers insights and recommendations, paving the way for early detection and management.

How we built it

The development of DermAssist was a collaborative effort, leveraging the robust features of LLaVA-Next for multimodal understanding and the analytical precision of the Mistral-7B language model. The integration of Google's SCIN dataset allowed for a broad representation of skin conditions. Our approach combined deep learning, computer vision, and natural language processing techniques to create a tool that's both accurate and user-friendly.

Challenges we ran into

A primary challenge in developing DermAssist was generating clinically appropriate and teledermatology qualified question-answer pairs. This required not only a deep understanding of dermatological knowledge across diverse skin types and colors but also a keen awareness of the subtleties involved in telemedical consultations. Balancing the computational intensity of our advanced AI models with the need for instantaneous, accurate responses was another significant hurdle. Ensuring inclusivity and precision in diagnostics, while navigating the complexities of medical language and the specifics of skin conditions, demanded continuous refinement and testing of our model. Achieving a level of interaction that mirrors the nuanced inquiry of a dermatological examination proved to be both challenging and critical to our mission.

Accomplishments that we're proud of

We are immensely proud of creating a tool that democratizes access to dermatology, making it possible for individuals around the world to get early insights into their skin conditions. The successful integration of the SCIN dataset to train a model that is both inclusive and accurate stands out as a significant achievement. Additionally, developing an intuitive interface that simplifies complex diagnostics into understandable insights for users is an accomplishment that resonates with our mission.

What we learned

Throughout this journey, we've gained invaluable insights into the complexities of skin diagnostics and the power of AI to address them. We learned about the importance of dataset diversity for training effective models and the challenges of interpreting medical imagery through AI. The project underscored the significance of interdisciplinary collaboration, bringing together technologists, medical professionals, and researchers to create impactful solutions.

What's next for DermAssist: the Future of Skinformatics

Looking ahead, the future of DermAssist is bright with possibilities. Our roadmap includes expanding the dataset to cover even more conditions, enhancing the model's accuracy through advanced AI techniques, and integrating DermAssist into healthcare systems for broader accessibility. We're committed to fostering community collaboration, inviting contributions that can propel DermAssist forward. The journey of revolutionizing dermatology with AI is just beginning, and we're excited for what lies ahead.

Built With

• google-scin-dataset
• llava-next
• mistral-7b