DermaInsight

• 

  Example of DermAI Output

• 

  The user provides the DermiInsight Program with the input of text and picture of his skin

Inspiration

I am someone who suffers with skin issues and have lots of difficulty getting dermatology appointments. And when I do they are over 6 months apart. This is a real issue with the healthcare system in Canada, and is also what was mines as well as my group members inspiration for the project.

What it does

This app, DermaInsight works by having the user give input in the form of text or image. The app then processes thought multiple Vision Transformers, and a RAG model built from the ground up to give a strong and reputable diagnosis. This is certified information from reputable papers and books.

How we built it

The ViT were from hugging face, RAG was made by pinecone and gemini as the core frameworks.

Challenges we ran into

Multiple issues connecting the outputs of the vision transformers, as well as hooking up with the frontend

Accomplishments that we're proud of

Created a RAG system from scratch, created a multi model AI system, managed to stay on campus for the entire duration of the hackathon.

What we learned

We genuinely learnt so much from this idea and we are so happy we managed to reach a minimum viable product

What's next for DermaInsight

add more dermatology literature and spend some much more needed time on developing it further

Built With

• bge
• fastapi
• gemini-api
• huggingface
• pillow
• pinecone
• python
• requests
• torch
• uvicorn.-jinja2.-python-multipart.-transformers