Inspiration
What it does💡 Inspiration
Access to dermatological care can be expensive, time-consuming, and out of reach for many — especially those in rural or underserved communities. We were inspired to build SnapCare to bridge this gap using AI. Our goal: create a tool that gives people a private, fast, and informative first step toward skin health using just their phone.
With the rise in preventable skin conditions and early-stage cancers being missed due to delayed checkups, we saw an opportunity to combine computer vision and natural language generation to offer an impactful solution.
🧠 What We Learned Over the course of this project, we dove deep into:
Transfer learning and image classification with PyTorch
Building scalable and clean frontends using Streamlit
How to integrate AI/ML inference with LLM-driven health advice
The importance of dataset quality for real-world medical applications
Handling user input, file uploads, and output interpretation in a web-based system
We also learned a lot about the ethical limits of AI in healthcare, and how to present results responsibly and accessibly.
🛠️ How We Built It Frontend: Built with Streamlit, allowing users to upload images of their skin conditions quickly and easily.
Model: We trained a convolutional neural network using transfer learning (ResNet-50), fine-tuned on a large dataset of dermatological images with nearly 200 condition classes.
Post-processing: The model’s top predictions were passed to a medical language model (LLM), which generated personalized, plain-language summaries and next-step advice.
System Flow:
User uploads an image.
Image is processed and classified into likely skin conditions.
SnapCare returns the top results and generates a summary with care suggestions.
🧗 Challenges We Faced Dataset issues: Most dermatology datasets are either too small, unbalanced, or not publicly available. We initially tried SD-198 but found the quality and class balance problematic. This affected model performance significantly.
Model accuracy: With nearly 200 classes and limited high-quality data, training a model with high top-1 accuracy was tough. We worked around this using top-k predictions and supplementing with LLM-based interpretation.
Medical responsibility: It was important for us to make clear that SnapCare is not a diagnostic tool. We spent time making sure the UI and advice were framed carefully to avoid overpromising.
Frontend/backend integration: Streamlit was smooth to use, but connecting the image upload, model inference, and LLM interaction required careful threading and state management.
🚀 What’s Next Incorporating confidence scores and visual explainability (e.g., Grad-CAM)
Expanding the dataset using public medical archives or clinical APIs
Adding a symptom-based chatbot for users without images
Exploring partnerships or feedback from medical professionals to refine the tool
Log in or sign up for Devpost to join the conversation.