Herbify – AI-Powered Skin Analysis & Personalized Herbal Skincare
Inspiration
Skin conditions are common, but identifying them and finding suitable skincare can be confusing. Most skincare solutions provide general recommendations without considering both the skin condition and individual skin type. Herbal skincare offers natural alternatives, but choosing the right remedy is not always easy. This inspired us to create Herbify, a platform that combines artificial intelligence with herbal skincare to provide personalized and accessible skin insights.
What It Does
Herbify allows users to upload or capture an image of their skin. The system analyzes the image using deep-learning models to identify possible skin conditions such as Acne, Dermatitis, Eczema, Psoriasis, Hyperpigmentation, Rosacea, Fungal Infection, Vitiligo, and Ringworm. It also identifies the user's skin type as Oily, Dry, Sensitive, or Combination.
Based on the detected condition and skin type, Herbify provides personalized herbal remedies and skincare guidance. It also displays prediction confidence and supports English, Hindi, and Gujarati.
How We Built It
The frontend of Herbify was developed using React.js and Vite, while Python and FastAPI were used for the backend. For skin condition classification, we used PyTorch and EfficientNet-B3 with transfer learning. A separate deep-learning model was used for skin-type classification, while Torchvision and Pillow were used for image preprocessing.
The detected condition and skin type are connected to a JSON-based herbal knowledge base to provide suitable recommendations. Gemini API was integrated to enhance explanations and provide multilingual responses.
Challenges We Faced
One of the main challenges was distinguishing between skin conditions with visually similar symptoms. Variations in lighting, camera quality, image angles, and skin appearance also affected model predictions. Another challenge was working with limited and imbalanced datasets.
We experimented with data augmentation, transfer learning, model freezing and unfreezing, and different training configurations to improve model performance. Integrating the AI models, recommendation system, backend, APIs, and frontend into one complete application was another major challenge.
What We Learned
Building Herbify gave us practical experience in developing an end-to-end AI application. We learned about image classification, transfer learning, image preprocessing, model evaluation, FastAPI development, API integration, and connecting machine-learning models with a React application.
The project also helped us understand the importance of data quality, model reliability, responsible AI, and clearly communicating limitations when developing healthcare-related AI solutions.
What's Next
We plan to improve Herbify using larger and more diverse datasets, increase prediction reliability, expand the herbal recommendation knowledge base, and improve the handling of uncertain predictions. Future versions could also include explainable AI, expert-validated skincare information, and support for additional skin conditions.
Herbify – Detect with AI. Heal with Nature.
Log in or sign up for Devpost to join the conversation.