-
App Demo
Inspiration
The healthcare industry is constantly pressured to provide fast, reliable diagnoses, especially in dermatology, where access to specialists is often limited and expensive. For general practitioners and dermatologists, quickly identifying skin conditions can be challenging due to the vast range of possible issues and visual similarities between them. Patients in many areas experience delays in receiving proper dermatology.
What it does
Aesthetech is a web application that serves as a virtual skincare assistant, providing users with personalized insights and recommendations for skin issues. Here’s how it works:
Image Analysis and Diagnosis: Users upload a selfie, which is analyzed by our convolutional neural network (CNN) to identify potential skin issues. The identified condition is then sent to a LLM for further contextual information and recommendation generation.
Personalized Product Recommendation: Based on the diagnosed skin condition, users are presented with the following:
- Identified Skin Issue: The app displays the identified condition and offers an overview of what it entails.
- Recommended Product and Usage: A specific product recommendation is provided, along with usage instructions tailored to address the user’s skin condition.
- Interactive Options; Learn More: Users can click the "Learn More" button to receive additional details about their diagnosis. This section includes; A brief summary of the skin condition, related links with more in-depth resources, a few alternative products that may also be effective, and the generation of a skincare routine. With these features, Aesthetech offers a comprehensive, user-friendly experience that empowers users to take control of their skincare with trusted recommendations and routines tailored to their unique needs.
How we built it
We used a combination of machine learning and computer vision techniques to build a highly accurate model, along with a user-friendly web interface to ensure accessibility.
Machine Learning: We trained a CNN using TensorFlow on a labeled dataset of skin conditions. Although many available datasets are focused on acne, we sourced a more comprehensive dataset to cover a wider range of conditions. However, an unidentified label called "3" required additional processing, so we manually classified one-third of this data and then used the correct skin blemish classifications to label the rest.
Generative AI Integration: We used the OpenAI API to connect with ChatGPT, which returns context-specific information, recommended treatments, and suggested products based on the skin issue identified.
Frontend Design: The interactive interface was built using web technologies that we learned on the go, enabling users to upload images, view diagnostics, and receive tailored recommendations. We focused on creating a clean and accessible design to make the user experience as seamless as possible.
Challenges we ran into
Dataset Limitations: Finding a comprehensive dataset was challenging, as most available data focused on acne alone. We sourced a larger dataset, but it required extensive cleaning and manual labeling due to unidentified labels.
Demographic Representation: Ensuring that our model was effective across a wide demographic, including various skin tones, genders, and age groups, required careful consideration to avoid bias.
First Hackathon Experience: Since this was our first hackathon, we faced challenges with organization and workflow management. Working as a team and using Git for version control was a learning experience.
Accomplishments that we're proud of
End-to-End Product Completion: Completing a functional web application that integrates machine learning, generative AI, and a user-friendly interface was a significant achievement for our team.
High Accuracy in Classification: We achieved a high level of accuracy in identifying skin conditions, particularly for common issues, making the tool both reliable and practical.
User-Centric Interface: We successfully created an interface that is both accessible and intuitive, despite our team’s initial lack of experience with frontend development.
What we learned
Model Optimization: We learned techniques for optimizing our CNN to handle varying input qualities and learned about strategies for managing data diversity and bias.
Cross-Disciplinary Collaboration: Building AesticAI required coordination across machine learning, web development, and user experience design, which strengthened our teamwork and project management skills.
Frontend Development: As a team new to frontend technologies, we gained valuable skills in building interactive web interfaces and integrating backend functionality.
What's next for Aesthetech
Mobile Compatibility: A high priority for us is to port Aesthetech to a mobile environment, making it accessible for users directly on their phones. This would allow users to receive diagnoses, treatment recommendations, and follow-up guidance on the go, creating a seamless and convenient experience.
User Profiles for Personalized Tracking: We plan to add user profiles, enabling users to view past recommendations, track which treatments have worked for them, and access direct links to recommended products. This feature will enhance personalization and allow users to monitor their progress over time.
Expanded Condition Database: Adding more conditions and sourcing datasets with diverse demographics will allow us to improve diagnostic accuracy across a broader range of skin concerns, ensuring more inclusive and precise diagnoses.
Log in or sign up for Devpost to join the conversation.