Pure Skin AI

Inspiration

From the patient perspective, skin health access has been a barrier across many user groups for various reasons: skin health awareness, skincare eudctaion, or easy connection to professionals. From the clinic perspective, often understaffed small businesses also need automation to speed up their workflow.

What it does

1. It displays information about the dermatology clinic we propose.
2. It takes the user's facial image and provides a quick diagnosis of their skin conditions.
3. It also provides a chatbot/voice agent that gives treatment suggestions and helps schedule appointments.

How we built it

1. The chatbot/voice agent is built with Voiceflow that takes skin type, age, gender input from the patient, recommends products, and interfaces Calendly.
2. The AI diagnosis feature is implemented by an ML (image classification) model. It is trained on Teachable Machine based on an image database containing with 3 labels (acne, bags, redness).
3. The website is built in React, while integrating AI chatbot and diagnosis features.
4. The AI voice agent was built using blandai and the voice assistant can answer any questions based on information it contains in the knowledge base, ranging from company details, details about the experts, etc, as well as schedule an appointment for the user.

Challenges we ran into

1. Dataset availability: available datasets are extremely limited -- some are for commercial uses (nexdata) while others can be complex and hard to port (Google scin).
2. Model training: we ran into logistic issues when integrating pre-trained models from Kaggle notebooks, including saving a model and exposing APIs.

Accomplishments that we're proud of

• In this project, we successfully developed a fully functional AI chatbot that provides personalized skincare advice and streamlines appointment booking with professionals through Calendly. We also integrated a voice assistant, allowing users to engage in real-time conversations, making the experience feel more personal and interactive. Our face scanner feature is another major accomplishment, providing users with a detailed analysis of their skin to tailor recommendations. We’re proud of how these technologies work together to create a seamless, user-friendly platform that elevates the way people care for their skin.

What we learned

• Take full advantage of existing skillset because learning new things under time limit can be stressful.
• Teamwork from idea orchestration to labor distribution can be difficult.

What's next for Pure Skin AI

1. Better diagnosis: (a) improve the accuracy of the model by refining the model and expanding training datasets; (b) try CV techniques that identify skin issues with bounding boxes.
2. Ethical use of AI diagnosis and treatment suggestions: clarify that AI diagnosis or treatment suggestions are in no way replacement of professional medical advice.

Built With

• airtable
• css
• html5
• javascript
• react
• teachable-machine
• voiceflow