The following questions are answered in a brief manner, there is a lot more detailed information regarding such questions in the uploaded presentation slides.
Inspiration
When I was 10 years old, I began to deal with some unwanted skin conditions including eczema and hives, at one point my whole body was covered with these conditions and it was truly a burden to live with. My family and I had no idea on how to deal with it, or even what it was at the time. If only I had a way to classify or identify my skin conditions before they got worse, or a way to find self care options and educational content. That is what inspired me to build this app years later now that I have programming and machine learning knowledge. It truly is a tool I wish the younger me had access too, and I hope that many other kids dealing with such skin conditions can have more help in the future.
What it does
The app simply takes in a photo from the user, analyzes it through it's trained model and gives a confidence report back to the user across the 6 different categories it is trained on. There are also questions in the app that don't affect the models prediction, but however do affect the educational and self care content displayed afterwards.
How I built it
I used google colab and pytorch to build training / testing models, grabbing my sources from the kaggle dataset "DermNet". I decided to roll with transfer learning, a specific machine learning technique that involves retraining a model for a specific task related to it's original purpose. In this case, that would be Efficient-net's model for image preprocessing. After training and testing I used streamlit and huggingfacehub free tiers to convert my model code into an app. Huggingface handled the trained model which then allowed me to access it through streamlit, where the rest of the app's UI was programmed as well as the model uploading.
Challenges I ran into
I ran into many challenges, but the ones I want to highlight most include original data set sourcing (Dermnet bias and some categories lacking data compared to others), implementing transfer learning as someone who is unfamiliar with actually using it, and app deployment. I would say these were the most time consuming for their own indiviual reasons and had me a little frustrated to deal with at times, which is why I'd highlight those as my biggest challenges.
Accomplishments that I'm proud of
Having an actual working model and app that is at least much better than classifying off of pure guesses. It's not a final model for me, almost more like a first step in this app's journey, but I'm pretty happy with what I've got and excited to work on/improve this project in the near future.
What we learned
App deployment, working with streamlit for app programming, transfer learning implementation.
What's next for AI Based Skin Condition Visual Similarity Tool
Adressing bias, better data sources, improvement with image processing, overall model structure improvement (whether that be through unfreezing more of the current model backbone or investigating outside of this), and having the questions actually affect model prediction.
Built With
- colab
- dermnet
- efficient-net
- github
- huggingfacehub
- kaggle
- machine-learning
- python
- pytorch
- streamlit
- transferlearning
Log in or sign up for Devpost to join the conversation.