DermDetect

• 

  Log from the different testing models

Inspiration

We’ve all had those moments when something feels off with our skin, so we go to the doctor. They take a quick glance, tell us to ice it or come back in a few weeks, and somehow that ends up costing hundreds or even thousands of dollars. Therefore we wanted to create an application that can identify what’s wrong with our skin, saving us both time and money.

What it does

The app analyzes the user’s skin through the camera and detects whether it’s a wart, acne, eczema, or a mole. After detecting the issue, it provides the user with recommendations on how to address it.

How we built it

We built the app using JSON to send data between our web application and servers. The web application itself was built with React and flask, while HTML was used to structure the content and design elements. JavaScript to add interactivity to the web page, and Python powers the backend. We used PyTorch to run the machine learning models through the dataset for skin condition detection.

Challenges we ran into

The challenges we encountered included effectively storing data related to the questions we asked the Gemini and its responses, enabling users to maintain a conversation with the AI. We also struggled with ensuring that the data in the backend could be implemented in the frontend. Our limitation of using only free datasets made it difficult to find enough good data to feed into the model to detect the skin conditions. We created many models from scratch which had accuracy of up to 56% on our test data, but this simply wasn't high enough for our application. Our solution was to use transfer learning with a general purpose classification model, resulting in an accuracy of over 85% on our test data. Additionally, we faced the challenge of trying to train the application to identify correct skin conditions with people who had darker skin tones with a relatively small non diverse dataset. We used many different transformations on the image in order to lower the model's over reliance on color, particularly when it came to identifying moles.

Accomplishments that we're proud of

The main thing we're proud of is being able to use this technology to build something end to end that can have a impact in the world. Many of the technologies we used in this project we had 0 prior experience with, so we are very proud of all the learning we had to do to get the project to function.

What we learned

We learned a lot about machine learning and how to implement different APIs effectively. This project also emphasized the importance of collaboration; working with others allowed us to combine our strengths and create a more sound solution.

What's next for DermDetect

Being able to expand the range of skin conditions that can be identified is what's next for DermDetect.

Built With

• flask
• gemini
• javascript
• python
• pytorch
• react
• vscode