DermoScan

Inspiration

• Doodling the future with a focus on inclusivity.
• Healthcare is a critical industry in the US and people do not have equal access to it
• We want to incorporate inclusivity into high tech especially in the healthcare industry
• Cutting edge AI research is often times limited to the best hospitals and not accessible to the public - thus we want to bridge this gap

What it does

• Takes a scan of a user's affected body part and outputs a diagnosis for the skin condition

How we built it

• Used React JS, Node JS, and Mongo DB to create a web app that directly interfaces with the user's webcam
• Implemented a ResNet-50 Convolutional Neural Network to perform image classification on the webcam input
• Used a Wave2Vec Natural Language Processing neural network to have the program speak to the user and take in user voice input

Challenges we ran into

• Getting the image sent from the front end React JS to the backend Node JS
• Finding a skin disease dataset
• Figuring out whether to make a custom CNN or use a pre-trained ResNet
• Connecting the web app to the Python script
• Connecting Node JS to Mongo DB
• Connecting React JS to Node JS

Accomplishments that we're proud of

• Getting the NLP and voice input to work
• Training a CNN using Tensorflow

What we learned

• How to use Tensorflow
• How to use MERN stack and Full stack dev
• How to how to deploy a web app

What's next for DermoScan

• Adding more skin diseases
• Improving the web app

Built With

• javascript
• mongodb
• node.js
• python
• react
• tensorflow