NeuroDerm

Inspiration

The growing refugee crisis brought our attention to the standard of living that many people face in developing nations. Here, we found that there is a significant lack of dermatology practice in third-world countries, whether that be due to a lack of funding/infrastructure or lack of awareness. Health workers assisting civilians in these areas must rely on the help of a licensed medical professional to make correct diagnoses about various skin conditions and their severity. With the lack of these licensed professionals available in these areas, civilians do not receive the quality of treatment they deserve. This is where NeuroDerm comes in!

What it does

NeuroDerm provides anyone with a smartphone with the ability to take a picture of any concerning cutaneous areas and receive predictions about the severity of them. The prediction is determined through a machine learning algorithm that was trained on 10 000+ images of various skin lesions. The mobile app provides a user-friendly interface to a tool that will provide a method for underresourced populations to monitor their own dermal health.

How we built it

We built the mobile app with React Native and Expo while utilizing Firebase storage to store image data taken with the app’s camera feature. The machine-learning algorithm was implemented in TensorFlow and trained in Google Cloud's AI Platform by making a custom Docker container and pushing it to Google Cloud's Container Registry. To limit computational load, an existing convolutional network for image processing was retrained to classify images of skin lesions.

Challenges we ran into

Our primary challenge developing NeuroDerm was figuring out how to connect the data pipeline between the mobile application and the machine learning model that’s hosted on Google Cloud. After speaking to a Firebase expert, our solution was to utilize Firebase Storage to hold images and Firestore to store image URLs. This allowed us to apply ML predictions onto the images taken through the app.

Thankfully, our team persevered through these challenges with the help of fantastic mentors and volunteers at HTN 2019!.

Accomplishments that we are proud of

Our team was able to successfully develop a functional convolutional neuro network for the first time, and a working mobile application that utilized Firebase Storage to hold image data. The team learned these new technologies in a short frame of time in order to achieve our goal of developing a functioning application. In addition, our team was able to achieve 68% accuracy with our model, making it very close to dermatologist accuracy(75%)!

What we learned

NeuroDerm provided every member of our team with their first opportunity to work with neural networks that handle image data for a mobile application. While the task seemed daunting at first, our team researched and applied various techniques and technologies to execute this idea. For instance, our team utilized React Native and Firebase for the mobile app, both of which are technologies that we had limited experience in. In addition, our team learned how to retrain existing models to fit our requirements, and we were able to train the model on 10,000+ images!