With Machine Learning as an emerging technology, we felt inspired to use its capabilities to help people especially in healthcare. In data analytics, machine learning helps predict outcomes based on certain features. In previous hack-a-thins and in popular tech (such as Apple's Siri) we saw how machine learning can create cool and useful technologies and we were inspired to create something of our own.
What it does
Our mobile app uses a camera and takes a photo. The photo is then sent as data to our backend where it is analyzed, then passed through a function for analytics. Finally it is returned with two parameters "cancerous" and "healthy" along with a confidence index indicating the strength of its classification to each group.
How we built it
We used Google's Inception Image Classifier model and retrained the data to classify categories of skin (healthy or cancerous). The overall idea is that we will be using transfer learning. The neural network is already set up and we use that information as the input to the final classification network which makes it highly efficient while also distinguishing between a variety of different classes. We did this by altering the library and photos for which the model uses as training data. We used a directory of over 100 photos of cancerous skin to retrain our model. We built this all in TensorFlow and then created a python file that references a function called runTensorFlow. This python file is also built with flask to connect the backend to the front end and receives the image as input. The scripts from TensorFlows image recognition are run and the image is then classified using our retrained model. We also used ionic to create the front-end and application itself.
Challenges we ran into
One of the biggest obstacles we faced in the beginning was learning how to use Deep Learning and TensorFlow since none of us had any experience with it. We ran into countless errors and spent a lot of time debugging. The biggest overall obstacle we had was combining the backend to the front end. This required the use of flask and ngrok and sending the data live over the internet to the backend and returning information back to the front end.
Accomplishments that we're proud of
Overall we are happy with the model that was created and that we were able to implement mobile app development in combination with a machine learning algorithm. The use of data is important and using it to make better predictions in the realm of healthcare can have a positive impact.
What we learned
Deep Learning, Ionic, Mobile Development, Flask, Terminal, TensorFlow, combining front end with backend, http
What's next for DemiCheck
Our model can be retrained for detecting and diagnosing many other diseases and used for many other medical applications. Hopefully if more precise and detailed data is used we can train the models to notice even the slightest of symptoms. DemiCheck can be used in healthcare and ultimately prevent and give early treatment of illnesses.