Main screen (without screening results)
Main screen (with expanded action menu)
Main screen (with finished and processing screening results)
Notification of analysis result
Detail view of screening result
Melanoma - the most serious type of skin cancer - looks quite like nevus and often people would pay little attention to those nevus-like melanoma until it becomes severe. If the melanoma is diagnosed in its late stage, the survival rate of 5 year interval is only 14%. Instead, if it is detected early, the survival rate can reach up to 97%. In addition, recent success in the research of applying deep learning in the diagnosis of melanoma inspired many people to train different deep neural networks and they obtained good result . If there is a way to utilize successfully trained deep neural networks in common people’s daily lives, people will be more aware and there will be less patients suffer from this deadly disease.
What it does
This project focused on using a mobile app to raise people’s awareness and recommend to take further examination if necessary. A mobile app combining the advantage of deep learning on the cloud and the indispensable role of mobile phone in daily lives was successfully developed.
Users can take pictures on the place they have doubts and pictures will be automatically sent to a fully trained deep learning network on a cloud for the analysis. One number representing the probability is sent back to mobile phones. This convenient app will definitely prevent additional medical cost on cases of late detection of melanoma and make people live better.
How we built it
With the aid of the previously trained deep learning neural network parameters on melanoma provided by the "The Learning Titans" team online, we revised the tensorflow code to make the model compatible with Amazon EC2 cloud platform, serving as out backend .
For the frontend, a proper mobile app was written in Android Studio with a user-friendly interface and the ability of efficient communication between the cloud and users’ mobile phones. After a user uploaded an image, our backend server notifies his device using Firebase cloud messaging once the classification is performed.
Challenges we ran into
We tried to make the whole computing process run feasibly on users’ mobile phones. However, due to the huge size of the parameters of the deep learning neural networks, the required computing power is way more than a mobile phone can provide. This leads us to implement the app in the direction of deep learning on the cloud, which is also a great solution.
We also faced the typical challenges of setting up a complex project, backend infrastructure, and dealing with changed APIs.
Accomplishments that we're proud of
Although our initial approach wasn't successful and despite the related challenges we faced, we're glad that we finished a working prototype of our idea within the short time frame of HackZurich 2017.
In the end, by means of a light app, we can make peoples’ lives different and safe!
What we learned
Recent technological developments, especially in the area of deep learning, are promising to revolutionize health care in the future. However, while today's smartphones come with immersive computational capabilities, those aren't always sufficient for complex neural networks.
What's next for Dr. Jon's Skin Cancer Screening
In the short term, development of a comparably light deep neural network to make the dream of a truly offline app for skin cancer detection come true.
In the long term, assuming further advances in deep learning hardware for mobile devices, development of an equally powerful network usable within an app on a smartphone.
Beyond that, even better deep learning models.