molecroscope

• 

  OpenCV image processing to detect differences in moles, with perspective and skew correction

Inspiration

With the thinning of the ozone and the increasing risk of skin cancer, early detection is the key to recovery. The changes in moles have become the most important indication factor when it comes to diagnosing malignant cancer cells. We wanted to create a tool that helps people better track the change in their moles, and understand their risk of skin cancer in a simple, accessible method.

What it does

With a simple hardware expansion hack, we turned our normal phone camera to be a microscope-like lens (up to x70 magnification ability). An Android app takes in the microscope picture as input and processes the images to track the change in shape of skin moles. The app then calculates a person's risk of skin cancer depends on the characteristics of their mole, using a Keras machine learning model running as a service in a cloud instance. Molecroscope is your personal skin monitoring app!

How we built it

It runs on a native Android app written in Java, calling an OpenCV image processing script, written in Python. We trained the Molecroscope's cancer prediction ability with a CNN neural network (VGG16) in a compute engine VM instance on the Google Cloud Platform, using the ISIC Archive's dataset of moles. Data is sent through HTTP request to a Google Cloud Platform storage bucket, and the returned data is visualized using the open-source MPAndroidChart library.

Challenges we ran into

We have noticed that we had some issues with utilizing GPUs on the Google Cloud Platform for our machine learning VM without upgrading our accounts. We ended up using CPUs but we wish we had a better option for that. Separating and analyzing the differences of skin mole pictures has also been challenging, especially under different lighting conditions, angles, and positions. We circumvented the issue using OpenCV, which has been a great help at analyzing the unique mole shapes.

Accomplishments that we're proud of

We have managed to deploy our instances in the cloud computing environment using the credits. We also created a machine learning model to predict the risk of skin cancer. We also created an efficient and accurate skin mole compare system, capable of accurately analyse skin moles under diverse conditions. Everyone in our team has different types of skill sets. With our diverse set of specialized skills, we were able to problem-solve as a group.

What's next for molecroscope

It is time for sleep! Just kidding. After the cuHacking event, we would like to containerize our app with Docker and orchestrate it with K8S, as well as implement a pub/sub or trigger-based communication pipeline to properly integrate everything together.

Built With

• android
• cloud
• google-cloud
• keras
• opencv
• python
• tensorflow