AI Breast Cancer Image Annotation

• 

  ML training results

• 

  Malignant tumor detection

• 

  ML Test Results

Inspiration

Cancer is a big killer when not detected early, and training models to accurately detect a malignant cancer that needs treating can be extremely valuable and save lives. We wanted to use this opportunity to apply computer vision techniques to efficiently analyze thousands of images of samples and make predictions.

What it does

Our model utilizes a CNN to identify parts of a microscopy tissue sample that are IDC positive (Invasive Ductal Carcinoma) malignant cells. We then generate a heatmap that helps show what areas are malignant and why the model made its prediction. We also built a front end to turn it into an application that takes a microscopy image as an input and then can annotate it to show whether it is malignant and which sections of the image suggest so. It will create a gradient heatmap of which the color will show up a redder color for regions of more significant value for the prediction that it made.

How we built it

The CNN model was built using pytorch and trained on a breast cancer imaging dataset sourced from Kaggle link This included 70,000+ malignant and 70,000+ benign samples. The heatmap is generated using a technique called GradCAM which takes a peek into the layers of the ml model and determines which part of the image is causing it to make the prediction that its making. The front end was created with NextJS and the back end was made using Django Rest API.

Challenges we ran into

One of our main challenges was the speed at which we were able to test model parameters for our ML model. Each run could take at least 30 mins and often an hour plus which limited the speed at which we could get accurate parameters. We were also working with libraries and software we were unfamiliar with, so it took time to get all of the parts working together.

Accomplishments that we're proud of

Our model was able to predict the right diagnosis 83.5% of the time which shows that it is able to function as we hope it too. Additionally, adding in front end features and the heatmap involved implementing libraries we were unfamiliar with, so to see those up and working was a proud moment indeed!

What we learned

We learned so much about CNN models and how they can be used for computer vision, especially in a biological context. We learnt about image processing and how to turn that into a functional application.

What's next for Machine Learning Supported Cancer Image Annotation

Diversifying the data input types, extending past breast cancer to other types of cancer as well. Our model could be further tweaked to be a bit more accurate, perhaps dually looking at imaging as well as other type of data such as mammograms and patient metadata.

Built With

• cnn
• django
• gradcam
• nextjs
• python
• pytorch
• react