RetnoScan: AI-Based Retinopathy Diagnosis

A mobile app to predict diabetic retinopathy from eye fundus images using computer vision.

Comment

Inspiration

According to the World Health Organization, there are approximately 2.2 billion people globally who have issues related to their eyes. Out of these 2.2 billion, it’s estimated that 1 billion of these issues could be prevented or have gone undiagnosed. The leading causes of eye issues within the world are due to addressing these issues once it has been too late. Today we will be investigating Diabetic Retinopathy, one of the possible eye complications. Diabetic Retinopathy is a complication of diabetes that is caused by damage to the blood vessels in the tissue at the back of the eye, which is the retina.

Currently to combat eye diseases and treatment options are not affordable for everyone. There is an expensive diagnosis that is required in order to know if you have a complication. If the diagnosis is taken late, then the complication could have caused more damage to the eyes resulting in more expensive treatments. This expensive diagnosis isn’t even covered with insurance! Additionally there are a shortage of doctors right now within the COVID-19 situation, where doctors should be more focused on helping patients who are prone to risks.

What it does

RetnoScan is a fast and easy at-home self diagnosis with functionality to share results with your doctor. We used machine learning to train a model to detect healthy and Retinopathic eye images and packaged it into an android app which can be used by anyone anywhere. After taking a picture of the eye, the model predicts if your eye is retinopathic or healthy. You can also share the results with your doctor if needed.

How We built it

To standardize the images, the dataset we used was cropped to remove any excess black zones, and then resized to a 256x256 resolution.

After this initial image processing, we needed to extract the features that are correlated with diabetic retinopathy. To do this, we tried a variety of approaches, but the best approach was as follows. Since many of the features indicating retinopathy were small details and dots in the image, we first blurred the original image, and then took a global average difference between the blurred and original image to better extract these features. The end result was a nice preprocessed version where hemorrhages and aneurysm contrasted nicely with the background.

See the image gallery for a comparison of original vs preprocessed.

After preprocessing all the images, we used Transfer learning to train a Convolution Neural Network where the first 5 layers were the first 5 layers of a model called EfficientNetB0. The remaining 3 layers were added with the Keras API. The last layer was 1 neuron with a sigmoid function to map the output between 0 and 1 (healthy and retinopathic, respectively).

See the image gallery for our CNN architecture, confusion matrix, and an example prediction

Challenges We ran into

Our first challenge was an extremely imbalanced dataset that had differing resolutions and zooms. We first tried to make this a categorical problem by trying to distinguish between levels of disease but that didn’t work. There was not enough data for the higher levels of the disease to train a model. Instead, we decided to make it binary between healthy and signs of disease so that the user can decide whether or not to go see a doctor.

Our second challenge was hyper parameter optimization while training the model.

Our third challenge was loading in the tflite model into our android app and predicting from a bitmap.

Accomplishments that we're proud of

  • Preprocessing the Images so nicely
  • Training a Deep CNN with Transfer learning
  • Packaging the CNN into a tflite model and running on an android app with Java

What We learned

  • ML Concepts - Keras, some advanced regularization methods, How to do transfer learning
  • Building Android apps - especially how to load in a tensorflow tflite model
  • OpenCV2 (preprocessing)

What's next for Diabetic Retinopathy App

  • login system to save previous image scans
  • One-click email to doctor button
  • Multi-class classification model to predict different stages of D.R.

Built With

Share this project:

Updates