CNNs to diagnose COVID-19 from X-Ray Images of the Chest

Confusion matrix showing the 100% accuracy on the validation set for the model v3
Heat map showing the areas of high activation from the model v3
Sample of the beginning training progress of model v3
Sample of 25 images from the training dataset

Inspiration

Being trapped at home and forced to read and watch so many stories on the devastation of COVID-19 made me want to just do something to help out. I decided to focus on the disease detection, since I reasoned that if one can detect the COVID-19 sufficiently it would decrease the number of new cases, and thus help flatten the infection curve. Watching the youtube video by MedCram, I noted that a big issue is the RT-PCR, the gold standard for diagnosis of COVID-19, had a unacceptably low sensitivity of 71%. That means that many people get false negatives back and think they are free to walk around because they don't have the disease when they are indeed infecting many people! Thus, I looked into alternative ways of detecting/diagnosing COVID-19 that could supplement the RT-PCR result with a result with much greater sensitivity. I learned that CT scans have been widely published to have much higher sensitivies of 98%! However, the issue that immediately struck me was the CT scans are not widely available and are also very expensive, so it may not be feasible to depend on in the setting of a global pandemic. Thus, I reasoned that X-ray images could also give interesting results, as they are in many aspects similar to CT scans, and are also much more widely available and cheap.

What it does

Given an X-ray image of a patient, this model is able to predict whether the patient has COVID-19, no disease, or if the disease was instead caused by viral or bacterial influenza (which can often be confused with COVID-19 in terms of symptoms). In addition, the heatmaps generated by the model provide insights into possible biomarkers for the disease.

How I built it

The X-ray images were sourced from multiple online open-source datasets for the purpose of maximum generalizability. To train my model, I used transfer learning from the ResNet50 architecture. The first model was trained on raw data files without data augmentation and achieved a test set accuracy of 90.4%. The next model was trained using data augmentation and achieved a test set accuracy of 94.2%. The final model was trained after normalizing the images and achieved a test set accuracy of 90.7%. This model is advantageous because it is more generalizable to other datasets.

Challenges I ran into

The first model did not perform significantly better than some of the published models on analyzing X-ray images to diagnose COVID-19. I hypothesized that the reason was because there was not enough training data (as the open-source dataset had very limited images of COVID patients), so I performed data augmentation: rotating, changing contrast, sharpening, blurring, and minimum filtering. Using this augmented data, my next model (4class v2) was able to achieve a 100% accuracy on both the test and validation, which raised an eyebrow for me. Indeed upon looking at the heat maps of the images, it seemed that the model was taking advantage of the differently zoomed images (all the COVID images were different zooms from the other datasets). Thus, I manually cropped all the images to be the same size so the model would instead focus on the biomarkers.

Accomplishments that I'm proud of

The final model (v3) achieved a 100% accuracy on the validation and test set. This was really astounding, since it was able to get 100% correct on images it had never seen before. This means that, although the data is limited, the model has a 100% sensitivity, which is on par with the sensitivity of CT scans. It was able to generalize much better to an outside dataset that contained images of patients without disease: although it predicted incorrectly that some of those patients had COVID-19, it was much better than v2 and is the first reported instance of a COVID-19 model performing very well between datasets that were not in the training set.

What I learned

CNNs are able to learn much about a dataset despite a limited number of images, and are an extremely powerful tool in differentiating between images, even if the coder himself has a limited understanding of radiology!

What's next for CNNs to diagnose COVID-19 from X-Ray Images of the Chest

I am in contact with multiple hospitals, specifically MD Anderson, and I hope to gather more test images of COVID-19 images to further test the accuracy of this model. In addition, I am currently working on a multi-label model that specifically looks for biomarkers in COVID-19.