Pytorch Covid Fighter

Front Page
Classification Component
Comparison X-rays/CT-scans and generate reports
Analysis X-rays/CT-scans and generate reports

Inspiration

As COVID-19 peaks, there is a shortage of health workers (especially in developing countries). So it means medical trainees and interns have to step up and understand this disease. This tool can help users in understanding and analysing chest x-rays and CT scans. It can also help users in differentiating between COVID and similar chest infections like viral pneumonia. Intended Audience: medical professionals and data analyists.

What it does

This tool uses pre-trained resnet18 model from torchvision to classify chest X-ray images as COVID/viral pneumonia and normal. As a medical trainee (or even a non-medical regular user) I can use this classifier to get an initial idea of a patient's image i am analysing. Then this tool can help me analyse the image and look for certain COVID specific imaging features like ground glass opacities, white consolidations and pavement patterns along lung peripheries. I can use image processing features (invert/zooming/contrast etc) to better recognise such imaging features.

This tool also gives me a comparison component. Using that I can compare patient's image with other COVID/pneumonia or normal images. Such intuitive comparison can further tell me about features such as lymph nodes, lung cavities which are present in similar chest infections but absent in COVID.

This tool also provides report generation which a medical trainee can share.

How I built it

Classification part uses pre-trained resnet18 model from torchvision. It is trained for 1000 imageNet classes. I changed the last fully connected layer features from 1000 to 3 and did training on COVID-19 radiology dataset (https://www.kaggle.com/tawsifurrahman/covid19-radiography-database). The accuracy reached is greater than 95% on the dataset. For classification in browser, i converted the pytorch model to ONNX model using torch.onnx module. Then i used the onnxjs NPM package for classification.

Report Generation was done in the browser itself taking care of data privacy. Image comparison uses new Javascript APIs such as ResizeObserver. Image processing was achieved using CSS3 filters.

Challenges I ran into

Running the trained model in browser/nodejs environments was the biggest issue. Converting pytorch model to ONNX model reduced the accuracy of the model. After a lot of searching, I failed to debug the exact issue. Initially wanted to just run the model in browser itself (for data privacy concerns) but the converted model was around 43MB, so it took a lot of time for download by browser. So i had to go the nodejs way.

Accomplishments that I'm proud of

Completing all basic functionalities of classifying, analysing and comparing. Learning a lot about ONNX and interoperability (or lack of) of ML models. Learning a bit about COVID along the way and making a tool that can have a positive social impact.

What I learned

Pytorch, Torchvision. Taking advantage of already pre-trained model like resnet18. Standardising ML model formats using ONNX. On the front-end, using Suspense with React for lazy loading components and new JS/CSS APIs.

What's next for Pytorch Covid Fighter

Implement reading dicom images. Have a classifier for CT-scans too. Try to incorporate more resources and learning material in the app, so that medical trainees/interns can level up and reduce the shortage of front-line health warriors. Ability to add more public datasets for comparison component.

Github: https://github.com/akhil-vij/pytorch-covid Model and Jupyter notebook: https://github.com/akhil-vij/pytorch-covid/tree/master/model