Smart_Hospital

Smart_Hospital is developed to be a secured AI platform with the purpose to assist radiologists with fast and accurate pneumonia dectection.

Comment

Dataset:

For the data analytics of COVID-19 pandemics, we used data collected by the Johns Hopkins University Center for Systems Science and Engineering updated on 4/30/2020.

For the chest X-ray detection models, we used combined 2 sources of dataset:

1) The first source is the RSNA Pneumonia Detection Challenge dataset available on Kaggle contains several deidentified CXRs with 2 class labels of pneumonia and normal.

2)The COVID-19 image data collection repository on GitHub is a growing collection of deidentified CXRs from COVID-19 cases internationally. The data is collected by Joseph Paul Cohen and his fellow collaborators at the University of Montreal

Eventually, our dataset consists of 5433 training data points, 624 validation data points and 16 test data points.

Inspiration

What will be working situation for medical staff in hospitals during and after the COVID-19 pandemic?How can we automate the process of COVID-19 diagnosis so precious time can be saved for both medical doctors and the patients? How can our solution for hospital later be scaled and implemented to be a essential tool for automating the daily operation at hospital even after the COVID-19 pandemics is over?

To answer these core questions, we did some background research to identify the main challenges in order to develop the best solutions around those:

Alt text

problems occur in the healthcare system during the pandemics are:

Long wait time for chest X-ray result during COVID-19 and after:

Alt text

Fig : Current chest X-ray diagnosis vs. smart_hospital

Patients can first be screened for flu-like symptoms using nasal swap to confirm their COVID-19 status. After 14 days of quarantine for confirmed cases, the hospital draws the patient’s blood and takes the patient’s chest X-ray. Chest X-ray is a golden standard for physicians and radiologists to check for the infection caused by the virus. An x-ray imaging will allow your doctor to see your lungs, heart and blood vessels to help determine if you have pneumonia. When interpreting the x-ray, the radiologist will look for white spots in the lungs (called infiltrates) that identify an infection. This exam, together with other vital signs such as temperature, or flu-like symptoms, will also help doctors determine whether a patient is infected with COVID-19 or other pneumonia-related diseases. The standard procedure of pneumonia diagnosis involves a radiologist reviewing chest x-ray images and send the result report to a patient’s primary care physician (PCP), who then will discuss the results with the patient.

What it does

Using the power of pretrained machine learning models from open source, Smart_Hospital is created as a full-scaled AI tool for radiology clinics and hospitals. It can automate the process of security log-in, PPE safety check for medical staff and assist radiologists determine sign of COVID-19 on chest X-ray images with high accuracy indicates pneumonia. This tool of cutting edge technology can be used to reduce the workload for clinicians, and speed up patients’ wait time for pneumonia lab results in this critical time of the COVID-19 pandemic.

Alt text

fig:Scan x-ray image

Smart_hospital Chest X-ray:

In the last step, the medical staff take patients’ chest X-ray images using the specialized machine and then upload the taken images to the database of web-app for testing for sign of COVID-19 infection or bacterial pneumonia. It is due to the fact that an AI system can review, highlight the pneumonia sign and classify each X-ray image all in less than 10 seconds (comparing the radiologist’s 20 minutes that we mentioned earlier), and it can do that same task effortlessly for 24 hours without taking a break. This time cut is especially critical in the time amid the pandemic of COVID-19. With this spreading rate, it will be overwhelming for radiologists to review a massive number of chest X-ray images of potential COVID-19 infected patients. With the assistance of CovidScan.ai, it can automatically highlight the suspected signs of pneumonia for the radiologists and speed up the process of chest X-ray review. Therefore, more COVID-19 positive-tested patients will get their result back faster and receive appropriate care sooner to prevent the spread of the virus. Alt text fig:Message to patient through mail

Game for Free time

under game option on menu

First view: Alt text

second view: Alt text

How we built it

Chest X-ray Classification: For this feature, we developed a Pytorch model. This project’s goal is to draw class activation heatmaps on suspected signs of pneumonia and then classify chest x-ray images as “Pneumonia” or “Normal”. For this project, we are going to use a dataset available at Kaggle consisting of 5433 training data points, 624 validation data points and 16 test data points. C. For the model, we load the pre-trained Resnet-152 available from Torchvision for transfer learning. ResNet-152 provides the state-of-art feature extraction since it is trained on a big dataset of ImageNet. ResNet-152, as the name sounds, consists of 152 convolutional layers. Due to its very deep network, the layers are arranged in a series of Residual blocks. These Residual blocks skip connections to help prevent the vanishing gradients which are a common problem with networks with deep architecture like ours. Resnet also supports Global Average Pooling Layer which is essential for our attention layer later on. For the attention layer to draw the heatmap, we use the global average pooling layer proposed in Zhou et al. Global average pooling layer explicitly enables the convolutional neural network (CNN) to have remarkable localization ability. We achieve 97% accuracy on the training dataset and 80% on the testing dataset.

Technical Requirements:

The packages required for this project are as follows:

Torch (torch.nn, torch.optim, torchvision, torchvision.transforms)

Django

Numpy

Matplotlib

Scipy

PIL

Tensorflow

jQuery

Challenges we ran into

This is the first time we all were working with such project and creating endpoints of the pre-trained Pytorch model. we had a little problem of focus with certain cameras so we had to experiment with several webcams that we had available to find one that didn't require to focus.Due to poor web-cam it probably lack of resolution.

Accomplishments that we're proud of

We manage to finish the project in such a limited time of 2 weeks in our free time from school and work. We still keep striving to submit on time while learning and developing at the same time. We are really satisfied and proud of our final product for the hackathon

What we learned

Through this project, we learn to implement a complicated image-recognition deep learning models. We also learn the deploying the models in web-app. This project can’t be done without the efforts and collaboration from a team with such diverse backgrounds in technical skills.

What's next for Smart_Hospital

In the next 2 months, our plan is:

1) We will partner with research lab to collect more dataset and find hospitals to test our solution. One of our memeber has published his newly collected dataset on this open-source github: https://github.com/nihalnihalani/COVID19-Detection-using-X-ray-images-/

2) Regarding our R&D, we plan on improving the performance of the platform, preferably by reading more scientific literature on state-of-art deep learning models implemented for radiology.

3) We also plan to add the bound box around the suspected area of infection on top of the heatmap to make the output image more interpretable for the radiologists. We are working to implament the multilabeling model of COVID-CXR on our dataset to improve our application. This model is published by The Artificial Intelligence Research and Innovation Lab at the City of London's Information Technology Services division and has accuracy 0.92, precision 0.5, recall 0.875, auc 0.96.

4) In many pieces of literature, they mentioned developing the NLP model on radiology report with other structured variables such as age, race, gender, temperature... and integrating it with the computer vision model of chest X-ray to give the expert radiologist’s level of diagnosis. (Irvin et al., 2019; Mauro et al., 2019) We may try to implement that as we move further with the project in the future.

5) With the improved results, we will publish these findings and methodologies in a user-interface journal so that it can be reviewed by expert computer scientists and radiologists in the field.

6) Eventually, we will expand our classes to include more pneumonia-related diseases such as atelectasis, cardiomegaly, effusion, infiltration, etc. so that this platform can be widely used by the radiologists for general diagnosis even after the COVID-19 pandemics is over. Our end goal is to make this tool a scalable that can be used in all the radiology clinic across the globe, even in the rural area with limited access to the internet like those in Southeast Asia or Africa.

References:

Irvin, Jeremy & Rajpurkar, Pranav & Ko, Michael & Yu, Yifan & Ciurea-Ilcus, Silviana & Chute, Chris & Marklund, Henrik & Haghgoo, Behzad & Ball, Robyn & Shpanskaya, Katie & Seekins, Jayne & Mong, David & Halabi, Safwan & Sandberg, Jesse & Jones, Ricky & Larson, David & Langlotz, Curtis & Patel, Bhavik & Lungren, Matthew & Ng, Andrew. (2019). CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison.

Kent, J. (2019, September 30). Artificial Intelligence System Analyzes Chest X-Rays in 10 Seconds. Retrieved from https://healthitanalytics.com/news/artificial-intelligence-system-analyzes-chest-x-rays-in-10-seconds Lambert, J. (2020, March 11). What WHO calling the coronavirus outbreak a pandemic means. Retrieved from https://www.sciencenews.org/article/coronavirus-outbreak-who-pandemic

Built With

Share this project:

Updates