Third Eye

Results screen
Home screen
Labels of test images
Plotted labels of detections

Inspiration

We were inspired by the work of Rock Ridge's Computer Vision club. There are many computer vision projects being done there, many of which deal with medical help and telemedicine, so we decided to go with something similar and created Third Eye.

What it does

Third Eye takes in images of a chest x-ray and can detect COVID-19 with 98% accuracy. It can also detect Atelectasis, Cardiomegaly, Infiltrates, Mass, Nodules, Pneumonia, Pneumothorax, and Effusion, but it isn't complete on these diseases.

How we built it

We created a ResNet50 model for our COVID detection system and created a separate YOLOv5 model for our other diseases. We then implemented these models on a Django web application and set up forms for file upload. We created a separate page for the display of confidence values and bounding box values for detected diseases.

Challenges we ran into

Due to many dependency and repository errors with source code, we couldn't find a good way to represent our detection data for the YOLOv5 model's results. We had to resort to displaying bounding box data, which was the best we could do in the situation.

Accomplishments that we're proud of

We are proud of completing both models in time however because it took great time and effort to manage training and testing files for these custom models. They also had impressive metrics, such as precision, recall, and mAP (Mean Average Precision), which made us even more proud.

What we learned

We learned how to train custom datasets using YOLOv5 and ResNet50, a valuable skill that will aid us in our future projects. We also learned significant web development strategies, mostly implementation of CNNs and computer vision in the Django web framework.

What's next for Third Eye

We definitely want to get good visualization for the data on the YOLO model. Bounding box values won't cut it. YOLOv5 has a very specific loading for CNNs that was a very bumpy road, and a road we couldn't complete, so we will persevere and attempt to get visualization with properly formatted and plotted bounding boxes on images.