MaskIndex

Homepage of MaskIndex
Example of Viewing Map
GIF
OpenCV Algorithm Testing

Inspiration

Nowadays during this “new normal”, it’s hard to tell whether a certain location poses a heavy risk of contracting the COVID-19 virus. Certain locations throughout the day have different amounts of people visiting, and it’s not guaranteed that everyone will be wearing masks. Masks are scientifically proven to significantly lower the risk of receiving and transmitting the virus. It would be nice to tell what percentage of people at a certain location at a given time are wearing masks.

What it does

Our web app delivers a simple, easy-to-use interface that allows users to view how many people are wearing masks at locations around the world. Videos of public spaces can be processed by our computer vision algorithm, which calculates an average of the amount of people wearing masks. The camera can upload this data and its location to our database. Our website then pulls this data and displays it on a Google Maps page on our website.

How we built it

We created a computer vision algorithm using OpenCV, Darknet, and the YOLOv4 framework. This model was trained on Google Colab using datasets from Kaggle and blogs. The algorithm then calculates an index based on how many people are wearing masks
The algorithm uploads data to our Google Firebase Realtime Database via the pyrebase API
Our website then pulls data from Firebase and displays it in a user-friendly format using the Google Maps API and Javascript

Challenges we ran into

Time taken to train CV models. Creating robust data pipeline from the CV algorithm to Firebase to the website

Such an ambitious idea is hard to realize without its challenges. Some roadblocks and difficulties encountered during implementation included the various package and dependencies involved, causing a web of interrelated errors in installation and execution. Finding reliable images to annotate and incorporate was also required for accurate and precise recognition of masked and maskless individuals. Getting all the components - the website, the firebase database, the openCV algorithm, and much more - to synchronize and work with one other in real-time was a challenge on all levels of abstraction.

Accomplishments that we're proud of

Computer vision algorithm that detects masks with high accuracy. User-friendly, intuitive web format that lets people easily see mask-wearing at areas.

What we learned

Machine Learning, Computer vision, full-stack web design

What's next for MaskIndex

Incorporating with AWS Cloud Computing to speed up the computer vision algorithm processing rate
Integrating with local businesses so customers can know a list of “Top 10 Places” near them that are best following mask-wearing guidelines

Built With

Submitted to

SummerHacks

Created by

I worked with training the Yolov4 object detection network using darknet framework on google colab. Worked on opencv and its dnn module to include features for analyzing webcam feed, pre recorded videos, and images.

Shreyas Kudari
Computer Engineering @ UT Austin
I worked on creating a pipeline between the OpenCV algorithm and Google Firebase using Python. I also worked with Michelle to help integrate Firebase and the website using Javascript

Austin Tsao
Computer Engineering @ UT Austin
I worked on the full-stack web application, focusing on front-end UI and integrating the Google Maps JavaScript API and Google Firebase JavaScript SDK on the back-end.

Michelle Wen
Computer Engineering Student at UT Austin
I worked on YOLO image annotation, video formatting and parsing, and establishing connections from the program to Google Firebase.

Isaac Lee