Mask detection for public spaces using Google Cloud AutoML

Our Inspiration

As COVID 19 continues to take over the globe, every country has been doing its best to minimise the communal spread of this destructive virus. One of the most potent ways to avoid the spread of this virus is by wearing a mask. Our product is aimed at enforcing safe social interactions, especially in spaces with larger social gatherings (eg. malls. restaurants, etc.). Our product alerts the administrative staff about whether their patrons are wearing their masks or not and therefore aims to keep everyone safe and healthy during these trying times.

What it does

Our program would ideally be linked to a camera located at the entrance of any social space where it can take a still image of the person entering. This still image is sent through a pythonIO program that detects the human objects in the image and outputs a zoomed-in photo of the peoples' faces. The close up of people's faces are compared to the mask and no-mask model that we created using the Google Cloud AutoML. If the person is not wearing a mask it will alert the administrative staff can then go and remind the visitor to wear a mask and save lives! :)

How we built it

Our mask detection program was based on the principles of object detection and classification. We started by exploring the different services offered by Google Cloud and found the Computer Vision services to be versatile and useful. Then we thought about how we could leverage this technology to solve an issue. We saw that on college campuses, in shopping centres, and in selected public spaces, there are rules to wear a mask that is often not followed. This presents the risk of increasing the likelihood of COVID-19 transmission. Hence, our computer vision model aims to solve this problem. We knew we wanted to custom train our AutoML model, to do so we created a web crawler that would search the internet for appropriate images to build a robust data set. Once trained, this AutoML model serves multiple purposes. It would most effectively be deployed at entry points of the aforementioned spaces where it can scan entering visitors and see if they are wearing a mask.

Challenges we ran into

Detecting whether the subject of an image was wearing a mask seemed like a simple enough task, but it wasn't without its difficulties. Data collection was a bit of a hassle. With Selenium and geckodriver, we were able to scrape over 850 training images for the AutoML model. But along the way, we were occasionally halted by certain failed URL requests and full-blown crashes during testing.

Once we were accumulated a sufficient amount of training data, much of it had to be 'cleaned'. There were many barely relevant and completely irrelevant images that ended up being downloaded (like a picture of a pig with a 'human-like' face) and had to be discarded manually after.

Concerning training and model evaluation, the Google Cloud API and structure of AutoML was definitely a bit of a head-scratcher at first, but a few hours in, we were able to hit the ground running.

Accomplishments that we're proud of

We trained our own models!!! And they turned out to be 99.4% accurate!!

What we learned

Overall, WHACK 2020 was an edifying experience, in more ways than we expected. Despite the thousands of miles of land and water separating us, we were able to work as if we were regular study buddies. We learned how to seamlessly transition from strangers to collaborators, and effectively communicate each step of the way. On the technical side, apart from our exposure to AutoML, we learned much more about the development process for object detection and image processing models. From data collection to model evaluation, there is a constant interplay of automated processes and manual 'cleaning' and refining.

What's next for Mask detection for public spaces using Google Cloud AutoML

Our next steps would be to make the interface more graphically friendly as well as work on integrating it with security systems and surveillance cameras so it can be employed in selective public spaces.