Mask Detector

Inspiration

We were inspired to create this because wearing masks is paramount to slowing the spread of COVID-19. Being able to detect whether a person in an image is wearing a mask or not has a variety of applications, including surveillance for businesses to allow or not allow customers into a space, or sampling a public space to compute statistics about mask-wearing at a given time.

What it does

The web app allows users to upload a picture and detects which people in the picture are wearing masks and which people are not.

How we built it

We used the Google Cloud AutoML Vision API to train an ML model to recognize faces with masks on them and faces without masks. We started on an artificial dataset of pictures of people with and without masks, training a simple single-label classification model to determine if a person was wearing their mask in a headshot. Later, we trained an object recognition model to tag faces with and without masks using some hand-labeled public images. We then created the web application using React and integrated our model using Tensorflow.js.

Challenges we ran into

As this was the first hackathon for the whole team, we ran into quite a few roadblocks. Some hackathon-essential languages and frameworks came into play that we’ve never worked with at length before, like React. We ended up with a trained model and a website, but we struggled to get the image submission to play well with our API call, which ultimately led us to fail to deliver a working product.

Accomplishments that we are proud of

Although we had some knowledge gaps that definitely slowed us down, the team was diligent in trying to catch up to speed and learning together to build and debug this project.

What we learned

We learned how to use the Google Cloud Platform to train a machine learning model, as well as the basics of front and back-end development!

What's next for Mask Detector

Mask Detector could be improved in a variety of ways, including:

UI improvements. Our inexperience with frontend development was a challenge we had to overcome. Naturally, a simple improvement to our project would be to improve the user’s interaction with our software by adding functionalities and utilizing more modern, minimalist UI/UX Design principles.

Improving training data. All ML models have bias, and ours is no exception. We could improve Mask Detector by not only expanding the size of our training set, but including a wider variety of training images (surveillance footage, close-up and far away shots, low angle, high angle, side angle, etc.). In this regard, a number of ethical concerns also become apparent - are we aware of the racial biases that our model has developed? How equitable is it?

Mask Detector also has various potential applications into the real world:

Community integration. Our ML models could be implemented into home, school, or business surveillance systems, to improve security and help keep public spaces safe.

Data collection and statistical analysis. Snapshots from traffic cameras or surveillance systems can act as random samples of populations in various times and locations. Such snapshots can be used to deduce crucial statistics about the Coronavirus’ spread and the effectiveness of current safety measures. Having a reliable face mask recognition system could expedite and expand data collection and pave the way for deeper and more insightful statistical analyses.

Facial recognition. A potential extension of our project could be into the field of facial recognition. In an effort to normalize mask wearing in our current society, could we improve facial recognition systems to recognize individuals even when they wear face coverings? Such a model could have deep implications into other fields such as the justice system where criminals often wear masks to conceal their identities. Perhaps the recent interest in face mask recognition is a step in that direction.