Mask Detect

Mask-Detect uses a custom trained model to recognize, detect, and notify people who are not wearing their masks or are wearing them incorrectly!

Comment

Inspiration

Since many of us are mentally overloaded during this time of quarantine, we find that it is easy to forget some important things. Our group found that we tend to forget our masks when we walk out our doors. Perhaps it is better to mentally live in a rosy world where we don’t need them. However, in today’s world, we certainly do need our masks, and we’ve all experienced that moment of sudden realization and subsequent shame when we find we’ve left home without a mask on. Then we have to turn around and do the walk of shame as we trek back inside to retrieve one, or perhaps we don’t realize until it is too late and we are well away from our homes after driving. Mask-Detect is a functional proof-of-concept that is meant to help stop this mental slippage from ruining your day by pinging you on your phone when it sees that you left your home without a mask on.

What it does

The program, when running, searches a video feed for a person in frame. It analyzes frames of the video feed, and if it detects that a person without a mask or a person with an improper mask is in frame, it will ping them on their phone with an SMS/MMS alerting them that they need their mask, as well as sending them the image with the detected mask error outlined.

How we built it

See the pdf below/on our Github for the detailed workflow of our project.

Challenges we ran into

The biggest challenge ultimately came down to formatting numerous XML files to a master .csv file. This was due to a slight lack of understanding on how to parse the data provided by the XML’s. That said, we were able to quickly grasp this knowledge, afterwhich we got our csv to compile, fed it into Google Cloud’s AutoML, and trained a model to detect whether or not someone is wearing a mask, and more importantly, wearing it correctly. After the training of our model, progress began picking up steadily, and we were able to wade through unfamiliar territory to form our overall project: Mask-Detect. Another challenge was sending images with text messages. It was easy to send text messages with the Twilio api but sending images with the message is more difficult because the local image has to be accessible to Twilio’s servers. So in order to have a local server we start a python http server and then use ngrok to create a tunnel to that server.

Accomplishments that we're proud of

When we came into BostonHacks2020 we did not have a single idea, however a short brainstorming session quickly solved that. What is one issue everyone has come upon during these months of quarantine? Leaving their house without their mask on. Perfect, so we use object recognition to detect a mask, print something out in console and we are done, right? Well, we hoped to take a different approach; an approach that we felt could have been slightly too ambitious, however we carried on, and 24+ hours, we can all confidently say that we were not wrong in our decision. BostonHacks2020 has reinforced all of our problem solving capabilities, and shown us that with our combined determination, there are no bounds. In the end, our biggest accomplishment is that we created a functional project that is applicable to today’s world.

What we learned

We first learned how to parse XML files to create a .csv file labeling our image set, thereby gaining an understanding of XML formatting and libraries used to parse XML. In our next step, we learned how to use Google Cloud’s AutoML Vision to recognize objects. It is amazing how effectively the model learned to accurately recognize correctly worn masks, incorrectly worn masks, and unmasked faces. Our final task was integration. We had a trained model, software to capture images from a webcam, and an app to query twilio’s API to send text alerts to users, and all we had to do was integrate these pieces together so create a script that automatically captures webcam images, queries the Google Cloud API to run our object recognition model, and send SMS/MMS alerts to users when they are not correctly wearing a mask. The most exciting part of our BostonHacks experience was seeing all of our work come together as we learned how to take individual working parts to create a functional project.

What's next for Mask Detect

In our Covid dominated world, improvement of this project would entail deployment on larger scale use cases, such as apartment complexes or university dining halls. Provided with certain hardware, images could be captured only when motion is detected and notifications could be expanded past simple text messages. The past few months have shown how important it is to wear a mask, and our project could even theoretically deny those not following guidelines from accessing certain spaces. Overall, the message we hope to convey is that Mask-Detect goes far past being only a “smart-home” application ... Mask-Detect is a project contributing to a smart-world.

Share this project:

Updates