User Interface
Backend Flow

Putta Mask On It

Group:/HackRealm
Members: Sai Coumar, Daniel Trager, Albert Lua, Jason Leong
Category: Health and Wellness

What Inspired Us?

While we were brainstorming for the category of Health and Wellness for COVID-19, we were inspired by how stores use their employees to stand by the entrances of their franchises to verify if guests are wearing masks. As a result, we came up with the idea to essentially make an application that would use a camera -webcam or security camera- to monitor people passing by the camera in real-time, to complete the task of the employee.

What Does PMOI Even Do?

Putta Mask On It(PMOI), will take in a live video from your Webcam and provide real-time updates to the user interface on whether or not the user has a mask on or not.

How Is It Built?

The only prerequisite to run the application is to have Python 3 installed. The first key requirement of our project was the facial recognition model, which simply utilized a pre-trained classifier to find specific locations of faces in an image. The output of this model is sent into a MobileNetV2 model, which we trained to classify face images into a mask or non-mask wearers. In the python script, we use OpenCV to take the default system webcam input frame by frame, input it into those two linked models, and output each frame to a locally-hosted webpage. To do all of this the user must run our desktop application.

Challenges and Issues Encountered

A challenge we were initially presented with was connecting the backend video output to our frontend user interface. For this issue, we had to research possible ways to connect the two; one failed idea was the use of WebSocket. The main issue we faced with the WebSocket approach was that it required multiple instances of a video feed, which most webcams do not support. One video feed would be on the HTML-front end, and it would send frames to the ML model back-end, which would send processed frames back. This approach ended up too slow and convoluted. To fix this issue we changed it to a local application. Another issue that we had was when we were able to use our real-time video to take snaps of the video each second that is stored in a folder. The issue was that when it executed, the Website User Interface cycled through every picture within less than a second. To fix this, we swapped to Flask through the use of Python.

Accomplishments Achieved

One of our biggest accomplishments has to be when we were able to connect our model output to the user interface. The result was that we could have real-time updates to the user interface so it can tell us if the user is wearing a mask. We made it usable to anybody who has Python 3 installed.

What Have We Learned?

We have learned how to incorporate the use of a webcam into machine learning as well as several techniques to transfer the processed video output into our user's local webpage.

Future Goals for Putta Mask On It?

Our future goals for PMOI is to further advance mask recognition beyond having a mask or no mask. An example would be whether or not the mask is properly worn.