Masked

masked logo
no mask
regular disposable mask
circular disposable mask
washable cloth mask

Inspiration

With the current outbreak of COVID-19, health professionals urge citizens to wear masks to help mitigate the spread of the coronavirus. Many countries and states in the United States have implemented measures and laws require the use of facial coverings and masks in public areas. Furthermore, many businesses require masks. However, especially during this time, many of those businesses lack the resources to thoroughly enforce mask policies at all times. That's where Masked comes in.

What it does

Masked is a real-time face covering and mask detection system that ensures that masks and facial coverings are being worn at all times. Masked uses an array of cameras (if available) and machine learning models to instantly detect if an individual is not wearing a mask. Within milliseconds, the individual is notified and reminds all citizens to respect mask and facial covering policies and laws. As a result, businesses and public locations, as well as everyday citizens, are able to live safely knowing that everyone is masked.

How I built it

Masked uses a TensorFlow machine learning model based in Python to identify whether an individual is wearing a mask or not. The TensorFlow training model was extensively trained on a variety of scenarios and angles to ensure optimal and precise accuracy while preserving detection speed and delay.

Within milliseconds, frames are analyzed through OpenCV and sent to the TensorFlow model to identify the location of a face within the frame as well as whether masks are being worn or not. The analyzed data is sent to a Node.js frontend to display the result in a simplistic and easy to understand user format.

If available, multiple video sources can be allotted within a singular Node.js frontend to monitor data from all available cameras in one easy-to-monitor user frontend.

Challenges I ran into

Analyzing raw video proved to be difficult in that the video source tended to be rather lagging, thus producing delayed and often incorrect identifications. As a result, I decided to use individuals frames, captured through OpenCV, in rapid succession as this allowed for the TensorFlow model to maximize identification efficiency while maximizing the reduction of both lag and end-user identification times.

Analysis speed within the TensorFlow machine learning model was a challenge in that both lag and processing time had to be accounted for, especially within the entire identification process. I solved this problem by compressing frames through specific video codecs as well as streamlining the TensorFlow model to focus on optimization.

Accomplishments that I'm proud of

Within the allotted time scope for the hackathon, I was not able to extensively train the TensorFlow machine learning model to the entire extent in which I intended to. However, with less than 750 sample images, we were able to achieve over 98.7% accuracy on mask identification, demonstrating the efficiency and robustness of our model.

The angled identification model for determining side profiles was also a large undertaking that involved developing my own library to modify testing data to fit this method of identification. I'm happy with the overall accuracy for the angle detection within side profiling and off-center identification.

What I learned

I learned how to connect both Python, as the backend, and Node.js, as the frontend, to fully get the best out of both worlds, with Python for machine learning and Node for routing and the end-user frontend. I also learned how to work with TensorFlow and OpenCV Computer Vision to develop models that can label individual frames with such high accuracy.

What's next for Masked

I would like to improve the identification of multiple individuals within the scope of the OpenCV frames. As the current version of Masked can identify multiple individuals within a singular frame, the accuracy decreases slightly when involving various degrees of movement from multiple isolated and connected individuals.

I also want to improve end-user and frontend experience. Due to the time I spent developing the TensorFlow model and identification system itself, the frontend itself is quite bare and can be fully flushed out with infrastructural features, such as live notifications and recording capabilities, within a matter of time.