MaskDetech

A web application that is geared towards detecting if users are wearing the proper protective coverings or not during the pandemic.

Comment

Inspiration

We are all going through an increasingly unprecedented time as for many of us, we are experiencing for the first time shutdowns, quarantines, and an ominous tension in the air. However, as lockdown protocols continue to ease up, the public’s inability to adopt face masks has become increasingly worrying. This is incredibly dangerous to public health and safety, as face masks would help contain and prevent the spread of an especially contagious virus. Although in many states, wearing face masks in public has become government-mandated, we believe the manual enforcement of these policies will be difficult. To aid in the suppression of this devastating virus, we developed an intelligent system that can in real-time detect the presence of face masks, while providing other services like quick news feeds and COVID-19 statistics.

What it does

Our app uses machine learning libraries, TensorFlow, and Keras with OpenCV to use live camera data from the end-users' webcam to detect instantly whether or not they're wearing a protective face mask. Our app then displays a certain message accordingly to notify the user and serve as a reminder before heading outside or making contact with others. Our app also has a page that displays COVID-19 statistics in a nicely nested format that's interactive for anyone to play around with. We also have a page that serves a constant feed of new news articles all relevant to COVID-19 to keep our users constantly informed with quick descriptions, grabbing the key points of these articles.

How we built it

The Machine Learning Model

Our machine learning algorithm is based upon the OpenCV MobileNet v2 classification model with a TensorFlow backend. It is fully trained and implemented in Python. Our algorithm first detects and identifies faces within the video frame, and then predicts whether each person’s face is wearing a mask or not.

COVID-19 Data Visualisation

Our web app fetches and displays real-time COVID-19 data on our statistics page in the form of graphs such as bar charts and pie charts. We fetched the data from a COVID-19 API based on Johns Hopkins CSSE data and used Google Charts to visualize it within our web app.

Live News Feed

Our live news feed page enables the user to get constant updates on the current COVID-19 situation. Using the Python News API, a JSON API for live news, we were able to fetch the latest COVID-19 news headlines and display them in an elegant format with a focus on readability, as well as a link to the original news story.

The Stack

Our web app is built upon Flask, a Python micro web framework. In our Python backend, we run all of our API calls and our camera vision models, which can later dynamically load HTML/CSS/JS templates to be used as our web app’s UI. Our app is hosted online via the Google Cloud Platform App Engine.

Challenges we ran into

A big challenge we endured was deploying the app. As the use of our machine learning model required the use of a heavy web service, we chose the Google Cloud Platforms App Engine, which took a lot of trial and error to perfect our configuration files to support our specific app with the heavy machine learning files on top of it. Another challenge we ran into was adjusting our web application from using the camera via localhost to then using it when deployed to a cloud service. We had to use some native JavaScript code for allowing access to camera in order for this to occur which was quite lengthy as it required us to have to rearrange our code in several areas and write new code in a language less familiar to us.

Accomplishments that we're proud of

Our crowning achievement is the working development of a powerful machine learning model that utilized computer vision algorithms. In addition, we’re proud of our successful integration of a flask app with this model using the bootstrap library to build HTML/CSS pages. We’re proud of being able to deploy something that very well can have real-world usage and could be of major benefit to our communities. Another big accomplishment would be our live deployment as we were able to figure out how to utilize Google Cloud Platform and use it to host our web app.

What we learned

We learned how to effectively use and extract the data from the COVID API, and then later visualize it using Google Charts. We were able to expand upon our limited experience with OpenCV to expand upon our computer vision skills, which we later developed into a machine learning model. We improved upon our collaboration skills on Github and expanded our knowledge of the version control software Git. Another service we learned was following Google Cloud Platforms docs more efficiently and getting the right information in a shorter period of time.

What's next for MaskDetech

While we were able to host a web app, we want to extend the use of our machine learning model with video cameras and video footage. If we were able to connect our software with security cameras, we could in real-time detect the usage of face masks, being able to track not only data related to the pandemic but also be able to help law enforcement keep track of people who aren't taking the proper safety precautions and ensure the public safety of all in the midst of a global pandemic.

Built With

Share this project:

Updates