We are all going through an increasingly unprecedented time as for many of us, we are experiencing for the first time shutdowns, quarantines, and an ominous tension in the air. However, as lockdown protocols continue to ease up, the public’s inability to adopt face masks has become increasingly worrying. This is incredibly dangerous to public health and safety, as face masks would help contain and prevent the spread of an especially contagious virus. Although in many states, wearing face masks in public has become government-mandated, we believe the manual enforcement of these policies will be difficult. To aid in the suppression of this devastating virus, we developed an intelligent system that can in real-time detect the presence of face masks, while providing other services like quick news feeds and COVID-19 statistics.
What it does
Our app uses machine learning libraries, TensorFlow, and Keras with OpenCV to use live camera data from the end-users' webcam to detect instantly whether or not they're wearing a protective face mask. Our app then displays a certain message accordingly to notify the user and serve as a reminder before heading outside or making contact with others. Our app also has a page that displays COVID-19 statistics in a nicely nested format that's interactive for anyone to play around with. We also have a page that serves a constant feed of new news articles all relevant to COVID-19 to keep our users constantly informed with quick descriptions, grabbing the key points of these articles.
How we built it
The Machine Learning Model
Our machine learning algorithm is based upon the OpenCV MobileNet v2 classification model with a TensorFlow backend. It is fully trained and implemented in Python. Our algorithm first detects and identifies faces within the video frame, and then predicts whether each person’s face is wearing a mask or not.
COVID-19 Data Visualisation
Our web app fetches and displays real-time COVID-19 data on our statistics page in the form of graphs such as bar charts and pie charts. We fetched the data from a COVID-19 API based on Johns Hopkins CSSE data and used Google Charts to visualize it within our web app.
Live News Feed
Our live news feed page enables the user to get constant updates on the current COVID-19 situation. Using the Python News API, a JSON API for live news, we were able to fetch the latest COVID-19 news headlines and display them in an elegant format with a focus on readability, as well as a link to the original news story.
Our web app is built upon Flask, a Python micro web framework. In our Python backend, we run all of our API calls and our camera vision models, which can later dynamically load HTML/CSS/JS templates to be used as our web app’s UI. Our app is hosted online via the Google Cloud Platform App Engine.
Challenges we ran into
Accomplishments that we're proud of
Our crowning achievement is the working development of a powerful machine learning model that utilized computer vision algorithms. In addition, we’re proud of our successful integration of a flask app with this model using the bootstrap library to build HTML/CSS pages. We’re proud of being able to deploy something that very well can have real-world usage and could be of major benefit to our communities. Another big accomplishment would be our live deployment as we were able to figure out how to utilize Google Cloud Platform and use it to host our web app.
What we learned
We learned how to effectively use and extract the data from the COVID API, and then later visualize it using Google Charts. We were able to expand upon our limited experience with OpenCV to expand upon our computer vision skills, which we later developed into a machine learning model. We improved upon our collaboration skills on Github and expanded our knowledge of the version control software Git. Another service we learned was following Google Cloud Platforms docs more efficiently and getting the right information in a shorter period of time.
What's next for MaskDetech
While we were able to host a web app, we want to extend the use of our machine learning model with video cameras and video footage. If we were able to connect our software with security cameras, we could in real-time detect the usage of face masks, being able to track not only data related to the pandemic but also be able to help law enforcement keep track of people who aren't taking the proper safety precautions and ensure the public safety of all in the midst of a global pandemic.