please note that the website mask detection functionality does not work and refer to the video for reference!


The effects of COVID 19 are still being felt by all of us and we must not let our guard down. Masks offer supreme protection to all of us and I felt that a system to monitor people wearing masks would be beneficial to society, with applications in crowded / indoor areas. More about that on the About Page on my Heroku website.

What it does

It uses your camera to send data back to the server, which applies a ML model to draw bounding boxes around all human faces and label them as wearing mask, not wearing mask, or wearing mask incorrectly, all done in real time. More about that on the Website.

How we built it

I started with using tensorflow / Keras to train the model, then combined it with opencv to detect faces, predict the model, then draw bounding boxes. Then I started creating the Django website and integrating them all together. Then I attempted to deploy it on Heroku but it unfortunately fails when using opencv.

Challenges we ran into

The machine learning model does not work accurately in real life as it flickers between the 3 states. The processing of the live camera frames was quite difficult, hitting many errors on the way. There were many problems with deployment to Heroku, such as the storage of static files and especially with accessing the camera, which remains unsolved since the solution is too complex and due to time constraints. To clarify it is due to the fact that openCV cannot access the client camera via Heroku. Therefore the actual TRY IT NOW functionality does not work at the moment and only works with local host. However please have a look at the video to see what the final product would be when the bug is fixed and deployed.

Accomplishments that we're proud of

I am proud of being able to create my first 'acceptable' convolutional neural network to detect whether a person is wearing face masks as well as having a very cool website design. It is also my first time using opencv which had its difficulties but I am glad I got through them.

What we learned

I have learned much more about using Django and Keras, as well as how to deploy to Heroku. Nonetheless, I learned about how persistence is key to getting things accomplished!

What's next for Mask.AI

Firstly I can implement the solution to the problem of not being able to capture the live video on Heroku with OpenCV and instead use WebRTC, which is quite complex to implement. Secondly, the machine learning model can be much improved on or trained to be much more accurate by using a larger and varied dataset. A more efficient model such as EfficientNet can be used to decrease CPU usage. Thirdly can be integrated with CCTV cameras and monitored in order to promote mask wearing. Additionally an app can be further developed for greater accessibility.

Share this project: