maskurity

Inspiration

We were inspired by the job of one of our team members. Nathan works at a grocery store in downtown Calgary that is plighted by individuals often walking into the store without a mask on, despite the bylaw in Calgary, which requires mask use. This causes unnecessary stress for the security guard, as they are responsible for kicking these individuals out after requesting they apply a mask. This distracts the security guard from their main purpose: preventing theft.

Calgary is not the only city that has bylaw enforced mask use, most cities in Alberta do, along with the majority of Ontario, all of Quebec, and British Colombia among many other locations and cities in Canada. As indicated in the following article, enforcing these bylaws in many areas may be very difficult. It seems that COVID-19 will be around for some time now, and as a result, so will the issue of enforcing these mask wearing laws. It is unrealistic to expect the police to be able to enforce such laws so the onus falls upon store owners, which often feels like an insurmountable task without sacrificing the time of employees i.e. Nathan's security guard.

We believe maskurity offers an easily deployable solution to this issue that helps not only save time of employees but also keep everyone safe, employees and shoppers alike.

What it does

Maskurity uses any device that has a web browser and camera to take a picture of your face after a voice activated command, which is then sent to our image classifier that will grant or deny access to the building based off the result (with an override for individuals with exemptions). This allows for a hands-free and therefore COVID-safe automation to this issue.

How we built it

Maskurity uses a react app front end which can be seen using a website hosted on heroku found below! The front end uses the Web Speech API to detect a voice command which trigger an image to be taken. The image is then sent to the backend which uses a Node.js Express web app which is also deployed on heroku. This backend then talks to the image classifier. The image classifier was made using Google Cloud’s AutoML Vision and roughly 2000 pictures of masked people and 2000 pictures of unmasked people as the training dataset.

Challenges we ran into

Although we met many challenges during our development process our biggest ones would be: Google Cloud's AutoML authentication being deployed to heroku and getting all of the used libraries to mesh together well.

Google Cloud's AutoML authentication caused us a fair few issues with gaining access to our model after we trained it, neither of us had worked with AutoML before so the whole authentication process was confusing and tedious for us. We ran into a large issue with it when we were deploying our backend on heroku. The way google Cloud's AutoML authentication works seemed encourage us to put our authentication credentials inside our github repository. If we did this it would have solved our issue but it would also cause a huge security risk. Our solution to this problem ended up using a combination of heroku's config var's and a custom buildpack.

Our second large issue was getting everything to work together smoothly, if you inspect our repository, alot of the effort came into thinking of solutions for getting many of these api's and languages services to interact with each other smoothly often needing an intermediate data structure at times this was very challenging.

We registered a maskurity.tech domain, but could not get it properly point to our herokuapp url in time.

Lastly effectively picking a dataset and accounting for the training time required was another challenge. If we had begun training almost immediately after the hack started it would have saved us a lot of time. This is mainly because we had this limbo state in the middle of the hack where the only thing we were waiting on was the training of our model to be complete and we had no way to add to the project until it was done.

Accomplishments that we are proud of

We are proud that we tackled a problem that is relevant to us in our daily lives and did not let the need for bleeding edge technology to deter us. We are also proud that despite our struggles and challenges we managed to make a functional end product with more features then we initially thought we would have time to implement. Lastly we are proud of our teamwork, in our last hackathon our group distracted each other a lot, asking for help on issues before fully trying to resolve them on our own. This disrupted our workflow quite considerably. This time around we made an effort to seek alternative sources of help before distracting a teammate!

What we learned

We both learnt a lot during our development of maskurity, neither of us had any experience with google cloud in general none the less Google Cloud’s AutoML Vision, so it was quite the learning experience for us. We learned about how influential your dataset is for ML as well as how to quickly identify good vs bad datasets for simple vision problems like ours. We also learned a lot more about JavaScript and how to effectively communicate between a backend and frontend.

What's next for maskurity

We would love to leverage Twilio to send texts to managers for our exemption case, or when there is a lower confidence probability (<90%) . We would also like to implement actual hardware or emulated hardware as a proof of concept. Although not too far off we currently only display and read out a message that the door will be unlocked, ideally, in addition to the message we would want a Smart Lock or some other easily computer engageable/disengageable lock to be utilized.