One of the main issues surrounding everyday life in the pandemic is ensuring that people in public places are wearing masks. Businesses have had to station employees outside of their stores making sure that customers are wearing masks and following social distancing guidelines. This forces already struggling businesses to divert employee attention away from their business. It also forces employees to come into contact with people who are not wearing masks, which can lead to tense standoffs between employees and customers. We wanted to create an app that addresses this issue by automating the process of checking customers for mask usage.
What it does
The app’s main use is in scanning customers at the entrance. Businesses will use our app to place a phone by the entrance, and customers wanting to enter the store will be scanned to check that they’re wearing a mask. If the customer isn’t wearing a mask, then they will not be allowed to enter the store and the store owner will be notified via automated text messages of a maskless customer.
Such a system ensures that maskless customers won’t be able to enter buildings and threaten the safety of other employees and customers, allowing businesses to allocate employee attention towards their business and the mask-wearing customers, and helping promote and support proper mask-wearing protocols to control the spread of COVID throughout communities.
How we built it
We used Pytorch to retrain google’s MobileNetV2 architecture to enable it to detect whether or not a person in a photo was wearing a mask. The model was trained with this data and we ended up obtaining an accuracy of 96% with it.
We also built a simple iOS application that the neural network can be seamlessly integrated into. Unfortunately, we didn’t have enough time to fully create the rich UI that we had planned for the app, but included in the project is a sample mockup of what we intended the UI of the application to look like had we had more time.
Challenges we ran into
The main issue that we ran into was time constraints. None of us are very experienced coders, and we all worked with technologies and programming-languages that we weren’t very comfortable with. As a result, we dedicated more time towards the back-end logistics of the app, leaving us little time to focus on the UI.
Another issue was finding suitable data to train the network on as many datasets online were formatted poorly or had other issues making their use impractical. Even the one we ended up using was formatted strangely and included invalid images, but we were able to programmatically sort out these issues.
Accomplishments that we're proud of
We were able to train a neural network using google’s MobileNetV2 architecture that can correctly identify if a mask is being worn 96% percent of the time. We also designed a simple UI that walks users through the process of scanning themselves.
What we learned
Each member of the team utilized challenging technology, and as a result learned a lot during this week of coding. We became much more familiar with iOS development and the intricacies behind developing user interfaces and layout constraints, as well as the challenge in manipulating data to input into the NN model. We learned how to design, train, and test a neural network in Python and learned how to port a network built in Python into the iOS environment using a new language: Objective C++.
What's next for MaskOn
First and foremost, we plan on building a functional IOS application using the UI models and autonomous algorithms we have already created. Later down the line, we hope to add the functionality of detecting if masks are being worn improperly. We hope that our app can be used by small businesses who need a way to ensure that their customers are abiding by COVID-19 guidelines but do not have spare employee resources to dedicate to this.