Connection to the Theme
We stood upon the shoulders of giants in many ways when building this app. First and foremost, inexperienced developers like us are only able to implement machine learning in IOS applications thanks to the work done in developing the PyTorch python library. Furthermore, we built off of the people who put together the image dataset that we used to train the neural network of our app.
One of the main issues surrounding everyday life in the pandemic is ensuring that people in public places are wearing masks. Businesses have had to station employees outside of their stores to make sure that customers are wearing masks and following social distancing guidelines. This forces already struggling businesses to divert employee attention away from their business. It also forces employees to come into contact with people who are not wearing masks, which can lead to unwanted confrontation between employees and customers. We wanted to create an app that addresses this issue by automating the process of checking customers for mask usage.
What it does
The app’s main use is in scanning customers at the entrance. Businesses can use our app by placing a phone by the entrance of their stores, and the app will scan customers wanting to enter the store to check that they’re wearing a mask. If the customer isn’t wearing a mask, then they will not be allowed to enter the store and the app will direct the user to return with a mask on. Such a system ensures that maskless customers won’t be able to enter buildings and threaten the safety of other employees and customers, allowing businesses to allocate employee attention towards their business and the mask-wearing customers, and helping promote and support proper mask-wearing protocols to control the spread of COVID in our community.
How we built it
We used Pytorch to retrain google’s MobileNetV2 architecture to enable it to detect whether or not a person in a photo was wearing a mask. The model was trained with this data and we ended up obtaining an accuracy of 96% with it. We also built a simple iOS application that the neural network can be seamlessly integrated into. Unfortunately, we didn’t have enough time to fully implement the texting feature we had planned for the app.
Challenges we ran into
The main issue that we ran into was time constraints. None of us are very experienced coders, and we all worked with technologies and programming-languages that we weren’t very comfortable with. As a result, we dedicated more time towards the back-end logistics of the app, leaving us less time to focus on the UI. Another issue was finding suitable data to train the network on as many datasets online were formatted poorly or had other issues making their use impractical. Even the one we ended up using was formatted strangely and included invalid images, but we were able to programmatically sort out these issues.
Accomplishments that we're proud of
We were able to train a neural network using google’s MobileNetV2 architecture that can correctly identify if a mask is being worn 96% percent of the time. We also designed an intuitive UI that walks users through the process of scanning themselves.
What we learned
Each member of the team utilized challenging technology, and as a result learned a lot during this week of coding. We became much more familiar with iOS development and the intricacies behind developing user interfaces and layout constraints, as well as the challenge in manipulating data to input into the NN model. We learned how to design, train, and test a neural network in Python and learned how to port a network built in Python into the iOS environment using a new language: Objective C++.
What's next for MaskOn
In the future, we plan on implementing an automated-messaging feature to alert store owners about results of the scans. We hope that our app can be used by small businesses who need a way to ensure that their customers are abiding by COVID-19 guidelines but do not have spare employee resources to dedicate to this.