SpacedOut

Inspiration

As students, we’ve seen the dangers of indoor occupancy across the world. While during the beginning of the pandemic the majority of states strongly denounced or prohibited indoor gatherings and indoor work, many areas are now allowing limited and controlled indoor gatherings. There is no better example than universities and workplaces, which are inviting students back to campus and employees back to offices. Headlines like “there are over 800 cases at UT Austin”, “University of Georgia reported 1400 new classes”, and “return of Defense department leads to a 35% spike of cases” flood the news. Unfortunately, regulations at many workplaces and universities are not easily enforced and are often ignored. We have seen this on a personal level as one of our team members works at our university library, a place often used to collaborate and inspire new ideas. The library has numerous safety precautions in place like seats set 6ft+ apart, a sign-in system to monitor the number of people, the removal of any shared objects, and mask requirements. However, these precautions are monitored by 2 students who are worried about their own safety. To address any of the multiple infractions, the student workers would be forced to put their own health in danger. Also, there is almost no way that these two workers would be able to monitor an entire room, especially since many tables and areas are out of direct eye view. The library was also warned that it was highly likely that they would close because the university understood the difficulty of balancing monitoring a space and maintaining the safety of the workers. We wanted to create a product that would make re-introducing indoor life safe and easy, which is why we came up with SpacedOut. It’s more effective, accurate, and safer than having real-life monitors.

What it does

It acts as an extension for security cameras by tracking the number of people in a room, whether people are wearing masks or not, and whether people are 6 feet or more apart. It notifies the person watching the feed when there is a capacity violation, a mask-wearing violation, or a distance violation.

How we built it

We built it by utilizing various deep-neural network libraries in Python that allowed us to combine motion-detection, facial recognition, and face-mask detection into a compact product called SpacedOut. The first step was to train a neural network to be able to detect the difference between someone wearing a mask and someone who wasn’t. Here we utilized TensorFlow, Keras, and SciKit-Learn to train and fit a neural network on a dataset of over 600 sample images of people with and without masks. We did so by combining various convolutional, pooling, and dense neural network layers over a pre-trained ResNet image model. After producing a model capable of detecting masks with over 98% accuracy, we move our way over to OpenCV. In OpenCV, we combined a pre-trained ResNet10 face detection model with our mask detection to allow our webcam to easily identify human faces and determine whether or not they are wearing a mask in real-time. First, utilizing the face detection model, we identify the pixel locations that mark a rectangular frame around each face it detects. Using these frames, we then generate a list of predictions as to whether or not these frames contain faces with masks. Within these frames, we also generate a centroid, which is simply a circle that marks the center of each rectangle generated. Using these centroid positions, we are then able to make estimations as to how far each person is away from each other. This is done by constantly making predictions as to how far each centroid is from each other based on their pixel positions. Finally, to give users a clear picture as to how many people are in the room, we also placed a live counter in the top left corner that updates based on how many centroids are currently on the screen.

Challenges we ran into

The main challenge we faced was having a good product and balancing it with a great presentation. We believe that no matter how useful and thorough a product is,** if we can’t show the benefits and usefulness of it, then there is no point in creating the product.** Thus, we struggled with balancing both tasks. On the coding front, a big issue we ran into was how to create an accurate face-mask detecting model. Utilizing Keras and understanding how to extract features from images utilizing an efficient set of convolutional, pooling, and dense layers was an initial hurdle we faced. Furthermore, training on over 75gbs of images is extremely different compared to a CSV of text information. Due to time constraints, finding that perfect balance between learning rates, information dropout, and accuracy was a big hurdle that we were able to overcome. Another challenge we faced was how to combine face detection and mask detection in OpenCV. A challenge was how to figure out how the outputs for both models could be combined optimally in order to generate face and facemask detection in real-time without causing the live-feed to be too sluggish. On the presentation front, we struggled with consolidating all the information we wanted into a two-minute presentation. We had an in-depth analysis of our problem, our solution, and our impact. Getting across all the information in a 2-minute video that is also captivating was hard to balance.

Accomplishments that we're proud of

We're proud of not only our project but also our presentation. Our team spent countless hours developing both so that we can showcase the talent of all the team members. With a variety of backgrounds, ranging from computer science to business to design, our team members were all able to use their special talents to create a great project.

What we learned

Besides learning more about the libraries, languages, and methods we used, we also learned a lot about creating a thoughtful presentation that will make potential investors interested in our product. We all learned some basics in design, art, and management since we were all relatively inexperienced in those areas.

What's next for SpacedOut

Our next steps are first getting our own university to implement it, perhaps in the library that our team member works at. If all goes well, we will hope to have all the buildings in our university using it, and then expanding to other universities, companies, but also places like stores and public transportation. While our product mainly targets the coronavirus pandemic, we believe that it will be relevant for quite some time as mask-wearing integrates itself into our life. However, our product can also simply be used to control room capacity in the far future.