What is Ice Bear Security Systems?
The coronavirus 2019 (COVID-19) disease has changed all of our lives. The pandemic has disrupted our schools, pushed the hospital system to its capacity, and created a global economic slowdown. Now, months into quarantine, as we look forward to reopening our economy and businesses, we must constantly adopt new safety measures. We're working to provide a cheap, easy, and scalable solution to in-person consumer and retail stores based on CDC guidelines.
Ice Bear Security Systems (IBSS) - inspired by our favorite cartoon character who is always protecting others - ensures that a given store is fully operating within safety criteria. IBSS features a developed Raspberry Pi Camera that stores would place outside their entrances – it streams 24/7 live data to a web-accessible dashboard that screens for three key safety criteria:
1. mask presence 2. fever symptoms 3. total store capacity
How we built it and how it works
Rasberry Pi, OpenCV, and TensorFlow We wanted our hardware to be affordable and modular. Our Rasberry Pi is connected to a camera and IR sensor module. In-house python scripts apply a mask detection software we trained using OpenCV and the live feed along with OpenCV frames are hosted onto a local server. Meanwhile, the IR sensor captures a nested array of temperature values we use to create a heat map and note the highest seen temp.
Flask, Firebase With the Rasberry Pi constantly streaming two types of videos and updating customer values we needed a real-time Firestore server and flask to do the heavy lifting in our back-end. Flask helps us host our live feed in a web-accessible manner by sending constant updates for each frame. Firebase on the other hand handles a constant push and update of our customer data collection for our front-end to pull down.
React Here's where it all comes together. By accessing API endpoints hosted by our flask framework we can stream a live feed from both our camera and IR module from the Rasberry Pi onto our dashboard. React also connects to our live Firebase server in order to get live updates on a customer's mask presence, temperature, and store capacity. All this information is laid out in an easy to read manner following our Figma designs.
Challenges we ran into
For most of this project, it was a first for our team. Here's what we learned:
Developing a full-stack application is definitely tough. One of the main challenges we ran into was finding creative solutions to be able to stream live OpenCV data through Flask so React can access it through API endpoints as well as displaying data from our live Firebase database. As a team, we learned some of the ins and outs to connecting a server, back-end, and front-end together.
Our first time utilizing TensorFlow and machine learning. In order to get an accurate reading of whether or not a customer is wearing a mask, we learned to work and struggle with training models.
Our first time with React. Although we had some experience with html and css on our belt, we needed a stronger framework and technology to achieve our goals this time around. Connecting to databases using RESTAPIs definitely helped us better grasp how clients and servers communicate.
What we are proud of
We were able to build a full-stack application! There are plenty more functionalities we would love to add to assist store owners and improve the customer experience, but as of submission, we are proud to have an application that can update and store live data, something maybe even small businesses can look into today.
Having a clean and easy-to-use interface! We were lucky to have Selina Tieu, a first-time hacker, as a designer as part of our team. Our goal was to create professional designs that store owners can easily access to keep their store safe. We cant wait to bring our design to its full potential.
Learning a ton of new technologies! From scipy and pygame to render our IR heatmap, to tensorflow and OpenCV to detect masks and Firebase and React to display our data, getting all of these to work together hand-in-hand was as equally frustrating as it was rewarding to us.
Most importantly as a team, we were proud to create a prototype application that impacts communities around us. Each of us was able to take what we were passionate about in computer science and ui/ux design, combine all the technologies and techniques we learned throughout this hackathon, and pour it into a passion project that is relevant to the problems we face as a society today.
There is plenty of more work to do here, but we are so happy to have gotten our embedded system this far!
What's next for Ice Bear Security Systems
More accurate infrared temperature readings. Rendering day by day store statistics such as peak hours and percent admitted. Combining multiple cameras in one store system to help decrement # of customers in the store.