Currently, businesses are unable to effectively track, monitor, and predict customer traffic. How can businesses know when is best to implement promotional activity? How can businesses know what is the optimal number of working staff at a given time, and adjust their hiring needs? How can businesses quickly and more accurately forecast customer traffic and revenue projections, and change their operations accordingly? The questions go on and the problem spans across any businesses where customer traffic matters, from retail stores to shopping malls to restaurants to coffeeshops to co-working spaces, and more.
What it does
Our solution to the problem is Shopalyze. Shopalyze enables businesses to better understand their customer traffic flow while accounting for conditional variables that may impact a customer’s decision to frequent the business. It provides and can predict customer traffic at any given time, accounting for many variables including time of day, day of the week, season, temperature and weather conditions. Based on this, Shopalyze can equip business owners with the information needed to implement incentives to drive additional traffic and boost sales, and adjust their operations to boost both top and bottom line.
How we built it
To build Shopalyze, we leveraged both a camera and CNN (Convolutional Neural Network) to perform regression on the number of people in a given space. We implemented bounding boxes via Google Cloud, uploading images to Firebase Storage. We updated Firebase with day, month, year, hour (in 24hr format), minute, second, day of the week (with 0 as Sunday, 1 as Monday, ….., and 6 as Saturday), temperature (in Fahrenheit), weather (0=sunny, 1=cloudy, 2=rainy, 3=windy, and 4=snowy), if a store-wide promotion is going on (0=no promo, 1=has promo), and number of people detected. Our firebase updated in real time along with the camera and offered the highest-level of data for our businesses. Temperature and weather data were from python library API.
Challenges we ran into
We initially opted for facial detection with the hopes of later being able to recognize people’s faces and gather more data in that regard. However, even the most powerful face detection API, Google Cloud Vision API, had lackluster performance when it came to detecting more than 5 people at a time. Our team then made the pivotal decision to abandon facial detection and switch to object detection because our application is meant for the real world market, and we prioritized being able to detect people on the order of 100.
Accomplishments that we're proud of
We’re proud of putting together different technologies in a seamless manner and using them all to the best extent possible. On the computer vision front, we’re using object recognition and localization in order to figure out how many people are in the store. In term of the cloud and big data, we’re updating our Firebase database every second and populating it with valuable information for business owners. And lastly, our statistics analysis helps our business owners understand their own business.
What we learned
This hackathon was another reminder to us of the limitations in deep learning. Although deep learning is constantly evolving, it’s also equally important to remember that there are always situations and edge cases in which deep learning isn’t trained well enough or that there isn’t enough data for. In our situation, we came into the hackathon believing we could use a simple desktop webcam to recognize the faces of all the people in a room with a relatively fast frame rate. However, coming out, we all understand that the limitations of our hardware. The raspberry pi, was our limiting factor this time and didn’t allow us to have such an extreme level of sophistication.
What's next for Shopalyze
Moving forward we plan to implement several additional features to further advance Shopalyze. This includes ability to read customer sentiments real-time to understand how customers are experiencing products or services upon first interactions. This enables businesses to adjust their investment plans accordingly - whether that be adjusting their product/service offerings, changing communications strategy, or identifying other ways to improve customer experiences. Second, we plan to implement security features to promote Shopalyze as a multi-fold system and tapping into the global security market that is projected to increase to US$23.32 billion by 2025.