Facebook Messenger Bot
User Onboarding

HastyPick

Seamless 'just walk out shopping' for small stores, built with off-the-shelf technologies.

Inspiration

After reading a Medium post about making your own Amazon Go and seeing the huge press they've gotten, our team realized how significant the impacts of this technology will be. The uncle of our team member Raksha, owns a small convenience store in Bangalore, India that would be threatened by this technology in the future. Our goal was to develop a low-cost 'just walk out shopping' system that would work in shops like his around the world.

During this process, we realized that there is a huge opportunity to help brands better understand consumer behavior around their products in the store and improve their advertising, packaging, and messaging. By using data available from the sensors we're utilizing, we will develop a massive dataset which can provide actionable intelligence, as more stores adopt HastyPick.

What it does

Our device has three key functions: 1) product movement detection 2) customer tracking 3) automated billing

How it works

We connected a raspberry pi to a load cell to detect when a product moves on the shelf. This signals a camera on top to match the customer to the product(s) taken and add them to their virtual cart. Once the customer leaves the store they are sent a receipt on Facebook Messenger.

How we built it

Facebook Messenger Bot
Kairos Facial Recognition API
Google Firebase
Raspberry Pi
React Native

Challenges we ran into

One major challenge was packaging all this functionality into a discreet device. If we had access to additional resources, we would fabricate a custom body and use smaller components to make the device less bulky and more space-efficient.

Another challenge was speeding up the facial recognition part of our process. Part of the problems is that we are only working off one image of the customer and that there are multiple steps between taking an image and getting a response that results in significant lag.

The last big challenge was integrating the hardware and software components of the project. The unique wifi situation left us tethering our Pi to someone's phone, using another phone to capture images, and trying to ensure all our different services are running correctly.

What's next for HastyPick

Making it better: 1) Capture and analyze consumer behavior data for brands 2) A sleeker, more discreet design 3) Reduce lag in the system 4) System improvements for larger scale retail spaces.