It's 4am on the first day of Hack the North, the team is tired, mildly cranky, but most importantly, hangry for Bear Paws. We walked to the closest snack bar, but alas none were there. Desperate, we made the journey to another floor, and another, until we finally found the last one at the last booth we checked. One team member exclaimed, breathless, "Geez golly! If only we knew that there were no more Bear Paws left!"
And this was the story of how Snack the North was born.
What it does
Snack the North is a hack designed to help your hungry programmer identify if their desired snack is available in their location of choosing using machine learning for image recognition, and various APIs for inventory cross-referencing. The interface is done through a slackbot, making it easy to communicate with.
How we built it
Snack the North uses the AWS DeepLens to identify snacks available on the table, pass the camera feed to IBM Watson, where the snacks are identified and counted. This count is then updated to the NCR inventory database, using their Catalog API. Finally, this information is then retrieved by a slack bot upon your humble programmer's request for a snack at a certain location.
Challenges we ran into
Turns out DeepLens didn't quite work for our purposes... DeepLens is designed to work with SageMaker, however, SageMaker makes it difficult to make a custom dataset, something that we needed in the project. At the same time, it was also difficult to gather enough pictures to build a custom dataset and train with ML.
Accomplishments that we're proud of
We were able to set up NCR and utilize the API to its potential. In our first iteration of the project, we learned A LOT about blockchain and smart-contracts. We're also pumped about having the interface done through Slack. Finally, even though we had to change our project idea 3/4 of the way into the hackathon, we were able to bring our second idea into fruition.
What we learned
We learned many ways to interface with DeepLens and AWS, train IBM Watson to categorize different snacks, Solidity and smart-contract integration, how to use the versatile NCR APIs and how to build a slackbot!
What's next for Snack the North
The next step for Snack the North would be to improve our snack detection system by switching to a more suitable camera (a webcam might be a good option for this). It would also be very helpful to have a larger dataset of pictures of snacks to train the neuro-network for higher accuracy. This isn't just helpful to hackers, but also makes snack restocking easier for the hackathon organizers. We also believe that Snack the North could be a great addition to industry, allowing employees of tech companies to keep tabs on the stock of their favorite treats. At the same time, it will also improve the restocking logistics that the facility managers have to do.