Scantry

App interface
Website with grocery list

Inspiration

We wanted to build this app for two main reasons. First, at this hackathon we were able to listen to a presentation about Machine Vision, which we found incredibly interesting. We decided to focus to create a project that could integrate and apply Clarifai's Machine Vision API in order to demonstrate the power of this concept.

Second, we love food. But in all seriousness, the issues of the tons (literally) of food waste created every day due to expired goods, as well as the lack of a simple and accessible way to cataloguing a personal inventory of such groceries, provided us with an opportunity to solve them and create something new. We also wanted to extend our project so that users could find cooking recipes using ingredients they already have.

The combination of these two factors thereby led to our idea, a simple management system taking advantage of a new technology. The choice was as easy as eating another slice of cold cheese pizza.

What it does

Scantry is comprised of a few elements interacting with one another-to illustrate how they work, let's imagine how Scantry would work in a situation where its user just bought some groceries. The first element is a mobile app, built with Expo.io. The user uses their camera to scan the grocery items in front of them. Clarifai's Machine Vision API then identifies the different items (like an apple, orange, or soup can) and records them down.

Data is then pushed real-time to Firebase. Finally, a website draws data from Firebase to display this information online, which the user can access. In this way, the user can create a catalog of what they've bought simply by scanning items with their phone, and can similarly very easily view the list of food they already have. An added feature is that Scantry will also provide the user with possible recipes they could create with ingredients they already have.

With this, the website can also come up with recipes that fit the catalog of food listed in the database. This can be done through if statements and combinations (an apple and an orange in our fridge could make a fruit salad!).

How we built it

We used the following technologies:

Implementation of Clarifai's Machine Vision API to identify and categorize items
React Native through Expo.io
Database hosting on Firebase
Website deployment using Bootstrap/Netlify

Then there's your usual HTML, CSS, and JS.

Challenges we ran into

Getting the Clarifai API to work was not easy. In fact, as one of the team members is writing this README another is still trying to troubleshoot the code. However, in the end we were able to create a functioning app that used the API, which was a big accomplishment for us.

Also, because we are bad at brainstorming ideas, this project was built in 10 hours more or less. So time management was a big thing too.

Accomplishments that we're proud of

As mentioned above, using the Clarifai Machine Vision API was a great hurdle to leap. Getting to integrate it into an app and having it function was probably our biggest achievement at this hackathon.

We're also beginner hackers. All of us have only been to one hackathon beforehand, so getting to create a working project like this one is tremendous in itself.

What we learned

Integrating and using Expo.io, and the Clarifai API
Learning how to deal with the concept of APIs in general, and how they can be implemented.
Working with Firebase

What's next for Scantry

Given some more time, we would like to make the user environment a little more friendly, like adding some more style to the interface. We'd also like to set up a function that suggests recipes users can make using the groceries listed in their database. While ambitious, another goal would be to add functionality with Google Home/Amazon Alexa devices, so that users can access and edit their grocery lists via their smart home devices.