Add item screen with image recognition
Confirm item screen
Pickup details screen
About 1/3 of the food produced for human consumption in the world is wasted. Last year, the average Canadian consumer threw out an estimated 170 kg of food. Food waste has become an extreme market inefficiency and has taken a toll on our environment. We created this app to counteract this issue by making it EASY for people to reach over to their phones and tell Potato Hub that "Hey, I have about half a dozen of eggs leftover and a gallon of milk that I haven't opened yet that I know I won't be consuming."
What it does
Simply take a picture via our mobile app, or let Google Assistant or Google Home know about the food you have in your house which you won't be eating. Image classification and speech recognition will help you fill in all of the details. Your item along with a "pick-up" location and an expiry date will be posted to a local community group, and from there someone in your designated hub can choose to request the food item either manually by clicking on a button in the app or by talking to Google Assistant. After the request has been sent and confirmed by the poster, you will choose a meet-up point and time to exchange the food.
How we built it
We created a speech recognition AI on Actions on Google and trained it on DialogFlow to enable it to have natural language conversations with people. It can be accessed by Google Assistant on mobile phones and by Google Home.
We built a native Android app in Kotlin and utilized Retrofit to make network calls. The Android app allows users to add items to the marketplace, browse it, select items that they want to receive, and confirm their selection in order to access the pickup details (e.g. when and where).
We utilized the Google Image Recognition API to recognize keywords that can be used to categorize user submissions based on the photos they post. This makes the experience more seamless and will enable fast categorization and searching of the inventory in future iterations of the app.
Our backend is a RestAPI server built with Python. It provides endpoints and database support to best suit the project's need. It also abstracts our API calls to third-party services like Google and NCR. It is hosted on a Google Compute Engine VM with a Gunicorn set up for a Python API server, an Apache2 for reverse proxy, and https for better security and interaction with Google Cloud services.
Challenges we ran into
One challenge arose in trying to integrate our application with the NCR API. We had a very difficult time sending images across http requests between the Android app and Python server. Instead, we kept the images on the server side and explored the Google Vision API for determining image labels on the client-side. We followed a tutorial, wrangled with dependencies and API keys until we finally got a response back containing the labels for the images. Android network requests with solely JSON also proved to be difficult. Part of the trouble came from deprecation of old libraries and the shift from Java to Kotlin. After many failed attempts and lots of time on Stack Overflow, we managed to create successful POST and GET requests to our server.
Accomplishments that we're proud of
The proudest moment for us was when we were able to take a picture of a banana and have a text field in our app populate with the string: "banana, banana family, produce, food, fruit, cooking plantain, local food." We almost cried happy tears (one of us actually did). The first high-five occurred when Google Assistant was able to send a message "Hey Google, I have some extra eggs to give away." and the server received it and added a new product into our database with an image and the name "eggs."
What we learned
We learned how to create a Google project on the cloud, get API keys, and make requests to harness the power of Google's machine learning APIs.
What's next for Potato Hub
Our goal is to have Potato Hub be a community-based app where each location will have a couple of "Hubs" for people within a certain radius to join. These could include "Fitness Enthusiasts of Waterloo" or "Cooking Lovers of the 6". Food is a unifying aspect of our lives and brings people together. Increasing interaction with those in our communities and creating a social marketplace will create a better environment. Potato Hub has the potential to bring together communities as a social network of sharing food. In the future, we would like to add location-based recommendations and user profiles.