What Goes Where?

Inspiration

Prior to 2017, the United States completely outsourced its waste management system to developing countries. In the past 3 years, there have been drastic changes to this trend. China and many other countries in Asia banned waste exports from the United States. The country's underdeveloped waste management system is left with a huge amount of waste to deal with. Our group was inspired to do something about our unsustainable economy. We want our nation to develop a circular economy, where resources can be used efficiently and effectively through a mature system that sorts and manages waste. Upon meeting with representatives of Cal Zero Waste, we learned that if waste is not sorted properly, it can impose a significant challenge for compost and recycling centers. The challenge is so great that some centers refuse to waste that might have been cross-contaminated. Meeting with our peers, our group realized that many people wanted to contribute to a more sustainable future, but lacked the knowledge to sort waste properly. Our group thought that we could solve this problem by enabling people to effortlessly identify the types of trash and the correct ways to dispose of such trash. It was then, the four of us came up with the idea to create "What Goes Where?".

What it does

"What Goes Where?" is a mobile application that enables users to effectively recycle by identifying and sort trash.

How we built it

We split the work on this project between the back-end and front end. Where the front end was on Expo in React JS while the other half of the team worked with the Google Computer Vision and Google Cloud Storage APIs. We were able to implement camera and image modules in Expo for our front end to send the captures or selected images back into firebase storage, which would set an Http post request to the Google Computer Vision API. This API allowed us to look for key labels, which were then sent into our classifier to determine the confidence in a photographed item's recyclability.

Challenges we ran into

The biggest challenge that we faced is the inconsistency in the ability of the Google Cloud Computer Vision API being to read public HTTP URL's. This issue has been documented by multiple users on stack-overflow and Google's community development forum. It took a lot of time and effort to debug and determine that the issue was not caused by our program, but rather the API's incomplete response to the data we gave it. Nevertheless, we were able to find a workaround using 64bit encoding of images and passed that into the Google Computer Vision API. This is likely a future addition to the project as we were limited by time constraints. (We will future elaborate in "what's next for What Goes Where?”) Issue: https://github.com/googleapis/nodejs-vision/issues/270 Our workaround: https://base64.guru/developers/javascript/examples/convert-image

Another challenge we ran into involved connecting the front end and back end of the program. Our group did not have the knowledge nor experience to design a smooth architecture of our front end and backend. We had to learn on the fly. We ran into problem after problem, but we were able to adapt and overcome these hurdles on our path. By choosing to use Expo to develop the UI of our app, we gained the ability to simultaneously implement the app on both Android OS and iOS, but we were limited in our options when debugging. Furthermore, we ran into an issue with transferring the images from Firebase to the Google Computer Vision API.

Accomplishments that we're proud of

The four of us are very proud of how much we learned in the process of building something that non of us had prior experiences in. The fact that we created something from scratch using a language we weren’t comfortable with, building on a platform we were unfamiliar with, and integrating API’s that were introduced to us on the opening night of CalHacks. The four of us were deliberate in choosing to "seek discomfort” so our learning would be unrivaled.

What we learned

For the front-end we learned a significant amount about starting from a very simple react js template for a web app and turning it into a cross-platform application that can integrate both a storage database as well as a computer vision API. At first we hoped that we could just connect the computer vision back-end to the react js front end, but realized that this would only work for a single platform because of the way the API was designed. Therefore, to keep our accessibility high, we chose to find a workaround. We first researched and learned how to use a database to just store primitive data types, and then furthered the use of the database so that we were able to upload images and create a public HTTP URL that the API could access. The extensive amount of network features we had to develop for this application served as a highly valuable learning opportunity for our group, especially since our experience up to this point was mostly concentrated to each of our local machines.

What's next for What Goes Where?

In the future, we hope to integrate AutoML Vision in order to create our own machine learning classifier that would be more specific and efficient than the more basic vision classifier offered by Google Cloud’s Computer Vision API. With AutoML Vision’s Image Classification API, we would be able to train a custom model that classifies the images instantaneously, rather than sorting out labels given to us by the Computer Vision API. In addition, we could also use AutoML Vision to build a model that is able to detect multiple objects in a single image, which would make the process even more efficient.

Looking even further into the future, we hope to eventually automate the waste sorting technology by implementing it in the waste bins themselves. By converting this into an effortless process, we hope to create a lasting change in how waste management is optimized.

Built With

• expo.io
• firebase
• google-cloud
• google-cloud-vision-api
• javascript
• python
• react