For Hack Princeton 2018 spring, we decided to hack an Android app that used image recognition to classify objects as trash or various types of recycling. When you break down the many rules of recycling, it comes down to a lot more than what number is on the bottom of a plastic bottle. Most college students can't even name which numbers are collected by their town's recycling collection. We aim to simplify the many rules of recycling by training our classifier to identify images and then query Wolfram to determine whether or not it's recyclable. This project is easily expanded by identifying more materials and items that have special recycling rules and directing people to nearby recycling locations to dispose of them in a convenient and environmentally conscious way.
What It Does and How We Built It
RUcycle consists of three components: an Android app, a Stdlib interface, and a Wolfram custom Classifier. The Android app uses the phone's camera to take a picture of an object. This picture is passed using HTTP through Stdlib to a Wolfram Classifier model. The model then returns what type of material is in the picture, and passes it back to the Android app through StdLib, which then tells you whether the object is trash, recycling, or electronics. If the item is electronic, the app will take you to Google Maps and show the nearest electronic recycling stations.
We created the Wolfram Classifier using labeled training data to learn a variety of materials including plastic, metal and cardboard. Some of the data was found online, but we also had to create our own dataset. Any materials that are not recyclable are grouped under trash which used training data of common trash items such as chip bags and candy wrappers. This data is then returned to Stdlib which relays the material type back to the app. We also wrote a function in Stdlib to query Wolfram's search engine with an unknown object type for additional data on a material.
Choosing Our Technology
Stdlib: In the initial iteration of our idea, we wanted to connect our recycling identification app to a hardware auto-sorting trashcan. Stdlib offered us a flexible way to implement our identification functions in an easily accessible, lightweight and platform-compatible manner. Although we drifted from our original hardware plan in favor of advancing our app's additional capabilities, Stdlib provided a seamless way to call our various functions in any platform, which means any future features and improvements, either hardware or software, just need to call one function to get our recyclable functions such as the material classifier.
Wolfram: Prior to today, we weren't even aware that Wolfram offered image recognition tools. At the beginning of the hackathon, we used Wolfram's built-in image recognition classifier. However, after tinkering with the various specificity levels, we decided that it's method of identification was too broad for our purposes and decided to write our own classifier. Much of this data was collected manually. Writing this new classifier in Wolfram fit well since the access and return methods proved the same as the original built in recognition but this new classifier was much more accurate at discerning the particular material of most common items.
Android: We wanted to implement the platform in a way that was easy to use by everybody, and decided on a mobile app. Since most of us had an Android phone, we decided to develop in Android Studio. It was great for rapidly prototyping the app which we needed to test several different ways due to the modular features we were individually working on.
Challenges we ran into
Glass vs Plastic It was really difficult for our classifier to tell the difference between glass and plastic leading to a lot of mislabeling of water bottles, a common item to be recycled and in a different manner than glass items. We tweaked our training data set to have less bottle shaped glass to make the common case more reliable.
HTTP Requests We faced a LOT of issues with sending HTTP requests from the Android app to Stdlib due to the required input type of a base64 image string. After trying many many different techniques, we were able to figure out a way to use Java's built-in HTTP library to query the Stdlib.
Accomplishments that we're proud of
We were very proud to have taken 3 different unfamiliar technologies, Android Studio, Wolfram One, and StdLib. It was initially very difficult to integrate them but we're very happy with how seamlessly the final project is able to access the Wolfram mathematical model, all the android's user interface tools by communicating with StdLib.
We were especially proud of integrating Android and Stdlib using HTTP. This was the most difficult connection to make, and we were learning HTTP for the first time. Our code also helped the developers of StdLib have more documentation for Java and Android connection.
What's next for RUcycle
More training data is easily added to the classifier under labeled folders. By expanding the data used by the classifier, not only can the accuracy of predictions improve, we can also label a greater number of materials and provide more detailed information about various recycling methods. Certain items are eligible for company specific recycling programs including non-traditional items such as shoes, inhalers and cosmetics that help the environment and may contribute to other good causes.
RUcycle can also be put in use on the industrial side in waste management plants to use computer vision to sort through recycling and trash.