Every day, millions of Americans face the same problem; they don't know how to recycle?!?! Discrepancies between different states' rules about trash often lead to confusion in the general public about how to throw things away. As a group of college students, we understand why most people find it difficult to learn about recycling and take the extra time to learn the regulations. There has never been a solution to help people learn about recycling... until now!

What it does

Re, or Recycling Elevated, is a full-stack web app which leverages machine learning and computer vision, to help educate and make a fun recycling experience for users! People who join the site, can start by taking a photo of their trash and our machine learning model determines what method it needs to be disposed (recycling, compost, or trash). Users are also able to register an account to login to track the trash they have disposed. For every item recycled and litter disposed, users earn points which are tracked on a leaderboard system so that they can compete with friends to recycle the most!

How we built it

Re, is a full-stack, we application where we used React.js and JavaScript in the frontend, Python and Flask with SQLAlchemy in the backend and a PyTorch computer vision model in order to categorize images. React was used to dynamically render the data and accessing the web camera to allow device portability across multiple devices. This was connected with web requests to our Python Flask endpoints which implement a REST API to send data to our frontend based on specific end points designed for data. This Python backend server was also connected to our SQL database (we used sqlalchemy and sqlite for this instance) and also connected our computer vision model. We took our training data for the computer vision model from the Taco Dataset containing thousands of images of trash in different context. An interaction with the user with the frontend would be facilitated by the backend to connect to the model. For instance, the React.js frontend would be used to take a photo by the user and convert it to a Base64 string which would be sent over web requests to the Python backend where it would be decompressed from the Base64 string back into an image file where it then be run through the model to get the best predictions of what trash items it could be. This was then mapped to a dictionary of known recycling principles for these trash items and then sent back to the user as another JSON object contain data about the top three possible choices of what the trash item was, the confidence of each possible guess, and the output of how the user should dispose/recycle the trash.

Challenges we ran into

Some of the challenges we ran into include:

  • Difficulties with connecting to a cloud database service. Initially we tried to get a Google Cloud instance of the application but that became difficult to set up so we had to switch to a more realistic option with sqlite.
  • CORS or Cross-Origin-Requests which prevent requests from particular origins was something we were never quite able to figure out for our production model.
  • Masking with the machine learning model. Background images and items can some times make it difficult for our model to accurate predict what piece of trash there is and not having enough time to build a mask to filter out the irrelevant parts of an image prevented true optimization of the computer vision model.

Accomplishments that we're proud of

One thing that was exciting about this project was that a majority of our members interfaced with this technology for the first time! For example, some of our members were exposed to SQLAlchemy and backend development for the first time while others worked on the machine learning model for the first time! We had no idea how to integrate machine learning into a full stack web application so there was a lot of "learning on the fly." It was incredibly fulfilling when we were able to connect our model with the frontend and see it correctly identity a plastic bottle!

What we learned

Through the course of the hackathon, we were exposed to many new technologies. For instance, none of our members had ever set up a production SQL database before and being exposed to SQLAlchemy and SQLite allowed us to understand how data was stored more and plan our application design around it. We also learn about React Hooks and Lifecycle which became incredibly important for us to figure out how to manipulated complicated components on the frontend such as accessing the web camera, taking photos, encoding image data, and handling the outputs of a machine learning model. Our entire group had never written an application that incorporated machine learning to this extend and being exposed to the production process of how to develop a model from scratch and connecting it with other parts of the application. It was particularly interesting learning how to hand dependencies because the weight and requirements of Python Flask modules such as PyTorch requirement of complicated virtual environments and heavy dependencies.

What's next for Re (Recycling Elevated)

The next steps of Re would be to refine some of the parts of our application that we did not get the time to complete during the hackathon. The first thing would be to address scaling of the application by working on a more permanent hosting solution on some cloud platform and also to set up a more persistent database. We would expand what we were able to hand such as adding more data the a user's profile such as storing user photos and allowing for the display on pages to excite more users about recycling by facilitation connections! We also plan to add data aggregations via SQL queries to show more detailed statistics to inform our users about recycling and help the public improve analysis of recycling to improve the environment! We will also love to try and connect with more people with our application to excite others and hopefully provide a useful application for them to stay informed on how to recycle!

Share this project: