Inspiration

OVER 1 MILLION MARINE ANIMALS ARE KILLED EACH YEAR DUE TO PLASTIC IN THE OCEAN. It is clear that littering and human ignorance is a major threat to the environment. Due to very tight funding and limited manpower, environmentalist groups often find it difficult to clean a good amount of litter. This is what led to the creation of Littermap. Cities and counties across the globe have task forces to combat litter and illegal dumping. Yesterday, teams had to drive city streets in search of litter all the while wasting fuel and time searching for litter. Today, Littermap gives these teams the capability to go exactly where the garbage is without having to roam streets not knowing where to go. Littermap is open-source and is capable of running on almost any device with a camera.

What it does

Littermap offers environmentalists a way to detect litter hotspots around the globe. Although there are already existing apps that can detect litter, none of them are able to display this litter data in a real-time, global setting. Littermap is a web application that detects the amount of trash in a specific location and maps this trash data, along with others, on a global heatmap. In the application, the user can take a webcam photo of a littered area, and this photo gets passed into our Flask backend where our PyTorch machine learning model analyzes it for the amount of trash. A geolocation API gets the location of where the photo was taken, and the output of the ML model and the coordinates returned from the API get stored in a Firestore database. When the page is refreshed, an updated Google Maps heatmap can be found at the bottom of the app. Our initial plan was to use a DJI Spark drone to demo but we were unable to fly because of COVID-19 restrictions in our area.

How we built it

The frontend of Littermap is built entirely with Svelte, and the backend is built with Flask. The ML model is built with PyTorch and we used Python to pre-process our training images. A Firestore database is used to store trash and coordinate data, and we used Google Maps' Heatmap API to chart this data.

Use of Open-Source Software

We used a variety of open-source software projects to make Littermap possible. The main one being YOLOv3 which stands for You Only Look Once. We used https://github.com/ultralytics/yolov3 as a reference and adjusted code to fit our needs and accommodate for our hardware. We were also going to use DJI's opensource SDK to transmit live video from a DJI Spark drone but due to time constraints and COVID-19 complications we were unable to fulfill this goal. Last but not least we used https://github.com/pedropro/TACO/ to download our training data.

Challenges we ran into

The main challenge we ran into was setting up our YOLO (You Only Look Once) machine learning model. We used TensorFlow to train our model but quickly ran into problems with GPU memory errors as well as a problem with tensorboard which forced us to downgrade to TensorFlow 2.0. A challenge for the frontend was working with asynchronous tasks in Javascript, as there were countless times that crucial variables did not return anything after running an asynchronous function that would give a return value for the variables.

Accomplishments that we're proud of

With every Machine Learning based hack, it is always a relief to see it work as intended. We were especially happy to see our Machine Learning loss was exceptionally low given the small amount of data we had as well as our short time frame. At the end of the day, a finished product no matter what it's function may be is always something to be proud of.

What we learned

Going into this hack we all had different skill sets. Our team consists of Frontend, Backend and Machine learning enthusiasts all of which are very experienced. This hack forced us to work on parts of the project that we don't usually work on and that helped us to learn something new. For example, people experienced in building the backend of the application also worked on machine learning this hack and they learned something new. Our teammates responsible for the frontend explored with and learned to use a couple of new APIs for the project, including the Google Maps Heatmap API. Each and every one of us will leave this hack with a new skill as well as an improvement in our teamwork skills and problem-solving skills given the circumstances of remote collaboration due to COVID-19.

What's next for Littermap

We plan on enriching our machine learning model even more with even more data and time to train. We also are planning on making our setup much more intuitive so users can more easily use our software without any fuss.

Built With

Share this project:

Updates