Track: Sustainability

Team

Prince (discord username: blackprince#0072)

Rasha (discord username: Rasha(she/her)#1414)

Summary

Our robot turtle Herman, is going to save the ocean like a superhero! Herman is designed to detect, monitor, and quantify marine debris with the use of machine learning algorithms, real-time high precision object detection programs and an eco-friendly look in the ocean; because the fishes need to feel safe too. Herman serves to provide safe automated surveillance in the ocean and also report data essential for predicting waste hotspots based on other factors like ocean current, temperature, and concentrations of algae that develop due to the presence of waste and greenhouse gases -- which is very important in solving the challenge.

Inspiration

We realized that there wasn't any map that shows you the concentration of wastes dumped in the water bodies and due to this most of the people were unaware of the fact, as to how badly the marine debris is affecting the organisms within as well as we human beings. So we wanted to communicate with the mass about this buried and serious issue of marine debris by building a live map that would monitor the amount of plastics and their age; hence we chose this problem statement.

How We Built it

Our project is divided into two sections : the hardware (the model that's structured in the form of a turtle and likewise would be released on to the water bodies) and the software (the server and user end part that would update the map information regarding the concentration and hotspots of marine debris).

Hardware

Model view of what we intended to 3D-Printed. Our hardware on utilizing a developed AI route-mapping program that uses an OpenCV trained image data Tensorflow model to detect, monitor, and quantify plastic waste under the ocean and report instances of found material as API queries to the server. Live frames from the raspberry pi camera modules will be analyzed for signs of plastic waste from trained data on how plastics in the ocean will look, so we enlarged the eye openings for more space for the ESP32 cams. Output data will then be recorded and saved on the raspberry pi model B storage device. The Quantified data will include picture frames of debris in the ocean, which will be classified by the model and updated to our server. Our hardware model will be Integrated with the Saildrone(NASA) platform API for the sole purpose of sending parsed data from observational routines in the ocean to a server for live map showcase and analysis. Circuitry and body parts for construction:

  • ESP32 cams - 4 pieces for the anterior part of the turtle and for the pivot areas of the turtle propellers.
  • White LEDs for illumination under the sea.
  • Raspberry pi 4 - running the TensorFlow model and sending feedback through saildrone. Covered by a 3D printed water resistance shell-like case.
  • Wheels - Servomotors -> mobility in the ocean using tracked distance mapping and pathfinder.

Software

  • Initially we started with creating a web based UI for our web app. While using our webapp, our end user would initially encounter our home page, which has a navigation bar, to redirect you to pages that contain our live map which shows the hotspots of the shores where marine debris were identified, virtually reality based hardware model, documentation (giving a brief insight about the available datasets) and the creators of this project. The UI was built purely through HTML,CSS and JS.
  • The live map was developed using ArcGis Software tool and the visualization for the datasets were made using Jupyter Notebook.
  • For object detection we used tflite model(single shot detectors).
  • Furthermore, we would be integrating multiple datasets from the NASAs portal, so that it can contribute in finding the hotspots of the marine debris and also develop a roadmap for the hardware model. For eg : the sentinel data obtained from esa that shows the chlorophyll concentration in the water bodies, can help us predict which zones are likely to have wastes. Greater concentration of chlorophyll, means there is a greater chance of the debris being identified. Also the data from NASA can be used to measure the oceanic temperature and the greenhouse gas rate and likewise predict the chances of marine debris being there.

Challenges we ran into

Challenges faced during the design

  • We faced a couple of challenges for selecting the components for hardware as we wanted to make it as much as environmental and pocket friendly.
  • Choosing the algorithm was a difficult task and we had to resort to a heuristic approach
  • During code execution we faced a couple of problems as our systems didn't provide a very high computational speed .
  • Sometimes our model gave wrong results or very less accuracy, for eg: when our model was shown the picture of a plastic bottle it detected it as a plastic bag
  • It was quite hard to find accurate datasets and also cleaning the datasets took a long time as the datasets were huge
  • Direct datasets regarding the marine debris weren't available. So had to resort other way and perform analysis and assumptions from the existing datasets. Also, the software tool we are using to develop a map that said the concentration of marine debris on the shores only supported importing a limited amount of data

How we plan on advancing our prototype

  • Initially we used the datasets from noaa that contained the data on the debris waste updated by the app users and likewise we optimized and built a precise map that shows the sea shore hotspots of marine debris along with a detailed explanation of the type and concentration of the waste found.
  • We would be integrating all these datasets and henceforth derive a predictive model that would predict the concentration and hotspots for marine debris and build a road map for our hardware model.
  • We would be integrating multiple datasets from the NASAs and other space agency's portal, so that it can contribute in finding the hotspots of the marine debris and also develop a roadmap for the hardware model
  • The sentinel data obtained from esa that shows the chlorophyll concentration in the water bodies, can help us predict which zones are likely to have wastes. Greater concentration of chlorophyll, means there is a greater chance of the debris being identified. Also the data from NASA can be used to measure the oceanic temperature and the greenhouse gas rate and likewise predict the chances of marine debris being there.
  • The sentinel-5P data obtained from esa would allow us to monitor the tropospheric nitrogen level over the ocean and derive assumption about the concentration of the waste. The same applies for the methane level tracker datasets obtained from Global Monitoring library. We would combine these two datasets along with other datasets from NASA such as the NASA Ocean Biogeochemical Model assimilating satellite chlorophyll data(that measures the oceanic temperature), the polar path finder data from nsdic that monitors the motion of the ice (as Sea ice accumulates more than its fair share ) so as to monitor where the plastic waste is coming from and henceforth build a roadmap for our hardware model, the information from airsar along track giving us an idea about the oceanic wave speed and direction.

What we learned

Our team compromised of diverse backgrounds. So in a nutshell, we got to learn a lot from each other and also were able to help the other when they were stuck at some instance. We also expanded our idea, by considering and putting forward each one of our thoughts . Through this approach we were also able to resolve the problem we were facing. For eg: when one of our team member had difficulty in visualizing the data , the other would work along with team member and together they would solve the problem , quickly as well as precisely

Conclusion

Since our team is majorly focused on fusing tech with climate change , we hope to advance and take our project further, so that as human beings we can utilize our potential and play a part in fighting climate change

Source Code

GitHub Repository

Share this project:

Updates