Summary

Flying Squirrel is the gliding hero of the earth! The project uses a programmable drone to scan an area for environmental hazards like littering, water pollution, wildfire, animal trafficking, drought, and soil erosion. The project uses the drone to capture pictures of the area in real-time and evaluate if there's any hazard by using Cloud Vision with Machine Learning.

Inspiration

The 50th anniversary of Earth Day is not complete without supporting our planet with a cool project. We have developed Flying Squirrel to be Earth's guardian. Our drone glides just like a Flying Squirrel to detect and report any dangers to our planet. Developing an innovative and meaningful project is the perfect way to celebrate Earth's day!

What it does

Flying Squirrel uses a programmable drone to fly around an area, detect potential hazards, and report them. The drone has a built-in camera that takes pictures in real-time and sends it to our webserver to decide if the picture contains an environmental hazard. If an environmental hazard is found, we update the dashboard with the location of the drone, picture, and hazard category to improve reporting and increase the speed of mitigation.

Our hazard categories describe what type of danger our little friend, Flying Squirrel, has found. The main categories include Pollution, Fire, Animal, and Geological Hazards. Each category allows us to identify the following environmental problems:

Potential Pollution Hazard:

  • Littering found on the floor/grass
  • Littering found on a body of water
  • Waste found in the picture

Potential Fire Hazard:

  • Smoke found in the picture
  • Fire detected on trees & grass
  • Active Volcano detected
  • Explosion detected

Potential Animal Hazard:

  • Wounded animals covered in Blood.
  • Caged Animals (Animal Trafficking)
  • Trapped Animals (Animal Trafficking)
  • Netted & Chained Animals (Animal Trafficking)

Potential Geological Hazard:

  • Soil Erosion
  • Inundation
  • Drought

It's as simple as fly, detect, report!, and mitigate.

How we built it

We build this project with a lot of patience, caffeine, and social distancing! We started with a simple idea of using a drone to identify if trash was found in a picture. We used a programmable drone with a Node.js and Socket.io server to make the drone move and take pictures autonomously. The pictures of the drone are saved on the server and used to evaluate any potential hazards. To classify the pictures, we started playing around with a simple dataset of littering images and an open-source machine learning algorithm for trash annotation. However, we realized that our project could be scaled up to the next level by using Cloud Vision. Therefore, we implemented Google Cloud Vision to our project in order to provide accurate labels and object annotations. By doing this implementation, we were able to narrow down the problems we want Flying Squirrel to identify and report them quickly, and directly to the dashboard.

Challenges we ran into

Time and distance were the biggest challenges! We spent quite some time looking at datasets and image annotation algorithms that were replaced for an algorithm in the Cloud. Due to the Coronavirus, my teammate got stuck in the airport because his connecting flight was canceled. In the meantime, while he worked on Flying Squirrel in-between flights, I worked to fully implement Google Cloud and machine learning algorithms in our platform. We lost some time but my teammate was diligent and we worked together to overcome this challenge (Teamwork makes the Dreamwork!) Besides that, it was our first time using Google Cloud Vision in a project, so we had to dig down the documentation to get the proper image properties and ensure accuracy and speed. As this was not enough, we had connectivity issues with sockets.io and the programmable drone, limiting the speed of the drone. It was also very challenging to interact with the drone's stream data while simultaneously getting regular screenshots of the scanned area with potential environmental hazards and classifying this data with one of our machine learning algorithms.

Accomplishments that we're proud of

This is our biggest project so far, and we are incredibly proud of working on Flying Squirrel. We implemented way more features than we could've possibly imagined in such a short span of time! This was our first hackathon project using Machine Learning and Cloud Vision, so it was super exciting to learn new things while developing Earth's gliding guardian! Besides that, we are proud of our organization regarding all the challenges we overcame.

What we learned

We learned that funky challenges can be solved with a lot of patience, clear communication, and Googling! We learned how to fully operate a drone from the computer by streaming and receiving data in real-time. In addition, we learned that Cloud Vision is an excellent way of taking a project to the next level without building our own datasets and models. Besides that, we learned more about environmental hazards and their consequences on our planet.

What's next for Flying Squirrel

Flying Squirrel's team is working on a collaborative platform to allow developers to use our algorithm to learn more about environmental hazards. With that, people will be able to use their own drones and pictures to classify their surroundings and expand Flying Squirrel around the globe!

Video Demo URL

https://www.youtube.com/watch?v=RKuu5RTdNZc

+ 22 more
Share this project:

Updates