Inspiration

Our team wanted to create a product that would make lawn jobs much easier. One of the most annoying things about owning lawns is controlling weeds so we decided to make an automated process to help with that.

What it does

In conjunction with Weedy's mobile app, the Weedy robot is able to remove unwanted weeds through the help of the Weedy drone.

How we built it

Components:

  • Drone: takes several overhead images of a plot of land.
  • Wheel robot: Drives to a weed and pulls it out
  • Android app: displays all data and deploys drone/robot

The drone, wheeled robot, and the app are connected through the Firebase Real-Time Database. The drone reports the percentage of weeds relative to the plot of land and crop weed ratio. The robot outputs the number of weeds it pulled and classifications of weed types.

To locate and classify weeds, we used Yolov3 trained on the 6 most common weed types in Virginia. We augmented the dataset (changing lighting, blur, compression) to have the detections work in any environment. Because running Yolo is computationally expensive, the drone and robot make a request to a remote server on a Linux VM. The VM checks if the state has changed in FireBase and runs the Yolo model. Because we used Cuda GPU acceleration and firebase reads and writes data in real-time, the object detections occur in real-time.

The robots have a Raspberry Pi and the drone has a Jetson Nano. Both run a python script to update firebase and take pictures. The Jetson Nano calculates its position relative to the plot of land and records these coordinates in the database whenever it detects a weed underneath. We use OpenCV to draw a path for the robot to follow in order to pull out the weed.

Challenges we ran into

Trouble-shooting Android studios became very frustrating at times, especially when we were first getting started. Making the mechanism to pull out the weeds on the robot was also difficult.

Accomplishments that we're proud of

Having the entire system operate in real-time was something we are proud of. Running Yolo ono the Pi and Nano took far too long and tiny Yolo, an optimized version, did not generate accurate results. That’s why we migrated to a Linux VM and got CUDA working to have Yolo cooperate in almost real-time.

What we learned

Not only have we learned more about firebase, android-studio, xml, raspberrypi, and more, but we also learned about the streangths and weakness about dividing and conquering. We learned quite a lot about Android Studios (many of our group mates are familiar with SwiftUI). We also learned about systems administration when setting up CUDA.

What's next for Weedy

A function that will colelct the actual weed and store it for disposal so that weed cannot regrow and making the path calculation for the wheeled robot more efficient.

Share this project:

Updates