Inspiration
Farmers and gardeners usually find crop disease by walking the rows and looking. By the time a leaf has visible spots, the infection has often already spread. Stressed plants also release volatile organic compounds (VOCs), sometimes before you can see anything wrong. We wanted a small, cheap robot that does the walking for you. It looks at each plant, smells it, and marks anything suspicious on a map, so a person only has to check the plants that need it.
What it does
LeafLink is a plant-health scouting system with two halves:
The rover (plantbot): a Raspberry Pi 4 robot with a webcam. You drive it from a phone or laptop dashboard, stop at a plant, and press Scan. It takes a photo and sends it, along with its sensor stream, to the LeafLink server. A 500 ms watchdog stops the motors if the controls are released, the tab is closed or the connection drops.
The LeafLink dashboard: a web app that runs the vision model on a GPU laptop. For each photo it: removes the ground and background with a SegFormer-B2 segmentation model, splits the scene into individual leaf candidates, classifies each leaf as healthy or infected with a CUDA FP16 classifier, flags the photo as an event when enough of its leaves look infected, and pins it on a field map.
The dashboard has a Field Overview tab (map, rover path, VOC and vision event panels), an Images tab for browsing every scan, and a Rover tab that lets you drive the robot with live video straight from LeafLink.
We also designed a VOC “sniff” channel. A gas sensor at the plant is compared against a background sensor at the back of the rover. A smell both sensors pick up, like the room or dead leaves on the ground, doesn’t count against the plant. Only a sustained rise at the plant triggers an anomaly.
How we built it
Hardware: Raspberry Pi 4B on a USB power bank, a TB6612FNG motor driver on a 9 V battery driving two DC motors, and a Logitech C270 webcam. The chassis was designed in parametric CadQuery and 3D printed. It went through three versions (v1 → v3), each simpler than the last, ending with flat plates, blocks and posts that print without supports.
Rover software: one Python process on the Pi handles motors, camera, sensors, the scan state machine and a FastAPI dashboard. Every device sits behind an interface with a mock version, so the whole team could develop on laptops with --mock while the hardware was still being built. If a device fails at startup, the dashboard shows it red and swaps in the mock, so a dead sensor never stops the rover from driving. We deploy to the Pi with an rsync script and run it as a systemd service.
Vision pipeline: SegFormer-B2 for ground removal, a heuristic leaf splitter, and a fine-tuned classifier running in FP16 on an RTX 3050. Healthy/infected is the only output we use; we don’t identify plant species.
LeafLink server: Node.js with a Python inference worker and SQLite. It handles image ingestion with retries, threshold snapshots, the event and cache lifecycle, and per-device VOC telemetry. It has 27 regression tests. The frontend is a compiled web app with a hand-drawn, sketchbook-style design. Sponsor integrations (written, gated behind API keys): Gemini “Plant Doctor” diagnoses and natural-language driving commands, Tiger Data hypertables for the VOC stream, Solana devnet certificates for each scan, a Snowflake sink, and a Trace PCB design for a single rover HAT.
Challenges we ran into
The model can’t run on the Pi. The classifier needs CUDA, so we split the system: the rover captures images and LeafLink does inference on a teammate’s GPU laptop over the network.
Speed. Full-image processing takes a median of 15.5 s per photo, because the segmentation model currently reloads on the CPU for every frame. Fine for stop-and-scan, but too slow for continuous driving. False positives. On a 20-photo test from the PlantDoc dataset, the full pipeline caught every infected photo (100% sensitivity) but also flagged most healthy ones (22% specificity), for 63% accuracy overall. Background leaves and leaf-splitting artifacts are the main cause.
Hardware versus time. We designed for 4WD, a VOC sensor pod, ultrasonic auto-stop and a tactile leaf probe. In the end we built a “skateboard MVP”: Pi, motor driver, two motors and a webcam. Both motor channels share the same control pins, so the rover drives forward and back but can’t steer yet. CAD iteration. Our first chassis designs were too complex to print in the time we had, so we cut them down twice until every part printed flat and fast.
Accomplishments that we’re proud of
A working end-to-end path from the rover’s camera to an infected/healthy verdict pinned on a map. Driving the physical rover from the same dashboard that shows the results.
A mock-first architecture that let 3 people build software, ML and hardware in parallel without blocking each other.
Honest evaluation: we published our confusion matrix, per-photo timings and test methodology instead of a single flattering accuracy number.
A parametric chassis where a new measurement is a one-line change.
What we learned
Detecting disease on cropped, lab-style leaves is much easier than in a real scene. Isolating the right leaf matters as much as the classifier itself.
Report more than accuracy. Sensitivity and specificity told us exactly where the pipeline fails.
What’s next for LeafLink
Give each motor its own pins so the rover can steer, then add ultrasonic auto-stop and row following. Wire in the MQ-2 and BME680 sensors and calibrate the plant-versus-background VOC differential on real healthy and stressed plants.
Keep the segmentation model loaded on the GPU to bring per-photo time well under a second. Retrain on photos taken by the rover itself to cut false positives, and test on a properly held-out set. Build the Trace HAT so the breadboard of modules becomes one board, with an RP2040 handling the motor watchdog.
Test it in a real garden row instead of on a table.
Built With
- 3d-printing
- cadquery
- css3
- cuda
- express.js
- fastapi
- gpiozero
- html5
- hugging-face
- javascript
- node.js
- numpy
- opencv
- python
- pytorch
- raspberry-pi
- segformer
- sqlite
- systemd
- vite
- websockets
