Inspiration
A strawberry grower can buy a weeding robot today. What they cannot buy is a better version of the one they already own.
Every embodied system ships with a control policy, and the moment it is deployed, improving it becomes someone else’s specialist job. The operator owns the weights and has no path forward. The four things they would need — domain knowledge, a labelled dataset, a reward function, and training infrastructure — are exactly the four things they do not have, and a consultant who does have them charges for months of work.
We kept coming back to one observation: of those four, three are already solvable with infrastructure that exists. The fourth — writing the reward function — is the one everyone treats as irreducibly human. We wanted to find out whether it actually is.
What it does
Accelerate.ai turns one sentence into a measurably better control policy.
The user uploads the weights their machine already runs and writes something like “pull pigweed and purslane seedlings in my strawberry beds without touching the strawberries.” Seven agents take it from there:
- Foreman parses the sentence into a structured task spec — what to act on, what to protect, what to prioritise.
- Scout reads the open web through Bright Data and extracts quotable measurements, each stored with the page it came from.
- Curator mines and verifies an image set for this specific task, including deliberately adversarial look-alikes, and fits a perception probe.
- Architect writes the reward function — actual Python — from the priorities in the request, then dry-runs it before anything trains on it.
- Trainer runs PPO across parallel Daytona sandboxes, 32 vectorised environments each, several reward variants racing simultaneously.
- Judge reads the per-component reward breakdown, rewrites the reward function, and goes around again.
- Report explains the outcome in plain language, with every figure traced back to the database row it came from.
The key design decision: the reward function is the thing being searched. Everyone hand-writes one reward and trains against it. We generate four, race them on identical seeds, and have a Judge agent rewrite them between generations. Three generations deep.
The vertical arrives in the sentence, never in the code. Agriculture is our focus, but the same pipeline runs unchanged for mining, logistics, inspection or manufacturing — the nouns in the simulator and the report are read from the task spec.
How we built it
Stack. Next.js 16 / React 19 / Three.js for the console and the 3D arena. Mastra for agent workflow orchestration. Python, PyTorch and stable-baselines3 for PPO. CLIP ViT-B/32 with a logistic-regression probe for perception.
Bright Data is the ground-truth layer. SERP API finds the sources, Web Unlocker retrieves them, and multi-region targeting means a mission reads the web the way its own geography does. For the run we demo: 54 queries, 64 pages, 11 distinct domains, 14 cited facts, 301 images of which 265 survived verification. Crucially, we mine hard negatives — the look-alike plants that fool a field robot — rather than clean examples.
Gemini verifies every scraped image against what the search claimed it was, at 24–32 concurrency. 36 images were rejected. Nobody hand-labelled anything.
Vultr serverless inference authors the reward function and environment config through tool calling, emitting structured code that is dry-run before use.
Daytona gives each training variant an ephemeral sandbox, 32 vectorised environments inside each, deleted the moment the run finishes.
Neon is the spine rather than just storage. Every agent writes to Postgres,
so a mission is fully reconstructable from rows. Serverless branching gives each
forked mission a copy-on-write branch, so a what-if run costs storage instead of
a re-run, and the demo replays from a frozen branch. Trainers claim work
atomically with FOR UPDATE SKIP LOCKED, and because state lives in Postgres
rather than memory, a mission survives the server restarting — which it did,
more than once.
Tiger Data handles the telemetry firehose: 16,514 episode stats, 3,324 live
arena frames and 2,647 run metrics from a single mission. Hypertables absorb it,
continuous aggregates keep rollups warm, lttb() downsamples curves to what a
screen can actually draw, and stats_agg gives distribution without a table
scan.
Challenges we ran into
The reward function priced the crop at nearly zero. Generation 0 valued a trampled strawberry at −0.0007 and a pulled weed at +0.0164, so the robot correctly concluded it should drive straight over the crop. The policy was not broken; it was obeying a badly written law. Watching the Judge find and fix that is what convinced us the reward search was real.
Telemetry was double-counting by 2.85×. Our drain was select-then-update,
so concurrent drains inserted the same rows twice. Fixing it meant an atomic
UPDATE ... WHERE id IN (SELECT ... FOR UPDATE SKIP LOCKED) RETURNING, plus
release-on-failure. Verified: 1.500× duplication before the fix, exactly
1.000× after.
Building wiped the running server. next build replaces .next underneath a
live next start, which killed in-flight missions repeatedly. We moved builds to
an isolated dist directory and made missions resumable from Postgres.
A wedged connection pool looked exactly like data loss. Late in the build, every panel went to zero at once. It was the long-running server holding dead Neon sockets after an autosuspend — a restart fixed it — but for ten minutes it looked like we had lost the run.
Getting the comparison honest. Early numbers flattered us because the baseline and the trained policy were not evaluated identically. The fair version runs both on the same 32 fixed seeds through the same perception model, and the numbers got smaller and trustworthy.
Accomplishments that we’re proud of
The Judge rewriting its own reward function and not simply turning every dial
up. One generation it wrote: “Leave anti_idle roughly unchanged — it
successfully prevented idle collapse and raising it risks over-penalizing
action.” It found a term that was already working and protected it. That is
reasoning, not pattern-matching.
The measured result, on identical seeds through the same detector:
| Measurement | Uploaded model | After Accelerate.ai |
|---|---|---|
| Act precision | 22.6% | 75.5% |
| Weed detection | 73.2% | 89.3% |
| Miss rate | 0.697 | 0.226 |
| Damage per weed removed | 1.117 | 0.889 |
| Crop damage rate | 88.8% | 73.5% |
| Weeds removed | 79.5% | 82.7% |
Two out of every three actuations used to be wasted. Now three out of four land. Total compute for three generations, 14 trained controllers and 19 evaluations: $0.09.
We are equally proud that the system reports what went wrong. The hard-negative ablation improved held-out look-alike accuracy from 75.0% to 87.5%, but pool separation got worse by 0.039 — and the report says so. Crop damage is still 73.5%, which is not shippable, and the report says that too.
What we learned
A reward function is law-making, not engineering. You are specifying what counts as good behaviour, and the optimiser will find every gap in your specification. Our generation-0 robot was not malfunctioning when it flattened the crop — it was complying.
Hard negatives are where the accuracy actually lives. Scraping clean, obvious examples barely moved the needle. Deliberately mining the look-alikes moved held-out accuracy on the hard cases by 12.5 points.
Pick the engine per workload, not per vendor. Reward authoring is text reasoning; image verification is high-volume multimodal. Splitting them across Vultr and Gemini let both run in parallel and each do what it is good at. The same logic put relational state in Neon and measurement streams in Tiger.
Make the system defend its own numbers. Every figure in the console traces to a row. That started as a correctness tool and became the most persuasive thing about the demo.
What’s next for Accelerate.ai
Sim-to-real. Everything today is measured in simulation grounded on scraped data. The next step is a hardware-in-the-loop evaluation on a real rover.
Upload-your-own-everything. We support uploaded weights; next is uploaded field footage and uploaded telemetry, so the simulator is grounded in the operator’s own ground truth rather than only the open web.
More generations, better search. Three generations of four variants is a small search. The reward-search loop is the most valuable part of the system and the one with the most headroom.
Verticals beyond agriculture. The pipeline is already domain-agnostic. The work is in templates for mining, warehouse and inspection tasks so a first-time operator in those fields gets the same one-sentence experience.
Built With
- bright-data
- clip
- daytona
- drizzle-orm
- google-gemini
- mastra
- neon
- next.js
- postgresql
- ppo
- python
- pytorch
- react
- reinforcement-learning
- scikit-learn
- server-sent-events
- stable-baselines3
- tailwindcss
- three.js
- tiger-data
- timescaledb
- typescript
- vultr
