Inspiration

I kept watching my coding agent burn expensive frontier-model tokens on stuff that really didn't need a frontier model. Like, writing a tiny is_palindrome function, or doing arithmetic in the middle of a bigger task. A 1.5B model on my laptop can do that just fine. So I wondered: what if the big model could just... hand that bit off? Send it on an errand, basically.

The catch is trust. If my agent sends a job to some random cheaper model, how do I know the answer is right? And how does that model's owner get paid? That's where PayPal came in. I didn't want payments to be a bolt-on; I wanted the money to be the thing that keeps everyone honest.

What it does

Errand is a marketplace where AI agents (Senders) hand well-scoped jobs to cheaper specialist models (Runners).

  • Your agent makes one call: delegate(task, check). The check is the answer you expect, or a few Python assert lines for code.
  • Errand figures out what kind of job it is, picks the best specialist for it (or the one you pin), and runs it.
  • The money is held in escrow while the specialist works. Errand runs the check. Right answer? The Runner gets paid. Wrong? It's retried on another Runner, and you never pay for the wrong one. No right answer at all? Full refund.
  • Senders top up with PayPal Checkout. Runners cash out with PayPal Payouts. Bigger one-off jobs can be posted as a PayPal bounty: the payment is authorized up front, then captured only if the work passes, or voided if it doesn't.
  • Hidden test questions keep checking every Runner. If one quietly swaps in a worse model after onboarding, it gets caught, loses half its deposit, and the Senders it served get refunded.

It works with basically any agent: there's an MCP server (so Claude Code, Cursor, VS Code and friends get a delegate tool in one line), a REST API, and a Python SDK.

Does it actually work?

I benchmarked it with real open models on 44 math and code tasks where every answer gets checked automatically:

Model Right (and checked)
phi3:mini 35 / 44
qwen2.5:1.5b 31 / 44
qwen2.5-coder:1.5b 26 / 44
Errand, picking the best model per job 39 / 44 (89%)

The bit I didn't expect: no single model was best at everything. phi3 was great at math and meh at code, the coder model was the opposite. Routing each job to whoever is best at it beat every single model. And it came out about 98% cheaper than an estimated frontier-model price for the same answers, with $0 paid for the 5 wrong ones. (Accuracy and Errand's cost are measured; the frontier price is an estimate. errand bench reproduces it.)

In the demo video, a real coding agent (qwen2.5 7B, running on my laptop) splits a task into two functions, Errand sends each one with its tests to a different specialist, both pass, and each job costs under a tenth of a cent.

How I built it

  • Backend: Python, FastAPI, SQLite. Every cent goes through a double-entry ledger where each transaction sums to zero, so money can't appear or disappear.
  • PayPal: Orders v2 (CAPTURE for wallet top-ups, AUTHORIZE for bounties), Authorizations (capture on pass, void on fail) and Payouts (Runner withdrawals), all against the free sandbox. Every PayPal call is logged with its sandbox ID in the dashboard. There's also a clearly labelled mock mode so anyone can run it without keys.
  • AI: Runners are real models served through Ollama (qwen2.5, qwen2.5-coder, phi3) or any OpenAI-compatible endpoint or webhook. Picking a Runner uses a Bayesian rating per job type (think "how often is this model right, and how sure are we") plus Thompson sampling, so new Runners still get a fair shot. Answers are checked by deterministic verifiers: exact math answers, and Python tests run in a separate process.
  • Agents: an MCP server (remote over HTTP and local over stdio), plus tool schemas for OpenAI-style and Anthropic function calling. For the demo I built a tiny coding agent on qwen2.5:7b that delegates through Errand, and a real local model writes the code.
  • Frontend: server-rendered pages with AG Grid for the dashboards (errands, ledger, PayPal activity), no build step.
  • Hosting: a Docker image deployed on Render. It seeds itself with demo specialists and real activity on every boot, so judges never land on empty tables.

Challenges

  • Small models are chaotic agents. My first coding-agent attempts on 1.5B and 3B models skipped the tool, sent no tests, or wrote tests that didn't even parse (an unescaped apostrophe in 'I'm Adam'!). That actually made Errand better: it now bounces broken tests before anyone gets paid to try, and it lets tests import from your own project.
  • Onboarding was too strict. A model that aced code but failed math got rejected entirely. Now Runners get listed for just the job types they pass, which is how a specialist market should work anyway.
  • Micro-payments vs real payments. A $0.0003 job can't be its own PayPal transaction, so Errand uses a prepaid wallet and batched Payouts, and keeps per-errand escrow in its own ledger.
  • Payouts in the sandbox. My first sandbox business account couldn't send Payouts at all (PayPal returned PAYOUT_NOT_AVAILABLE for its country), so I had to test withdrawals against a second sandbox account on the hosted demo.

What I learned

Honestly, that verification is the whole product. Routing is the fun part, but "pay only when it checks out" is what makes it safe to let an agent spend money at all. Also that a 1.5B model on a 4 GB VRAM laptop GPU is way more useful than I assumed, as long as something is checking its work.

Accomplishments that we’re proud of

  • Routing beat every single model. Errand picked the best model per job and got 39/44 right, more than any one model managed alone, at about 98% less than an estimated frontier price. And I paid $0 for the 5 wrong answers.
  • A real coding agent running on my 4 GB laptop GPU delegates end to end: it splits a task, sends each function with its tests to a different specialist, both pass, and each job costs under a tenth of a cent.
  • Real PayPal money flows. Checkout for top-ups, authorize then capture or void for bounties, and Payouts for Runners, with real sandbox Payouts going out from the hosted demo.
  • The money can’t leak. Every cent runs through a double-entry ledger where each transaction sums to zero, and the test suite checks it.
  • It’s genuinely plug and play: one line connects Claude Code, Cursor, VS Code and other MCP clients. I built all of it solo in about two weeks, including the live demo and a fully scripted demo video.

What we learned

  • Verification is the whole product. Routing is the fun part, but “pay only when it checks out” is what makes it safe to let an agent spend money at all.
  • PayPal already had escrow built in. Authorize up front, then capture on a pass or void on a fail, mapped almost one to one onto how I wanted bounties to work.
  • Micro-payments need a different design. A job worth a fraction of a cent can’t be its own PayPal transaction, so a prepaid wallet plus batched Payouts is the way to go.
  • Small models are way more useful than I assumed, as long as something is checking their work. They’re also chaotic as agents: they skip tools and write broken tests, so the platform has to catch that before anyone pays.
  • No model is best at everything, which is exactly why a marketplace of specialists makes sense.

What’s next for Errand

  • More job types (data extraction, translation, search, etc).
  • Better ways to evaluate answers, beyond exact matches and unit tests.
  • Smarter matching between models and jobs, learned from real usage data.
  • Ask several models at once and take the answer they agree on.
  • Approve PayPal once and let your agent top itself up within limits you set.
  • Run a specialist from your own laptop in one command, and get paid for it.

I’m open to funding and collaborators: https://x.com/shashvatsingh05 or [email protected]

Built With

Share this project:

Updates

Submission history