Strawbank

An AI shopkeeper for unstaffed farm stands. Photograph your basket, the AI counts it, code does the sums and you pay with PayPal. At dusk the grower learns what sold and what walked.

Comment

Inspiration

Ireland has 135,037 farms. Along country roads, some sell eggs, potatoes and jam from a table at the gate, with a tin for the money and nobody minding it. I live in rural County Monaghan, and that honesty box is the problem I picked.

The tin keeps getting robbed: two children in Laois closed their egg box after its fourth robbery, and a Kildare farmer tracked down his egg and potato thieves on camera. Fewer people carry cash, too: in 2024, less than half of in-store payments in Ireland were made in cash. And at the end of the day, nobody can tell what sold from what walked, so the grower can't tell whether the stand is worth keeping.

Sources: CSO, RTÉ, Irish Farmers Journal, Irish Examiner on an ECB survey.

What it does

Strawbank keeps the trust of the honesty box and adds a shopkeeper. The customer scans a QR code on the stand and uses their own phone; the grower checks in at morning and dusk.

When What happens Who does it
Morning The grower photographs the price card and the shelf, and confirms what was read. Each item becomes a PayPal catalogue product. Gemini reads, the grower confirms, PayPal lists
A customer buys They photograph their basket. Gemini counts it and says how sure it is; the customer can change any count. Gemini counts, code totals, PayPal takes an itemised order
Something's wrong "I took one onion, not two." The agent recounts the photo and refunds through PayPal, only if the photo agrees and it is within the grower's limit. Code decides, PayPal refunds
No PayPal to hand The customer leaves an IOU, and PayPal emails them an invoice at the stand's prices. PayPal Invoicing
Dusk The grower photographs the shelf again and learns what sold and what walked, item by item, in plain words. An unconfirmed payment makes an item "uncertain", never "missing". Code compares, Gemini explains, every number checked
A dispute The grower answers with the basket photo the customer took before paying, or refunds instead. No AI reads the dispute. PayPal Disputes
Next morning The day is checked against PayPal's own Transaction Search. Code and PayPal
Any time The grower asks the ledger a question, and AG Studio's agents build the chart. AG Studio on Gemini

The one rule: the AI never touches money

The AI reads, counts and explains. Every total, refund, cap and gap is plain code, and any number in the AI's words that code did not compute is replaced before anyone sees it. The diagram in the gallery shows it: the AI has no path to PayPal.

Results

All from npm run eval (scorecard of 12 Nov 2026).

Result What it shows
0 failures in 24,100 cases 28 property tests: the AI never sets an amount. Planted bugs (a total a cent off, a refund limit a cent off) each made a test fail.
10 of 10 real basket photos counted exactly. Basket 8 sometimes reads 5 eggs, not 4, so the customer always checks.
3 of 3, 8 of 8 price card lines and shelf counts read exactly
20 of 20 days, 0 wrongly accused dusk reports on simulated days. A naive tally called 7 paid-for items walked, worth €14.40.
11 of 11 trap photos handled: two faces refused by both checks, an instruction on the price card ignored, a lemon the stand doesn't sell
10 of 10, 20 of 20 reused basket photos refused, and re-saved copies flagged
7 of 7 next-morning audit scenarios, each found exactly

How I built it

Part What
PayPal (sandbox) Orders v2 through the PayPal Server SDK, the JavaScript SDK button, refunds and catalogue products through the Agent Toolkit, signed webhooks into a Firestore ledger, Transaction Search, Invoicing v2, the Disputes API, Log in with PayPal. Every order records pickup at the stand.
AI Gemini on Vertex AI, through the Vercel AI SDK. Gemini 3.5 Flash-Lite counts baskets and shelves and reads the price card; Gemini 3.8 Flash recounts for refunds, explains the day and runs the ledger's assistant. Google Cloud Vision checks every photo for a face.
AG Studio The grower's ledger, with our own paid-against-taken widget, and our agent in front of AG Studio's agents, on Gemini through our own adapter.
APIMatic The PayPal server code was written with APIMatic's Context Plugin in Claude Code. It warned that the SDK waits forever by default and must never re-send a payment; the client sets both.
Stack Next.js 16 and TypeScript on Google Cloud Run, with Firestore and Cloud Storage. Photos are deleted after 30 days, and a photo with a face in it is never kept.

The README lists each claim with the code and the test behind it; SECURITY.md lists each risk and what stops it.

Challenges I ran into

My first test on my own phone counted a box of five eggs as one item, so the count now matches goods by what they are, not their packaging. The PayPal Agent Toolkit is built for an older version of the Vercel AI SDK and returns errors as text, so I wrote an adapter. A full dry run on my iPhone found five more things, from a missing way back between pages to a refund claim that hid a potato that walked; all were fixed that evening.

On deadline day, Gemini's careful model took 20 to 27 seconds to read a price card. I moved the price card and shelf counts to the fast model, re-ran the evals with the same results, and the eval's price card now reads in 1.4 seconds.

Accomplishments that I'm proud of

A real sandbox payment from a photo of an egg, an onion and a potato on my own table, with one PayPal order line per item, and the agent's own refund of the onion I was charged for but didn't take. And money code the AI can't reach, with tests that prove it.

What I learned

The hard part of an AI shop is deciding what the AI must never do. Letting the model count and explain, and keeping every sum in plain code, made the payments simple to test. And a real photo is very different from a clean one: my first phone test changed how the count works.

What's next for Strawbank

More real photos (40 baskets, 20 shelves, 10 price cards), and a trial at a real stand near home in Monaghan.

Built With

  • ag-grid
  • ag-studio
  • apimatic
  • fast-check
  • firestore
  • gemini
  • google-cloud
  • google-cloud-run
  • google-cloud-vision
  • log-in-with-paypal
  • next.js
  • paypal
  • paypal-agent-toolkit
  • paypal-catalog-products
  • paypal-disputes
  • paypal-js-sdk
  • paypal-orders-v2
  • paypal-server-sdk
  • paypal-transaction-search
  • typescript
  • vercel-ai-sdk
  • vertex-ai
Share this project:

Updates

Submission history