CatalogPay-Agent

CatalogPay Agent turns text, photo and voice orders into payments using AI, SKU matching, and PayPal. It automates commissions and vendor payouts with cent-accurate accounting and idempotent payments.

Comment

Inspiration

Watching independent cafés reorder supplies: a WhatsApp message to one supplier, a photo of a whiteboard to another, a voice note to a third. On a marketplace, someone then has to turn that into SKUs, compute each vendor's commission without rounding errors, collect the payment, and pay every vendor separately. We wanted an agent that does the whole loop and that a finance team could actually trust with money.

What it does

CatalogPay Agent turns a messy wholesale order (typed, photographed, or spoken) into a SKU-verified PayPal checkout that automatically splits and pays every vendor.

  • Gemini 2.5 Flash extracts product phrases and quantities with structured outputs.
  • A hybrid matcher maps each phrase to a catalog SKU, combining fuzzy matching, character n-gram TF-IDF and unit normalization ("half kg" = 500g). It shows a confidence score and lets the operator switch candidates in an AG Grid workbench.
  • Prices and commissions are computed in integer cents, with the fee charged once per vendor.
  • PayPal Orders v2 collects the payment. After a verified capture, PayPal Payouts v1 pays each vendor its net share automatically.
  • A React 19 Command Center with two AG Grid v33 workbenches shows the resolution, the economics and the full settlement ledger, with CSV export.
  • The same flow is exposed as Gemini function-calling and MCP tools.

How we built it

  • Backend: FastAPI + SQLAlchemy 2.0 on Python 3.12+.
  • AI: Gemini 2.5 Flash via google-genai, with a prompt-injection-hardened system instruction and item-by-item validation of model output.
  • Matching: a local hybrid SKU matcher (rapidfuzz + numpy TF-IDF + unit agreement), so it is fast and deterministic.
  • Payments: an httpx-based PayPal client (OAuth2 token caching, Orders v2, Payouts v1, webhook verification), plus an in-process PayPal simulator that speaks the same HTTP API, so the full lifecycle runs without credentials.
  • Frontend: React 19 + TypeScript + Vite + Tailwind v4 + AG Grid v33 (Theming API, custom cell renderers and editors, pinned totals rows, CSV export).
  • Deployment: one multi-stage Docker image serving the API and the SPA, deployed with a Render Blueprint.
  • Tests: 532 pytest and 94 Vitest tests, with mutation checks on the safety guards.

Challenges we ran into

  • Rounding drift. Charging commission per line made two identical lines cost 1¢ more than one combined line. We moved to charging the fee once per vendor and apportioning it by largest remainder, so splitting or merging lines never changes what a vendor pays.
  • Ambiguous payment failures. A timed-out PayPal call may have succeeded. We separated ambiguous failures (retry with the same PayPal-Request-Id) from definitive ones (rotate IDs), and made our simulator replay cached responses like PayPal does. That exposed a bug where a declined capture would have replayed its rejection forever.
  • Concurrency. Two simultaneous captures must never oversell. We added a capture lease plus conditional UPDATE … WHERE stock_quantity >= qty, with database CHECK constraints as a second line of defense.
  • Trusting AI with money. We let the model extract, never price: buyer text is fenced as untrusted data, quantities must be real integers, and tests prove an injected "$0.01" can't change the charge.
  • Unit parsing edge cases. "1,000g" originally normalized to "1g" and "500 gm" was read as a quantity of 500. Both were caught in review and are now covered by regression tests.

Accomplishments that we're proud of

  • The full lifecycle (messy text → verified SKUs → PayPal order → verified capture → per-vendor payouts) takes about 60 ms of server time in sandbox simulation (9 lines, 4 vendors, measured median), and runs with zero setup for judges.
  • The ledger invariant GMV = platform fee + vendor payouts is enforced at every level, including by the database itself.
  • Retries are provably safe: lost responses, timeouts and duplicate webhooks never charge a buyer or pay a vendor twice.
  • Two AG Grid workbenches that are the actual operating surface: in-cell SKU switching, live re-pricing, pinned totals and finance-ready CSV.

What we learned

  • In payments, "did it fail?" is the wrong question; "could it have succeeded?" is the right one. Idempotency keys only help if you know when to reuse them and when to rotate them.
  • A simulator that faithfully reproduces the real API's quirks (request-ID replay, duplicate-ID rejection) finds real bugs before production does.
  • Structured outputs make LLMs dependable extractors, as long as you still validate every field and keep them away from prices.

What's next for CatalogPay-Agent

  • Live PayPal Sandbox → production onboarding for real marketplace vendors (PayPal for Marketplaces / partner onboarding).
  • Inventory sync and SKU learning from operator corrections, so a pinned candidate becomes a learned alias.
  • Voice-first reordering over messaging channels, with the agent asking clarifying questions for ambiguous lines.
  • Postgres plus background payout reconciliation workers for scale.

Built With

Share this project:

Updates

Submission history