Legible

AI agents are starting to shop. Legible makes web apps agent-ready, routes every payment through PayPal with limits and human approval, and proves it.

Comment

Inspiration

AI agents are starting to shop for us: they search, compare, fill carts and check out. But the web was built for humans. An agent clicking through HTML is fragile, and an agent holding a payment API is a liability. A single prompt injection hidden in a product description can make it buy the wrong thing. We asked: how does a merchant know that AI agents can use their store safely, and that the money is right? Nobody could answer that with evidence, so we built Legible.

What it does

Legible makes a web app safe for AI agents, and proves it, with PayPal.

  1. Scan. Legible reads the site's code. An LLM describes every route, and deterministic rules sort each action into read, write or money.
  2. Approve. A human decides, in a dashboard, which tools agents are allowed to use.
  3. Publish. Approved tools go live as an MCP server that any AI agent can connect to.
  4. Safe Pay. Every money action becomes a PayPal Orders v2 order. A spending limit is checked before PayPal is called, a human approves in PayPal, and only then is the payment captured. Agents never see card data.
  5. Crash Test. A real AI agent runs shopping scenarios through the MCP server. Every payment is verified against PayPal's own record and the store's database, and the result is an Agent-Readiness Score.

In our demo store with planted bugs (the agent is not told about them), Legible scores 20/100. It catches:

  • a store that marks orders paid before PayPal captures (caught only by checking PayPal's record before approval)
  • a coupon that works on the website but crashes through the API
  • a prompt injection in a product description: the agent obeyed it and added a leather jacket, and Safe Pay's limit blocked the $143 payment before PayPal was ever called

After the fixes, the same agent scores 100/100: three payments captured, one simulated decline handled cleanly, one blocked by the limit.

How we built it

  • TypeScript monorepo (pnpm), with modules that talk only through Zod contracts and HTTP, so four people could build in parallel and join cleanly.
  • Scanner: a 3-stage pipeline. ts-morph discovers routes deterministically → an LLM describes each one through a forced tool call (with a JSON + Zod fallback) → rules classify risk, and disagreements lower confidence and get flagged for review. Money tools can only bind to a cart route through Safe Pay.
  • Publisher: a stateless MCP server (official TypeScript SDK, Streamable HTTP) that serves only approved tools and logs every call per run.
  • Safe Pay: Express + SQLite on PayPal Orders v2 (Sandbox): OAuth, idempotent create (PayPal-Request-Id), approval, capture-once, negative-testing declines, and a Verify API that compares PayPal's record with the store's.
  • Crash Test: an LLM tool-use agent (openai/gpt-oss-20b via Groq, provider-agnostic through an OpenAI-compatible helper), Playwright for the sandbox test buyer, and grading by failure kind (tool error, state mismatch, injection followed, …).
  • Dashboard: Next.js + Tailwind. Risk lanes with a visual limit gate, an agent trace timeline, a Store-vs-PayPal ledger, and a before/after score.
  • pnpm demo runs the whole golden path with one command. There are 98 unit tests and 17 contract tests, one per module join.
  • We used Claude Code and Codex as coding agents, with humans owning every commit.

Challenges we ran into

  • INR isn't supported in our PayPal Sandbox accounts (422 CURRENCY_NOT_SUPPORTED), so we switched the demo to USD.
  • Free-tier LLM limits: one provider's quota ran out mid-build and another hit a daily token cap. We made the LLM layer provider-agnostic, added backoff and a prompt cache (rescans cost zero calls), and recorded which model produced every number.
  • A bug that hides itself: the "paid early" bug looks fine after capture, because both sides say paid by then. Legible only catches it by verifying before and after approval.
  • Our own scanner had a safety gap: a smaller model exposed an internal order-status route as a money tool. We fixed the binding rule (money tools may only bind to cart routes; internal order routes are never exposed) and added a regression test that replays that exact failure.
  • Models sending invalid tool arguments (null for optional fields) broke runs at random. We added a repair-and-retry loop.

Accomplishments that we're proud of

  • It works end to end on real PayPal Sandbox, not mocks, and it passes a fresh-clone test.
  • The hybrid scanner is safer than the LLM alone: on our 11-route labelled set, 100% tool F1 vs 87.5% for LLM-only, because the rules stop internal routes leaking to agents.
  • A hijacked agent was contained: prompt injection succeeded against the model, and the payment layer still held.
  • Honest by design: human approval, spending limits, verification against PayPal's records, planted bugs disclosed, and a test harness clearly labelled as one.

What we learned

  • For agentic commerce, the payment rail is the safety boundary. You can't trust the agent, so you verify against the payment provider's ground truth.
  • LLMs need guardrails and evals. Rules plus an LLM beat the LLM alone, and only a measured baseline showed us that.
  • Contracts first let four people and several coding agents build in parallel without a painful merge.
  • Agents follow instructions they find on web pages. Prompt injection is a real commerce risk, not a theory.

What's next for Legible

  • More frameworks: Next.js, FastAPI and Rails, beyond today's Express + TypeScript.
  • Server-side and cumulative spending limits, service auth, and PayPal webhooks for captures.
  • Refunds and disputes in the Crash Test lifecycle, and a comparison against the PayPal Agent Toolkit.
  • A larger benchmark across real open-source stores, more injection scenarios, and model-vs-model readiness scores.
  • A hosted version merchants can point at their staging site before opening to AI agents.

Built With

Share this project:

Updates

Submission history