Tollgate — the free-first agentic AI gateway

Ops

## Inspiration

Most AI gateways meter every call — you pay per request, forever. But a huge share of agent traffic is simple stuff a free model handles fine. We wanted the opposite default: free unless a task actually earns the spend — and when it does, let the agent pay for the better model itself, with no human and no credit card. x402 + stablecoins made the "agent pays on its own" part finally possible, so we built it.

## What it does

Tollgate is an OpenAI-compatible gateway. For every request, a small routing agent scores the task and decides:

Free tier — routed to free model providers. Most requests cost $0.
Premium tier — a frontier model (GPT-4o), unlocked only when the task warrants it. The agent autonomously pays a fraction of a cent in USDC on-chain via x402, within a budget you set.

For questions that need current info, the premium path first grounds the answer in live web results (Nimble) and cites its sources. Every routing and payment decision is logged to ClickHouse (with a natural-language "ask the data" endpoint), and model calls are traced in Datadog (Lapdog).

It supports two payment models:

Autonomous — the agent pays from a managed budget wallet.
Client-pays x402 — any external agent pays us from its own wallet; the facilitator settles on-chain. True agent-to-agent commerce.

## How we built it

Gateway: Python + FastAPI, OpenAI-compatible, with a per-request router (heuristics + an optional free-model difficulty classifier).
Payments: Coinbase CDP server wallets (budget + revenue) on Base Sepolia; autonomous transfers, plus the canonical x402 flow (EIP-3009 USDC, facilitator-verified) for client-pays.
Grounding: Nimble web search injected into premium prompts, with citations.
Analytics: ClickHouse for the spend/decision ledger, aggregate views, and a guarded natural-language → read-only SQL endpoint.
Observability: Datadog Lapdog auto-traces every model call.
UI: a chat that shows the agent's reasoning trace + cited sources, and an ops dashboard for spend analytics.

## Challenges we ran into

CDP auth — a mismatched API key/secret cost us an hour of 401s until we verified the JWT claims and regenerated the key.
Real x402 client-pays — wiring EIP-3009 signing + facilitator settlement was the hardest part; we did it on a branch so a failure wouldn't break the demo. It worked: a separate wallet really paid us on-chain.
Free-tier rate limits & latency — a single key throttled fast, so we rotate multiple keys and pin a fast small model; web grounding is slower, so we only trigger it when a question actually needs current data.

## What we learned

x402 + EIP-3009 makes agent payments genuinely gasless for the payer — that unlocks "pay only when needed" as a real default, not a gimmick.
The cheapest request is the one you don't pay for: free-first routing plus honest spend analytics is a strong, different angle from pay-per-call.

## What's next