Inspiration

Two agents investigating the same framework can spend seconds processing identical code before either writes a line. Hosting a shared prefix can avoid that work, but somebody must keep it warm and decide how to pay the maintainers whose code was reused. On our GPU, processing a cold 26,000-token prefix took more than eight seconds before the first text appeared. We chose the unit our inference engine could verify on each call, one reused token.

What it does

A publisher selects licensed Starlette source, then adds the FastAPI layer that depends on it. Cachet verifies each source hash, warms it and checks the actual cache-hit ratio with another request. A developer approves one PayPal Sandbox spending cap. The authorization can fund several coding tasks for up to 72 hours. On each task the agent compares the cost of buying access to the shared prefix with computing privately. A model explains the quote before the first metered turn.

The developer sees hits, time to first text and the charge for every turn. A missed public token costs zero. The royalty tree shows what each maintainer contributed. The dashboard compares a token-proportional split with the Shapley split that determines royalties from measured prefill costs. When accrued usage reaches $2, Cachet captures whole cents. At close, it waives outstanding usage below $1, preserves the maintainers’ earned royalties and releases the unused authorization. If a warm call breaches its latency guarantee, Cachet can reverse the charge or refund the captured public portion. PayPal Payouts pays eligible maintainers. The settlement dashboard reconciles each provider receipt against the ledger.

The public demo currently offers two built-in coding tasks: FastAPI rate limiting and a Starlette custom route class. Free-text tasks are disabled because the current model did not reliably follow custom response-format requirements. Existing custom run records remain available for audit.

How we built it

The PayPal JavaScript SDK presents the sandbox wallet. Orders v2 creates an AUTHORIZE order and authorizes the buyer-approved budget. Payments v2 captures accrued usage, sets the final-capture flag where appropriate, voids leftover authorizations and refunds eligible captures. Payouts v1 groups individual payees. PayPal Webhooks verifies signatures and handles redeliveries. Reconciliation reads the original orders, authorizations, captures, refunds and payout objects. Ops Copilot uses the PayPal Agent Toolkit for scoped order lookup and Transaction Search. Sandbox reporting can lag, so the object GET determines settlement.

SGLang 0.5.23 serves Qwen2.5-7B-Instruct-AWQ on a 24 GiB NVIDIA RTX 3090 Ti. The buying agent uses real model output from previous turns. A separate, platform-funded model request explains its deterministic quote decision. The gateway intersects SGLang’s cached prefix with token ranges for each source layer. FastAPI and SQLite record the charges and royalties as balanced integer microdollar entries. Structured Ops Copilot routes explicit requests about earnings, mandates and refunds to workspace-scoped tools. A refund requires confirmation. Next.js hosts the judge guide. Our AG Studio dashboard integrates a custom theme, Prefix Royalty Tree and a custom Ops Copilot agent alongside AG Grid. Singapore hosts the Docker API and the Next.js frontend on Render. The GPU host registers its tunnel with the API. In a controlled outage, the public service switched to labeled replay after 22.19 seconds and returned to live inference 16.59 seconds after the tunnel was restored.

Challenges

A changed token at a file boundary can invalidate everything that follows. We pinned both source revisions and the tokenizer, measured the shared segments separately and kept each consumer’s private dialogue under a tenant salt. A rolling conversation window bounds the live prompt plus generation reserve at 31,000 tokens.

A successful API call can still make a poor payment. This US sandbox merchant received $0.00 net on a $0.50 capture. On a real $2.00 capture PayPal reported a $0.54 fee and $1.46 net. A separate final $1.03 capture cost $0.52 and the remaining $1.97 of the $5 authorization was released. We use the actual fee in reconciliation and show the estimate before a judge makes a small test capture.

Accomplishments

  • Both production cameras recorded the same original close and payout. After sampling, a further $0.003887 of uncaptured SLA charges was reversed. Closing waived the remaining $0.214572 and released $1.00. A single PayPal Sandbox batch paid $0.64 to Starlette and $0.31 to FastAPI, with $0.25 in fees per recipient. Nine provider objects reconciled with zero differences.

  • A separate long-running agent used one $5 sandbox authorization for 1,020 metered turns on the public deployment. Usage reached $4.218459. PayPal captured $2.00 twice, charging $0.54 per capture and leaving $1.46 each. Two completed $0.01 refunds brought the merchant’s net to $2.90. Before close, $0.218459 of usage remained uncaptured. Starlette had $0.610403 and FastAPI $0.295582 in funded royalties available to settle. At that checkpoint the mandate was still open, no publisher payout had been submitted for it, and six PayPal objects reconciled with zero differences.

  • In the deployed AG Studio settlement panel, the operator compared publisher earnings, traced the original mandate and confirmed a proposed $0.01 refund against an eligible call. PayPal independently reported the refund completed. The workspace reconciled ten provider objects without differences.

  • On the Singapore deployment, the two built-in tasks completed ten measured turns each, and both generated applications passed HTTP checks in isolated Docker containers. The same sandbox budget funded the two tasks. The $0.09 judge capture incurred a $0.09 fee, and closing released $4.91 of the original $5 authorization.

  • We measured time to first text across five prefix lengths, with five cold/warm pairs at each length. For the 26,000-token prefix, the GPU-host medians were 8,258.4 ms cold and 69.3 ms warm. Measured from the Singapore API across the GPU tunnel, the corresponding medians were 8,864.1 ms and 644.1 ms. The two published source layers occupy 11,992 and 13,908 tokens. An ordinary browser workspace observed 99.992% and 99.996% cache hits on their publication probes.

  • Two independent Chromium sessions completed all six judge steps on the public deployment. Each ran ten live turns, captured $0.06 against its own $5 sandbox authorization, refunded $0.01 against three deliberately delayed calls and released $4.94. In each run PayPal reported SUCCESS for both payout items ($0.02 and $0.01), and eight judge payment objects reconciled with zero differences. Judge Mode explicitly lowers the payout minimum and allows an early test capture. Separately, we verified 16 PayPal-signed webhook deliveries across seven event types.

  • In the public wallet flow, the browser approved a separate $1 sandbox authorization. We dropped its streaming connections mid-task. The event stream resumed from event 107 at 108, with ten unique live turns and $0.034087 in metered usage. Closing waived the outstanding amount and released the full $1. Starting with a clean checkout of release commit 5885857, the uncached Docker Compose test finished in 181.69 seconds without a GPU or PayPal configuration. All seven routes returned HTTP 200, and the same workspace survived replacement of both containers.

What's next

Measure other eligible sandbox and merchant fee schedules, including micropayments pricing for production. Evaluate independently hosted native inference providers against the same cache counters and latency measurements, then extend publishing to more maintainers and opt-in private organization contexts.

Built With

  • ag-grid
  • ag-studio
  • fastapi
  • nextjs
  • paypal
  • paypal-agent-toolkit
  • paypal-orders-api
  • paypal-payouts
  • paypal-webhooks
  • playwright
  • python
  • qwen
  • render
  • sglang
  • typescript
Share this project:

Updates

Submission history