CounterSign

Inspiration

Marketplaces and platforms that pay many sellers through PayPal have to connect buyer disputes to maker payouts. Enough cash in the account does not make a disputed share payable. A stuck payout also needs investigation before a resend.

AI can help read the complaint, tracking and payout status. Its suggestion still needs an application permission check. We built a boundary where the agent proposes, code checks the records, and the owner approves the exact payment or buyer refund.

What it does

CounterSign gives the merchant one workspace for disputes, maker payouts and the decisions between them. It is a focused prototype for marketplaces and platforms that want AI assistance with PayPal payouts.

Command center shows the next owner action, maker balances and the staged week. Decisions shows the proposal, applicable checks, exact payload hash and passkey countersign. Plain approval remains available and is labelled separately.
Incident desk filters disputes and stuck payouts. Payout runs exposes maker lines, attempts and resends. Cash shows freshness and the spendable-cash calculation. Money graph links the disputed order to maker shares and holds; the maker's money map opens that maker's records in a dialog.
Execution history traces verdicts, approvals and PayPal operations. Evaluation replay explores historical mock runs with AG Grid, filtered totals and CSV export. Sandbox proof links observed PayPal behavior to recordings.
Try to trick the agent runs the proposal a fooled agent would make through the real shield, then rolls back the preview. It makes no Gemini call. Light and dark themes, a phone layout and reduced-motion support carry the same decisions and simulation label across the product.

The Gemini Incident Commander reads sanitized incidents, buyer messages and read-only tools. It returns one validated structured proposal with action, parameters, evidence references, rationale and expected effect. A deterministic shield evaluates the applicable rules. Payments and buyer refunds require a bound owner approval; the executor rechecks before its adapter writes to PayPal.

An outside MCP agent gets the same boundary. Hosted HTTP is a rollback preview; local stdio can record a proposal for review. No MCP tool approves or executes it. The recorded SDK exchange returns BLOCK, names the payee and injection rules, and says written: false for a refund-to-another-account attempt.

The demo opens on unsigned Friday. It runs on a local PayPal mock and says Simulated: mock PayPal. A naive read-only comparison would pay $1,500, including $414 of disputed shares. CounterSign holds A's $360 and B's $54, and asks the owner to approve $1,086 for B and C. A staged lost response is recovered by following the original-batch link in a duplicate 400 response; the intended payout is paid once. In the following week, A's staged seller win is reconciled before the owner approves its $360. Available cash ends at $354; B's separate $60 refund remains unapproved. The money is simulated, not a claim of customer savings.

How we built it

PayPal sandbox research shaped the design. Across checks D1–D13 we recorded 1,041 HTTP exchange attempts, including transport failures. The 47 successful webhook signature checks include the simulator event. Four summary checks differed from our assumptions: duplicate payout responses, dispute creation and holds, evidence availability, and balance freshness. The dashboard links those findings to the actual recordings.
A local PayPal mock reproduces the recorded behaviors. Faults include a timeout after acceptance, crashes, stuck items and forged or reordered webhooks. A fresh resource GET confirms state; an event alone does not settle the record.
Python, FastAPI and PostgreSQL 16 hold the execution path. State machines, record versions, write-ahead operations, a double-entry ledger and a PostgreSQL webhook inbox keep the checks close to the records. There is no graph database or separate message queue.
16 named safety boundaries comprise 14 pure shield rules, execution recheck and webhook authenticity. A payout proposal has 10 applicable pre-execution checks; unrelated checks are not shown as if they ran. Import-linter contracts enforce the agent, pure-shield, write-adapter and gateway package boundaries.
Gemini through LangChain investigates and proposes. The PayPal Agent Toolkit exposes five read-only tools behind an exact allowlist and HTTP guard. Model output, buyer text and tool results remain untrusted. Only the owner can change the trusted payout address.
The official MCP Python SDK carries read and proposal tools. WebAuthn verifies device signatures on the server while retaining the original approval binding and expiry checks.
AG Grid Community powers operational tables and the historical evaluation explorer. A Render Docker Blueprint packages the app, PostgreSQL and loopback mock, with a health check and no PayPal or model keys. Public Render deployment and hosted verification remain pending while the URL is a placeholder.

Challenges we ran into

Observed PayPal behavior changed the design. A duplicate payout POST returned a 400 with an original-batch link, rather than a replayed create response. Disputed captures became PENDING without an increase in withheld. Evidence required the right dispute state and action link. We built against those recordings.
The model did not follow the script. In the recorded Gemini week it chose WAIT on both unsafe baits, so those particular shield-block scenes were absent. We kept that result instead of inventing agent actions.
A shared demo needs a reviewable starting point. The hosted image restores the unsigned Friday state from real scripted steps. It retains passkeys, clears challenges and waits for active decisions before an idle restoration.

Accomplishments that we're proud of

The merchant can inspect a proposal, sign its exact details and follow its outcome through the execution history and money map. The sandbox proof page and historical evaluation explorer make the supporting evidence inspectable too.

Historical fault evaluation: mock, model disabled. The full published experiment contains 48 scenarios, five runs per scenario, for each system:

System	Passing runs	Scenarios passing 5/5
CounterSign, model disabled	170/240	34/48
Rules-only workflow	215/240	43/48

Both systems recorded zero for each of four measured invariant counters: duplicate settlement, disputed-order payment without approval, refund without owner approval, and payment to an unverified payee. Rules-only completed more tasks in this model-off experiment. Agent-only actions were missing when the model was disabled. Targeted follow-ups do not replace the original full score. Zero measured violations do not establish universal safety.

Real Gemini evidence is separate. The recorded gemini-3.5-flash-lite week ran on the mock and finished COMPLETE_WITH_GAPS. It submitted evidence, cancelled the unclaimed payout, drafted a maker note and left the refund to the owner. It chose WAIT on the injection and settled-retry baits. This is not a real-model benchmark score. The public demo instead uses a labelled scripted model, with no provider calls, so visitors can replay the same safety scenes.

What we learned

AI helps investigate unstructured complaints and evidence. Deterministic records, checks and owner authority still have to decide whether a financial write is permitted. Webhooks need verification and resource confirmation. Recorded platform behavior is a stronger design input than an untested assumption.

What's next for CounterSign

Apply the decision and approval layer to PayPal platform products for real marketplaces collecting for third parties. This project uses Payouts as a focused prototype; it is not a production platform integration.
Add owner-tunable policies, partial refunds, more dispute types and multi-currency.
Extend the gateway as an approval layer for PayPal's hosted MCP workflows.