Inspiration
A support agent can say the right thing and still send money twice. In a PayPal sandbox experiment, we simulated a timeout after a completed $50 refund. A retry with a new request ID completed a second $50 refund on the same $100 capture. We built Still to expose this kind of decision before an agent receives refund access.
What it does
Still runs customer scenarios against an agent, records its tool calls, and checks payment decisions against deterministic rules. Synthetic personas cite public source records. Repeated mock trials report pass@1 and pass^N. A failure map groups related mistakes and links to example traces.
A passing run can produce a mock-scoped permission file. Still's gateway enforces its amount ceiling and other conditions. Developers can set checked per-refund and per-order limits, reopen results, and inspect saved sandbox comparisons. A connected agent's mock session can be promoted to a regression and rerun against a fresh mock. The CLI and GitHub Action support repeatable runs.
How we built it
The Node.js app connects scripted agents, local HTTP agents, and configured MCP hosts. Its MCP interface mirrors selected PayPal Agent Toolkit tools. A host must accept Still's synthetic scenario and supplied connection; unchanged toolkit agents are not guaranteed to work. Thousands of trials use isolated deterministic mock ledgers. Separate sandbox runs capture actual PayPal responses. Codex-backed agents choose tool calls and AI can explain failures; deterministic checks decide verdicts. A scripted option needs no model login.
Challenges
A successful HTTP response is not always a completed refund. Sandbox payments can remain pending, and chargebacks can behave differently from wallet disputes. We kept uncertain outcomes unverified and separated mock scores from provider evidence. The retry timeout was injected.
Accomplishments
Saved sandbox evidence shows two completed refunds for one decision. A separate fixed run reuses the request ID and receives the same refund back. Both scripted agents ran 1,000 personas five times: the buggy example failed 5,000 mock trials and the corrected example passed 5,000. The mock gateway blocked an over-limit request and allowed an eligible one. These examples do not estimate real-world failure rates.
What's next
Expand toolkit coverage and agent compatibility, verify selected MCP traces against PayPal sandbox, improve mock agreement, and establish complete payment and dispute history before considering production use. Current earned permissions and gateway examples are scoped to the mock.
Judge setup
Clone the public repository with Node.js 22 or later, run npm start, and open http://127.0.0.1:3000/upgrade. The scripted mock path needs neither a Codex login nor PayPal credentials. Saved sandbox responses are separate from fresh mock trials.
Public repository: https://github.com/VasuBansal7576/still-agent-lab
Hosted saved sandbox replay: https://vasubansal7576.github.io/still-agent-lab/
