Inspiration
A support team testing an AI refund agent may have only a handful of examples. Seeing no costly mistakes in that small sample does not establish safety.
RiskGraph asks: what could an agent’s mistakes cost, how uncertain is that estimate, and which actions should be stopped before execution?
What it does
RiskGraph combines a synthetic refund agent, an independent safety monitor, incident memory, and a dashboard.
Claude proposes refund, decline, or escalation decisions. A separate monitor checks trusted order information and explicit rules covering ownership, refund windows, remaining balances, duplicate refunds, approval limits, and action consistency before the sandbox records an execution.
The dashboard separates proposed risk from residual risk after monitoring. Every decision has an inspectable trail linking the request, proposal, policy checks, simulated outcome, and recalled memories.
How it estimates risk
Expected monetary loss is decomposed into frequency × severity.
Beta and Dirichlet models provide estimates and 95% credible intervals when harmful examples are scarce. A separate conservative, time-uniform loss bound supports repeated monitoring under documented assumptions, while a betting e-process tracks evidence against a chosen harmful-proposal reference rate.
Zero observed loss still leaves visible uncertainty. Adaptive-memory evaluations disable the stationary loss bound because memory can change the decision distribution.
How it was built
The application uses Python, FastAPI, NumPy, SciPy, SQLite, and a JavaScript dashboard, deployed on Render.
Claude generates locally validated proposals through the Anthropic Messages API. SurrealDB Agent Memory stores blocked incidents and retrieves them into later decisions, preserving source, turn, and retrieval trace IDs. Recalled information supplies context; trusted policy checks determine authorization.
What is working
The current MVP passes 80 automated tests and has completed 26 Claude decisions across synthetic evaluations.
In a fresh 20-case evaluation, the monitor blocked two inconsistent escalation proposals carrying nonzero amounts. No unauthorized monetary loss was observed. Those protocol violations are reported separately from financial harm.
Hosted memory storage and retrieval in later evaluations are verified. Public synthetic evidence exports are included in the repository.
Challenges and lessons
Key challenges included adapting to current model API requirements, preventing duplicate execution during retries, preserving evidence across deployments, and keeping failed provider requests out of the observed safety sample.
An important lesson: policy violations, observed monetary harm, and uncertainty are different quantities and should be shown separately.
Next during Build Week
Planned improvements include broader adversarial evaluations, calibration and prior-sensitivity analysis, richer time-aware memory, bounded policy verification, and a clearer demo.
Disclosure and limitations
All orders, customers, messages, and money are synthetic. No real payments are executed. These evaluations establish a working prototype, not production safety.
The project was developed during Build North with AI coding assistance from Codex. It combines established statistical methods; the complete AI system has not been formally proved.
Log in or sign up for Devpost to join the conversation.