-
-
Independent Verification — ReAgent confirms the $500 transfer from system state and suppresses the unsafe retry.
-
Commit-Then-Timeout — A controlled failure scenario where a successful transaction loses its response, creating duplicate-execution risk.
-
UNKNOWN Is Not ABSENT — When independent observation is unavailable, ReAgent escalates instead of blindly retrying.
-
Nemotron Advisory Architecture — Model-proposed contracts are validated, while deterministic policies retain execution authority.
-
Evidence-Driven Investigation — Tracing agent actions, independent observations, contract checks, and recovery decisions.
-
ReAgent — An independent reliability layer for autonomous agents
-
Reliability Evaluation — Quality gates, test coverage, and scenario-level recovery results.
-
480-Trial Fault-Injection Benchmark — Comparing ReAgent's state-governed recovery against alternative policies.
Inspiration
Most of the energy in agent engineering goes into making agents more capable: better planning, better tool use, longer workflows, more autonomy. A quieter question matters just as much: once an agent can act on the world, how do we know that what it says it did is what actually happened?
An agent can reason correctly about an action but fail to execute it. A tool can return an error after the action has already committed. A success response can arrive even though only part of the intended operation was completed. And sometimes, the agent loses the ability to determine what happened.
These situations expose four different things that are often treated as one: what the agent intended, what it or its tools claim happened, what actually happened in the external system, and what can be independently established with evidence.
Consider a $500 transfer. The payment service commits the transaction, but the response is lost. The agent sees a timeout. If it retries, the user could be charged twice. If it assumes success, it could report an outcome it cannot establish.
The deeper problem is not the timeout. It is the gap between an agent's understanding of an action and the actual state of the system it operates.
We built ReAgent to explore how that gap can be closed.
What it does
ReAgent is a reliability and verification layer designed to sit between autonomous agents and the systems they operate. It focuses on a question that ordinary task execution often leaves unanswered: did the intended outcome actually happen?
Before an operation executes, ReAgent uses a structured postcondition contract to define what must be true for the operation to count as successful. These conditions describe observable properties of the expected result, rather than relying on a generic success message.
When execution becomes uncertain, ReAgent independently observes the relevant system state and compares it against the validated contract. The decision is based on what the evidence establishes, not simply on what the original tool response says.
There are three possible outcomes:
- VERIFIED: The observed state satisfies the contract. ReAgent records the evidence and prevents unnecessary duplicate execution.
- NOT ESTABLISHED: The evidence does not establish the intended outcome. If the absence is authoritative and the execution boundary guarantees that a retry is safe, ReAgent may permit one bounded retry. Otherwise, it escalates.
- UNKNOWN: The observer cannot establish what happened. ReAgent preserves uncertainty and escalates rather than treating an unknown outcome as a failed action.
The principles behind this approach are simple: an agent's claim is not proof of its action, and UNKNOWN is not ABSENT.
How we built it
We designed ReAgent around a sequence of distinct responsibilities: contract proposal, semantic validation, contract freezing, execution, independent observation, deterministic verification, and a recovery decision.
An agent or model first proposes a structured contract describing the expected outcome. A semantic validation layer checks whether the proposed conditions are specific, meaningful, and grounded in the operation. A vague assertion such as a generic success status is not sufficient when the task requires concrete state changes.
Once accepted, the contract is frozen so that the conditions for success cannot be weakened after execution. ReAgent then uses an independent, read-only observation path to inspect the system of record. Deterministic verification logic compares the observed state with the original contract.
For model integration, NVIDIA Nemotron through Nebius Token Factory is available as an optional contract-proposal component. The model helps express the expected postconditions; it does not certify the outcome or make the final recovery decision. The model path is opt-in. The demonstrations and benchmark run offline using deterministic fixtures.
We built wallet and hotel-booking scenarios to demonstrate the approach.
In the wallet scenario, a $500 transfer commits but its response disappears. ReAgent independently observes the original transaction, checks the relevant details against the contract, and prevents a duplicate execution.
In another scenario, the observer itself becomes unavailable. The transfer may have committed, but ReAgent cannot establish that from the available evidence. It returns UNKNOWN and escalates. The demonstration's oracle reveal then exposes the hidden ground truth: the transfer had actually committed. This makes the consequence of confusing unknown with absent visible.
A third scenario demonstrates partial execution. A ledger record is created, but the corresponding balance update fails. The existence of a record alone would suggest success; checking the complete contract reveals that the intended outcome was not achieved.
Challenges we ran into
The first challenge was reproducing two different execution outcomes that look identical to the agent: an action that never committed and one that committed successfully before its response disappeared. The distinction only becomes visible through independent observation.
The second was moving beyond superficial verification. A record can exist while containing the wrong amount, referring to the wrong account, or representing only part of an operation. We needed contracts that describe the actual outcome, not merely the presence of evidence.
The third was determining when recovery is justified. Even if an observation finds no effect at one moment, a delayed original write could still arrive afterwards. Real integrations need execution-boundary protections such as enforced idempotency or fencing before a retry can be considered safe. Where those guarantees are unavailable, ReAgent must fail closed and escalate.
These challenges shaped the project around evidence and execution safety rather than simply adding more retry logic.
Accomplishments that we're proud of
We built a working verification architecture, demonstrated it across wallet and hotel-booking workflows, and developed a synthetic SQLite fault-injection benchmark to compare five recovery policies across 480 trials: 12 tasks, 8 injected fault modes, and 96 trials per policy.
In the benchmark, ReAgent correctly verified 29 trials, made no false verifications, and made no unsafe retries. It escalated 67 cases, meaning it completed fewer than a third of the trials without human intervention. That is the cost of refusing to guess when the evidence is insufficient.
An existence-only baseline declared success in 43 additional cases where the state was partial or incorrect. A naive retry policy made 24 unsafe retries, producing 4 duplicate primary writes and 5 duplicate audit rows.
These are synthetic SQLite results with injected faults, not production results. The existence-only baseline is deliberately limited; an observer applying the same field-level checks would close much of the gap. The benchmark therefore demonstrates why verification must examine the complete expected state, not that ReAgent outperforms every possible observer-based design.
What we value most is that the system makes uncertainty visible and allows that uncertainty to influence whether another action is permitted.
What we learned
Building ReAgent changed how we think about autonomous systems.
An agent's ability to perform an action and its ability to establish the result of that action are two different capabilities. A tool response is not necessarily the outcome. A database record is not necessarily proof of completion. And a model's confidence cannot substitute for independent evidence.
We also learned that recovery is not always about trying again. Sometimes the correct decision is to verify, sometimes it is to retry under enforceable safety conditions, and sometimes it is to stop and escalate.
Most importantly, we learned that a reliable autonomous system must be able to represent what it does not know, rather than turning uncertainty into an assumption.
What's next for ReAgent
Today, ReAgent focuses on verifying individual actions. Our next step is extending this approach to multi-step tasks, where each operation can be independently verified and recovery can resume from the appropriate point.
We also aim to strengthen execution-boundary protection against delayed writes and duplicate delivery through enforced idempotency and fencing, expand the adapter architecture to real external services, and evaluate model-proposed contracts across previously unseen tools and capabilities.
As autonomous agents take on increasingly consequential tasks, verifying their effects will matter as much as planning their actions.
An agent can tell you what it did. ReAgent asks the world.
Log in or sign up for Devpost to join the conversation.