EVE — Evidence Verification Engine

Inspiration

AI agents are starting to act on our behalf: they book, buy, change records and call other services. When something goes wrong, or simply when someone asks afterwards, one question is hard to answer: what was the agent allowed to do at that moment, and why?

Logs say what happened. They rarely say which evidence and which approval an action was based on, and whether that evidence actually covered the action. And if the model that proposes the action also decides whether it is allowed, the decision is only as good as the model's reasoning.

So we separated the roles:

AI agents make decisions. EVE makes the evidence behind those decisions verifiable.

What it does

EVE (Evidence Verification Engine) is an independent verification layer. Before an agent's consequential action runs, EVE checks the sealed evidence chain behind it against an operator-pinned policy and returns a determination (allow, escalate, ...) together with a sealed record.

EVA is the delegation app built on EVE. An AI agent proposes; a deterministic gate in EVA (plain Python, no model) asks EVE before any consequential tool runs, and only EVE's allow lets the tool execute. Anything else - escalate, block, unknown, unreachable - stops the action, and EVA reports the outcome with EVE's record id.

The demo in the video (household delegation, synthetic data):

• A third-party AI agent (Hermes) is asked to book a dishwasher repair quoted at $275. It only proposes the booking.
• Evidence version 1 contains the owner's confirmed mandate with a $200 limit. EVE returns ESCALATE (human review required). The gate stops the booking - nothing is booked.
• EVA then explains why automatically, built only from the preserved records (mandate, quote, EVE's own gap) - no model text. The explanation is itself sealed as a record.
• Evidence version 2 adds an approval of this exact $275 offer. EVE returns ALLOW, and exactly one (synthetic) booking is made.
• Version 2 supersedes version 1, but nothing is overwritten: both decisions, their evidence and their audit trails remain verifiable.

Same request. Two evidence versions. Two decisions. Both verifiable.

The human approvals in this scenario were recorded on Oct 6, 2026. No approval was given live in the video; the two EVE decisions shown were made live by EVE against those sealed evidence versions (Oct 21 and Oct 22, 2026).

What is live, what is simulated, what is next

How we built it

• EVE core (existing, private, hosted). Pinned to a frozen build (eve-core-v1, git tree a698922c...) whose regression suite passes 1,293 tests. It runs on loopback behind EVE MCP and is never exposed directly.
• EVE MCP v1 (built for this hackathon). A self-hosted MCP server - Streamable HTTP, protocol 2025-11-25 (mcp==1.30.0) - with exactly one tool, eve_pre_action. It returns EVE's envelope verbatim: no extra "approved"/"proceed" field, no second authority surface. The caller can only select an operator-pinned policy by reference; the policy bytes used are bound by their SHA-256. Frozen as tag eve-mcp-v1 and vendored into EVA with a byte-level verifier.
• EVA on the Strands Agents SDK. The deterministic gate is a Strands BeforeToolCallEvent hook. In the core gate, a consequential call is authorised once, bound to the exact tool name and arguments. Which evidence chain applies is operator-owned configuration ("binding selects evidence, it never establishes it"); a changed binding or policy mismatch means no action.
• BookingGate (delegation flow). Before EVE is even asked, the gate refuses a target or price that is not in the evidence (TARGET_NOT_IN_EVIDENCE, PRICE_NOT_IN_EVIDENCE).
• Human review semantics (locked decision). A human can add evidence, which creates a new chain version and a new EVE evaluation. A human can never turn an earlier escalate into allow, and earlier evidence, decisions and action records are never changed.
• "Ask EVA why" / automatic explanation. Answers are classed ESTABLISHED, NOT_ESTABLISHED or OUT_OF_SCOPE and built only from verified records; on an escalation the explanation is written automatically as its own sealed record.
• Hermes as proposer. The agent runs on a separate server. Each model step of the local Strands agent becomes one one-shot Hermes run that must return exactly one structured proposal. The adapter fails hard on anything else - there is no fallback to another model. Hermes never receives one of our tools.
• Amazon Bedrock + Amazon Nova Lite. The same frozen gate runs with amazon.nova-lite-v1:0 in eu-north-1 through Strands BedrockModel (supplier-risk flow, Oct 6, 2026): EVE allow -> tool executed once; EVE escalate -> not executed; EVE unreachable -> not executed, no record. Pass/fail is decided by observed system behaviour, never by the model's text.
• Spoken output is built from observed fields and verified records only (EVE outcome, gate result, execution, record id, the sealed explanation), never from model text.
• Testing. The current EVA suite passes 499 tests on Windows; every live run writes one new self-hashed evidence record and records are never rewritten.

Challenges we ran into

• Getting the first Bedrock call through on a new AWS account. Account verification blocked the first calls although model access looked ready, then the applied Nova Lite quotas were 0 ("Not adjustable") and surfaced as "Too many tokens per day". A support case resolved it on Oct 6. All failed attempts are kept as evidence and described in our friction log.
• A careful agent is not a cooperative one. In early live runs Hermes stopped to ask for approval instead of proposing the booking - a good safety reflex, but it meant the agent did not know the approval on record existed. We added the sealed, read-only history to its context, clearly labelled as historical, instead of loosening any rule.
• Keeping history honest. One early run silently used a scripted proposer instead of Hermes. We kept that record as a documented deviation rather than deleting it, and changed the run procedure.
• Resisting convenience. It would have been easy to let the agent supply policy, add a friendly boolean, or let a human "flip" an escalation. Each of those would have created a new authority outside EVE.

Accomplishments that we're proud of

• A third-party agent, an independent verification engine and a deterministic gate working together live - with EVE deciding and the gate enforcing.
• The same request leading to two different, explainable and verifiable decisions as the evidence changes - without rewriting anything.
• Explanations that are built from the records, not from the model.
• The same EVE boundary in front of different proposers (Hermes, Amazon Nova) in two different flows.

What we learned

Making the model more accurate does not answer the question "who verified that the evidence supported this action?". That question needs its own boundary - and it has to fail closed: unknown, malformed or unreachable must never become permission to act.

We also learned how much trust depends on what you don't do: no silent defaults, no caller-supplied policy, no overwritten history.

What's next for EVE

• Natural, conversational replies in EVA while keeping every statement grounded in observed fields.
• Running Amazon Nova as the proposer on the delegation flow.
• Governed execution: verifying not only before an action, but while it runs.
• Packaging EVE as a verification layer that any agent platform can call.

Built With

• amazon-bedrock
• amazon-nova
• boto3
• httpx
• mcp
• python
• starlette
• strands-agents
• uvicorn