SEAL | Devpost

Each new version of the harness is recorded
Agent Harness Architecture is evaluated using the code
All evolutions are logged
Every successful run shows the traces and relevant data
Database Query Agent with database viewer on the right
Every question that is unanswered is logged as a capability gap

Inspiration

AI agents often fail because they lack the right tool, even when the information they need is available. We wanted to build an agent that could recognize that limit, learn from it, and gain a reusable capability.

What Seal does

SEAL (Self Evolving Agent Layer) sits around a Database Query Resolution Agent. Suppose a user asks a question that requires combining information from several collections, but the agent’s tools can only retrieve one document at a time. The agent acknowledges that it cannot answer reliably and records a capability gap.

An Analyst agent examines the failure in Arize Phoenix traces. If the tools caused the limitation, it creates a tool for the missing query pattern. SEAL tests that tool against the original question, unseen variations, and existing regression cases. Only after it passes does it add the tool to the harness for future requests.

The result is a new class of capabilities, rather than a one-off answer.

How we built it

We built SEAL around a bounded Database Query agent whose initial tools can inspect a table, read limited pages of rows, and perform simple calculations. When a larger question exceeds those tools, SEAL records the failed run and investigates its trace in Arize Phoenix. It proposes a change to the agent’s tool code, then runs the original and candidate versions in separate Daytona sandboxes against the same frozen data and evaluation cases. SQLite stores the datasets, run history, and promotion evidence. SEAL activates a new capability only when it solves the original task and passes unseen cases and regressions.

Challenges and what we learned

The hardest distinction is between an agent that reasoned poorly and an agent that genuinely lacks a tool. Adding a tool also requires more than passing the question that exposed the gap: it must work on new cases without breaking existing ones.

Building SEAL taught us to treat an agent’s improvement as something that needs evidence, reproducible evals, and a promotion decision.