Agent Black Box

Every Run tells the truth.

Comment

Inspiration

What it does

Inspiration

AI coding agents can do far more than return text. They can inspect files, edit code, run commands, call models, and keep working across multiple turns.

But when an agent fails, the operator often sees only something like failed, an error message, or even an overly optimistic final response. That is not enough to answer the questions that actually matter:

  • What really happened during the run?
  • Which observable step failed first?
  • Did a command actually succeed?
  • Can I inspect the evidence without exposing secrets?
  • Can I recover without rewriting or losing the original failure?

That led me to build Agent Black Box: lightweight middleware for Agent Launchpad that turns every observable Agent Run into a correlated, bounded, and redacted timeline.

The goal is simple:

A black box for AI agents: trace every run, diagnose failures, and recover safely.

What it does

Agent Black Box extends the existing Track 1 Agent Launchpad instead of replacing it.

Every Agent Run gets a structured timeline containing observable events such as:

  • Run queued and started
  • Runtime started and completed
  • Codex turn lifecycle
  • command execution and exit status
  • workspace file changes
  • model usage
  • explicit errors
  • cancellation, timeout, and restart interruption
  • final Run outcome
  • linked retry and recovery information

Each event is correlated using the Agent ID, Run ID, trace ID, sequence number, status, timestamp, and safe metadata.

The middleware intentionally does not expose hidden model reasoning or raw command output.

Before diagnostic information reaches storage or the browser, Agent Black Box applies server-side bounds, an allowlist, path sanitization, and secret redaction.

It also adds explicit failure classification for conditions such as:

  • non-zero command or Runtime exit
  • timeout
  • cancellation
  • Runtime startup failure
  • output limit
  • Codex errors
  • interrupted Runs after a server restart

When a Run fails, the operator can create a linked retry from the persisted workspace. The failed Run remains unchanged and inspectable while the retry becomes a new attempt with its own Run ID and recovery mode.

A failure should stay a failure

One of the most interesting moments during testing came from a real ModelArk-backed Agent Run.

The Agent created a TypeScript CLI and test. Its first test command failed because of an incorrect import. The Agent fixed the source code afterward and then reported that the test had passed — without actually rerunning the test.

A normal chat interface could easily leave the operator with the impression that everything succeeded.

Agent Black Box preserved the observable evidence:

  • the test command ran;
  • it returned a failure;
  • the source was modified afterward;
  • no second test command appeared in the trace;
  • the overall Agent Run still completed.

A separate zero-inference verifier later confirmed that the corrected files did pass.

That experience reinforced the central idea behind the project: the final Agent message is a claim; the execution trace is evidence.

Failure and recovery demo

For a reproducible negative case, Agent Black Box includes a controlled Runtime failure that exits with code 17.

The failure travels through the same:

Runner → JSONL adapter → redaction → persistence → API → frontend timeline

used by normal Runs.

The injected failure contains a safe fake bearer-token canary. The server removes the value before it reaches trace persistence or the browser, displaying only:

[REDACTED]

The operator can then select Retry from persisted workspace.

Retry does not pretend to restore an exact process checkpoint. Instead it creates a new immutable Run, links it to the failed attempt, reuses the persisted Agent workspace, records the actual recovery mode, and keeps both attempts available for inspection.

How I built it

The project preserves the architecture of the provided Agent Launchpad and adds middleware at the existing Runtime and control-plane boundaries.

Frontend

The existing React + TypeScript Playground was extended with an Agent Black Box panel containing:

  • Run history
  • ordered trace timeline
  • sanitized event details
  • failure-boundary highlighting
  • redaction information
  • retry controls
  • links between related attempts

Control plane

The Fastify backend owns the trace and recovery lifecycle.

I added:

  • typed trace events
  • stable failure codes
  • Run correlation
  • trace APIs
  • retry APIs
  • idempotency protection
  • serialized Run transitions
  • restart-interruption handling

Runtime instrumentation

Both the local-process and disposable-container Agent runners use a shared adapter for the observable Codex JSONL protocol.

Only explicitly supported event types are recorded. Unknown protocol events, hidden reasoning, final-response internals, and raw command output are not copied into the trace store.

Redaction and bounded storage

Diagnostic data passes through one server-side sanitization boundary before persistence.

The implementation:

  • redacts common tokens, passwords, API keys, Authorization headers, URL credentials, and sensitive query parameters;
  • converts absolute or traversal file paths into safe placeholders;
  • limits command previews;
  • bounds metadata size;
  • limits file-change lists;
  • caps the number of trace events while reserving space for terminal evidence.

Recovery

Retries are represented as new Runs rather than mutations of the original failure.

A retry records:

  • source Run
  • root Run
  • attempt number
  • recovery mode
  • persisted workspace reuse
  • idempotency key

This preserves the complete history of what actually happened.

Challenges

Preserving the starter application

One of the main constraints was avoiding a redesign of the provided platform.

Agent CRUD, Playground chat, asynchronous Runs, persistent workspaces, resumable Codex sessions, the JSON store, and the Runtime architecture all had to continue working while the middleware was added around them.

Capturing useful evidence without capturing too much

Raw Runtime output would have been easy to store, but it could contain credentials, sensitive user data, or information that should not be exposed.

I instead built an allowlisted event contract that keeps useful operational evidence while deliberately discarding raw output and hidden reasoning.

Making failure reproducible

Depending on an actual provider outage would make the demo unreliable.

I created a controlled Runtime failure that uses the same execution, parsing, persistence, API, and UI path as a normal Run while remaining deterministic and clearly labelled as injected test evidence.

Defining honest recovery semantics

"Resume from failure" can easily imply process checkpoint restoration.

Agent Black Box does not make that claim. Recovery creates a new attempt using persisted state and explicitly records whether recovery uses only the workspace or both the workspace and an existing session.

External model availability

Model access was another challenge during the hackathon. I therefore kept the middleware independently testable with credential-free Runtime proofs while preserving the normal ModelArk-backed Agent path.

I later verified the full model-backed path using a live free-quota ModelArk Run.

What I learned

The biggest lesson was that observability is only valuable when the evidence boundary is trustworthy.

Collecting more data is not automatically better. For an Agent system, a smaller set of well-defined, bounded, correlated events can be more useful than dumping every internal object or model message.

I also learned that recovery needs precise semantics. A retry from persisted state is useful, but it should not be described as exact checkpoint restoration when it is not.

Finally, the live ModelArk run demonstrated something I did not expect so clearly: an Agent can produce a confident final response that does not perfectly reflect the commands it actually executed.

That made the value of execution-level evidence much more concrete.

Verification

The project includes automated tests covering areas such as:

  • Agent lifecycle regression
  • trace ordering and deduplication
  • secret redaction
  • bounded trace storage
  • safe workspace paths
  • JSONL event mapping
  • malformed and unknown events
  • timeout and output-limit failures
  • cancellation
  • restart interruption
  • one-active-Run concurrency
  • immutable retry behaviour
  • retry idempotency
  • Runtime command construction
  • historical JSON-store compatibility

The repository also includes a one-page architecture diagram, local setup instructions, a controlled failure demo, documented limitations, and evidence from a real ModelArk-backed Agent Run.

Limitations

Agent Black Box is intentionally scoped for the hackathon:

  • the JSON store is single-process;
  • trace capture is allowlisted rather than exhaustive;
  • ordinary containers are not hardened multi-tenant isolation;
  • existing Playground prompts and final chat messages are outside the new trace-redaction guarantee;
  • controlled Runtime proofs are clearly distinguished from model inference;
  • retry is a new execution attempt from persisted state, not restoration of an exact instruction pointer;
  • retries involving arbitrary external side effects are not guaranteed to be idempotent.

These boundaries are deliberate so the system can be clear about what it actually proves.

What's next

Future work could include:

  • OpenTelemetry-compatible trace export
  • pluggable model and tool-provider adapters
  • configurable retention policies
  • richer anomaly detection across Runs
  • external append-only trace storage
  • policy-aware retry for side-effecting tools
  • comparison of Agent claims against execution evidence

For this version, I focused on making one capability reliable and demonstrable:

turning opaque Agent execution into evidence that an operator can inspect, trust, and recover from.

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Agent Black Box

Built With

Share this project:

Updates

Submission history