Inspiration
The official Track 1 Starter Kit already works. It gives you Agent CRUD, a Playground, persistent workspaces, and a Codex runtime that really executes commands. So we asked what it doesn't give you, and the answer was uncomfortable: a Run is a chat blob — {status, prompt, output, error}. The Codex --json events stream in, get parsed, and are thrown away. When something goes wrong, nobody can say which step failed. And key-like strings land unredacted in launchpad.json.
That is not a UI problem. It is a missing plane.
What it does
Launchpad Trace Plane is control-plane middleware. It does not rebuild the UI, the CRUD API, the Playground, the Codex integration, or the ECS path.
On every Playground message the backend opens a root invoke_agent span, runs a policy check for secret exfiltration, and — if allowed — spawns the runner and maps each Codex JSON event into a child span: chat {model}, execute_tool shell, execute_tool apply_patch, runtime.spawn. Attributes are redacted twice: once when the span is written to disk, and again before the HTTP response leaves the server. The trace is served from GET /api/runs/:id/trace.
The Playground renders it as a waterfall — indented by parentSpanId, offset by startedAt, bar width proportional to durationMs, coloured by span kind — with token usage, an estimated cost, span filters, JSON export, a token delta against the previous Run, retry linkage, and a two-Run compare.
If the prompt asks for a secret, the Run is denied in the control plane: a policy.check span with status=denied, the Run fails, and the runner is never invoked. The trace of a refusal has the same shape as the trace of a success — most guardrail layers return an error and leave no audit artefact at all.
How we built it
Four pieces, all behind Fastify:
- TraceCollector — mints traceId/spanId/parentSpanId, redacts, and persists spans with debounced writes.
- PolicyGate (
policy.ts) — a pre-run prompt check plus a live per-command check during the run. parseCodexEventLine— maps Codex events to spans; unknown item types become a generic span recording the event keys, never the raw line.- The React panel — reads stored spans. It enforces nothing.
Span names follow the OpenTelemetry GenAI semantic conventions. We say aligned, not compliant: every gen_ai.* attribute is still at Development stage in open-telemetry/semantic-conventions-genai, breaking changes are permitted, and there is no tagged release to pin. policy, runtime and sandbox have no well-known equivalent and remain custom values, which the spec explicitly allows.
Token counts are recorded as gen_ai.usage.input_tokens / output_tokens, and each span also carries gen_ai.usage.source naming where the number came from — Codex turn.completed, not a billing API. The dollar figure stays labelled est. because the per-million rates are a fixed placeholder.
TRACE_CAPTURE_CONTENT mirrors OTel's Opt-In rule for content attributes. It defaults on, because the failing command and its exit code are the deliverable of an audit trace; turn it off and commands and error text become [content capture disabled] while status, exit codes, tool names and the tree survive.
Challenges we ran into
The "Open failing step" button was landing on an empty span. A failure marks every span on the path to it — the root invoke_agent and the runtime.spawn envelope get flagged too, but they carry no diagnostics. The first-match selector therefore opened {promptChars: 62} during the exact beat the 30-second judging test measures. The fix ranks problem spans by depth and picks the innermost one that actually carries errorText / exitCode / ruleId / reason, falling back to the deepest. Three regression tests lock it in: the deny path resolves to policy.check with its ruleId, a mid-run failure resolves to the failing command, and a spawn error still falls back to the root.
Failed Runs reported no token usage. usage was only assigned on the success path, so a Run that spent 900 input tokens and then failed persisted usage: null — which is precisely the case the Track A gate asks about ("a failing step and available model usage"). It is now recovered by summing the model spans already collected.
The documented Windows quickstart could not start Codex. npm i -g @openai/codex writes codex, codex.cmd and codex.ps1 but no .exe, and the runner spawns without a shell — deliberately, because the user prompt is an argv element and shell: true would be a command-injection hole. The result was a bare ENOENT and a UI claiming Codex was not installed. Fixed by documenting CODEX_BIN pointed at the package's .js entrypoint, which the runner already knows to run under Node.
A debounced write could kill the server mid-demo. The interim trace persist was a bare void store.mutate(...); on Node 22's default --unhandled-rejections=throw, one failed write — entirely plausible on Windows under an AV scanner — would take the process down. Now caught and dropped; the final trace is written again by persistTrace.
Accomplishments that we're proud of
The denial is provably backend-enforced, not cosmetic. glass-box-http.test.ts drives a real Fastify app through happy path → deny and asserts runnerCalls === 0, and a separate test asserts the protected fixture's SHA-256 is unchanged across the denial. The acceptance checklist explicitly disqualifies middleware implemented only in the browser; we have the assertion.
Redaction happens at the write path, not at render. redactDeep runs inside startSpan, endSpan and snapshot(), so a configured secret never reaches launchpad.json in the first place — and a test asserts it is absent from the entire serialized HTTP trace body.
48 automated tests, all hermetic: no Docker, no network, no live model, ~3 seconds.
What we learned
- A failure marks the whole path, so "which span failed" is the wrong question. The useful one is "which span on that path can actually explain it."
- Alignment is a claim you have to bound. The OTel GenAI conventions are genuinely the right vocabulary and genuinely unstable; saying "OTel-compliant" would have been the easy sentence and the false one.
- Renaming a token field is not a semantic migration.
gen_ai.usage.input_tokensis specified to be the billed, cache-inclusive count. Ours is whatever Codex reports, so the span says so rather than implying a conformance we did not verify.
Limitations
Single-user JSON store; ordinary containers are not multi-tenant isolation. Redaction is a best-effort regex net, not a secret scanner — no regex survives a value the agent transforms, and the control that actually keeps a secret out of a Run is the policy gate, which denies before the runner is invoked. The policy is a pattern denylist and a novel prompt may bypass it. Unknown Codex item types are recorded as generic spans, and one of them — Codex's non-fatal error item — is currently rendered without diagnostic attributes; the fix requires threading its message through and is deliberately deferred rather than shipped untested.
What's next
Native OTLP export, so these spans land in Jaeger, Tempo or Datadog with no new UI to adopt — the span model is already the right shape. Session grouping above the trace: a Codex thread is resumed across turns, so a multi-turn session is today N unlinked traces, and OTel already names the level above the trace (gen_ai.conversation.id). A quality signal on a green Run — exit code 0 with no error span counts as success even when the output is wrong; gen_ai.evaluation.* is reserved for exactly that.
Log in or sign up for Devpost to join the conversation.