Runtime Command

Independent review of whether Codex actually fulfilled the original development instruction, based on the evidence in its execution report.

Comment

Inspiration

AI coding agents are becoming faster, more capable, and more autonomous. But a confident completion report is not the same as verified execution.

While building with Codex, we repeatedly encountered a practical problem: an execution report could sound complete while omitting evidence for required deliverables, runtime behavior, routes, or tests.

That led to one question:

Did Codex correctly execute the original development instruction?

What it does

Runtime Command independently compares:

  • the Original Development Instruction
  • the Codex Execution Report

It returns a structured review containing:

  • Review Result
  • Confirmed Facts
  • Findings
  • Evidence Assessment
  • Human Test Requirement
  • Risks
  • Omissions
  • Evidence Gaps

Runtime Command does not write or repair the code. It reviews whether the execution report provides enough evidence to support its completion claim.

How we built it

The public Build Week prototype includes:

  • a public-safe request and response contract
  • a server-side Review Authority boundary
  • a loopback-only Local Web Host
  • a structured Review interface
  • a synthetic local Stub Authority
  • schema validation
  • explicit error taxonomy
  • automated tests and runtime smoke tests
  • a reproducible local Demo

Architecture:

Browser
→ Local Web Host
→ Private Web Adapter
→ Review Authority interface

In the public synthetic Demo:

Browser
→ Local Web Host
→ Private Web Adapter
→ Loopback Synthetic Stub Authority

The browser receives no API key and does not call OpenAI directly. No deterministic parser fallback is used.

The production Review prompt and production Review Authority are intentionally not included in the public repository.

Challenges we ran into

The hardest challenge was maintaining a strict separation between execution claims, supplied evidence, confirmed facts, product judgment, and human validation.

During development, Runtime Command repeatedly found real gaps between what Codex reported as complete and what the report actually proved. These included missing HTTP layers, route mismatches, incomplete runtime verification, and incorrect public error mappings.

Accomplishments that we're proud of

We built a public-safe local prototype with:

  • structured request and response contracts
  • a server-side Review Authority boundary
  • a loopback-only Web Host
  • a synthetic Stub Authority
  • evidence-aware Review output
  • explicit error taxonomy
  • automated tests and reproducible runtime validation

The product also validated its own value during development by identifying several genuine execution-report gaps.

What we learned

Execution and execution verification are different capabilities.

A model can be highly capable at implementation while still producing an incomplete or overly confident completion report.

The stronger AI becomes, the more important independent verification becomes.

What's next for Runtime Command

The current public prototype demonstrates the core review workflow using a synthetic local Authority.

Future validation will focus on more real development cases while preserving the same principle: Human Test remains part of the system, and a completion claim is never treated as proof by itself.

Built With

Share this project:

Updates