CounterWorld

Inspiration

AI agent failures often begin with a sentence that sounds reasonable: “repository text is only data,” “the model will follow the safety prompt,” or “this tool permission should be harmless.” Documentation review can challenge the wording, but it does not show a concrete configuration in a declared test catalog where the assumption fails. Testing directly in production is expensive and unsafe, while asking another model for a safety verdict is difficult to reproduce.

We wanted a preflight step between prose review and production: let a strong model understand what the developer means, but make deterministic application code—not the model—decide each catalog run's outcome.

What it does

A developer gives CounterWorld three small artifacts:

1. an agent design note;
2. an explicit capability configuration;
3. a safety claim that should remain true.

GPT-5.6 compiles those artifacts into an inspectable HypothesisSpec: the claimed invariant, observed trust-boundary facts, exact evidence anchors, ambiguities, and implicit assumptions. The developer inspects and locks that intermediate representation.

From that point on, no model participates. A pure state machine exhaustively executes the complete 32-world v0 catalog across five Boolean dimensions: treatment of untrusted text, symbolic-secret read capability, external-sink write capability, tainted-sink enforcement, and prompt safety wording.

CounterWorld returns:

• all 32 worlds checked;
• the two worlds that falsify the no-exfiltration claim;
• the failing world nearest to the observed configuration under a declared stable ranking;
• a deterministic, hash-addressed trace;
• a catalog neighbor at Hamming distance one whose run does not falsify the claim;
• a controlled replay showing the exact policy bit that changes the result while four dimensions stay fixed;
• a portable JSON refutation record containing both traces, both trace hashes, and its own deterministic digest;
• a certificate that explicitly limits minimality to catalog v0.

The simulation uses only a symbolic protected value. It never executes pasted code, reads a real credential, or contacts an external sink.

How we built it

CounterWorld is a Next.js 16 and TypeScript application with three deliberately separated layers.

Semantic compiler. The server uses the OpenAI Responses API with GPT-5.6, medium reasoning, store: false, and Zod Structured Outputs. Model output is untrusted. Application checks validate exact quotes and line ranges, required evidence sources, explicit configuration facts, schema shape, and the resolved GPT-5.6 model before a hypothesis can be locked.

Deterministic verifier. Pure TypeScript enumerates all 2^5 = 32 declared, versioned worlds, runs a side-effect-free state machine, derives the catalog verdict from trace events, ranks falsifiers, and finds a one-bit non-failing neighbor. Controlled Replay then re-executes both runs, verifies that exactly one bit changed, rejects inconsistent traces, and emits a portable record with canonical SHA-256 evidence. The core verifier can replay-validate an exported record; its hashes are content addresses, not signatures. GPT-5.6 cannot create worlds, transitions, verdicts, rankings, or catalog-minimality claims.

Fail-closed product boundary. Requests are limited by content type, UTF-8 validity, byte count, exact production origin, credential patterns, timeout, and strict schemas. The judge deployment also requires a high-entropy access code and a short-lived signed HttpOnly session before either the UI or compile route is reachable. Without an API key, only the exact checked-in demo preset can use the verified offline fixture; arbitrary edited input never silently receives a fake model result.

The 88-case suite covers deterministic enumeration, runtime traces, the rendered comparison UI, access control, known SHA-256 vectors, stable ranking, prompt injection, prototype pollution, evidence fabrication, conflicting configuration, wrong resolved model, cross-origin requests, payload limits, inconsistent comparison traces and record hashes, fake two-bit remedies, and attempts by the model to own the verdict. Eighty-five cases pass in the default suite; three conditional cases are skipped until the real local Codex-child dogfood mode is enabled.

How we used GPT-5.6

GPT-5.6 performs the part that a rigid parser cannot do well: reconcile informal architecture prose, configuration facts, a developer's claim, evidence locations, ambiguity, and hidden assumptions into a constrained semantic intermediate representation.

This is a material runtime feature, not generated copy. General user input requires a validated GPT-5.6 result. The product makes the model's contribution visible through evidence anchors and provenance while deliberately preventing it from deciding safety.

We also dogfooded this boundary without API credits using an isolated, capability-disabled local Codex child explicitly pinned to GPT-5.6 Sol. The baseline and one-bit-safe cases must compile to their expected configurations. Instruction-shaped artifact text can pass only as fully grounded validated output; otherwise CounterWorld returns a sanitized fail-closed rejection with no unverified fallback.

How we used Codex

Codex was the primary engineering collaborator for the hackathon build. It helped:

• challenge the original fixed lesson concept and narrow the product to a finite counterexample preflight;
• separate GPT semantics from deterministic catalog verdicts;
• implement the Next.js UI, Responses API boundary, typed contracts, exhaustive catalog, state machine, search certificate, and tests;
• adversarially review prompt-injection, secret-handling, evidence-grounding, and fallback behavior;
• diagnose the repository's cross-CLI hook control plane and stage protected fixes with attack and false-positive corpora.

Human decisions included choosing Developer Tools as the primary audience, limiting v0 to one symbolic no-exfiltration invariant, rejecting real repository execution and real credentials, requiring a model-independent verdict, and keeping every minimality claim catalog-scoped.

Primary /feedback Session ID: 019f70a1-759e-7b53-a8e1-6102e5fa88e7

Challenges

The hardest design problem was using GPT-5.6 meaningfully without turning the product into an LLM safety judge. Structured output alone is not a trust boundary: a schema-valid result can still fabricate evidence or contradict explicit configuration. We added application-level evidence and cross-field validation, then placed the runtime behind a visible human lock.

The second challenge was being honest about “minimal” and “causal.” CounterWorld does not claim to prove a real agent safe. It certifies only that every world in a named finite catalog was checked, the selected witness is first under the declared ranking, and one controlled local intervention changes the outcome in the deterministic runtime. That limitation is part of the UI and exported record rather than hidden in fine print.

Accomplishments we are proud of

• A complete inspect → lock → exhaustive catalog check → controlled replay loop on one screen.
• Reproducible model-independent traces and a first-under-declared-ranking certificate.
• A replay-validated, hash-addressed JSON record with both worlds and both sides of the experiment.
• A meaningful GPT-5.6 role with exact evidence rather than decorative chat.
• A verified offline preset that remains fail-closed for all non-preset input.
• An 88-case unit, integration, UI, deterministic, security, and adversarial suite, including three conditional local Codex-child dogfood cases.

What we learned

The useful boundary is not “AI versus no AI.” It is semantic uncertainty versus catalog execution. Models are excellent at compiling meaning from messy artifacts; small deterministic programs are excellent at exhaustive evaluation inside a declared universe. Making that handoff inspectable produced a more trustworthy and more legible developer tool than either approach alone.

What's next for CounterWorld

The next catalog will add a second audited invariant for confused-deputy tool use, not an arbitrary world DSL. We also want a repository adapter that extracts a small, reviewable artifact bundle without executing repository code; comparison runs across two configuration revisions; shareable signed certificates; and short validation sessions with agent-framework maintainers and security engineers to test whether the witness changes a real configuration decision.

Built With

• codex
• gpt-5.6
• next.js-16
• openai-node-sdk
• openai-responses-api
• react-19
• sha-256
• structured-outputs
• typescript
• vitest
• zod