IntentTrials

Turn ambiguous software requests into evidence-backed alternatives, then lock reviewed human intent into executable criteria before agents build.

Comment

Inspiration

AI coding agents are becoming remarkably good at turning instructions into working code. But there is a deeper problem that conventional testing does not solve: an agent can implement the wrong interpretation of a request perfectly.

A request such as “add sharing to the dashboard” sounds clear, yet it hides several product decisions:

  • Should sharing use a public link or invitation-only access?
  • Should access be permanent or expire automatically?
  • Which behavior should become part of the product contract?
  • Which rejected alternatives should be remembered?

A coding agent can silently choose one interpretation, produce clean code, and pass every test while still building something the user never intended.

I built IntentTrials because AI-assisted development needs more than faster implementation. It needs a reliable way to expose ambiguity before code is accepted, compare real alternatives with evidence, and preserve the final human decision as an executable contract.

The core principle behind IntentTrials is:

AI should build and prove the alternatives. The human should make and lock the product decision.

What it does

IntentTrials is a developer tool that turns an underspecified feature request into runnable, evidence-backed counterfactual implementations.

For the current demonstration, the request is to add sharing to a dashboard. GPT-5.6 identifies material decision axes such as:

  • Public-link access versus invitation-only access.
  • Permanent access versus seven-day expiration.

Instead of allowing the model to silently select one interpretation, IntentTrials creates three isolated implementation variants using Codex inside separate Git worktrees.

Each variant is evaluated through deterministic verification, including:

  • Type checking.
  • Automated tests.
  • Production builds.
  • Browser journeys.
  • Accessibility analysis.
  • Screenshot comparison.
  • Artifact hashing.
  • Source allowlist validation.
  • Git and execution provenance.

The dashboard presents one product decision at a time. The human reviewer sees the alternatives, examines the supporting evidence, selects the intended behavior, and records a rationale.

IntentTrials then generates a versioned intent.lock.json containing:

  • The selected human decisions.
  • The rejected alternatives.
  • The reviewer’s rationale.
  • Evidence references.
  • Acceptance criteria.
  • Protected executable tests.
  • Integrity hashes.

The selected intent persists across refreshes and application restarts. If protected tests or locked decisions are silently modified, IntentTrials detects the tampering and rejects the lock.

IntentTrials does not ask AI to decide which product behavior is correct. It uses AI to expose and implement the alternatives, deterministic systems to prove their differences, and human judgment to make the final decision.

How I built it

IntentTrials is implemented as a TypeScript monorepo with four principal workspaces:

  • A React and Vite dashboard for reviewing alternatives, evidence, and intent decisions.
  • A Fastify and Node.js orchestrator for workflow execution, state management, Codex integration, Server-Sent Events, and SQLite persistence.
  • A shared contracts package containing versioned Zod schemas and TypeScript types.
  • PulseBoard, a trusted synthetic application used to demonstrate counterfactual feature implementation.

The main technologies include:

  • TypeScript.
  • React.
  • Vite.
  • Node.js.
  • Fastify.
  • SQLite.
  • Zod.
  • OpenAI Codex SDK.
  • GPT-5.6.
  • Git worktrees.
  • Vitest.
  • Playwright.
  • Axe accessibility analysis.
  • PowerShell automation.

How GPT-5.6 is used

GPT-5.6 performs structured ambiguity analysis. Its output must conform to strict schemas and identify meaningful product-decision axes rather than superficial implementation differences.

For the PulseBoard scenario, it identifies the access model and expiration policy as decisions that should remain under human control.

GPT-5.6 is also used inside bounded Codex workflows for implementation tasks that require deeper repository-level reasoning.

How Codex is used

Codex was used throughout the development process to:

  • Inspect and understand the repository.
  • Design the architecture.
  • Implement the monorepo.
  • Create tests and verification gates.
  • Diagnose Windows-specific problems.
  • Improve accessibility.
  • Validate Git-worktree isolation.
  • Strengthen secret handling and security boundaries.
  • Verify each development milestone.

IntentTrials also launches real Codex builders inside isolated Git worktrees. Each builder operates under explicit constraints:

  • A registered source allowlist.
  • Predefined verification actions.
  • A sanitized environment.
  • Bounded retries.
  • Restricted network access.
  • Explicit workspace-write permissions.
  • Commit and parent-SHA provenance.
  • Mandatory cleanup verification.

The model can generate implementations, but it cannot declare itself successful. Deterministic verification remains authoritative.

Challenges I ran into

The hardest part was not generating code. It was creating safe, auditable boundaries around agent-generated code.

One challenge was ensuring that three Codex builders could modify separate implementations without touching the trusted checkout. I addressed this with isolated Git worktrees, frozen base commits, source allowlists, and cleanup checks.

Another challenge was preserving reliable provenance. Every accepted variant needed to remain connected to its original builder commit, verification attempt, evidence receipts, and generated artifacts. IntentTrials records commit relationships and verifies artifacts using SHA-256 hashes.

Windows introduced additional complexity around process handling, paths, Git worktrees, browser automation, and Codex CLI execution. I created dedicated PowerShell workflows and bounded Windows concurrency to make the process reproducible.

I also had to prevent secrets from reaching browser bundles, child processes, worktrees, or evidence artifacts. The solution combines ignored environment files, sanitized child-process environments, browser-bundle secret scanning, source-free evidence traces, and dependency auditing.

During the G5 evidence gate, the first real run produced 36 passing receipts and six identical accessibility failures inherited from the shared PulseBoard base. Instead of bypassing the gate, I registered and applied one permitted ARIA correction, preserved every builder parent SHA through auditable child commits, and reran the full verification process.

The corrected run passed the evidence, artifact, repository, audit, accessibility, and secret gates.

Accomplishments that I’m proud of

IntentTrials is not a mock multi-agent interface and does not display invented model scores.

Codex builds real source-code variants inside real isolated Git worktrees, and the evidence shown in the dashboard comes from actual deterministic checks.

The verified submitted scope can:

  • Analyze ambiguous requests with GPT-5.6.
  • Identify material product-decision axes.
  • Build three isolated Codex variants.
  • Collect deterministic evidence for each implementation.
  • Compare alternatives one decision axis at a time.
  • Keep the final decision under human control.
  • Persist decisions across refreshes and restarts.
  • Generate acceptance criteria.
  • Generate protected executable-test documents.
  • Create a versioned Intent Lock.
  • Detect tampering with locked intent and protected tests.
  • Verify repository, browser, artifact, audit, and security boundaries.

The final offline verification passed:

  • 18 test files.
  • 99 automated tests.
  • TypeScript checks.
  • ESLint with zero warnings.
  • Production builds.
  • Two Playwright end-to-end journeys.
  • Browser preflight.
  • Browser-bundle secret scanning.
  • Dependency auditing with no known vulnerabilities.

The deeper development gates also passed:

  • G3: live GPT-5.6 structured-analysis gate.
  • G4A: deterministic Git-worktree gate.
  • G4B: real Codex-builder gate.
  • G5: deterministic evidence and artifact gate.
  • Milestone 6: human review and Intent Lock workflow.

I am especially proud that IntentTrials treats human intent as a durable engineering artifact rather than a temporary chat message.

What I learned

The most important lesson was that passing tests does not necessarily mean the correct product was built.

Tests can preserve an incorrect assumption just as effectively as they can prevent a regression. If the original product decision was never made explicitly, a technically excellent implementation can still be wrong.

I also learned that AI-generated summaries and confidence scores should not be treated as proof. Model reasoning is useful for proposing and implementing alternatives, but builds, tests, browser behavior, accessibility checks, hashes, and repository provenance should determine whether an implementation is valid.

Another lesson was that human oversight becomes more valuable when it is structured. Asking a user to review an entire generated application is overwhelming. Presenting one meaningful product decision at a time, together with focused evidence, makes review more practical and defensible.

Finally, I learned that the real value of AI-assisted development is not simply producing code faster. It is creating better collaboration between machine reasoning, deterministic verification, and human judgment.

What’s next for IntentTrials

The next major milestone is verified synthesis.

After the human selects the intended behavior, IntentTrials will generate a final implementation that combines those selected decisions while remaining constrained by:

  • The Intent Lock.
  • The reviewer’s rationale.
  • The acceptance criteria.
  • The protected executable tests.
  • Repository safety policies.
  • Deterministic verification gates.

Future directions include:

  • Supporting additional frontend and backend frameworks.
  • Making Intent Locks portable across coding tools.
  • Integrating IntentTrials into CI pipelines.
  • Adding reusable decision templates.
  • Supporting product managers, developers, and reviewers in the same workflow.
  • Enabling multi-reviewer approval.
  • Providing hosted sandbox environments for easier evaluation.

IntentTrials is based on a simple belief:

AI-assisted development should not hide product decisions inside generated code. It should expose the alternatives, prove the differences, and keep the human in control.

Built With

Share this project:

Updates