AskFold

Short description

AskFold is an ambiguity firewall for coding agents. Before code changes, it compares two independent interpretations, asks at most one decision-critical question, locks the answer, builds in a disposable Git worktree, and audits the complete diff and test evidence against that lock.

Inspiration

Coding agents are already fast. The expensive failure mode is a fast, polished implementation of the wrong interpretation: retry the delivery or the whole job, redact a field or remove it, preserve compatibility or break it cleanly. Those choices are often buried in prose and discovered only after code review.

AskFold moves the acceptance contract in front of implementation. It treats unresolved intent as a first-class engineering risk, makes the decision inspectable, and refuses to silently improvise when a second independent critical ambiguity remains.

What it does

Two independent, read-only planner threads interpret the same issue and bounded repository evidence.
A comparator finds decision-critical divergence. AskFold asks no more than one focused question; another unresolved critical decision stops as NEEDS_SPEC.
The answer becomes a checksummed intent.lock.json containing required and rejected behavior, allowed paths, and exact validations.
A lock-bound builder returns a bounded full-file change plan. AskFold applies it only inside an AskFold-owned disposable Git worktree; the original source tree is not the writable model workspace.
Trusted validations run by exact executable and argument mappings without a shell. An independent auditor reconciles the lock, complete diff, and test evidence.
AskFold returns ALIGNED, MISALIGNED, or INCONCLUSIVE and writes hash-committed JSON plus a self-contained three-surface HTML report.

How we built it

AskFold is a TypeScript CLI for Node.js 20.19+, npm 10+, and Git 2.40+. Its runtime uses strict schemas, runtime-owned model and thread identity fields, path and evidence bounds, secret redaction at two persistence boundaries, owned-worktree cleanup proofs, and fail-closed state transitions.

Codex was the primary development environment from product framing through implementation. It helped turn the PR/FAQ into acceptance criteria, trace implementation/test/document contracts, generate and review TypeScript, run adversarial security checks, diagnose cross-platform CI, and reconcile the final evidence. GPT-5.6 was used for the architecture, security boundary, evaluation design, and final live semantic and sandbox review. That combination compressed the loop from design decision to tested evidence while keeping the final claims tied to artifacts.

Routine CI and the judge-ready offline path make zero model calls and require no API key. The local product path uses OpenAI account authentication and the user's Codex allowance. Paid submission evaluation is isolated in a manual protected GitHub environment and pinned to gpt-5.6-luna.

The 92-second demo was captured with OBS from a browser-only source, so no desktop, taskbar, background monitor, or unrelated application can appear. It transparently replays the exact evidence from one completed gpt-5.6-sol run, makes zero additional model calls, then pans through that run's real self-contained HTML report from top to bottom and finishes on an unobstructed full-report overview. The Sol run used ChatGPT OAuth, inherited no API key, and recorded 50,142 input plus 2,676 output tokens across five bounded turns. The original OBS audio was discarded; the final video contains English narration and burned-in English subtitles only.

The public commit keeps its low-cost Spark default. For this requested Sol demo, an isolated copy of that frozen commit changed only the product model binding and its evidence-schema literal; the exact two-line patch and complete Sol evidence bundle are included with the submission collateral. The public repository and prebuilt judge package were not modified.

Challenges

Distinguishing a genuine product decision from a cosmetic implementation difference without letting a model assign its own severity.
Asking one useful question while stopping safely if one answer cannot resolve every independent critical ambiguity.
Letting a coding agent produce real changes without giving it a writable source repository or free-form shell access.
Separating deterministic contract evidence from semantic model-quality claims so a clean offline score is never presented as an accuracy benchmark.
Producing useful audit evidence without persisting secrets, unbounded diffs, or unverifiable cleanup claims.

Accomplishments

The frozen commit passed the same-commit proof chain:

Three-OS CI: Windows, macOS, and Ubuntu all succeeded.
Full GPT-5.6 Luna evidence: 8/8 semantic cases and 3/3 adversarial sandbox probes passed.
Semantic usage: 201,819 input + 14,111 output = 215,930 observed tokens.
Sandbox usage: 41,397 input + 454 output = 41,851 observed tokens.
Full run usage: 257,781 observed input+output tokens.
Cumulative paid evaluation usage: 751,811 observed tokens, below the explicit 2,500,000-token ceiling.

The sandbox probes verify denial of a read-only-workspace write, a repository root escape, and outbound network access. The offline verifier separately proves the real Git/worktree, registered node:test, audit, cleanup, and artifact-integrity path with no model call.

What we learned

The most useful intervention point is before code, when two plausible interpretations can still be compared cheaply.
Model output becomes easier to trust when runtime code owns severity, paths, identity, evidence limits, and terminal verdict rules.
A polished verdict is not enough. Reviewers need the locked intent, rejected behavior, complete change evidence, validation result, and limitations on one inspectable surface.
Deterministic and live evidence answer different questions and should remain separate.

Why it matters

AskFold targets “right code, wrong intent,” a failure mode that ordinary tests often miss because the tests were written from the same mistaken assumption. It can fit between an issue and any coding-agent workflow as a decision and evidence layer. The interaction is intentionally small—zero or one question— while the resulting contract is precise enough for builders, auditors, and humans to inspect.

What’s next

Add adapters for more issue trackers and coding-agent runtimes while preserving the same lock and evidence contract.
Improve the one-question explanation so teams can see why the decision is critical before choosing.
Add signed evidence bundles and policy packs for organization-specific paths, validations, and approval boundaries.
Measure avoided rework in real team workflows without weakening the fail-closed behavior.

Judge path

The prebuilt askfold-0.1.0.tgz installs without compiling AskFold. Run askfold verify --offline from an empty directory to produce a 5-check, hash-linked acceptance bundle with no key and no model call. Exact Windows, macOS, and Linux commands and limitations are in judge-testing.md.

Exact evidence identities

Semantic result SHA-256: 59d80102ba1dd9b179d6938fd9b21dbd2f94005bf1c396246e593a5b9431371a
Sandbox result SHA-256: 6cbf3158510bf268ec8d6f6ba065ebcbc3ea767f8dcc8832379f51fdeb2c7877
Prebuilt package SHA-256: 95678aa13ecdab549e23799e623bd36c33f121b7713386f2dadefef90a710ab7
Final narrated video SHA-256: de31eb5f143035b90a024a7f4c500edd0144feaff3f823f6c9ba91442c4c1908

Built With

git
github-actions
gpt-5.6
json-schema
node.js
openai-codex
typescript