escrow

• 

  Escrow running in GitHub Actions: a successful instruction integrity check with a downloadable evidence artifact.

• 

  Escrow’s local-only interface checks the instruction chain, keeps network access opt-in, and runs approved commands in isolated worktrees.

• 

  A verified monorepo instruction file: pnpm, the real test:unit script, and a repository health check.

• 

  Escrow finds four stale instructions: npm conflicts with pnpm, a required file is deleted, pnpm test is missing, and Jest is not installed.

Escrow

Coding agents increasingly rely on AGENTS.md and other repository instructions. The problem is that those files can quietly become wrong: a project moves from npm to pnpm, a test script gets renamed, documentation disappears, or a dependency changes.

Escrow makes those instructions testable.

It reads repository guidance, turns it into structured claims, and checks those claims against the actual codebase. It can verify package managers, package scripts, dependencies, paths, nested instruction scopes, and explicitly approved commands.

Our core idea is simple:

AI interprets the instruction. Deterministic repository evidence decides whether it is true.

What inspired us

We kept seeing the same failure mode with coding agents: the repository’s instructions sounded confident, but they were stale. An agent following outdated guidance can use the wrong package manager, run a nonexistent command, or make incorrect assumptions before it even begins real work.

We wanted a tool that treats agent instructions like something worth testing, not just documentation that developers hope stays accurate.

How it works

Escrow finds the effective AGENTS.md and AGENTS.override.md files for a target directory, including nested scopes.

GPT-5.6 is used where language understanding is useful: extracting candidate claims from natural-language instructions and proposing restricted documentation repairs.

But GPT-5.6 does not decide whether a claim passes or fails. Escrow’s TypeScript validators compare claims with deterministic repository evidence:

• lockfiles and package.json metadata for package-manager claims
• nearest package.json scripts for documented commands
• dependency metadata for tool and framework claims
• Git-root-bounded path resolution for referenced files
• temporary Git worktrees for approved command execution
• scope resolution for nested instruction overrides

Escrow produces console, JSON, Markdown, HTML, browser UI, and GitHub Actions reports. It can also preview a restricted repair, but only permits changes to instruction files and rejects source code, tests, lockfiles, build files, and CI changes.

How we built it

Escrow is a TypeScript and Node.js CLI with a local browser interface and a GitHub Actions integration.

We used Codex throughout development to plan the architecture, implement features, write and review tests, debug workflow issues, document the project, and integrate the GitHub Action.

At runtime, GPT 5.6 extracts structured instruction claims and can propose narrowly constrained repairs. For a no-cost self-hosted GitHub Actions demo, Escrow can also use local Ollama models while preserving the same deterministic validation layer.

Challenges we faced

The hardest part was drawing a real trust boundary around AI.

It would have been easy to ask a model whether an instruction “looks correct.” But that is not enough for a tool that agents and teams may rely on. We redesigned the flow so the model only interprets language; deterministic code makes every final verdict and records the evidence behind it.

We also had to make command verification safe. Escrow never runs documented commands in the active repository. It uses temporary Git worktrees, blocks unsafe commands, and disables network-capable commands by default.

Finally, smaller local models can extract inconsistent metadata. We improved the pipeline so explicit instructions are normalized against their original source text before validation, preventing model interpretation mistakes from becoming misleading results.

What we learned

We learned that the best use of AI in developer tooling is often not replacing verification, but making unstructured human language usable by reliable systems.

Escrow is not “AI that reads documentation.” It is repository instruction integrity infrastructure: AI helps understand what a repository claims, and deterministic evidence establishes whether that claim is true.

Built With

• actions
• codex
• css
• git
• github
• gpt-5.6
• html
• node.js
• ollama
• openai
• qwen
• typescript
• vitest
• zod