SCOPELOCK

Change only what you meant. SCOPELOCK checks an AI-generated code diff against your planning contract and verifies the exact decision behind Aligned, Unplanned, or Conflict.

Comment

Inspiration

AI coding agents are getting very good at producing code that is technically reasonable. But technically correct code can still be the wrong change.

An agent may add a feature that feels like a natural extension even though the team explicitly moved it to “Later.” The code may compile, tests may pass, and the UI may look good — but the implementation has drifted from the product decision.

That gap between technical correctness and product alignment inspired SCOPELOCK.

The idea is intentionally narrow: use the planning artifacts we already create — scope, PRD, specification — as a contract, then ask one question about a proposed change:

Does this change still belong to the product we decided to build?

What it does

SCOPELOCK takes two inputs:

  • a Markdown planning contract;
  • one coherent proposed code change, pasted as a diff.

Claude proposes one of three product-alignment verdicts:

  • Aligned → Proceed
  • Unplanned → Review plan
  • Conflict → Stop

But the model does not get to certify its own evidence.

For an Aligned or Conflict result, it must propose one exact planning quote and the full Markdown heading path where that decision lives. SCOPELOCK then verifies that evidence deterministically against the original planning text.

The quote must match a complete normalized planning block, and it must exist under the exact claimed heading path. If the evidence cannot be verified, the result falls back safely to Unplanned. Model/API failures are kept separate as Check failed.

The central proof is the flip test:

  1. The sample diff adds CSV export.
  2. The plan explicitly puts “Export results as CSV” under Scope > Later.
  3. SCOPELOCK returns CONFLICT · Stop with verified evidence.
  4. The developer moves only that planning decision from Later to Now.
  5. The diff stays exactly the same.
  6. SCOPELOCK returns ALIGNED · Proceed, with the same quote now verified at Scope > Now.

Same code. Different human decision. Different action.

How we built it

SCOPELOCK is deliberately small:

  • Node.js 22
  • Express 5
  • plain HTML, CSS and JavaScript
  • Anthropic Claude through the server-side API
  • structured JSON output
  • a dependency-free deterministic Markdown parser and evidence verifier
  • no database, authentication, persistence or frontend framework

The architecture separates interpretation from proof.

model.js asks Claude to propose a verdict, explanation and evidence. markdown-blocks.js parses the planning contract into citable blocks and full heading paths. verify.js checks the proposed quote and location without AI. check.js combines both into the final product result.

The UI state machine is also isolated from the DOM so states such as Checking, Outdated, Not checked and Check failed can be tested directly.

The final build has 78 automated tests, all running without network access. I also ran a bounded live-model smoke set: 12/12 live smoke checks passed, including 6/6 canonical flip-test runs.

Challenges we ran into

The hardest problem was deciding how much trust to place in the model.

At first, verifying that a quoted sentence existed somewhere in the planning document looked sufficient. During scope review, the flip test exposed the flaw: the exact same sentence exists before and after the flip — only its location changes from Later to Now.

That led to the stronger rule: exact quote + exact full heading path.

The second challenge was keeping that rule strict without making the model unusable. Instead of weakening verification when the model missed, I treated model reliability as a separate hypothesis. The deterministic verifier was fixed first, then the prompt was tested against the live API.

There were also SDK-level surprises. The installed Anthropic SDK required a top-level object for the JSON-schema helper, and its default schema transformation weakened part of the discriminated evidence contract. I adapted the schema wrapper and disabled that transformation while preserving the product rules.

Finally, several edge cases were about honesty rather than features: a stale verdict after editing, an API failure being mistaken for Unplanned, or a technically unverified change looking approved. Those became explicit product states rather than hidden implementation details.

Accomplishments that we're proud of

I’m most proud that the trust boundary is visible in the product rather than being only an implementation detail:

The model proposes. SCOPELOCK verifies.

A model cannot turn its own confidence into approval. Verified evidence is required for Aligned and Conflict.

I’m also proud of how small the final proof stayed. It does not pretend to be a code reviewer, security scanner, project-management platform or CI system. It proves one narrow idea end to end.

The canonical demo makes that idea observable in seconds: the same diff changes from Conflict to Aligned only after the human changes the planning decision.

The implementation is backed by 78 automated tests and a bounded live validation set of 12/12 smoke checks, including 6/6 canonical flip-test runs.

What we learned

The biggest lesson was that better agentic development did not come from giving the coding agent more instructions. It came from making the product contract smaller and more precise.

The Devpost Learn process also showed me the value of flipped interaction. Instead of asking an agent to immediately generate an application from a finished specification, I let it question the idea, expose ambiguities and force decisions before implementation.

One of the most important product rules — verifying the evidence location, not just the quote — emerged from reviewing the plan visually, not from coding.

I also learned to separate two different questions:

  • Can the model interpret the intent?
  • Can the application prove the evidence behind that interpretation?

Keeping those concerns separate made the system easier to test and much easier to trust.

What's next for SCOPELOCK

The POC intentionally uses pasted planning text and one coherent diff.

The next useful step would be to connect the same trust model to the developer workflow:

  • read planning documents and diffs directly from a repository;
  • evaluate pull requests before merge;
  • add CI, IDE or coding-agent hooks;
  • handle mixed diffs by separating them into coherent changes;
  • support multiple evidence items when a change depends on several product decisions.

The principle would stay the same:

AI can interpret intent, but approval should be backed by evidence the system can independently verify.

Built With

Share this project:

Updates

Submission history