ChangePlane

Keep GitHub. Let agents ship. ChangePlane autonomously repairs fixable agent PRs, then independently verifies the exact commit because agents should never certify themselves.

Comment

ChangePlane

Keep GitHub. Let agents ship.

Autonomous repair without autonomous self-approval.

About the project

Why I built it

An AI agent can write the patch, edit the test, push the commit, and report green. If the same actor controls both the change and the verdict, green is not assurance. It is self-approval.

I ran into this problem in RouteThai's production route-planning workflow. A routing change can look perfectly reasonable in review and still place a stop outside its service window. I wanted the speed of autonomous repair without giving the model a way to weaken the evidence or certify its own work.

That gap became ChangePlane: an independent control layer between coding agents and GitHub. The normal path stays autonomous. Only exceptions go to a human.

What ChangePlane does

ChangePlane watches each pull request update and turns it into a revision-bound contract:

  1. Bind the exact commit, approved paths, and one meaningful behavioral check.
  2. Reproduce the failure with deterministic CI evidence.
  3. Give GPT-5.6 Luna only that evidence and source from approved paths. Luna returns one strict unified diff.
  4. Validate the proposal in a clean job against the current head, protected files, evidence integrity, and the two-attempt, 15-minute budget.
  5. Let a separately credentialed controller apply an accepted patch.
  6. Start fresh CI on the new commit. Only that new exact head can receive ChangePlane / guard as PASS.

That separation is the product. The model can propose a repair, but it cannot push, merge, edit the evidence, publish the required Check, or decide PASS. GitHub remains the merge authority. Protected, stale, ambiguous, provider-failed, or exhausted work fails closed and names the next human action.

How I built it

I built ChangePlane on the infrastructure teams already trust: a repository-scoped GitHub App, GitHub Actions, one pure evaluator, a small controller and repair helpers, and GitHub Checks, comments, artifacts, and refs as the audit trail. There is no new code host, proprietary agent workspace, database, queue, or merge service.

The OpenAI adapter calls the Responses API with GPT-5.6 Luna. It uses high reasoning, disables storage, and requires structured output containing one raw diff. Terra and Sol are allowlisted behind the same contract. Every response is untrusted input. Refusals, timeouts, malformed output, extra paths, protected-path changes, or clean-apply failures are rejected before any write.

Users bring their own OpenAI key per repository. The browser sends it once; ChangePlane verifies it, encrypts it with GitHub's repository public key, and stores it only as the OPENAI_API_KEY Actions Secret. The plaintext is cleared. ChangePlane has no database in which to keep it.

Setup is one GitHub-native path: install the App, choose one repository, bind one real check, and merge one protected setup pull request. There is no ChangePlane CLI. After setup, developers stay in their coding agent and GitHub.

How I used Codex and GPT-5.6

I used Codex as an engineering and verification partner throughout Build Week. The most useful loop was concrete: I would state an invariant, such as “the proposal model never receives write authority,” then use Codex to trace every path that could violate it, write the failing test, implement the boundary, and verify it in the browser and live canary.

Codex accelerated the OpenAI adapters, GitHub App onboarding, installation-scoped repository discovery, BYOK flow, signed repair authorization, fail-closed test suite, browser journeys, production verification, and release evidence. I made the product, security, production-data, and authority decisions.

GPT-5.6 is part of the runtime, not a label in the interface. Luna proposes bounded patches and performs advisory review. It never judges its own work. A separate deterministic harness does that.

The hard parts

Generating a patch was the easy part. Making the result trustworthy required solving five harder problems.

The target keeps moving. A pull request can change while an agent is working. ChangePlane binds the contract, evidence, review, repair grant, preview, and result to one commit SHA. A stale head is never reused.

Model access cannot imply write access. The proposal job has no GitHub credential. The controller rechecks the signed grant and current head before minting a short-lived, repository-scoped App token for one bounded push.

A repair needs genuinely fresh evidence. A workflow-token push does not trigger the pull-request run needed for independent re-verification. ChangePlane applies through its dedicated App identity so the new commit starts new CI rather than inheriting an old green result.

The repair must not rewrite the exam. Tests, evidence configuration, dependency manifests, managed files, and repository-declared protected paths are immutable during autonomous repair. Touching one routes the change to a human.

Production proof must not expose production secrets. ChangePlane is used with RouteThai in production, but its repository, routes, customer context, and operating data are trade secrets. The public workspace is a recorded, sanitized replay with synthetic identifiers and values. It never contacts RouteThai production systems.

What I learned

Autonomy and authority are different things. A model can be more useful precisely because it has less authority: it can investigate and propose freely inside a bounded contract, while deterministic evidence and a separate controller own the decision.

I also learned that the commit SHA is the unit of trust. A PASS, preview, or receipt without a visible revision is decoration. A new commit must invalidate the old assurance.

Models will change. The safety boundary should not. Luna, Terra, and Sol can evolve behind the same contract: the model proposes, deterministic evidence decides, and a trusted controller applies.

Result

ChangePlane is now a hosted, repository-scoped GitHub App used with RouteThai in production. The public experience recreates that workflow safely: judges can replay the failure, Luna proposal, clean validation, trusted apply, new-head evidence, and PASS without credentials or access to private data.

The replay ends with zero human repair actions. That is the normal path I wanted: agents handle the fix, independent evidence earns the result, and GitHub decides what ships.

Built With

Share this project:

Updates