Tenure

GitHub's Agent Merge merges as soon as GitHub allows. Tenure decides what GitHub allows: per-agent merge rights granted by people, proved in a sandbox by Nemotron, revoked by evidence.

Comment

Inspiration

Coding agents now open more pull requests than people can review. Every vendor draws the same line: agents propose, people click merge. That keeps review as the bottleneck. GitHub's Agent Merge now merges a PR as soon as GitHub allows it, so the real question is what GitHub should allow, and who decided that. We wanted the answer to work the way access works anywhere it matters: granted by a named person, scoped, expiring, and withdrawn when the evidence turns bad.

What it does

Tenure is a GitHub App that decides which agent-written pull requests may merge without a person.

  1. It holds every agent PR by default and says why on a check run. Agents are identified by GitHub's numeric bot ID, never a name anyone can type.
  2. A maintainer grants autonomy with a PR comment, for example /tenure grant dependabot deps 30d @owner because …. A grant covers one repository, one agent and one kind of change (docs < tests < deps_patch < deps < code). It has an owner and always expires. A bot's comment is never treated as a command.
  3. Under a grant, the verification agent proves the PR. NVIDIA Nemotron 3 Super, running on Nebius Token Factory, plans a build-and-test recipe from the base branch only, and repairs the recipe when a step fails. From one sandbox checkpoint, the tests run at the base, with the PR, and after git revert. Exit codes and test counts decide; the model never does.
  4. Tenure merges exactly the commit it verified. It can merge without a review, but never without CI.
  5. Evidence takes the grant away. If a Tenure merge is reverted, or main goes red at its merge commit, the grant is withdrawn and nobody has to notice.
  6. Everything goes into a hash-chained audit log, shown on a public live page.

Some changes are never merged unattended, whatever has been granted: CI config, agent instruction files (AGENTS.md, CLAUDE.md, …), MCP configs, licences and Tenure's own policy.

It ran on real PRs (26 Oct 2026, hyperion, Dependabot as the agent):

  • #39–#42 changed CI, so they were never merged.
  • #43 was held, then granted, verified, and merged by Tenure.
  • #44 was verified (101 tests, three runs), but GitHub refused the merge because CI was still running.
  • #45 reverted #43, and Tenure withdrew the grant on its own.

How we built it

  • The GitHub App is in Python with Starlette. It verifies webhook signatures, handles each delivery once, writes check runs, pins merges to a sha, and checks that a granter has maintain or admin rights before acting.
  • The verification agent uses nvidia/nemotron-3-super-120b-a12b through Token Factory's OpenAI-compatible API. It runs at reasoning_effort=low, under a daily token cap that survives restarts; past the cap, the PR is simply held.
  • Sandboxes: Tenure supports Nebius Token Factory Sandboxes (ConTree, branching snapshots). Sandboxes weren't activated on our project before the deadline, so the live deployment runs the same interface on a private runner of our own, with no permissions and no secrets. One setting switches back to Nebius Sandboxes.
  • Evaluation: we replayed 34,891 real agent merges from the AIDev dataset, 212 of them later reverted, in time order:
    • Merge-when-green let every revert through.
    • Path rules alone caught 12%.
    • Tenure's policy merged 15.4% of PRs without a person, at a 0.02% leak rate.
  • Quality: 226 tests, and CI on Python 3.11 and 3.13.

Challenges we ran into

  • Empty answers from Nemotron. The first call returned nothing because reasoning used the whole 2,000-token budget. reasoning_effort=low fixed it: a correct plan in 2 s and 95 tokens.
  • Sandboxes returned 403 for every call. The whoami endpoint showed every permission false. With days left, we built an isolated runner behind the same interface rather than run PR code next to our keys.
  • Choosing the right proof suite. A Python-only change first got the Rust suite. Now the planner is told which top-level folders the PR touches; those names come from the base tree, so PR content never reaches the model.
  • Making CI bind Tenure too. We split the rules into two rulesets: Tenure's grant counts as a review, and CI is required for everyone.

Accomplishments that we're proud of

  • The whole loop ran on real pull requests, end to end: hold, grant, verify, merge, refusal by CI, and revocation on revert.
  • The model never holds merge authority. It plans the proof; the evidence decides.
  • A measured result on 35k real merges, rather than a claim.

What we learned

  • Reverts cluster, so revoking a grant on the first revert does most of the safety work.
  • Path rules alone fail, because most reverts land in ordinary code.
  • Agents behave differently: 28% of Claude Code's PRs and 24% of Copilot's touch never-auto paths, against 7% for Codex.
  • Reasoning models need an explicit effort setting when an agent parses their answer.

What's next for Tenure

  • Nebius Sandboxes on the live deployment, once activated.
  • Grants proposed automatically from an agent's track record (simulated in the replay, not shipped yet).
  • More repositories, and deeper monorepo awareness in the planner.

Built With

Share this project:

Updates

Submission history