Permit

• 

  !28, opened by GitLab's Developer flow, ends with the claims the agent stands behind. Captured 2026-10-11.

• 

  Permit's answer to each claim of !28, with the evidence of that exact commit: 4,123 tests, 0 failures.

• 

  A Permit flow session (!21): a summary by the flow's one tool-less model, then the engine's own lines, verbatim.

• 

  The dashboard: each agent's level per change category, its ceiling, and progress toward the next level.

• 

  Issue #18, after the hands-off merge and deployment: Permit reports back on the issue the work came from.

• 

  Staged: a deliberately careless demo agent claimed only_touches docs/** but changed README.md. Blocked (!20).

The problem

An AI coding agent finishes its work with a summary: "tests pass", "no new dependencies", "only touches the docs". A reviewer either believes the summary or re-does the checking. And how much autonomy the agent gets is set by hand, all at once or rule by rule, by how brave the team feels, not by anything the agent has shown.

Both problems have the same answer. The claims in that summary are checkable against evidence GitLab already holds for the commit. If they are checked every time and the results are kept, the record says, per agent and per kind of change, how much checking by a person is still needed.

What Permit does

1. Agents state claims. AGENTS.md tells every agent to end its merge request with a ## Claims section: tests_pass, tests_added, lint_clean, no_new_dependencies, no_migrations, no_api_change, only_touches: <paths>, no_new_secrets, no_new_vulnerabilities and others.
2. Permit checks each claim against the evidence of the exact head commit: the JUnit report and coverage of that commit's pipeline, the lint and image build jobs, GitLab's secret detection, SAST and dependency scanning reports, and the diff, each compared with the merge base where that matters. No language model takes part. The answers are verified, contradicted, stale (the evidence is for an older commit) or unverifiable (no evidence either way, which counts neither for nor against the agent).
3. A ledger. Verifications (with the verdict on every claim), approvals, merges and deployments are records in a hash-chained file on its own branch, one commit per record. Only records pushed by the identities the policy trusts count.
4. Earned levels. Three clean, merged merge requests in a row in a change category move an agent one level up: assisted (a person approves every step; for now the only step is the merge), supervised (a person approves the outcome), hands_off (nobody). One contradicted claim sends it back to assisted. The policy sets a ceiling per category: an agent can reach hands_off for backend code and tests, but never for the pipeline configuration, migrations, authentication code or Permit itself.
5. A GitLab Duo flow and a merge executor. The flow verifies a merge request and, where the level asks for it, stops at an approval gate that shows the engine's own lines. Every step of the flow that can write is a fixed command; the one model in it has no tools and only writes a summary. A flow cannot merge on GitLab, so a scheduled job on main verifies again and merges what the ledger allows.
6. After the merge: the image is built, deployed, checked from outside (the running instance says which commit it is), recorded in the ledger, and watched every hour. A release's notes are generated from the ledger.

Hands-off, end to end

The loop without a person has run end to end on a real issue (#18, 2026-10-11, all times UTC, from GitLab): 09:19 issue created and assigned to GitLab's Developer flow, which is the last thing a person did; 09:24 the agent opened !28; 09:30 its pipeline passed, and the agent completed its claims and marked the merge request ready; 10:26 the hourly merge executor verified the claims itself (the agent had earned hands_off for tests), recorded the result and merged; the push pipeline built, deployed and verified the deployment from outside; 10:34 Permit posted the summary on the issue. Seventy-five minutes, fifty-five of them waiting for the hourly run. On GitLab the owner's account (gyb07190) appears as the author of !28 and as the one who merged it and posted Permit's notes: a flow acts for the person who triggered it, and the executor, the deployment job and the monitor use the owner's token, limited to this project. The same loop stops and waits for a person as soon as a change touches a category where the agent has not earned that level.

What actually happened in this project

Everything below is in the project's history and on the dashboard. As of 2026-10-17:

• GitLab's Developer flow worked on 13 issues, 12 mirrored from upstream linkding and one flaky test found in this project's own pipeline (#18), and opened 13 merge requests with 98 claims. None was contradicted. It went from assisted to supervised to hands_off for backend code and tests, and is at supervised for documentation.
• !15, !22 and !28 were merged with no approval, because the agent had earned that level for what they touched. !9, by the same agent, still needed an approval of every step: it touched the code that creates the initial administrator, which the policy treats as security-sensitive.
• !15 is worth reading. Its first pipeline had one failing test. The agent did not claim tests_pass; it wrote that the failure looked unrelated but that it could not establish the cause. A person re-ran the job, it passed, and the agent updated its description to say exactly that.
• The demonstration of a false claim is staged, and says so: in the README and PERMIT.md, with a "staged demo" label on its merge requests and issues, and in a note on !20. Since the real agent never made one, we added a deliberately careless agent that pastes the same claims into every merge request. We gave it three issues asking for changes under docs/, where the pasted claims are true; all three were merged and it was promoted. The fourth changed README.md while claiming only_touches: docs/**: contradicted, blocked, demoted (!20).
• The monitor has had one real incident. On 2026-10-14 the free instance stopped serving: the hourly check got HTTP 502 for its whole four-minute look and opened issue #19 with what it saw and when; the next check got no answer at all; the check two hours later recorded on the same issue that production was healthy again and serving the commit the ledger names.
• Four faults of Permit itself showed up while it was in use or under review, and each is written up with what it affected and how that was re-checked: a comparison baseline that could be a pipeline without tests; a ledger record lost when two writers committed within half a second; a way for one merge request to make the whole ledger unreadable with a Unicode line separator; and a way to earn a level from merges into a branch other than main. After the first fix, all thirteen agent merge requests merged until then (ten by the Developer flow, three by the staged agent) were verified again with the fixed engine; every verdict held.

Lifecycle stages

What was added on top of linkding (Path B)

linkding was imported on 2026-10-10 at upstream commit 27b7303. Everything after that is this project's work, all of it after 2026-10-05: Permit (permit/: an engine that uses only the Python standard library, its pytest tests and documentation, and .permit/policy.json), the GitLab pipeline (converted from upstream's GitHub Actions and extended), AGENTS.md, PERMIT.md, the demo deployment (docker/demo/), and the changes to linkding's code and documentation (bookmarks/, docs/, .env.sample), all of which were made by agents in merge requests that went through the loop. Outside the repository: the Permit flow and the staged QuickFix agent in the AI Catalog, a pipeline schedule, two protected CI/CD variables, the dashboard on GitLab Pages, and the deployment.

How it was built

• The engine is deterministic Python with no dependencies, so it runs in a flow container, in CI and on a laptop alike.
• The flow is a GitLab Duo Agent Platform custom flow: deterministic steps for verification, the approval record and the hand-off; a conditional router that reads the engine's one-word decision; a human gate; one model without tools for the summary.
• A flow's token cannot merge or approve on GitLab. The merge executor is a scheduled CI job with a fine-grained token limited to this one project.
• Permit's own code was written with Claude Code, an AI coding agent, and most of Permit's own merge requests were verified by Permit like any other agent's; they appear on the dashboard under the account gyb07190.

What was hard

• Finding out what a flow may and may not do. Much of it is undocumented: after a human gate a flow continues in a new container on the default branch; a router compares the whole output of a command including its first line. One documented rule shaped the design: a flow starts only on an event caused by a person, so at hands_off the merge executor makes the verification itself.
• Concurrency on a ledger that is a file in a Git branch. GitLab's check of the last commit happens before the commit is created, so two writers could both be told "success" while one record replaced the other. The fix is a marker file per extended commit, a read-back, and an audit of the branch history.
• Not overstating. The documentation says for each part whether it had run on GitLab or only in tests when that part was written, with the date, and which records are staged. The README also records the monitor's first real incident (2026-10-14).

Related work

groundtruth (https://github.com/MSal2020/groundtruth, MIT) also checks an agent's claims deterministically, by running the tests itself and reading the diff, as a CLI, a Claude Code hook and a GitHub Action. Its README mentions no record across runs and no level per agent. Permit differs in using the CI evidence of the exact commit instead of running anything again, in the ledger and the earned levels, and in being built into GitLab as a flow with a gate and a merge executor. This is what one search found; there may be others.

Limits

Permit verifies claims against available evidence. It does not show that code is correct, and it says nothing about what nobody claimed. The ledger branch cannot be protected with the role we have in the hackathon workspace, and main, where the policy names the trusted writers, takes pushes from every Developer of the workspace; someone with that access can stop the executor and, by changing the policy, make their own records count. The deployment is a free instance without persistent data. The monitor's token may open an issue but not close it, so after the recovery note the issue stays open until a person closes it.

What is next

A second gate for assisted (ready, then merge) so that the first two levels differ in more than name; evidence from more sources (performance, accessibility, licence scans); a hosted ledger for projects that do not want one in a branch.

Built With

• django
• docker
• gitlab
• gitlab-ci
• gitlab-duo
• gitlab-pages
• python
• render