SpoonCheck

GitLab Duo agents that keep every release easy to use for people with the fewest spoons left.

Comment

Inspiration

We all start the day with a limited number of "spoons" (Spoon Theory). People with ADHD, autism, chronic illness or a cognitive impairment often have very few, and every extra step, every hash on the screen, every confusing error message uses one up. One spoon less is nothing for some of us and everything for others. Most pipelines check that code works. None of ours checked what a release costs the person who has to use it.

What it does

SpoonCheck turns cognitive accessibility into a CI budget, after the code is written:

  • A meter runs the app's core task in a real browser, in English and German, and counts spoons per rule: steps, choices, reading load, machine codes on screen, negative wording, relative dates, motion, error messages, lost progress on reload, moved buttons and visual friction such as crowded or overlong lines. Every rule names its W3C COGA pattern; the weights are team-set heuristics, not a clinical measure.
  • A report issue always shows what the task costs today.
  • The spoon coach, a GitLab Duo agent, is started by assigning it that issue (or mentioning it). It lowers the most expensive rules in the demo UI, in both languages, and opens a merge request.
  • A second, read-only fidelity check agent confirms that no deadline, amount or duty was lost and posts a receipt bound to the exact commit SHA.
  • GitLab Duo Code Review must approve the current head.
  • The gate merges only with a matching receipt, the change inside the allowed paths, every language within budget and Duo's approval. Budgets ratchet down after each improvement. Nobody merges by hand.
  • After the merge, the pipeline deploys to Google Cloud Run (keyless, workload identity federation), measures the live site and reports the live cost on the merge request. A daily scheduled job measures both public demos again.

A good score never excuses a lost fact or a skipped confirmation: four functional guardrails (error feedback, input and task kept, progress kept after reload, consistent measurement) must hold in every view.

Live before and after. With the first meter, FolderHome's accident journey cost 36.14 / 36.84 spoons (en / de) before and 19.92 / 19.30 after one coach run; codes on screen fell from 14.5 to 0.5 spoons. The stricter v3 meter measures both live demos on the same day: 38.84 / 39.34 spoons before, 20.30 / 19.63 after, and only the After version keeps your progress when the page reloads. The two meter versions are never compared with each other.

The After demo now also offers an opt-in Calm view, a clear correction when the confirmation is mistyped, a reload that never repeats an action, and the same task through a small assistant client that uses the app's real HTTP API: request, preview, exact confirmation, result.

How we built it

  • GitLab Duo Agent Platform: a custom flow with two agent components, coach and fidelity, registered in the AI Catalog, triggered by mention or assignment; GitLab Duo Code Review on every merge request.
  • Trust boundary: the pipeline lives in a separate project (spooncheck-ci), so a merge request cannot change how it is judged. Meter, rules, budgets and gate load from the default branch. The app under test runs as an unprivileged user without CI variables. Agent changes may only touch the demo UI (fail closed).
  • Meter: Python + Playwright; reports per rule, language and view (Standard / Calm), compared with the default branch under the same meter, rule and browser identity.
  • Deploy: Cloud Build builds an immutable image as a dedicated builder account; Cloud Run serves the demo behind a small gateway (exact Origin on POST, body cap, one synthetic workspace per visitor). The CI deployer can only roll out the one service; no Google key is stored anywhere. /spooncheck/version.json shows the deployed commit, image digest and revision.
  • For assistants: the repo ships SKILL.md and llms.txt, so an assistant uses the real interfaces and leaves counting to the deterministic meter.
  • Lifecycle stages: plan (report issue), create (coach MR), verify (meter, tests, Duo review), secure (separate CI, default-branch tooling, unprivileged app, SHA-bound receipt), package (Cloud Build image), release (gate merge, Cloud Run revision), configure (rules, budgets and policy as code), monitor (live measurement after every deploy, daily scheduled check), govern (budget ratchet, required review, protected branches).

Challenges we ran into

  • Agents have no merge tool, and bots cannot trigger flows. So the gate decides by measurable rules, and when Duo asks for changes on a coach merge request, a person restarts the coach with one mention.
  • Our own measurement had to be trustworthy: agents optimise what you measure, so the measured app must not reach the measurement. Duo Code Review found real holes on the way (a job token reachable through .git, a CSP bypass that would have measured something users never see) and later a client that hid a half-finished execution behind a generic error. All were fixed with regression tests.
  • Shared runner minutes: our first gate polled for the review and burned most of them; it now waits at most four minutes, heavy jobs only run when their inputs change, and our CI jobs run on the project's own runner.

Accomplishments that we're proud of

  • The first merge without a person, and the first deploy from CI with a live check, both by the gate bot.
  • Duo Code Review caught an accessibility regression the coach introduced (a label screen readers would not announce), and the gate held the merge.
  • A full v3 coach run went from one explicit start to a live deployment: fidelity receipt, native Duo approval, 199 tests, budgets and gate, then a provenance-checked live measurement.
  • The same meter gives the same numbers in CI and against the live URL.

What we learned

Accessibility can be a release criterion like tests, if the rules are transparent and the numbers are reproducible. "Simpler" is only better if nothing a person needs gets lost; that is why the fidelity check and the guardrails exist. And a keyboard test in a browser is not a screen-reader test: screen-reader output is still untested.

What's next

SpoonCheck will grow. Spoons differ per person: profiles for ADHD, colour vision, blindness and low vision, and motor impairments; tests with real assistive tools and the people who use them; and more tasks an assistant can do for you, offered as tools over APIs, MCP or WebMCP, with cowork between a person and their assistant. Not only spoons, but forks too: from spoons to full assistance.

Evidence for judges

Built With

  • cloud-build
  • docker
  • gitlab
  • gitlab-ci
  • gitlab-duo
  • gitlab-duo-agent-platform
  • google-cloud-run
  • playwright
  • python
  • workload-identity-federation
Share this project:

Updates

Submission history