Autonomy Thermostat

AI agents earn autonomy. A Pre-mortem agent imagines how each merge request could fail, then trust and risk decide: ask a human, or merge itself. Good deploys raise trust; failures lower it.

Comment

Inspiration

The hackathon gave us three levels of autonomy to pick from: assisted, supervised and hands-off. It felt odd to choose one and stick with it forever. People don’t work that way. A new teammate asks before touching anything, and after a few good months you let them ship small things alone. If something breaks, you go back to checking their work.

So i asked myeslf: what if the agent’s freedom wasn’t a setting, but something it earns and loses based on evidence?

What it does

Autonomy Thermostat decides, for every merge request, how much freedom the AI gets.

When an MR opens, a Pre-mortem agent reads the diff and pretends it’s six months later and this change caused an outage. It writes the incident report, lists three ways it could have failed, suggests a test or monitor for each, and gives the change a risk score from 0 to 100.

A Thermostat agent then combines that risk with the service’s trust score, which is a plain JSON file in the repo. The result is one of three levels:

  • Assisted: a human has to approve before merge.
  • Supervised: one human approval is requested.
  • Hands-off: the agent merges it itself once the pipeline passes. (Furture plan)

After every pipeline on main, a second flow, the Trust Updater, records what happened. A passing pipeline adds trust, and a failure takes it away. It opens an MR with the new score so the whole history is reviewable in git.

How we built it

Everything runs on the GitLab Duo Agent Platform, starting from a tiny Python demo API that the agents look after.

  • Flow 1 (Pre-mortem → Thermostat): two agents in one custom flow, triggered by merge request events. The first reads the MR and scores risk. The second applies the policy and posts the verdict as an MR comment.
  • Flow 2 (Trust Updater): triggered by pipeline events on main. It updates trust/state.json and opens an MR with the change.
  • CI/CD: a GitLab pipeline runs unit tests and SAST security scanning on every change.
  • Trust logic: a small dependency-free Python script (thermostat.py) holds the rule: effective freedom = trust minus 60% of the risk score, with cutoffs at 30 and 60.
  • Lifecycle stages touched: create, verify, secure, release, monitor and govern.

Challenges we ran into

Plenty, and most of them were small and sneaky.

  • The classic import error. Our first test run failed because pytest couldn’t find our own scripts folder. A two-line pytest.ini fixed it.
  • A misplaced include:. I indented the SAST template under a job instead of at the top level, which would have quietly broken the pipeline.
  • Committed junk. Adding .gitignore doesn’t untrack files already in git, and own pre-mortem agent caught tracked pycache files in an MR.
  • Tool names differ by place. Duo Chat told it had no run_command tool, while the flow docs list it. i stopped depending on it and had the agent compute the decision from the policy directly, so the demo couldn’t hinge on one tool.
  • Infinite loops. The Trust Updater’s own MR merges into main, which runs a pipeline, which would trigger the Trust Updater again. We added a guard in the prompt and [skip ci] in the commit message.
  • Triggers are picky. MRs opened through push options and by service accounts didn’t always fire flows, so i learned which events do and don’t, and used reviewer assignment to trigger runs.
  • Time. i built this in a short window, so i cut anything that wasn’t part of the core loop.

Accomplishments that we're proud of

  • A real feedback loop: trust changes because of what actually happened in the pipeline, not because I typed a number.
  • The pre-mortem reads diffs closely. It noticed that we’d deleted a CI job and caught committed bytecode files that i hadn’t noticed ourselves.
  • Every decision and every trust change is visible in git, so the autonomy history can be audited like code.
  • The whole thing is small enough to understand in one sitting.

What we learned

  • Autonomy works better as a dial than as a switch.
  • Writing the failure story first makes risk concrete, and a vague “this feels risky” becomes a test someone can write.
  • Agents are best given boring, checkable policy. Simple arithmetic in a prompt or a script beats asking a model to “use judgment” about permissions.
  • Flows are powerful, but guardrails like loop protection and narrow toolsets matter from the start.

What's next for Autonomy Thermostat

  • Make the pre-mortem read SAST findings and raise the risk score for real vulnerabilities.
  • Track trust per service and per area of code, instead of one score for everything.
  • Add time-based decay so trust fades when a service hasn’t shipped cleanly in a while.
  • Deploy to Google Cloud so the trust updater can watch real deployments and rollbacks.
  • Add approve and reject buttons in chat for assisted changes, so a human can approve from their phone.
  • Alerts: verdicts and pipeline results are sent to a Discord channel through GitLab’s built-in Discord integration.

Built With

Share this project:

Updates

Submission history