Atari

Inspiration

Teams work on trust. Small, low-risk work goes to people who are still proving themselves, bigger work goes to people who have earned it, and trust drops when someone ships a mistake. I wanted autonomy that moves on its own, based on what kind of change it is and who is making it.

What it does

Atari sorts every change into docs or feature and gives each category its own tier. Docs are hands-off up until merge. Features are supervised, with a human approval gate.

Three agents (A, B and C) each have a trust score per category, kept in trust.json. When an issue is created, Atari reads it, decides the category, and gives it to an agent by score. Docs go to the lowest-scoring agent, because that is where agents earn trust. Features go to the highest-scoring agent, and only if that score is 50 or more. If nobody qualifies, a human gets it.

If a merged change breaks main, Atari finds the merge request and the agent behind it. It opens a revert merge request and a merge request that lowers that agent's score by 5. Humans can use it too. A tag in a commit message ([merge]) hands their branch to the system, which opens the merge request and marks it as a human change, so it never counts against an agent.

No flow merges anything. Every change, including reverts and trust-score changes, is a merge request that a person reads first.

How we built it

We built it entirely on GitLab Duo Agent Platform custom flows, in three flows:

• Triage, Worker and Gate: started by a new work item. Triage picks the category and the agent. Three worker steps do the change and open a merge request. Features go to a human approval step.
• Watcher, Reverter and Ledger: started by a failed pipeline on main. It reports the defect, opens a revert, and lowers the trust score.
• Intake: started by a passing pipeline. It opens a merge request when a human tags a commit.

The guardrails sit outside the AI. A CI job fails any docs: merge request that touches a non-doc file, and unit tests run on every merge request. The trust ledger is a plain file in the repo, so every score change is a reviewed commit. Our demo project is a photo editor written in Python.

Challenges we ran into

• Custom flows don't support structured output, so we couldn't have an agent return a neat form. We had every step end with one exact word, like C_docs, and had the routers read that word.
• Some issues look like docs but need a code change. A prompt rule alone isn't a lock, so we added a CI check on the real list of changed files.
• Closing an issue at merge is wrong when the change might still be reverted, so we left closing to a human.
• The intake agent misread the branch name as main, because our prompt had the wrong wording. A lot of refining prompts in order to get the desired result.

Accomplishments that we're proud of

• Autonomy that changes by itself. Work goes to different agents depending on the category and their track record.
• A full safety loop that works end to end: a bad merge, the defect report, the revert merge request, and the lowered score.
• Humans stay in charge. Nothing merges without a person reading it, and every score change has an audit trail.
• Humans and agents share one system, and a failure after a human merge never lowers an agent's score.
• We touched five lifecycle stages: plan, create, verify, monitor and govern.

What we learned

• The hard part of agent systems is the handoffs, not the agents. Most of our debugging was about what one step passes to the next.
• Put the hard rules in CI and not in a prompt. A prompt asks the agent to behave, and a CI check makes it.
• Simple designs hold up better. Our first plan had three layers of analysis. Routing work by trust score replaced two of them.
• Make failures explain themselves. When every "skip" has its own answer, debugging takes minutes and not hours.

What's next for Atari

• More categories, such as configuration and security-sensitive code, each with its own tier.
• Closing issues automatically once main is green after the merge.
• Adding the Secure stage, with a scan whose findings affect the trust score.
• Trust that spreads, so a bad change in one category can slightly lower an agent's score in others.

Built With

• duo
• gitlab
• python