One flow. Five AI agents. A contributor goes from "where do I start?" to "I shipped it" — automatically. For any repo, any team, zero maintainer effort.
Contributor Compass — Our Story
What Inspired Us
Every developer has felt it. You find an open-source project you love, you want to contribute, and then you open the issue list and freeze. Which issue is safe to touch? Where does this function even live? Why is my pipeline red? You close the tab and move on.
That moment of friction kills thousands of contributions every week — in open source and inside enterprise teams where new engineers spend their first month just building a mental model of a codebase that a 10-minute tour would have given them.
We wanted to answer one question: what if every repository had an always-available senior engineer who could onboard anyone, review any MR, explain any conflict, and diagnose any failure — without ever burning out?
That's Contributor Compass.
What We Learned
Building this taught us things we didn't expect.
Prompt engineering at the agent level is a different discipline. Writing a prompt for a chatbot is forgiving — the user corrects you in the next message. Writing a prompt for an autonomous agent that posts a public comment and stops is not. Every instruction has to be precise, ordered, and defensively written. We learned to think in terms of steps with exit conditions, not conversations.
Idempotency is not optional for event-driven agents. Our biggest mistake early on was treating the flow like a one-shot script. It isn't — it fires on every trigger, potentially multiple times a day. Without a dedup gate and hard caps, a weekly schedule became 1,957 duplicate issues overnight. The math of runaway agent loops is brutal:
$$N_{\text{issues}} = T \times R \times C$$
where $T$ is triggers per day, $R$ is runs per trigger, and $C$ is issues created per run. With $T=10$, $R=3$, $C=10$ you get 300 issues in a single day. We learned this the hard way and built the dedup gate, the default_route → end safety, and the hard cap of 3 as a direct result.
A router that fails open is a liability. Our original router had default_route: issue_forge_agent — meaning any unmatched output silently ran the issue-creation branch. LLMs occasionally emit stray whitespace or an unexpected word. Every such misroute created issues and spawned a CI pipeline. Changing it to default_route: end was a one-line fix with an enormous impact.
The ambient environment is a constraint that sharpens your thinking. No shell means no git diff, no pytest, no grep. Everything has to be reasoned about statically from file contents and API responses. This forced us to write agents that explain their reasoning rather than just running commands — which, it turns out, produces much better output for contributors.
How We Built It
Contributor Compass is built entirely on the GitLab Duo Agent Platform using the custom flows YAML format.
The architecture is a router + five specialized agents:
Event
│
▼
mode_router ──▶ one word: DISCOVER · EXPLORE · IMPACT · RESOLVE · TRIAGE · SKIP
│
▼
Specialized agent reads files + GitLab APIs → posts exactly ONE result
Each agent is a prompt with a strict step-by-step structure, a bounded toolset, and hard output rules (one comment, one note, cite every path you read). The router is intentionally minimal — it outputs a single uppercase word and nothing else, which makes it fast and predictable.
The five agents and what makes each one interesting:
DISCOVER (Issue Forge) — the hardest to get right. It has to explore a repo it has never seen, identify safe contribution opportunities, match them to a skill profile, check for duplicates across open and closed issues, and create well-scoped issues with Mermaid call-graphs and roadmaps — all without a shell. The dedup gate is the most critical piece.
EXPLORE (Tour Guide) — traces the call chain from entry point to the target function and builds a mental model in prose + Mermaid. The constraint of max 6 stops forces it to prioritize, which produces better tours than an unconstrained walkthrough.
IMPACT (Guardian) — computes blast radius by reading every file that imports a changed symbol, up to depth 3. Risk scoring follows a simple rule: $\text{risk} = f(\text{affected files}, \text{untested symbols})$. LOW if $|\text{affected}| < 5$ and all tested, HIGH if $|\text{affected}| > 15$ or any entrypoint is touched.
RESOLVE (Peacemaker) — reads both sides of a conflict statically and explains in plain language what each branch was trying to do. Never claims to have run git. The tone constraint ("warm, reassuring") is load-bearing — a conflict comment that sounds like a compiler error helps nobody.
TRIAGE (Firefighter) — reads
.gitlab-ci.ymland the source files the failing job touches to reason about the most probable cause. No log access needed for the most common failures (wrong assertion, missing import, syntax error) because the cause is visible in the source.
The demo project (demo/) is a small Python package with a deliberate structure: cli.py → runner.py → parser.py → utils/validate.py. It gives every agent something real to reason about, and we crafted specific demo artifacts for each mode:
- A clean issue for EXPLORE to tour
- An MR with a new
normalize()function for IMPACT to review - A branch that conflicts on
sanitize()for RESOLVE to explain - A test that deliberately asserts the wrong value for TRIAGE to diagnose
Challenges We Faced
The 1,957-issue incident. The most dramatic challenge. A combination of default_route: issue_forge_agent, a soft "max 10" instruction, and a dedup check that only looked at open issues meant that every trigger — including MR events and pipeline events that should have been ignored — silently ran the issue-creation branch. We hit the GitLab pipeline rate limit, filled the project with duplicates, and had to diagnose the root cause under time pressure. The fix was three targeted changes: safe default route, hard cap of 3, and a dedup gate that checks open and closed issues and stops entirely if any contributor-compass issue already exists.
Prompt precision vs. prompt length. Every agent prompt is a balance between being specific enough to produce consistent output and short enough to leave room for the agent's reasoning. We rewrote the DISCOVER prompt four times before it reliably stopped at the dedup gate instead of creating issues anyway.
Static reasoning without execution. The IMPACT agent has to predict what a code change will break without running the code. The TRIAGE agent has to diagnose a CI failure without reading the job logs. This sounds limiting but it's actually a feature — it forces the agents to explain their reasoning in terms a human can verify, rather than just running a command and reporting the exit code.
The ambient environment constraint. No shell means no package managers, no git commands, no test runners. Early versions of our prompts included instructions like "run pytest to verify" — which the agent would faithfully attempt and fail. We had to audit every prompt for implicit assumptions about shell access and replace them with static reasoning equivalents.
Rate limits and pipeline costs. Every flow trigger starts a CI pipeline. With broad triggers enabled during testing, we hit GitLab's "too many pipelines in the last minute" limit repeatedly. This pushed us to design the minimal trigger set documented in docs/flow-setup.md and to make SKIP the default for anything ambiguous.
The Result
A single 500-line YAML file that turns any GitLab repository into a self-serve contribution platform. Open a repo, enable the flow, set seven triggers — and every contributor gets a personal onboarding experience, every MR gets a pre-screen, every conflict gets explained, and every red pipeline gets diagnosed. Without a single maintainer lifting a finger.
"One flow. Five modes. A contributor goes from 'I want to help' to 'I shipped it' — without ever pinging a maintainer."
Log in or sign up for Devpost to join the conversation.