Revert Radar

• 

• 

• 

• 

• 

• 

Inspiration

The worst thing I've ever shipped was a revert. We had a bad change in prod, so reverting it felt like the safe move. The diff was tiny and looked self contained, it just deleted a little module, and CI was green. About twenty minutes after it merged, a few other parts of the app started falling over with import errors. Turned out a bunch of code in completely different files had been calling into that "self contained" module the whole time, and none of it showed up in the revert diff.

That one stuck with me. A revert is really a delete, and a delete never asks who's still depending on you. The reverts that look the cleanest are often the ones that bite. The only thing I needed to know before merging was "does anything still call this," and grep can't answer that. The call graph can, and GitLab's Orbit graph already has it. So I built a thing that asks it for me, on every revert, before I hit merge.

What it does

Revert Radar leaves one comment on a revert MR: a green "SAFE to merge," or a red "UNSAFE, live callers exist."

It looks at the functions your revert would delete and walks the Orbit CALLS graph backwards, across file and package boundaries, to find everything that still imports and calls them.

If nothing does, you get the green banner, and it lists the functions it checked so you can see it didn't just whiff. If something does, you get the red one with a count and a table of the exact functions and files that will break, plus the raw Orbit query trace so you can confirm it really hit the graph.

On my demo repo, reverting payment.processor looks harmless in the diff but comes back UNSAFE with 9 live callers (checkout, returns, the receipt mailer, and the test suite). Reverting a little formatter helper comes back SAFE. Same tool, both answers.

How we built it

It's a small Python script. No third party libraries, just the standard library and the Orbit API. It runs as a GitLab CI job on merge request events, and it's also published as a Duo Agent flow.

The whole thing rests on one query, run once per changed module, and figuring out the right query took a while. In Orbit, a call from one module into another isn't a Definition to Definition CALLS edge like I assumed. It goes through an ImportedSymbol node. So the query that finds callers is really "find every Definition that CALLS the ImportedSymbol whose import_path is this module." I run that per module, map each calling function back to its file, throw out the ones inside the revert itself, and what's left is the caller set. That set is the answer.

A few things I cared about. The comment is built only from graph data, never from the diff or commit messages, and I scrub symbol and file names before they go into the table, so nothing in the code under review can inject text into the bot's comment on a public MR. It's honest about whether it actually checked: every comment prints effective_mode: orbit, and if Orbit is down it does not guess and does not post a green SAFE, it posts UNVERIFIED and fails the job, because a merge gate that says "safe" during an outage is worse than no gate. The graph read retries a few times with backoff and jitter so a random 5xx doesn't tank the run. And it works in CI over HTTP and locally through the glab CLI, which is how I iterated.

Challenges we ran into

The ImportedSymbol thing. My first few queries returned almost nothing because I was looking for direct Definition to Definition edges. Once I read the schema and saw that cross module calls route through ImportedSymbol, it clicked, and that is also the whole reason a text search misses these callers in the first place.

The scary bug. My first version said "safe if zero callers found." Then I realized that if every Orbit query failed, it would also find zero callers, and happily post a green SAFE on the one check that is supposed to catch a breaking revert. I changed it so any failed query means UNVERIFIED and a red exit, and I wrote tests that fail if an outage ever reads as safe.

Flaky graph calls in CI. Early runs hit the odd transient error. Retrying with backoff and jitter sorted that out without spamming the API.

Making it readable. The verdict has to be obvious in a second, but I still wanted the caller table and the query trace there for anyone who would rather verify than trust.

Accomplishments that we're proud of

It works on a real MR and it's right both ways: UNSAFE with 9 callers on a "clean" revert, SAFE on an actual leaf.

It doesn't lie. SAFE means "I asked Orbit and found nobody," not "I didn't find anything," and there are tests holding that line.

No dependencies. It's stdlib plus the Orbit API, so it drops into any GitLab pipeline with nothing to install.

It's live. A published flow posts these on real merge requests. It isn't a mockup.

What we learned

A graph answers stuff grep and an LLM just can't: who calls this, in other files, even through an alias or a re-export. Once you can ask that, a lot of "use your judgment" review steps turn into a lookup.

The ImportedSymbol pattern is the key to cross module reachability in Orbit, and it's reusable. The same backwards walk ended up powering my test picker and my choke point tool too.

For anything that blocks a merge, the failure case matters most. Building the "I couldn't check" path first is what made me trust the rest of it.

What's next for Revert Radar

Going deeper than direct callers, so it also flags the things two hops away and the tests that reach the change. Ranking callers by how many call sites they have and whether they sit on a hot path, so the worst ones are at the top. Other languages, since the backwards walk doesn't care which one you're in, the only language specific part is the import resolution Orbit already does. And pulling it together with the three sibling tools I built on the same query layer (test selection, choke points, cycle cutting) into one "what does the graph think about this change" comment on every MR.

Built With

• ai-catalog
• gitlab
• gitlab-ci-cd
• gitlab-duo-agent-platform
• gitlab-knowledge-graph
• glab
• md
• mypy
• orbit
• pytest
• python
• rest-api
• ruff