Code Health Auditor

A GitLab Duo Agent Platform flow that uses the Orbit knowledge graph to find dead code and test gaps, verifies every finding by trial removal, and posts a report as a merge request comment. It is read only with respect to your source. It recommends, it never deletes on a guess.

Live and published: https://gitlab.com/explore/ai-catalog/flows/1011855/

Inspiration

Two things. First, the AI Catalog is already crowded with blast radius agents that answer "what breaks if I change this?" That is the obvious read of a call graph, a risk lens. We pointed the same Orbit graph at the quieter verb, hygiene: what does nothing reference (dead code), and what does nothing test (coverage gaps).

Second, we watched the naive version of this idea fail. Off the shelf dead code tooling, and a neighbouring hackathon project that auto deletes code it is "80% confident" about, breaks on the thing that matters most: cross file and dynamic references. Delete a function the call graph thinks is unused and you ship a bug, because the graph cannot see self.method(), template calls, getattr, or a public API's external callers. So we built the careful version: graph to narrow, a mandatory cross check to verify, and a confidence model that would rather say 0 than lie.

What it does

• Finds dead code. Functions and methods with no inbound reference, after resolving cross file calls and ruling out first class value, dispatch, and template usage. • Finds test gaps. High fan in (high blast radius) definitions that no test reaches. • Verifies before recommending. It trial removes a finding in a sandbox and checks the package still imports and the tests still pass. Evidence, not 80% confidence. • Performs an action. It posts a Dead Code and Test Gap Report as a merge request comment, and files an issue when there is no MR. Every finding cites path:line, the Orbit inbound count, and the cross check evidence.

It is live. We published the flow to the GitLab AI Catalog and triggered a real run on a merge request, where it posted its report as a comment.

How we built it

• A custom Duo flow (deadcode-orbit.inline.yml): a deterministic git step, then an agent whose prompt encodes the funnel, the per language rules, and the post a comment action. The Orbit recipes are inlined because custom flows do not support Jinja includes. • It reaches Orbit through glab orbit, remote (hosted graph, JSON query_graph DSL) or local (DuckDB SQL), the same method either way. • The funnel: graph liveness (CALLS, EXTENDS, IMPORTS edges plus imported symbol name rescue), then reachability from entry points, then a textual cross check folded into liveness (it scans code and rendered templates, tagged by language), then confidence tiers. • A standalone reference implementation (auditor/deadcode_audit.py) and a trial removal verifier (auditor/verify_dead.py). The sample-app result is fully reproducible offline: orbit index sample-app, then run deadcode_audit.py. The multi repo table was measured by indexing each public repo locally.

The heart of it: precision, and a correction we made on ourselves

The naive "no inbound call" query flags 35,258 definitions as dead on Django. Pure noise. We first claimed our hardened query funnelled that down to "232 high confidence dead." Re measuring proved us wrong. The cross check rescued nearly all 232, because they were private methods dispatched via self._x(). So we rebuilt the tool around the honest rule: only a textually orphan function is confirmed dead, methods always go to a review tier, and a language whose call graph is under 10% resolved is flagged advisory, not confident.

Result on Django: 0 confirmed dead functions, and 73 methods routed for review. It refuses to cry wolf on a mature framework, which is exactly what makes the findings it confirms on the demo app trustworthy. Measure, do not assume, is baked into the product, not just the process.

The sample app row reflects the published flow (2 confirmed plus 1 review). Every other row is the reference auditor (deadcode_audit.py) run locally on that repo.

• Scale. Django, about 48,667 definitions and 181,709 edges, indexed in seconds, with queries answering sub second. • Multi language. Verified on Python, Go, and JavaScript. It also auto detected and sectioned C#, Ruby, and Bash with no per language code, deriving callable types, visibility, and the coverage gate for each. • Multi edge liveness, proven on real code. A definition is live if it is called, imported, or subclassed, not just called. On Django that subclass rule alone keeps 693 abstract base classes alive, like AbstractBaseModel, that a call graph only tool would wrongly flag as dead, and across the five Python repos it rescues about 953. Inheritance is a real reference, so the tool counts it. • Proof, not confidence. On the demo app the funnel runs 41 to 15 to 10 to 3. Of the 3 textual orphans, the published flow confirms send_fax and legacy_vat as dead and routes audit_stock to review, because it is a public function that could have external callers, so the live run reports 2 confirmed plus 1 review. The bare reference auditor (deadcode_audit.py) confirms all 3, the flow adds the public function rule on top. Removing the 2 confirmed dead functions breaks nothing. Removing a rescued candidate (cmd_add, reached through a dispatch table) throws NameError, proof that the cross check is load bearing.

Challenges we ran into

• Remote is not local. Hosted Orbit takes a JSON query_graph DSL, not the local DuckDB SQL. We reverse engineered the live surface and ported the funnel to it. • Only the default branch is indexed. A feature branch returns 0 nodes, so a graph only audit misses the dead code an MR introduces. We added an MR delta pass. • Do not trust the synthesis, measure. The 232 correction, and the discovery that Go init(), JS named function expressions, and vendored copies like six each need bespoke handling, all came from measuring on real repositories.

What we learned

The graph is a bulk filter, not an oracle. Its job is to cut tens of thousands of definitions down to a handful, so a precise low tech cross check and a trial removal can finish the job correctly. And honesty is a feature. A tool that says "0, and here is why" earns trust that a tool screaming 35,258 never will.

What is next

• Severity and autofix suggestions on the exact diff lines. • External caller analysis to disambiguate library public APIs. • Scheduled repo health trend runs, and broader verified language coverage.

Built with

GitLab Duo Agent Platform, Orbit (the codebase knowledge graph), glab, a custom AI Catalog flow (YAML), Python, DuckDB, and a GitLab CI Docker runner.

Built With

• duckdb
• gitlab
• gitlab-ci
• gitlab-duo
• gitlab-duo-agent-platform
• glab
• orbit
• python
• yaml