Test Gap Sentinel

Inspiration

Every engineering team faces the same question with no good answer: "We have limited time to write tests — what should we test first?" Coverage tools report which lines ran, not which untested code is actually dangerous. The truly risky code is the code that many other things depend on and that changes often. That isn't a coverage percentage — it's a graph question, joining the call graph with merge-request history. Code search can't answer it. The GitLab API can't answer it in one hop. GitLab Orbit can, because it indexes code structure and SDLC metadata as one queryable graph. That unlock inspired Test Gap Sentinel.

What it does

Test Gap Sentinel finds the riskiest untested code in a project. It identifies definitions (functions, classes, methods) that are heavily relied upon by other code, frequently changed across recent merged merge requests, and not referenced by any test file. It scores each one on a transparent, explainable formula — dependents × log2(1 + churn) — keeps only the untested ones, and posts a ranked table as a comment on the merge request. Instead of a coverage number that hides risk, a developer gets: "these three functions are relied on by 20+ others, change constantly, and have zero tests — write these first."

How we built it

The agent is an Agent Skill (a SKILL.md following the Agent Skills specification) paired with a custom flow on the GitLab Duo Agent Platform. The skill encodes all Orbit interaction: it discovers the graph schema at runtime, gathers candidate definitions in scope, then runs three queries per candidate — a neighbors query for incoming dependents, an aggregation query for merge-request churn, and a reference check that gates on whether any test file references the definition. The flow is flow-registry-v1 YAML running in the ambient environment; it loads the skill via the workspace_agent_skills context and posts exactly one comment. Orbit is reached through its REST API (POST /api/v4/orbit/query, GET /api/v4/orbit/schema, GET /api/v4/orbit/status) and the equivalent MCP tools. The agent is published to the GitLab AI Catalog.

Challenges we ran into

Orbit's ontology is in beta, so edge and node names can change. Hardcoding them would make the agent brittle, so the skill discovers the schema at runtime and adapts its queries to whatever the live schema returns. A subtle correctness trap was direction: dependents come from incoming references, not outgoing ones, and getting that backwards quietly inverts the whole analysis. We also had to define "tested" carefully — a definition isn't covered just because its file has sibling tests; we require an actual reference edge from a test file to the definition itself.

Accomplishments that we're proud of

Test Gap Sentinel answers a question that is genuinely impossible with code search or the GitLab API alone — it only works because Orbit joins the call graph and merge-request history in one graph. The risk score is fully traceable: every row in the report maps to a real query result, with no fabrication. And because the skill is schema-adaptive, it keeps working as Orbit evolves.

What we learned

The real power of a knowledge graph isn't any single query — it's the join across domains that no existing tool spans. We also learned that a transparent, explainable score earns more trust from developers than a black-box number, and that delivering the answer inside the merge request, where the decision is made, matters as much as the analysis itself.

What's next for Test Gap Sentinel

An auto-generated test stub for the top entry, handed off to a code-writing flow so the gap closes in the same session. Scheduled runs that open issues for newly risky code as the graph evolves, not just on merge requests. Suggested-author routing using Orbit's User-to-MergeRequest edges, so the report also names who last understood the affected code. And a combined risk surface with our companion agent, Security Reachability — code that is both reachable and untested is the highest-priority work of all.

Built With

• agent-skills
• ai-catalog
• anthropic-claude
• clickhouse
• gitlab
• gitlab-duo-agent-platform
• gitlab-knowledge-graph
• gitlab-orbit
• grpc
• markdown
• mcp
• rest-api
• yaml