OrbitSense

Intelligent Blast Radius & Impact Analysis

Comment

Inspiration

Every developer has experienced it: you merge a "small" change to a shared utility and three microservices go down in production. The blast radius of a code change is invisible until it's too late. We were inspired by the idea that GitLab Orbit's Knowledge Graph already knows every dependency relationship in your codebase — it just needed an agent smart enough to ask the right questions at the right time. OrbitSense was born from the belief that the best code review happens before the merge, with full dependency context surfaced automatically.

What it does

OrbitSense integrates directly into the GitLab Merge Request workflow as a CI/CD-powered AI agent. When a developer opens or updates an MR, OrbitSense automatically:

  1. Parses the MR diff to extract all changed functions, classes, and files.
  2. Traverses the Orbit Knowledge Graph with multi-hop queries (up to 3 hops deep) across CALLS, IMPORTS, DEPENDS_ON, and TESTS relationships to map every downstream dependent.
  3. Computes a weighted risk score across 5 dimensions: downstream dependents (30%), security sensitivity (25%), test coverage (20%), cross-service impact (15%), and change complexity (10%).
  4. Posts a rich Markdown report directly as a comment on the MR, including a risk level (LOW / MEDIUM / HIGH), impact metrics table, detailed findings, and prioritized recommendations.
  5. Auto-merges low-risk MRs when confidence is high, reducing review bottlenecks.

How we built it

Tech Stack: GitLab Duo Agent Platform, GitLab Orbit Knowledge Graph, Python 3.12, GitLab CI/CD, D3.js v7, GitLab REST API v4, Vanilla CSS with Glassmorphism.

The architecture consists of four composable agent skills defined in .gitlab/duo/:

  • parse_mr_diff — Extracts changed symbols from raw git diff output.
  • orbit_blast_radius — Issues multi-hop graph traversal queries to the Orbit API to discover all downstream callers, importers, and dependent services.
  • impact_analysis — Applies the weighted scoring model and classifies findings by severity.
  • report_generator — Formats the findings into a Markdown comment and calls the GitLab Notes API to post it directly on the MR.

The flow is orchestrated via config.yaml (the Duo Agent Flow Registry) and triggered automatically by CI_PIPELINE_SOURCE == "merge_request_event" in .gitlab-ci.yml. A separate deploy_group.py script enables group-wide deployment so the agent protects every repo in the organization.

Challenges we ran into

  • CI/CD token permissions: GitLab's CI_JOB_TOKEN cannot post MR notes — it is scoped only to the Package Registry and artifact endpoints. We had to engineer a PRIVATE-TOKEN authentication flow and handle graceful soft-exits when the token was misconfigured, so the pipeline would never block a developer's work.
  • YAML parsing strictness: GitLab's CI YAML parser is stricter than standard YAML libraries — emoji characters in script: blocks, unquoted if: conditions, and bash substring expansion (${VAR:0:4}) all caused silent parse failures. We validated every change locally with Python's yaml.safe_load() before committing.
  • Graph traversal depth vs. performance: Orbit queries with unlimited depth caused timeouts. We settled on a configurable ORBIT_MAX_DEPTH=3 default that captures the vast majority of blast radius impact without runaway query costs.
  • Making the agent group-wide: GitLab does not expose a simple API to inject a CI configuration into every project in a group. We built a deploy_group.py script that creates a shared CI project and documents the manual group-level CI config path setting required.

Accomplishments that we're proud of

  • Zero-setup MR integration: Once GITLAB_TOKEN is set as a single CI/CD variable, every new MR in the project automatically gets a full blast radius report — no per-repo configuration needed.
  • Graceful degradation: The agent never blocks a developer's pipeline. If the token is missing or the Orbit API is unreachable, it soft-exits with code 0 and logs the report to the job output.
  • Production-quality risk model: The 5-factor weighted scoring model handles real-world edge cases: security-critical paths auto-escalate to HIGH regardless of overall score, and the test coverage factor correctly handles projects with no test infrastructure.
  • Interactive dashboard: A standalone D3.js force-directed dependency graph that brings the Orbit Knowledge Graph to life visually, with animated risk meters, severity-coded findings, and glassmorphism dark-mode design.
  • End-to-end working pipeline: From git push → MR diff → Orbit graph query → risk score → MR comment in under 2 minutes.

What we learned

  • The Knowledge Graph is the killer feature: GitLab Orbit's relationship graph (CALLS, IMPORTS, DEPENDS_ON, TESTS) is the missing ingredient that makes automated impact analysis actually useful. Without it, you can only analyze the files changed — not the consequences of those changes.
  • CI token scoping matters enormously: The difference between CI_JOB_TOKEN and PRIVATE-TOKEN is not just authentication — it's a fundamentally different permission model. Understanding this boundary was key to making the agent actually post to the MR.
  • Agents need to fail gracefully: The best AI agents in a CI/CD context are ones that help when they can and stay out of the way when they can't. A hard-failing agent that blocks your pipeline is worse than no agent at all.
  • YAML is deceptively fragile: Special characters (emoji, bash expansion, unquoted conditions) that work in development YAML parsers can silently break GitLab's stricter pipeline parser. Always validate locally.

What's next for OrbitSense

  • Full GitLab Duo Agent Platform integration: Publish OrbitSense to the GitLab AI Catalog so any team can install it in one click, with reviewer-assignment and @orbitsense mention triggers powered natively by the Duo platform.
  • Historical risk trend analysis: Track blast radius scores over time per repository to identify which parts of the codebase are becoming increasingly risky and need architectural attention.
  • Smart merge gating: Automatically block merges that exceed a configurable risk threshold and require a senior reviewer sign-off, integrated with GitLab's approval rules.
  • Cross-project dependency mapping: Extend the Orbit traversal beyond a single project to discover blast radius across multiple repositories in a group, crucial for monorepo-to-microservice migrations.
  • Team-level dashboards: Engineering manager views showing aggregate risk scores, most-impacted services, and code health trends across the entire organization's merge activity.

Built With

  • css
  • d3.js-v7
  • gitlab-ci/cd
  • gitlab-duo-agent-platform
  • gitlab-orbit-knowledge-graph
  • gitlab-rest-api-v4
  • glassmorphism
  • python-3.12
  • vanilla
  • with
Share this project:

Updates