CompatGuard

Inspiration

Most merge request reviews focus on the files that changed. The dangerous breaks often live outside the diff: API clients, GraphQL queries, package users, CI templates, docs, deployment scripts, and tests that still expect the old contract.

GitLab Orbit gives AI the missing context graph for this problem. Instead of asking an agent to summarize a patch, CompatGuard asks a sharper review question before merge: who still depends on the public contract this MR is changing?

What it does

CompatGuard is an Orbit-powered merge request compatibility gate. It detects public contract changes, maps downstream consumers through the Knowledge Graph, validates old expectations against the source branch, and writes the result back into GitLab.

It focuses on contract surfaces that regularly break real teams:

• REST and API response fields
• GraphQL schemas
• exported shared types
• environment variables and config keys
• CI templates
• package exports
• CLI output contracts
• shared configuration files

When CompatGuard finds a risky or breaking change, it creates GitLab-native artifacts instead of a chat summary:

• MR compatibility report
• contract-risk::high label
• owner follow-up work items
• structured CompatGuard memory notes for future reviews
• generated compatibility-plan branch
• report JSON/Markdown artifacts
• inspectable agent trace

How we built it

CompatGuard is built as an Orbit-first GitLab Duo Agent Platform project with a seven-agent workflow:

• Contract Detector
• Orbit Consumer Mapper
• History Memory Agent
• Runtime Replay Agent
• Compatibility Judge
• Test Planner
• MR Gatekeeper

The live AI Catalog agent and flow are published and enabled in the provisioned GitLab Showcase project. The custom agent is configured with Orbit graph tools, merge request tools, pipeline/job tools, repository search tools, and GitLab write tools. The flow is attached to a native GitLab Flow Trigger for mention, assign, assign reviewer, pipeline events, and merge request ready events.

We also added a reusable repository-memory skill. It makes CompatGuard history-aware without pretending to train a model: the agent reads GitLab-native issues, work items, notes, labels, prior MRs, pipelines, and previous CompatGuard reports as evidence, then writes structured memory notes back into GitLab for later reviews.

The repository includes a deterministic GitLab CI proof path. In a merge request pipeline, CompatGuard compares the target and source branches, detects changed contract manifests, loads Orbit-style graph evidence, runs compatibility replay, and posts the same artifacts back to the MR. This makes the demo reproducible for judges while the live agent and flow remain available in the AI Catalog surface.

Challenges we ran into

The first challenge was avoiding a static AI demo. We did not want a chatbot that only explains a diff. CompatGuard had to behave like a GitLab workflow: read MR context, use graph context, make a compatibility decision, and create artifacts reviewers can act on.

The second challenge was keeping the project honest about Orbit usage. The live AI Catalog agent is configured with Orbit graph schema, graph status, list/invoke commands, and query graph tools. The proof MR also uses recorded Orbit-style graph evidence so the run is deterministic and reviewable even if a judge opens it outside the same indexing window.

The third challenge was making the gate language-agnostic. CompatGuard supports a contract manifest and adapter boundary instead of relying on one framework parser, so the same workflow can cover APIs, schemas, package exports, environment variables, CI contracts, and CLI output.

Accomplishments that we're proud of

CompatGuard has a working proof MR that judges can inspect end to end. The pipeline posts an MR note, applies a risk label, creates owner work items, uploads report artifacts, and publishes a generated compatibility-plan branch.

We also made GitLab-native memory part of the product. CompatGuard writes structured memory notes into issues and work items so future reviews can find prior compatibility breaks through GitLab records and Orbit-indexed history instead of hidden local state.

The narrow scope is the strongest part: CompatGuard is not a generic MR risk bot. It gates public contract compatibility, which makes the output easier to trust, easier to test, and easier for other developers to replicate.

What we learned

The strongest developer agents are workflow-native, not chat-native. In GitLab, that means acting inside merge requests, pipelines, labels, branches, work items, and review notes.

We also learned that broad impact analysis is less useful than a specific merge-blocking workflow. Contract compatibility gives the agent a clear reason to intervene, concrete evidence to collect, and clear artifacts to produce.

What's next for CompatGuard

Next, we want to expand adapters for OpenAPI specs, database migrations, package exports, and shared GitLab CI components. We also want to run CompatGuard across more public GitLab projects and use live Orbit query results directly in every proof path where the target namespace has Orbit indexing available.

Built With

• ai-catalog
• gitlab
• gitlab-ci/cd
• gitlab-duo-agent-platform
• gitlab-graphql-api
• gitlab-orbit
• gitlab-rest-api
• mcp
• mermaid
• python