Lade

Inspiration

Pipelines go green and production still breaks. When it does, someone spends hours reading CI logs, scan reports, and cloud metrics to find out why. Most AI tools help write the code before the commit, and very few investigate what happens after it. We wanted an agent that investigates the way a careful engineer would: collect evidence, form a theory, try to disprove it, and show its proof.

What it does

Lade investigates a post-merge incident inside GitLab. A CI job runs five gated phases: catalog the evidence, extract findings, correlate across change, pipeline, and runtime, test competing explanations, then write the report. Every finding has to quote the exact log line it relies on, and the quote is checked against the stored tool output. The alternative-narrative phase tests rival explanations (platform problem, traffic spike, slow startup, an unrelated vulnerability) and records a verdict for each before anything is concluded. The result becomes a GitLab post-mortem issue, and a GitLab Duo flow reads it and drafts a remediation merge request for human review. It never merges. We demonstrate it on a staged incident: a dependency update silently removes a cache size limit, and staging runs out of memory.

How we built it

A GitLab CI pipeline runs the investigation, verifies the audit log, and publishes the issue. A GitLab Duo custom flow turns the issue’s remediation section into a draft merge request. A Python toolkit (lade_core) provides eight read-only evidence tools (Trivy reports, dependency diffs, CI logs, git diffs, Cloud Logging, memory metrics, heap traces, status pages), the evidence gate, the phase gates, and the report writer. [Claude models drive the Planner, Executor, and Analyst roles.] [Deployed on Google Cloud Run.] A hash-chained audit log records every tool call, and verify recomputes the whole chain.

Challenges we ran into

Verifiable claims. Quotes had to match exactly what the model saw, so we had to unify how outputs are serialized, stored, and hashed. Tests that lied. Our first tool tests used invented data and passed, while six of eight parsers failed on realistic fixtures. We rebuilt the tests on realistic evidence. Untrusted evidence. Logs can contain text that looks like instructions, so every prompt treats evidence strictly as data. Limited model quota, which pushed us to test the plumbing on small models first. [If true: We replaced an early prototype’s external base with new code written for this hackathon.]

Accomplishments that we're proud of

A fake or misquoted citation is rejected automatically. Change one character in the audit log and verification fails at that exact record. The agent argues against its own conclusion before writing it. [N tests passing.] [A full run from a CI job to a draft merge request on GitLab.]

What we learned

Gates and verification mattered more than prompts. Tamper-evident is an honest claim, while “tamper-proof” is not. And a tool is only as good as its tests against real data.

What's next for Lade

Run real Trivy and Syft scans on live images, add Kubernetes and service-mesh evidence, cover infrastructure-caused incidents to prove the hypothesis phase is unbiased, move to ATT&CK v18.1 mapping, and add merge-request approval policies for a hands-off mode.

Built with: GitLab Duo Agent Platform, GitLab CI/CD, Python, Claude, [Google Cloud Run], Trivy, SQLite.

Before you paste:

Fill every [bracket] from the real run, and delete any line that isn’t true. If you submit under Path A, the new-work declaration has to be true when you click submit. The bracketed line about replacing the early prototype is only for when that’s actually done. Say the incident is staged. It’s stated above, so keep it.

Built With

• gitlab
• gitlabduo
• python