Inspiration
I've merged evaluation code into UK AISI's inspect_evals, DeepEval and TruLens. I keep seeing the same failure: teams have great unit tests and no tests for how their model behaves. A "make the bot friendlier" prompt tweak passes CI, ships, and then the bot starts inventing features or reciting its system prompt to anyone who asks. Code generation is solved. Knowing whether the AI feature still works after the code is written is not.
What it does
EvalGate turns LLM evaluations into a GitLab-native release gate that Duo agents operate across all nine DevSecOps stages:
- Plan: the EvalGate Planner flow reads an issue and writes acceptance eval cases (including adversarial ones) into evals/suite.yml, then opens an MR.
- Create: the EvalGate Reviewer custom agent and the EvalGate MCP server let Duo run evals and explain failing cases from chat.
- Verify: every MR runs the suite against the main-branch baseline; a scorecard note says exactly which cases newly fail and why.
- Secure: SAST, Secret Detection and Dependency Scanning, plus an LLM red-team category: prompt injection and a planted secret canary that must never appear in output.
- Fix (Supervised autonomy): when the gate blocks, a pipeline event triggers the EvalGate Fixer flow: triage agent -> fixer agent (smallest fix plus a new regression eval case) -> human approval gate -> publisher agent commits and posts the new scorecard.
- Package / Release: container image to the registry; the GitLab Release carries an evidence pack and scorecard.
- Configure / Deploy: Cloud Run staging and production; production waits for a passed canary plus a manual click.
- Monitor: canary evals against the live URL after every deploy and hourly; drift opens an incident issue that feeds the Fixer.
- Govern: CODEOWNERS protects policy and suite, a hash-chained tamper-evident ledger records every gate decision, and the evidence pack maps to NIST AI RMF and EU AI Act Art. 15.
How we built it
GitLab Duo Agent Platform (two custom flows with a HumanInputComponent approval gate, one custom agent, an MCP server), GitLab CI/CD (JUnit and Metrics reports, Security templates, Container Registry, Releases, Environments, Schedules, Jobs Artifacts API), Python 3.12, FastAPI, PyYAML, any OpenAI-compatible LLM (mock mode; Gemini), and Google Cloud Run (asia-south1).
Challenges we ran into
- The agent needs a main-branch baseline to compare against, not just a pass/fail number.
- Keeping the fixer honest: an agent can "fix" evals by deleting them. The prompt and CODEOWNERS forbid touching the suite and policy except to add cases.
- Making the demo reproducible without API keys.
Accomplishments that we're proud of
All 9 stages covered by real jobs or flows; a supervised agent loop that leaves the codebase with more tests than it started with; 8 automated tests including flow-schema validation and ledger tamper detection.
What we learned
Agents are most trustworthy when the eval suite is their contract. The same eval definitions drive pre-merge gating, agent self-checks and production monitoring.
What's next
LLM-as-judge scorers, cost and latency budgets per MR, and publishing the flows to the AI Catalog.
Path: A: Start Fresh | Autonomy: Supervised | Stages: 9/9 (plan, create, verify, package, secure, release, configure, monitor, govern)
Log in or sign up for Devpost to join the conversation.