Inspiration

Every developer has felt it — that sinking feeling when a pipeline goes red at 2 AM and you have no idea why. Or worse, pushing code that accidentally exposes a secret key, only to find out hours later. Modern DevOps teams are drowning in GitLab notifications, CI/CD noise, and security alerts that demand constant human attention.

We asked: what if your repository could audit itself?

That question became PULSE — a system that doesn't wait for a human to notice something went wrong. It watches every push, every pipeline, every merge request, and responds in seconds with a structured, actionable report posted directly back to GitLab. No dashboards to check, no tools to context-switch into. Just intelligence, inline, where you're already working.


What it does

PULSE is a 7-agent autonomous SDLC intelligence system that attaches to any GitLab repository via webhook and activates on every push, pipeline event, or merge request — with zero human intervention.

The moment an event fires, PULSE:

  1. Routes the event type (push / pipeline / MR) via the Watcher agent
  2. Fetches the relevant diff and pipeline logs via the GitLab REST API (Extractor)
  3. Scans the code for leaked secrets and security vulnerabilities using Vertex AI Gemini 2.5 Flash (Security Sentinel)
  4. Diagnoses the root cause of any pipeline failure using the same model (Deploy Doctor)
  5. Computes a MoSCoW priority grade and Change Failure Rate score (DORA Calculator)
  6. Formats everything into a clean Neobrutalist Markdown report (Format Forger)
  7. Posts the report as a commit comment or MR note directly on GitLab (Emissary)

Beyond the main pipeline, PULSE LIVE (/pulse-live) lets developers ask plain-English questions like "Why did my pipeline fail?" and get actionable answers in under 2 seconds, powered by Groq's llama-3.1-8b-instant.

The DORA Calculator computes a Change Failure Rate impact score using a straightforward formula:

$$\text{CFR Score} = \frac{\text{Failed Pipelines}}{\text{Total Pipelines}} \times 100$$

Combined with a MoSCoW priority grade, this gives teams an at-a-glance measure of release health without any manual tracking.


How we built it

PULSE is built on Python + FastAPI, with three AI backends working in concert:

  • GCP Vertex AI (gemini-2.5-flash) — powers the Security Sentinel and Deploy Doctor agents, chosen for its strong instruction-following and code comprehension at low latency
  • Groq (llama-3.1-8b-instant) — powers PULSE LIVE for sub-2-second natural language query responses
  • GitLab REST API v4 — used by the Extractor and Emissary agents to pull diffs/logs and post reports back

Each agent is defined as a standalone YAML spec under agents/, and the full 7-agent orchestration is described in flows/pulse_flow.yml using GitLab Duo Agent + Flow primitives. The FastAPI backend wires them together into a single /webhook endpoint.

Authentication uses GCP Application Default Credentials (ADC) — no JSON key files, no secrets committed to the repo. All sensitive values live in .env, which is gitignored.


Challenges we ran into

Multi-model orchestration was the biggest surface area for failure. Vertex AI and Groq have different client interfaces, different error modes, and different latency profiles. Making each agent fail gracefully — so a Vertex AI hiccup doesn't crash the whole pipeline — required careful isolation at every step.

Content truncation was a subtle but real problem. GitLab diffs and pipeline logs can be enormous. Feeding raw logs into a model context blows past limits instantly. We capped all fetched content at 2,000 characters, which required tuning to preserve the most diagnostically useful portions without losing the signal.

Webhook event diversity was underestimated early on. GitLab's push, pipeline, and merge_request payloads are structured very differently. The Watcher agent went through several iterations before it could reliably extract project_id, commit_sha, and pipeline_id across all three event types without breaking.

GCP ADC setup on unfamiliar machines has friction. We wrote test-keys.py specifically so contributors can validate all three integrations (GitLab, Groq, Vertex AI) with a single script before touching the server.


Accomplishments that we're proud of

  • A fully autonomous end-to-end pipeline — from GitLab webhook to formatted report posted back as a commit comment, with no human in the loop
  • Clean agent isolation: every agent can fail independently without taking down the system
  • Dual-model architecture that uses the right model for the right job — Gemini 2.5 Flash for deep code analysis, LLaMA 3.1 8B via Groq for real-time NL queries
  • A 6/6 integration test suite (test_local.py) that validates the full pipeline locally before any deployment
  • Zero hardcoded secrets anywhere in the codebase — ADC-first, .env-first design throughout

What we learned

Building a multi-agent system taught us that the orchestration layer is where complexity lives, not the individual agents. Each agent in isolation is straightforward; making them compose reliably under real network and API conditions is the hard part.

We also learned that prompt design for diagnostic tasks is fundamentally different from generative tasks. The Security Sentinel and Deploy Doctor prompts went through many iterations — vague prompts produce vague reports. Structured, constraint-heavy prompts that specify exactly what format the output should take produced dramatically more useful GitLab comments.

Finally, ADC over service account JSON keys is the right call for GCP integrations. It's more secure, easier to rotate, and removes an entire class of accidental secret exposure from the threat model.


What's next for PULSE

  • Claude integration — Anthropic's Claude is a natural fit for the Security Sentinel role, given its strong performance on code auditing and structured output. We want to add Claude as an optional backend and make the model choice configurable per agent.
  • Historical trend analysis — storing DORA metrics over time so PULSE can detect degradation patterns across sprints, not just per-commit
  • Slack / Teams notifications — routing PULSE reports to team channels in addition to GitLab comments
  • Custom rule configuration — letting teams define their own security patterns and MoSCoW weighting via a pulse.yml config file in their repo
  • Dashboard UI — a lightweight frontend to visualize CFR trends, security findings, and pipeline health over time

Built With

  • custom-integration-test-suite-(test-local.py
  • fastapi
  • gcp
  • gcp-application-default-credentials-(adc)-libraries-vertexai
  • gitlab-duo-agent-yaml
  • gitlab-duo-flow-yaml
  • gitlab-duo-flow-yaml-cloud-&-auth-google-cloud-platform-(gcp)
  • gitlab-rest-api
  • google-vertex-ai-(gemini-2.5-flash)
  • groq
  • python
  • python-dotenv-dev-&-testing-ngrok-(local-webhook-tunneling)
  • requests
  • uvicorn
  • uvicorn-ai-/-ml-google-vertex-ai-(gemini-2.5-flash)
Share this project:

Updates