Privacy Gate

• 

  23 test merge requests on Saathi, reviewed blind: the review agent flagged all 15 leaks and blocked none of the 8 safe changes.

• 

  All nine DevSecOps stages, each covered by a GitLab Duo agent, a CI/CD job, or both.

• 

  The loop: an issue becomes a reviewed merge request, the pipeline checks every promise, a person presses Merge, production is audited.

• 

  A blind test case: the gate's own jobs went red 17 s after opening, and the reviewer explained the broken promise at 61 s.

• 

  Someone else's app: the agents fixed hat.sh's upstream bug #116, from issue to an approved merge request in under two minutes.

• 

  Each layer catches what the others miss: static 10/15, runtime 9/15, pipeline 12/15, with the AI review 15/15. 0 of 8 safe changes blocked.

The problem

Most apps make privacy promises in plain words: "your files never leave your browser", "we don't track you", "only you decide what your family sees". Nothing in a normal pipeline checks them. Tests pass while a change quietly loads a font from a third-party CDN, posts an error report with the user's data, or turns on sharing by default. As agents write more of the code, fewer people read every diff with those promises in mind.

Two real examples:

• hat.sh encrypts files in the browser; its promise is that files, passwords and keys never leave it. It has 2,289 stars on GitHub, no release since June 2022, and its domain now redirects to a parking page. Issue #116, "password saved in browser", has been open since December 30, 2025.
• Saathi is my medicine companion for elderly parents: two phones share an end-to-end encrypted schedule, and its promise is that medicine names never leave the elder's phone unless she chooses.

What it does

A project writes its promises in one file, privacy-gate.json: the promises in plain language, the only hosts it may contact, test values ("canaries") that must never leave the browser, and a short user journey. Everything else reads that file.

• The Privacy Gate, a reusable CI/CD component (templates/): static rules on the source (trackers, unpromised hosts, cookies, plus project plugins), and a runtime audit that drives the app's real journey in headless Chromium and checks every request, header, WebSocket frame and service-worker fetch for a canary or a host the project never promised. It runs on every merge request, before deploy, against production after every deploy, and every 6 hours.
• Three GitLab Duo Agent Platform flows, public in the AI Catalog:
  • Issue to production: triage → implementer → privacy reviewer ⇄ fixer → reporter. A person opens an issue; agents plan it against the promises, write the change and its tests, review it promise by promise, and report when it is safe to merge.
  • Privacy review: reviews every merge request, including ones written by people.
  • Incident responder: investigates any failed pipeline and opens an incident with the log lines and the likely cause.
• Guardrails on the agents themselves: a gate-integrity job fails any merge request in which an agent's commit touches the gate, the policy, the rules, the pipeline, the flows, the scorecard or an existing test. Agents may change the app; only a person may change what judges it.

How I built it

GitLab CI/CD components, GitLab Duo Agent Platform custom flows (YAML, five agents in the main flow), Node.js and Playwright for the gate, Firestore as Saathi's relay, GitLab Pages for production, and Google Cloud (Firebase Hosting) for a second production copy.

The pipeline on main has 12 jobs: unit tests (53), the gate's static and runtime jobs, SAST and secret detection, a checksummed package in the package registry, GitLab Pages deploy, a GitLab Release, the Google Cloud deploy, a config-drift check that production serves exactly this build, and live privacy audits of both production URLs.

Google Cloud, keyless. The deploy-google-cloud job exchanges its GitLab OIDC token for a 10-minute Google token through Workload Identity Federation. The trust is pinned to this project's protected main branch (a run from another branch was refused), the service account can only manage Hosting and has no keys, and the Google Cloud project has no billing account.

Path B: before and after

• Saathi before (github.com/chxmq/saathi, MIT, created Oct 11): no CI, no tests, a static GitHub Pages site.
• After, on GitLab: a 12-job pipeline on every commit to main, three agent flows, 53 unit tests, a release and a package per build, and privacy audits of production on two hosts every 6 hours. Everything in the repository except app/ was built during the submission period.
• hat.sh before: CodeQL, Node.js CI and Snyk on GitHub, and still a password-autofill bug open for months. After, in the GitLab copy: the gate flagged it in the first pipeline, agents produced a reviewed fix, and a person merged it.

Results

hat.sh #116, fixed by the agents. On issue #1 the triage agent quoted hat.sh's own promise ("Passwords and private keys never leave the browser and are never stored by it") and posted a plan; the implementer changed 9 fields in 5 files, including 2 the rule did not scan; the reviewer approved it promise by promise. About two minutes from issue to an approved merge request; a person merged it later. Before: the gate's static job failed with 5 findings. After: green.

A blind leak scorecard. 23 merge requests against Saathi, written the way a well-meaning contributor would: 15 quietly break a promise, 8 don't. They were opened from neutrally named branches, in shuffled order, with the answer key held out of the repository until every review was in.

Each layer catches what the others miss. Static rules see error paths and fallbacks the journey never runs. The runtime audit sees what the code assembles at run time, like a host built from config parts. Some leaks send nothing unusual at all, like sharing medicine names by default; only the reviewer catches those. The full table, with every merge request linked, is in benchmark/RESULTS.md. An earlier run, whose branch names gave the answers away, is documented there and superseded.

It refuses, too. Asked to add Google Analytics, triage answered in 53 seconds with four reasons it would break the app's promises (an unpromised host, a tracker, cookies, and dose events leaving the phone), offered three alternatives that keep them, and handed the decision to a person. No merge request was created.

It watched our own gate. The live audit once failed in production. The incident responder found the cause: our gate was building a malformed URL (#5). A person fixed it (309c3e37); since then every main and scheduled pipeline has passed.

Why Supervised

Agents plan, build, review, fix, deploy and watch; a person approves the outcome by pressing Merge. That click is required: GitLab rejects the merge quick action from AI workflows ("Quick actions merge cannot be used with AI workflows"). The opt-in AUTOPILOT_TOKEN job lets a Maintainer remove even that click; it is available but has not been exercised in this project.

Limits, honestly

• The runtime audit sees only what the journey runs; error paths and fallbacks rely on the static rules and the reviewer.
• The review agent is a model and also runs the static rules, so it is not an independent check. The scorecard shows where it was right and wrong.
• It checks the web client. A backend that receives data legitimately and then misuses it is out of scope.
• I wrote the 23 scorecard changes myself.
• hat.sh's GitLab Pages are members-only on this provisioned project, so its live audit is off.

What's next

Publish the component to the CI/CD Catalog, add journeys for React and Next.js apps, and turn the scorecard into a public benchmark other privacy tools can run against.

Built With

• chromium
• firebase-hosting
• firestore
• gitlab
• gitlab-ci
• gitlab-duo
• gitlab-duo-agent-platform
• gitlab-pages
• google-cloud
• javascript
• node.js
• playwright
• semgrep
• workload-identity-federation