-
-
HTTP 200, but the DB audit log catches the cross-tenant write — exit 1 blocks the merge.
-
The fix proven: 13/13 invariants hold at API and database — exit 0.
-
Anti-cheat guards reject editing a test or weakening an invariant — exit 1 each.
-
All green: lint, 6 invariants, 3 Duo flows, 40 tests — 6/6 bugs caught, zero false positives.
The problem
Cross-tenant data exposure — Broken Object Level Authorization — is #1 in the
OWASP API Security Top 10 and the single most damaging bug a B2B SaaS can ship.
It's invisible to pattern-based scanners because they don't know your tenancy
model. And AI coding assistants now produce "looks-right" code that quietly drops
a tenant_id filter faster than any human can review it.
What it does
TenantFence turns the intent of every change into executable tenant-isolation invariants, proves them against a live app, repairs failures, and keeps proving them in production.
- Intent → invariants. A GitLab Duo flow (
tf-intent) reads the MR diff, the linked issue, and the discovered tenancy model, and writes executable invariant specs intoinvariants/. - Deterministic proof. A Python harness (
tenantfence verify) attacks a live app and checks both the HTTP response AND the database side effects. A cross-tenant write that returnsHTTP 200is still caught, because a Postgres audit trigger records every write. The verdict is an exit code, never an LLM opinion. - Repair & re-prove. On failure,
tf-remediatecommits the minimal fix (routing the query through the tenancy helper) and the pipeline re-proves it. - Production probes + self-hardening. The same invariants run in production
as safe, canary-only probes. A failing probe auto-rolls-back, opens an
incident, and
tf-incidentopens an MR that permanently adds the failing condition to the pre-merge gate — the gate hardens itself after every incident.
Core rule: the LLM proposes, deterministic code decides. Agents derive specs, attack parameters, and patches; a deterministic harness decides pass/fail.
How we used the GitLab Duo Agent Platform
tf-intent(custom flow) — derives and commits invariant specs from change intent, posts a summary note.tf-remediate(custom flow, Pipeline-events trigger) — commits the minimal tenant-scoping fix.tf-incident(custom flow) — root-causes incidents and opens self-hardening MRs.- AGENTS.md — repo-wide rules every flow obeys, enforced by CI, not trust.
- GitLab CI/CD pipeline:
lint → test → secure → package → review → tenantfence → deploy → probe → promote → rollback, plus a scheduled monitor. - GitLab security report artifact surfaces findings in the MR widget.
- Feature Flags, a protected
productionenvironment with required approval, and review environments withon_stop. - Two deterministic governance guards:
invariant-guard(invariants may only be strengthened) andprotected-paths-guard(an agent can't edit the check to make it pass).
How we built it
FastAPI + SQLAlchemy + PostgreSQL for a realistic multi-tenant app ("Ledgerly"),
a Postgres audit trigger for database-level side-effect verification, a
schema-validated invariant format, six generic attack executors derived from the
tenancy model (not hard-coded endpoint names), and three GitLab Duo custom flows
governed by deterministic CI guards. Google Cloud deploy (Cloud Run, Cloud SQL,
Artifact Registry, Workload Identity Federation) is fully expressed as Terraform
in /infra.
Challenges we ran into
Phase 0 revealed that non-human users cannot activate a GitLab Duo trigger, so the planned "CI posts a mention to trigger the incident flow" was impossible. We pivoted the deterministic→agent bridge to a Pipeline-events = Failed trigger on the monitor/rollback pipeline, with the flow classifying the payload itself.
What we learned
The decisive design choice is keeping every decision deterministic. Letting an LLM propose freely is powerful — but only because a verifier it cannot edit has the final say. Verifying database side effects (not just HTTP responses) is what catches the leaks real scanners miss.
Accomplishments
A generic verifier that catches 6/6 planted isolation bugs — including a
silent cross-tenant write that returns HTTP 200 — while the correct app and two
negative controls pass cleanly (no false positives), all proven by an automated
scenario test matrix. The same invariants run in production as safe, canary-only
probes.
Built With
- api-security
- artifact-registry
- cloud-run
- cloud-sql
- fastapi
- gitlab
- gitlab-ci
- gitlab-duo
- gitlab-feature-flags
- google-cloud
- multi-tenancy
- openapi
- owasp
- postgresql
- pytest
- python
- sqlalchemy
- terraform
- workload-identity-federation
Log in or sign up for Devpost to join the conversation.