PromiseCheck

• 

  Pipeline passes after the agent’s fix: privacy, tests, security scans.

• 

  Duo agents trace the privacy violation and list every field leaving the service.

• 

  Estimated pipeline footprint (~0.7 g CO2e), shown with assumptions.

Inspiration

Most privacy leaks are accidents, not attacks: someone adds one analytics field or SDK and the product collects more than its policy promises. Code review rarely catches it, because nobody compares the diff to the policy.

What it does

PromiseCheck turns a privacy promise into an automated release gate, built for teams that handle personal data. PRIVACY.yml lists what the app may collect. On every pipeline, PromiseCheck verifies it two ways: a static AST trace of the code, and a dynamic run with a synthetic user that inspects what the app actually emits. If a promise is broken, the pipeline fails and saves evidence (field, file, line).

A GitLab Duo Agent Platform flow with three cooperating agents (trace, fix, report) then locates the leak, makes the smallest fix, commits it, and posts the evidence on the merge request. The pipeline re-verifies independently, runs SAST and secret detection, publishes a staging report, and waits for a human to approve the release. The pipeline also records an estimated carbon figure (assumptions shown); it is informational only, not a measured SCI score, and does not block releases.

How we built it

Python and PyYAML, GitLab CI/CD with the SAST and Secret Detection templates, and GitLab Duo Agent Platform custom flows (Flow Registry v1) kept in .gitlab/duo/flows/. The remediate flow runs in the project through a service account and is triggered by mentioning it on a merge request. A second flow, which turns an issue into a merge request, is included in the repo but is not demonstrated in the video.

Challenges we ran into

Making the privacy gate run independently of unit tests so evidence is always produced, keeping security templates compatible with our pipeline setup, and the Duo agent's sandbox not having the checker's dependency, so we let the pipeline be the independent verifier. We also kept every claim scoped to what we can actually test.

Accomplishments that we're proud of

A complete, visible failure-and-fix loop in real pipeline history: a planted precise-location leak fails the gate, a Duo flow session traces it and commits the fix, and the pipeline passes on independent re-verification.

What we learned

Agents are most useful when a deterministic check defines "done". The checker decides pass or fail; the agents do the work to get there.

What's next for PromiseCheck

More event types and SDK signatures, real network-egress checks in test environments, measured energy data for the carbon estimate, and generating a draft PRIVACY.yml from an existing privacy policy.

Built With

• gitlab
• gitlab-ci/cd
• gitlab-duo-agent-paltform
• gitlab-secret-detection
• python
• pyyaml