About the project

HoldWatch is an automation that runs after your code merges and answers one question honestly: what actually happened, and what do we know for sure?

The problem, in plain terms

When something goes wrong after a release — a payment is blocked, a payout is returned, a build behaves oddly — the people who need answers are handed a pile of signals: logs, status codes, event payloads, pipeline results. Most post-deploy tooling responds by inventing a plausible cause. It patterns. It guesses.

That is worse than useless. A wrong guess in production costs someone real money and real trust. Once a tool confidently blames the wrong thing, people stop reading it — and then they miss the next real failure.

HoldWatch is built on the opposite rule: it would rather say "unknown" than guess.

What HoldWatch does

It runs as a GitLab CI pipeline (verify → secure → govern → publish), and each stage has one job:

  • verify — checks a claim against the actual evidence and reports how well it holds up (recall, precision, false positives). If the evidence doesn't support the claim, it says so.
  • secure — confirms a payload is genuinely what it claims to be. If the signature doesn't check out, it stops. It fails closed rather than trusting by default.
  • govern — assembles the final report and, critically, marks what it could not prove. It does not paper over gaps.

The honesty, shown rather than claimed

The demo video walks through a real run. In the govern stage, the report openly lists two claims it could not verify — the synthetic ground truth and the live-money paths — instead of claiming full coverage. For a tool whose entire thesis is "never invent a cause," admitting what it can't prove is the feature, not a limitation. It is what makes the rest of its output believable.

On the dashboard, that same rule is visible to a human operator: when HoldWatch cannot determine a cause, it displays CAUSE UNKNOWN and lists the resolution steps it can verify — it does not fabricate an explanation to fill the blank.

How it runs

Every claim HoldWatch makes is reproducible from the pipeline history rather than taken on faith. The pipeline is live and public, and each stage emits measured JSON results — recall, precision, fail-closed holds, verified-versus-unverified claims — not prose.

What we learned

That "I don't know" is a valid, useful engineering output — and that a tool willing to produce it is more trustworthy than one that never admits uncertainty.

What's next for HoldWatch

Extend verification across more lifecycle stages, but only ones that can hold the same evidence-backed, fail-closed standard. Breadth that undermines honesty isn't progress.

Honest scope

HoldWatch currently runs three verified stages, not the full nine-stage lifecycle. That is a deliberate boundary: adding stages whose output could not be independently proven would contradict the project's own rule. The value is in the honesty of what it reports, not the length of the list.

Built With

  • gitlab-ci
  • gitlab-duo-agent-platform
  • json
  • python
  • render
Share this project:

Updates

Submission history