TwinTrace

Compare the failure before blaming the change: GitLab Duo selects authorized checks; deterministic BASE/HEAD replay shows whether the observed CI failure is new, pre-existing, worse or inconclusive.

Comment

Inspiration

A red CI log shows a symptom on HEAD. It does not, by itself, show whether the current merge request introduced it. We wanted a small, auditable investigation that helps developers avoid debugging the wrong revision instead of asking AI to fix the latest change on assumption alone.

What it does

TwinTrace combines GitLab Duo Agent Platform check selection with deterministic revision comparison. A bounded catalog tells the agent which checks are permitted. A strict gate verifies the selection, revision references and one-time authorization. The replay engine runs the same trusted check twice on BASE and HEAD, records outcomes and environment/failure fingerprints, and classifies the specific observed failure as PRE_EXISTING, NEW_REGRESSION, WORSENED or INCONCLUSIVE.

Duo communicates the supported next investigation. It cannot replace measurements with an explanation or override the deterministic classification. A human decides what to do next. PRE_EXISTING does not mean that the entire merge request is safe.

How we built it

An independent MIT-licensed Python CLI, immutable Git fixtures, JSON and Markdown evidence, GitLab CI/CD, a GitLab Duo custom flow and a local browser interface. The runtime is pinned to Python 3.12.14. The Python engine needs no third-party runtime packages. Model output selects known check identifiers, never arbitrary shell commands. Invalid, duplicated, stale or unauthorized selections stop safely.

The Balance V3 interface presents the same deterministic results, source provenance and expandable technical evidence. Its animation is recorded-evidence playback, not an indicator of a fresh Duo execution.

What genuinely ran

Actual GitLab Duo session 9636121, runner job 17254794251, coordinated an authorized replay with two selected byte-budget checks. For case_1, both revisions measured 105 bytes against a 100-byte budget, supporting PRE_EXISTING for that check. For case_2, BASE measured 95 bytes and HEAD 105, supporting NEW_REGRESSION. The agent explained the different next actions while retaining human review.

These were controlled synthetic fixtures investigated using genuine, previously recorded failing CI evidence. The self-contained public evidence package contains the observed selection, sanitized runner trace, transcript, screenshots and limitations. Some original tool output was truncated; we do not claim to possess complete original live replay JSON or reconstruct missing hashes.

Separately, public release pipeline 2971530396 passed 121 Python tests and reproduced seven comparison cases. This is fresh deterministic release validation, not another Duo run. The no-key local demo is reproducible without access to the private development project.

Challenges we ran into

Real integration attempts exposed runtime/authorization handoff problems and an agent tool/output contract mismatch. We retained strict validation, pinned the runtime, made the authorization boundary explicit and used bounded historical CI artifacts. An infrastructure-green runner was never treated as proof that a valid comparison had occurred.

Accomplishments

A genuine supervised Duo investigation selected authorized checks, launched controlled replay and explained two different evidence-supported classifications. The public MIT source includes automation configuration, runnable no-key demonstrations, visible release CI and independently inspectable supporting evidence. No private Observer modules are required.

What we learned

GitLab already compares historical test reports. TwinTrace's narrower contribution is a controlled paired replay and comparability contract: retry observes HEAD again, while git bisect searches history for a first bad revision. TwinTrace evaluates a specific BASE/HEAD attribution question under explicit boundaries. We have not measured comparative productivity gains, and two repeated observations do not eliminate all flakiness or establish universal causal certainty.

What's next

More complete downloadable live evidence and additional explicitly authorized real-world check integrations. Arbitrary untrusted-repository execution would require stronger isolation; this trusted-fixture prototype does not claim to provide an OS sandbox. No automatic repair, merge or production deployment is implemented.

Judge access and attribution

A Melvenx project, developed in the Observer Lab context. Observer Core is not part of this release.

Complete MIT source and no-key instructions: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43317536/twintrace-release

Self-contained genuine Duo evidence: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43317536/twintrace-release/-/blob/main/public_duo_evidence/README.md

Successful public non-Duo validation: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43317536/twintrace-release/-/pipelines/2971530396

Open START_HERE.html for recorded-evidence playback. For a fresh local comparison, use Python 3.12.14 and Git, run python -B release_checks.py then python -B launch_judge.py --serve, and open the printed loopback URL. These local runs do not invoke Duo or require API keys.

Built With

Share this project:

Updates

Submission history