Examen

Inspiration

Concurrent customers expose defects that a single happy-path test can miss. An AI-written patch needs evidence from the transaction ledger. Examen applies that check to a small synthetic ticket marketplace.

What it does

Examen runs 3 model customers in independent browsers against 1 marketplace. A trusted observer checks 5 SQL invariants over committed rows. The checks cover overselling, payment consistency, confirmed bookings and customer ownership. Payments and refunds are synthetic.

A controlled seeded incident gives Duo a concrete failure to investigate. Duo proposes competing hypotheses and bounded experiments. It writes a focused repair and a regression test. Independent Duo review identifies a test weakness. Duo corrects it, and the maintainer adds further regression hardening.

The same assertion runs against the old and candidate images. All 5 old runs fail the intended capacity assertion. All 5 candidate runs pass. The seeded image reproduces the oversell in 20 of 20 controlled replays. The candidate records no violation in 20 repetitions and 10 timing variations. Each variation runs once.

How it is built

The marketplace and control service use separate Django applications and PostgreSQL 17 databases. Pinned Linux images and hash-locked dependencies define the local runtime. Playwright drives real Chromium over HTTP. The observer reads committed rows through a restricted role. Captured artifacts and image digests bind each recorded result to its declared inputs.

GitLab holds the incident, Duo work, repair commits, review notes and visible source CI. The public source pipeline passes after a marketplace job retry. The video includes actual local browser footage and public GitLab records.

Results

• 5 of 5 candidate regression runs pass against 5 intended old-image assertion failures.
• 20 candidate replay repetitions and 10 timing variations report no invariant violation.
• 5 fresh model namespaces each confirm 1 ticket and 1 synthetic payment of 1,500 pence.
• Those trials make 88 actual provider calls, including 9 invalid responses that trigger bounded retries.
• The control cost ledger records USD 0.071098 for those 5 trials.
• 2 fresh-volume installs pass all 5 local stages on 22 October 2026.
• Their 16 scripted and random negative-control trials report no invariant violation.

The https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/8727406/showcase/-/blob/main/docs/evidence.md links the exact records, source revisions and limits. The https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/8727406/showcase/-/blob/main/docs/judge-guide.md supplies installation and inspection steps.

Challenges and lessons

The trusted test must prove the intended old failure. A setup error or incomplete request overlap cannot count as that failure. Review identifies a calendar dependency and weak overlap accounting in the regression. The corrected test keeps the concurrency mechanism explicit.

Record integrity and release authority also need separate checks. Source CI, local replay and a local image digest do not establish protected release authority. The submission uses the available Developer access and declares the remaining release limits.

Limits

The runtime and evidence are local. The project uses an intentionally seeded defect and synthetic payments. The fresh-model trials validate repaired purchasing behavior. They do not replace the declared comparative discovery cohorts. The cost figure comes from the control ledger without independent invoice reconciliation.

Protected evaluation and release remain deferred. The local release rehearsal uses a synthetic approval ID. Authenticated human approval, policy-consumed release and rollback remain incomplete. The original M1 and C20 acceptance targets remain incomplete.

What is next

Extend the declared comparative benchmark with its unexecuted cohorts. Complete protected evaluation and the authenticated release lifecycle under a reviewed design. Apply the same committed-ledger checks to another transactional application.

What we learned

What's next for Examen

Built With

• chromium
• django
• docker-compose
• gitlab-duo-agent-platform
• playwright
• postgresql
• python