Cust

• 

  MR !40 renames RefundWorker with no shim. Cust's check fails and the merge is blocked.

• 

  cust:mr-check replays the retained jobs and reports failed_declared_checks, with the three missing refund rows.

• 

  Duo session 10018104 runs the Cust flow on !40 and writes the compatibility shim.

• 

  Spec MR !42 records the human approval that lets CI prepare the ordered release.

• 

  Plan issue #6 tracks compatibility, producer switch and cleanup with exact commits.

• 

  Step 1, compatibility (!52), merged after its gate and staging deploy passed.

• 

  Mastodon near-miss replay. Without the worker, the old reply job fails and cleanup is blocked. Lab data.

• 

  Cleanup gate job 17459517125 returns ready_for_review and lists what it can't prove.

• 

  Render free-tier demo production. One worker and one producer on the producer-switch commit. Not customer production.

• 

  With the maintainer's fix, both jobs deliver and the declared checks pass. Lab data.

Inspiration

Sidekiq jobs outlive deploys. A job can sit in Redis scheduled, retrying or queued for hours. Rename a worker or change its arguments, and every test still passes because tests only create new jobs. The old ones fail after the deploy. Mastodon hit this in 2018 when it deleted ActivityPub::ReplyDistributionWorker and a maintainer restored it two days later because queued jobs would have failed.

What it does

Cust is a GitLab workflow for Ruby, Sidekiq and Redis apps.

• cust:mr-check replays retained old-shape jobs against the MR's worker code in a real Redis and checks business results, not just crashes. Failing checks are written out in the job.
• A Cust flow on the GitLab Duo Agent Platform writes a compatibility shim, replays the same jobs to test it, and splits the change into three release steps. Duo's approve, merge, deploy, issue-edit and variable tools are removed, so it can't release anything itself.
• CI turns an approved release spec into a plan issue and three ordered MRs: compatibility, producer switch, cleanup. Each step needs a human cust-approve on its exact commit, a passing gate and a staging deploy.
• The cleanup gate fails closed. It checks that no old-shape jobs are waiting, that registered workers and producers run the new commit, and that evidence is fresh. If it can't tell, it says inconclusive.
• A read-only census reports which job shapes in a Redis the cleanup code can't run.

How we built it

Ruby harness and gates in GitLab CI, Sidekiq 7.3, Redis, a Duo Agent Platform flow, a sample app deployed to a free Render service by a GitLab job, and a Mastodon replay built from Mastodon's official Docker images.

Evidence

• Repo: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43066430/cust (start with docs/demo.md)
• Blocked change: MR !40. Duo session 10018104 and spec MR !42.
• Ordered release: plan issue #6, MRs !52, !53, !54. Cleanup gate job 17459517125 returned ready_for_review.
• Render free-tier demo production: https://cust-demo.onrender.com, observation job 17459415593.
• Mastodon near-miss replay: job 17460707996.

Challenges

Live runs found real bugs that Cust failed closed on: a Render rollout overlap where old and new instances shared a producer id (!55) and a plan refresh that missed merges while deploys ran (!56).

Limitations

Sample-app data is synthetic and the Mastodon run is lab data on a stand-in image. Render's queue was empty, so it proves deployment binding, not a drain. Fleet size is declared in release/fleet.json and checked against live Sidekiq processes. All approvals came from one maintainer.

What's next

A read-only census on a real team's Redis, then advisory mode on their worker MRs.

Built With

• docker
• gitlab
• gitlab-ci
• gitlab-duo
• redis
• render
• ruby
• sidekiq