InvenTree Lifecycle

Shipping is day one. Upgrading is every day after. One GitLab comment upgrades the InvenTree you already run, rehearsed on a copy of your real data before production moves.

Comment

Inspiration

A small company's InvenTree holds its stock, bills of materials and part documents. Nobody upgrades a self-hosted site for them. Doing it carefully takes about twelve manual steps: back up the database and media, build, test, scan, stand up a copy, restore into it, compare the data, switch web and worker together, check the worker, merge, close the ticket. Skip one and inventory can be lost, so many sites stay on old releases with known vulnerabilities. Writing code is the easy part here. The risk is in the release and in running the site afterwards, and that is where we wanted agents.

Any business that holds stock needs something like InvenTree. It's our first example; the same approach fits any self-hosted app with a database, files and a background worker.

What it does

You point Lifecycle at a running InvenTree. It inspects the site over SSH, a GitLab Duo onboarding flow proposes how to adopt it, and one owner approval enrolls it. propose 1.6.1 opens an issue and a merge request for the upgrade. From then on each decision is one comment on that issue:

  • /inventree rehearse runs everything except the switch.
  • /inventree complete also upgrades production, merges the tested revision and closes the issue.
  • /inventree remediate asks Duo to fix dependencies the verifier refused.

Between comments nothing waits for a person. A custom Duo flow, the Release Assessor, reads the request, the MR, CI and the upstream changes, rates the risk, names risky migrations and can block or add checks. A protected verifier pipeline in a separate private project builds the exact revision, runs InvenTree's own backend test suite, scans the image and signs a receipt. The controller then pauses writes, snapshots, clones the data into an isolated copy and runs InvenTree's update steps there. On the copy it compares stock, BOM and attachment fingerprints, checks the schema and the new web interface, fetches attachments over HTTP and sends a job through the background worker. Only after all of that does it switch production, check the route through the site's own proxy and post the evidence on the issue and MR. A managed monitor keeps checking the site afterwards.

Autonomy: Supervised

The owner states the goal and approves each outcome with one comment. Duo flows and the controller do the work in between. Duo can block a release or add checks, but its output never becomes a deployment command and it cannot approve or waive a check. When a run stops, nothing retries automatically. The owner reads the reasons and decides.

How we built it

  • GitLab Duo Agent Platform: three custom flows. The Release Assessor judges each request, the Remediation Planner picks fixing releases from the verifier's signed findings, and Onboarding plans the adoption of a site.
  • GitLab CI: source CI on every MR, plus the protected verifier project (build, InvenTree's backend suite, live API checks, Trivy, signed receipt).
  • A Python controller with a crash-safe journal, one authority reservation per release, and recovery that never replays an uncertain step.
  • Google Cloud: the demo site on Compute Engine (deploy/gcp), and Lifecycle itself on a small always-on controller VM plus a verifier build host that powers off after 5 idle minutes (deploy/gcp/lifecycle). The controller's service account can start that one VM and nothing else.

What we added on or after October 5

The InvenTree application is upstream: the fork starts at stable 1.5.6 (commit 284a670413cc), and upstream 1.6.1 was later merged in as the upgrade target. Everything else was built during the hackathon, from October 5:

  • The controller and delivery pipeline (delivery/): adoption, release, rehearsal, probes, monitor, inbox, recovery, about 760 tests.
  • The protected verifier pipeline with signed receipts.
  • The three Duo flows and their prompts.
  • Google Cloud deployment of both the site and Lifecycle (deploy/gcp, deploy/gcp/lifecycle).
  • SCI measurement scripts (deploy/sci).
  • The evidence record and changelog.

InvenTree's own files changed very little: dependency pins for advisories the verifier found, the container build (base-image pins, a reproducible Python environment), a worker-timeout setting with its tests, and tests that no longer need the public internet so the verifier can run them in isolation.

Evidence (all public on the GitLab project)

  • #124, 27 Oct: a stock InvenTree 1.5.6 installed exactly as InvenTree's Docker guide describes, on the official image, upgraded to 1.6.1 by one /inventree complete. Time 1h25m44s, 22 migrations, writes paused about 20 minutes. Getting there took three product fixes, found on #121 to #123, each listed in the evidence record.
  • #106 and #111, 24 Oct: our Google Cloud site, 1.5.8 to 1.6.1. The verifier refused upstream 1.6.1 for a critical brotli advisory published two days earlier. Duo's Remediation Planner chose brotli 1.2.1 and pypdf 6.21.2, and the controller committed the pins in 2m37s. Two more refusals followed, for a merge mistake of ours and a broken behaviour check, and each was fixed without touching the gate. Then one /inventree complete delivered it: 1h54m44s.
  • #101, #97, #92, #89: earlier rehearsals, promotions and completions on two other sites.
  • The verifier project stays private by design: Lifecycle refuses to enroll a site while it is public, because it holds the signing setup. Each public summary names the verifier pipeline it came from, and the video shows those pipelines' real logs.

Challenges we ran into

Getting a stock InvenTree, installed exactly as InvenTree's Docker guide says, from 1.5.6 to 1.6.1 took eight attempts over two days. Google Cloud had no capacity for our build VM in our zone for a while. Our own pre-check refused the official image's Python because it lived somewhere we didn't expect. A Duo session returned nothing. One of InvenTree's tests failed by 7 milliseconds (2.507 s against a 2.5 s limit) on a cheaper VM. The new image laid out Python differently from the old one. The official install keeps its static files in a folder our rehearsal doesn't mount. Each failure was a real bug or a real limit, and each stop left production where it was.

The upstream release itself was refused. InvenTree 1.6.1 shipped with a critical brotli advisory published two days earlier. Our verifier blocked the build, and then blocked it twice more, both times for our own mistakes: a merge that kept five old backport files, and a behaviour check that used a field 1.6 removed.

The hardest part was not changing the gate when it kept saying no late at night. Every fix went into the product, not into an exception or a hack.

Accomplishments that we're proud of

  • A site we didn't build, set up the way InvenTree tells people to set it up, went from 1.5.6 to 1.6.1 with 22 database migrations after one comment (#124, 1h25m44s). Duo assessed it, the protected verifier built and tested it, the upgrade ran on a copy of the data first, and then production moved.
  • The gate caught a real critical vulnerability in a real upstream release. Duo picked the fixed versions, and the controller wrote and committed the pins in 2m37s.
  • Everything is on the record: 31 defects that real sites exposed, each with its cause and fix, every refusal linked. The video plays back real pipeline logs and a real terminal session.
  • We measured the carbon of real releases: about 75 gCO2e for a feature upgrade, against about 368 g done carefully by hand. The build VM switches itself off when idle.

What we learned

  • Duo is most useful as a reviewer that can say no: judging risk, naming risky migrations, picking package versions. We never let it run commands against someone's inventory, and we never needed to.
  • When you don't know whether a step happened, stop and check. Retrying is how you corrupt data. That rule brought production back unchanged three times.
  • Measuring energy changed the code. Sharing one SSH connection per site cut a release's VM work from about two hours to forty minutes.
  • Security advisories arrive faster than upstream releases. The newest release is not automatically safe.

What's next for InvenTree Lifecycle

  • Native Linux installs. Lifecycle connects to them today; their upgrade hasn't been rehearsed yet.
  • Shorter write pauses. A completion pauses writes for 20 to 38 minutes now. The site could stay readable during the rehearsal and lock only for the final switch.
  • Automatic cleanup after a clone whose outcome is unknown. Today the operator clears it through the controller's own state store.
  • A hands-off mode for patch releases: the same gates, with the owner told afterwards instead of asked first.
  • Other self-hosted apps. A database, files and a background worker make up the same twelve-step upgrade everywhere. InvenTree is the first one we did.

Sustainability

We measured releases with the Green Software Foundation's Software Carbon Intensity specification (ISO/IEC 21031:2024): VM and host CPU sampled every five seconds, the Duo session's real token counts, and Google's published grid intensity for us-central1. The 1.6.1 completion on #111, with 22 migrations, used 34 to 40 Wh, or 17 to 19 gCO2e, and about half of that was the one Duo assessment.

Counting Lifecycle's VMs whole and the owner's review time, a feature upgrade like this comes to about 75 gCO2e. Done carefully by hand it is about 368 g, and with a frontier AI coding agent about 688 g. Across InvenTree's 36 releases of 2026 that is 1.9 kg and 10 hours of review, against 5.7 kg and 184 hours by hand.

Two engineering changes came out of measuring:

  • The verifier host powers off when idle.
  • One shared SSH connection per site cut a release's VM work from about two hours to about forty minutes.

A quick admin doing a patch release in two hours roughly ties with us, and the controller's always-on monitor (about 47 g a day) is reported separately. Method, sources and a calculator: delivery/docs/SUSTAINABILITY.md and SCI-BY-HAND.md.

Built With

Share this project:

Updates

Submission history