Based

Based catches compute regressions, lets GitLab Duo repair them, and proves the same work uses less CPU—with evidence, cost accounting, and human approval.

Comment

Inspiration

A merge request can pass its tests and still make every request more expensive.

Correctness tests answer, “Does this work?” They rarely answer, “How much compute does the same useful work now require?” Small inefficiencies can repeat across every request after deployment.

We built Based to make software sustainability an engineering decision inside GitLab: detect unnecessary compute, let an agent propose a repair, and require independent evidence before accepting the result.

Our principle: same work, less compute—with the cost of proving it included.

What it does

Based is a supervised GitLab Duo Agent Platform workflow for detecting and repairing compute regressions.

In the recorded demonstration, Duo’s repair used 59.1% less CPU per request, with matching output across all 24 paired measurement blocks. CPU fell from 51.870 to 21.209 milliseconds per request.

The workflow connects detection to a reviewable decision:

  1. Detect: GitLab CI compares baseline, candidate, and unchanged-code control executions for a defined unit of useful work.
  2. Investigate: An opted-in failed pipeline triggers GitLab Duo. The flow checks authorization and evidence before investigating.
  3. Repair: Duo inspects the request profile, diagnoses repeated database lookups, and opens a separate draft merge request.
  4. Verify: Independent CI tests the repair against the existing verifier and correctness checks. The agent cannot award itself a pass.
  5. Review: Based presents the result, paired observations, costs, and remaining approval requirements. It never automatically merges.

The public evidence desk lets reviewers inspect exact observations and recompute retained results in their browser—without another CI job or model call.

The demonstration is a disclosed replay of a historical regression in a prepared Healthchecks workload. We supplied the workload and regression, not the repair. It is not a claim of discovering a new upstream incident.

Explore Based · Inspect the demonstrated repair

How we built it

Based is new MIT-licensed automation built for this hackathon, entered under Path A: Start Fresh, Supervised.

GitLab Duo performs the investigation and authors application repairs. GitLab CI independently validates correctness and compute behavior. Merge requests, job artifacts, and retained receipts connect findings to their source revisions. GitLab Pages hosts the evidence desk.

Codex assisted with the measurement harness, evidence validation, interface, tests, and integration. Historical third-party fixtures retain their original licenses and attribution.

The project links execution evidence across all nine DevSecOps lifecycle stages:

  • Plan: issues define the work and human acceptance requirements.
  • Create: Duo produces reviewable draft repairs.
  • Verify: independent CI checks correctness, useful output, and compute.
  • Package: evidence is published as an inspectable package.
  • Secure: code, dependencies, and secrets are scanned.
  • Release: the Based evidence desk is released and published.
  • Configure: runner and branch configuration are checked.
  • Monitor: probes check evidence availability and integrity.
  • Govern: changed source requires renewed human review.

This coverage describes the overall product workflow. Package and Release publish Based’s evidence desk; upstream application repairs remain unmerged and undeployed.

Sustainability accounting

We separate measured, modeled, and unknown quantities.

CPU per useful request is measured. Electricity is modeled, not power telemetry. Reference carbon scenarios expose the Software Carbon Intensity relationship:

$$ SCI = \frac{E \times I + M}{R} $$

The methodology identifies energy, carbon intensity, embodied emissions allocation, and the functional unit. Sourced assumptions remain visible rather than being presented as observed runner characteristics.

The cost ledger also exposes verification costs, token-based agent energy scenarios, and missing costs. Unknown costs are not treated as zero. The retained ledger shown in the video predates the fresh demonstration run.

Challenges we ran into

Separating a real improvement from measurement noise. Paired observations, unchanged-code controls, matching output hashes, and sampling intervals make a percentage more defensible than a single timing run.

Preventing the agent from validating its own work. Repair generation and independent verification have distinct responsibilities. In the demonstration, Duo changed five lines in one application file while leaving the verifier and tests unchanged.

Making uncertainty actionable. Invalid experiments must remain inconclusive. A performance improvement must not override governance. The demonstrated repair passed its sustainability gate, but the overall pipeline remained blocked because changed source required a new review.

Accounting for the automation itself. Finding an optimization consumes resources too. Based exposes those costs rather than reporting only the attractive reduction.

Accomplishments that we're proud of

  • Recorded a real GitLab sequence from failed gate to Duo investigation, draft repair, and independent verification.
  • Demonstrated 59.1% less CPU per request, with matching useful output and valid unchanged-code controls.
  • Made all 24 paired measurement blocks inspectable and browser-recomputable.
  • Preserved the human approval boundary even after performance passed.
  • Connected evidence across all nine lifecycle stages.
  • Delivered a public evidence desk that can be inspected without signing in or rerunning the benchmark.

What we learned

A sustainability result needs a denominator. “Less compute” becomes meaningful when it is tied to the same useful work.

It also needs a boundary. We demonstrate reduced CPU demand in controlled tests, not realized production energy or carbon savings. Actual runner SCI and net environmental benefit remain unknown.

The useful role of an agent is to turn evidence into a proposed change. Independent checks and human judgment determine whether that change should ship.

What's next for Based

Next, we want to validate the workflow with independent maintainers, broaden workload adapters, and add runner energy measurements where available.

The longer-term test is production impact: whether accepted repairs reduce resource use enough to repay the cost of finding and verifying them.

Based turns sustainability from a claim into a reviewable change: less compute for the same work, with evidence attached.

Built With

Share this project:

Updates

Submission history