Adaptive Vision Pipeline Guardian

AI-assisted regression investigation and release safety for computer vision models

Comment

Project name

Adaptive Vision Pipeline Guardian

Elevator pitch

Catch ML regressions hidden by better averages, with real GitLab Duo investigation and human-controlled promotion review.

Path

Path A — Start Fresh. Guardian is a new project created October 26, 2026, within the hackathon period. It does not import the existing InventoryLens application; it uses owner-authorized derived numeric evidence as inputs. No deployment is claimed or required for this Path.

Autonomy

Assisted. A human separately authorized publication and the bounded Duo investigation. The agent's next action remains unapproved. Read-only inspections were preapproved within the investigation scope; there was not a manual click before every read. No autonomous correction, merge or deployment is claimed.

Built with

Python; GitLab; GitLab CI/CD; GitLab Duo Agent Platform; HTML; CSS; JavaScript. Python standard library only. No training or model-inference dependency.

Repository and working-version links

Demo video URL — required before submission

MISSING: paste the actual public YouTube URL after the 2:50 video is recorded, uploaded with authorization, and checked while signed out. No URL is invented.

Deployment / Google Cloud bonus

Not applicable to this Path A entry. No deployment or Google Cloud bonus is claimed. The offline dashboard opens from a local checkout without hosting.

Inspiration

An ML experiment improves overall candidate-ranking agreement, but three previously successful predictions regress. Adaptive Vision Pipeline Guardian discovers those failures, GitLab Duo investigates the evidence, and the release review withholds promotion of an insufficiently verified change.

Our audience is developers reviewing computer-vision changes. A better mean score does not reveal which previously working cases were lost, and an agent's plausible explanation is not evidence that a model is safe.

What it does

Guardian replays archived numeric experiment records and discovers item-level ranking transitions across a selected cohort. The archive improves from 266/290 to 271/290 agreements with provisional reviewer choices. It contains eight gains, three correct-to-wrong regressions and four wrong-to-wrong changes.

Production discovery does not hardcode expected failing IDs. Reports retain stable sample IDs, saved score margins, source hashes, all 266 originally successful samples and a deterministic same-cohort control policy. A small self-contained offline dashboard makes the actual evidence searchable.

These are archived candidate-group agreements, not verified commercial SKU accuracy or fresh model inference. Archive metadata labels a gallery representation change, but independent per-candidate gallery provenance is incomplete. The real-evidence recommendation remains INCOMPARABLE. This is conservative promotion withholding, not an existing production deployment block.

How we built it

The new MIT Guardian project uses standard-library Python, strict JSON/numeric validation, SHA-256 provenance, frozen fixtures, registered diagnostics and GitLab CI artifacts. All 111 archived cases replay across separate ranking, detection, mask-summary, crop and count-review cohorts. We do not pool unlike metrics or turn missing predictions into zero.

Actual pipeline 2993838999 at core commit ffc5f3f4 succeeds: 59 tests, real-evidence replay and the separate SYNTHETIC release fixture gate. Its downloadable archive contains JSON/Markdown and 111 per-case reports. Real reports remain INCOMPARABLE; the synthetic GOOD fixture returns REVIEW.

Genuine GitLab Duo developer/v1 session 10119226 ran in native Plan mode and performed 56 tool executions. It fetched MR/CI evidence and checked targeted raw score records. Its useful additional reasoning was to challenge the chosen control: the 21-match scene has no IL-NEW-008 reviewer matches. A uniform boost to that candidate could restore a target while reversing existing gains. Duo selected gain sentinels and proposed rank_margin_audit.

It stopped at input_required. No command, diagnostic, model correction or measured improvement followed. Its proposed request has unresolved fingerprints and an inaccurate target/control statement. It requested Build mode; we did not approve that switch. Public evidence preserves these limitations rather than manufacturing a completed loop. Internal reasoning, credentials and host details are excluded from the public record.

Challenges we ran into

Ranking agreement is not accepted-prediction safety or commercial identity. The archive lacks verified SKU identities, acceptance/unknown-rejection metrics and independent gallery membership. The largest stable scene is also not necessarily a useful candidate-specific control.

Duo faced truncated/empty API responses, a log rejected as binary and no shell for canonical hashes. Its critique of circular tests also missed synthetic generalization coverage. The native session is not anonymously accessible, so we prepared a redacted actual-call record, clear summary and authentic video footage rather than relying only on a private session link.

Accomplishments

Working public CI verifies 59 tests and 111 archived cases. Dynamic discovery exposes losses hidden by a better average. Real GitLab Duo checks source evidence and adds a defensible control-coverage challenge. The record preserves actual tools, failures, actor attribution and the unexecuted human request.

What we learned

Useful agents must challenge generated evidence, identify collateral regressions and admit missing tools or data. A request hash is not authenticated human approval, and a proposed diagnostic is not an implemented improvement.

What's next

After a separately reviewed action request and supported execution route, verify candidate identities and gallery provenance, then design a matched comparison that preserves existing successes and measures false accepts. No further experiment, inference or feature implementation is part of this submission.

Lifecycle coverage

Plan: genuine investigation/proposal. Create: authorized MR publication and historical Duo review. Verify: automated tests/replay. Package: report archives only. Secure: input validation and the local read-only boundary, no security scan. Release: conservative promotion review, no deployment. Govern: provenance, sealed records and scoped human decisions. Configure and Monitor are not implemented; archived comparisons are not live monitoring. These stages combine CI, agent, human and coordinator actions, not seven autonomous Duo workflows.

License and limitations

MIT Guardian code/documentation; owner-authorized derived numeric evidence with InventoryLens attribution. No private photographs, overlays, checkpoints, source implementation or credentials are distributed. The native session's full export remains private; the public record is explicitly redacted. No GPU, training, new inference, deployment, production monitoring or verified correction is claimed.

Final form checklist

  • [x] Public GitLab project and MIT detected by GitLab.
  • [x] Visible successful pipeline and downloadable reports; anonymous project/pipeline/report checks passed.
  • [x] New Guardian project within the submission period; Path A supported by history.
  • [x] Genuine Duo Agent Platform activity; Assisted action boundary and preapproved reads disclosed.
  • [x] English description and dependency-minimal branch-specific setup prepared.
  • [ ] Publish the reviewed dashboard, quickstart and redacted evidence package.
  • [ ] Record the real 2:50 demonstration and upload a public YouTube video after authorization.
  • [ ] Paste its verified URL; do not leave this required field blank when submitting.
  • [ ] Owner confirms Devpost registration, eligible entrant/team, rights and applicable DCO terms.
  • [ ] Owner authorizes and completes submission before October 27, 2026, 13:00 UTC / 14:00 Zagreb.
  • [ ] Keep code/reports available through judging ending November 14, 2026, 17:00 UTC.

Requirements checked against the official overview and rules. Path A deployment is optional. Final personal eligibility/registration and form submission are owner actions; no entry or video has been submitted by this preparation.

Built With

Share this project:

Updates

Submission history