Ami Ama Guardian

Supervised MLOps for offline AI farmers trust: GitLab agents carry every change to an offline coffee-leaf app from a field report to the farmer's phone, and one human expert approves only the release.

Comment

Links
GitLab project (public, MIT) https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/35646169/showcase
Deployed app (Google Cloud Run, production) https://ami-ama-prod-sqcnb353pa-lz.a.run.app (how this build was released: https://ami-ama-prod-sqcnb353pa-lz.a.run.app/guardian)
Staging https://ami-ama-staging-sqcnb353pa-lz.a.run.app
Original project (Path B, MIT) https://github.com/vansyson1308/ami--ama (imported at 964419a, the last commit before 5 Oct 2026)
Demo video (< 3 min) https://youtu.be/hzrONHCp3hw
Re-check every claim bash scripts/transcend_evidence.sh (public API only, no token)

Tagline

Supervised MLOps for offline AI that farmers trust: GitLab agents carry every change to a 4.4 MB offline coffee-leaf app from a field report to the farmer's phone, and one human expert approves only the release.

Inspiration

When I was hired to pick coffee in Buôn Ma Thuột, in Viet Nam's Central Highlands, I stayed with an old farmer who has grown coffee for many years. I knew nothing about leaf disease. When a leaf looked wrong, I took a photo and sent it to him, and he would tell me what it was.

That stayed with me. Farmers there know a great deal, but most of it comes from years in the field. It lives in people's heads, not in a form a newcomer can learn or check. I was lucky: I had him to ask.

Ami Ama is a small attempt to put some of that knowledge, taken only from vetted sources, into a phone that works offline: a photo, a careful answer in Vietnamese, and, when the model is unsure, "Chưa chắc — cùng kiểm tra thêm" (not sure, let's check further), with a "🙋 Hỏi cán bộ" (ask an officer) button. It is meant to send people to experts like him, not to replace them.

That is also why every release is supervised. A phone may stay offline for weeks, so a wrong line of advice cannot be quietly fixed later. Each change is checked by code and approved by a person before it ships, the same way I checked with him before trusting my own guess.

What it does

A wrong diagnosis reported from the field becomes a safe fix on farmers' phones:

  1. Plan. An extension officer's field report arrives as a GitLab work item. The Field Triage flow classifies it (model confusion, out of scope, image quality, duplicate, needs info). It quotes only numbers from the model card and recommends one option from a fixed menu.
  2. Create. A project member comments @…fix-it option A. The Fix-it flow drafts one short Vietnamese tell-apart line, citing a source vetted for that card, and code opens a draft MR. Code also writes the line into the card's audio text; the clip is regenerated at build time.
  3. Verify, secure, package. The MR pipeline runs deterministic gates. When the MR is marked ready, the Release Guardian flow posts the verdict (decided by code), an Advice Safety agent review and a bilingual explanation. The gates:
    • eval gate on a frozen eval pack: per-class precision, coverage, ECE, never-assert, non-coffee false accepts;
    • browser vs Python parity in airplane mode;
    • content safety lint: no doses, brands or over-claims; vetted sources; short lines; voice follows the screen;
    • SAST, secrets, dependency, container and licence scans;
    • Lighthouse, and SCI.
  4. Release. The one human step: a listed approver comments approve <sha8>. The Release flow re-checks the whole chain from GitLab records and hands the merge to the approver with exact approval trailers. Then release:guard re-verifies them, and the pipeline:
    • tags the commit and publishes a GitLab Release (approval record, notes, SBOM, evidence package with checksums);
    • deploys keyless to Cloud Run staging, runs an airplane-mode smoke test, then deploys to production.
  5. Monitor and govern. The Release flow posts a summary on the original report, labels it release::live and closes it. A weekly Field Health flow and scheduled pipelines check production offline. A failure opens an incident by code, and the recovery closes it. Every release carries a governance report.

Agents write words; code decides. Every flow starts with deterministic steps that compute the facts, and the LLM only explains them. Gate code always runs from main, never from the MR being judged.

Why does it sometimes say "ask an officer"?

On field photos (RoCoLe test, model card), Ami Ama answers 79.2% of them and is right 91.1% of the time it answers. For the rest it says "Chưa chắc" instead of guessing, because a confident wrong answer can cost a farmer a needless spray. When unsure, it first asks up to three yes/no tell-apart questions (content/questions.vi.json), then prepares a message with the photo and the answers for an extension officer.

Every officer answer comes back as a field report, through the same report → fix → supervised release loop shown in the hero runs (whose reports are simulated). Local, expert-labelled photos are how coverage grows in Tây Nguyên: today there are no Vietnamese photos in training.

How we built it

  • GitLab Duo Agent Platform: five custom public flows and one custom public agent in the AI Catalog, published by catalog-sync on tags. Each flow uses deterministic run_command steps, typed gate steps, then an agent:
    • Field Triage, Fix-it, Release Guardian, Release, Field Health;
    • the Advice Safety agent.

Triggers: work_item created, mention, merge_request_ready, pipeline_hooks, scheduled. Every chain starts from a human action.

  • Foundational flows were used on our own work: Fix CI/CD Pipeline, SAST false-positive detection and Code Review. Code Review found two real bugs before a merge (FLOW_EVALS.md).
  • CI/CD: one .gitlab-ci.yml covering all nine stages (STAGES.md), with rules:changes so the ML evaluation runs only when the model, content or ML code changes. The CI and flow logic lives in small, unit-tested scripts (ci/*.py).
  • Google Cloud: Cloud Run (europe-north1) for staging and production, Artifact Registry, and Workload Identity Federation from GitLab OIDC id_tokens. There are no service-account keys anywhere.
  • Tests:

Proof that it runs (not a proof of concept)

  • Three hero runs, report → release, linked step by step in the README:
    • #8 → !33 → v2.1.0;
    • #23 → !51 → v2.1.2;
    • #24 → !64 → !66 → v2.1.3. !64 was approved but merged once without the approval trailers, so nothing was released and production did not change. Its change was then re-released through !66, approved and merged with the trailers. The v2.1.3 provenance wrongly said no vetted source was cited; the correction is linked from the Release.

The photos and model errors are real public test images; the reports are simulated.

  • Two releases the gates blocked:
    • !19: a pesticide dose and brand in the advice;
    • !20: a threshold change that dropped rust precision 0.928 → 0.906.
  • An incident opened and closed by code on staging (a time-limited test, FLOW_EVALS.md).
  • 361 pipelines, 54 MRs and 183 Duo agent sessions since 5 Oct, as of 9 Oct 2026 (BUILT_DURING_TRANSCEND.md). The session ledgers in BUILD_LOG.md list every Duo session with its trigger and LLM steps. In M3, 44 of 57 sessions ended in deterministic steps with no LLM call.

Sustainability (SCI, ISO/IEC 21031:2024)

Measured in every pipeline by govern:sci (SUSTAINABILITY.md, every coefficient cited):

  • 0.16–0.19 g CO2e per 1,000 diagnoses on the phone over the last 10 main pipelines;
  • the same diagnoses with cloud inference would be ≥ 34 g, a lower bound dominated by uploading the photo, so at least 178× more;
  • the change rules saved 241.8 CI minutes ≈ 20.8 g CO2e over the last 40 pipelines (measured from the GitLab API).

Cloud Run runs in europe-north1 (30 gCO2eq/kWh). Because the app works offline, the region adds no latency to a diagnosis.

Challenges we ran into

Each one is written up with its evidence and fix in FLOW_EVALS.md and DECISIONS.md:

  • Flows cannot merge. The flow token's scope excludes the merge endpoint, and the first approved release failed silently. We redesigned it: the flow verifies, and the human merges with trailers that CI re-verifies (D11).
  • "Not released" for a live release. The flow runtime cannot reach Cloud Run, so release status is now decided by code from the deploy job's evidence (D12).
  • run_command reports success on a non-zero exit code, and read_file on a missing file. Stopping a flow without an LLM call needed a flag file and a typed gate step.
  • Content safety beyond doses and brands:
    • an unvetted citation passed the gate, which led to vetted sources per card (R11);
    • a hedge-less tell-apart line ("…là dấu hiệu bệnh rỉ sắt") on the healthy card passed lint in hero run 2. The coordinator caught it in review, and lint rule R4-certain was added.
  • A transient ConnectionResetError turned a release pipeline red. Every API read now retries transient errors, and the runner retries only platform failures.

Accomplishments that we're proud of

  • The whole post-code lifecycle of an offline AI model, end to end on GitLab, with one human decision per release.
  • Gates that cannot be talked past. They are CI code, run from main, and re-verified after the merge.
  • Honest evidence: simulated data is labelled everywhere, numbers link to artifacts, and a script anyone can run re-checks the claims from the public API.

What we learned

  • Put the LLM where words are needed and nowhere else: deterministic steps first, then the agent, then code for state.
  • A human review step still catches things the gates miss, and each catch should become a gate (R4-certain).
  • Platform limits are design inputs. Record them, design around them in the open, and never work around a permission.

What's next

  • Vietnamese field photos labelled by extension officers, to replace the Ecuadorian field test set.
  • A labelled photo set built with a Tây Nguyên extension centre or WASI (Viện Khoa học Kỹ thuật Nông Lâm nghiệp Tây Nguyên), and τ recalibrated on it.
  • Officer review of the whole advice library before real use.
  • More crops through the same pipeline (pepper, durian, cashew).
  • A Bahnar language pack with a native speaker.

Built with

GitLab Duo Agent Platform (custom flows and agent, AI Catalog, triggers, foundational flows) · GitLab CI/CD (SAST, Secret Detection, Dependency and Container Scanning, Releases, Package and Container Registries, Environments, schedules, incidents) · Google Cloud Run · Artifact Registry · Workload Identity Federation · React · TypeScript · Vite · ONNX Runtime Web (WASM) · MobileNetV3 (int8) · Python · Playwright · Lighthouse · Piper TTS · CycloneDX.

Path B — what we added since 5 October 2026

The original project, https://github.com/vansyson1308/ami--ama (MIT), is the offline PWA and its model. Its history was imported unchanged, with the original dates. Everything below is new work in the GitLab project, on or after 5 Oct 2026 (dated list: BUILT_DURING_TRANSCEND.md):

area added
agents and flows Field Triage, Fix-it, Release Guardian, Release and Field Health flows, and the Advice Safety agent, all public in the AI Catalog with their triggers; deterministic flow steps (ci/flow_boot.py, ci/*_facts.py, ci/release_check.py, ci/fixit_apply.py, ci/field_health.py)
CI/CD, all nine stages eval gate on a frozen eval pack, browser–Python parity, content safety lint (R1–R7, R11, R12, R4-certain), recalibration job, security scanners, licence check, SBOM, Releases, evidence packages, Environments, scheduled monitoring, incidents, governance and SCI reports
deployment Dockerfile (nginx), Cloud Run service YAML for staging and production, keyless OIDC → WIF deploys, offline smoke tests, rollback job
app (build-time only, no behaviour change to diagnosis) Vietnamese audio generated at build time with a manifest check; a static /guardian page showing how the build was released; the three content fixes shipped through the hero runs
evidence and docs scripts/transcend_evidence.sh, flow evals with simulated field reports, BUILD_LOG with session ledgers, DECISIONS, STAGES, SECURITY, SUSTAINABILITY, CONTENT_SAFETY_RULES

The original Vercel deployment (https://ami-ama.vercel.app) and the GitHub repository were not changed.

Built With

  • artifact-registry
  • cyclonedx
  • gitlab-ai-catalog
  • gitlab-ci-cd
  • gitlab-duo-agent-platform
  • google-cloud-run
  • lighthouse
  • mobilenetv3
  • onnx-runtime-web
  • piper-tts
  • playwright
  • python
  • react
  • typescript
  • vite
  • webassembly
  • workload-identity-federation
Share this project:

Updates

Submission history