-
-
MR-A, the Safety Case note: the oracle scored all 38,026,560 situations. 972,648 less severe, 0 more severe; labelled safety::regression.
-
Ranoka: people approve outcomes. Built with GitLab Duo Agent Platform flows. Synthetic data only.
-
The app in Malagasy, same answers on both methods: 2.0.0 said Alert (Loza), 2.1.0 says Good (Tsara). A less severe verdict.
Inspiration
In Antananarivo, people wash clothes, bathe and water animals in canals and streams that also carry sewage, industrial run-off and, some seasons, toxic algae. Ranoka (Malagasy for "water") is a free app that turns a short visual check by the water's edge into a verdict, the warning signs behind it and plain advice, in Malagasy, French and English, offline.
The hard part is not writing the rules; it is changing them safely. A rule change that looks harmless can tell a mother that a sewage canal is fine for laundry, and a change that looks alarming can be the right call. One maintainer cannot review that by hand across 38,026,560 possible checks. The post-code lifecycle (review, verification, security, release, monitoring, governance) needed help, but not from agents that guess. So we built agents that cannot.
What it does
Ranoka hands its lifecycle to three custom GitLab Duo flows and a deterministic pipeline, under one rule: deterministic tools decide, a model explains, a person approves the outcome.
- Mpiambina Safety Case (Malagasy for guardian) runs when a person opens a merge request, marks a Draft ready or asks the flow for a review. If the change touches the method, the scoring code or a health text, an exhaustive oracle scores every situation before and after the change and classes each one (less severe, more severe, assessability change, unchanged). A read-only model explains the result; a model with no tools drafts the note in English with French and Malagasy machine drafts; a deterministic step posts it only if every number in it is in the oracle's output. The safety label comes from the oracle, never from the model, and so do the figures of the hazard-log entry it drafts for a person to decide (on MR-A, that draft gave the entry a new id instead of the merge request's existing one: a known issue, to be fixed in its own merge request).
- Remediation Steward triages issues against the project's invariants and opens Draft merge requests that fix real defects, each with failing-then-passing evidence from its battery of 7 oracle checks or from a new unit test. It never touches the method, a health text or the pipeline: 37 protected paths are refused before a change leaves its sandbox.
- Release & Monitor Sentinel writes the incident analysis after the pipeline has already rolled production back, and drafts French and Malagasy release notes for a native reviewer.
- GitLab Duo Code Review, GitLab's foundational flow, has review instructions in the repository that restate our safety rules per path; it has not reviewed one of our merge requests yet.
- The pipeline (32 jobs in 5 lanes) runs the gates on each merge request, the heavy ones when their inputs change. After a merge that changes what ships, its main lane builds the app twice to the same bytes, writes a CycloneDX bill of materials and in-toto provenance, deploys GitLab Pages, smoke-tests the live site and, if that test fails, rolls back by itself to the last good build (no rollback has succeeded on GitLab yet: see "Limits"). A release starts only when a person pushes a protected tag; from there, the release lane runs every gate again, then packages, signs and publishes the release. An announced drill publishes a tampered file to prove the rollback.
In the demo, MR-A is real science, and MR-B is a realistic refactor slip we built on purpose, of the kind no example covers:
- MR-A adds the oil stick test (method 2.0.0 → 2.1.0): a rainbow film
that breaks into flakes is natural iron, not fuel. On all 38,026,560 situations of
both methods (method 2.1.0 adds an answer), 972,648 become less
severe, 640,152 alerts become "not assessed" and
0 become more severe; 6,082,560 lose or lower a warning sign
and 6,082,560 lose protective advice. The smallest example goes from
Alert (89) with chemicals or fuel suspected to Good (100) with no warning sign. The oracle labels it a regression,
because something loses protection; whether that is justified is a person's call.
A second person, a code owner, accepted the hazard-log entry that cites outcome
321bc46e3f8b, and approved the merge request (its author did not); it was then merged while its pipeline was still running, and that pipeline passed afterwards, the guard green on the same digest. GitLab required neither the approval nor a green pipeline: both are Maintainer settings, off on this project. That digest covers the outcome of every situation, so the decision covers exactly what the oracle measured: any later change to the outcomes changes the digest and turns the guard red again. https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/merge_requests/3 - MR-B is a tidy refactor of the code that builds each warning sign, with the slip built
in: it counts a sign's certainty over its top-tier cues only.
Every example-based job passes on its tip, run as CI runs it: 533 tests with none failing, all
26 vignettes, the HL7 validator and the end-to-end harness.
No verdict moves.
The exhaustive oracle finds 1,774,080 situations with
a warning sign held with less certainty: the smallest goes from Alert (79) with chemicals or fuel likely to
Alert (79) with chemicals or fuel suspected. The properties job and the guard fail on its tip, so its pipeline
goes red; it is a demo, never to be merged.
The slip is not cosmetic: on the branch, the app's own FHIR check rejects the export of that
smallest example (rule
rnk-hz-3: a sign's certainty follows from its cues), so the app would ship records that break its own published profile. https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/merge_requests/4
This is the Supervised level as the hackathon defines it: people approve outcomes, not steps. After the merge, no person and no model sits on the critical path of the main lane. A release is a person's act; after it, no person and no model sits on the release lane's path either.
The nine lifecycle stages, and who does each:
- Plan. Issue triage: the invariant at stake, labels from an allow-list, the blast radius of the files the issue names, a command that reproduces it. Done by: Remediation Steward (triage) ·
scope. Evidence: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/work_items/1 · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/blob/v2/AGENTS.md. - Create. Draft merge requests that fix real defects, each with failing-then-passing evidence from an oracle check or a new test. Never the method, a health text or the pipeline. Done by: Remediation Steward (fix) ·
agent-scope. Evidence: (pending: not recorded from GitLab yet). - Verify. Format, lint, strict types, unit tests, properties over every situation, the vignettes, the pinned HL7 validator, the end-to-end harness with axe. The Safety Case explains the oracle's verdict on every change to public-health meaning. GitLab Duo Code Review has its review instructions in the repository, but has not reviewed a merge request here yet. Done by: Safety Case ·
format·lint·types·unit·oracle·properties·advice-guard·fhir·e2e. Evidence: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/merge_requests/3 · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/merge_requests/4. - Package. On every merge that changes what ships, the app built twice to the same bytes, a CycloneDX bill of materials and in-toto provenance; on a release tag, the files packaged in the Generic Package Registry. Done by:
build·sbom·provenance·package. Evidence: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/pipelines/2986274667 · (pending: not recorded from GitLab yet). - Secure. Secret Detection on every merge request and SAST on every one that changes code, with a job that fails the pipeline on critical and high findings; a dependency audit; the agents' prompt-injection and regression evals, with the unit tests; the threat model. Done by:
semgrep-sast·secret_detection·security-gate·deps-audit. Evidence: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/pipelines/2976110078 · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/blob/v2/test/agents/evals.test.js · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/blob/v2/docs/threat-model.md. - Release. Only from a protected tag a person pushes: every gate again, checksums signed keyless with cosign, a release with the bill of materials and the provenance attached. French and Malagasy release notes drafted by the Sentinel for a native reviewer. Done by: Release & Monitor Sentinel (release notes) ·
sign·release. Evidence: (pending: not recorded from GitLab yet). - Configure. The pipeline and the agent configuration linted against GitLab's own schemas; the production environment on GitLab Pages; Code Owners with a second approver in every governing section, and review instructions, kept in the repository. Done by:
ci-lint·agents-lint·deploy-pages. Evidence: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/blob/v2/.gitlab/duo/agent-config.yml · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/blob/v2/.gitlab/CODEOWNERS · https://ranoka-b233a5.gitlab.io. - Monitor. A smoke test of the live site after each deploy and on a schedule: file digests, offline reload, no CSP violation, no outside request, axe. On failure, an automatic rollback to the last good build and an incident the Sentinel analyses; an announced drill proves the loop. Done by: Release & Monitor Sentinel (incident) ·
smoke·monitor-smoke·rollback·verify-rollback·incident·drill-report. Evidence: (pending: not recorded from GitLab yet) · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/pipelines/2985943440. - Govern. A default branch that cannot be force-pushed (any Developer of the project may still push and merge to it: see the limits); a hazard log in which a second person, a code owner, decides any change that loses protection, and a guard that turns the pipeline red until then (GitLab does not stop a merge on a red pipeline here, since that is a Maintainer setting, so people must not merge on red); after the merge, an audit that checks through GitLab's API that the person the decision names approved, once a Maintainer sets its read token (until then it fails and reports the decision as self-attested); agent commits kept off human-only paths; the submission texts checked against the repository; a ledger, read by code from GitLab's API, of the flow runs that reached a model on agent-labelled merge requests and issues, and of the human decisions on the flows' work. Done by:
advice-guard·agent-scope·approval-audit·submission. Evidence: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/merge_requests/3 · (pending: not recorded from GitLab yet) · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/blob/v2/docs/safety/hazard-log.yaml · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/blob/v2/.gitlab/CODEOWNERS · https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43307237/ranoka/-/blob/v2/docs/agents/LEDGER.md.
The Secure stage's pipeline belongs to a merge request that was merged before that pipeline finished: its security gate passed only after that merge request was merged (see "Limits").
How we built it
- The method is data.
method/ranoka-method.json(method 2.1.0, 18 questions) drives the scorer, the FHIR Questionnaire and terminology, and the documentation. The oracle (node tools/ranoka.mjs) enumerates every situation, checks 7 properties over the answer space, replays 26 vignettes (field scenarios whose expected results we wrote from the method's rules; 0 signed by a public-health reviewer so far) and pins the result in a golden matrix (2badd1f7ac79). - Flows built for distrust. Each flow is Flow Registry v1 YAML with the same shape: a deterministic scope gate (out of scope, the flow ends before any model runs, so no credits are spent), deterministic evidence, a reader that sees untrusted text but can write nothing, a writer that sees only the reader's brief and the facts and holds no tool, and a deterministic publisher. GitLab's executor reports a command that fails as a success, so each decision is gated by listing a marker that only a passing step writes. Across the flows that is 27 deterministic steps and 7 model components. The steps run the default branch's copy of the tools, so a merge request cannot change the checks that judge it.
- GitLab's own sources, not memory. Every keyword, schema field, trigger type and tier we rely on was read in GitLab's repositories at a pinned commit; 9 JSON schemas are vendored byte for byte and our agent lint validates the flows against them. What we could not confirm is listed with a fallback.
- A pipeline that is the same command locally. Every gate in CI is an
npm runornode tools/…command a contributor runs on a laptop; the CI configuration is checked against GitLab's CI schema before GitLab sees it. - Standards all the way down. HL7 FHIR R4 export with profiles and invariants generated from the method, checked offline by the pinned HL7 validator 6.10.4 on a corpus of 40 Bundles, while 59 tampered Bundles must be rejected. WCAG 2.2 AA with axe on every screen and 60 contrast pairs checked from the design tokens. A strict Content-Security-Policy with no network access at all, zero runtime dependencies, a reproducible build within 97 KB gzip.
What was added during the hackathon. The project existed before: the first Ranoka was built for
the OneAquaHealth IEEE Global Hackathon 2026, and commit 9ad52f6 of 2026-10-04, tagged
byo-baseline-2026-10-04, is the project as it stood before this hackathon.
Every later commit is dated on or after 2026-10-05, the first day of the hackathon; the
first one is from 2026-10-13.
Everything about agents and the lifecycle is new: the oracle and its golden matrix, the
three flows and their tools, the skills, the review instructions, the pipeline, the
approval audit, the ledger, and this text's generator. So are method 2.1.0, the FHIR
implementation guide 2.1.0 and the rebuilt app they protect.
Challenges we ran into
- A justified change and a regression look the same in a diff. The oracle does not judge; it measures every situation and names what lost protection. A person decides, in a hazard log that cites the exact outcome digest.
- A second person, without the settings that would force one. Required approvals, Code
Owner approval and "Pipelines must succeed" are Maintainer settings, and both of us are
Developers; whether GitLab lets authors approve their own merge requests here is a setting we
cannot even read. So every governing section of our Code Owners names a second person, every
merge request merged so far was approved by that person and not by its author, and
approval-auditis built to check after each merge, through GitLab's API, that the person the hazard-log decision names approved. On the default branch it runs without the read token it needs, so there it reports the decision as self-attested. - Only people can fire triggers. Agents' merge requests are therefore Drafts, and the person who marks one ready starts the Safety Case. The platform's rule became our Supervised loop.
- A failed command is not a failed step. GitLab's flow executor reports a command that exits with an error as a success, so a gate built on exit codes would never stop a flow. Every decision now writes a marker only when it passes, and a listing of that marker is the gate; a model of the executor replays each flow in our lint with every gate refusing.
- Prompt injection. No component both reads untrusted text and can act outside its sandbox. 15 evals (injection in an MR description, in a translated string's comment, an invented figure in a French summary, a scorer that tries to read the flow's token, and regressions of contrast, privacy, CSP and FHIR) run with the unit tests in CI, with deterministic expectations.
- No GitLab tool restricts the paths a commit touches. A CI job,
agent-scope, fails any agent-authored commit that touches a human-only path or decides in the hazard log, judged by the policy of the branch it merges into, never by the one it brings. - Scarce credits and minutes. Every flow exits before any model when nothing governed changed, and each has a cap: 2 Safety Case runs per merge request, 8 agent Draft MRs per day, 3 Sentinel incidents per day.
- Designing for a platform we could not try yet. 24 platform facts remain unverified; each has a first-day check and a fallback, and the deterministic gates still run if no flow runs.
- Not entering Hands-off. "No human in the middle" conflicts with our rule that no agent ships a change to public-health meaning without a named person.
Accomplishments that we're proud of
- An oracle that shows a person exactly what a change costs on every situation rather than a sample, so that a person can tell a justified correction from a disguised regression, and an agent that explains it without being able to change a number.
- MR-B: every example-based test green, caught anyway, with the smallest counter-example a person can read.
- Notes that cannot lie about numbers: a model's draft with one figure the oracle did not produce is withdrawn, and the oracle's own table is posted instead.
- 21 of 21 project invariants enforced by a command
today, each named in
AGENTS.mdnext to the command that checks it. - This text: every number in it is generated from the repository, and the pipeline fails if the text and the repository disagree.
What we learned
- Deterministic oracles are what make agents trustworthy; the model's job shrinks to explaining what a tool measured, and that is a job it does well.
- Unit tests pin examples. For a change to public-health rules, the safety net has to be exhaustive, and a person has to sign for what it finds.
- The platform's constraints (human-only triggers, the default branch's agent configuration, composite identity) are good security design when you build with them instead of around them.
- Reading GitLab's source at a pinned commit is faster than debugging an assumption.
What's next for Ranoka
- Native review of the French and Malagasy health texts, and a public-health reviewer among the code owners.
- A field pilot with paired laboratory samples to calibrate the indicative index; the 6 method proposals already written wait for the method owner.
- Publish the flows in the AI Catalog for other small public-health and environmental projects.
- Map the FHIR export to the OneAquaHealth implementation guide once it is published.
Limits
- French and Malagasy are machine drafts: 0 French and 0 Malagasy strings out of 573 are reviewed by a native speaker so far.
- Every merge request merged so far was approved by the second code owner, who is not its author; GitLab did not require it. The public-health reviewer the method calls for is not recruited yet; 4 of 4 hazard-log entries have a decision.
- GitLab enforced none of our review process, except that a new push removes earlier approvals:
both of us are Developers, and required approvals, Code Owner approval, "Pipelines must
succeed", the branch protection's rules and the audit's read token are Maintainer settings. The
default branch is protected only against force-pushes and deletion: any Developer of the
project, the hackathon group's members and the flows' service accounts included, may push and
merge to it directly, and which tags are protected a Developer cannot read. The gates turn a
pipeline red; people keep the rest, and it slipped: the commits that brought up CI were pushed
straight to the default branch before any merge request existed, with no review; the method
freeze was merged before any decision on the changes already in force existed (the code owner
decided them hours later, in a note on that merge request, and they were recorded in their own
merge request); the method freeze and the fixes to the smoke test and to the agents' proxy were
merged while their merge request pipelines still had blocking manual jobs nobody had run
(
properties, and for the freeze and the smoke-test fixfhirande2etoo), which passed on the default branch after each merge; MR-A was approved early once and the approval withdrawn, closed by mistake and reopened, and in the end merged while its pipeline was still running (its guard passed on the same digest minutes later, and every job of that pipeline passed); andapproval-auditruns on the default branch without its read token, so there a hazard-log decision is self-attested. - No rollback has succeeded on GitLab yet: the first deploys after a merge failed their smoke test, and the rollback had nothing to restore, since no build had been promoted yet. Once the smoke test was fixed, in its own merge request, the next deploys passed it and were promoted; the announced drill is what will show a rollback.
- Synthetic data only: no field pilot yet, and the index is not calibrated against laboratory samples.
Built With
- agent-skills
- agents.md
- ajv
- axe-core
- code-owners
- custom-flows-(flow-registry-v1)
- cyclonedx
- esbuild
- eslint
- generic-package-registry
- gitlab-ci/cd
- gitlab-duo-agent-platform
- gitlab-pages
- hl7-fhir-r4
- hl7-fhir-validator
- in-toto
- javascript-(es-modules-jsdoc-typescript-checks)
- json-schema
- node.js
- playwright
- prettier
- sigstore-cosign
- sushi