SwarmMedic

A service can return the right answer while continuing to call its provider after success. SwarmMedic uses MIT otel-swarm instrumentation to expose that hidden retry storm and pass a bounded repair to GitLab Duo. In the recorded controlled recovery, a separate owner-side publisher and controller carried the repair through merge, exact-image delivery and three live checks without a human mid-loop. Requests fell from sixteen to two and fallback events from eight to one, with the same fixture result. This was not a customer incident or paid-model deployment.

The mechanism closes the gap between observing a problem and proving recovery. OpenTelemetry exposes the repeated work. Assigning a fresh incident starts a native GitLab Duo flow whose toolless agents feed fixed triage and publication steps. Triage verifies the configured incident and source, then reproduces the storm. The repair agent proposes replacement source. Publication checks its syntax tree, runs tests and enforces the HTTP budget before publishing a candidate that changes only service/run.mjs. Model output cannot select arbitrary commands, files, branches or deployment hosts. GitLab's native execution records and exact-source CI give the separate controller evidence it can verify before acting.

On 24 October, one reviewed controller command assigned incident #5 and continued through repair, integration, delivery and closure without an intermediate human decision, correction or restart. Native session 9997833 produced candidate acd10d7b and MR !30. After candidate CI passed, the controller independently tested fallback recovery, normal-primary success and bounded complete failure. Legitimate failure retries remained intact. It required three fresh pre-merge probes at 16 requests and 8 fallbacks, merged only into lab/swarmmedic-delivery-20261024-2, and waited for pipeline 2984012469 on actual merge d2dd6dc11f2042015edc8326e179a674f5741826.

The controller assembled an immutable local Docker image from that exact revision and delivered it to the existing SwarmMedic host. Three live HTTPS probes measured 2 requests and 1 fallback before its summary and automatic incident closure. A separate readback fetched all three reports by their exact IDs and hashes. The dated public recovery record links the native session, executor, MR, candidate and merged CI, deployment identity and sanitized proof bundle. These are real HTTP fixture requests and OpenTelemetry events, with no paid model API or customer outage involved.

This demonstrates deployed Path B Hands-off recovery with explicit limits. The recovery ran under a shared team identity; recorded actions are traceable through the session log, pipeline history and controller receipt linked here. The Duo executor is separate, and native MR publication uses the owner identity. Delivery ran off CI from a reviewed private controller checkout mirrored in the public repository. The isolated delivery target is unprotected and ordinary Developer grants remain. The running console's bundled evidence ledger is historical; the dated record and retained reports supply current proof. Mutation controls are disarmed. The 22 October episode, which required a continuation, is preserved separately from this fresh uninterrupted run.

Attribution here is to the shared GitLab identity and recorded execution, not independent identification of the individual operating that identity. The native session 9997833 requires GitLab sign-in and is not anonymously accessible. Public evidence is available in the executor job log, dated recovery record, candidate pipeline 2984010361 and merged pipeline 2984012469.

SwarmMedic builds on the existing MIT-licensed otel-swarm instrumentation project, retaining its history, license, source and original tests. On or after 5 October 2026, we added the incident console, controlled HTTP service, native Duo recovery automation, controller policy gates, independent checks and deployed recovery path around that library. Imported upstream work is identified separately in UPSTREAM.md. The contribution is a traceable incident-to-recovery workflow whose final check is the running service preserving its result.

The controlled service kept its fixture result while reducing HTTP calls from sixteen to two. Duo repaired the loop; the owner-side publisher and controller carried that repair through merge, delivery and live checks. Codex assisted engineering and evidence assembly, Claude reviewed read-only, and Deepgram narrates the edited retained run evidence. This is fixture recovery with shared-owner and unprotected-target limits, not a customer incident or paid-model savings claim.

Lifecycle stages demonstrated: monitor, release, secure, verify, package.

Built With

• caddy
• docker
• docker-(rootless)
• gitlab-ci/cd
• gitlab-duo-agent-platform
• javascript
• node.js
• opentelemetry
• python