Safeview

• 

• 

• 

• 

• 

• 

Inspiration

In a laparoscopic cholecystectomy, the moment that decides whether the patient goes home fine or comes back with a severed bile duct is a single clip. Surgeons have a discipline for earning that clip: the Critical View of Safety. You expose three things before you cut, and if you cannot see all three, you do not clip. It is a rule precisely because the eye under time pressure wants to say "close enough."

We wanted to build the thing that says "not yet" out loud. A gate that watches the operation and answers one question before the clip goes on: was the critical view actually established? Clear, or hold.

Then the "Life After Code" part clicked into place. A safety gate is only as good as the rule inside it, and rules like that erode quietly, in a refactor six weeks later that looks harmless in a diff. Nobody sets out to weaken a patient-safety check. It happens when an "audit second opinion" feels symmetric and someone lets it vote both ways. So the gate needed a second product next to it: an agent living in the merge request whose whole job is to notice when a change makes the gate lie.

What it does

Safeview is two things that share one invariant: an application that runs the gate, and a GitLab Duo automation that defends the gate's code on every merge request.

The application — the clip gate

Safeview replays real gallbladder operations from the SAGES CVS Challenge 2024 and Endoscapes2023 datasets and runs the gate over them frame by frame. The pipeline is a short, honest chain:

video → per-frame vision scoring → calibrated yes/no per criterion → Claude's per-frame call → oracle verdict

Two image-to-text models read every frame against a small set of surgeon-labelled reference frames and score three questions, each 0 to 10. A three-frame window and a per-criterion logistic regression (fitted only on training footage) turn those scores into a yes or no for each of the three criteria:

• C1 — the cystic duct and cystic artery are both isolated, each seen as its own tube entering the gallbladder.
• C2 — the hepatocystic triangle is cleared of fat and fibrous tissue.
• C3 — the cystic plate is exposed, with the lower gallbladder lifted off the liver.

Claude then makes the calls a surgeon would actually hear in the room: each frame gets a state per criterion, and an action — stay silent (the default), log a change, or speak up when an instrument is at a duct and the view is not complete. The notes are short and concrete, like "Two of three. Cystic plate not exposed."

The one thing Claude does not do is decide the verdict. That is the point. The clear-or-hold decision is made by a few lines of plain Python in oracle.py: the gate is clear only when all three criteria read exactly "met" on the final frame, and anything ambiguous — partial, missing, a weak yes, a typo — is a hold. A second Claude pass can audit the gate's log and offer a debrief, but it can only pull a clear down to a hold. It can never push a hold up to a clear.

The console shows all of this. You pick a case, watch the clip, and read the gate: each criterion side by side with what the dataset's surgeons said and what Safeview said. A ledger underneath shows every frame's action and the moment the gate would have spoken. It runs entirely on saved readings and needs no API key; a "Run it" button re-decides the final frame with a live Claude call for anyone who wants to see the model work in real time.

The automation — the Safeview Safety Auditor

The gate's rule lives in code, so it can be broken by code. The Safeview Safety Auditor is a custom GitLab Duo flow that reviews every merge request touching the gate and refuses to let that happen quietly.

It is three agents working in sequence. An auditor reads the merge request and every changed file under backend/ and tests/, looking for specific ways the invariant can be defeated: a gate that can clear with fewer than three criteria met, an audit path that can turn a hold into a clear, a verdict read from the model's own output instead of the oracle, a safety test deleted or loosened. For each finding it hands a single defect to a generator, which writes a pytest function that fails on the broken code and passes on a correct one — a test that exposes the bug, not one that rubber-stamps it. A scribe posts one merge-request comment: a verdict line, a table of findings with the exact invariant clause each one breaks and the file and line, and the generated tests as suggestions. If any guard is weakened, the first line reads PATIENT-SAFETY RISK: BLOCKS MERGE.

Then it stops. The auditor never commits, never edits a file, never merges, never approves. It pauses and waits for a clinician or a maintainer to decide. That is the assisted model, on purpose: the agent does the reading, the reasoning and the proof, and a human owns the call.

The demo is merge request !3, on the branch feature/audit-reconcile. The commit there is titled "Let the audit's verdict stand in reconcile," and it looks like a reasonable simplification. It rewrites reconcile() so the audit's verdict is returned directly whenever the audit says clear or hold. The effect is that reconcile("hold", "clear") now returns "clear" — an audit second opinion can upgrade a hold into a clear, which is exactly the direction the rule forbids. In the operating room, that is a surgeon being told "clear to clip" over a view that was never established. The auditor catches it, names the clause it breaks, and writes the test that proves it.

How it works (architecture)

The application is a FastAPI service on a single port. It serves a routed single-page console in vanilla JavaScript and a small case API: a list of cases, one case with its frames and gate, and a run endpoint for the one live decision. GET /healthz returns {"ok": true}.

Under the API, eyes.py runs the two vision models and caches every frame's scores. verifier.py ensembles those scores over a three-frame window and applies the fitted per-criterion regressions from calibration.json. agent.py asks Claude for the per-frame states, actions and notes, then calls oracle.py to settle the verdict and applies the audit's downgrade if there is one. Everything the console shows is cached, so the whole thing runs with no key; only the live button spends tokens.

oracle.py is deliberately tiny and dependency-free, because it is the part that must never be wrong. tests/test_safety.py pins its behaviour from every angle: all three met is the only clear, every not-met-like value is a hold, a missing criterion is a hold, a model claiming "clear" does not make it clear, and the audit can only downgrade.

The automation is defined in the repo as flows/safeview.yml (the three-agent flow) and agents/safeview-auditor.yml (a read-only reviewer agent). A catalog-sync CI component publishes both to the GitLab AI Catalog on a version tag. The flow runs on GitLab's managed Anthropic Claude Sonnet, served on Google Vertex AI.

GitLab CI carries the safety story end to end. A safety-test job runs the full pytest suite on every merge request. An oracle-coverage job runs the oracle tests under coverage with a 90% floor. A ui-test job boots the app and drives it in a real Playwright browser. A Kaniko docker-build job builds the image on the default branch. The container is deployed to AWS App Runner, which is the live URL above.

How we built it

We built Safeview fresh, starting from the rule and working outward. oracle.py and its safety tests came first, before any model or any pixels, so the invariant existed as code and as red-green tests before anything could depend on it. Then the vision layer: we tried three image-to-text models and dropped the slowest (Kimi-K2.6 was around 70 seconds a call and kept timing out), kept two, and ensembled them. We calibrated per-criterion regressions on held-out training footage so a yes means a yes at a sensible threshold rather than at the model's whim.

The console came next, built to be read rather than admired: criterion, what the surgeons said, what we said, in three columns, with the frame that justifies each call one click away. Finally the Duo automation, and the planted defect in MR !3 that gives the auditor something real to catch. We seeded that defect as a plausible refactor rather than an obvious mistake, because an auditor that only catches obvious mistakes is not worth having.

Challenges we ran into

The hard problem was keeping the model out of the verdict. It is genuinely tempting to let Claude return a clear-or-hold decision — it is good at it — but then the safety guarantee lives in a prompt, and prompts drift. Drawing a hard line, where the model reads frames and narrates and the oracle alone decides, took discipline and a lot of tests that assert the model cannot override the rule even when it says "clear" in its own output.

Calibration was the other one. Real operations rarely look like a textbook, and C2 in particular sits on a boundary a lot of the time. We leaned on reference frames where two of three surgeons agreed, labelled "only just," so the models could learn where the line actually falls instead of only seeing pristine examples.

Getting the auditor to write exposing tests rather than passing ones took several tries. The useful instruction turned out to be explicit: assert the safe behaviour, then confirm by reading the code that the assertion would fail on the defect. A test that passes on a bug is worse than no test, because it looks like coverage.

Accomplishments we're proud of

A patient-safety rule that a language model cannot quietly override, and an agent that cannot quietly let someone weaken it. The gate clears only on all three criteria, the oracle owns that decision in plain Python, and the Safeview Safety Auditor sits on every merge request that touches it — reading the diff, naming the exact clause a change breaks, and writing the test that proves the break, before pausing for a human. On MR !3 it catches a one-line reconcile change that would have let an audit upgrade a hold to a clear.

What we learned

The safest place for a safety rule is a small, boring, well-tested function that nothing clever is allowed near. Everything interesting — the vision models, the calibration, Claude's narration — can stay interesting precisely because none of it decides the verdict. And "assisted" is not a weaker kind of autonomy here; it is the right one. For a clip-gate rule, the valuable thing an agent does is catch the regression and prove it, then hand a clinician a decision they can actually make.

What's next

Running the gate on live endoscopic video instead of replayed datasets, with the per-frame latency budget that implies. Widening the auditor's scope so it reviews any change to the calibration and the reference frames, not only the oracle and the agent, since a quietly re-fitted threshold can weaken the gate as surely as a logic change. And a second clinical procedure with its own critical view, to show the same shape — rule in code, model out of the verdict, agent on the merge request — holds up beyond the gallbladder.

Built With

• amazon-web-services
• anthropic-claude-(sonnet)-on-google-vertex-ai
• app
• docker
• ensemble
• fastapi
• gitlab-ai-catalog
• gitlab-ci/cd
• gitlab-duo-agent-platform
• glm-5.3-flash)
• image-to-text
• numpy
• playwright
• pytest
• python-3.12
• readings
• two
• vanilla-js-single-page-console