Inspiration
A laparoscopic cholecystectomy goes wrong in a way that is hard to see coming. The surgeon mistakes the common bile duct for the cystic duct, clips it, divides it, and the injury is done before anyone knows. The defence the field settled on is the Critical View of Safety: before you clip or divide anything, you confirm the anatomy. It is a checkpoint, and the whole point of a checkpoint is that you do not skip it when you are busy or sure of yourself.
We started Hemowatch as a retrospective tool that watches laparoscopic video, measures how much of the field is covered in blood, and flags when a safety checkpoint belongs before an irreversible step. Classical computer vision does the measuring. A language model reads the measurements and writes the explanation.
Partway through we noticed the more interesting problem was not the measuring. It was the silence. A tool like this fails safely only if it refuses to say "all clear" when it cannot actually see. A fogged lens, a camera halfway out of the body, an open procedure that was never laparoscopic in the first place: in every one of those cases the honest answer is "I can't assess this," and the dangerous answer is a confident "nothing to raise." So the question that shaped the project was: once the code is written, who stops a well-meaning change from quietly turning "I can't tell" into "it's fine"? That is the "Life After Code" part, and it is what the GitLab Duo automation here is built to catch.
What it does
Hemowatch is two things that defend the same rule.
The application reads a laparoscopic clip and measures it. OpenCV segments the share of the visible field covered in pooled blood, tracks how fast that share is changing, estimates a bleeding onset, recovers a rough scale from instrument shafts, and runs a set of refusal gates: out of focus, lens fogged, occluded, exposure clipped, out of domain, no scale reference. A Claude reasoning layer then reads a digest of those numbers and decides whether to raise a Critical-View-of-Safety checkpoint before the irreversible clip-and-divide step, and explains every refusal in plain words. The console ships with five real cases. One is a laparoscopic cholecystectomy where the agent raises a checkpoint before dissection of the cystic duct and artery. One is an open cholecystectomy that the engine refuses as out-of-domain on 45% of its frames, so the verdict is cannot_assess, not a clean bill. The others cover a suture clip refused as out-of-focus, a pancreatic dissection, and a bleeding ulcer.
The verdict is never the model's to give. It lives in about a hundred lines of plain Python (backend/oracle.py) with one invariant: when the evidence is absent, refused, unmeasurable or untrustworthy, the answer is cannot_assess, and cannot_assess is not safe. The model may be more cautious than the oracle and raise a checkpoint the numbers alone would not. It is never allowed to be less cautious. A held checkpoint is resolved only by a named person who gives a reason, never by the model, the system, or a timer.
The automation is the Hemowatch Safety Auditor, a custom GitLab Duo flow that guards that invariant on every merge request. Mention it on an MR and it reads the diff against the oracle rules. If a refusal gate was loosened, a threshold relaxed, a safety test deleted, or a path opened where an unmeasurable case could return do_not_raise, it hands that single finding to a generator subagent, which writes one pytest that fails on the defective code and passes on a correct version. A scribe subagent posts a single safety report on the MR: the verdict line (PATIENT-SAFETY RISK: BLOCKS MERGE when the invariant can break), a table of findings with file and line, and the exposing tests as suggestions. Then it stops. It never commits, never merges, never approves. A clinician or maintainer decides.
The demo that shows it working is merge request !3, on branch feature/relax-refusal. The change there looks like a reasonable cleanup. It is not.
How the demo defect works
The planted change edits backend/oracle.py in two places and nothing else of substance:
# checkpoint_decision(), before:
if refused or not measurable or not trustworthy or peak_blood_pct is None:
return CANNOT_ASSESS
# after ("a refusal is just a missing reading; no need to hold back"):
if peak_blood_pct is None:
return CANNOT_ASSESS
# reconcile(), the guard it deletes:
if model_decision == DO_NOT_RAISE and (refused or not measurable or not trustworthy):
return CANNOT_ASSESS
Read together, the two edits remove the one thing that kept doubt from being read as safety. After the change, a refused or unmeasurable or untrustworthy field no longer forces cannot_assess, and a model that answers do_not_raise under exactly those conditions is no longer overridden. The open-surgery clip that should come back cannot_assess can now come back do_not_raise. The model is allowed to stay silent under doubt, which is the failure the whole system exists to prevent. The commit message calls it "stop holding back on refused or unreadable fields," which is how this kind of regression usually arrives: described as a fix.
The Safety Auditor catches it, names both edits by line, generates a pytest that asserts refused evidence never yields do_not_raise, marks it PATIENT-SAFETY RISK: BLOCKS MERGE, and pauses.
How it works (architecture)
The vision is classical OpenCV 5, not a neural net doing the measurement. Blood is a colour decision in a chroma-per-lightness space, tuned for precision on hand-labelled real frames, with specular highlights masked out so the laparoscope's own light does not erode a pool. A small YOLOX-tiny model assists only with spotting instrument shafts for scale; if the scale wanders too much across a case, the volume figure is withheld and the field-fraction stands. Every gate has its operating point written next to it in config.py with the sweep that set it, not a number tuned until a demo looked good.
The reasoning layer never sees a pixel. It is handed a digest: the measured series one row per second, the refusal spans, the phase track, and the validation figures for the segmentation so it can weigh the numbers by how much they deserve trust. It returns a decision, a per-refusal explanation, and a surgeon-facing summary. Every number it cites is checked back against the digest; a citation that does not match what OpenCV measured is flagged on screen rather than hidden. The boundary is strict on purpose: OpenCV measures, Python decides, the model explains.
On the GitLab side, .gitlab-ci.yml is a full post-code pipeline, not just a build. On a merge request it runs a safety-test pytest job, an oracle-coverage job that fails under 90% coverage on the oracle specifically, and a Playwright ui-test that drives the running app. On the default branch it goes the rest of the way: a container-build that pushes the image to the GitLab Container Registry, a release job that cuts a tagged GitLab Release, a deploy-cloudrun job that deploys the service to Google Cloud Run, and a verify-smoke job that smoke-tests the live URL after the deploy. On a tag, a catalog-sync job publishes the flow and agent to the GitLab AI Catalog. The flow (AI Catalog item 1016272, "Hemowatch Safety Auditor") and the agent (item 1016271, "Hemowatch Auditor") are defined as YAML in the repo and versioned there. The app is a FastAPI service with a vanilla-JS console, containerised and served on Google Cloud Run (us-central1), which scales to zero between requests. The five demo cases are cached, so the live app boots and replays real results without any API key; set ANTHROPIC_API_KEY to run a clip live.
How we built it
We built the measurement engine first and spent most of that time learning where it should refuse. Real laparoscopic footage is unkind: the lens fogs, the camera leaves the body, an "open cholecystectomy" clip turns out to show gloved hands and drapes. Each of those became a gate with a threshold set by a sweep and recorded. Only once the engine knew how to say "I can't tell" did the oracle and the reasoning layer go on top, with the invariant pulled out into its own file so it could be tested and guarded on its own.
The Duo flow came last and is deliberately small: an auditor supervisor with a generator and a scribe, a strict system prompt pointed at the exact files where the invariant lives, and a hard instruction to stop and wait for a human. We planted the relax-refusal defect to prove the flow earns its place, then wrote the oracle's own safety tests so the regression fails in CI too, not only in the agent's report.
Challenges we ran into
Getting a model to expose a defect instead of papering over it took work. A generator asked for a test will happily write one that passes on the broken code, which is worse than useless. The prompt now forces it to assert the safe behaviour and to check, by reading the code, that its assertion actually fails on the defect.
The honest-limits problem was the other one. The blood segmentation has high precision and low recall by design (around 64% precision, 8% recall on held-out real clips), the volume estimate was never validated against a real volume, and the onset detector has not fired on any of sixteen real clips because the camera is rarely still long enough. We decided to surface every one of those on screen rather than quietly round them off, because a tool that overstates what it measured is exactly the kind that lulls a surgeon into trusting a silence.
Accomplishments we're proud of
A single rule, written once in plain Python, that the application obeys at runtime and a GitLab Duo flow defends at merge time. The failure mode we most wanted to prevent, a confident "nothing to raise" when the system cannot actually see, now has to get past the oracle, the oracle's tests in CI, and an auditor that reads every change to those files and blocks the merge. The open-surgery case coming back cannot_assess instead of a clean reading is the whole thesis in one clip.
What we learned
Refusing well is harder than measuring well, and more important. Most of the engineering here went into the cases where the right answer is "I don't know," because those are the cases where a tool does real harm if it guesses. We also learned that a supervised agent is most useful when its job is narrow and its authority is capped: the Safety Auditor is valuable precisely because it cannot merge, cannot approve, and cannot resolve a checkpoint. It reads, it exposes, it reports, it stops.
What's next
Run the auditor on a branch-protection schedule so the invariant is checked on every MR without a mention. Widen the oracle's named-human resolution path into a full audit trail of who resolved which checkpoint and why. And validate the volume estimate against a phantom with a known blood quantity, so the one figure we currently refuse to trust can earn its place on the field.
Log in or sign up for Devpost to join the conversation.