Scopewatch Intelligence

Inspiration

In the United States, postpartum haemorrhage causes about 11% of maternal deaths and is the leading cause of death on the day a baby is born. California reviews every one of those deaths. Across 2.4 million births, an expert panel judged 63% of haemorrhage deaths highly preventable and found delayed recognition in 72% of them (California Pregnancy-Associated Mortality Review, Obstet Gynecol 2025).

The American College of Obstetricians and Gynecologists names the cause directly: imprecise estimation of actual blood loss is "a leading cause of delayed response to hemorrhage" (Committee Opinion 794).

Measured against haematocrit, visual estimation underestimated the loss in 90% of cases. A fifth of those patients had lost over a litre, and by eye none of them had haemorrhaged at all. Surgery has the same shape: across 548 recorded gallbladder operations, the critical view of safety, the step that prevents bile duct injury, was recorded in only 61% of cases.

In both, the information is already in front of a camera that is already in the room, and a person under time pressure is asked to judge it by eye.

What it does

Scopewatch watches the laparoscopic camera and measures the operating field. Five real surgical clips are bundled, and each one gets:

• Blood-covered field: the share of the visible field that is blood, per frame, with the trace across the whole clip and the surgical phase marked.
• Volume, refused unless the scale holds: millilitres need a scale taken from instrument shafts of known diameter. When the scale is not steady the row reads CANNOT MEASURE and names the reason instead of printing a number.
• Bleeding onset: detected only when the motion gate allows it. When it does not fire, the screen names the gate that held it.
• Refusals: focus, fog, occlusion, exposure, and out-of-domain (the open-surgery clip is refused as a whole case).

Then NVIDIA Nemotron, on Nebius Token Factory, reads those measurements. It decides whether a safety checkpoint belongs before the irreversible step and says why, explains each refusal in plain words, and writes the surgeon summary. Every figure it cites is checked back against the OpenCV series and shown as matches OpenCV, or flagged when it does not. A named surgeon has to answer the checkpoint. Scopewatch blocks nothing; it asks, and it records who decided.

How I built it

This is an honest split of work, and I want it to be clear who does what.

OpenCV 5 does all of the measuring. A pixel counts as blood when it clears three tests together: Lab chroma above an absolute floor, chroma-per-lightness above a multiple of the scene's own median, and a hue inside a band around that median. The three constants were selected by maximising F-0.5 under leave-one-clip-out, so precision is weighted over recall. The instrument scale comes from YOLOX-tiny (Apache-2.0, exported to ONNX and run through OpenCV's cv2.dnn). Nothing here is a language model.

NVIDIA Nemotron 3 Super (on Nebius Token Factory) does the reasoning, over the measured digest and never over the pixels. It receives the per-clip numbers, the refusals and their reasons, and returns a checkpoint decision, a plain-language explanation of each refusal, and a surgeon summary. It cannot measure anything and it never sees the video.

Every figure Nemotron cites is checked back against the OpenCV series. A cited number that matches is shown as "matches OpenCV"; one that does not is flagged on screen. The model's prose is never trusted on its own.

This is the Scopewatch concept from my OpenCV AI Competition entry, rebuilt for this hackathon. The measurement engine is carried over unchanged; the Nemotron reasoning layer, its grounding check, the new UI and the Nebius integration are what was built here. The clips are openly licensed (CC BY) surgical case-report videos, credited in the app. Results are cached so the bundled clips load pre-computed; the live call runs behind the reasoning button, and the API key stays server-side.

What makes it agentic

Accomplishments

Measured on hand-drawn masks over real frames, split by clip so a shared scope, light source and patient cannot leak between halves:

• Precision-weighted by design. A region Scopewatch marks is usually blood, and it under-counts the blood a person drew. I would rather it mark less and be right than mark more and be wrong, so recall is low on purpose and I am not claiming it finds most of the blood.
• 78.2% precision pooled over all 15 laparoscopic clips (23.2% recall). On the strict held-out split: 63.6% precision, 7.7% recall; dev clips 82.2% / 40.1%. A region it marks is usually real blood.
• Refusing is the feature. It refuses volume and withholds onset when the scale and motion gates do not hold, and it names the gate. A tool that prints a number through smoke is worse than one that stops.
• A grounding check on every cited figure. The language model's numbers are verified against the measurement, and a mismatch is shown rather than hidden.
• A clean line between measuring and reasoning: OpenCV measures, Nemotron reasons, and a named human decides.

What's next

• More labelled clips. Each one added roughly 3.5 points of held-out precision in the original work, so the next gain is data rather than modelling.
• Image registration so bleeding onset can fire on a handheld, moving laparoscope.
• Labelled clinical video through the registered datasets (Endoscapes, CholecT50), and a real reviewer study, before anything is said about clinical use.

Built With

• fastapi
• nvidia-nemotron-3-super-on-nebius-token-factory
• onnx
• opencv-5
• python
• yolox-tiny