Lot Watch

A watchtower over surgical-device safety reports: it reads FDA stapler and staple data across both product codes, surfaces what summary reports hid, and ranks what to act on.

Comment

Inspiration

Between 2011 and 2018 the FDA counted 109,997 reports about surgical staplers and staples. Among them were 412 deaths, 11,181 serious injuries and 98,404 malfunctions. (FDA Executive Summary, General and Plastic Surgery Devices Panel, 30 May 2019.)

Then the same document says something that matters more than any of those numbers. 56,277 of those reports, 51%, were filed as Alternative Summary Reports, a program that kept them out of the public MAUDE database. For the years before 2017, FDA says those reports "were not made publicly available."

The search itself also hides things. Staplers have two product codes, GAG for the stapler and GDW for the staple. Of the 412 deaths, 404 were filed under the staple code and 8 under the stapler code. Someone who searches for "Stapler, Surgical" and counts what comes back finds eight.

This is not only a theoretical problem. In Hershberger v. Ethicon Endo-Surgery, Inc. (S.D. W. Va., 277 F.R.D. 299), a circular stapler cut tissue but fired no staples during a colostomy reversal. The maker told the plaintiffs it had "no documents" about similar incidents. Its own complaint database held "would not staple" reports that the magistrate judge found "very similar [to] identical". They had been coded differently, and nobody had searched them. The court sanctioned Ethicon, $53,350.36.

I want to be careful about what that case shows. The sanction was for discovery, not a verdict. I read the stapler cases I could find, and none of the patient verdicts that I found survived appeal (a $320,000 jury award in Gillies and a $538,281.73 award in Meyer were both reversed). Lot Watch makes no claim about who is liable. The point is narrower and easier to check: the information was in a database, and the way it was coded kept anyone from finding it, including the company that held it.

FDA acted. It reclassified surgical staplers from class I to class II with special controls in 2021 (86 FR 56195). But the failure mode of the search has not changed. A death can sit under an instrument code while the recalled reload it involved is named only in the free text.

What it does

Lot Watch reads the narratives of FDA stapler reports, not only their fields, and shows what a field search misses.

The demo opens on a board of findings. The first one is the case I built it around:

  • The death a field search walks past. Searching MAUDE for the recalled Intuitive SureForm 30 gray reload (48230M) returns 24 reports and no deaths. Report 2955842-2026-01596, received 16 Jan 2026, is a death. It is filed under the stapler instrument, so the reload appears only in the narrative ("PRODUCT NUMBER 48230M, GRAY RELOAD"). The surgeon describes firing on the pulmonary artery with no error messages, and "no staples were applied to the PA... massive bleeding." That report arrived about eight weeks before Intuitive's 11 Mar 2026 recall. Whether it is the death FDA cites in the recall is not verified, so the screen says it is consistent with the recall and no more.
  • Lot match. 89 reports on the Ethicon Echelon VASECR35 reload name 28 of the 57 lots FDA later listed. 14 of those arrived before Ethicon's 22 Apr 2025 letter, including a death report on lot 326D66 received 10 Apr 2025.
  • Failure modes from 300 narratives. Nemotron Nano reads 300 reports on the circular stapler CDH29A and sorts them into failure modes (incomplete firing, uncut washer, leak), the same family as the 2019 Class I recall.
  • An honest negative. Lockout-type narratives for VASECR35 ran at 0 to 3 a year before 2025, then 64 in 2025. By volume alone Lot Watch would not have flagged it early, and the board says so.
  • "Reads like this" search. Pick a narrative and find others that describe the same failure in different words, with different codes.

Every finding carries a "what this cannot show" box. There is no rate anywhere in the app. MAUDE has no denominator (no count of units sold or procedures), so a rate computed from it would be made up.

How I built it

The backend is FastAPI and serves both the API and the built React app from one port. The frontend is React and Vite with AG Grid and AG Charts, adapted from an earlier dashboard of mine. The board is a findings list on the left, a finding with its evidence in the centre, and a data table and small charts beside it. I use only the free Community editions of AG Grid and AG Charts. An earlier version pulled in the commercial trial build, and I removed it.

Everything model-related runs on Nebius Token Factory, through its OpenAI-compatible API:

Tier Model Job
Reader Nemotron 3 Nano (nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B) Classifies each report narrative into a failure mode and pulls out what the form fields drop: the tissue involved, the device or lot named only in the text, the harm, and whether the device was returned. One cheap call per report, so it runs over the whole set.
Writer Nemotron 3 Super (nvidia/nemotron-3-super-120b-a12b) Writes each finding's brief from the numbers already computed in the data. The brief may not state a figure the code did not produce.
Search Qwen3-Embedding-8B Embeds 772 narratives for the "reads like this" search. It is not an NVIDIA model. I first planned a different embedding model that turned out not to be in the Token Factory catalogue.

The data comes from openFDA (CC0): the device event, recall and classification endpoints, filtered to product codes GAG and GDW (and NAY for Intuitive's robotic stapler).

Decisions and gotchas:

  • The counts are code, not model output. Report totals, the by-year chart and the lot matching are computed from the records. The models read text and write prose. I do not let a model produce a number that ends up on screen.
  • Both product codes are searched together, always. The 404-versus-8 split is the reason. A search on one code is the bug I am showing.
  • The result is cached. A single Nemotron pass produced the Nano extractions and the Super briefs, and the app serves those. A scan button re-counts live and runs one Nano read, capped, so the page cannot spend credits on an open URL.
  • Reasoning models return nothing on a small token budget. The budget goes to hidden thinking, and answers can land in reasoning_content. My first calls looked broken until I raised max_tokens. Extraction runs at low reasoning effort.
  • Report counts are not patient counts. Manufacturer reports include follow-ups, so I de-duplicate by report number before counting.

What makes it agentic

Perception Decision Action
A report's narrative names a device or lot that its fields do not the form and the text disagree, so the field search is wrong for this report attach it to the right device and lot, and flag it where the field search shows nothing
A recall lists lots which reports already name those lots, and which came earlier match them, put the earlier ones on a timeline, and open the report that arrived before the letter
A pattern rises on volume whether the volume actually supports an alert say when it does not (the VASECR35 lockouts), instead of presenting it as early warning
A finding is ready to write what the data can and cannot support write a brief that cites report numbers, with a fixed "what this cannot show" section
New reports since the last run whether any change a finding re-read them and mark what is new

The last row is a design decision as much as a capability. The agent's most useful output is often "cannot tell."

Accomplishments

  • A hidden fatal report brought to the top: 24 reports and no deaths by the fields, one death by the text.
  • The 28-of-57 lot match against FDA's recall list, with 14 matching reports earlier than the recall letter.
  • 300 of 300 narratives read and classified, and 772 embedded for search.
  • The 404-versus-8 code split shown as a chart, using FDA's own figures.
  • A board with no rates, no liability claim and no number the code did not compute.
  • Deployed on AWS App Runner and tested end to end on the live URL.

What's next

  • De-duplication I can show. Follow-up reports inflate counts. A visible report-number grouping would let a reader audit it.
  • More devices. The pipeline is not specific to staplers. The same coded-differently gap exists for other implants and reprocessed devices.
  • A scheduled re-pull. A nightly job that fetches new reports and marks "new since last run". Today it is a manual scan.
  • UK data. MHRA reports have not been examined.

Built With

Share this project:

Updates

Submission history