Binder

The regulatory desk a principal investigator doesn't have: agents that draft every US clinical trial submission, cite every claim to the CFR, and never miss a statutory deadline.

Comment

Inspiration

A principal investigator with a good scientific question rarely fails on the science. They fail on the paperwork around it — and then they keep failing on it for the life of the study.

The numbers are worse than most people outside clinical research expect. Study start-up runs about seven months from site identification to site initiation, and can consume 30% of the entire trial timeline. Sites spend an average of $13,901 per study on regulatory compliance and are reimbursed a median of $3,000 — about 22% of the real cost. 54% keep four or more binders per study, each typically over 250 pages. At any given moment, 26% of sites are two weeks or more behind on their compliance tasks.

Two failure modes cost the most. The first is mundane: an incomplete application comes back from the IRB and burns an entire review cycle. The second is worse because it is silent — a registration window closes, an annual report lapses, a consent form drifts out of sync with an amended protocol across three sites, and nobody notices until a monitor or an inspector does.

That is not a document-generation problem. It is a continuous problem, running in the background, over years, across dozens of interlocking rules. It is exactly the shape of work this hackathon asked for, and it is why we built a fleet rather than a chatbot.

What it does

Binder is a governed fleet of specialist agents that share one live model of a study.

A PI uploads a protocol draft, an investigator brochure, a grant, and a staff roster. Within minutes the fleet has:

  1. Extracted a structured Study Profile — phase, article under study, population, funding source, sponsor-investigator status.
  2. Determined which regulatory regimes actually bind this study — IND under 21 CFR 312 or IDE under 812 or neither; Common Rule, FDA rules, or both; whether it is an applicable clinical trial for registration; whether the single-IRB policy applies — each branch returned with the rule text it relied on.
  3. Assembled first drafts of every submission package, with a citation on every regulatory claim.
  4. Attacked its own output with an adversarial reviewer that returns line-item deficiencies before anything is submitted.
  5. Stood up permanent watchers on the study's statutory clocks.

From that point the fleet runs unattended. Amend the protocol and the change-impact agent computes the blast radius across the consent form, the IRB modification, the IND amendment and the registry record, proposing each edit as a reviewable diff. When FDA publishes new guidance, the intelligence agent re-scores every open study against it.

Nine agents, each separately deployed with its own service account and its own narrow tool set: Reader, Applicability, Dossier (fanning out into per-package workers), Consent, Deficiency, Sentinel, Change Impact, Intelligence, and Registrar.

The guardrail, stated first

Binder never files anything. It drafts, checks, cites, and reminds; the PI and the IRB remain the decision-makers, and every outbound package requires an explicit human signature gate.

Every regulatory statement the fleet emits carries a retrieved citation to a specific CFR section. An agent that cannot ground a claim is required to return an open question instead of prose. In a regulated setting that is an ethical requirement, not a feature — but it is also the thing that makes the system usable by a real investigator, who can check any sentence in ten seconds.

How we built it

Models. Gemini 3.5 Flash handles the constant, high-volume work — document extraction, routing, the sentinel's sweeps. The reasoning tier handles the rare and expensive work: applicability determinations, drafting, and adversarial review. Splitting the tiers is itself an architectural argument, not a cost optimization.

Agents. Google ADK, with a registry rather than hard-wired calls, so agents can be added, versioned and revoked without touching the others.

Infrastructure. One Cloud Run service per agent. They never call each other directly — coordination is an orchestrator plus a Pub/Sub event spine (study.created, protocol.amended, clock.due, guidance.changed, package.ready). A failing agent degrades one capability instead of the fleet, and any agent can be replayed against the event log. Firestore holds an event-sourced case file per study; Cloud SQL with pgvector holds the regulatory corpus; Cloud Scheduler drives the sentinel's heartbeat; Cloud Storage with object versioning holds the evidence, append-only.

Bonus models. Gemma runs a de-identification pass so participant-adjacent free text never reaches a hosted model. Veo generates a short plain-language explainer video from the approved consent form — a real accessibility win for participants who will never read eleven pages of legal prose.

The grounding layer, in detail

Everything above depends on retrieval that returns a citation, so we built that first, and it is the part we would defend hardest.

binder-corpus ingests 21 CFR 50, 56, 312 and 45 CFR 46 from the public eCFR API into pgvector. Three decisions carried it:

Chunks are paragraphs, not character windows. A retrieval hit has to name something a human can open. 21 CFR 50.25(a)(1) can be checked in ten seconds; a chunk spanning (a)(1) through (a)(4) cannot be cited at all.

Ancestors travel as context, not as text. (2) A description of any reasonably foreseeable risks is meaningless alone. Its lead-in goes into the embedded text and the grounding block, while the stored body stays exactly what the citation points at — context improves recall without corrupting the quote.

Retrieval is hybrid. Half the real queries an agent issues contain exact tokens — Form 1572, 312.32, single IRB — that dense embeddings blur. Vector and lexical result lists are fused with reciprocal rank fusion, which needs no score calibration between the legs.

Challenges we ran into

Reconstructing CFR paragraph nesting from a flat XML stream. eCFR emits every paragraph as a sibling <P>, so the hierarchy has to be inferred from designators, which nest by character class:

(a)     lowercase letter
  (1)     arabic numeral
    (i)     lowercase roman
      (A)     uppercase letter

The ambiguity is real and common: after (h), a (i) is the letter i and a sibling; inside (1), a (i) is roman one and a child. A naive parser mis-cites the result, which is strictly worse than not answering — a confident wrong citation is the exact failure this product exists to prevent. We solved it with a stack that asks "is this the successor of a level I am already in?" before "is this the first child of a deeper level?", and pinned it with a test that walks a section through (h) → (i) → (i)(1) → (i)(1)(i) → (i)(1)(ii) → (i)(1)(ii)(A) → (j).

Prompt instructions do not hold under load. Telling a model "only cite retrieved regulation" works until it would rather be helpful. So the guardrail became a type, not a sentence: retriever.ground(q) returns a Grounding that either carries citations or is empty, and an empty one hands the caller a ready-made refusal to emit. Then grounding.validate(answer) re-reads the model's own output, extracts every N CFR X.Y(z) it used, and rejects the response before it leaves the service if any is unsupported.

Making re-ingestion mean something. chunk_id hashes the citation, never the content, so a paragraph keeps its identity across amendments and re-ingest is idempotent. That turns a changed content_hash into a signal: it is an amendment, journalled to a history table and drained onto Pub/Sub as guidance.changed. The change feed came out of the schema for free, rather than being bolted on.

Accomplishments we're proud of

  • The parser handles the ambiguous-designator case correctly, which is the difference between a citation you can trust and one you cannot.
  • The whole ingestion pipeline runs fully offline — a deterministic fake embedder and a SQLite store mean the parser, chunker, store, retriever and CLI are all verifiable with no cloud project, no credentials and no cost. Every test in the repository runs in under a second.
  • The refusal path is demoable. We can show the system declining to answer, which is a stranger and better demo than another confident paragraph.
  • Storing as_of_date on every row means a determination made in March can be replayed against the regulation that was in force in March, not today's.

What we learned

The domain punished shortcuts that would have been invisible in a normal RAG build. Chunk boundaries are usually a tuning parameter; here they are the product, because they decide whether output is checkable. Retrieval quality is usually measured on semantic similarity; here the exact-token queries mattered as much as the fuzzy ones. And "the model should cite its sources" turned out to be an architecture decision — a return type and a validator — rather than a line in a system prompt.

We also learned to build the boring layer first. Starting with the corpus rather than the agents meant every agent that came after inherited grounding by construction instead of having it retrofitted.

What's next

  • Widen the corpus to 42 CFR 11 (the ClinicalTrials.gov registration and results clocks), 21 CFR 11, 54 and 812.
  • Rank amendments by materiality. content_hash detects that a paragraph changed, not whether the change matters — that judgment belongs to the Intelligence agent.
  • Multi-site studies, where consent-version drift across sites is the expensive failure.
  • Institutional deployment. The buyer is an academic medical centre's research office, not the individual PI: sites already spend an average of $13,901 per study on this and recover about a fifth of it.

Sources for the figures above

Binder is a drafting and checking aid, not legal advice. Retrieval returning a citation means the text exists and says what it says — not that it has been applied correctly to a study. Every determination goes to the investigator and the IRB for a human decision.

Built With

Share this project:

Updates

Submission history