Threshold

Explain the problem

Finding a cancer clinical trial is hard. Being allowed into one can be harder.

Every trial has eligibility criteria, the rules that decide who can join. Some are scientifically necessary. Many are copied from old protocol templates and shut people out for reasons that have nothing to do with their cancer: HIV that has been controlled for years, a past cancer, treated brain metastases, or not speaking English.

FDA has told sponsors since 2020 that many of these exclusions should be loosened. Protocols have been slow to change. When we read the eligibility criteria of all 1,011 cancer trials recruiting in Massachusetts, we found:

About 1 in 5 drug trials still exclude people with HIV outright, even when it's well controlled.
About 3 in 10 supportive-care and quality-of-life studies require a specific language, usually English. In Suffolk County's environmental-justice neighborhoods, 1 in 7 households has limited English.
Access is concentrated: 945 trials recruit in Suffolk County, 24 in Hampden, and 1 in Berkshire. Restrictive criteria slow enrollment, which delays new treatments, and they decide who gets access at all. Recent research links restrictive criteria to lower minority enrollment. Patients can't see these rules until they're turned away, and the people who write protocols rarely see how their rules compare with everyone else's.

Inspiration

Two of us worked in oncology research and watched patients miss out on trials over criteria that had nothing to do with their disease. The Hackachusetts and DS+X brief put it simply: finding a trial is harder than joining one. We noticed most access tools focus on where trials are. We wanted to look at who is allowed in once a patient gets there, and to fix the problem where it starts: in the protocol.

What it does

Threshold helps patients explore cancer trials and helps researchers see who their eligibility criteria may exclude. It has four tools:

Find Trials. A patient shares their cancer type, age, county, and relevant health information to explore trials that may fit, see which rules could be a barrier, and find study contacts. It's a starting point for a conversation with their care team, not an eligibility decision.
Trial Landscape. Charts and searchable tables show how often 13 categories of restrictions appear across all 1,011 trials, filterable by category, phase, and sponsor. Every number links to the exact sentence it came from.
Interactive County Map. Trial restrictions are shown next to CDC PLACES health data and MassGIS environmental-justice data. A funnel shows how overlapping rules narrow the trials available to an example patient, quoting the sentence behind each step.
Criteria Linter. A protocol writer pastes draft criteria. Claude flags restrictive rules, ranks what to fix first, suggests wording other Massachusetts trials already use, cites the FDA or ASCO–Friends guidance behind each suggestion, and shows how the draft compares with every recruiting Massachusetts cancer trial. Keyword rules take over as a fallback if the model is unavailable.

How we built it

Data: We pulled every recruiting cancer trial with a Massachusetts site from the ClinicalTrials.gov API, mapped each of the 1,537 sites to a county, and joined CDC PLACES county prevalence (diabetes, smoking, obesity) and MassGIS environmental-justice data (limited-English households).
Extraction: Claude (Haiku) reads each trial's criteria with a fixed 13-category schema enforced through tool use. Every flag must quote the trial word for word; any flag whose quote can't be found in the source is dropped. Results are cached per trial.
Evaluation: We hand-labeled 42 trials (a seeded random sample with quotas by phase, sponsor, and cancer type, plus 10 held-out trials drawn after tuning) and scored the extractor per category.
App: A Next.js web app (Landscape, Map, Linter, Find Trials) was deployed on Vercel. The live Linter calls the same prompt as the pipeline through an API route.
Kiro: We explored in Kiro's Vibe sessions and built the extraction engine and a data bugfix through full Kiro specs (requirements, design, tasks), with Hypothesis property-based tests. A root AGENTS.md served as always-on steering, holding our data contracts, ownership rules, and a change log.
Tests: 43 tests, including parity checks that keep the live Linter's prompt and suggestions identical to the pipeline's, and a guard that fails if numbers quoted in our pitch drift from the data.

Challenges we ran into

"Recruiting" doesn't mean every site is recruiting. ClinicalTrials.gov's recruiting filter applies to the whole trial, so closed and not-yet-open Massachusetts sites leaked in, inflating our data to 1,170 trials. A Kiro bugfix spec and property-based tests fixed it (1,011 trials) without breaking county matching.
Conditional vs. outright. "Excludes HIV" and "eligible if HIV is controlled" are opposites for patients, but both mention HIV. Separating them changed our headline from a misleading "a third of trials restrict HIV" to an accurate "1 in 5 drug trials exclude it outright."
The AI over-flagged in specific ways. It counted the standard "18 or older" rule as an age restriction, and a handful of long trials came back empty without an error. Our eval surfaced both, and we fixed them.
Messy text. ClinicalTrials.gov escapes characters (\<, \[), and criteria contain line breaks, so quote matching and row counting needed care.
Defining the categories. Is "adequate organ function" a kidney rule? Is "primary brain tumor" the same as brain metastases? We logged each decision in a labeling guide so the labels and the prompt stayed consistent.
Two codebases, one truth. The Python pipeline and the TypeScript app both carry the prompt and the guidance text, so we wrote tests that fail if they ever disagree.

Accomplishments that we're proud of

An AI we measured, not just demoed: 98% precision and 92% recall on 10 trials the prompt was never tuned on.
Every flag has a receipt: all 3,839 flags quote their trial word for word, so anyone can check one in seconds.
Real findings from real data: 1,011 trials, not a toy sample, with every number traceable to its source.
Both sides of the problem: a patient-facing search and a protocol-facing linter built on the same data.
Honest limits on screen: county estimates are never added together, weak guidance is labeled weak, and categories with too few examples aren't scored.

What we learned

The wording matters as much as the rule. Half the trials that mention HIV already use FDA-style conditional wording. The fix already exists; it just hasn't spread.
Measure your AI before you trust it. Our first numbers looked great until a held-out set and a per-category breakdown showed where the model was wrong.
Discover fast, then lock it down. Vibe sessions were great for exploring; specs and property tests kept the parts that had to be right from drifting.
Policy moves slowly. The key FDA guidance is from 2020, and many protocols still don't follow it. That gap is the opportunity.

What's next for Threshold

Pilot with protocol teams at Massachusetts cancer centers and biotechs, to test the Linter on real drafts before they're locked.
Validate patient matching with clinicians and patient navigators, especially how conditional rules are explained.
Drive-time catchments in place of site counties, so impact reflects where patients actually travel from.
Expand beyond Massachusetts and refresh trial data nightly.
Support more languages in Find Trials, starting with Spanish.
Automate the pipeline with Kiro hooks: clear the cache when the prompt changes, and run the full test suite after every task.