About the Project

Inspiration

Drug repurposing offers the possibility of finding new treatments for existing diseases, but promising hypotheses don't always survive rigorous testing. Scientific literature is often contradictory, mechanisms are uncertain, and a plausible biological explanation doesn't necessarily translate into clinical benefit.

An established drug may have a well-understood safety profile and a plausible mechanism of action, but that doesn't mean it will work for a new disease. Evidence can be scattered across cell studies, animal models, observational research, and clinical trials, with findings that conflict or apply to very different biological contexts. A mechanism that works in a laboratory model may not hold at clinically achievable drug concentrations or translate into meaningful patient outcomes.

These challenges create a difficult prioritization problem for pharmaceutical R&D. Researchers must decide which hypotheses deserve further investment, which uncertainties matter most, and which experiment would provide the most useful information before moving forward. Reviewing more papers isn't necessarily enough; the critical question is whether the existing evidence actually supports the assumptions a hypothesis depends on.

Our question: Can AI identify the weakest assumption in a drug–disease hypothesis and recommend the most useful next experiment to challenge it?

That question inspired FalsifyRx, an AI-powered scientific reasoning tool designed to do more than summarize research. Instead of simply collecting evidence in favor of a hypothesis, FalsifyRx actively searches for contradictory findings, identifies gaps in the evidence, and translates those gaps into actionable experimental recommendations.

Our goal is to help researchers move from "What does the literature say?" to "What would we need to test next to find out if this hypothesis actually holds up?"

What it does

FalsifyRx takes a drug-disease hypothesis and investigates whether the available scientific evidence supports or challenges it.

Retrieves scientific literature: Searches Europe PMC using multiple queries to find relevant studies, supporting evidence, contradictory findings, and mechanistic research.
Challenges assumptions: Identifies the strongest counterarguments, conflicting findings, and unsupported assumptions underlying a hypothesis.
Grounds findings in evidence: Links findings to retrieved papers and verifies supporting quotations against the original abstracts.
Assesses evidence quality: Distinguishes clinical trials from preclinical studies and considers study design rather than simply counting positive and negative papers.
Recommends the next test: Identifies a consequential unresolved question and proposes an experiment designed to distinguish between competing explanations, explaining how different outcomes would affect the hypothesis assessment.

Each investigation concludes with one of three verdicts: Challenged, Not Challenged, or Insufficient Evidence. Importantly, Not Challenged does not mean proven; it means the retrieved evidence has not provided a comparable-design challenge to the hypothesis.

How we built it

FalsifyRx around a retrieval AI pipeline, combining live scientific literature search with structured evidence analysis.

Input: Our website tool prompts the user for a drug, disease, and proposed mechanism.

Literature retrieval: We integrated Claude in the backend to run five queries in parallel, targeting clinical studies, supporting trials, contradictory evidence, mechanisms, and general literature, in places such as Europe PMC. This helps prevent a single search strategy from dominating the evidence pool.
Iterative search refinement: Rather than trusting the first batch of results, the AI reviews retrieved titles, flags potentially irrelevant papers, and generates refined queries when important studies appear to be missing. If no results turn up for the user's query, the AI modifies the prompt for the user and reruns the literature retrieval.
Evidence analysis: From the retrieved literature, the AI evaluates supporting and contradictory findings, distinguishes direct evidence from indirect evidence, and identifies the most consequential unresolved assumption.
Citation and quotation verification: Every finding is associated with a retrieved paper and a supporting quotation. The backend checks that the quotation appears verbatim in the paper's abstract and drops invalid paper references.
Interactive Scientific Knowledge Graph FalsifyRx visualizes the relationships between a drug, its target disease, proposed mechanisms, and the scientific literature in an interactive, clickable knowledge graph. Each node represents a key component of the hypothesis or a relevant research paper, while connections reveal which findings support or challenge the hypothesis. Users can click individual nodes to explore study details, inspect evidence, and trace conclusions back to their sources—making complex, sometimes contradictory scientific literature easier to navigate and understand.
Verdict and Next-test generation: The AI uses the identified evidence gap to propose a feasible experiment, explain why it would be informative, and describe how different outcomes would distinguish competing explanations.

We also built a benchmark using 14 drug–disease hypotheses with established, mixed, or unknown real-world outcomes. The set includes failed clinical trials, successful drug repurposing cases, an unresolved hypothesis, a fictional drug, and four held-out cases.

Challenges we ran into

Finding the right evidence was harder than generating an answer. Initial searches sometimes returned studies about related drugs rather than the exact drug in our hypothesis. For example, our first search for thalidomide surfaced papers about related immunomodulatory drugs. We addressed this by having the AI inspect the retrieved titles and refine its search when key evidence was missing.

Avoiding confirmation bias required searching both sides. A system that retrieves mostly supportive papers can produce a convincing but misleading argument. We designed parallel searches for supporting and contradictory evidence and added rules to ensure that strong null results from randomized trials count as evidence against a hypothesis.

Citation accuracy wasn't enough on its own. A real paper can still be used to support a claim it never made. We therefore added verbatim abstract-quote verification, helping us distinguish evidence-linked claims from unsupported model-generated statements.

Connecting evidence gaps to useful experiments was the central challenge. We didn't want the AI to append a generic experiment to an otherwise conventional literature summary. The proposed next test needed to follow from a specific unresolved question and distinguish between plausible explanations.

Finally, evaluating the system exposed the importance of honest uncertainty. We found that borderline cases can receive different verdicts across runs, reinforcing the need for explicit limitations, reproducible