Clinical Trial Matchmaker

Inspiration

Clinical trial recruitment is one of the biggest bottlenecks in modern healthcare research. Many patients who could benefit from experimental therapies never discover the trials they are eligible for, while researchers struggle to recruit participants on time.

What inspired us was the gap between the amount of clinical data available and the lack of intelligent systems capable of reasoning over eligibility criteria in a reliable and explainable way. We wanted to build something beyond a simple RAG pipeline: an AI agent able to retrieve, reason, rank and justify decisions like a clinical assistant would.

The TREC Clinical Trials benchmark was especially interesting because it reflects a real-world problem: matching noisy free-text patient descriptions against highly heterogeneous clinical trial criteria.

What we built

We built an agentic AI system for automated patient–clinical trial matching using:

• LangGraph for orchestration
• ClinicalTrials.gov API v2 as the retrieval backend
• Gemini 3.1 Flash Lite as the reasoning model
• Deterministic scoring and reranking pipelines
• Criterion-level eligibility evaluation

The pipeline consists of seven stages:

1. Patient normalization
2. Multi-query planning
3. Trial retrieval
4. Lexical semantic-aware pre-ranking
5. Criterion-by-criterion eligibility evaluation
6. Deterministic scoring + LLM reranking
7. Dossier generation

Instead of evaluating trials holistically, the system decomposes eligibility into individual inclusion/exclusion criteria and classifies each one as:

• met
• not_met
• NEI (not enough information)

The final score combines eligibility evidence, critical exclusions, uncertainty burden, recruitment status and trial phase:

[ score = 0.80 \cdot met_ratio

• 0.55 \cdot badness_critical

• 0.25 \cdot NEI_burden

• 0.10 \cdot phase_bonus

• 0.10 \cdot recruiting_bonus ]

We also implemented a final LLM cross-encoder reranker that jointly compares top candidates and improves ranking consistency.

Challenges we faced

The hardest challenge was not ranking — it was retrieval.

The TREC benchmark revealed that many relevant trials are indexed under adjacent MeSH concepts rather than the exact disease name used in the patient profile. For example:

• Anaplastic Astrocytoma vs Glioma
• Burkitt Lymphoma vs generic Lymphoma

This meant that naive lexical retrieval missed a large portion of relevant trials.

We experimented with:

• multi-query retrieval
• query expansion
• embedding-based retrieval
• Reciprocal Rank Fusion (RRF)
• semantic-aware lexical ranking

Interestingly, generic embeddings actually hurt performance because biomedical similarity requires domain-specific representations such as PubMedBERT or SapBERT.

Another challenge was operating under strict free-tier constraints:

• 15 RPM
• limited token budgets
• no fine-tuning
• only two days of development

To make experimentation feasible, we implemented a content-addressed caching system for all LLM evaluations, allowing us to iterate on scoring formulas and reranking strategies without recomputing expensive inference steps.

What we learned

This project taught us several important lessons about building reliable AI systems for healthcare applications:

1. Retrieval quality dominates downstream performance

Even strong reranking models cannot recover trials that were never retrieved. The main bottleneck in clinical trial matching is semantic retrieval, not LLM reasoning.

2. Structured reasoning is more robust than free-form prompting

Transforming patient descriptions into a structured PatientProfile dramatically improved evaluation stability and reduced hallucinations.

3. More complexity is not always better

Some of our most complex experiments performed worse:

• excessive query expansion diluted relevance
• generic embeddings reduced ranking quality
• evaluating too many borderline trials confused the reranker

Careful system design mattered more than adding components.

4. Explainability is essential in healthcare AI

Deterministic scoring formulas and criterion-level evidence made the system auditable and interpretable, which is critical for medical applications.

Results

Our final system achieved: | Metric | Score | | Recall@20 | 0.056 | | NDCG@10 | 0.375 |

While Recall@20 remained limited by lexical retrieval, our NDCG@10 reached the range of published BERT reranker baselines without fine-tuning.

The strongest result came from the AVM patient case, where the retrieval stage successfully captured the correct semantic neighborhood and the system achieved:

[ NDCG@10 = 0.726 ]

This validated that the reasoning and reranking architecture works effectively when retrieval succeeds.

Future work

With more time and compute resources, we would focus on:

1. Dense biomedical retrieval using PubMedBERT/SapBERT
2. Fine-tuning the reranker on historical TREC qrels
3. Specialized medical LLMs
4. FHIR/EHR integration
5. Continuous monitoring of trial updates
6. Scaling to large patient cohorts

Ultimately, we believe agentic and explainable AI systems can significantly improve clinical trial recruitment and help connect patients with potentially life-saving therapies faster and more reliably.

Built With

• 3.1
• a
• al
• api
• apis:
• basada
• bases
• biblioteques
• cache
• ciencia
• clau:
• clinicaltrials.gov
• control
• d'ia
• d'usuari
• dades
• dades.
• de
• desenvolupament:
• eines
• emmagatzematge:
• en
• estandard
• fastapi.
• flash
• frameworks
• gemini
• git
• github
• i
• interficie
• l'estructuracio
• langgraph
• lite.
• llenguatge
• llenguatges:
• llibreries
• local
• models
• orquestracio:
• pandas
• per
• pydantic
• python
• sha256
• streamlit.
• v2.
• versions.