BioEvidence MCP
Inspiration
Large language models have made biomedical literature much more accessible, but one problem remains: it's difficult to verify exactly where each generated statement comes from. Researchers and developers often spend as much time validating AI-generated answers as they do reading the original literature.
I wanted to build an infrastructure layer that sits between biomedical databases and AI assistants—not another chatbot, but a system that retrieves evidence, validates it, and ensures every substantive claim can be traced back to supporting sources.
That idea became BioEvidence MCP.
What it does
BioEvidence MCP is an open-source Model Context Protocol (MCP) server that connects AI applications to authoritative biomedical resources such as PubMed and ClinVar.
Instead of simply retrieving documents, the server:
- Parses biomedical queries into structured components (genes, variants, transcripts, diseases)
- Performs source-specific searches
- Normalizes heterogeneous evidence into a common schema
- Scores evidence relevance
- Generates grounded summaries
- Validates every generated claim against retrieved evidence
- Ensures citation integrity before returning the final response
The result is a structured response that AI assistants can consume while preserving evidence provenance.
How we built it
The project is implemented in Python as an MCP server with a modular architecture.
Key components include:
- Biomedical query parser
- Source-specific retrieval connectors
- Normalized evidence model
- Evidence relevance scoring
- Grounded summarization pipeline
- Deterministic post-validation
- Response-wide citation integrity checks
- Offline evaluation framework with reproducible benchmarks
One of the core design decisions was separating deterministic validation from LLM generation. Rather than trusting the language model to cite evidence correctly, BioEvidence MCP validates citations in code after generation.
This approach allows every retained claim to be checked against normalized evidence before it is returned.
Challenges we ran into
The biggest challenge wasn't retrieving biomedical literature—it was making AI-generated summaries reliably traceable to supporting evidence.
Some of the problems we addressed included:
- Preventing unsupported claims from surviving generation
- Distinguishing bibliographic references from supporting evidence
- Handling title-only versus abstract-backed literature differently
- Normalizing evidence from different biomedical databases
- Preventing unrelated variants from appearing in variant-specific searches
- Building deterministic validation rules that do not depend on model behavior
We also learned that evaluation is just as important as generation. Instead of relying only on manual inspection, we built an evaluation framework that measures parsing accuracy, citation integrity, retrieval quality, safety, and robustness.
What we learned
Building BioEvidence MCP reinforced an important lesson:
Trustworthy biomedical AI is as much a software engineering problem as it is a language-model problem.
Retrieval, normalization, provenance, validation, and reproducible evaluation are all essential pieces of the pipeline.
We also learned that deterministic validation dramatically improves transparency. Rather than asking users to trust an AI-generated summary, BioEvidence MCP makes it possible to inspect exactly which evidence supports each retained claim.
Accomplishments that we're proud of
Built an evidence-first MCP server that connects AI applications to biomedical databases instead of relying solely on language model knowledge.
Designed a normalized evidence model that unifies heterogeneous sources like PubMed and ClinVar into a consistent, machine-readable format.
Implemented deterministic claim validation, ensuring every substantive statement in the final response is linked to supporting evidence before it is returned.
Achieved response-wide citation integrity by validating claim-level evidence references and removing unsupported claims rather than leaving uncited statements.
Developed exact-variant retrieval logic, preventing unrelated gene-level records from appearing in variant-specific searches and improving retrieval precision.
Built a reproducible offline evaluation framework that measures query parsing, retrieval quality, citation integrity, safety, and robustness, making system improvements measurable instead of subjective.
Created a modular, extensible architecture where new biomedical evidence sources can be integrated through a common evidence model without changing downstream validation or summarization components.
Released the project as open source, making the evidence retrieval and validation pipeline available for researchers and developers building trustworthy biomedical AI applications.
What's next
Future work includes:
- Additional biomedical evidence sources
- Expanded benchmark datasets
- More sophisticated evidence ranking
- Improved handling of conflicting evidence
- Broader MCP ecosystem integrations
Our long-term goal is to provide a reusable evidence layer that helps researchers and developers build more transparent biomedical AI applications.
Open Source
BioEvidence MCP is released as an open-source project and is designed to integrate with any MCP-compatible AI assistant or application.
We hope it serves as a foundation for building biomedical AI systems where evidence provenance and transparency are first-class design principles—not afterthoughts.
Disclosures
All details are in Github repository. Didn't get much time for creating a video. Used Codex and GPT5.5 starting from defining requirements & architecture. Planning implementation module by module. Executing validation and tests.
Log in or sign up for Devpost to join the conversation.