Inspiration
Reports often use real data. But they sometimes attach that data to a claim the evidence does not support.
A study can report reduced stress. A report can then claim improved academic performance or reduced disease risk. The number is real. The conclusion is not.
We built ClaimCheck to make this evidence gap visible.
What it does
ClaimCheck is an Evidence Integrity Workbench.
Users paste a short report and up to three source passages. ClaimCheck extracts factual claims and maps each claim to stable source passages.
For every claim, it returns an inspectable verdict:
- Supported
- Overreach detected
- Contradicted
- Needs review
- Unsupported
It also shows the exact source excerpt, the evidence relationship, and the reasoning issue.
How we built it
We built ClaimCheck as a web application with Google AI Studio, Google Cloud Run, and the Gemini API.
The app has two main modes:
- Evidence Workbench — analyze a report against source documents.
- Compare Two Claims — test two claims against one source passage.
The system segments sources into passages with stable IDs, such as S1-P1. It cites the exact passage used for each verdict.
For a supported claim, ClaimCheck says that no revision is required. For an overreaching claim, it suggests a grounded revision that stays within the evidence.
What makes it different
ClaimCheck does not give a generic truth score.
It separates three things:
- What the source establishes
- What the claim adds or changes
- Why the claim is supported, overreaching, contradicted, or unsupported
For example, a source may establish a 25% reduction in self-reported stress. A report may claim a 25% reduction in heart-disease risk. ClaimCheck identifies this as a metric mismatch and shows the exact evidence.
Human review
ClaimCheck supports human judgment. It does not replace it.
Reviewers can inspect the source, confirm or override the AI result, and save a verification note. They can also apply a grounded revision while the original claim remains visible.
Challenges we ran into
The main challenge was to make the AI explain its reasoning, not only give a verdict.
We needed ClaimCheck to:
- Cite only relevant source passages
- Quote the evidence exactly
- Detect metric substitutions
- Avoid treating an unmeasured outcome as a contradiction
- Produce revisions limited to the evidence
We improved the verification logic with a clear reasoning taxonomy:
- Metric mismatch
- Population leap
- Causation overreach
- Duration overreach
- Scope expansion
- No direct evidence
What we learned
We learned that clear evidence review needs more than a good AI model.
It needs stable evidence IDs, exact quotations, clear verdict rules, and a simple review workflow.
The strongest result is not “this claim is false.” It is:
“This source establishes X. The claim adds Y. Therefore, the claim is overreach.”
What is next
We want to add source-file upload, reusable review templates, and exportable evidence-audit reports.
ClaimCheck makes evidence integrity practical—claim by claim, source by source.
Log in or sign up for Devpost to join the conversation.