Check the evidence. Then write the claim.

A higher published accuracy does not automatically mean a better method. Researchers choosing baselines need to know whether the experimental conditions actually support their comparison.

Relay Research turns a question into source-linked observations, reviewable comparisons, and a cited manuscript. NVIDIA Nemotron on Nebius Token Factory powers the reasoning. Researchers can inspect the evidence, challenge a judgment, and revise the claims that depend on it.

From source table to manuscript

  1. Ask and discover. Nemotron proposes question-specific outcomes and comparison criteria. OpenAlex and arXiv supply scholarly records and available text; optional Tavily runtime searches support initial discovery and follow-up investigation.
  2. Select and inspect. Guided mode scans retrieved text for tables without model calls. Choose rows and exact supporting context, inspect the paper/topic map, and retain observations even when they are outside scope or unresolved.
  3. Compare and challenge. Nemotron checks experimental conditions and rechecks proposed comparable pairs against original table context. Inspect the reasons, verify one result against a specified condition, or investigate missing evidence. Corrections are proposals for review, not automatic facts.
  4. Write from reviewed evidence. Generate a provisional abstract, cited related work, comparison discussion, and proposed study. Inspect and edit LaTeX/BibTeX, then export an Overleaf-compatible bundle. Changed evidence marks dependent claims stale for review and regeneration.
  5. Explore another path. Saved execution checkpoints support independent branches with revised questions or criteria, preserving the earlier state.

The gallery illustrates source selection, focused verification, the evidence graph, discovery, comparison challenges, correction proposals, the LaTeX manuscript workspace, and execution branches. These are views of the workflow—not evidence that every investigation finds a correction or produces a supported conclusion.

Hosted PDF compilation is disabled: edit and export on the website, then compile locally in the supported sandbox or in Overleaf. Relay does not claim a live Overleaf integration or independently performed scientific experiments.

A result you can inspect

The recorded guided DeiT investigation retained eight top-1 accuracy observations and assessed sixteen pairs: three comparable, twelve incompatible, and one insufficient-evidence judgment.

The three source-rechecked pairs report 72.2→74.5, 79.8→81.2, and 81.8→83.4 for matched 224-resolution variants. These are descriptive within-paper comparisons, ready for human review—not reproduced experiments or causal findings. Pending pairs remain distinct from incompatible results.

Earlier false accepted comparisons motivated the original-source recheck. Relay's purpose is to make such judgments inspectable and revisable, not to turn every number into a confident answer.

Architecture that supports the workflow

  • Nemotron 3 Super on Nebius Token Factory (nvidia/nemotron-3-super-120b-a12b) handles planning, extraction, comparisons, source rechecks, investigations, synthesis, and manuscript drafting through real inference calls.
  • One or two bounded paper workers return selected evidence to a coordinator before comparison selection. Shared call/time budgets, Stop controls, and checkpoints keep work manageable. An in-flight request may finish and be billed after Stop.
  • Tavily provides optional runtime discovery and investigation searches, preserving URLs. The recorded guided study includes two discovery queries. Search hits are evidence to assess, not verified results.
  • IndexedDB stores portable browser copies of projects and source text, excluding credentials. The Python backend maintains source hashes and execution checkpoints. No vector database is required for exact source-address checks.
  • Python, native JavaScript, and Render provide the application. We use Token Factory inference without provisioning GPU instances; we do not claim Nebius Serverless Jobs or Endpoints.

What we measured—and what it means

Software checks: 168 Python tests and 12 JavaScript tests passed, covering source references, review separation, checkpoint integrity, worker budgets/cancellation, preserved results, and recorded loading without new inference.

Worker trial: identical selected rows from two papers produced twelve observations and two extraction calls in each mode: 40.05 seconds sequentially versus 25.05 seconds with two workers. This single trial excludes planning and comparison time. Its later six comparison judgments produced zero accepted comparable pairs, two incompatible, and four unresolved; thirty candidates remained unassessed. Faster extraction did not solve interpretation accuracy.

Three-model diagnostic: Super, Nano, and GPT-OSS-120B received six identical source tasks through Token Factory. All returned the expected values and status labels, but Nano explained one variant conflict incorrectly as a resolution mismatch. Super and GPT-OSS identified that distinction correctly. Times were 15.73s, 31.11s, and 3.68s respectively. Citation coverage remained incomplete.

These are small development diagnostics with coding-assistant-checked labels, not independent scientific benchmarks, full-pipeline model rankings, or measured researcher time savings. Super remains the application's primary model. Inspect the complete diagnostic receipts.

What we learned

Broad extraction was difficult to audit, so we added guided evidence selection. Domain-specific comparison rules became question-derived contracts. Correct-looking labels still concealed incorrect reasoning, making source context and human review essential. Codex assisted implementation, testing, and failure analysis.

Next: independent domain annotation, paper-disjoint evaluation, stronger table interpretation, and studies of researchers using Relay to choose and audit baselines.

Try the complete workflow

Open Relay → Open guided results → inspect a comparable pair and its source → inspect a conflicting judgment → explore the manuscript workspace and execution history.

Recorded inspection requires no inference key. Fresh inference uses your Nebius key; Tavily is optional. Use a dedicated deployment for private research.

Relay keeps the path from source evidence to a manuscript claim visible, challengeable, and revisable.

Built With

Share this project:

Updates

Submission history