-
-
OmicsTrust: evidence before interpretation for high-dimensional omics research.
-
Clean omics results can still hide confounding, missing metadata, or unstable structure.
-
The engine audits quality, null calibration, confounding, stability, and reproducibility.
-
An audit can pass QC and stability yet remain unsafe when batch risk is high.
-
GPT-5.6 explains deterministic evidence without changing statistics or trust decisions.
-
In VANISH, PC11 emerged as the strongest internal candidate among 25 screened axes.
-
Permutation, bootstrap, multiple-testing, and metadata checks challenge the PC11 signal.
-
OmicsTrust can stop misleading findings before researchers invest in costly validation.
-
Every report separates supported claims from conclusions that remain unsafe or unvalidated.
-
Built and hardened with Codex, with 119 automated tests passing locally and on GitHub.
Inspiration
Most omics tools are built around a familiar question:
Can we detect a molecular signal?
OmicsTrust began with a more difficult question:
Is the evidence strong enough to interpret that signal, publish it, or spendmoney validating it?
High-dimensional molecular datasets can produce striking patterns even when theapparent result is actually caused by batch effects, donor imbalance, missingmetadata, unstable components, technical covariates, or random variation. Ananalysis may pass quality control and be perfectly reproducible at thecomputational level while still being scientifically unsafe to interpret.
For that reason, OmicsTrust was never intended to become another biomarkerdiscovery tool. The goal was to create a scientific trust layer between ananalysis result and the important decisions that may follow from it.
The project is built around one central principle:
[\text{Signal strength} \neq \text{scientific interpretability}]
That principle became especially clear when a real spatial transcriptomicsdataset initially received a high-trust result even though it contained nobatch, donor, or biological-label metadata. The statistics had not identifiedconfounding because the information needed to test for it was simply missing.That experience led to a major redesign of OmicsTrust around a conservativerule:
When a risk cannot be assessed, the answer is unknown, not low.
Why We Built It: The Cost of Acting Too Early
This problem is not limited to a particular dataset or disease area. Inbiomedical research, statistically attractive findings often move quickly intowet-lab experiments, external contracts, biomarker-development programs, orclinical-study planning. Yet there is rarely a consistent evidence gate betweenthe first promising result and those expensive next steps.
A widely cited economic analysis estimated that more than half of preclinicalresearch was not reproducible and linked this problem to approximately $28billion in annual spending in the United States. The exact figure remainsuncertain, and OmicsTrust does not claim that all of this spending could beprevented or addressed through software. The estimate does, however, show thescale of the underlying issue: incomplete or weak evidence can consume majorresources, delay genuinely useful discoveries, and create false hope before thelimitations become visible.
Freedman, Cockburn, and Simcoe, The Economics of Reproducibility inPreclinical Research
OmicsTrust focuses on an important and potentially preventable part of thatproblem: the omics-evidence stage. If a result is dominated by batch, lacksdonor metadata, fails under resampling, sits at the empirical-null resolutionfloor, or cannot be reproduced from its recorded inputs, the research teamshould know that before it pays for the next expensive stage.
The purpose is not to discourage ambitious research. It is to make the nextdecision explicit. A team should be able to decide whether to proceed, repairthe analysis, collect missing metadata, acquire more samples, lock thehypothesis for independent validation, or stop making a claim that the currentevidence does not support.
What OmicsTrust Is
OmicsTrust is a Research Use Only evidence-audit platform for high-dimensionalmolecular research. It evaluates whether a result is stable, reproducible,adequately protected against known failure modes, and supported strongly enoughfor a clearly defined level of interpretation.
The platform goes beyond conventional data quality control. It audits the fullevidence chain, from the input matrix and accompanying metadata to statisticalstructure, confounding, stability, multiplicity, reproducibility, and thescientific claims that can reasonably be made.
Rather than producing a generic success score, OmicsTrust returns aconservative decision:
safe: the configured evidence gates support the stated interpretation
limited: useful structure is present, but important claims remain restricted
unsafe: a detected failure mode prevents interpretation
needs_validation: the finding remains a research hypothesis and requires anindependent test
Each decision is accompanied by a failure hierarchy, a claim matrix, anevidence ledger, and a reproducibility record. The system is designed not onlyto say yes or no, but to answer four practical questions:
What was tested?
What failed, or what could not be assessed?
What can legitimately be claimed from the available evidence?
What must happen before stronger claims become acceptable?
The Product
Researchers can work with OmicsTrust through a private local web console, acommand-line interface, a REST API, or a Docker deployment. The samedeterministic workflows remain available even when no OpenAI API key is used.
The platform supports a connected research lifecycle:
Inspect and validate .h5ad, .csv, .tsv, and text matrices
Audit structural signal, empirical-null support, batch and donor risk,technical dominance, metadata sufficiency, and resampling stability
Search public dataset metadata without automatically downloading large files
Run registered single-cell and treatment-response workflows
Discover research-stage treatment-response axes
Lock a discovered axis before evaluating it in an independent cohort
Reproduce a previous run from its recorded inputs, configuration,environment, random seeds, and fingerprints
Export reviewer-ready HTML, PDF, and JSON evidence packages
This makes OmicsTrust a scientific control plane rather than a single-purposeanalysis script. Discovery, auditing, validation, reporting, and claimboundaries are all governed by the same evidence contract.
From a Result to a Defensible Next Action
OmicsTrust is intended to provide more than a warning. Every failure is linkedto supporting evidence and to a prioritized next research action.
When core metadata are missing, the platform marks confounding risk asunknown and specifies which batch, donor, sample, condition, or label fieldsare needed.
When the observed structure is dominated by batch, it recommends within-batchnull models, residualized sensitivity analyses, or study redesign before anybiological interpretation is attempted.
When an axis is unstable, it blocks strong claims and points toward a largersample, a more reliable assay, or a locked replication analysis.
When a discovery is internally convincing but has not been validatedexternally, the system converts it into a locked validation contract insteadof presenting it as a biomarker.
When the evidence passes the configured gates, the platform records exactlywhich claim is supported and preserves the full provenance required forreview.
This is the practical service model behind the product:
Bring the molecular data and the research question. OmicsTrust audits theevidence and returns the most defensible next scientific step.
It does not instruct clinicians how to treat patients. Its role is to tell aresearch team what the current evidence allows them to do next.
How It Works
Research question or molecular dataset | v GPT-5.6 Evidence Copilot (optional) | v Validated workflow registry | v Deterministic OmicsTrust engine | +--> data and metadata gates +--> structural signal and empirical null +--> batch, donor, and technical-risk tests +--> stability and multiple-testing controls +--> trust decision and failure hierarchy +--> claim matrix and evidence ledger +--> HTML / PDF / JSON reports | v GPT-5.6 evidence explanation (optional, non-authoritative)
The deterministic engine remains the scientific authority. GPT-5.6 helpsinterpret natural-language intent, maps that intent to a registered workflow,and explains the completed evidence in accessible language. It cannot alter thestatistics, override a workflow that was explicitly selected, silently redirecta request, or transform an unsafe result into a safe one.
To protect privacy, raw expression matrices, patient-level rows, and local filepaths are excluded from requests sent to GPT-5.6. The integration usesstore=False. Requests involving clinical decisions, diagnosis, dosing, orpatient-level treatment are rejected.
What Makes It Different
Many tools are designed to maximize signal discovery. OmicsTrust is designedto handle uncertainty with discipline.
Its main contribution is the combination of:
Deterministic statistical auditing
Conservative treatment of missing metadata
Clear separation between stability and interpretability
Explicit workflow contracts and locked validation
Machine-readable provenance and evidence ledgers
Claim boundaries that are enforced computationally
A constrained AI interface that explains the evidence without replacing it
In OmicsTrust, a limitation is not a standard disclaimer added at the end of areport. It directly influences the final decision.
Real Research Proof: VANISH PC11 / VasoGate
The main integration case is a retrospective transcriptomictreatment-response analysis involving 116 patients from the VANISHseptic-shock cohort.The primary integration case also shows why a scientific trust layer matters inreal research.
OmicsTrust screened 25 transcriptomic principal components and identified PC11as the strongest internal candidate interaction between randomized vasopressorallocation and 28-day mortality. The preserved evidence includes:The original VANISH randomized trialcompared early vasopressin with norepinephrine in septic shock. In the overalltrial population, vasopressin did not improve the primary outcome ofkidney-failure-free days.That population-level finding, however, does not answer a differentprecision-medicine question: might patients in different molecular states showdifferent associations between vasopressor allocation and outcome?
Published transcriptomic work from VANISHlater examined predefined Sepsis Response Signature endotypes. OmicsTrust askeda broader, de novo question. It did not assume that the previously definedendotypes represented the only meaningful structure in the transcriptome, andit did not search for a single gene.
What OmicsTrust discovered
The analysis used baseline whole-blood transcriptomic and clinical informationfrom 116 eligible VANISH patients with complete data. Beginning with 28,220expression features, it selected 5,000 variable features, reconstructed thedominant transcriptomic structure, and screened 25 principal components for aninteraction between randomized vasopressor allocation and 28-day mortality.
The component labeled PC11 in this analysis emerged as the strongest internalcandidate. PC11 is not a newly discovered gene. It is a latent multi-gene axisthat was not reported in the original VANISH analyses. Each patient receives acontinuous score that represents a coordinated expression state distributedacross many genes.
The interaction model was:
[\text{death}_{28} \sim \text{vasopressor} \times \text{PC} +\text{steroid} + \text{SRS} + \text{age} + \text{APACHE II} + \text{sex}]
Among the 25 screened axes, the vasopressin-by-PC11 interaction produced anodds ratio of 0.16881 for each one-standard-deviation increase in PC11.Internally, this indicates that the association between vasopressin allocationand mortality changed substantially across the PC11 continuum.
A descriptive split makes the pattern easier to understand:
In the PC11-high group, observed 28-day mortality was approximately 40.0%with norepinephrine and 12.1% with vasopressin.
In the PC11-low group, observed mortality was approximately 23.8% withnorepinephrine and 45.9% with vasopressin.
These subgroup percentages are descriptive observations. They are not causalestimates and should not be interpreted as a treatment rule. Their importancelies in the reversal of direction across the axis, which suggests a candidatemolecular treatment-response interaction that could remain hidden in anoverall average comparison.
How OmicsTrust challenged the finding
The platform did not stop after identifying the smallest p-value. It examinedwhether the candidate remained credible under several internal challenges:
28,220 expression features
5,000 variable features used for discovery
25 candidate axes included in the screening family
Interaction odds ratio per 1 SD PC11: 0.16881
Wald (p = 0.00377)
Likelihood-ratio (p = 0.001258)
Benjamini-Hochberg FDR: 0.03145
Permutation (p = 0.003996)
Bootstrap direction stability: 99.9%
Metadata explanation (R^2 = 0.01793)
The key result is not simply that OmicsTrust detected a strong internal signal.It is also that the platform refused to exaggerate what that signal means.PC11 is classified as a retrospective research hypothesis that requiresexternal validation. It is not classified as a biomarker, a causal effect, ora treatment-selection rule.
The low metadata-explanation value suggests that PC11 is not an obviousreconstruction of the recorded clinical variables. The permutation,multiplicity, and bootstrap results also support internal robustness. None ofthese findings proves that the axis will generalize to an independent cohort.
Why the discovery matters, and where it stops
The defensible novelty claim is not that the original VANISH investigatorsoverlooked a gene. The novelty is that OmicsTrust identified a previouslyunreported, multi-gene candidate treatment-response axis by asking a differentquestion of the available transcriptomic data.
The same 116-patient subset was used both to discover and evaluate the result.The finding is therefore exposed to discovery-set optimism, cohort specificity,and differences in PCA reconstruction. The original trial was neither designednor powered to validate PC11, and its primary endpoint was not thistranscriptomic interaction.
For that reason, OmicsTrust will not label PC11 as a biomarker or use it toguide vasopressor treatment. The next scientifically valid step is to lock thegene weighting, axis orientation, preprocessing procedure, model, covariates,and success criteria before testing PC11 in a compatible independent cohort.Any eventual clinical use would require prospective validation in abiomarker-stratified trial.
If independent validation succeeds, PC11 could help explain why a treatmentwith a weak average effect might still behave differently in molecularlydistinct groups of patients. If the validation fails, OmicsTrust preserves thatnegative result as well. Both outcomes have scientific value.
This case captures the full philosophy of the platform: discover evidence,challenge it, preserve it, and enforce the boundary between an interestingresult and a justified scientific claim.
Built During OpenAI Build Week
OmicsTrust started as a deterministic research prototype. During Build Week, Iused Codex to turn that foundation into a focused and testable product.
The Build Week work included:
Building the optional GPT-5.6 Evidence Copilot
Implementing a central workflow registry and explicit-workflow precedence
Restoring and refining the private web console
Adding privacy-preserving construction of AI payloads
Strengthening failure prioritization and conservative metadata handling
Creating a dedicated PC11 evidence experience
Hardening public metadata search against SSRF and unsafe XML
Improving delivery of HTML, PDF, and JSON reports
Removing unrelated drug-design features so the product remained focused
Expanding routing, privacy, security, scientific, and regression tests
Producing a reproducible judge package and an English-language demo video
The focused release passes 119 automated tests. Seven tests that depend on thedataset are skipped because the private VANISH fixture is not included in thepublic repository.
Challenges
The most difficult part was not adding more metrics. It was preventing thesystem from becoming overconfident.
That required keeping several concepts separate even though they are oftentreated as if they were equivalent:
A stable signal is not automatically biological
A reproducible computation does not necessarily mean the discovery itself isreproducible
Failing to detect a batch association does not imply low batch risk when thebatch metadata are missing
A statistically interesting treatment interaction is not a clinical rule
An AI-generated explanation must never become the source of the scientificdecision
Other challenges included extending treatment-response discovery beyond onecase study, preventing unnecessary transmission of biomedical data to AIsystems, and producing reports that are useful to both computationalresearchers and decision-makers.
What I Learned
Scientific trust cannot be reduced to a single number. A credible system mustpreserve the reasoning behind a result: what information was available, whatwas tested, what could not be assessed, which failure mattered most, and whichnext experiment could change the conclusion.
I also learned that GPT-5.6 is most useful in this setting when it serves as aconstrained interface between human intent and deterministic evidence. Thelanguage model makes a technically complex platform easier to use and explain,while the statistical engine keeps the conclusions grounded and reproducible.
Most importantly, claim boundaries should be generated as computationaloutputs rather than added afterward as legal language.
Potential Impact
Biotechnology companies and academic research teams can spend months and largebudgets validating molecular findings that later turn out to be batch-driven,unstable, poorly documented, or impossible to reproduce.
OmicsTrust is intended to detect those weaknesses earlier and help directresources toward findings that survive serious scrutiny.
Its product promise is straightforward:
Send us your omics evidence. We audit whether it is interpretable before youspend money acting on it.
If PC11 is independently validated, it could motivate better-stratified sepsisresearch and new mechanistic experiments. More broadly, OmicsTrust could becomethe evidence infrastructure used before a result moves from computationaldiscovery into wet-lab validation, partnership discussions, publication, or aprospective study.
The statistical trust problem is not specific to any one disease. The sameevidence gates can support transcriptomic and other high-dimensional researchin oncology, infectious disease, neurology, psychiatry, autoimmunity, raredisease, and drug-response research, provided that the required data andmetadata are available.
For researchers, the result can be clearer analyses, reviewer-ready provenance,and fewer months spent pursuing artifacts. For biotechnology andpharmaceutical teams, it can provide an auditable go, repair, validate, or stopdecision before substantial capital is committed. For patients, the benefitwould be indirect but meaningful: fewer premature claims, better-designedvalidation studies, and a greater chance that limited resources are focused onfindings capable of surviving independent testing.
With further investment, external benchmarking, independent validation, andprospective pilots, OmicsTrust could become a standard trust gate for omicsresearch worldwide. Its long-term ambition is that no high-impact molecularclaim should advance merely because it produced a clean plot or a smallp-value. It should advance because the complete evidence package survived atransparent and reproducible audit.
What's Next
The next milestones are locked validation in independent cohorts, additionalbenchmarked case studies, external scientific review, simpler privatedeployment, stronger report visualization, and prospective pilot projects withbiotechnology and academic research teams.
OmicsTrust is Research Use Only. It is not intended for diagnosis, prognosis,treatment selection, dosing, or clinical decision-making.
Log in or sign up for Devpost to join the conversation.