DEfensible

Differential gene expression you can defend

Comment

Inspiration

Differential gene expression is one of the most widely used approaches for understanding how disease, treatment, or environment changes biological systems. It can guide expensive laboratory experiments and influence which hypotheses eventually reach biological or medical trials.

But a successful analysis is not necessarily a valid analysis.

A pipeline such as DESeq2 can execute perfectly even when treatment is inseparable from sequencing batch, paired samples are treated as independent, or the requested comparison is absent from the experiment. The result may still contain convincing coefficients, plots, and adjusted p-values—but the data cannot support the biological claim.

We built DEfensible to catch these failures before analysis begins.

Our guiding principle became:

GPT-5.6 interprets the biology. Deterministic code proves whether the design is estimable.

Instead of using AI to generate another plausible analysis script, we wanted to use it where it is most valuable: translating messy human descriptions and inconsistent sample metadata into explicit, reviewable scientific intent.

What it does

DEfensible is a scientific preflight check for differential gene expression studies.

A researcher uploads a CSV or TSV sample sheet and describes the biological comparison in plain English. No count matrix is required.

GPT-5.6 converts that request into a structured design specification containing:

  • The outcome factor
  • Target and reference conditions
  • Subsets and time points
  • Blocking and nuisance factors
  • Pairing intent
  • Normalized sample labels
  • Exact source evidence for every interpretation
  • Ambiguities that require human review

The researcher can inspect and correct these assumptions before continuing.

A deterministic validation engine then checks:

  • Biological replication
  • Paired-sample completeness
  • Covariate overlap
  • Treatment–batch confounding
  • Missing or duplicated samples
  • Absent factor levels
  • Design-matrix rank
  • Residual degrees of freedom
  • Imbalanced or unsupported comparisons

For a proposed model matrix X, DEfensible verifies:

rank(X) = number of model columns

If the design is singular, confounded, or otherwise unable to identify the requested effect, DEfensible explicitly abstains and withholds executable analysis code.

The included confounding example initially appears convincing: six drug-treated samples and six vehicle controls. However, every control belongs to batch one and every treated sample belongs to batch two. Treatment and batch encode the same division, producing a rank-deficient design. DEfensible labels the comparison Not estimable and explains why.

Switching to the balanced example places both treatments in both batches. The design becomes full rank, and DEfensible releases the defensible formula:

~ batch + treatment

Researchers can export:

  • A cleaned sample sheet
  • A human-readable audit report
  • A machine-readable design specification
  • A DESeq2 analysis starter

How we built it

DEfensible uses a hybrid architecture that separates semantic interpretation from statistical verification.

The application is built with:

  • Next.js 15, React 19, and TypeScript
  • GPT-5.6 Terra through the OpenAI Responses API
  • Strict Structured Outputs and Zod validation
  • Papa Parse for local CSV and TSV processing
  • ml-matrix SVD for matrix-rank verification
  • Vitest and Testing Library
  • Playwright browser testing
  • OpenNext deployment

Uploaded files are parsed locally and are never persisted. Metadata is sent to the API only when the researcher explicitly requests interpretation, and API requests use store: false. The public interface also warns users not to upload patient-identifiable information.

GPT-5.6 is responsible for bounded semantic work: understanding the research question, identifying factors, normalizing inconsistent labels, and preserving source provenance. Its output must satisfy a strict schema and cannot add, remove, or duplicate samples.

The model never determines the final scientific verdict. Tested TypeScript code performs the algebraic and experimental-design checks.

Codex was our primary build partner throughout development. It helped us:

  • Translate the scientific concept into an implementation plan
  • Build the deterministic validation engine
  • Implement SVD-based matrix-rank checks
  • Create synthetic studies with known expected outcomes
  • Design and implement the responsive interface
  • Add editable interpretation evidence and explicit abstention
  • Build the four reproducible export formats
  • Write unit, component, and browser tests
  • Discover and repair pairing and mobile-layout edge cases
  • Produce the documentation, deployment, and submission materials

Human decisions remained central. We defined the scientific boundaries, reviewed the fixture truth, chose when the system must abstain, and constrained GPT-5.6 to interpretation rather than unsupported statistical judgment.

Challenges we ran into

The hardest challenge was deciding what the language model should—and should not—be trusted to do.

Messy metadata requires semantic reasoning. A label such as Drug_A_24hr, a free-text research question, or inconsistent representations of the same donor cannot always be interpreted with simple string rules. GPT-5.6 is well suited to this problem.

Confounding and estimability are different. They require mathematical verification, not a persuasive explanation. We therefore designed a strict boundary between AI interpretation and deterministic proof.

Maintaining sample integrity was another challenge. The application must reject any model response that changes sample identity, omits rows, duplicates samples, or invents factor values without source evidence.

Paired designs introduced important edge cases. A dataset can contain repeated donors without every donor contributing to both conditions. We had to distinguish complete pairing from merely having a donor column and make incomplete pairing visible to the researcher.

We also had to turn technical failures—such as rank deficiency or zero covariate overlap—into explanations that a biologist could act on. Instead of showing only an error code, DEfensible displays the implicated factor cells, observed overlap matrix, failing checks, and a concrete next step.

Finally, the complete audit had to remain understandable on both large screens and narrow mobile displays. Browser testing helped us identify overflow and interaction problems that were not apparent from component-level tests.

Accomplishments that we're proud of

We are especially proud that DEfensible is designed to say no.

Many AI tools are optimized to always produce an answer. DEfensible withholds a formula when producing code would create false confidence. Abstention is treated as a successful scientific outcome—not as a product failure.

Other accomplishments include:

  • A working, zero-login public demonstration
  • Exact source-cell evidence for AI-normalized metadata
  • A human checkpoint before statistical validation
  • Explicit separation between AI interpretation and deterministic verification
  • Interactive confounded, balanced, and paired examples
  • Immediate recomputation after researcher corrections
  • Four reproducible export formats
  • Sixteen passing unit and component tests
  • Four passing end-to-end browser journeys
  • Responsive containment tested down to 320 pixels
  • A privacy-conscious workflow that requires metadata rather than expression counts

Most importantly, a judge can understand the central failure within seconds: twelve samples can still produce an unusable comparison when treatment and batch are inseparable.

What we learned

We learned that correct code and correct science are not the same thing.

Statistical software fits the model it receives. It cannot determine whether that model represents the researcher’s real question or whether the experiment contains the information needed to answer it.

We also learned that language models are most trustworthy when their responsibilities are bounded. GPT-5.6 is valuable because it can interpret biological intent and messy terminology. Deterministic code is valuable because it can prove rank, overlap, replication, and pairing properties exactly. Combining them produces a stronger system than asking either one to do everything.

Provenance also matters as much as prediction. Researchers need to see why Drug_A became drug, which source cell supported that mapping, and where uncertainty remains. An inspectable assumption is far more useful than an unexplained confidence score.

Finally, we learned that refusing to generate code can be a powerful feature. Preventing one invalid study from reaching an expensive experimental stage may be more valuable than generating hundreds of additional analysis scripts.

What's next for DEfensible

Our next priority is to extend DEfensible from a competition prototype into a broader experimental-design assistant.

Planned directions include:

  • Support for edgeR, limma-voom, and pseudobulk single-cell workflows
  • Prospective design checks before samples are collected
  • Power, balance, and replication guidance
  • Safe import of public-study metadata from repositories such as GEO
  • More complex longitudinal, nested, and repeated-measures designs
  • Collaborative review and signed audit histories
  • Domain-specific validation profiles for different experimental systems
  • Integration with reproducible workflow tools
  • Community-contributed synthetic failure cases
  • Exportable reports suitable for methods review and preregistration

The long-term goal is simple: every differential gene expression study should pass an explicit, evidence-backed design audit before computational results are allowed to influence biological decisions.

Built With

Share this project:

Updates