Quant Integrity Gate

Quant Integrity Gate detects duplicate trades, split leakage, and unstable setup progression before flawed backtests advance—producing row-level evidence and portable audit reports.

Comment

Inspiration

A backtest can look profitable and still be untrustworthy. Duplicate trades may inflate performance, repeated events can overstate the sample, and train/validation/test leakage can invalidate the result entirely. Even after those problems are removed, a promising setup should not advance simply because one period looked good.

I learned about OpenAI Build Week on July 18 and used the final days of the challenge to build a complete, auditable research-governance tool with Codex and GPT-5.6.

What it does

Quant Integrity Gate provides two connected quality-control modules.

Integrity Gate

The first gate inspects a trade CSV before its results are trusted. It detects:

  • exact duplicate rows;
  • economically equivalent trades;
  • repeated source events;
  • leakage across train, validation, and test splits;
  • material distortion in performance metrics.

The included synthetic demonstration starts with 26 operations and produces 21 canonical rows. Net P&L falls from 297.0 to 197.5, and the gate returns FAIL with understandable row-level evidence.

It also compares six before/after metrics and exports a portable HTML report, machine-readable JSON evidence, and a cleaned canonical CSV.

Setup Lifecycle Gate

The second module evaluates synthetic candidate summaries across:

  1. exploration;
  2. out-of-sample review;
  3. forward-paper confirmation.

Nine deterministic controls examine sample sufficiency, fold stability, regime concentration, performance retention, drawdown expansion, profit-factor stability, and expected-versus-observed cost drift.

Candidates receive READY_FOR_PAPER_REVIEW, REVIEW, or REJECT. Progression is sequential: when a phase fails, later phases are marked BLOCKED, while their independent diagnostic evidence remains visible.

READY_FOR_PAPER_REVIEW is a research-governance status, not authorization to trade.

How I built it

The application uses Python's standard library for the analysis engine, HTTP server, validation, and artifact generation. The responsive interface is built with vanilla HTML, CSS, and JavaScript.

Codex and GPT-5.6 supported architecture selection, deterministic policy design, implementation, test construction, documentation, adversarial review, and iterative correction. Model output was treated as code requiring verification rather than as evidence by itself.

Every export is generated from the same completed analysis result. HTML, JSON, and CSV retain a common analysis_id, analyzed_at, and policy identifier, while recording their own generation time.

The repository contains only synthetic data and has no external Python packages, web resources, credentials, trading execution, signals, or proprietary strategy logic.

Challenges

The hardest challenge was making the evidence semantically precise. A duplicate group must distinguish canonical rows from removed rows. A lifecycle phase must distinguish independent evaluation from permitted progression. Exports must preserve provenance without silently rerunning the analysis.

These issues were addressed through explicit data contracts, deterministic decisions, adversarial inputs, and regression tests.

Accomplishments

  • Two connected governance modules in one local application.
  • Row-level evidence for every integrity finding.
  • Sequential exploration, review, and confirmation gates.
  • Portable HTML, JSON, and CSV audit artifacts.
  • Responsive desktop and mobile interface.
  • 37 unit and integration tests.
  • Zero third-party runtime dependencies.
  • Fully synthetic demonstrations with no sensitive data.

What I learned

Research quality is not just another performance metric. It is a sequence of evidence gates. A useful tool should explain why a result cannot advance, preserve the underlying diagnostic evidence, and make every decision reproducible.

What's next

Future versions could add configurable policies, temporal-overlap detection, statistical significance controls, multiple-testing adjustments, signed artifacts, and integrations with external research pipelines—without turning the gate into a signal generator or trading system.

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Quant Integrity Gate

Built With

Share this project:

Updates