Strategy Lab

Strategy Lab stress-tests algorithmic trading hypotheses with reproducible OOS replay, execution stress scenarios, deterministic evidence, and grounded GPT-5.6 explanations.

Comment

Inspiration

Many algorithmic trading strategies look impressive in backtests but fail when they meet unseen data, higher fees, slippage, or stricter execution conditions.

Strategy Lab was created to answer a harder question:

Can we find evidence that a trading hypothesis is fragile before real money is put at risk?

It is a PAPER/RESEARCH laboratory, not a trading bot, signal service, or investment-advice product. Real trading and mainnet execution are disabled.

What it does

Strategy Lab evaluates trading hypotheses using public Binance Spot candles.

The research pipeline includes:

  • bounded candidate search;
  • walk-forward and out-of-sample validation;
  • OOS ranking;
  • selection of one winner;
  • a sealed Final Holdout;
  • deterministic PASS/FAIL verdicts;
  • optional manual transfer into PAPER simulation.

For OpenAI Build Week, I added an Evidence-Backed Robustness Check for the already selected OOS winner.

The check does not rerun the optimizer, change strategy parameters, select a new winner, reevaluate Final Holdout, start PAPER, or enable live trading.

It performs:

  • exact reproduction of the saved OOS baseline;
  • fees increased by 1.5×;
  • fees increased by 2×;
  • additional slippage of 5 bps;
  • additional slippage of 10 bps;
  • an optional stricter liquidity cap when the original execution profile supports it.

If the reproduced baseline does not match the saved result, the process stops with INTEGRITY_MISMATCH.

The deterministic classifier can return:

  • ROBUST;
  • CONDITIONALLY ROBUST;
  • FRAGILE;
  • NOT ENOUGH EVIDENCE;
  • BASELINE INELIGIBLE;
  • INTEGRITY MISMATCH;
  • ERROR.

Every conclusion is linked to structured evidence IDs.

How GPT-5.6 is used

GPT-5.6 is integrated through the OpenAI Responses API.

It receives only the deterministic classification, validated metrics, evidence IDs, and limitations.

GPT-5.6 does not:

  • calculate metrics;
  • select the strategy;
  • assign the classification;
  • change Final Holdout;
  • start PAPER or live trading;
  • provide BUY, SELL, or HOLD recommendations.

Its response must satisfy a strict Ukrainian, Russian, and English JSON schema.

The output is validated for:

  • schema completeness;
  • allowed evidence IDs;
  • numeric grounding;
  • trading-safety rules.

If the API call or validation fails, the deterministic result remains valid and the interface shows a localized fallback explanation.

The final production verification showed:

Model: gpt-5.6 · source: gpt · fallback reason: none

How I built it

The broader Strategy Lab project existed before Build Week.

During Build Week, I implemented the complete robustness extension primarily with Codex:

  • immutable replay artifacts;
  • baseline integrity checks;
  • execution stress scenarios;
  • versioned deterministic classification;
  • evidence IDs;
  • GPT-5.6 integration;
  • dashboard and printable report integration;
  • Ukrainian, Russian, and English localization;
  • admin/viewer authorization;
  • read-only version history;
  • regression tests.

I made the key product and safety decisions: GPT cannot influence deterministic research outcomes, Final Holdout is never recalculated, old runs are not reconstructed from fresh data, and no result can automatically enable trading.

Claude Code was later used to diagnose one production Responses API issue. A realistic three-language response was being truncated because the output-token limit was too low. A live synthetic smoke test identified the real cause and verified the final fix.

Challenges

The main challenge was preserving semantic honesty.

A stress scenario should not be marked as passed merely because the calculation completed. It must preserve economic eligibility and have enough evidence.

Another challenge was making GPT useful without allowing it to become the decision-maker. The solution was a strict evidence allowlist, numeric validation, safety validation, and a deterministic fallback.

Accomplishments

  • Exact reproducible baseline replay.
  • Fail-closed integrity verification.
  • Transparent versioned rules instead of an opaque AI score.
  • Structured evidence linked to every conclusion.
  • Real production GPT-5.6 explanations.
  • Read-only history for robustness versions.
  • No changes to live-trading restrictions or Final Holdout.
  • 1,411 passing tests and 18 passing subtests.

What I learned

I learned that reproducibility requires preserving exact replay inputs, not only metrics or fingerprints.

I also learned that AI works best here as an explanation layer over deterministic evidence, not as a replacement for validation rules.

Production integration must be tested with realistic payload sizes, not only mocked responses.

What is next

Possible future research-only extensions include:

  • parameter sensitivity analysis;
  • regime slicing;
  • bootstrap and Monte Carlo analysis;
  • comparison across multiple historical runs;
  • more efficient replay-artifact storage.

Strategy Lab will remain fail-closed, PAPER/RESEARCH only, and separate from real-money execution.

Built With

Share this project:

Updates