Edge or Noise

Describe a trading strategy in plain English. Nemotron turns it into exact rules, and an honest backtester tells you whether it's a real edge or just noise. Can also search the web for a strategy

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Edge or Noise

Inspiration

I've been trading for over five years. The same thing kept happening, to me and to almost everyone in the trading communities I've been part of: a setup looks amazing on a chart whose ending you already know. Over time I realized a lot of things we see on social media are just cherry picking. Having a background in math, I am also quite familiar with statistics and how sometimes a seemingly overwhelming edge can turn out to be just luck in a way that's difficult to fully understand right away, at least for me.

Backtesting should fix that, but the tools make it worse. Most discretionary traders describe their setups in words ("a sweep of the Asia low, then a fair value gap in the New York morning"). That's hard to code, so they test by eye. The backtesters that do exist reward the wrong habit: tweak a parameter, rerun, keep the best curve, repeat. After 40 tries something always looks great, and it's noise.

I wanted a backtester that's on the trader's side by being honest: it makes the rules exact, then makes overfitting visible and hard instead of easy.

What it does

You describe a strategy in plain English. NVIDIA Nemotron turns it into an exact rule spec, asks about anything you left out, and shows you a rules card to approve. A deterministic engine runs it on 1-minute data from 2020 to 2026 (Nasdaq 100, S&P 500, Bitcoin, Ether) and gives a verdict: probably noise, can't tell yet or promising.

  • Every try counts. Any change to the rules is a new numbered trial, and the verdict is deflated for how many you've run.
  • A sealed holdout. The last 20% of the data (about 16 months) is hidden. You get one final exam on it, and taking it locks the rules.
  • A random baseline. 1,000 random-entry runs matched to the strategy's sessions, exits, costs, long/short mix, trade count and stop distances. Beating random is the bar.

If you don't have a strategy yet, it can find one: Tavily searches the web, and Nemotron writes what it finds as complete rules, with every number checked against the sources.

How I built it

Results are in R, multiples of what you risk on each trade, so they don't depend on account size or market. A stop-out is −1R, a hit 2R target is +2R, and a gap through the stop can lose more than 1R:

$$ R = \frac{\text{exit} - \text{entry}}{\text{entry} - \text{stop}} - \text{cost}_R $$

  • Engine (Python, pandas, Numba). Building blocks (sessions and killzones, prior day/week and Asia/London highs and lows, swings, fair value gaps, sweeps, displacement, EMA, ATR) feed a rule interpreter. Signals are vectorized, and a Numba loop resolves each trade on the 1-minute bars inside the strategy's timeframe. The engine is a standalone package with its own tests.
  • Rule spec (Pydantic). All/any/not filters, "A then B within N bars" setups, and references like the gap's top or the signal candle's low. The canonical JSON is hashed, and the hash is the trial's identity: rewording a prompt into the same rules doesn't cost a trial, and changing one number does.
  • Nemotron 3 Super on Nebius Token Factory, in json_object mode, temperature 0, thinking off. The prompt is generated from the block registry, so adding a block teaches the model about it. The model only ever produces JSON: it never writes code that runs, and never computes a number you see.
  • The verdict, computed by the engine. The expectancy interval is

$$ \bar{R} \pm 1.96\,\frac{s}{\sqrt{n}} $$

and the Deflated Sharpe Ratio (Bailey and López de Prado) asks whether the trial's per-trade Sharpe beats what the best of N no-edge trials would show by luck, where V is the variance of the trials' Sharpes:

$$ SR_0 = \sqrt{V}\left[(1-\gamma)\,\Phi^{-1}!\left(1-\frac{1}{N}\right) + \gamma\,\Phi^{-1}!\left(1-\frac{1}{N e}\right)\right] $$

$$ \text{DSR} = \Phi!\left(\frac{(\widehat{SR} - SR_0)\sqrt{n-1}}{\sqrt{1 - \text{skew}\cdot\widehat{SR} + \frac{\text{kurt} - 1}{4}\,\widehat{SR}^2}}\right) $$

  • App. Django REST and Postgres, React and TypeScript with TradingView Lightweight Charts, one Docker image on Railway, with spend caps on every model and search call.

Challenges

Lookahead hides everywhere. pandas labels resampled bars by their start time, so a naive join hands a 5-minute strategy the bar's close before it happened. A swing low isn't a swing low until later bars confirm it, a fair value gap exists only at its third candle's close, and the Asia low is final only when Asia ends. Every block has a truncation test: computed on the data up to a random bar t, it must equal the full-data value at t, on every market.

Fills that don't flatter. When a 5-minute bar touches both the stop and the target, which came first? The engine looks at the 1-minute bars inside it, and if it still can't tell, the stop wins. Gaps fill at the open, limit orders need price to trade through, and the spread is charged on the buy side. Dukascopy's quoted spread changed regime between 2020 and 2026, so the spread is a fixed schedule from the last six months. For crypto it's a share of price: Bitcoin went from about 4,000 to 126,000 dollars, and a fixed dollar spread would have overcharged every 2020 trade.

The model wanted to help too much. Nemotron reads trader language well, but when a trader left out a stop, it would quietly pick one. An honest backtester can't allow that, because the result would test the model's idea instead of the trader's. The fix is deterministic checks against the trader's own words: an entry, stop or target they never stated is dropped and asked about, ticks and pips are always asked about, and a session they named but the spec lacks is sent back. I built a golden eval of 14 trader prompts with expected specs and ran it on every change. Temperature 0 isn't fully deterministic on a hosted model, so I judged each change over several runs.

Honest answers are a hard sell. Most strategies land on can't tell yet, because that's the truth. To make that useful rather than disappointing, the verdict also says how many more trades it would take to know.

What I learned

  • How much data an edge needs. The demo's first trial earned +0.05R per trade over 277 trades and beat 68% of random entries. For its 95% interval to exclude zero at that spread of outcomes, it would need

$$ n \ge \left(\frac{1.96\,s}{\bar{R}}\right)^2 \approx 3{,}100 \text{ trades,} $$

about 60 years of that setup. Most of the edges traders argue about can't be proven with a few hundred trades.

  • The holdout tells the truth. The demo's best tweak earned +0.07R per trade in development. On the sealed 16 months it earned +0.006R over 28 trades.
  • The model is a translator, not a judge. Nemotron turns "sweep of the Asia low" into a spec and asks about what's missing. Everything that decides a result stays deterministic and tested.
  • Measure model settings instead of guessing. I expected the stricter options to do better. On the eval, json_schema mode scored 10/12 (it picked a direction instead of asking), thinking on scored 9/12 and took about 9 seconds per parse, and json_object with thinking off scored 12/12 three runs in a row at about 1 second.

What's next

A prop-firm simulator that turns R into pass and breach odds under a firm's daily and max loss limits, more of the rule language (break-even and trailing stops, multi-timeframe conditions, RSI, Bollinger Bands), more stress tests, and accounts, so trials count per trader across projects and not just per project. A lot more additional features to come.

Built With

Share this project:

Updates

Submission history