Reading the fine print

Inspiration

Our original idea came from pharmaceutical patent fights. We wanted to read Hatch-Waxman court filings with an LLM, score how each judge tends to rule, and trade the branded drug maker against the pharma sector ETF (XPH). We committed that hypothesis before running any backtest. Then we found we could not collect enough historical court data in a weekend.

So we kept the core idea and widened the source. Every material event at a listed drug company has to be disclosed in an SEC 8-K: trial readouts, financings, FDA letters, deals, delisting notices. These filings are long and technical, and most investors react to the headline or press release instead of reading them. Small biotech stocks are also expensive and risky to short, which keeps arbitrage capital away. Our bet was that an LLM which actually reads each filing could find mispricings that persist for a few days.

What it does

For every material 8-K from a U.S. pharma or biotech company, the system:

Pulls the filing from SEC EDGAR the moment it is accepted.
Has an open-weight LLM (gpt-oss-20b, served through UF NaviGator) score its sentiment, surprise and materiality, and tag the event type.
Combines those scores with price and liquidity features in a gradient-boosted model that predicts whether the stock will beat XPH over the next five trading days.
Goes long or short the stock, hedges it 1:1 with XPH, and lets a second model pick the exit rule.
Keeps learning once live: every week it labels new filings and retrains a challenger model, which replaces the current one only if it does at least as well on recent data neither has seen.

How we built it

Data. From SEC EDGAR we pulled every 8-K filed under SIC codes 2834, 2835, 2836 and 8731, keeping only the material items. That gave 39,650 filings from 2016 to 2024, of which 16,390 were tradable (price at least $5 and average daily dollar volume at least $1M). The held-back period, 2025-02-15 to 2026-10-02, had 10,674 filings, of which 4,174 were tradable. Daily and 1-minute prices came from Massive (Polygon).

Label. For filing $i$ we model the stock's return relative to the sector:

$$ y_i = \mathbf{1}\left[ r_{i,t \to t+5} - r_{\text{XPH},t \to t+5} > 0 \right] $$

Signal. A HistGradientBoosting classifier estimates $\hat p_i = P(y_i = 1 \mid x_i)$ from the LLM scores, 5- and 20-day relative returns, volatility, liquidity, timing and item codes. The trade rule is simple:

Long if $\hat p_i \ge 0.54$
Short if $\hat p_i \le 0.46$
No trade otherwise

Entries happen at the first regular-hours minute after the SEC accepts the filing, so the model never sees the future.

Costs. Each trade pays 10 bps per side. Shorts also pay Webull-style borrow fees each day:

$$ \text{daily fee} = V \cdot \frac{b}{360} $$

Here $V$ is the position's value, and $b$ is the annual borrow rate: 0.5%, 2%, 8% or 25% depending on the stock's liquidity (XPH pays 0.5%). The portfolio starts at $100k, puts 5% of equity into each trade and holds at most 20 positions.

Model selection. We used walk-forward validation over 2020–2024: each year is predicted by a model trained only on earlier years, with a 45-day gap between them. We searched 37 settings × 12 cutoffs = 444 combinations and picked the one with the highest t-statistic:

$$ t = \frac{\bar r}{s / \sqrt{n}} $$

To qualify, a combination needed at least 150 trades and a positive average trade in at least four of the five years. The winner (5-day horizon, depth-2 trees) made 3,850 trades averaging +1.5% with a 57% win rate and $t = 6.8$. It lost money only in 2022, a biotech bear market.

Results on the held-back test (2025-02-18 to 2026-10-02), net of all costs:

	Strategy	XPH buy-and-hold
Total return	+60.8%	+41.2%
Sharpe	2.12	1.02
Max drawdown	−10.3%	−18.7%
Trades	1,095	–

Regressing daily returns on SPY and XPH gives an XPH beta of −0.02, a market beta of −0.12, an $R^2$ of 2% and annualized alpha of 35% ($t = 2.9$). With costs doubled, the strategy still returns +46.6% (Sharpe 1.69). We estimate capacity at about $1.5–2.5M.

Stack: Python, pandas, scikit-learn, gpt-oss-20b, SEC EDGAR, Massive, matplotlib, and the Webull OpenAPI for paper execution. python run_all.py reproduces everything from raw filings to the metrics.

Challenges we ran into

Pivoting mid-hackathon. Court data was not available at scale, so we moved from litigation to 8-Ks and disclosed the change in hypothesis.md. Our own results later backed up the pivot: trades after patent-litigation filings lost 1.6% on average.
Data gaps. Our market data API had no 8-K history before 2025, so we wrote an EDGAR scraper with correct acceptance timestamps.
Scoring about 20,000 tradable filings with an LLM under rate limits. We batched five filings per request and required structured JSON output.
A backtest that disagreed with its labels. The first held-back run checked stops every minute, but the training labels checked them at the close. That run returned +23%. Aligning the two gave +104% with a Sharpe of 3.02, which looked too good to be true. We retuned using only 2020–2024 data, added borrow fees, and reached the +60.8% headline. With minute-level stops, the same signals earn about zero (−0.6%), and we report that alongside the headline.
Honest out-of-sample testing. We ran the held-back period three times, so we treat it as a validation set, not a locked test.

What we learned

A Sharpe ratio above 3 is a bug report, not a result. Our +104% run came from a mismatch between how the labels and the backtest handled stops.
Selection beats bias. Under identical exits, model-picked trades averaged +1.36%, against +0.12% for shorting every filing. Most of the edge comes from the model choosing which filings to trade, not from a short tilt.
Where the edge lives. Clinical-trial readouts (+3.2% over 168 trades) and M&A filings (+6.8% over 17 trades) did best. FDA decisions and litigation lost money.
Multiple testing is humbling. We checked our result with the Deflated Sharpe Ratio, which corrects for how many variants were tried:

$$ \text{DSR} = \Phi\left( \frac{(\widehat{SR} - SR_0)\sqrt{T-1}}{\sqrt{1 - \hat\gamma_3 \widehat{SR} + \frac{\hat\gamma_4 - 1}{4}\widehat{SR}^2}} \right) $$

Here $SR_0$ is the Sharpe ratio that the best of $N$ unskilled trials would reach by luck. With $N = 454$, our per-trade Sharpe of 0.110 falls short of $SR_0 = 0.144$, giving a DSR of only 3%. Many of our trials differ only in the cutoff, so this test is conservative. Still, we cannot claim that the walk-forward edge survives a strict correction for selection, and we say so in the quant note.

What's next

Paper trade on Webull with the frozen model, so the first evaluation is on genuinely fresh data.
Switch on the champion/challenger learning loop (live_learning.py).
Test exits at the next open, and measure exposure to size and momentum factors.