Alpha Signal Memory Agent

• 

Inspiration

Citadel GQS recently announced they are actively looking to purchase alternative signal sources. Could we make a product that would be valuable to industry leading quant firms?

Recent advances in large language models have opened a massive opportunity for extracting trading signal from unstructured text. Models can now read news articles, research papers, and press releases, then translate technical content into structured financial predictions. Lopez-Lira and Tang (2024) documented this empirically: GPT-4 sentiment scores predict next day stock returns with a Sharpe ratio of 3.8, and predictability is strongest for smaller stocks and low readability (highly complex) news. We wanted explore this potential for LLMs to process information that human analysts struggle with, at scale and speed that no research team can match.

What it does

tl;dr: A self-improving agent that translates complex alternative data into actionable cross-sectional trading signals. Unlike stateless models that process each article in isolation, this agent maintains persistent memory across sessions, accumulating sector knowledge, learning from its own prediction errors, and generating behavioral rules that improve its accuracy over time.

Built on Alibaba Cloud infrastructure (DashScope API + ECS), the agent demonstrates that persistent memory can substitute for both model scale and expensive fine-tuning. A small 8B model with memory outperforms larger 14B models and fine-tuned models at the most actionable trading horizon.

The Alpha Signal Memory Agent is an autonomous market intelligence platform that:

1. Ingests 7 Alternative Sources: Programmatically monitors news RSS, arXiv preprints, USPTO patents, SEC Form 4 insider trades, job postings, GitHub commit velocity, and conference presentations.

2. Maintains a Persistent Memory Store: Built on SQLite, it implements Semantic Memory (facts about companies), Episodic Memory (past predictions and their market outcomes), and Procedural Memory (learned behavioral rules).

3. Runs a Self-Improving Feedback Loop: Matches predictions to actual 5-day stock returns, computes running accuracy stats, and calls Qwen Cloud API to generate advanced behavioral rules from its mistakes.

4. Generates Cross-Sectional Signals: Outputs a weighted score vector across all 10 quantum tickers simultaneously, accounting for competitive dynamics and revenue exposure.

How we built it

Inference & Reasoning: Powered by qwen-plus via the DashScope API (Alibaba Cloud Model Studio).

• Memory Layer: SQLite persistent database with TTL-based forgetting and relevance pruning.

• Infrastructure: Provisioned via Terraform on Alibaba Cloud ECS (Singapore region, ap-southeast-1) for the backend API, and hosted on HuggingFace Spaces for the custom frontend.

• Data Pipeline: Python modules using yfinance for market data, requests for USPTO/SEC/GitHub APIs, and feedparser/arxiv for preprints and news.

• Evaluation Framework: We built a rigorous walk-forward evaluation pipeline grounded in quantitative finance methodology with alphalens. Each model configuration processes 200 chronological articles from January to June 2026. For each prediction, we compute the 5-day forward abnormal return (actual stock return minus the market model expected return, controlling for SPY beta). The primary metric is the Spearman Rank Information Coefficient (IC), which measures the correlation between the model's predicted signal scores and the realized abnormal returns across all ticker-date pairs. We also compute direction accuracy (what percentage of non-trivial predictions correctly predicted the sign of the forward return). To isolate the contribution of memory, we ran a 6-way comparison: 8B base, 8B + memory, 14B base, 14B + memory, 14B fine-tuned, and 14B fine-tuned + memory. Additionally, we conducted temperature sensitivity experiments (t=0.3, 0.5, 0.7, 1.0) on the 14B model to rule out model collapse as an explanation for its poor performance. The evaluation uses a minimum score threshold of |0.3| to filter out low-conviction predictions, and only evaluates tickers with meaningful quantum exposure (excluding GOOGL, MSFT, NVDA which are always scored at 0.0).

Challenges we ran into

The "Rising Tide" Fallacy in Larger Models

Our most significant challenge was discovering that larger models (8B vs 14B) systematically underperformed the smaller 8B model on directional prediction. Through a case-by-case comparison of sampled articles, we identified the root cause: the 14B model treated every quantum computing breakthrough as sector validation (assigning bullish scores to all stocks), while the 8B model correctly identified competitor breakthroughs as competitive threats. In 37 out of 40 strong disagreements between the models, the 8B model was correct 76% of the time. The 14B model predicted IONQ would go up in 95% of articles; IONQ actually went up in only 45% of them.

We hypothesize this stems from pre-training data composition. Larger models absorb more financial media, which overwhelmingly frames emerging technology breakthroughs as positive for the entire sector. The 8B model, with fewer parameters, reasons more directly from first principles: "Company X achieved something. Company X competes with Company Y. Therefore Company Y faces a threat." This naive reasoning happens to be more correct for short-term price action in speculative, pre-revenue stocks.

Fine-Tuning Did Not Fix the Bias

Fine-tuning the 14B model on 881 labeled examples produced almost no improvement (IC went from +0.006 to +0.010). The bullish bias persisted, suggesting it is deeply embedded in the model's pre-trained representations rather than something correctable with a few hundred examples.

Iterative Training Data Refinement (V3 to V4)

Our initial dataset (V3, 187 examples) produced sobering results: overall IC of +0.055, with three tickers actively anti-predictive (NVDA at IC -0.175, MSFT at -0.033, GOOGL at -0.023). ArXiv articles had an IC of -0.058, meaning the model's predictions for academic papers were worse than random.

V4 was a complete rebuild with 16 empirically grounded fixes: removing anti-predictive tickers (MSFT, GOOGL, NVDA hard-coded to 0.0), adding an arXiv score cap (papers rarely move stocks directly), enriching every example with market context (5-day returns, 30-day returns, 52-week position), repairing 307 placeholder chain-of-thought entries, rebalancing the arXiv distribution (training was 1% arXiv but evaluation was 37% arXiv), and adding a technology validation rule (grounded in the empirical observation that RGTI gained 89% on Google's Willow announcement).

The V4 prompt improvements alone (without retraining) improved IC by 46% (+0.055 to +0.093), fixed the arXiv category from anti-predictive (-0.058) to positive (+0.037), and made the overall signal 10x more statistically significant (p=0.03 to p=0.0003). This confirmed the LIMA finding that prompt engineering on the teacher model matters more than scaling data quantity.

Accomplishments that we're proud of

Statistically Significant Results: Our evaluation on 200 chronological articles proved that the memory agent achieved a highly significant Information Coefficient (IC) of +0.107 (p<0.001) and 56.8% direction accuracy, beating both the base 8B model (+0.047 IC) and the 14B base model (+0.006 IC).

Autonomous Learning: We successfully demonstrated the agent's accuracy improving steadily from 46% (Batch 1) to 58% (Batch 4) as it accumulated episodic memories and generated behavioral rules from its mistakes.

Full Results:

Config	IC @1d	IC @2d	IC @5d	IC @10d	IC @20d	Dir Acc @5d	N
8B Base (no memory)	+0.047	+0.042	+0.047	+0.065**	+0.037	53.8%	993
8B + Memory	+0.011	+0.026	+0.107*	+0.199*	+0.035	56.8%	887
14B Base (no memory)	+0.025	-0.001	+0.006	+0.034	+0.121***	49.9%	1013
14B + Memory	+0.015	-0.017	-0.007	-0.030	-0.067**	53.2%	899
14B Fine-tuned (no memory)	+0.038	+0.001	+0.010	+0.045	+0.100***	51.9%	1041
14B FT + Memory	+0.026	-0.008	-0.004	-0.039	-0.047	52.9%	933
30B Base (qwen3-30b-a3b)	+0.113*	+0.096*	+0.139*	+0.144*	+0.081*	45.3%	1090

What we learned

Bigger is not always better, and the relationship is not monotonic. For short-term directional prediction in speculative sectors, the 8B model outperformed the 14B. The 30B model had the best ranking ability (IC +0.139) but the worst direction accuracy (45.3%). These are fundamentally different capabilities, and the right metric depends on your trading strategy.

Persistent memory can substitute for both model scale and fine-tuning. The 8B model with memory (+0.107 IC) outperformed the 14B fine-tuned model (+0.010 IC) at zero training cost. Memory provides the historical context that makes predictions more accurate without requiring any gradient updates.

Prompt engineering on the teacher model matters more than scaling data quantity. The 16 empirical fixes in V4 improved IC by 46% without changing the model or adding articles. This aligns with the LIMA finding that "almost all knowledge in large language models is learned during pretraining."

Evaluate on the right metric. If we had only measured IC, we would have concluded the 30B model was the best. If we had only measured direction accuracy, we would have concluded the 8B was the best. A quintile analysis revealed the 30B model captures a 7 percentage point spread between its top and bottom quintiles, which is a tradeable signal only visible through ranking metrics.

What's next for Alpha Signal Memory Agent

Executable Trading Strategy with Full Backtesting

The immediate next step is translating the signal into a fully backtestable trading strategy. This means building a position sizing engine that converts raw signal scores into dollar-weighted portfolio allocations (accounting for volatility scaling, sector concentration limits, and liquidity constraints). We plan to implement this using a standard event-driven backtesting framework (e.g., Zipline or Backtrader) with realistic transaction costs, slippage, and borrowing costs for short positions.

The 30B model's quintile spread (7 percentage points over 5 days between top and bottom quintiles) suggests a market-neutral long/short strategy: go long the tickers the model scores highest, short the tickers it scores lowest. The backtesting will measure Sharpe ratio, maximum drawdown, turnover, and capacity to determine whether the signal is tradeable at meaningful scale.

Built With

• huggingface
• python
• qwen