Stock picker photo

Inspiration

The key to improving Large Language Models (LLMs) in specific domains lies in comprehensive, high-quality evaluations. Our project, the LLM Stock Trader, addresses this need in the financial sector. By creating a tool for evaluating LLMs' stock trading capabilities, we're paving the way for future research into making language agents better at stock trading.

What it does

Our tool allows LLMs to simulate buying and selling stocks. Users can:

Select custom start and end dates for trading simulations
Watch as the AI makes real-time buying and selling decisions based on market data and newspaper headlines
Analyze the AI's performance and decision-making process

How we built it

For each day in the specified time range, the LLM receives its current portfolio and crucial news headlines. Then, we tell the LLM to output a JSON of the trades it wants to make. For each trade, it also outputs a justification for why it made the trade (chain of thought prompting). This makes the model less likely to hallucinate nonexistent stock tickers or make poor decisions, because it's forced to explain out loud why it made a given decision.

Challenges we ran into

API Rate Limits:

Groq: Optimized our trading bot's efficiency to reduce API calls and avoid rate limits
yfinance: Implemented local caching of stock data to minimize repeated downloads and avoid yfinance's rate limits

Frontend Development:

Tackled numerous UI bugs, which tested our patience in debugging very precise issues. Debugging front end bugs took longer than any other portion of the project.