Inspiration
Ever since I was 17, I’ve been deeply interested in the stock market. I started with technical indicators—moving averages, RSI, MACD—and spent hours analyzing charts and patterns, hoping to predict price movements. While I occasionally found success, I started noticing something important: sometimes, even when all the indicators pointed to a "sell," the price kept rising.
That’s when I realized a crucial factor was missing in traditional analysis—market sentiment. Trader psychology, public opinion, news buzz—these emotional factors were influencing price movements in ways technical indicators couldn't capture.
What it does
This project predicts buy and sell signals for stocks by integrating:
Price and volume anomalies using Z-scores
Trader sentiment from Twitter and news headlines
Deep learning classification, trained on historical market behavior
It computes a CSDI score, which measures the deviation between sentiment momentum and actual price behavior. If the crowd is excited but price hasn't moved—or vice versa—the model interprets this gap and outputs a decision: buy, sell, or hold.
How we built it
Data Collection: Used yfinance to fetch historical stock price and volume data, and the Twitter API to collect tweets mentioning the stock.
Sentiment Analysis: Applied TextBlob for polarity scoring and BERT embeddings for deep textual understanding.
Feature Engineering: Calculated Z-scores for price and volume to detect anomalies, and computed sentiment momentum.
Model Architecture: Built a neural network using PyTorch that takes BERT vectors and engineered features (Z-score, sentiment momentum) as input.
Training: Trained the model on historical data with known outcomes (price surges or drops) and used binary classification to predict signals.
Challenges we ran into
Data Noise: Tweets and news headlines often contain sarcasm or slang, which made sentiment analysis less accurate without fine-tuning.
Sparse Labeling: Reliable labeling of buy/sell ground truth was difficult, especially when market movement wasn't linked to any single factor.
Computational Resources: BERT embeddings were computationally heavy, requiring optimization and batching for faster training.
Accomplishments that we're proud of
Successfully implemented a hybrid model that combines both statistical and emotional signals.
Designed a system that detected buy signals that traditional indicators missed.
Gained deeper insight into the connection between human behavior and market movement.
Built an explainable, modular architecture that can be adapted to other market signals or sectors.
What we learned
Sentiment, when combined with traditional signals, significantly improves market prediction accuracy.
Z-score-based anomaly detection is a powerful tool when interpreted in context, especially when fused with NLP-based features.
Deep learning models benefit greatly from good feature engineering and real-world domain knowledge.
What's next for A Deep Learning-Based Stock Signal System Powered by CSDI
Integrate more sentiment sources such as Reddit, StockTwits, and financial news APIs.
Fine-tune sentiment models using domain-specific data to handle financial slang and sarcasm better.
Extend the model to multi-class predictions (buy/hold/sell) and portfolio optimization.
Build a real-time dashboard that tracks CSDI signals across multiple stocks.
Deploy the model as a web service to test performance in live market conditions.
Log in or sign up for Devpost to join the conversation.