Inspiration
Stock markets are influenced by numerous factors, including technical indicators, fundamental data, options flow, and sentiment analysis. Many forecasting models rely on a single method, limiting their effectiveness. We wanted to build a robust prediction system that integrates multiple strategies to increase accuracy and reliability.
What it does
StockSense is a stock price prediction system that combines different forecasting techniques to improve accuracy. It integrates:
- XGBoost, LSTM, and SVR models for predictive analytics.
- A Gradient Boosting Regressor to intelligently weight different forecasts.
- Technical indicators like RSI, MACD, Bollinger Bands, and VWAP.
- Fundamental data such as P/E ratio, dividend yield, and price-to-book value.
- Options flow analysis to detect unusual trading activity.
- Sentiment analysis from financial news using NLP techniques.
- Wavelet denoising to remove market noise and focus on trends.
By combining these signals, StockSense aims to provide a more accurate short-term forecast of stock price movements.
How we built it
- Data Collection: We gathered stock market data from Yahoo Finance, including price history, fundamental metrics, and options activity.
- Feature Engineering: We computed technical indicators, processed options data for insights, and applied sentiment analysis using NLP.
- Modeling Approaches:
- XGBoost was optimized using grid search to find the best hyperparameters.
- LSTM (Long Short-Term Memory) neural networks were trained for sequential forecasting.
- SVR (Support Vector Regression) was used for additional price prediction.
- A Gradient Boosting model was employed as the final layer to combine predictions from different models.
- Evaluation: Performance was measured using RMSE and a rolling backtest to simulate real-world conditions.
Challenges we ran into
- Data Noise: Short-term stock prices are highly volatile, making predictions challenging. We used wavelet denoising to mitigate this issue.
- Model Overfitting: Deep learning models tend to overfit on limited historical data. We controlled this by applying dropout layers and early stopping techniques.
- Options Data Processing: Identifying meaningful signals from options activity required filtering outliers and aggregating volume-weighted strike prices.
- Sentiment Complexity: News sentiment can be inconsistent and sometimes misleading. We experimented with different sentiment scoring methods to refine our approach.
Accomplishments that we're proud of
- Successfully integrating diverse forecasting methods into a single ensemble model.
- Using wavelet transforms to smooth out price fluctuations for better trend detection.
- Demonstrating that options flow data can provide useful predictive signals.
- Creating a backtesting system that evaluates model performance in realistic conditions.
What we learned
- Ensemble models combining multiple forecasting approaches tend to be more reliable than single-method predictions.
- Options activity, when analyzed correctly, can provide valuable insights into future price movements. Specifically, options calls tend to mirror stock behaviour
- Market sentiment is highly contextual, requiring careful preprocessing to be useful in models.
- Short-term predictions benefit from a combination of technical and fundamental analysis rather than relying solely on one approach.
What's next for StockSense
- Expanding Sentiment Analysis: Integrating social media sentiment from platforms like Twitter or Reddit.
- Exploring Transformer Models: Testing attention-based deep learning models such as Transformers for improved sequence prediction.
- Developing a Web App: Creating a user-friendly dashboard for traders to interact with StockSense predictions in real-time.
StockSense is an evolving project. We’re excited about pushing the boundaries of predictive analytics with real-time data in financial markets!
Log in or sign up for Devpost to join the conversation.