Inspiration
I wanted to explore how machine learning models could improve predictive accuracy by leveraging different strengths from each model.
What it does
The project predicts the 5-day return and price for selected equities using a combination of CatBoost, LightGBM, and a neural network. The model takes in historical stock data and generates forecasts based on learned patterns. The final prediction is obtained using a weighted ensemble of the three models, where CatBoost contributes the most, followed by LightGBM and then the neural network.
How we built it
I started by collecting historical daily stock data from January 1, 2015, to February 21, 2025. The dataset includes features such as last price, open price, high price, low price, volume, moving averages (20, 50, 200 days), RSI (14-day), volatility (30-day), implied volatility, VWAP, and other market indicators.
- Data Preprocessing: Missing values were handled using rolling window averages, and numerical features were standardized using MinMax Scaling for compatibility with the neural network.
- Feature Engineering: Lagged features were created to capture temporal dependencies, and the 5-day future return was defined as the target variable.
- Model Training: Three models were trained separately:
- CatBoost, which is optimized for handling categorical and time-series data.
- LightGBM, which is designed for large-scale datasets and efficient training.
- A neural network with two hidden layers to capture non-linear patterns.
- CatBoost, which is optimized for handling categorical and time-series data.
- Hyperparameter Tuning: I used Optuna to optimize parameters such as learning rate, tree depth, and regularization for the boosting models, while adjusting hidden layer sizes, learning rate, and epochs for the neural network.
- Ensembling: The final prediction was computed as a weighted combination of the three models:
Final Prediction = (0.6742 × CatBoost) + (0.2760 × LightGBM) + (0.0498 × Neural Network)
Challenges we ran into
One of the biggest challenges was handling missing data and feature selection. Some features, such as market order bid size and market order ask size, were mostly blank, requiring careful preprocessing. Another challenge was tuning hyperparameters effectively, as financial data is highly volatile, and overfitting needed to be minimized. I also considered using intraday data to improve predictions but couldn't incorporate it in time.
Accomplishments that we're proud of
I was able to implement an ensemble learning approach that combined multiple models to improve predictive accuracy. The hyperparameter tuning process helped extract the best performance from each model, and automating data preprocessing, feature engineering, and training made the system scalable for future improvements.
What we learned
I learned that ensemble models generally perform better than individual models by leveraging their unique strengths. CatBoost contributed the most to the final predictions, indicating that gradient boosting models handle stock market data well. The R² scores varied significantly, showing that some stocks are harder to predict due to their volatility.
What's next for Yash project
The next step is to refine the current approach by testing it on new stock data and iterating based on performance feedback.
Final 5-day Predictions
- ALT US Equity: Predicted price = $7.03
- CELH US Equity: Predicted price = $31.01
- CVNA US Equity: Predicted price = $241.02
- FUBO US Equity: Predicted price = $3.49
- UPST US Equity: Predicted price = $71.35
