🧵 About the Project

🎯 Inspiration

This project was born from the challenge of predicting product-level production for the Mango Datathon. With more than 90,000 training samples and 2,250 test items, the dataset combined rich categorical information (family, silhouette, season, fabric, color…) with continuous features and image embeddings.

We were inspired by a simple question:

Can we combine different modeling philosophies to build a forecasting system that is not only accurate, but also stable, interpretable, and robust across product types?

From this idea, two complementary approaches emerged:

a Stacking model (LightGBM + CatBoost), and
a Super-Ensemble that merges v3, Stacking, and Ridge Regression.

⚙️ How We Built It

1. 🔦 Stacking Model: LightGBM + CatBoost

(Document: “LGBM + CatBoost”)

The stacking pipeline uses the strengths of two gradient boosting libraries:

LightGBM → excellent for numerical features and speed
CatBoost → state-of-the-art for categorical features
Meta-model LightGBM → learns the optimal combination of both predictions

After internal train/validation, the model achieved:

MAE: 5088.93
RMSE: 8781.88
R²: 0.9343

Mathematically, the stacking layer learns:

[ \hat{y} = f_{\text{meta}}\big(f_{\text{LGBM}}(X),\ f_{\text{CatBoost}}(X)\big) ]

We also built a product explorer to inspect errors, target leakage, and where the model over/under-estimates.

2. 🧬 The Ensemble Supermodel: v3 + Stacking + Ridge

(Document: “Mango Ensemble Supermodel”)

The ensemble was designed to reduce variance and provide smoother, more controllable predictions across the catalog.

It combines:

v3 model (0.55 weight): large feature set, high stability
Stacking model (0.35): powerful but higher variance
Ridge model (0.10): PCA-based smoothing and regularization
Global boost factor: (1.08\times) to align production scale

Final ensemble formula:

[ \hat{y}{\text{final}} = 1.08 \cdot \big(0.55\hat{y}{v3} + 0.35\hat{y}{stack} + 0.10\hat{y}{ridge}\big) ]

This allowed us to:

Avoid extreme outliers
Smooth predictions while maintaining accuracy
Keep interpretability at a product level
Generate a clean, ready-to-submit CSV

We also added visual validation: distribution histograms, scatter plots (v3 vs ensemble), and per-product model contribution breakdowns.

📚 What We Learned

How stacking can exploit different feature types better than any single model.
The importance of variance control when working with forecasting models.
How ensembles often outperform highly tuned single models because they merge different “perspectives” on the data.
The usefulness of explaining predictions product-by-product to detect anomalies and justify decisions.
That sometimes the most powerful improvements come not from more tuning, but from combining models wisely.

⚠️ Challenges We Faced

Managing high-cardinality categorical variables, especially with image embeddings and fashion taxonomy.
Avoiding overfitting in high-capacity models like CatBoost and v3.
Selecting the right weights and boost factor for the ensemble without causing prediction drift.
Building a balanced validation strategy that fairly evaluates stacking and baseline models.
Keeping training times reasonable while experimenting with large model combinations.