Inspiration

Satellite-based monitoring of industrial emissions is critically hampered by cloud cover and sensor limitations, leaving critical data—most notably methane—missing in 65% of the training dataset. Furthermore, models trained on specific facilities often struggle to generalize to new, unseen sites (domain shift). As a Ph.D. in Materials Science and Metallurgy, I wanted to apply combustion thermodynamics directly to computer vision, ensuring the model respects physical laws rather than just memorizing pixel patterns.

What it does

This project introduces a multi-modal PyTorch neural network (CNN + MLP) that predicts five industrial pollutants (CH₄, NO₂, CO, AAI, FIRMS) directly from 14-band Sentinel-2 satellite imagery and meteorological data.

It features a custom Masked MSE Loss to train on incomplete data, and a Physics-Informed Penalty. If the model predicts high fire intensity but zero smoke, the loss function penalizes it, as this violates basic thermodynamic principles. The model is deployed in an interactive Streamlit dashboard that flags these physically impossible predictions in real-time.

How I built it

  • Data Pipeline: Built a custom PyTorch Dataset with an in-memory RAM caching mechanism, reducing training epoch times from 200 seconds to just 25 seconds.
  • Architecture: A late-fusion model combining a 3-layer CNN (image branch) and an MLP (tabular branch).
  • Training: Utilized 5-Fold GroupKFold cross-validation (grouped by site_key to prevent data leakage), AdamW optimizer, and learning rate scheduling.
  • Dashboard: Streamlit, Matplotlib, and Torch for real-time inference and validation.

Challenges I ran into

The biggest challenge was handling severe "Domain Shift." While local cross-validation yielded an impressive RMSE of 0.892, the unseen Kaggle test set revealed that purely visual CNNs easily overfit to site-specific textures (like soil color or building layouts). However, this led to the paper's core insight: while raw error degrades on unseen sites, the physics-informed guardrails never fail, always catching thermodynamically impossible predictions.

Accomplishments that I am proud of

Successfully bridging the gap between materials science and deep learning. The Streamlit app organically flags "Combustion Violations" and "Dispersion Violations" when fed the most extreme images in the dataset, proving that the model "knows what it doesn't know."

What I learned

That raw RMSE is not the only metric that matters. In real-world environmental monitoring, a model that flags its own physically impossible predictions is far more valuable than a model that confidently outputs garbage due to domain shift.

What's next for Physics-Informed Multi-Task Learning for Satellite Emissions

Integrating a Vision Transformer (ViT) pre-trained on Sentinel-2 data, and adding a temporal ConvLSTM component to track the dispersion of smoke plumes across consecutive satellite passes.

Built With

Share this project:

Updates

Submission history