DataMinors

Inspiration

Air pollution is one of those problems where knowing the current number is not enough. A PM2.5 reading tells you what the air is like right now, but people and organizations also need to know what is likely to happen next and what they should do about it.

We wanted to build something that could move from simply reporting air quality to helping people make decisions. This led us to HackInsight: an AI-powered system focused on PM2.5 forecasting and decision support for Pakistan, particularly Lahore and Karachi.

What it does

HackInsight combines environmental sensor data and weather data to forecast PM2.5 levels 24 hours ahead.

The system:

Processes and validates large-scale PM2.5 sensor data.

Builds a trusted gold dataset from raw environmental data.

Uses a machine-learning model to forecast future PM2.5 levels.

Provides prediction uncertainty rather than presenting a forecast as absolute fact.

Compares model performance against a persistence baseline.

Uses a Data Quality Agent to detect and handle data-quality issues.

Uses an Insight/Decision Agent to turn numerical forecasts into understandable and actionable recommendations.

Automates the pipeline using Prefect and presents the results through an interactive dashboard.

Our model achieved a test MAE of 40.8 µg/m³ compared with 42.3 µg/m³ for the persistence baseline, representing a 3.63% improvement, while the 80% prediction interval achieved 82.4% empirical coverage.

How we built it

We designed HackInsight as an end-to-end data-to-decision pipeline:

Data Ingestion → Cleaning & Validation → Data Quality Agent → Gold Dataset → ML Forecasting → Insight Agent → Dashboard

We used Python, pandas, NumPy, scikit-learn, DuckDB, Streamlit, and Prefect, along with custom Python-based AI agents.

The data pipeline separates raw, cleaned, and processed/gold data so that results remain traceable and reproducible. The forecasting model uses a chronological train/validation/test split to avoid leakage and is evaluated against an untouched test set.

The AI components are designed around trusted project data rather than allowing an LLM to freely generate numerical claims. This helps keep the system grounded in the data used by the forecasting pipeline.

Challenges we ran into

One of our biggest challenges was dealing with the complexity and quality of environmental data. Sensor datasets can contain missing values, invalid measurements, duplicates, inconsistent timestamps, and other issues that can affect downstream predictions.

We also had to make sure our model evaluation was scientifically meaningful. A forecasting model can appear accurate if information from the future accidentally leaks into training, so we used chronological splits and an untouched test set.

Another challenge was turning a technically complex ML pipeline into something understandable to a normal user. We didn't want the dashboard to simply display another number. We wanted users to understand the forecast, its uncertainty, and what it means for their decisions.

Finally, building the entire system under hackathon time constraints required us to combine rapid AI-assisted development with testing, validation, and careful review.

Accomplishments that we're proud of

We are proud that HackInsight became more than just a machine-learning model.

We built an integrated system with:

176,392 validated records

30 sensors across Lahore and Karachi

A reproducible environmental data pipeline

Automated data-quality validation

A 24-hour PM2.5 forecasting model

Uncertainty estimation with 82.4% empirical coverage

A measurable improvement over the persistence baseline

Agentic AI for data quality and decision support

Automated orchestration and run logging

An interactive dashboard that turns predictions into actionable insights

Most importantly, we built a system designed around the complete journey from data → prediction → explanation → decision.

What we learned

This project taught us that building an AI system is not just about choosing a powerful model.

Data quality matters as much as model complexity. A sophisticated model cannot compensate for unreliable inputs.

We also learned the importance of rigorous evaluation. Beating a simple baseline on a genuinely held-out chronological test set is much more meaningful than reporting a high score from a random split.

Most importantly, we learned that AI becomes much more useful when it is connected to a complete workflow. Forecasts become valuable when they are accompanied by uncertainty, explanations, trusted data, and actionable decisions.

What's next for DataMinors

Our next step is to make HackInsight more real-time and geographically comprehensive.

We want to integrate continuously updated sensor feeds and live weather forecasts, expand coverage to more Pakistani cities, provide more granular station- or neighborhood-level predictions, and introduce automated alerts when forecasted pollution reaches unhealthy levels.

We also want to make the system more proactive by allowing its agents to investigate unusual pollution patterns, identify potential contributing factors, and provide more personalized recommendations.

Our long-term vision is to turn HackInsight into a real-time environmental intelligence platform for Pakistan — helping citizens, organizations, and decision-makers understand not just what the air is like today, but what is coming next and how they can respond.