DataCrew

Inspiration

India has many open datasets, but most are messy, inconsistent, and hard to use directly for AI. We wanted to make public data cleaner, safer, and ready for responsible AI use.

What it does

DataCrew converts messy public datasets into clean, trusted AI-ready datasets. It checks missing values, duplicates, wrong data types, inconsistent columns, and generates an AI-Readiness Score with a quality report and cleaned export.

How I built it

We used Python with Pandas and NumPy for cleaning and feature engineering, and Matplotlib/Seaborn/Plotly for visual insights like distribution plots, correlation heatmaps, and ranking charts. The workflow loads data, validates it, cleans it, and outputs final results.

Challenges I ran into

Handling large files efficiently, fixing inconsistent formats across files, preventing divide-by-zero errors, and keeping the pipeline fast while still producing meaningful analysis.

Accomplishments that I'm proud of

A complete end-to-end pipeline, clear decision-ready summaries (Top states/districts), visual dashboards, and reproducible outputs that can be reused as open-source.

What I learned

Data quality directly impacts AI accuracy. Documentation and reproducibility matter as much as analysis. Responsible AI needs privacy awareness and clean metadata.

What's next for DataCrew

Build a simple web app dashboard, add automated dataset documentation (dataset card), support PDF table extraction, and integrate AIKosh-ready export formats.

Built With

• advanced
• but
• correlation-plots
• dashboards/graphs)
• exporting)-numpy-(math-+-statistics-calculations)-matplotlib-(basic-graphs:-bar
• github
• grouping
• histogram
• hosting
• interactive
• line)-seaborn-(heatmap
• merging
• open-source
• optional
• optional)
• plotly
• project
• python-(main-programming-language)-jupyter-notebook-/-google-colab-(to-run-+-show-proof)-pandas-(data-cleaning
• recommended)
• version
• visuals)