Ask Your Data

Inspiration

I'm a Physics graduate transitioning into data analytics, and I kept running into the same gap in my own work: a dashboard or query only tells you what the data says, not whether you should trust it. Real datasets are messy — inconsistent formatting, missing values, data entry errors — and that context usually lives in someone's head or a Slack thread, not attached to the data itself. DataHub's whole premise is fixing that, so I wanted to build something that actually uses that metadata rather than just displaying it.

What it does

Ask Your Data is a Streamlit app that lets you ask a question about a retail database in plain English and get back a real SQL query, run against real data, with a results table and chart. The key part: before it writes any SQL, it pulls the table and column context live from DataHub's GraphQL API — including human-written descriptions and known data-quality notes I added directly in the DataHub UI, not hardcoded into the prompt.

How I built it

• Generated a synthetic but realistic retail dataset (customers, products, orders, order_items) with deliberate, realistic messiness: 17 different spellings of 5 real regions, ~6% null customer emails, ~10% of order dates in a different format, and a batch of negative-quantity data entry errors.
• Loaded it into Postgres, then pushed the schema into DataHub via the REST emitter API and documented the messy columns directly in the DataHub UI.
• Built a Streamlit agent that queries DataHub's GraphQL API at runtime for schema + descriptions, feeds that as context to an LLM (Llama 3.3 via Groq) alongside the user's question, and executes the generated SQL against Postgres.

Challenges I ran into

The most persistent one wasn't the AI part at all — it was Windows Smart App Control silently blocking compiled Python packages (sqlglot, then numpy) with no clear error message, which took real debugging through raw tracebacks to diagnose and fix. I also hit an OpenAI billing wall partway through and had to swap the LLM provider to Groq's free tier mid-build, which turned out to be a useful lesson in keeping the LLM call decoupled from the rest of the logic.

What I learned

The most interesting result wasn't a success, it was a genuinely useful failure: when I asked "what are total sales by region," the agent generated a technically correct query that still fragmented the same region into multiple rows, even though the data-quality note about inconsistent region spellings was sitting right there in the context it was given. That's a real, honest demonstration of the gap between an agent being grounded in metadata and an agent actually using it well, and it's exactly the kind of blind spot a human analyst (or a stricter prompt / validation layer) needs to catch.

What's next

Adding a validation step that checks generated SQL against known data-quality notes before running it, and writing query results back into DataHub so the metadata graph gets smarter with use, not just read from.

Built With

• datahub
• docker
• groq
• llama
• postgresql
• python
• sql
• streamlit