• 

  DataHub SQL Copilot interface demonstrating AI-powered SQL generation from DataHub metadata using an interactive Streamlit application.

Inspiration

What it does# DataHub SQL Copilot

Inspiration

Writing SQL is a common task for data analysts and data engineers, but understanding an unfamiliar database is often the hardest part. AI models can generate SQL, but they frequently produce incorrect queries because they don't know the actual database schema, relationships, or business context.

We wanted to solve this problem by giving AI access to trusted metadata from DataHub before generating SQL. Instead of guessing table names or columns, the assistant uses real metadata to create more accurate and reliable queries.

What it does

DataHub SQL Copilot allows users to describe the data they need in plain English.

The application:

• Reads metadata from DataHub.
• Understands available tables, columns, and relationships.
• Uses an AI model to generate SQL based on the real schema.
• Returns production-ready SQL together with a short explanation.

This approach reduces hallucinations and helps developers write correct queries much faster.

How we built it

The project was built using:

• Python
• Streamlit for the web interface
• DataHub as the metadata platform
• OpenAI API for SQL generation
• GitHub for version control

The workflow is simple:

1. The user enters a natural language request.
2. The application retrieves metadata from DataHub.
3. The metadata is combined with the user's request.
4. The AI generates SQL using the provided context.
5. The generated SQL is displayed to the user.

Challenges we ran into

The biggest challenge was ensuring that the AI generated SQL using the actual database schema instead of making assumptions.

Another challenge was selecting the most relevant metadata without overwhelming the language model with unnecessary information. We focused on providing only the tables and columns related to the user's request.

Accomplishments that we're proud of

• Successfully integrated DataHub into the SQL generation workflow.
• Built a clean and simple interface that is easy to use.
• Reduced SQL hallucinations by grounding AI with metadata.
• Demonstrated how DataHub can improve AI-powered developer tools.

What we learned

This project reinforced the importance of metadata in AI applications. Large language models become much more reliable when they are given structured context instead of relying only on their internal knowledge.

We also learned how DataHub can serve as the foundation for intelligent developer tools by providing trusted information about enterprise data.

What's next

Future improvements include:

• Support for multiple SQL dialects.
• Automatic query validation before execution.
• Direct integration with databases to preview query results.
• Support for dbt model generation and pipeline creation.
• Team collaboration features and query history.

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Generate accurate SQL using DataHub metadata and AI.

Built With

• ai
• datahub
• docker
• github
• metadata
• openai
• python
• rest
• sql
• streamlit