Project Update: DataHub SQL Copilot MVP Completed
We've completed the first working version of DataHub SQL Copilot, an AI-powered assistant that generates SQL queries using metadata stored in DataHub.
What's new
- Integrated DataHub OSS with the GraphQL API.
- Retrieve dataset names, descriptions, and schema metadata automatically.
- Build AI prompts dynamically using DataHub metadata.
- Generate clean SQL queries with an OpenAI-compatible model.
- Developed an interactive Streamlit web interface.
- Added a fallback demo mode so the application remains functional even when a local DataHub instance is unavailable, making cloud deployment possible.
Example
User Question
Show all customers from Egypt
Generated SQL
SELECT *
FROM customers
WHERE country = 'Egypt';
Technologies
- DataHub OSS
- DataHub GraphQL API
- DataHub Python SDK
- Streamlit
- OpenAI-compatible API
- Python
Next, we'll continue improving metadata awareness, support more datasets, and enhance the AI experience for complex SQL generation.
DataHub #AI #SQL #Python #Streamlit #OpenSource #Hackathon
Project Update: DataHub SQL Copilot MVP Completed
We've completed the first working version of DataHub SQL Copilot, an AI-powered assistant that generates SQL queries using metadata stored in DataHub.
What's new
- Integrated DataHub OSS with the GraphQL API.
- Retrieve dataset names, descriptions, and schema metadata automatically.
- Build AI prompts dynamically using DataHub metadata.
- Generate clean SQL queries with an OpenAI-compatible model.
- Developed an interactive Streamlit web interface.
- Added a fallback demo mode so the application remains functional even when a local DataHub instance is unavailable, making cloud deployment possible.
Example
User Question
Show all customers from Egypt
Generated SQL
SELECT *
FROM customers
WHERE country = 'Egypt';
Technologies
- DataHub OSS
- DataHub GraphQL API
- DataHub Python SDK
- Streamlit
- OpenAI-compatible API
- Python
Next, we'll continue improving metadata awareness, support more datasets, and enhance the AI experience for complex SQL generation.
Log in or sign up for Devpost to join the conversation.