Metachat

• 

  file_system in Microsoft Fabric

• 

  lakehouse set-up in Microsoft Fabric

• 

  Medallion Architecture in Microsoft Fabric

• 

  Notebooks in Micorosft Fabric

• 

  Pipeline Set-up in Microsoft Fabric

• 

  Chatbot in Azure Open AI Studio

Inspiration

With the rise in data usage, quick access to metadata is increasingly important. We set out to make metadata easily accessible and actionable by delivering it through intuitive, chat-style interactions.

What it does

Metachat allows users to ask questions and gain immediate insights from metadata about their data sources, all while keeping the underlying sources secure and undisclosed.

How we built it

We used Microsoft Fabric, OpenMetadata, and Azure services, including AI search and OpenAI for language models, to build a chat interface that seamlessly integrates and presents metadata insights.

Challenges we ran into

Getting more data for our use case - we would have loved to get data lineage data which would help in showing the capabilities of metachat. Setting up the correct configurations for AI search

Accomplishments that we're proud of

Making sure that we are able to run Metachat as an AI data cataloguing tool Linking source data to fabric, transformation and cleaning it

What we learned

Linking the medallion data architecture with pipelines Using AI search and sourcing data from fabric

What's next for Metachat

Finding a way, we can link data lineage with data users Having more details on owner information Making sure that all data changes are tracked and stored Having a dynamic way to extract all the tables Training the model on variety of data

Built With

• azure
• microsoftfabric
• openai