Inspiration

As a data engineer, I understand the complexity involved in transforming data from transactional databases to customer-facing reports or dashboards, especially within large companies. This process can be simplified.

What it does

My solution enables anyone within the company to query data and receive tailored insights based on their specific roles and interests. That's done by looking into the metadata (as in a data catalog).

How we built it

I developed a user-friendly interface using Streamlit, underpinned by robust backend logic powered by Snowflake Arctic.

Challenges we faced

One of the main challenges was providing the right context to the model to ensure accurate outputs. It felt a lot like writing unit tests to cover all possible scenarios.

Accomplishments we're proud of

I successfully leveraged my data engineering expertise to address a very common issue I am, and the area I work at is part of.

Lessons learned

LLMs are super powerful and will definitely revolutionize how we use data for decision-making.

What's next for Data Potion powered by Arctic I am passionate about open source and plan to develop a multi-tenant Django application. While connecting data sources is a logical next step, compliance issues might be challenging. I want to enable users to upload Excel sheets as well, so it's easy for them to control the data they want to share with the app.

Built With

• arctic
• python
• streamlit