Inspiration
The team has been severely understaffed with a growing number of data projects. It is imperative for us to develop an intelligent data access platform with intuitive tools for non-technical users reducing dependency on the data team. It needs to be capable of handling basic tasks (e.g. data queries) freeing the team to work on more complex tasks.
Inspired by K-2SO of Star Wars, Kaytoo is our version of a data droid. Developed with the purpose of assisting and augmenting the data team.
What it does
- Query - it can query internal data sources and help users find relevant information
- Code Interpreter - it leverages on the capabilities of Open AI's Code Interpreter to generate code and process files.
How we built it
We used the following technologies:
- LangChain - To create agents, tools and enable us to switch LLMs later on if needed
- OpenAI - To enable prototyping
- Google Chat API - To make it easy for our end users to interface with the agent we built.
- Databricks Serverless SQL - To enable the query capability on our internal data sources
- Azure Data Lake - Storage for data sources
- Azure Web App - To host our application
We created a LangChain agent that utilizes OpenAI's assistant API. We provided the agent with additional tools such as the query tool. The query tools translates user inquiries about our data into SQL statements. These SQL statements are then run on a Serverless SQL endpoint. The results are returned and sent back to the user.
Challenges we ran into
- Data Security and Privacy - Safeguarding sensitive information and ensuring compliance with data protection regulations poses a significant challenge. It needs to be equipped with robust security measures to prevent unauthorized access and data breaches.
- Data Quality and Consistency - The quality of data affects what is being returned by the queries.
- Model Serving with Databricks - We tried to implement model serving but was encountering issues with MLflow.
Accomplishments that we're proud of
- Integration with Google Chat - Making it available on the company's internal messaging app makes it accessible to everyone within the organization.
- Creation of Tools - This enables us to expand Kaytoo's capabilities by providing it more tools in the future.
What we learned
- We learned the difference of making our models available through model serving compared to deploying it on an azure web app. Several changes are needed before it can be made available on Databricks but it looks promising as logs are automatically stored there and experiments could be monitored better.
- We learned how Assistants API and LangChain can make prototyping LLM based apps faster and immediately valuable.
What's next for Droids
- Integration with Google Sheets and Google Drive - while the utility classes and functions were already available. The tool did not make it in time for submission. It is exciting to see what Kaytoo would be capable of with integrations on google sheets and google drive.
- Product Activations - this is another time consuming task that can be eventually be automated.
- Feedback and Suggestions - we can integrate gathering feedback and suggestions on the app
Log in or sign up for Devpost to join the conversation.