Financial Agentic RAG

An LLM-powered Agent acting as an assistant to financial analysts, to support in answering questions from internal data and or otherwise searching from the internet as a backup.

Comment

Inspiration

This project is inspired by our experience building RAG chatbots and agents for our clients. We wanted to challenge ourselves into building a smart assistant for financial analysts to quickly query and analyse financial reports and filings from public companies with SEC.

What it does

It is an agentic self-query rag chatbot that runs query on authoritative sources (SEC 10-Q and 10-K forms- comprehensive report of a company's business and financial performance) with a fallback to online search if the data does not exist. But it can generally be applied to any internal / proprietary data that the clients want to use as the authoritative source.

How we built it

  • First, we acquired the SEC edgar 10-Q and 10-K forms in markdown format for all companies that filed in Q1 2024 via the python-edgar library. We ingested the files into Databricks volume and split them into chunks semantically (via markdown headers and recursive character text split). The chunks, together with the metadata (form type, filing date, company name, ticker symbol, exchange, industry) were ingested and indexed by vector search with databricks-bge-large-en as the embedding model.
  • A naive retrieval method would not work for our use case as it could not filter for specific company / ticker / industry before running the similarity search. Hence, we utilised a more advanced self-query retriever which can extract filters from the user query on the metadata of stored documents and to execute those filters. Hence, we could ask question like what's the revenue of alphabet? and an llm will construct the query together with a filter on the company name. We utilised the databricks-dbrx-instruct chat model as the llm as it supports self-querying in Langchain.
  • In the event that there are no documents retrieved from the vector search (out of scope, or updates delay), we wanted the chatbot to fall back to online search. Hence, we built an agent via Langgraph, which runs the question through the self-querying llm rag chain (from the previous step), and determine if the question has been answered. If the question is not answered, it will use an online search (duckduckgo) tool to retrieve relevant search results, before generating the final answer. This is a cyclical flow and will be repeated if the question is not answered. We utilised azure openai gpt4 chat model as the agent llm.
  • Finally we deployed the agentic chatbot in a Gradio chat interface.

Challenges we ran into

  1. The initial vector search delta sync index creation took very long (>24 hours due to the documents/chunks volume) and failed to run after the wait. We resolved it after reducing the number of documents (edgar forms) by filtering for companies in tech industry.
  2. We could not save the Langgraph compiled model / runnnable to Unity Catalog registry as Langgraph compiled model is not supported by MLflow yet.

Accomplishments that we're proud of

We are happy to see the chatbot able to return coherent and relevant answer from the internal data (edgar forms), and failing which, fall back to the online search tool to query for relevant results. We are also extremely satisfied with the performance and capabilities of the databricks platform that enabled us to effectively utilise the instruct-tuned model and vector search.

What we learned

We learnt tremendously about the end-to-end development and delivery of an LLM RAG chatbot in the databricks platform. We are also impressed with the data engineering (pipeline, catalog and delta live) capabilities that made the development of such data-driven app a breeze.

What's next for Financial Agentic RAG

We plan to scale it to all industries and have a regular scheduled pipeline to ingest the edgar forms into a Delta Lake table and have it synced with the vector search index Delta Live Table to serve the most relevant and timely queries. We also plan to improve the Gradio chat interface and return the agent's message with the sources (retrieved document) and associated metadata.

Built With

  • databricks
  • edgar
  • gradio
  • langchain
  • langgraph
  • openai
Share this project:

Updates