GPT Journal: An LLM long-term memory system based on TiDB

GPT Journal is a long-term memory system for large language models, leveraging TiDB for efficient semantic data storage and retrieval. Make LLM recall any conversation you made at any time.

Comment

About the Project

Inspiration

The idea for GPT Journal was born out of frustration with large language models (LLMs) unable to remember past conversation. We wanted to use a database system to extend LLM's memory by storing conversation and bring relevant content to the current dialogue.
This approach is particularly beneficial for daily interactions (like AI girlfriend!), where conversations can be difficult to label and track. By utilizing TiDB's vector search functionality, we can quickly access the most relevant past conversations, making the LLM more personalized and user-friendly.

How We Built the Project

  • Chrome Extension: Our main product is a chrome extension. The extension has two functionalities, store and search. To store the current conversation, the extension captures text from web pages and sends it to the backend. To search for relevant content, the extension parses the text in the prompt bar and sends it to the backend, then retrieves the result.

  • Backend (Django): For storing, we built a Django API that processes text data into embeddings by sentence transformer and stores them in a TiDB vector database. For searching, we embed the text and retrieve vector search result in TiDB.

  • Dataset: We use the DailyDialog dataset to test the functionality of the application.

Why use TiDB

  • TiDB Serverless provides powerful vector search. which is a necessity for retrieving relevant dialogue.
  • TiDB Serverless is serverless, meaning that we only need to focus on the product itself instead of figuring out how to build a db
  • TiDB Serverless is scalable, this means the extension can be used by as many people as possible as long as they have a TiDB account.

Future

  • Optimization: Doing some optimization on user interface, server side logic
  • Exposure: try to make more people use the product.
  • Earn some MONEY: talk to someone with money about our product, see if they could hire us

GitHub Repository

You can find the code for GPT Journal on GitHub.

Built With

Share this project:

Updates