Germinate AI

• 

• 

• 

• 

• 

• 

• 

  From the Guo et al survey paper

What it does

Germinate AI is an experimental framework for distributed LLM-based multi-agent systems (LLM-MAS).

(More details here: https://germinate-ai.github.io/germinate-ai/)

Inspiration

Andrew Ng's short talk at Sequoia on the potential of multi-agent LLM systems.

• The MetaGPT paper https://arxiv.org/abs/2308.00352
• Multi agent LLM systems survey paper (Guo et al) https://github.com/tjunlp-lab/Awesome-LLMs-Evaluation-Papers
• Apache Airflow
• FastAPI and web frameworks in general

Motivation

I'd been meaning to mess around with/look into the internals of distributed systems like Map-Reduce and Apache Spark.

NATS looked like an interesting light weight tool to build distributed systems around.

The schematized dialogue/dataflow between role-playing agents in the MetaGPT paper (https://arxiv.org/abs/2308.00352) and using LLM workflows to write code for my projects seems like a especially worthwhile idea to implement a distributed system for.

Challenges we ran into

There's a lot to do to do justice to the idea I chose, which has a very broad scope.

Accomplishments that I'm proud of

Adapting Apache Airflow DAG specification internal DSL to specify parallel LLM based workflows. Langchain's LCEL is nice but I also like the "mathematical" terseness of Airflow's DAG spec

task1 >> [task2, task3] >> task4

A simple internal DSL for defining FSM transitions that AFAIK no-one else is using.

(state1 & condition1) >> state2

Composing task DAGs within state machine states -- which gives enough expressiveness to implement complex iterative workflows. (Unfortunately this part is kind of close to the finish line, but not quite there.)

Implemented a simple and easy (multi-process or multi-node) parallelization with topologically ordered DAG execution. Built a flexible Task executor protocol with DI and state transition conditions evaluated on worker nodes as Tasks as well.

Read a bunch of papers, and read a lot of langchain internal code.. and actually managed to understand some of it :).

What I learned

Creating a distributed system is hard!

It's quite easy to read some bleeding edge research on LLMs (specifically LLM based multi-agent systems in my case) and start coming up with all kinds of ideas of experimental and application related things you could do with it (with varying levels of usefulness and hidden difficulty).

What's next for Germinate AI

There's a lot of development that has to take place before its ready for actual use.

Built With

• docker
• langchain
• langgraph
• nats
• postgresql
• python
• weaviate