Monolith

Shared coding agent context powered by RLMs. One Context, Run Everywhere.

Comment

Inspiration

RLMs are an elegant solution to the problem of context rot that let LLMs easily scale to millions of tokens, basically for free. Monolith lets you take advantage of RLMs for coding agents by giving each repo its own single chat history across devs working on the repo. This gives the agent a native understanding of your previous sessions and insights, such as design decisions, from other devs' sessions without you doing anything.

What it does

Monolith is implemented as a plugin to any coding agent that supports MCP tooling, which allows it to query and interact with the shared context via the RLM harness described in the original paper. One context, run everywhere.

How we built it

We used a couple of key technologies:

  • Modal serverless functions
  • Modal sandboxes
  • Modal volumes
  • OpenAI API
  • Cloudflare Workers & Containers

For some quick background, the RLM paradigm exposes the context as an object in a REPL environment for the root LM to interact with and inspect with sub-LMs to avoid polluting the main context. We implement the root LM with a Modal serverless function, which can provision sandboxes for the sub-LMs to do analysis on chunks of the context. The context is exposed as a file system in a Modal volume, which is mounted to all sandboxes.

Challenges we ran into

Existing RLM implementations are very barebones, so we had to do a lot of engineering to make things work in the cloud on Modal. Designing the system was also an interesting problem. Initially, we wanted to store the monocontext in object storage (S3), however, we found that this was not a good primitive for slice and append heavy workloads, which is what our use case demanded.

Accomplishments that we're proud of

Getting the whole thing to actually work by 4 am-ish.

What we learned

A lot about designing systems, agentic harnesses (this is what the "RLM" is at the end of the day), and Modal.

What's next for Monolith

We don't know if this is THE way to implement a massive mutable shared context, maybe the best implementations just converge to some kind of database. However, we think it is an interesting idea that has potential. We envision Monolith eventually incorporating git history (commit messages, PR comments, etc.) into the context and having the option of automatic initialization of Monolith into GitHub repos.

Built With

Share this project:

Updates