Telepathy

• 

  Telepathy

• 

  Inside the Model

• 

  Architecture

Inspiration

Teams of AI agents are used for hard jobs. One agent plans, one checks the plan, one improves it, and one writes the answer. They pass work to each other by writing notes, and every agent has to read all the notes written before it.

That gets expensive. Anthropic reports that its own multi-agent research system uses about 15 times more tokens than a normal chat, and says these systems only pay off when the task is valuable enough to cover the cost (anthropic.com/engineering/multi-agent-research-system).

We wanted to see how fast the cost grows with team size. We ran the same 50 maths questions with teams of 2, 4, 6 and 8 agents on NVIDIA Nemotron 3 Super, through Nebius Token Factory. Tokens read per question went from 416 to 12,019, about 29 times more. Accuracy stayed flat: 92%, 92%, 96% and 96%, which is within noise at 50 questions.

You also notice it as waiting. In one recorded maths question, a four-agent team needed 68 seconds. On our coding tasks, the average was 188 seconds per task.

So we asked what would happen if the agents did not write notes, and passed on what they were thinking instead.

What it does

Telepathy lets the agents in a multi-agent system hand over their working memory instead of notes.

While a language model reads and thinks, it builds up a memory of what it has understood so far. This is called the KV cache. Normally that memory is thrown away, and the next agent has to rebuild it by reading text. Telepathy keeps it and passes it to the next agent.

The planner, critic and refiner think silently and write nothing. The solver reads on top of their shared memory and writes the answer. All four are the same model with the same job prompts as in the usual flow. Only the hand-off is different.

Telepathy also handles untrusted content. Our code marks web pages as untrusted. Only one agent may read them, and it passes on what it found as plain text instead of through the shared memory. In our research demos that text is checked for planted instructions. Nothing from a web page travels hidden inside the memory.

The website replays real recordings. One question is answered both ways on a single clock, so you can watch the usual team and Telepathy side by side. You can pick from featured maths, coding, science and web research questions. The results section shows the time and tokens for every question we tested.

How we built it

Challenges we ran into

An API cannot pass on a model's memory. Telepathy works by handing the KV cache from one agent to the next, and Token Factory returns text only. The hackathon credits covered Token Factory, so we ran Telepathy on a free Kaggle T4 and used Token Factory for what an API can do: the usual multi-agent flow at 2 to 8 agents, and a run on the bigger model. We never compare seconds across the two models. Every comparison between a single agent, the usual flow and Telepathy uses the same Nano 4B, the same GPU and the same questions.

Web pages can hide instructions, and shared memory cannot be read. In a normal agent team you can read the notes and scan them. Shared memory is numbers, so an instruction planted in a web page could pass to the next agent unseen. We made the code, not the model, decide what is untrusted. One agent reads web content and passes on what it found as plain text, produced on a copy of the memory so the page's own tokens never enter the shared one. In our research demos that text is checked for planted instructions.

Accomplishments that we're proud of

Results. Same model, same GPU, same questions, three ways of working. Telepathy compared with the usual four-agent flow:

• School maths, 150 questions: single agent 79.3%, usual flow 69.3%, Telepathy 72.7%. Telepathy was 4.3 times faster and wrote 78% fewer tokens.
• Coding, 60 tasks graded by running tests: single agent 43.3%, usual flow 43.3%, Telepathy 55.0%. Telepathy was 40 times faster (4.7 seconds against 188.4) and wrote 98% fewer tokens (55 on average against 2,723).
• Medical exam, 60 questions: single agent 33.3%, usual flow 33.3%, Telepathy 41.7%. Telepathy was 3.4 times faster and wrote 74% fewer tokens.
• Web research, 3 questions (a demo, not a benchmark): usual flow 2 of 3, Telepathy 2 of 3. Telepathy was 2.9 times faster and wrote 60% fewer tokens.

On coding, Telepathy solved 7 problems the single agent missed and missed none that the single agent solved (sign test, p = 0.016).

Every number can be checked. We tested 273 questions in all. Each one has a saved row with the answer, the time and the tokens, and the raw files are in the repo. The README has commands to re-run the tests on a free Kaggle GPU. 112 automated tests pass.

We tested our own idea and kept the results that went against us. Details are under "What we learned".

It runs on small hardware. Everything above ran on one 16 GB Tesla T4.

What we learned

• Most of the saving comes from not writing and rereading notes. In a self-test on 50 maths questions, Telepathy got 35 right. With no silent thinking it got 37, and when the silent agents thought about a different question it got 32. These differences are within noise, so we do not claim the silent thinking makes answers better. We claim speed and cost. The self-test was on maths, and repeating it on coding is our next experiment, because coding is where Telepathy's accuracy gain was largest.
• Most accuracy gaps are small. Maths: 109 right for Telepathy against 104 for the usual flow, out of 150 (p = 0.54). Medical: 25 against 20, out of 60 (p = 0.38). Coding against the usual flow: 33 against 26, out of 60 (p = 0.12). Only the coding result against the single agent is significant.
• A team does not always help. On school maths, one agent alone beat both teams.
• A bigger model helped more than anything else we changed. The usual flow went from 69.3% on Nano 4B to 91.3% on Nemotron 3 Super, so we expect Telepathy to gain from a bigger model too. Testing that is our next step.
• A readable "subtitle" of the silent thinking did not work. We tried to show a plain-English version of what each silent agent passes on, and could not verify it was faithful, so we removed it.

What's next for Telepathy

• Run Telepathy on a Nebius AI Cloud GPU, so the whole project runs on Nebius.
• Run it on bigger NVIDIA Nemotron models, where we expect higher accuracy for every way of working.
• Compare against a usual flow where agents pass short summaries instead of full notes.
• Repeat the "different question" self-test on coding, to find out where the coding gain comes from.

Built With

• fastapi
• github
• gsm8k
• hugging-face-transformers
• huggingface
• kaggle
• kv-cache
• latentmas
• mbpp+
• medqa
• multi-agent-systems
• nebius-token-factory
• nvidia-nemotron
• nvidia-tesla-t4
• python
• pytorch
• react
• tailwind-css
• tavily
• typescript
• vite