Inspiration
I run more than one AI assistant: OpenClaw for work, Hermes Agent for home, and a few that came with tools I pay for. Each one is good at something, but they don't talk to each other, and each knows a different slice of my life. Workflow tools can glue APIs together, but an assistant isn't an API call. It has its own skills, its own memory and its own judgement, and it can be wrong. I wanted one place where my assistants work as a team, share only what each should know about me, and stop for my yes before anything sends, spends or books.
What it does
Huddle is n8n for personal assistants. Each block on its canvas is a whole assistant, reached through a connector: OpenClaw, Hermes Agent, any A2A agent, OpenAI-style APIs, Ollama, the agent CLIs on my computer, and agents inside NVIDIA OpenShell sandboxes. I draw who hands work to whom, and where I want a say.
Most of the time I don't draw anything. With Just ask, I say one sentence, like "Plan my work for next week, and find something fun to do on Saturday." Nemotron 3 Super reads it next to my saved flows and assistants and plans who does what. Jobs that don't depend on each other run side by side, so OpenClaw and Hermes work at the same time, and Hud, Huddle's own voice on Nemotron 3.5 Lightning, sums up what each brought back. In the run in my video, side by side took 6.1 seconds, against 14.0 seconds one after the other.
My assistants share one memory, in slices. OpenClaw sees only my work notes and Hermes only my home notes. Anything an assistant learns waits as a suggestion until I keep it, and passwords, card numbers, one-time codes and keys never go in.
Approvals are a step in the flow, not a promise in a prompt. A run that reaches one stops and waits for me, even across a restart. Before I'm asked, Huddle can check the plan's facts on the web: Nemotron 3.5 Lightning lists them, Tavily searches each one, and Nemotron 3 Super weighs the sources and quotes them. A quote that isn't word for word in its source doesn't count. In my video's run, it flagged the plan's Mysuru Palace fee of ₹50 a head, because The Hindu reports the fee is now ₹120 per person, and it caught a wrong train time too.
The Huddle button sits on my desk: an ESP32-S3 board with a screen, a mic, a speaker and a camera. I hold it and talk. Nemotron 3.5 ASR hears me and Magpie TTS answers, both running on my own laptop through NVIDIA NeMo-Speech.cpp, so only the words go on to my assistants. When something needs my yes, the button reads it aloud: a tap is no and a one-second hold is yes, so a knock never approves anything. Its screen shows which assistant has the work, two side by side when they work together. It talks to Huddle over its USB cable, through the canvas in Chrome, so it needs no Wi-Fi setup and no administrator rights.
Agents can also run inside an NVIDIA OpenShell sandbox. When one reaches for a site its policy doesn't allow yet, OpenShell drafts a rule, and the request comes to my Approvals with OpenShell's own risk check. Once I allowed api.open-meteo.com, the sandboxed agent's run fetched Bengaluru's weather in 4 seconds.
Every run ends with a receipt: where my data went, which notes went with each hand-off, and what never left my laptop. It's worked out from the run's own log, so it records what happened rather than what a setting promises.
How I built it
Huddle is TypeScript on Node 24: a Hono server with the flow engine, a React canvas built with Vite, and one SQLite file for everything. It runs from one command, one Docker container or a Linux desktop app. The button's firmware is Arduino on the ESP32-S3.
All of Huddle's own thinking runs on NVIDIA Nemotron through Nebius Token Factory, with each model sized to its job. Nemotron 3.5 Lightning routes each ask to the right assistant, lists the facts to check and sums up a team as Hud. Nemotron 3 Super plans Just ask, weighs the fact-check sources, learns from what I say and tidies memory. Nemotron 3 Ultra drafts whole flows from a sentence. Nemotron 3.5 Super VL reads photos from the button's camera, like a bill. OpenClaw and Hermes themselves run in Docker on Nemotron 3 Super through Token Factory too, and the sandboxed Hermes reaches Token Factory through an OpenShell provider profile that lets only curl use the key, and only for that host.
Huddle reads Token Factory's live prices when it starts, so the router knows what each size costs. On 24 everyday asks, it sent every one to the right size, and quick asks and messages cost 89% less than sending everything to Ultra. The benchmark and how to rerun it are in the repo. Thinking stays off unless an assistant needs it: on a routing call, Lightning took 5.8 seconds and 1,344 tokens with thinking, and 0.4 seconds and 16 tokens without.
Every model call Huddle makes on its own goes through one meter with a daily budget and a per-minute limit, so a public demo can't run up a bill.
I started Huddle on October 9, 2026. Its memory ranking is ported from Sochen, my earlier project; everything else was built for Huddle. It's open source under the AGPL-3.0.
Challenges I ran into
Nemotron thinks before it answers unless told not to, and the thinking is slow and billed. The OpenAI-style reasoning_effort setting is ignored; the switch is enable_thinking in chat_template_kwargs. Finding it took Huddle's quick calls from seconds to under half a second.
Prompts alone didn't keep the models honest, so Huddle checks in code. Nemotron ignored "keep every fact" when merging notes, so a merged note may name no person, place or number its notes don't, and must keep most of what they say. A fact-check quote must appear word for word in its source. Hud's spoken plan is built from the plan as it will actually run, so "while" means at the same time and "then" means it waits.
Learning from what I say started on Lightning, but it saved my request itself as a fact and filed notes in the wrong slices. It runs on Super now, which got 4 of 4 right for about $0.00015 a message.
Token Factory has no speech models, so the button's voice runs on my laptop's own GPU through NVIDIA NeMo-Speech.cpp. That means the hosted demo has no voice, but my voice never leaves my machine.
I wanted to use NemoClaw, but the version I could install pins an older OpenShell that can't talk to the current OpenShell gateway, so I ran Hermes inside OpenShell directly.
Accomplishments that I'm proud of
A real button on my desk that hears me, hands the work to my assistants and only says yes when I hold it. Two assistants working side by side, done in 6.1 seconds instead of 14.0. A fact check that caught a wrong price before I approved a plan. A receipt for every run, built from what actually happened. And routing that picked the right Nemotron size for all 24 test asks.
What I learned
Pick the model size per job, not per app: Lightning for routing and short answers, Super for planning and judging, Ultra only when a whole flow has to be drafted. Turn thinking on only where it pays. And anything that has to be true, like a quote, a note or a plan, gets checked in code, because a prompt is a request, not a guarantee.
What's next for Huddle
Approvals on my phone, a Telegram trigger, keys per app for the router, and a hosted version for people who'd rather not run anything. I also want the button to work fully offline with a local model, and to speak Indian languages.
Log in or sign up for Devpost to join the conversation.