Nodus — a tiny local planner that makes agentic loops free (and observable)
The demo is a real production workflow: a 324M local planner reads a film script, plans the tool steps, a Gemini-on-Vertex executor writes the shoot-day brief and delivers it to Cloud Storage, and Grafana Cloud traces every step — one command, end to end.
Every agentic system pays a hidden tax: before it can do anything, a frontier model burns tokens just to plan — and for simple tasks, planning can cost more than the work itself. Nodus replaces that planning phase with a 324M-parameter transformer, fine-tuned on a single job: turn a natural-language task into an ordered sequence of tool names. No arguments, no replanning. The harness fills the details, executes, and verifies.
Because the planner is tiny and local, planning is free, private, and fast — your code never leaves the machine, and the planning cost stays flat however long the task grows. Every step of the run is streamed live to Grafana Cloud as an annotation, so an agent session becomes something you can replay, correlate, and audit.
The results are real: on a fresh holdout of 120 tasks — brand-new entities the model never saw in training — Nodus plans 119 correctly (99%, zero regressions). End-to-end, the plan significantly improves an executor (sign test, p = 0.0139), with the effect growing as plans get longer. Both arms — plan and no-plan — ran under the same deterministic guardrails, so the effect isolates the planner's contribution from the harness code.
Agentic Cinema compliance: the judged demo path uses only Google Cloud AI and Grafana Cloud MCP. Generative execution and slot-fill run on Vertex AI Agent Builder (vertex:gemini-2.5-flash, official google-genai SDK, vertexai=True, ADC). Observability uses mcp-grafana. Cloud Storage delivers the media artifact. No OpenAI, Anthropic, AWS, Microsoft, Ollama, or OpenRouter in the submission — enforced at runtime via NODUS_HACKATHON=1. See AGENTIC_CINEMA.md on GitHub: https://github.com/Rua987/nodus/blob/main/AGENTIC_CINEMA.md
The local 324M component is a non-generative tool-name planner (fixed 8-tool vocabulary, our PyTorch weights). It does not call third-party LLM APIs. All generative execution in the demo runs on Vertex AI.
The whole pipeline was validated live with one command — a real Vertex executor, a real gs:// upload ([real]), and real Grafana annotations (live run, exit 0). Demo video: https://youtu.be/pG2VAql7aog
Inspiration
Agentic loops are expensive twice over: planning burns frontier-model tokens, and re-planning makes loops slow and hard to audit. We wondered how much of that cost is actually necessary. A 324M model can't write code well — but "what tools do I need, and in what order?" is a much simpler question than "write the code." Train a small model on exactly that question, and the big model is left with only what it's good at: executing.
What it does
Plans — NODUS (the 324M planner) reads a task and outputs an ordered JSON array of tool names from a fixed 8-tool vocabulary. Step arguments are filled per step by a Vertex/Gemini LLM prompt.
Executes — the harness fills arguments, runs each tool, and verifies results against guardrails. The Agentic Cinema submission runs on Vertex AI — Google Cloud Agent Builder only (vertex:gemini-2.5-flash). Slot-fill and the ReAct loop use the same Vertex/Gemini backend. No third-party LLM APIs in the judged demo path.
Delivers — the media workflow (demo task media) reads a script, writes a shoot-day brief, and uploads to Google Cloud Storage via gcs_upload (mock-first without credentials; real upload with ADC).
Observes — every event is pushed to Grafana Cloud through mcp-grafana as a tagged annotation (live run, exit 0).
How it's different:
- Architecture: decoupled local planner (tool names) + Vertex/Gemini executor (arguments and execution).
- Planning cost: one small local forward pass instead of repeated frontier API calls.
- Reliability: 99% plan accuracy on a fresh 120-task holdout (unseen entities, 0 regressions).
- Observability: Vertex AI + Cloud Storage + Grafana Cloud MCP.
Try it:
Mock (no creds): python demo_agentic_cinema.py --task-key media
Live (judges): python demo_agentic_cinema.py --live --model vertex:gemini-2.5-flash --task-key media
Repo: https://github.com/Rua987/nodus Landing: https://rua987.github.io/nodus/
How we built it
The planner — 324M-parameter transformer (24 layers, 768 embed), SFT on 20k planning examples, ~3.5 GPU-minutes on a rented RTX 5090. Outputs only tool names.
The harness — ReAct loop, 8 native tools, guardrails for systematic failure classes.
Google Cloud — judged executor on Vertex AI (Agent Builder) via google-genai (vertexai=True, ADC). gcs_upload delivers artifacts to Cloud Storage. Mock-first without credentials.
Observability — Grafana Cloud MCP (mcp-grafana); each event becomes a create_annotation call.
Slot-fill — per-step Vertex/Gemini prompts extract argument targets; hardened JSON parsing.
Validation — 994 passing tests; live end-to-end run confirmed through Google Cloud and Grafana APIs.
Challenges we ran into
Generalization vs memorization — a fresh holdout (zero entity overlap) exposed real weaknesses the old holdout had hidden.
Fixed tool vocabulary — Grafana and GCS are harness capabilities, not planable tools.
Mock-first cloud — no credentials in repo or CI; [mock] markers keep the demo honest.
Geo-blocked Gemini API — pivoted to Vertex AI (ADC), the Agent Builder path of the brief.
Org policy blocking service-account keys — user OAuth ADC (gcloud auth application-default login).
Windows console encoding — UTF-8 output with safe fallback for timeline glyphs.
Accomplishments that we're proud of
99% plan accuracy on 120 never-seen tasks (0 regressions).
Statistically significant e2e impact — p = 0.0139 sign test.
30-second mock demo; real 324M planner when checkpoint is present.
Full Google Cloud stack live in one command — Vertex, GCS [real], Grafana annotations.
Live landing page: rua987.github.io/nodus
994 tests passing.
What we learned
Model plus guardrails is what ships — not raw plan accuracy alone.
Never trust a contaminated holdout.
Observability makes agents auditable when every step is a tagged annotation.
A 324M planner keeps planning flat-cost as tasks grow.
What we built during the hackathon
Nodus grew out of months of prior LINUS research. The hackathon period added:
Media workflow demo (shoot-day brief from a film script).
Google Cloud validated live — Vertex executor and real GCS upload.
Grafana Cloud live via mcp-grafana.
Shippable artifacts — landing page, demo video (https://youtu.be/pG2VAql7aog), release v1.0.0 weights.
Eligibility hardening — AGENTIC_CINEMA.md, NODUS_HACKATHON=1 guard, README/landing updated.
What's next for Nodus
Deeper slot-filling in the local planner.
Wider task distributions.
Edge deployment of the 324M planner.
Stronger Vertex/Gemini executors with the same A/B protocol.
Log in or sign up for Devpost to join the conversation.