-
-
architecture card
Message CanvasMind on Telegram like a studio assistant: send a logo, a reference photo or a link, ask "animate the café logo in my style", say "slower" or "👍 love the calm pace", ask "that sunrise logo from last week?". Everything you send is filed in a library on your own machine, described by Nemotron VL so you can find it by meaning later. Every 👍/👎 updates a readable style profile, and every new animation follows it, with a line saying which of your preferences it used. Every Monday it sends a short motion-design digest, picked from the web for your style, with one of your logos animated after a trend it found.
The agent is Hermes Agent running inside an NVIDIA OpenShell sandbox, installed with NVIDIA NemoClaw. It can't read your disk or hold your API keys: it reaches your library only through a small MCP server on your machine, over your private Tailscale network, and every model call runs on Nemotron via Nebius Token Factory.
What it does
- Files what you send. Photos, stickers, SVG logos, MotionLang animations and links go into
~/CanvasMind/on your machine. Nemotron 3.5 Super VL writes a title, caption, tags and palette for each one; duplicates are recognised. A studio scan also files anything new in folders you choose (for example your SketchDynamics exports), hourly, without spending a token when nothing changed. - Finds it again. "Something with wind", "everything for the café rebrand", "what I filed last week": full-text search plus Nemotron 3 Nano scoring each candidate, with thumbnails back in the chat.
- Animates in your style. Send or pick an SVG and describe the motion (or let it propose one). Nemotron VL names the logo's parts from a numbered render; Nemotron 3 Super writes a MotionLang animation, our declarative motion language, which a validator checks and repairs; the result comes back as a GIF that plays in the chat and is filed as a new version.
- Learns from your reactions. "Slower" retimes the animation instantly; other changes ("less bounce") go through Nano edit ops. 👍/👎 with a reason is recorded in a journal;
style.jsoncounts the verbs, easings and lengths you like and dislike, and Nano keeps a few notes in your own words ("avoids bouncy", "prefers hand-made"). The planner gets that summary on every new animation. - Works while you don't. A Monday digest: Nano writes search queries from your style profile, Tavily searches the web (it only ever sees those queries), Super picks three finds with a "why it fits you" line and cites only real sources, and one of your SVGs is animated after a trend.
- Your data, your call. Forget an item (its files and any feedback on it go, and the profile is recounted) or export everything as a zip.
Inspiration
Mood-board apps keep your references on someone else's server and never act on them. We wanted the opposite: a personal assistant that lives with your files, remembers your work across tools, and does something with it in your own taste. Our two other hackathon projects gave us the pieces: MotionLang, a motion language Nemotron can write and a validator can check (SketchDynamics), and the habit of verifying model output with programs rather than trusting it (MotionHarness). NVIDIA's NemoClaw/OpenShell stack gave us the "private and always on" part: a sandboxed agent that can only reach what you allow.
How we built it
- Agent: Hermes Agent (Nous Research) in an NVIDIA OpenShell sandbox, installed and managed by NVIDIA NemoClaw on a MacBook (Apple silicon, Docker). Inference goes to Token Factory through NemoClaw's "Other OpenAI-compatible endpoint" option, so the key lives in OpenShell's provider store, never in the sandbox.
- canvasmind (Python, ours): a library (files on disk, SQLite with FTS5), the MotionLang animation and style code, and a Streamable HTTP MCP server bound to localhost.
tailscale servegives it an HTTPS address on the user's private tailnet, which OpenShell accepts as a trusted private host; OpenShell injects the bearer token at egress and restricts the agent to MCP methods. - Getting files across the sandbox boundary: OpenShell caps an MCP request at 128 KB and a model can't put image bytes in a tool call, so a small script in the sandbox uploads attachments in chunks over the same MCP route (with tools hidden from the model). GIFs come back as MCP image content, which Hermes forwards to Telegram.
- Models (all on Nebius Token Factory):
| Job | Model | | --- | --- | | Hermes agent (chat, tool use, cron turns) | Nemotron 3 Super 120B | | Caption and tag filed items; name a logo's parts and propose an animation | Nemotron 3.5 Super VL (reasoning off, ~2 s) | | Plan MotionLang animations (structured output); write the digest | Nemotron 3 Super 120B | | Re-rank search results; style notes; digest queries; edit-op tweaks | Nemotron 3 Nano 30B (reasoning off, ~1–2 s) |
- Setup as code:
canvasmind hermes-setupinstalls the skill, a CanvasMind persona, two Hermes cron jobs and a small Hermes plugin into the sandbox, and sets Hermes's own options; it's rerunnable after a NemoClaw rebuild.canvasmind demo-resetrebuilds a seeded library for demos. - Tests: 29 tests with a scripted fake model cover filing, recall, chunked upload, the MCP server (auth, hidden tools, GIF content), animation, tweaks, style, forget/export, the studio scan and demo fixtures.
Challenges we ran into
- Making a sandbox reach the host. OpenShell rejects
localhost,host.docker.internaland stdio MCP servers; it wants HTTPS with a trusted certificate. Tailscale'sserveplus NemoClaw's--trusted-private-hostwas the clean answer, and it keeps the library off the public internet. - One model per sandbox. OpenShell routes every inference call from the sandbox to the model chosen at onboarding, so the agent can't use Nemotron VL for photos itself. We turned that into the design: the agent never sees images; canvasmind describes them on the user's side.
- Hermes under NemoClaw: tool search hid our MCP tools from the model (it searched the web for "current wind speed" instead), MCP resources made it report an empty library, and the persona file is read from
~/.hermes/runtime/, not~/.hermes/. We fixed each in config, in the persona, or on the server side. - Two Telegram adapter bugs:
.svgfiles sent as documents were dropped as "non-image data", and GIFs from tools were sent as still photos (then as unnamed bytes). A small Hermes plugin keeps SVGs and sends local GIFs as animations; Hermes loads the adapter under two module names, so it patches both. Reported in our feedback. - Photos can't be animated: redrawing a photo as an animatable vector was poor with every approach we tried, so CanvasMind animates SVG artwork only, and keeps photos as references.
- Cost of an agent turn: Hermes's own prompt and tool schemas cost 63–74k prompt tokens per turn at first; a trimmed toolset brought a typical turn to ~23–35k.
Accomplishments that we're proud of
- A personal agent that is private by construction: sandboxed, keyless, reaching the user's files only through an 8-tool MCP door on their own network.
- A style profile you can read, edit and trace back to your own 👍/👎, that visibly steers the next animation.
- Proactive work that costs nothing when there's nothing to do (the hourly scan runs no model).
- Everything verified by programs: the MotionLang validator checks every animation before it reaches the chat.
What we learned
- Agent frameworks are mostly prompt: Hermes's persona, tool exposure and image routing mattered more than our tool code for whether the right tool got called.
- "Private" is an architecture, not a setting: deciding which side sees images, keys and files shaped every component.
- Small, fast models carry most of the product: Nano does recall, notes, queries and edits in a second or two; the big model only plans.
What's next
- Running on any Linux VM (Nebius, or any other provider) for true always-on use; the setup is the same script.
- Feeding the style profile back into SketchDynamics, so the canvas app and the assistant share one taste.
Log in or sign up for Devpost to join the conversation.