Agent Muse

Your agent your terms

Comment

Inspiration

Agent chat apps treat the agent as the product. We wanted the opposite: the user's world as the product, with agents as interchangeable guests inside it. Claude Code, Grok, and whatever ships next month should all sit in the same chair — same tools, same guardrails, same transcript fidelity — without any of them owning your data, your credentials, or your phone. And it had to be permanent, daily-driver software: real persistence, real security posture, real fault isolation — not a demo that falls over when one MCP server hiccups.

What it does

Agent Muse is a native macOS app (Tauri: Rust + React) that hosts any ACP-speaking agent as a first-class session. Around that core:

  • Composers & Muses — curated agent personas with scoped tools, docs, and models; subagents are first-class entities one agent can delegate to, with permission prompts proxied to the parent chat.
  • A governed MCP gateway — every MCP server connects behind one hub with per-backend policy, tag-scoped tool access, audit logging, OAuth with reactive refresh, elicitation dialogs, sandboxed MCP-Apps widgets, and fault isolation: one broken server can never take down the rest.
  • Discovery builtins — models can list/read MCP resources, templates, and prompts even when their harness exposes no affordance for them.
  • EDITH — a phone companion over a zero-install embedded tailnet node (libtailscale compiled into the app). Your phone mirrors live sessions, replays history, and gates dangerous tool permissions — approve from your pocket.
  • Forensic transcripts — every ACP frame both directions is persisted; restore, inspector, and the phone mirror are all pure functions of that wire record.
  • A built-in mail engine, skills repository, EventKit integration, and a queryable log system with content redaction by default.

How we built it

Rust workspace (~15 crates) + React frontend, strongly typed end-to-end with ts-rs generated bindings — one schema, no drift. The MCP side rides rmcp with copy-on-write config reloads so live sessions survive edits. The phone link is our own EMP protocol over WebSocket on the embedded tailnet node, with P-256 pairing proofs and a byte-pinned golden-fixture contract keeping Rust and Swift in lockstep. The transcript layer is built on a transport-level wire tap: capture every JSON-RPC line before dispatch, persist it, and derive everything else from it — so live view and restored view can't disagree by construction. We work architecture-decision-first: every settled debate is an ADR, and every invariant that matters is pinned by a test.

Challenges we ran into

  • Protocol churn. We started on a pre-1.0 ACP SDK, vendoring a fork just to send cancellation. Migrating to SDK 1.3.0 meant absorbing a whole new programming model — the trait we implemented and the subscribe() hook our persistence rode were simply removed. We rebuilt capture at the transport layer and proved old transcripts decode forever with an 85-frame golden corpus of real production data, byte-frozen.
  • Streaming reassembly. Making live chat, restored chat, and the phone mirror render identically from chunked, interleaved, sometimes id-less agent output took three iterations and ended with render-time grouping by (messageId | turn, subtype) — one shared function, pinned by tests.
  • macOS realities. Hardened-runtime entitlements that fail silently, keychain ACLs invalidated by every dev rebuild, App Store sandbox vs. an app that spawns agent processes — each one a small war.
  • The little ones that bite hardest: a dependency's log-level prefix match writing 18 GB/day of logs; a disk that filled mid-build; a JSON key-ordering flip that nearly broke our byte-frozen fixtures.

Accomplishments that we're proud of

  • A full ACP 0.10 → 1.3.0 migration — ~1,600 lines through the two hottest files — landed with 1,186/1,186 tests green, zero clippy warnings, and provably identical rendering of every pre-migration transcript (no history migration needed: we proved the wire format additive, empirically and by schema diff).
  • The zero-install phone mirror: no Tailscale install required — the app IS the tailnet node, and machine identity is owner-only and backup-excluded.
  • Fault isolation that holds: any MCP backend can crash, hang, or lie without disturbing its neighbors.
  • Security posture as default: secrets keychain-only, logs length-redacted () at every subscriber, raw content confined to the local event store, SSRF-guarded icon fetches.
  • Byte-frozen contract testing across three languages (Rust <-> TypeScript <-> Swift) that has repeatedly caught drift before users could.

What we learned

Protocols in the agent space move fast — the survival strategy is owning your own envelope: translate at the boundary once, and your UI, persistence, and phone never chase upstream churn. Golden fixtures of real production data beat clever unit tests for wire compatibility. Capability honesty matters: advertising what you don't implement breaks agents in ways that take days to diagnose. And "the agent harness is the bottleneck" — specs advertise capabilities models can't actually reach, so bridging spec-to-model (our discovery tools) is where real usability lives.

What's next for Agent Muse

  • Live-smoke and ship the ACP 1.3.0 stack, then track the v2 draft as it stabilizes (1.3 is the on-ramp — several v2 concepts already landed).
  • Per-composer log attribution to match the per-backend granularity we have today.
  • Backend task proxying the moment rmcp ships client-side task APIs, and richer MCP Apps surfaces.
  • Refresh the embedded tailnet node on a stable-tag cadence, iOS head from spike to shippable, and multi-device EDITH.
  • Deeper Muse authoring: shareable personas with pinned tool scopes, docs, and skills — your agents, your rules, portable.

Built With

Share this project:

Updates