Resident

A human-agent record store where an agent digs, plays and curates in full view—while the listener stays in control.

Comment

Inspiration

Most browser-agent experiences happen behind the scenes. A user asks for something, waits, and receives a result without seeing what the agent searched, what it considered, or when control changed. Resident explores a different model: the agent joins the music interface instead of operating behind it.

What we built

Resident grew out of my existing 3D digital record crate at https://sephmartin.com/. For the WebMCP Challenge, I extended that same environment rather than building a separate demo. The original experience remains at https://sephmartin.com/, while the challenge build lives at https://demo.sephmartin.com/.

The result is a meaningful page-level WebMCP extension, not a separate wrapper. The agent can orient over the catalog, search and rank releases using conservative metadata-only Song DNA, focus and inspect records, play and navigate previews, move between tracks and releases, manage a reversible local My Crate, and prepare checkout.

The interface makes the agent's work legible. Agent Mode, Thinking, Digging, Listening, Returning and Human Mode are visible states; the orb mirrors the active release and operation; and the crate, player and detail panel update together. The same page remains usable by a person through pointer, keyboard and touch input.

How WebMCP is implemented

The adapter feature-detects document.modelContext and keeps a compatibility fallback for navigator.modelContext. It registers narrow tools with explicit schemas and read-only or UI-mutation annotations. The tools call a small page facade exposed by the existing crate runtime rather than reaching into arbitrary DOM nodes. A local browser diagnostic log records the tool sequence, visible state and failures so the interaction can be inspected during a demo.

The Song DNA layer is evidence-bound: it uses catalog fields such as tags, descriptions and track titles, labels text-derived matches, and reports unavailable audio analysis instead of inventing BPM, key or texture. Stable record IDs keep agent actions reliable across sorting and browsing.

Human control and safety

The human remains in the loop. A pointer, keyboard or touch gesture can reclaim the page immediately and return it to Human Mode. Crate changes are reversible. Checkout is intentionally split into review and confirmation: the on-site challenge simulator records no real payment, and the final download step ends at a copyright-safe DEMO COMPLETE boundary rather than delivering an original audio file.

What I learned

I don’t come from traditional software engineering; I come from music, where experience, rhythm and emotion matter as much as functionality. AI gave me a medium for turning that perspective into software. I didn’t get into AI to write code. I got into it to build experiences that didn’t exist before.

I built Resident iteratively with coding agents on a standard ChatGPT Plus subscription. I brought the product vision, music knowledge and interaction design; the agents helped me test, break and refine the implementation until it matched what I had imagined.

The original 3D crate was already a long-running creative project. The challenge gave me the spark to extend it into a shared human-agent space. WebMCP is valuable here beyond button automation: it lets an agent operate inside a stateful, visual application while the person can see, interrupt and review the work.

The strongest interaction is not a one-shot recommendation; it is the visible loop from natural-language intent to digging, focus, playback, curation and human handoff.

Try it

Open the live demo in ChatGPT's in-app browser, ask the agent to start the curator session, request a late-night disco or house selection, play and inspect a result, add it to My Crate, and prepare checkout. The purchase is a no-payment demo flow. If audio does not start immediately, click or tap the page once: browser autoplay policies require a user gesture before sound can play. A human gesture during Agent Mode demonstrates immediate priority.

Built With

Share this project:

Updates

Submission history