Pocket Producer

An AI memory agent for songwriters: hum an idea, and it remembers, reconnects, and rescues it. Nothing is lost.

Comment

Inspiration

I write songs, and my Voice Memos app is where they go to die. At last count: 147 recordings with names like "New Recording 52" and "chorus idea maybe??". I once spent an evening looking for a melody I hummed in March. Never found it. Probably never will.

Tools today either want to write the song for me (no thanks) or just record more stuff into the same black hole. What I actually needed was something that remembers what I've made and tells me when two ideas belong together. That's what I built.

What it does

You hum something into your phone — on a train, in bed, wherever. Gemini 3 Flash listens to the audio itself (not a transcript of it) and tags the emotion and structure. Takes a few seconds.

Then the interesting part happens. A Memory Agent embeds the idea with voyage-3 and runs $vectorSearch against everything you've ever captured, through the MongoDB MCP Server. In my own data it connected a hummed train rhythm to a piano sketch from four months earlier — 0.847 cosine similarity, two ideas I'd never have linked myself.

A Producer Agent then scores each project (richness, structure, coherence, freshness), notices when a dormant project shares DNA with an active one, and proposes a merge — with its reasoning written out in plain sentences. I can approve, edit, or ignore it. I delete tags it got wrong all the time, and it re-evaluates without complaining.

There's also a Creative DNA page. Mine says I'm a melancholy singer-songwriter who only works between 11 PM and 1 AM. I didn't tell it that. It's right.

How I built it

Google ADK runs the agent side: a Catcher layer for capture and tagging, then Memory and Producer agents for the actual reasoning. All vector search and memory reads go through the MongoDB MCP Server into one Atlas database — fragments, embeddings, decisions, everything in one place. FastAPI backend on Cloud Run, Next.js PWA on Vercel, Firebase for auth. The five domain skills (music-tagging, rescue-scoring, relationship-rules, and friends) are split out as open-source ADK skills, Apache 2.0.

Challenges

Short musical fragments are terrible search queries. "da da dum, like a heartbeat slowing down" embeds very differently from a clean lyric, and early versions matched everything to everything. Most of my tuning time went into thresholds and into having the Memory Agent classify the relationship between fragments instead of trusting raw similarity.

The other hard part was restraint. My first Producer Agent merged projects on its own, and it felt awful — like someone reorganizing your notebook while you sleep. Now it only proposes, and every proposal comes with its reasoning. That one change made the whole thing feel like a collaborator instead of a tool.

What I learned

The memory layer turned out to be the product. Cross-time connections and project bridges aren't features I wrote — they fall out of keeping everything in one searchable store. Also: if an agent can't explain a decision in one sentence, users (me included) won't trust it with two.

What's next

Audio-signal similarity to complement the semantic search, shared memory for co-writers, and DAW export. The numbers so far, for what it's worth: 42 real fragments over three weeks, organized into 9 projects, none of them grouped by hand.

Built With

Share this project:

Updates