The problem with how AI listens

Every AI system today runs on a request-response loop. You produce a complete input. Then it thinks. Then it acts. Voice makes that loop painful because speech is slow and the pause after you stop talking is dead time. But the deeper problem is that the loop locks in mistakes. By the time a model acts on "tell Esha I'll see her at Tuesday's meeting," the wrong day is already in the draft. Nothing in a request-response system can catch an error while you're making it, because nothing is running while you're talking.

People don't work like that. A friend who knows your calendar cuts in at the word "Tuesday." They were already thinking about the meeting while you were mid-sentence. I didn't think a better assistant would close that gap. The loop itself had to change.

What Ahem does

Ahem is speculative execution for conversation. It treats your sentence as a stream, not a message. On every partial transcript it searches everything it knows about you, forms a running belief about what you want, and commits to a guess before you finish. If you diverge, it rolls back. If what you're saying contradicts what it knows, it cuts in, out loud, at the word where the contradiction happens.

Say "Draft a message to Esha saying I'll present the VQE results at Tuesday's group meeting." At "Tuesday" Ahem says "Heads up, the VQE meeting is Monday at 8:30, not Tuesday. Say Monday?" You say yes. The corrected draft is already staged, because a coding agent started on it at the word "Esha."

Everything runs on my real Google Calendar and Gmail, imported read-only. Every change is a diff on a git branch. It never acts without asking, and "no" means it never happened.

Why voice, and why this form

I chose voice because it is the interface where the request-response loop hurts most. Typing lets you see and fix your mistake before you hit enter. Speech doesn't. The only way to catch an error in speech is to catch it while it's being made, and that forces the whole architecture to run continuously instead of once per turn. Voice is the hardest case, so it's the honest test.

It's also the interface people reach for when they can't use a screen. Driving, cooking, walking between classes, or living with a disability that makes typing hard. The HUD exists so judges can watch the belief form. You don't need it to use Ahem.

How I built it

Cartesia's Ink 2 streams the transcript, with a cumulative update about every 250 ms and an early end-of-turn signal that arrives 100 to 350 ms before the turn formally ends. That signal is the trigger for committing the speculation, and I would have built turn detection worse myself. Sonic 3.6 over the streaming WebSocket gives 68 to 85 ms to first audio, which is the difference between an agent that cuts in and one that merely responds. I went to the models directly rather than Cartesia's managed agent platform because the whole product lives inside the turn, so I had to own the stream and the turn logic.

Moss is the only data store. My calendar, contacts, and email are indexed once and loaded inside the server process. Each partial transcript runs one to four queries at about 2 ms each, roughly 50 per sentence. I measured the alternative. Through a simulated cloud vector store at 300 ms a query, the same correction lands 5 to 6 seconds after you've stopped talking. In process, it lands 13 to 20 ms after the word is transcribed. There's no database behind Moss and no cache in front of it, and conversation memory goes back into the same index, so "What did I tell Esha?" works a minute later and after a restart.

Factory's Droid does the real work. It's triggered mid-sentence when the belief tracker's confidence crosses 0.75, with no chat interface anywhere. Phase one opens exactly one file and plans. Phase two, fired on the early end-of-turn signal, makes the edit. Every plan step, tool call, and diff streams to the screen as it happens. Confinement is layered because I let a coding agent loose on my actual calendar. Droid's sandbox denies reads outside the workspace, the tool allowlist is five tools, a watchdog aborts on any path outside the workspace, and the child process gets a scrubbed environment with no API keys. Each speculation is a git branch. Yes fast-forwards main. No deletes the branch.

Claude Haiku turns each partial plus the top retrieval hits into intent, entities, and claims as strict JSON.

Challenges

Droid's Read tool bypassed its permission handler, so confinement became sandbox plus allowlist plus watchdog plus environment scrubbing, and I verified it with a canary secret two directories up. Staging took 25 to 39 seconds until I split the calendar into one file per event and collapsed Droid's three round trips into one, which brought it to 8 to 14 seconds from belief lock, most of it overlapping speech. Spoken "yes" was unreliable until it was matched on the first words of the next turn instead of going through the intent model. Corrections leaked the original word into drafts until the corrected value replaced it in every slot.

What I learned

Retrieval latency decides the architecture, not just the speed. Below about 10 ms you can afford to query on every partial, and the request-response loop stops being necessary. Above it, you're back to waiting for the user to finish.

Accomplishments

The interjection lands 13 to 20 ms after the wrong word. It runs on my real calendar and inbox. 500 tests across server, web, and contracts. Built alone.

What's next

Real send and calendar write-back behind the same confirm flow. An earbud form factor, since the screen is already optional. Schedule conflicts as a second kind of interjection.