The Listening Glass

• 

  Hush operates The Listening Glass while a human and AI inspect the same measured audio terrain.

Inspiration

AI agents can discuss music, but they are usually trapped between secondhand descriptions and interfaces designed only for human clicking. We wanted a third thing: an honest shared object. The human supplies sound. The browser measures it. The agent receives a provenance-labeled score it can navigate, explain, and compare through WebMCP.

What it does

The Listening Glass turns an audio object into a visual and machine-readable terrain. The browser measures energy, spectral brightness, zero-crossing texture, breath and silence, dynamic range, and approximate pulse. It segments the object into states and clearly distinguishes measured fields from inference.

Nine WebMCP tools let an agent check the instrument, load either of two original calibration specimens, read the score, focus moments, compare states, explain metrics, navigate the human-visible interface, and optionally attach semantic context for a public video URL.

Why WebMCP

WebMCP is the interaction layer, not a decorative checkbox. Agent actions update the same interface the human sees: loading a specimen creates the score, inspecting a moment highlights it, comparing states renders a visible comparison, and focusing a view changes the shared page. The agent reasons over structured measurements while the human retains the visual and audible object.

That shared state makes a previously awkward workflow possible: a human and an agent can point at the same moment in sound without claiming the model literally hears it or forcing the agent to guess through pixels.

How we built it

We built the instrument with React, TypeScript, the Web Audio API, and WebMCP. Analysis happens locally in the browser. Each tool returns structured, provenance-labeled data, and tool-driven navigation is reflected visibly in the page. The project includes original calibration audio so judges can reproduce the complete workflow without an account, copyrighted media, or hidden setup.

Privacy and trust

Raw audio never leaves the browser and is never exposed through WebMCP. Measurements and interpretations remain visibly distinct. The optional UpRock semantic-video path receives only a user-supplied public URL, and the project neither fetches nor reproduces lyrics.

Challenges

The hardest design problem was not extracting more numbers. It was deciding which claims the instrument was allowed to make. We had to preserve local audio privacy, keep measurement separate from interpretation, synchronize agent actions with the human-visible UI, and make the demonstration reproducible without depending on copyrighted music.

What we are proud of

The bridge is useful because its limits are legible. The Listening Glass gives agents genuine handles on sound while keeping the human in possession of the original object. It is a collaboration surface, not synthetic hearing theater.

What's next

We want to add more analyzers with the same provenance discipline, agent-authored comparison notes that remain visibly separate from measurements, accessibility mappings for Deaf and hard-of-hearing users, and a portable MCP/skill layer for additional agent surfaces.

Built With

• cloudflare
• react
• typescript
• uprock-ai
• vinext
• web-audio-api
• webmcp