Listening Delta

An AI-assisted listening experience that turns a first observation into a deeper second listen.

Comment

Inspiration

Many AI products are designed to generate content, provide answers, or complete tasks for us. I wanted to explore a different direction: using AI to cultivate attention.

Listening is often treated as passive, but a small question can completely change what we notice. Listening Delta began with a simple idea: instead of asking AI to interpret a sound for the listener, could AI help the listener hear more for themselves?

The project reflects my broader interest in creating the conditions for perception rather than replacing human judgment.

What it does

Listening Delta is a short, two-pass listening experience.

First, the user listens to a brief sound and writes what they noticed.

GPT-5.6 then generates a personalized follow-up question based on that specific observation. The question does not score, correct, or evaluate the response. Its purpose is to guide the user toward a second, more attentive listen.

Before listening again, the user can also observe the sound through a real-time stereo X-Y vectorscope. It visualizes the changing relationship between the left and right audio channels without interpreting the sound for the user.

After listening again, the user records what changed.

The experience presents both observations together as a “Listening Delta,” making the shift in perception visible.

Instead of asking AI to think for us, Listening Delta asks AI to help us notice more.

How we built it

Listening Delta was built as a lightweight web application using:

  • Next.js
  • TypeScript
  • OpenAI Responses API
  • GPT-5.6
  • Web Audio API
  • Vercel

GPT-5.6 is integrated in a server-side API route. The user’s first observation is sent to GPT-5.6 via the Responses API, which generates one concise follow-up question grounded in that observation.

In addition to the language-based interaction, the application includes a real-time stereo X-Y vectorscope built with the Web Audio API. It does not analyze the listener or generate interpretations. Instead, it visualizes the relationship between the left and right audio channels, offering an additional way to observe the sound itself.

This design places a language-based prompt alongside a non-interpretive visual representation, allowing both to support observation without evaluating the listener’s perception.

I used Codex to simplify the experience into two listening passes, build the GPT-5.6 follow-up-question flow, refine the interface and product copy, verify the model integration, run lint and production build checks, and confirm the production deployment.

The application is deployed on Vercel, and the source code is available under the MIT License.

Challenges we ran into

The main challenge was making the generated question specific enough to be useful without making it leading or overly interpretive.

Questions that closely reflected the listener’s observation could easily become too long or explanatory, while shorter questions often became generic. The prompt therefore had to balance grounding, brevity, and openness.

The prompt and output constraints focused on three goals:

  • remain grounded in the user’s observation
  • ask one concise, non-evaluative question
  • encourage a meaningful second listen without suggesting an answer

Another challenge was reducing the product to its essential interaction. Earlier versions contained more explanatory copy and additional interface elements. Through iteration, the experience became simpler: listen, observe, receive a question, listen again, and reflect.

I also needed to verify that the production route explicitly used GPT-5.6, that the model was correctly passed to the OpenAI Responses API, and that no secrets or private environment values were exposed in the repository.

Accomplishments that we're proud of

I am proud that Listening Delta gives AI a deliberately limited but meaningful role.

The system does not attempt to identify the correct interpretation of a sound. It does not grade the listener, summarize the experience, or replace their judgment. It generates only one grounded question and then returns attention to the listener.

I am also proud that the project was reduced to a clear and functional experience, deployed to production, documented, and released as open-source software.

The final interaction demonstrates that a carefully constrained AI can deepen human perception without replacing it.

What we learned

I learned that AI does not always need to produce the final artifact or answer.

It can also create a productive pause, introduce a new perspective, or direct attention toward something the user may have overlooked.

I also learned that strong constraints can make an AI interaction more meaningful. By limiting GPT-5.6 to one grounded, non-evaluative question, the role of the AI became clearer and the listener’s contribution became more important.

The development process also showed me how Codex can support not only implementation, but also product simplification, copy refinement, verification, documentation, and deployment preparation.

What's next for Listening Delta

The current version uses a single listening example to keep the experience focused and demonstrate the core two-pass interaction. The next step is to expand Listening Delta to additional sounds while preserving its minimal structure.

Possible directions include:

  • offering different kinds of sounds and listening exercises
  • allowing users to revisit past Listening Deltas
  • exploring visual, spatial, and environmental observation
  • supporting facilitators, educators, artists, and listening workshops
  • studying which kinds of follow-up questions produce the most meaningful changes in attention

The central principle will remain the same: AI should not replace perception, but help create the conditions for deeper observation.

Built With

Share this project:

Updates