Listener / Lens

Same words. Different ears. Hear how speech sounds may be recategorized across languages.

Comment 3

Inspiration

As a native English speaker, I cannot easily hear English as pure sound because meaning arrives immediately. But when I speak English around my uncle in Brazil, he can hear its rhythm and sound patterns without automatically understanding the words.

Listener / Lens began with a question: could teachers experience an evidence-informed approximation of that difference and better understand what a new language learner may be hearing?

What it does

Listener / Lens helps teachers and learners explore how speech sounds may be recategorized across languages.

The main comparison provides three tracks:

  • A — Source pronunciation: how the sentence is pronounced in its source language.
  • B — Listener approximation: an evidence-informed rendering using supported sound-category effects associated with the selected listener language.
  • C — Production comparison: a listener-language voice reading the original sentence.

Changed words are aligned visually, and an evidence receipt explains which rules were applied, absent, or could not be rendered reliably.

Sound Minus Meaning creates deterministic, meaning-opaque speech that preserves source-language syllable structure. This helps learners focus on sound without vocabulary supplying meaning first.

The prototype includes 30 languages and 835 directed language combinations. Its optional classroom tool uses GPT-5.6 Luna to turn bounded result metadata into a short teaching activity.

How we built it

A Cloudflare Worker handles request validation, rate limits, and API contracts. A Python service performs grapheme-to-phoneme conversion, applies supported listener-language rules, constructs aligned carrier forms, and produces SSML. Azure Speech renders the audio.

The classroom activity generator uses GPT-5.6 Luna through Azure AI Foundry. It receives bounded result metadata rather than the teacher’s original sentence or audio, and a curated fallback remains available.

Codex with GPT-5.6 supported the project throughout research, architecture, implementation, testing, debugging, acoustic analysis, and interface development.

Challenges we ran into

The hardest problem was separating an intended sound-category change from ordinary variation introduced by speech synthesis.

We evaluated multiple renderers and transformation architectures. Some produced robotic speech, list-like delivery, unwanted commentary, inconsistent pronunciation, or differences unrelated to the target sound. We built deterministic transformation contracts, acoustic and integrity gates, aligned comparisons, cache validation, and explicit failure states instead of silently presenting unsupported output.

Another challenge was balancing broad language coverage with honest claims. Listener / Lens therefore distinguishes registry coverage from successfully rendered evidence and describes its output as a research-informed approximation—not a recreation of anyone’s private perception.

Accomplishments that we're proud of

  • Built an end-to-end multilingual listening experience spanning 30 languages and 835 directed combinations.
  • Created aligned source, listener-approximation, and production-comparison tracks.
  • Added an evidence receipt showing what changed and why.
  • Developed Sound Minus Meaning for listening without semantic interference.
  • Integrated GPT-5.6 Luna through Azure AI Foundry for classroom activities.
  • Built extensive automated validation across the Python and Worker layers.
  • Preserved visible limitations and fail-closed behavior rather than overstating uncertain results.

What we learned

Perceptual recategorization is not the same as imitating an accent. A responsible education tool must model specific, cited sound-category effects, expose its reasoning, and identify where evidence or rendering support is missing.

We also learned that natural speech synthesis is not enough by itself. Controlled comparisons require deterministic plans, matched structure, integrity checks, and human listening alongside automated measurement.

What's next for Listener / Lens

Next steps include broader native-listener studies, stronger rule-level validation across voices and phonetic contexts, expanded renderer coverage, and classroom testing with language teachers.

We also want to add teacher-authored lesson templates, clearer practice activities for individual contrasts, and evidence-qualified support for more sound patterns while preserving the product’s transparent limitations.

Built With

Share this project:

Updates