Echoform

Your voice. Better understood.

Comment

Inspiration

I grew up in a community where hearing multiple languages in a single day is normal. Mandarin, Hindi, Spanish, family names from across the world, foods that do not translate neatly into English, and words carried between generations are part of everyday conversation. Yet whenever those same words enter a voice assistant, automatic captioner, or speech-to-text system, they often become something else entirely.

What bothered me most was how generic transcription repeatedly mangles the names, places, foods, technical terms, and heritage words that appear in recordings. Echoform reverses that relationship by learning how speech recognition misunderstands you.

What it does

Echoform is a personalized speech-recognition layer for the words that matter most: names, places, foods, heritage-language words, and other vocabulary that conventional voice systems repeatedly misunderstand. A user teaches Echoform a word with two short examples—first saying it alone, then using it naturally in a sentence—so the system can learn both how the recognizer mishears it and the contexts in which it appears.

Later, Echoform preserves the original speech-recognition transcript and independently evaluates possible corrections using previously observed ASR mistakes, approximate phonetic similarity, and contextual evidence. Rather than rewriting everything aggressively, Echoform abstains when the evidence is weak. A baseline like “I ordered a crescent this morning” can become “I ordered a croissant this morning,” with a Why this changed panel showing exactly why the correction was made.

How we built it

Echoform is built with React, TypeScript, and Vite and requires no backend or hosted speech-recognition API. Whisper runs locally in the browser through Transformers.js inside a Web Worker, using WebGPU when available and a local WASM fallback otherwise. Recordings are decoded, converted to mono 16 kHz audio, and processed locally; they are never uploaded to a speech-recognition server.

On top of Whisper, I built my own deterministic personalization engine. Echoform maintains a local lexicon of intended words, observed ASR confusions, approximate phonetic representations, and contextual evidence, then rescores possible corrections while enforcing safeguards against overcorrection. Missing evidence is excluded rather than fabricated, context cannot authorize a correction by itself, and all personalized data stays in IndexedDB on the user's device.

Challenges we ran into

My biggest challenge was realizing that making Echoform correct more often was not the same as making it better. If a name had once been misheard as a common English word like “shopping,” blindly memorizing shopping → [name] could cause Echoform to change the word even when someone genuinely meant shopping. We repeatedly attacked our own system with cases like this, eventually introducing independent evidence families, an evidence gate, and a form-anchor requirement so Echoform can choose not to intervene.

Another challenge was that Whisper does not mishear the same pronunciation consistently. The same word might become several different outputs across recordings. Instead of treating this as a failure, we built a confusion neighborhood: Echoform can compare a new ASR mistake not only with the intended word, but also with previous ways the recognizer misunderstood it. What began as one of our biggest problems became one of the project's most interesting mechanisms.

Accomplishments that we're proud of

I am proud that Echoform doesn't use LLMs. Every decision comes from an inspectable system: which candidate was considered, what evidence was available, what was missing, why the correction passed or failed, and why Echoform chose to act or abstain.

I am equally proud that privacy and evaluation are built into the product rather than added as marketing claims. Speech recognition and personalization happen locally, and Echoform can continue transcribing after the model has loaded even when the computer is offline. We also built an evaluation mode that reports actual target-term recovery from user-recorded trials instead of displaying a fabricated accuracy percentage.

What we learned

Building Echoform changed how I think about personalization. At first, personalization sounded like simply remembering what a user wanted. But a genuinely personalized system also needs to know when not to interfere. A system that remembers your name but starts replacing unrelated words is not personalized—it is just overconfident.

I also learned that recognition mistakes themselves contain useful linguistic information. When a speech recognizer repeatedly transforms one word into several related outputs, those mistakes reveal something about how the system is perceiving that speaker. Instead of discarding errors, Echoform learns from them. More broadly, the project reinforced why I care about language technology in the first place: a word can be rare in a dataset while being incredibly important to the person saying it.

What's next for Echoform

The current version deliberately prioritizes transparent, conservative text-based personalization. The next major step is trustworthy acoustic evidence: comparing a new pronunciation with locally stored reference pronunciation features so Echoform can distinguish cases where transcript text and context alone are insufficient, while still refusing to pretend approximate audio alignment is more precise than it really is.

I also want to expand support for multilingual and code-switched vocabulary, improve our lightweight phonetic representation, and let personal lexicons become richer over time. The larger goal goes beyond fixing a single name or word: everyone has vocabulary that technology understands unusually poorly and that they care about unusually deeply. Echoform is an attempt to make speech technology learn that vocabulary instead of teaching people to speak around its limitations.

Built With

Share this project:

Updates

Submission history