Vocal Tract simulator

Inspiration

Most people never learn why a voice sounds the way it does. The answer is the vocal tract: a tube of air, with a movable tongue, jaw, lips and larynx, that filters the buzz of the vocal folds into vowels and timbre. Spectrograms and textbooks explain it, but you can't touch them. As a singing teacher, I wanted a place where my students can move the anatomy and hear and see what changes.

What it does

Vocal Tract Simulator is an interactive, bilingual (English/Spanish) source-filter model of the voice. It loads with a sample voice already playing, so there is nothing to set up.

• Sliders for jaw, tongue body, tongue tip, lips, larynx, pharynx, nasalization, epilaryngeal constriction, pitch and vocal register (M1/M2).
• Live feedback in four views: a sagittal (side) cut of the tract, an F1/F2 vowel map, the harmonic spectrum, and the sound itself.
• An optional formant sweep with the microphone to estimate your own vowel formants, with a voice-type selector (male, female, child) that settles doubtful peaks. Results are saved in your browser and any single vowel can be repeated or corrected by hand.
• A sung-note reader: sing a vowel and the app measures pitch, vowel and formants.
• An expert mode to record paired measurements, one control at a time, and export them.
• An AI voice tutor (NVIDIA Nemotron 3 Ultra, running on Nebius Token Factory). Ask in plain language ("make this /a/ sound rounder", "how do I darken my voice like an opera singer?"). The tutor reads the simulator's state, moves the controls through range-checked tools, checks the resulting formants with code, and explains the change in beginner-friendly words.
• Sources you can check. For knowledge questions, the tutor searches trusted voice-science sources through Tavily and cites real links. When it finds none, it says so and labels the rest as general knowledge. Preprints (arXiv) are only a secondary source and are marked as not fully verified.

Everything audio-related runs in the browser: microphone audio is analysed locally and never uploaded. The tutor only receives text and numbers.

How the AI is used

The rule is: code measures and decides, the model converses and plans.

• The model never invents numbers. Formants, brightness and nearest vowel come from tool results computed by the app. The system prompt forbids stating any figure that did not come from a tool, and the app flags any it cannot verify.
• Every value the model sets is clamped to the control's range.
• The tutor tells a request for a different vowel (solved numerically for target formants) from a request for a different voice quality or style ("darker", "opera"), which it handles with larynx, throat and lips and never by closing the jaw.
• Every tutor turn is a runtime call to Nebius Token Factory (OpenAI-compatible API), through a server relay that holds the keys, with per-visitor rate limits.

How I built it

SvelteKit 2 + Svelte 5 + strict TypeScript + Tailwind 4, deployed on Cloudflare Workers. Sound is generated with the Web Audio API from a glottal source shaped by the model's formant filters. The acoustic model, measurement and tutor tools are pure TypeScript modules with more than 600 unit tests (npm run validate runs type-check, tests and build).

What is new for this hackathon. The core simulator (formant equations, vowel presets, sagittal geometry, audio engine) started in a previous personal project, voicexplorer. During the hackathon I built:

• the Nemotron tutor: server relay, tool loop, range-checked tools, code-side verification of its numbers and its grounding rules;
• source search with Tavily, with real citations or an honest "no sources found";
• the microphone formant-sweep assistant and personal profile, benchmarked against LPC;
• analysis of the recorded sweep at a fixed rate, the voice-type selector, and saved results;
• the sung-note reader;
• the expert measurement mode (paired A/B takes, M1 and M2) and local export;
• separate tongue-body and tongue-tip controls, and an experimental M2 (head voice) mode;
• the Cloudflare deployment, rate limits and documentation.

Challenges

• Honesty about accuracy. I benchmarked my formant-tracking method against LPC on synthetic vowels: it is comparable to a well-configured LPC, not better. The app presents itself as an educational approximation, not a voice clone.
• An ill-posed inverse problem. One vowel can be made with many jaw/tongue combinations, so fitting sliders from a few vowels is underdetermined. I calibrated each control by isolating one factor at a time (e.g. only the jaw, same tongue and larynx).
• Keeping the model grounded. Letting an LLM move the controls is only useful if the numbers it reports are real, hence tool-only numbers and range-checking. The tutor first treated "darker" as "close the jaw", the opposite of how opera singers do it, so I taught it to tell vowel changes from voice-quality changes.
• Choosing a model. I tested five Nemotron models on Token Factory with the same tool loop. Nano and Lightning failed (empty replies, or changes not applied); Super was close but slower and less reliable; Ultra was consistent. The relay also asks again once when a model returns an empty reply.
• Octave errors in pitch. A sung 300 Hz note was read as 615 Hz when the second harmonic was the strongest. I reproduced it on 540 synthetic cases (57 wrong) and added a spectral check that brought it down to 3.

What I learned

How to integrate an LLM into a learning tool without letting it make things up: give it tools, compute every number in code, check every value it sets, and make it cite real sources or admit it has none. I also learned that models behave very differently on the same tool loop, so it is worth measuring them instead of guessing.

What's next

Source spectral tilt that depends on pitch, a source × filter panel, a voice comparator, guided exercises verified by code, and re-analysis of saved takes with a different voice type.

Attribution

Includes data from the Dresden Vocal Tract Dataset (CC0 surfaces; CC BY tables). Font: Roboto (Apache 2.0). NVIDIA, Nemotron, Nebius, Token Factory, Tavily and Cloudflare are trademarks of their owners; this project is not endorsed by them.

Built With

• canvas
• claude
• cloudflare-workers
• nebius
• nvidia-nemotron
• nvidia-nim-api
• svelte-5
• sveltekit
• tailwind-css
• typescript
• vite
• vitest
• web-audio-api