Hearsay

• 

  Synthetic demo family: same-device comparisons with uncertainty, not clinical diagnoses

"What?" "WHAT?" "...never mind."

That dinner-table exchange is Hearsay's starting point: make hearing-in-noise differences easier to explore together, without pretending a laptop and ordinary headphones form a calibrated clinical test.

Try Hearsay · Watch the demo · Direct video fallback · One-page project PDF · Source-code PDF

What it does

Hearsay is an educational self-check, not a medical device or a diagnosis. Put on headphones, set a comfortable volume, hear three spoken digits in noise, and type what you heard. A Bayesian adaptive engine chooses the next noise level to reduce uncertainty. The live belief chart shows that process rather than hiding it behind a score.

The Family board compares results collected with the same device and headphones, using a leave-one-out median and an uncertainty-aware threshold. This design reduces a shared device-offset problem under the model; it does not establish clinical validity or eliminate differences in fit, attention, language or listening environment.

Hear-through plays audio through illustrative high-frequency attenuation profiles. These are teaching examples, not a simulation of any particular person's hearing. The Lab runs reproducible studies on synthetic listeners, plus a virtual-listener walkthrough that judges can use without headphones. The Honesty page explains provenance and limitations.

How it was built

TypeScript, Vite and Web Audio run entirely in the browser, with no backend or tracking. Results stay in localStorage.

Nine spoken digit words were synthesized offline with ElevenLabs, trimmed and RMS-equalized. Locally generated speech-shaped noise uses their long-term spectrum. Digits have opposite phase between ears while the noise is identical in both channels.

The adaptive engine maintains a grid posterior over threshold and psychometric slope, selects the next signal-to-noise ratio by expected information gain, and stops using a precision criterion with a maximum trial count. The family comparison accounts for estimated uncertainty before flagging a gap.

The repository includes 32 Vitest tests covering DSP, noise shaping, mixing, clipping, scoring, adaptive procedures, family comparison, shipped audio and simulation claims.

Results: synthetic listeners, not people

In the committed seeded simulation of 400 virtual listeners tested twice, the Bayesian procedure averaged 11.8 triplets, 1.01 dB RMSE and 94% coverage for nominal 90% intervals. A 12-trial staircase had 1.53 dB RMSE; a 24-trial staircase was more precise at 0.79 dB but took about twice as long.

Those figures evaluate the algorithms under simulated assumptions. They are not clinical performance, screening accuracy or proof of benefit for real families.

Challenges and lessons

Uncalibrated headphones make absolute clinical thresholds inappropriate. The prototype therefore focuses on same-device comparison and openly labeled uncertainty rather than a normal/abnormal label.

The simulation also made a trade-off visible: fewer trials can save time, but precision and interval coverage need separate measurement. The Lab exposes both, including the stronger precision of the longer baseline.

What's finished and what's next

The interactive check, Family board, Hear-through, Lab, deployed demo and reproducible simulation are implemented. The prototype has not been clinically validated. No human-subject data was collected; the demo family is synthetic and labeled that way.

A future ethics-approved study could compare it against a calibrated clinical test. Per-digit intelligibility balancing, more languages and an exportable appointment summary are future work.

AI and asset disclosure

Devin helped implement and test code and produce documentation, screenshots and demo media. ElevenLabs generated the app’s existing digit and sentence speech assets and the original demo narration offline. The current synthetic-only demonstration uses new offline Kokoro Heart narration. The app’s speech assets are unchanged. The live app makes no ElevenLabs or other model API calls and does not use a sponsor API. Its runtime AI is Bayesian active learning, with no LLM or trained neural model.

Code is MIT licensed. Generated audio and technical provenance are documented in PROVENANCE.md.

Built With

• bayesian-inference
• elevenlabs
• github
• typescript
• vite
• vitest
• web-audio-api