Sound for All

Turning up the volume never helped. Sound for All makes speech clearer instead, boosting the sounds hearing loss removes first, and backing every alert with light and touch.

Comment

Inspiration

Most accessibility tools online are built for Deaf and sign-language users, captions, avatars, that whole category. That's important work, but it misses a much bigger group entirely: elderly people wearing hearing aids who can hear, just not clearly.

The reason is mechanical, not vague. Age-related hearing loss removes high frequencies first, specifically the consonants, roughly the \(1.5\text{-}4\,\text{kHz}\) band where s, f, th, and k live. Speech doesn't go silent. It goes to mumble. And turning up the volume doesn't fix that, it just makes the mumble louder. Nobody was really building for that specific, mechanical problem, so we did.

What it does

Sound for All makes speech clearer, not louder, and backs every alert up with sight and touch so nothing depends on reading fast captions or knowing sign language.

  • Clarity Boost — live mic audio gets noise suppression, a peaking EQ lift on the \(1.5\text{-}4\,\text{kHz}\) band, and dynamics compression, all in real time in the browser. There's also a file mode: upload a recording, enhance it offline, download the result.
  • 3-way Alerts — every alert (doorbell, alarm, phone ringing) fires sound, screen flash, and vibration together. If one channel is weak, the other two still land.
  • Slow Speech — a hand-written overlap-add time-stretch algorithm slows down uploaded audio without the chipmunk pitch distortion, because older adults often need more time to process speech, not just louder speech.
  • What's that sound? — a real pretrained model (YAMNet, 521 AudioSet classes) runs entirely on-device via TensorFlow.js to identify doorbell, alarm, speech, or noise, then routes that into the alert system.
  • Live Call — two devices connect peer-to-peer over WebRTC, with Clarity Boost running live on the incoming audio during the actual conversation. Every device gets a permanent code, saved like a contact.

How we built it

The whole thing is a single Next.js app deployed as a Cloudflare Worker (via OpenNext), no separate backend for the core features. Layers 1 through 3 are pure client-side Web Audio API. The classifier runs client-side too, via TensorFlow.js. Live Call uses PeerJS for WebRTC signaling and ExpressTURN for NAT traversal, so calls connect across different networks, not just the same wifi. The one thing that reaches out to an external API is an optional Whisper transcript feature, proxied through a single Cloudflare Function with one secret.

We started with a heavier architecture, Cloudflare plus Cloudflare Workers plus a Render-hosted FastAPI backend running a hand-tuned heuristic classifier. Once YAMNet proved it could do that job client-side, for free, more accurately, we deleted the backend entirely. One deploy target, one secret, fewer things to break during a demo.

Challenges we ran into

Live Call surfaced two genuinely obscure bugs. First, AudioContext created inside a 'stream' event handler (which fires well after the click that triggered it) started suspended under Chrome's autoplay policy, silent with no error thrown anywhere. Second, even after fixing that, both devices logged ICE connected and track unmuted, real WebRTC confirmation that audio was flowing, while our Web Audio graph read zero. That turned out to be a specific Chrome quirk: createMediaStreamSource() on a WebRTC stream can silently receive no samples unless the stream is also attached to a real (even muted) <audio> element first.

We debugged both of these live, on real phones with no dev tools available, by building a visible debug log directly into the app itself, plus a live audio-level meter, so we could actually see what was happening instead of guessing blind.

Accomplishments that we're proud of

Replacing our own heuristic classifier with a real pretrained model once we had evidence it was better, 94% confidence correctly identifying a piano performance, with related instruments clustering sensibly underneath, is the kind of result that's hard to fake. Getting Live Call working end-to-end, including tracking down that Chrome audio bug through nothing but an on-page debug log, felt like real engineering under real constraints, not a demo hack.

What we learned

The biggest lesson was architectural restraint. It's tempting to keep every service you start with. Deleting the Render backend once a better, simpler answer existed was the right call, and the app is genuinely more reliable for it.

What's next for Sound for All

  • A hearing profile that learns each person's specific hearing curve over time, instead of one generic boost
  • A native companion that's reachable the way a real phone is, always on, not a tab you remember to open
  • Care that includes the people around them, so a missed alarm or a rough call isn't something faced alone
  • The same clarity idea reaching TVs, doorbells, and shared spaces, not just one device

Built With

Share this project:

Updates

Submission history