Inspiration

Growing up, my best friend's dad lived next door. When ALS took his voice, he carried a tablet everywhere and typed out every sentence one letter at a time. A conversation that should take seconds took minutes, and most of them never happened. He passed away a few years later.

At Brown, I shadowed physicians and saw the same thing again: patients with ALS, stroke, and Parkinson's who knew exactly what they wanted to say but couldn't be understood. My teammate and I are both engineers, so we decided to build what my neighbor never had.

What it does

Vocera is a band worn around the neck. Two piezo contact sensors and a MEMS microphone sit against the throat, so it picks up speech both as vibration through the skin and as sound in the air. It streams both signals to a laptop, where a speech model turns them into text in real time. Then the small speaker built into the throat pod, will read that text out loud.

How we built it

The band is built from a small ESP32-S3 board, a tiny microphone, two piezo discs that feel your throat vibrate as you talk, a little amplifier and speaker, and a rechargeable battery. We soldered all of it by hand.

The band sends what it hears to a laptop over a USB cable. There, a speech model turns it into text. We use a distilled version of WavLM, a speech model from Microsoft. Distilled means it's a smaller copy of the full model, so it runs fast on a normal laptop.

We designed the case in Fusion 360 and 3D printed it. A curved piece sits on the throat and holds the sensors, mic and speaker. A small box at the back of the neck holds the board, battery and amplifier, and an adjustable strap connects the two.

Challenges we ran into

Our soldering iron died halfway through the build, and we got kicked out of Brown's main workshop, so we door-dashed a soldering iron and finished in our dorm room. Our microcontroller turned out to be a clone with a different pinout from the board we'd planned around, so we had to remap every pin. Then we soldered the battery in backwards and watched it smoke. The board survived, and we wrote a self-test that checks every pin before anything else gets soldered. Carbon-fiber PLA also turned out to be too stiff for snap fits. Our buckle cracked, so we redesigned the clips with no flexing parts.

Accomplishments that we're proud of

The band works end to end. You speak while wearing it, and your words show up as text on the laptop in real time.

It records two signals at once: the piezos pick up vibration through the skin while the microphone picks up sound in the air, each on its own core of the ESP32-S3.

We fine-tuned Whisper on dysarthric speech, so it handles the kind of speech that stock models usually get wrong.

We went through four versions of the enclosure, all written in code, and printed every part ourselves. The final one fits around a neck, adjusts to different sizes, and holds every component.

We also got through a dead soldering iron, a lost workshop, and a battery wired backwards, and still finished with a board that passes every test we wrote for it.

Most of all, we're proud to have worked on a problem we first saw as kids. ALS took the voice of someone close to us, and building something for people in his place means more to us than anything else on this list.

What we learned

Voice APIs were new to us. We learned how speech-to-text, text-to-speech and latency fit together, and why off-the-shelf models struggle with impaired speech. On the hardware side, we picked up digital audio over I2S, ADC sampling, and how to handle LiPo power safely. Building something people wear all day taught us that it has to be comfortable and adjustable, but also tough enough to survive coming on and off. And we now check polarity twice and test after every joint :)

What's next for Vocera

First, we want to give it a voice. The speaker is already sitting in the throat pod, so the next step is adding text-to-speech so the band can say the words out loud instead of only showing them on a screen.

Then we want to cut the cable. Right now it has to stay plugged into a laptop. Running it on battery and streaming over Bluetooth would let someone wear it anywhere.

We also want to get more out of the two sensors. Combining the piezo and microphone signals in the model should help with whispers, weak voices, and noisy rooms.

After that, we want to test it with the people it's for. We'd love to team up with the clinicians we shadowed and collect speech data, with consent, from people with ALS, stroke, and Parkinson's.