FOR SPONSORS: How we use Gemini, ElevenLabs, Tiger Data and Vultr is set out under How we built it.

Ecko

Keep teaching, in your own voice. Ecko is a clip-on wearable badge for teachers with ALS. It listens as they teach, understands slurred speech, and writes the whiteboard live, drawing diagrams when asked. With one press of a button on the badge, it repeats their last sentence, recaps the lesson, or re-speaks them clearly, all in their own voice. Between classes, it tracks how their speech is changing and keeps their care team informed.

Inspiration

Alex Chen is 47, teaches high-school biology, and has ALS. He wants to be in a classroom in June.

His speech-language pathologist measured his speech at 165 words a minute at 9 am, and 118 by 4 pm. Over the same day, his intelligibility drops from 98% to 91%. Students have started asking him to repeat himself. His handwriting fades after about ten minutes at the board, and he has quietly stopped doing his end-of-day recap. The school offered him a microphone, but he never asked for it.

One line in the case stayed with us: most assistive devices end up in a drawer because nobody asked the person what they wanted. Alex turns down anything that makes him look sick. He doesn't want a machine that teaches for him. He wants to be the one to keep teaching.

So we built Ecko around one rule: Alex teaches, Ecko helps, and Alex decides when.

What it does

Ecko has two parts: a badge Alex clips to his shirt, and the board on the classroom projector.

1. The Ecko badge

The badge is a small wearable with a built-in microphone and four large buttons. Alex doesn't need to hold anything, find a screen or look down. He keeps teaching, and presses a button when he wants help.

  • Mic: turns listening on and off.
  • Say that again: repeats his last sentence.
  • Recap: sums up the lesson so far.
  • Clear voice: re-speaks everything he says clearly.

Why a badge? The school offered Alex a microphone, and he hasn't asked for it. He wants to be seen as a teacher, not a patient. A badge on his shirt looks like something any teacher might wear. It keeps the microphone close to his voice, and keeps the controls in reach of a weakening hand.

We sketched the badge and modelled it in CAD. In our demo, a phone runs the badge: the same four controls as large buttons, each with its own vibration so they can be used by feel, sending his voice to Ecko just as the badge will.

2. The live whiteboard

  • Understands slurred speech: Ecko sends Alex's actual audio to Gemini along with today's lesson plan, so even words that are hard to make out are recovered from the context of the lesson. Students talking in the background are ignored.
  • Shows what it inferred: any word Gemini had to work out from context is highlighted in the live caption.
  • Writes notes as he teaches: clean notes and section headings appear about 1.5 seconds after he pauses.
  • Waits for whole thoughts: if he stops mid-sentence, Ecko holds the words until he finishes, so half-sentences never reach the board.
  • Writes only what he said: it never answers a question he asks the class, and never adds anything he hasn't taught yet.
  • Draws when he asks: if he says "let me draw a neuron", a labelled chalk diagram appears in about 8 seconds. Ask for the Krebs cycle and it draws a flowchart loop.
  • Saves the notes: Download notes turns the whole board, notes and diagrams, into a PDF for his students.

3. His own voice, when he chooses

Three of the badge's buttons speak in his own cloned voice through the classroom speakers.

  • Say that again: shows his last sentence large on the board and speaks it.
  • Recap: writes a short recap of what he actually taught today and reads it aloud. That's the end-of-day recap he had stopped doing.
  • Clear voice: while it's on, everything he says is re-spoken clearly. That includes instructions and answers to students that never go on the board.

4. Tracking his speech over time

  • Measures every sentence: speaking rate, pauses, loudness, and how many words had to be inferred. It stores only these numbers, never his words.
  • His My speech page: shows today against a typical day, a calendar of past days, and his mornings against his afternoons, week by week.
  • Suggests, never decides: when his speech drops well below his usual mornings, Ecko gives him a distinct buzz and suggests Clear voice. Nothing switches on by itself.
  • Keeps his clinic informed: his ALS clinic sees him every three months. One tap writes a short summary of how his speech has changed in between, for his neurologist and speech-language pathologist. It reports numbers only, with no diagnosis.

How we built it

How it all fits together

Here's one sentence from Alex, end to end:

  1. The badge (a phone, in this demo) hears him speak and waits for a natural pause. It sends that clip of audio to our server on Vultr.
  2. Gemini listens to the clip, with the lesson plan as context, and works out what he said, including slurred words.
  3. Gemini turns his words into notes and headings. Our server checks them, then pushes them to the board instantly.
  4. If he asked for a diagram, Gemini's image model draws it in chalk, and the board places it on the page.
  5. At the same time, the clip is measured (speaking rate, pauses, loudness) and the numbers are saved in Tiger Data.
  6. When he presses a button, ElevenLabs speaks in his voice through the classroom speakers. Say that again and Clear voice use his own transcribed words; for Recap, Gemini first writes the recap from what he taught.
  7. Later, his My speech page and clinic summary read his trends from Tiger Data, and Gemini writes the summary for his care team.

The badge, board and My speech page never talk to each other directly. They all connect to one server, which keeps the state of the class and pushes every update to every screen at once.

The badge

We designed the badge around one weakening hand: four large buttons, a microphone close to his voice, and a clip so nothing needs holding. We sketched it, then modelled it in CAD.

For the demo, a phone runs the badge in the browser. It listens with the Web Audio API, learns the room's background noise, and keeps a moment of audio from just before he speaks, so his first syllable isn't cut off. Each clip ends at a 2-second pause, because at 1.3 seconds his halting sentences arrived as one-word pieces.

Vultr

Vultr hosts the whole of Ecko on a server in Toronto, close to the classroom for low lag.

  • Our Node.js server runs there with no framework. It keeps the state of the class, and pushes every update to the board and the badge instantly over Server-Sent Events.
  • HTTPS through Caddy: the microphone only works on secure pages.
  • Locked down: the site is password-protected, the server only accepts SSH-key logins, and API keys never leave the server.
  • Always on: the app restarts itself if it crashes.

Google Gemini

Gemini does all of the understanding.

  • Gemini 3.8 Flash transcribes his audio directly, using the lesson plan, his last few sentences and the newest board notes as context. It marks every word it inferred, and leaves a blank rather than making a word up.
  • Gemini 3.5 Flash turns his speech into structured board content: notes, headings, flowcharts, diagram requests, and a flag for "his sentence isn't finished yet". Our server then checks every reply:
    • it drops notes that repeat a recent one
    • it only draws a diagram when both Gemini and our own keyword check agree he asked for one
    • it holds an unfinished sentence until he completes it
  • Gemini 3.1 Flash Image (Nano Banana 2) draws each diagram in coloured chalk. The board then removes the image's background on a canvas, so the diagram looks like it was drawn on the board, not pasted in.
  • Lesson plans, recaps and summaries: Gemini reads uploaded lesson plans (a PDF, or a photo of handwritten notes), and writes the recaps and the clinic summary.
  • Backups: every model has a backup, and if one is overloaded, Ecko switches to the next automatically.

ElevenLabs

ElevenLabs is the teacher's own voice.

  • Ecko streams audio from ElevenLabs' Flash v2.5 model straight to the classroom speakers. It starts playing in about 0.27 seconds.
  • We send the previous sentence with each request, so his tone carries on instead of restarting every line.
  • Ecko recognises its own voice coming back through the microphone and ignores it, so nothing it says is transcribed as if he'd said it.

Tiger Data

Tiger Data stores his speech measurements in a TimescaleDB hypertable on Tiger Cloud.

  • Measurements: our server measures each clip as it arrives: speaking rate, pauses, loudness, and how many words were inferred.
  • Half-hour totals: a real-time continuous aggregate keeps them, and the My speech page and the clinic summary read from it.
  • Retention: raw measurements are deleted after 120 days, and only the totals are kept. That's less data about a teacher in a room full of minors, while the trends still work.
  • Compression: week-old data is compressed.

Challenges we ran into

Speech-to-text deletes the words that matter. Browser speech recognition drops any word it can't make out, so a slurred word disappears entirely. That's why we send the audio itself to Gemini.

Halting speech broke the board. Our first version wrote fragments like "terminal and then" as notes. We fixed it in three ways:

  • retuned every timing for slow speech
  • let Gemini hold back unfinished thoughts
  • showed it real fragments from our own testing as examples of what never to write

Keeping the AI to what he said. The fastest models answered questions he asked the class, and added facts he hadn't taught. We moved to a stronger model, wrote strict rules, and had the server double-check every reply.

Diagrams that looked wrong. Our first diagrams were SVG shapes generated by Gemini, and the parts floated loose. Moving to an image model fixed the anatomy, but left a white box around each diagram. A CSS blend couldn't remove it, so we wrote our own colour-to-transparency step.

Ecko heard itself. A recap played through the speakers came back through the mic and was transcribed as if he'd said it. Now Ecko ignores anything that mostly matches what it said in the last two minutes. A real sentence he says over the top still gets through.

Accomplishments that we're proud of

  • We designed a real wearable. The badge is sketched and modelled in CAD, and a phone prototype runs the same four controls end to end.
  • It understands slurred words. In our slurred test recordings, Ecko recovered every key term (neuron, axon, myelin). Browser speech-to-text dropped them completely.
  • It's fast enough for a live class.
    • Notes appear in about 1.5 seconds (median of 55 logged calls).
    • His voice starts in 0.27 seconds.
    • A labelled diagram takes about 8 seconds.
  • Every feature is his choice. Nothing turns on by itself.
  • Privacy by design. Only the teacher is transcribed, only numbers are stored, and raw measurements are deleted after 120 days.
  • It runs end to end on a real cloud server, with the phone prototype connected over HTTPS.

What we learned

ALS speech isn't just noisy speech. It's slower, with pauses in the middle of thoughts. Every timing in the system had to be rethought around that.

Context mattered more than a bigger model. The lesson plan is what lets Ecko recover his slurred speech.

Independence is a design requirement. Our first instinct was to switch Clear voice on automatically when his speech slipped. The case showed us why that's wrong for Alex, so Ecko only suggests.

What's next for Ecko

Build the badge. Turn our CAD model into a physical prototype, and test its buttons with people living with ALS.

Test it on real ALS speech. We'd start with the VOC-ALS dataset, and test with speech-language pathologists.

Voice banking from his own lessons. We want to help him bank his voice early, from his clearest lessons and only with his consent, so it never has to feel like giving up.

A badge that grows with him. As his hands weaken, we want the badge's buttons to need less and less strength to press.

A notes page for students, opened with a QR code at the end of class.

Ecko can't stop ALS. But it can help Alex keep teaching his way, for as long as he chooses.

Built With

Share this project:

Updates

Submission history