Inspiration
Alex has taught senior biology for nineteen years. His first sign of ALS was a student asking whether his marker had run dry. It hadn't; his handwriting was fading halfway across the board.
Two years later, his file showed us a pattern no single clinic visit captures. At 9 am he reads at 165 words per minute and is 98 % intelligible. By 4 pm it's 118 words per minute and 91 %. Students at the back have started asking him to repeat himself after lunch, and he has quietly dropped his end-of-day recap.
Two other details shaped everything we built. The school offered him a microphone, and he never asked for it. He hasn't started voice banking because "it feels like giving up." The prompt warned that most assistive devices end up in a drawer because nobody asked the person what they wanted. Alex had already told us: he wants to be in a classroom in June, and he wants to be remembered as himself, not as a patient.
So we set ourselves one rule: Lecturn never speaks for Alex. It helps his own voice reach the back row, for longer.
What it does
Lecturn listens through the microphone of the computer Alex already teaches from and notices when his voice is tiring. There's nothing new to wear, charge or set up.
- Captions only when he needs them. In the morning, nothing happens. As fatigue builds, live captions of his own words fade onto the bottom of whatever is on the classroom screen: slides, videos or any app. He can switch them Off / Auto / On at any time.
- Speech battery. One number and one sentence telling him how much voice he has left today, so he can plan instead of push.
- Stutter-aware captions. Fillers and repeats are cleaned out. When he stalls mid-sentence, a suggested ending appears in italics after an arrow, grounded in the lesson material and clearly marked as a guess.
- Live notes and term definitions. While he teaches, Lecturn builds class notes from his lesson, and a one-line definition appears when he uses a key term.
- The recap comes back. At the end of the day he ticks the sentences that matter and publishes them for students: his exact words, never rewritten.
- Sessions on his terms. The mic stays off until he presses Start session. End session turns it off and builds a report: how his speech changed during the session, how it compares with recent sessions, energy-conserving guidance and a breathing plan.
- Clinic view. Between his three-monthly visits, his ALS clinic sees daily speech trends, but only while he chooses to share.
How we built it
Signals from a single microphone. Every 5 seconds we analyse the last 10 seconds of audio from the computer's mic: loudness from frame RMS, voice activity, breath groups (speech runs split by pauses of at least 250 ms), and speaking rate from word timestamps. Each measure is compared with Alex's own fresh baseline, giving deficits scaled from 0 to 1:
$$d_{\text{rate}} = \min\left(1,\ \frac{(r_0 - r)/r_0}{0.30}\right) \qquad d_{\text{loud}} = \min\left(1,\ \frac{L_0 - L}{6\ \text{dB}}\right) \qquad d_{\text{breath}} = \min\left(1,\ \frac{(b_0 - b)/b_0}{0.40}\right)$$
Here $r$ is speaking rate, $L$ is loudness and $b$ is words per breath, with a $0$ subscript marking his fresh baseline. The 30 % full scale isn't arbitrary: his own drop from 165 to 118 wpm is 28 %. The fatigue score is a weighted average, smoothed so one bad window can't trigger anything:
$$F_t = \alpha \cdot 100\sum_i w_i d_i + (1-\alpha)\,F_{t-1}$$
Captions use hysteresis: they turn on at $F \ge 55$ and only turn off below $40$, after at least two minutes, so they never flicker mid-sentence. The battery he sees is simply $100 - F$.
Captions and language. ElevenLabs Scribe v2 Realtime transcribes word by word, boosted with key terms pulled from the day's lesson PDFs. Gemini handles the predicted endings, live class notes, term definitions and the clinic summary, with an instant offline fallback for each. For the recap, Gemini receives numbered sentences and returns only the numbers to keep; the server copies those sentences verbatim, so the model cannot change a word he said.
Clinical grounding. Our measures follow the four speech subsystems in Green et al. (2013): respiratory, phonatory, articulatory and resonatory. We say openly that resonance can't be measured with one microphone. Eshghi et al. (2022) gave us the thresholds for functional speech loss, 120 words per minute and 85 % intelligibility, and mean decline rates of 1.74 wpm/month for spinal onset and 4.16 for bulbar onset. The clinic view fits a line to his morning speaking rate, so daily fatigue doesn't pollute the trend, and estimates
$$t_{120} = \frac{r_{\text{now}} - 120}{\lvert \text{slope} \rvert}\ \text{months}$$
as a prompt for the clinic, not a prognosis.
Stack. A Python FastAPI backend with WebSockets, Tiger Data (TimescaleDB hypertables with continuous aggregates) for time-series storage, a desktop caption overlay in tkinter that sits on top of slides and videos, plain HTML/JS pages, and a simulator that replays Alex's day from his clinic numbers so the demo never depends on venue Wi-Fi.
Challenges we ran into
- Helping without taking over. Predicted endings are powerful but risky: a confident wrong guess on a projector could mislead a student. We kept predictions visually separate, gated them by a confidence threshold, let Alex turn them off, and never let them reach the recap.
- Fatigue vs. progression. His voice gets worse every afternoon and every month. Comparing against his own baseline separated the first; using only morning values for the clinic trend separated the second.
- Measuring breathing without a sensor. We had only a microphone, so we used pause patterns as an indirect breathing measure, the way the speech research does.
- Flickering captions. Our first version switched captions on and off mid-sentence. Hysteresis and a minimum on-time fixed it.
- Latency. On Windows,
localhosttried IPv6 first and added about 2 seconds to every caption request. Switching to127.0.0.1made captions feel live. - API limits. Free-tier Gemini quotas ran out and models got overloaded during peak hours, so every AI feature got throttling and an offline fallback.
- What not to build. Our first instinct was a throat-strengthening program. The research says oral strengthening isn't recommended in ALS and can tire the muscles further, so we designed for energy conservation instead.
What we learned
- In ALS, pauses are a breathing measure. Weaker breathing muscles mean shorter phrases and more frequent breaths, which a microphone can track every day, not every three months.
- Speaking rate falls before intelligibility does, which makes it an early warning that ordinary hardware can actually measure.
- The only bulbar/respiratory exercise with randomized-trial support in ALS is expiratory muscle strength training, and it belongs in a therapist's hands, not an app's defaults.
- Most of all, autonomy is a design requirement. Every feature has an off switch, nothing is stored without his say, and the most important thing the tool does is stay out of the way until he needs it.
What's next
A small clip-on microphone so Lecturn can follow Alex as he walks the rows, a clinical pilot with ALS clinic speech-language pathologists to tune thresholds on real dysarthric speech, and the same caption bar showing text from an eye-gaze communication device when speech becomes hard to understand, so his classroom setup never has to change.
