ALF: Real-Time Conference Transcription Platform

Inspiration

Nerdearla 2026 runs 30+ simultaneous sessions, most of them in English, while most of the audience is Spanish-speaking. Today, accessibility means either expensive commercial tools or manual subtitle operators — costly and hard to scale across dozens of rooms.

We asked: what if any conference could get live, translated subtitles for the cost of a Gemini API call?

ALF answers that with open-source tech, Google's Gemini Live API, and an architecture that isolates each stage into its own pipeline so it scales room by room.

How We Built It

Stack:

• Backend: Node.js + Express + Socket.IO (per-stage transcription pipelines)
• Frontend: Next.js + Socket.IO client (real-time subtitle sync)
• AI: Gemini Live API (gemini-3.5-transcribe-live) for transcription, Gemini Flash for context-aware translation
• Deployment: Docker Compose for local dev, Google Cloud Run (backend) + Vercel (frontend) for production

Core Design:

1. Operator's browser captures mic or file audio, streams PCM16 to the backend over a JWT-authenticated WebSocket namespace
2. One isolated Gemini Live session per stage transcribes in the original language, streaming incremental results as the speaker talks
3. A custom segmenter turns that growing, punctuation-free ASR stream into clean, sentence-level captions
4. Each finished sentence is translated with the last 2 sentences of context (so pronouns and technical terms land right), and broadcast over a public, read-only Socket.IO namespace
5. The audience picks a session + language and watches, no login required

What We Learned

• Incremental transcription is a different problem than batch transcription. Gemini's live interim text keeps growing and can revise a word it already sent as more audio arrives. Naively diffing against what you already committed breaks — you have to decide, deliberately, what "already shown to the viewer" means and never take it back.
• Real-time resilience has to be built, not assumed. Live sessions can go silent mid-utterance with no error and no close event. We ended up with three layers: audio buffering across reconnects, Gemini's own session resumption, and a stall watchdog that forces a reconnect if nothing arrives for 15 seconds — tested against 10+ minute live sessions today.
• Translation throughput needs its own concurrency control. A single sequential translation queue falls behind once sentences finalize every second or two; bounding concurrency (instead of firing every request in parallel or queuing strictly one-by-one) is what keeps translations from trailing minutes behind the live audio.
• Open source only helps if it's actually easy to run. MIT license, .env.example, and a one-command docker compose up matter as much as the AI plumbing.

Challenges

What's Next

• Speaker diarization (who said what)
• Searchable, exportable transcript archive (SRT/VTT/TXT)
• Scale-testing beyond 2 simultaneous rooms and documenting the real cost curve
• More language pairs, including Portuguese

ALF is open source (MIT), built for Nerdearla 2026 — and for any conference after it.

Built With

• api
• audio
• css
• docker
• express.js
• gemini
• google
• javascript
• next.js
• node.js
• open
• postgresql
• processing
• react
• real-time
• rest
• tailwind
• typescript
• websocket