Inspiration

Conferences like Nerdearla bring speakers from all over the world, but when the talk is in English, a big part of the audience is left out. We wanted to break that language barrier with something any event could afford: live captions and simultaneous translation on the public's own phones, with open-source software.

What it does

It converts any talk — mic in the room, a YouTube video/live, or an audio file — into real-time subtitles translated to the viewer's language. The audience scans a QR code and picks their language (EN/ES). Captions stream to the browser in ~4-second windows. It runs on cheap hardware (a mini-PC) and is fully Dockerized; a demo mode plays back a recorded session with no API key needed.

How we built it

Python 3.12 managed with uv, a FastAPI + Uvicorn API, and google-genai (Gemini) for transcription and translation in fixed audio windows. Subtitles are pushed to browsers via Server-Sent Events; mic audio arrives over WebSocket. yt-dlp + ffmpeg resolve YouTube URLs and export segments. Front-end is vanilla HTML/JS with a QR generator. Everything deploys to Cloud Run or the mini-PC from one Docker image.

Challenges we ran into

The Gemini Live API wasn't accessible from our GCP project (404s, empty errors), so we pivoted from word-by-word live transcription to windowed chunks — which actually made the pipeline far more available. Mid-demo we hit 429 RESOURCE_EXHAUSTED on the promo credit, so we added retries with backoff+jitter. We also debugged yt-dlp working in the container but not on the host, and the replay-vs-live mode confusion users hit.

Accomplishments that we're proud of

End-to-end run from a real microphone to translated captions in the browser. A session survived a 429 burst without dying. 45 automated tests green. The whole thing is open source and reproducible in one docker compose command.

What we learned

Windowed transcription (chunked) is more reliable than a streaming API you can't access; SSE + WebSocket is a simple, scalable shape for live subtitles; and promocional-cloud billing/quota limits are a real constraint you must design retries for.

What's next for nerdearla-live-translator

Deploy the live instance to Cloud Run, validate in-room with the hardware board, record the demo video, add more languages, multi-mic audio splitting, and a public dashboard per event room