Inspiration
Nerdearla has offered live Spanish transcription for years, because a conference should be understandable to the people in the room. That used to mean two commercial tools: one for Spanish-to-Spanish transcription, and another for live English-to-Spanish translation. It worked, until it did not. This year there are more than 30 English sessions, many of them at the same time. The old setup is expensive, manual, and hard to repeat at another event.
Almost every open-source conference has the same gap. EventLyra is our answer: an open, local live-caption platform that a conference can actually run. The point is not to replace human interpreters. The point is that a voice on stage should reach the audience in their language.
The name comes from that idea. Event is the room. Lyra is the lyre of Hermes, a symbol of communication. The product is a live signal from a voice to people across languages: every voice, every language, everyone included.
What it does
EventLyra takes live audio from a file, a microphone, or a public YouTube live or VOD, and turns it into captions in real time. Each session transcribes the original language (Spanish, English, or Portuguese) and can show a translation on top of the video. One Windows process, on one GPU, serves several sessions at once. They share a single copy of the models.
Around that engine there is a real event surface, not a demo box:
- An organizer board to set up the event and each session.
- An audience lobby where someone picks a session and a language.
- A production desk with session state, inference latency, and queue wait shown separately.
- A transparent browser overlay for OBS or vMix.
- Export of the full talk as SRT, VTT, or plain text.
- A short glossary so names and technical terms survive translation.
The code is MIT. Gemma weights stay out of the repository and follow their own license.
How we built it
The demo runs locally on a Windows PC with an NVIDIA RTX 4070 SUPER (12 GB). A FastAPI process owns the engine. A React app is the organizer, the audience, and the production desk.
Speech-to-text is faster-whisper turbo in INT8 on GPU (int8_float16). Translation is TranslateGemma 4B in BF16, behind a replaceable translator interface. This cut ships only the local provider. There is no Gemini, Vertex, or other cloud client in the path that produces captions.
One GPU holds one copy of each model. Health reports a single asr_id and a single translator_id for every session. The GPU transcribes one chunk at a time and round-robins across sessions, so two rooms advance together without loading the weights twice. To go further, raise the session count on that GPU, or run the same API on another GPU with its own count. Thirty rooms do not fit on this card.
YouTube is not ingested as RTMP. yt-dlp pulls the audio in short streaming slices. On the audience player, a live with DVR is delayed by the measured lag, and a VOD waits until captions are ready so the text never runs ahead of the talk.
Challenges
Twelve gigabytes is a hard ceiling. TranslateGemma 12B and 27B do not fit comfortably next to Whisper on this card, especially with a desktop already using VRAM. We stayed on the 4B.
Smaller is not automatically better. Loading TranslateGemma in 8-bit with bitsandbytes saved memory and ruined the translation. BF16 on the 4B was the setting that still read like the talk.
One shared model is a queue, not a magic multiplier. Sessions share weights, but they also share time. Latency on the production desk is inference time. Time waiting for a turn is shown apart, so the panel does not pretend the GPU is faster than it is.
Captions have to meet the picture. A transcript that arrives early or late is a different bug from a bad translation. Syncing a YouTube live, a VOD, and a local file forced us to treat timing as part of the product.
The machine fought the install. The CUDA wheel had to replace the default CPU build of PyTorch. Model caches could not live on the system drive. Gemma would not download until the Hugging Face license was accepted. None of that is the product, and all of it blocked the first real subtitle.
What we learned
A caption model is the smallest part of live accessibility. The hard product is everything around it: several sessions, one honest GPU budget, a language choice for the audience, a desk for production, and an overlay a streamer can drop into OBS.
We also learned to trust the output, not the memory savings. A quant that fits and a quant that speaks are different things. EventLyra keeps the signal precise on purpose: one shared engine, measured latency, and a voice that can cross languages without leaving the conference's own machine.
Log in or sign up for Devpost to join the conversation.