legenda-livre

Open-source, real-time multilingual captions for every conference room.

Comment 
## Inspiration

Technology conferences bring together people from different countries, backgrounds, and levels of hearing ability, but language remains a barrier. A great technical talk can become inaccessible when attendees cannot understand the speaker or follow the audio clearly.

I built **Legenda Livre** to make conference content accessible in real time. The idea was to create a lightweight, open-source system that conference organizers could operate without specialized captioning infrastructure.

## What it does

Legenda Livre converts live speech into multilingual captions for conferences.

An operator can:

- Capture audio from a microphone or a local audio/video file.
- Generate real-time transcription in the speaker's original language.
- Translate captions into another language.
- Create independent rooms for multiple simultaneous talks.
- Share an accessible audience caption page.
- Add a transparent caption overlay to OBS.
- Provide a technical glossary for names, tools, and specialized terminology.
- Export transcripts as VTT, SRT, or TXT.
- Choose between a modern interface and a Windows 95/98-inspired classic translator layout.

More than 80 BCP-47 language variants are available. Viewers reuse the caption events generated for each room, so adding viewers does not create additional AI transcription requests.

## How I built it

The application uses Node.js, Express, WebSockets, browser audio APIs, and the Google Gemini API.

The audio pipeline is:

```text
Microphone or media file
        ↓
Browser AudioWorklet
        ↓
16 kHz mono PCM audio
        ↓
Room-specific WebSocket
        ↓
Gemini 3.5 Transcribe Live
        ↓
Gemini 3.5 Flash-Lite translation
        ↓
Audience page and transparent OBS overlay

Each room owns an isolated transcription session and event stream. Interim transcription results are displayed immediately, while finalized segments are translated and stored for export.

Translations are streamed to the interface as they are generated. Each room uses two controlled translation workers, allowing captions to be processed in parallel without creating an uncontrolled number of API calls. Slow requests are cancelled locally and retried automatically.

The project also includes:

  • Persistent file-backed room state.
  • Translation and transcription latency metrics.
  • Technical glossary support.
  • A scripted demo mode that works without an API key.
  • Docker and Docker Compose configuration.
  • Automated tests for rooms, exports, languages, themes, continuous speech, retries, and parallel translations.
  • An MIT Open Source license.

Challenges I faced

The largest challenge was continuous speech. Live transcription naturally finalizes an utterance when it detects a pause, but conference speakers and recorded talks can speak for long periods without silence. This initially caused captions and translations to remain pending.

I solved this by creating automatic caption boundaries every few seconds. The audio stream is immediately reopened after each boundary, allowing continuous talks to produce regular subtitle segments.

Translation latency was another challenge. Most translations completed quickly, but some requests occasionally remained pending for more than 30 seconds. I added streamed translation updates, two parallel workers per room, a local timeout, and automatic retries. In testing, normal translations appeared in under a second, while stalled requests were recovered instead of leaving captions untranslated.

I also had to make the OBS overlay readable while keeping everything except the caption text transparent.

What I learned

I learned how real-time audio differs from ordinary request-and-response applications. Low latency depends on the entire pipeline: audio chunk size, voice activity detection, segment duration, translation concurrency, network behavior, and rendering.

I also learned that accessibility cannot be treated as a final visual feature. It has to influence the architecture, operational workflow, language selection, subtitle exports, and failure recovery from the beginning.

What's next

Future improvements include Redis Streams for multi-instance deployments, automatic reconnection of long Gemini sessions, collaborative glossary management, speaker identification, more deployment options, and direct integration with conference streaming platforms.

Legenda Livre is open source so event organizers can adapt, deploy, and improve it for their own communities.

Built With

Share this project:

Updates

Submission history