Inspiration
Nerdearla already invests in accessibility, but more than 30 English sessions and several simultaneous stages make commercial caption tools expensive and operationally difficult. The same problem affects open-source conferences everywhere. I wanted the caption pipeline to scale with active stages, not with audience size.
What it does
OpenStreamTranslate accepts live English or Spanish audio and publishes original and translated captions in near real time. Viewers choose the stage and language from a familiar video player. Producers get a separate Studio for microphone or video input, per-talk technical glossaries, and SRT/VTT/TXT export. OBS and vMix can consume a transparent overlay.
A production stream connects through RTMP, SRT, or HLS to a media server. The included FFmpeg bridge extracts PCM audio and sends it to OpenStreamTranslate over WebSocket. Every stage is inferred once; Redis Streams and Pub/Sub distribute the same caption event to reconnectable SSE gateways, so adding viewers does not add GPU work.
How I built it
- faster-whisper large-v3-turbo for bilingual ASR
- TranslateGemma 4B IT Q8_0 on llama.cpp for local translation
- FastAPI WebSockets for audio ingress and SSE for audience delivery
- Redis Streams for history/replay and Pub/Sub across replicas
- FFmpeg real-time input for live-stream integration and deterministic tests
- A dependency-free web player, operator Studio, and OBS/vMix overlay
- Prometheus metrics and structured JSON logs
- Bounded audio/context buffers plus retry and exponential backoff for long-running reliability
- An acceptance suite covering WER, latency, 2/5/10 concurrent sessions, bilingual isolation, reconnect replay, transcript export, and up to 5,000 SSE clients
Challenges I ran into
The hardest part was not raw inference speed. Partial hypotheses changed while people were reading them, short fragments disappeared too quickly, and naive fan-out made one source look expensive for every viewer. I anchored captions to the video clock, stabilized partials, limited them to two readable lines, and made one Redis subscription feed all local SSE clients for a session. I also bounded every long-lived buffer and isolated ASR retries from translation retries so a transient model or network failure does not terminate the live session or repeat successful work.
Accomplishments I’m proud of
- 3.70% English WER on a real technical video with a human reference
- Ten simultaneous sources with a 3.51 s maximum first-caption delay
- 100 viewers with 17.12 ms fan-out p95 and one shared inference event
- 1,000 viewers on one replica with 173.07 ms fan-out p95
- English↔Spanish streams running simultaneously without cross-talk
- Technical hotwords and exact translation mappings per talk
- SRT, VTT, and plain-text transcript export after every session
- No commercial inference API or per-viewer model cost
What I learned
Caption UX is a scheduling problem as much as an ML problem. A marginally more accurate model can still feel worse if it rewrites text too often or lets delay accumulate. I also learned to separate stage capacity from delivery capacity: GPU workers scale by concurrent talks, while inexpensive CPU gateways scale by viewers.
What's next for OpenStreamTranslate
- Word-level alignment and speaker labels
- Portuguese after evaluating it with human references
- Authentication, rate limits, and production ingress hardening
- Multi-GPU scheduling with Ray Serve/KubeRay
- A second-pass high-accuracy transcript after each talk
Log in or sign up for Devpost to join the conversation.