livesubs

low latency speech recognition and live translation engine that delivers really good and accurate subtitles. Ensures words are never cut and that tech terms are perfectly preserved

Comment

Inspiration

My motivation was only one: I wanted to challenge myself by building something I had never tackled before, from scratch, under extreme time pressure. In this project there is no safety margin, and any lag immediately ruins the viewer experience.

How I Built It

I built LiveSubs as a decoupled real-time pipeline. An ingestion worker continuously analyzes incoming audio using a local Silero VAD model (ONNX Runtime) to detect speech boundaries. When speech pauses, the chunk is sent to Groq Cloud running Whisper Large v3 Turbo for transcription, followed by an LLM stage for translation (I used GPT OSS 120B). The resulting subtitles are published to Valkey (Redis fork) and broadcast through a FastAPI ASGI gateway over WebSockets directly to an OBS Studio overlay and a web reader. When the source and target languages match, the LLM step is bypassed, delivering native captions in under 850 ms.

Challenges Faced

• Word Clipping: Early versions cut off initial consonants and word endings. I solved this by implementing an acoustic ring buffer with a 350 ms pre-roll and 120 ms post-roll window around speech events. • Technical Jargon: LLMs tend to translate dev terms literally. I introduced a domain glossary conditioning Whisper's prompts and added translation guardrails so words like deploy, pipeline, or merge are preserved. • Live Rate Limits: Live speech is continuous. To prevent crashes on API rate limits, I implemented a multi key round-robin rotator with automatic cooldowns that switches keys without dropping audio (this is only because i was using Groq free tier. But on paid plans you dont have to)

What I Learned

Real-time audio leaves zero room for blocking calls or unnecessary locks, every millisecond directly impacts user experience. More importantly, decoupling the AI inference from client distribution through an in-memory message broker is what allows a single stream to scale to thousands of viewers without multiplying costs or latency.

Share this project:

Updates

Submission history