linguastream

• 

  Dashboard

Inspiration

Nerdearla brings together speakers and attendees from all over Latin America and beyond. Talks happen in Spanish, English and Portuguese, often in several rooms at once. Part of the audience always loses some of every talk, and for deaf and hard-of-hearing people a talk without text simply doesn't exist.

This was my first vibeathon. I wanted to build something the event could actually use the next morning. A room plugs in a microphone, attendees scan a QR code, and everyone reads the talk live in their own language on their own phone.

What it does

LinguaStream turns a room's audio into live captions and translations (ES / EN / PT):

• Room console: captures a microphone or a browser tab, streams audio to Gemini Live and publishes the original transcript plus one translation per target language.
• Audience: scan a QR code or type a 4-letter code, pick Español, English, Português or Accessible in a popup, and read. No app, no sign-up. An optional read-aloud voice always stays on the live sentence instead of queueing old ones.
• Accessible mode: very large Atkinson Hyperlegible text, high-contrast themes, "pause to re-read", a visual Speaking / Silence indicator, an optional human sign-language (LSA) interpreter video, and vibration when the talk starts, pauses or ends (Android).
• Presentation + CC: the slides with the captions drawn on top, for the room's projector or the stream. There is also a transparent overlay for OBS/vMix.
• Production dashboard: a live funnel (live / with problems / scheduled / finished), latency, audience count, alerts with sound, automatic reconnection, and an archive of finished talks with SRT / VTT / TXT exports.
• AI glossary: upload the talk's slides (PDF or Google Slides link) and Gemini extracts the technical terms and proper nouns, so "Nerdearla" stops becoming "Nerdierla".

How I built it

• Next.js 16 + React 19 + Tailwind v4 on Vercel.
• Convex as the real-time backend: sessions, captions, stats, events, crons and file storage. Every phone subscribes to a reactive query, so there is no WebSocket code of my own.
• Gemini Live (gemini-3.5-live-translate-preview) opens one connection per target language. The first connection also provides the original transcript.
• Ephemeral tokens: the Gemini API key never reaches a browser. Convex mints short-lived tokens for each room console.
• Audio: an AudioWorklet turns the audio into 16 kHz PCM16 and sends it in 100 ms chunks. Line segmentation is driven by those audio chunks instead of timers, because browsers throttle timers in background tabs.

Cost scales with the number of target languages. At the preview price of about USD 0.0368 per minute per connection, one room costs

$$ \text{cost per hour} \approx 60 \times 0.0368 \times N_{\text{targets}} \approx 2.21 \cdot N_{\text{targets}} \ \text{USD} $$

The development was spec-driven (objectives, expected behavior, business rules, acceptance criteria). Every change went through its own branch and pull requestest suite covering reconnection, roomisolation and the stall watchdog.

Challenges

• Latency vs. readable lines. The translate model streams small text fragments. Cutting lines too early splits sentences, and cutting too late feels slow. the original and about 2.5 s for thetranslation after each sentence.
• Connections that die, or go quiet. Gemoughly every 10 minutes and sometimes keeps a connection open while returning nothing. I added session resumption and automatic reconnect, then a watchdog that replaces a translation that goes silent whilg.
• Quotas. The preview model limits concurrent sessions per project. Three rooms ran for 30 minutes without errors, but ten rooms were partially rejectenstead of hammering the API, and the READMEexplains how to request more quota.
• Phones on real networks. Scanning the hole app before redirecting. Resolving thecode on the server and preloading the session cut QR → popup from about 4 s to about 2.4 s on a simulated 4G phone.
• Small UX traps with big effects. A Spaset to English leaves the Spanish captionsempty, because Gemini has nothing to translate. Viewers now get a one-tap switch to the original instead of a blank screen.

What I learned

• Real-time systems are mostly about failureled tabs, quotas and silent streams.
• A reactive backend like Convex removes a whole layer of plumbing. The hard part moves to the edges: audio capture and the model.
• Accessibility is a design constraint from day one, not a feature added at the end.
• Measuring beats guessing. Most fixes startrom production data.

What's next

• Translated audio output for attendees who
• A self-hosted option with local models.
• More languages, and caption tracks for str

Built With

• 16
• 19
• next.js
• react