Project Drake

Inspiration

Live events increasingly bring together international audiences, but accessibility and real-time translation are still difficult to scale across multiple simultaneous stages.

Project Drake was built to explore a simple idea: what if one live audio stream could become an accessible, multilingual caption stream that could be reused everywhere it is needed?

What it does

Project Drake is an open-source platform for real-time multilingual captions and translation for live events.

It can:

Transcribe live audio in real time.
Translate captions between Spanish, English and Portuguese.
Run multiple stages simultaneously with independent language configurations.
Provide an Audience view for attendees.
Send the same canonical caption stream to OBS through a Browser Source overlay.
Persist completed talks and their translations.
Export transcripts as TXT and VTT.
Manage events, stages and talks from a unified interface.

How we built it

Drake uses a realtime pipeline designed around a canonical caption stream.

Live audio is processed through Gemini for speech-to-text. Final captions can then be translated into the configured target languages while the original captions remain independent from translation latency or failures.

The same caption stream powers Audience, OBS and persistent Results, avoiding separate transcription pipelines for every output.

The application is built with TypeScript, React, PostgreSQL and Google Cloud, and is deployed on Cloud Run.

Challenges

The main challenge was not simply generating a transcript, but building a reliable realtime system that could support multiple simultaneous stages while keeping transcription, translation, persistence and different consumption surfaces synchronized.

We also designed recovery and degradation behavior so that translation or persistence issues do not unnecessarily interrupt the original live captions.

Accomplishments

During the Vibeathon build window, Drake reached:

5 simultaneous stage pipelines.
Real-time transcription and translation.
Spanish, English and Portuguese language configuration.
Audience and OBS integration.
Persistent talk history and results.
TXT and VTT exports.
A complete event → stage → talk → live → results workflow.
Deployment on Google Cloud Run.

Most importantly, Project Drake became a working product that can be operated through its UI rather than only a technical prototype.

What's next

Future work includes post-talk intelligence, richer content generation, additional languages and distributed coordination for larger multi-instance deployments.