Verba

One conversation, no matter the language.

Comment

Inspiration

So much of how we communicate online is mediated by feeds, algorithms, and recommendation systems. They decide what reaches us, what gets translated, and often who we interact with in the first place.

For HackWesTX's Beyond the Feed theme, we wanted to think about what communication looks like when the focus returns to the people having the conversation.

Language is still one of the most basic barriers to that.

Today, two people who do not share a language often have to stop, translate, read or listen to a generic voice, respond, and repeat the process. The words may get across, but the conversation becomes less direct.

We built Verba around a simple idea: two people should be able to speak naturally in their own languages and still feel like they are talking directly to each other.

One conversation, no matter the language.

What it does

Verba is a real-time multilingual communication platform.

Two people join the same session and choose the languages they want to speak and hear. When one person talks, Verba transcribes their speech, translates it for the other participant, and synthesizes the translation using an enrolled version of the speaker's voice.

Instead of replacing the person with a generic translator voice, Verba is designed to preserve more of who is speaking.

A Verba session also supports:

  • live captions and transcription
  • video and screen sharing
  • shared and private whiteboards
  • post-call transcripts and summaries
  • session recommendations
  • study PDFs and Quiz Battle
  • persistent study memory

When the call ends, the session does not disappear. The same link can continue into the transcript, summary, and shared whiteboard.

How we built it

Verba is built as a full-stack real-time application using Next.js, TypeScript, Supabase, LiveKit, Deepgram, ElevenLabs, Anthropic Claude, and Google Cloud Translation.

LiveKit provides the real-time audio, video, and screen-sharing infrastructure.

Deepgram handles live speech transcription.

For translation, Verba uses Google Cloud Translation, with language-model support where needed.

ElevenLabs is central to our voice experience. Users enroll their voice, and Verba uses that voice for synthesized translated speech so the person on the other side can hear the translation without replacing the speaker with a generic text-to-speech voice.

Supabase handles authentication, application data, session state, and protected storage.

Anthropic Claude supports language and post-call experiences including summaries, recommendations, and study features.

On top of those services, we built the session system, language configuration, call interface, shared canvas, post-call experience, access controls, and the orchestration required to connect the different stages together.

Challenges we ran into

The hardest problem was latency.

A translated sentence has to travel through several stages before the other person can hear it:

speech → transcription → translation → voice synthesis → playback

Each stage adds time, and each external service introduces another place where the pipeline can fail. Making those systems work together while keeping the conversation responsive was much harder than simply getting each API to work independently.

Voice introduced another challenge. Translation alone was not enough for what we wanted to build. We wanted the translated result to preserve the speaker's voice while still making voice enrollment explicit and controlled by the user.

We also spent a lot of time on the product experience. With this much happening behind a conversation, it would have been easy to expose every technical step in the interface. Instead, we tried to make Verba feel like communication software: join, choose your language, and talk.

What we learned

We learned that real-time translation is an orchestration problem as much as a language problem.

A strong translation does not matter much if it arrives too late. Fast synthesis does not help if session state is wrong. And adding more technology does not automatically make the conversation better.

We also came away thinking differently about voice. In a conversation, meaning is carried by more than the words themselves. Who is speaking matters too.

That became one of the principles behind Verba: translate the language without unnecessarily replacing the person.

What's next for Verba

The current version of Verba focuses on conversations happening directly inside Verba.

The next step is to make that language layer available where conversations already happen. Our roadmap includes integrations with platforms such as Zoom, Google Meet, and Microsoft Teams, while keeping Verba's own calling experience available.

We also want to validate more language pairs, reduce end-to-end translation latency, improve voice preservation across more speakers and languages, and continue developing the post-call collaboration experience.

Beyond the feed, we think there is room for technology that does something simpler: help two people understand each other directly.

Built With

Share this project:

Updates