Talk Bridge

• 

  screen-capture

Inspiration

My family members and many other non-native English speakers often feel unprepared when an English phone call comes in. Fast speech, unfamiliar accents, poor phone audio, and the pressure to answer immediately can make an ordinary call about a medical appointment, home repair, or bank account surprisingly stressful.

I wanted to help without taking over the conversation. TalkBridge helps the user understand what was said, prepare a response, and speak for themselves with more confidence.

What it does

TalkBridge is a real-time assistant for English calls in desktop Chrome. It shows both sides of the English conversation, provides Simplified Chinese translations, and suggests three short ways to reply: answer directly, ask for clarification, or move to the next step. At the end of the call, TalkBridge can produce a summary and key action items.

It supports two ways to capture a call:

• For a call in Chrome, it captures the selected tab and the user's microphone separately.
• For a phone on speaker, one microphone captures both people. A short voice calibration identifies the user, then speaker diarization separates the conversation into You and Call partner.

TalkBridge never invents missing personal details. It uses placeholders when unknown user information is being asked. The user always decides what to say. TalkBridge does not join the call or speak to the other person automatically.

The project also includes real-pipeline prerecorded demos, local recording upload, and an interactive role-play in which OpenAI Realtime acts as a fictional property manager.

How we built it

The frontend uses React, TypeScript, and Vite. The backend uses Node.js, Express, and WebSockets.

An AudioWorklet converts browser audio to 16 kHz mono PCM.Chrome calls use two independent Deepgram Nova-3 streams, so each audio source already has a reliable speaker label. Phone-speaker calls use one mixed Nova-3 stream with diarization. The user's calibration phrase establishes the initial speaker mapping, and the user can swap labels if the acoustic conditions confuse the model.

Final transcript segments go to the OpenAI Responses API using GPT-5.6. Incremental translation, full-sentence correction, reply suggestions, and call summaries are separate tasks, so each can be tuned for the latency and reasoning it needs. OpenAI Realtime powers the interactive role-play over WebRTC, but TalkBridge still transcribes the actual audio through Deepgram rather than using Realtime text events as a shortcut.

Recordings remain in the current browser page for segment replay and an optional mixed WAV download.

Challenges we ran into

The hardest problem was identifying two people from one microphone. Deepgram returns speaker numbers, not roles, and those numbers can shift during a streaming call. TalkBridge asks the user to read one fixed sentence before the call, maps that speaker to You, excludes the calibration from the transcript and recording, and applies that identity to the rest of the session. Built-in demos use the same connection for calibration and playback so the test matches the live path.

Mixed audio also made feedback control important. Playing a suggested reply or replaying a transcript segment could be captured again as new speech, so TalkBridge pauses the mixed input during local playback. Noise, echo, and overlapping speech remain real limitations, so the interface also supports label swapping and warns when more than two speakers are detected.

Accomplishments that we're proud of

The full path works end to end: real audio, live transcription, speaker labels, Chinese translation, reply guidance, pronunciation, segment replay, recording download, and a call summary.

The phone-speaker mode brings the same experience to calls where a separate remote audio track is unavailable. It works with a live microphone, two built-in two-person scenarios, or a locally decoded uploaded recording.

The demos do not use prepared subtitles or prepared responses. Only the audio is prerecorded; it still travels through the real Deepgram and GPT-5.6 pipeline at approximately real-time speed.

What we learned

Clear speaker identity matters as much as transcription accuracy. A correct sentence under the wrong name can produce the wrong reply suggestion, so calibration, mapping, and correction controls have to be part of the product.

I also learned to balance latency and quality of AI models. Translation benefits from low latency, while reply suggestions and summaries need more reasoning. Keeping those paths independent made the experience faster and easier to test.

Realistic demos are most useful when they exercise the production pipeline. Synthetic audio is valuable for repeatable testing, but it should never replace the live transcription and assistance logic being demonstrated.

What's next for TalkBridge

• Test phone-speaker mode with more voices, accents, devices, room layouts, and noisy environments.
• Improve handling of overlapping speech and speaker-label drift.
• Expand accessibility testing, including keyboard navigation and screen readers.
• Add assistance for more languages after the English-call experience is reliable.

Built With

• audioworklet
• deepgram
• express.js
• gpt-5.6
• node.js
• realtime
• vite
• webrtc