Inspiration In an increasingly remote and global workforce, video calls bring together people from diverse linguistic backgrounds and technical domains. We noticed that non-native speakers, students, and professionals frequently struggle to keep up with fast-paced discussions, domain-specific jargon, or regional accents. Stopping a meeting to search for an unfamiliar word breaks conversation flow. We were inspired to create a tool that seamlessly bridges language barriers and turns live meetings into opportunities for effortless learning without leaving the call.
What it does VerbaSync is a Google Chrome extension designed for real-time video conference assistance. During live web calls, it:
Transcribes: Captures meeting audio and generates low-latency, real-time live captions.
Translates: Converts transcribed spoken dialogue into target languages instantaneously.
Defines: Automatically detects complex vocabulary or newly introduced terms appearing in conversation and provides context-aware definitions on the fly via a clean, non-intrusive visual overlay.
How we built it Frontend: Developed as a Google Chrome Extension (Manifest V3) using modern HTML, CSS, and JavaScript to construct a lightweight, customizable side-panel overlay.
Audio Capture: Leveraged browser-level audio capture capabilities via Web Audio APIs and the Chrome Tab Capture API to isolate and process live meeting streams.
Transcription & Translation Engine: Integrated natural language APIs to handle continuous speech-to-text processing and streaming translation.
Lexicon Processing: Built a lightweight keyword extraction script paired with dictionary and LLM-backed APIs to contextualize and define key terms in real time.
Challenges we ran into Audio Stream Synchronization: Capturing, chunking, and routing clean browser audio without introducing latency or echo back into the meeting proved tricky.
Real-Time UI Performance: Rendering dynamic live captions and updating vocabulary panels simultaneously without freezing or slowing down the browser page required careful state management.
Context-Aware Definitions: Ensuring the extension accurately identifies genuine new/complex words—rather than flagging common language or mistranscriptions—required fine-tuning our keyword filtering logic.
Accomplishments that we're proud of Building a fully functional Chrome extension within tight hackathon time constraints.
Achieving low-latency, synchronized live transcription and translation directly within active browser tabs.
Successfully designing an intuitive, non-disruptive user interface that enhances comprehension without distracting from the primary video call.
What we learned Hands-on experience working with browser media stream capture and real-time Web APIs under strict extension execution permissions.
The nuances of low-latency speech processing, prompt optimization for dictionary lookup, and efficient UI re-rendering in JavaScript.
Practical strategies for designing accessible tools that serve non-native speakers and neurodiverse learners.
What's next for VerbaSync Post-Call Summaries: Generate automated meeting notes, action items, and customized flashcard decks based on vocabulary introduced during the call.
Platform Expansion: Broaden cross-browser compatibility (Firefox, Edge) and standalone desktop integration.
Custom Dictionaries: Allow users or enterprise teams to upload custom glossaries, company-specific acronym lists, and technical documentation for personalized vocabulary extraction.
Log in or sign up for Devpost to join the conversation.