LinguaLoop

Inspiration

We noticed that millions of language learners using YouTube and other video platforms for “immersive learning” struggle with inefficiency and a fragmented experience. The constant loop of pausing, rewinding, and looking up words breaks focus and drains motivation. Existing tools either rely on cloud services (raising privacy concerns) or are too limited to deliver a smooth learning flow. We believe powerful, on-device AI can change this—creating an intelligent, seamless, and fully private language-learning companion.

What it does

LinguaLoop is an innovative Chrome extension that turns any YouTube video into your personal language tutor:

Smart Pause Looping: When you pause, LinguaLoop automatically snaps to the start of the current sentence and loops it, helping you focus on understanding.
AI-Powered Vocabulary Highlighting: On-device AI highlights the most learn-worthy words and phrases based on your proficiency level.
Seamless Navigation & Personalized Practice: Jump to the previous sentence to build context, save any sentence to a local word bank with one click, and get instant, personalized example sentences for saved vocabulary.
Privacy & Offline-First: All processing happens in the browser via Chrome’s built-in AI. Your study data and watch history never leave your device.

How we built it

Core stack: Chrome Extension (Manifest V3), Chrome Built-in AI APIs (Prompt API), modern JavaScript.

Architecture:

Content capture: A content script listens to and parses YouTube’s live caption stream to precisely obtain sentence text and timestamps.
AI processing core: A background service worker calls Chrome’s built-in AI—using the Prompt API for vocabulary extraction, example generation and instant translation.
User interface: A minimal, Material Design-inspired popup provides loop controls, highlighting, and word-bank management.
Data storage: All user data is saved locally via chrome.storage.local.

Challenges we ran into

Subtitle sync & sentence boundary detection: Getting complete sentences with exact time ranges in real time from YouTube’s dynamic captions was a core challenge.
Solution & credit: We drew on ideas from the open-source project za01br/yt-subtitle-extension to monitor and parse the YouTube caption container. By analyzing caption timing patterns and text content, we infer sentence starts/ends to achieve high-accuracy, sentence-level sync.
Prompt engineering for built-in AI APIs: Early outputs were unstructured or imprecise, requiring iterative instruction design for the Prompt API.
Performance & resource management: We tuned on-device models to avoid overusing system resources while keeping the UI responsive.

Accomplishments that we're proud of

A “magical” user moment: “Pause to loop” turns passive watching into focused, active learning.
Pushing the frontier: Among the first to put Chrome’s built-in AI APIs into real-world use—building a complex, fully client-side app with no cloud AI dependency.
Privacy by design: Powerful features with the principle that learning data never leaves the device.

What we learned

Client-side AI is ready: Models like Gemini Nano are now strong enough to handle complex language tasks on-device.
Prompts are product: In client-side AI, product intelligence and UX quality hinge directly on prompt design.
Chrome extension boundaries: We mapped what modern extensions can do across content-script messaging, complex state management, and media processing.

Built for the Google Chrome Built-in AI Challenge 2025.