Inspiration
Every day, millions of students—especially English Language Learners (ELL), students with dyslexia, and those behind grade-level reading—sit down in front of standard middle and high school science and history textbooks and hit an insurmountable lexical wall.
A typical 9th-grade textbook page averages over $26.5$ words per sentence and is loaded with dense academic syntax: $$\text{Cognitive Load} \propto \frac{\text{Syntactic Complexity} \times \text{Unfamiliar Lexicon}}{\text{Working Memory Capacity}}$$
The core concepts (such as cellular respiration or constitutional amendments) are not beyond their intellectual capability, but the syntactic density blocks comprehension. We built Bridge to turn every physical textbook into an open door, delivering genuine educational equity in any classroom and any language.
What It Does
Bridge is an end-to-end multimodal cognitive equalizer for classrooms and independent learners:
- Multimodal Textbook Ingestion: Snap a camera photo or upload an image of any physical textbook page.
- Dynamic Cognitive Leveling: Our Gemini multimodal engine extracts the text, measures reading difficulty, and reconstructs the content into bite-sized, accessible sentences (from Grade 9+ down to Grade 2–4 levels) while maintaining 100% conceptual and scientific rigor.
- Dual-Language Scaffolding: Pairs every leveled sentence with instant bilingual translations across 8 global languages (Spanish, Arabic, Hindi, Tagalog, Vietnamese, French, Mandarin, and simplified English).
- Synchronous Multi-Sensory Audio: Dual-channel synchronous speech narration with real-time text highlighting to enhance phonological awareness.
- Interactive Academic Vocabulary Anchors: Spaced-recall 3D flashcards with phonetic transcription guides (IPA) and native-language definitions.
- Adaptive Comprehension Checks: Multi-strand quizzes (Concept Recall, Cross-Anchor, and Cause-Effect) with instant bilingual pedagogical rationales.
- Accessibility & Readability Telemetry: Real-time algorithmic tracking of Flesch Reading Ease gains ($\Delta +64$ points), cognitive load reduction ($-75\%$), and student mastery trajectories.
- One-Click Printable Worksheet Export: Generates clean, printer-ready dual-language classroom handouts and teacher answer keys for hybrid or offline classrooms.
How We Built It
- Frontend Architecture: Modern React 18 + TypeScript + Vite styled with Tailwind CSS for responsive, accessible interaction.
- Multimodal AI Pipeline: Powered by Google Gemini Multimodal AI for high-precision OCR extraction, semantic analysis, syntactic simplification, and cross-lingual translation.
- Audio & Speech Engine: High-performance Web Speech Synthesis API providing synchronous bilingual voiceover and word-by-word tracking.
- Telemetry & Readability Algorithms: Real-time mathematical scoring using standardized Flesch-Kincaid Grade Level and Flesch Reading Ease formulas: $$206.835 - 1.015 \left(\frac{\text{total words}}{\text{total sentences}}\right) - 84.6 \left(\frac{\text{total syllables}}{\text{total words}}\right)$$
- Print & Export Engine: Custom CSS print media layout system producing ready-to-use classroom handouts with dual-column layouts and quiz modules.
Challenges We Faced
- Zero Instructional Distortion: Simplifying sentence structure without stripping away critical domain vocabulary or altering causal relationships in science and history.
- Cognitive Load vs. Engagement: Designing a multi-sensory UI that assists struggling readers without cluttering their visual field or overwhelming their working memory.
- Synchronous Multi-Language Alignment: Ensuring that audio read-aloud timings smoothly correlate with both the primary language and translated sentence anchors.
What We Learned
- Multimodal AI can act as a profound bridge between rigid, static physical learning materials and adaptive, personalized education.
- Providing dual-language scaffolding alongside simplified English significantly accelerates second-language vocabulary retention compared to translation alone.
- Real-time readability telemetry empowers educators to instantly see whether curriculum adaptations meet their students' specific IEP or ELL requirements.
What's Next for Bridge
- Speech-to-Speech Pronunciation Coach: Allowing students to read aloud into the microphone and receive gentle, real-time phonetic feedback.
- LMS Integration: Direct Google Classroom and Canvas export for automatic student gradebook synchronization.
- Offline Edge Mode: Local on-device lightweight models for rural classrooms with low or intermittent internet connectivity.
Built With
- accessibility
- artificial-intelligence
- bilingual-education
- edtech
- flesch-kincaid
- gemini-api
- google-genai
- literacy
- lucide-react
- multimodal-ai
- natural-language-processing
- ocr
- react
- responsive-design
- speech-synthesis
- tailwind-css
- typescript
- vite
Log in or sign up for Devpost to join the conversation.