Whale

Inspiration Modern classrooms are built around a one-size-fits-all model. A single lecture paced for the "average" student inevitably leaves someone behind. Hard-of-hearing students struggle with rapid speech and a lack of environmental awareness. International students fight through a double barrier of complex academic jargon and a non-native language. Neurodivergent students (ADHD, cognitive processing differences) often get overwhelmed by massive walls of text.

True accessibility isn't about giving everyone the same textbook; it's about giving everyone the textbook they can actually read. We wanted to build a "universal design" classroom companion that dynamically bends the lecture to fit the student, rather than forcing the student to fit the lecture.

What it does Whale is a real-time, AI-powered universal accessibility platform for education. The professor simply opens the Presenter Dashboard and speaks. Meanwhile, students join from their own devices and select an accessibility profile tailored to their needs:

Hard of Hearing (HoH) Profile: Provides a clean, high-contrast live transcript. Crucially, it uses browser-based audio detection to warn students of environmental sounds—popping up a visual alert if a fire alarm goes off or a bell rings. ESL / Translation Profile: Our AI intercepts the professor's dense academic jargon, simplifies it, and provides a real-time, side-by-side translation (e.g., Spanish) so international students never lose context. Cognitive Processing Profile: For students who easily suffer from cognitive overload, Whale aggressively condenses the transcript to a 3rd-grade reading level while strictly preserving core academic definitions. It dynamically builds a visual Knowledge Concept Graph on the fly, physically connecting the dots between concepts (e.g., ATP -> Mitochondria) as the professor speaks. Instant Study Guides: With one click at the end of the lecture, Whale analyzes the entire transcript and instantly broadcasts a customized, markdown-formatted Study Guide and Multiple-Choice Quiz to every student's device—automatically translated into their chosen language. How we built it Frontend: Built with React and Vite, featuring a custom, high-fidelity CSS design system. We intentionally avoided standard component libraries to ensure a highly accessible, distraction-free SaaS aesthetic. Real-time Infrastructure: We built a custom Node.js WebSocket relay to handle ultra-low latency broadcasting from the single presenter to hundreds of concurrent student viewers. AI Engine: We integrated Gemini 3.6 Flash-Lite to handle the heavy lifting. To minimize API roundtrips, we engineered complex system prompts that process transcription correction, cognitive simplification, language translation, and JSON graph-node extraction simultaneously in a single pass. Browser APIs: We utilized the Web Speech API and Web Audio API for continuous speech recognition and local environmental sound analysis. Challenges we ran into Concurrency & Latency: Sending raw audio text to an LLM every 2 seconds caused massive API bottlenecking and disjointed translations. We had to engineer a rolling-window chunking algorithm that intelligently stitches fragmented sentences together before sending them to Gemini. Schema Strictness with LLMs: Generating visual Concept Graphs requires strict, unbreakable JSON schemas. However, the lightweight/fast Gemini models we needed for real-time latency didn't fully support strict API responseSchema enforcement. We had to build a robust prompt-injection system that guarantees perfectly structured JSON outputs natively from the model. The Over-summarization Problem: When we asked the AI to simplify dense lectures to a 3rd-grade level, it began completely deleting the actual academic definitions! We spent hours prompt-tuning the model to strike the perfect balance: extreme cognitive accessibility without losing strict concept preservation. Accomplishments that we're proud of Achieving sub-second latency between the professor speaking and the AI-simplified, translated concept card appearing on the student's device. Successfully rendering dynamic, real-time knowledge graphs that visually build themselves as the professor teaches. Designing an interface that doesn't look like a rushed hackathon project, but rather a premium, enterprise-grade educational tool. What we learned The raw power and flexibility of WebSockets in modern React applications for decentralized state management. How to "tame" incredibly fast, lightweight LLMs to reliably output highly structured JSON schemas for UI consumption. Accessibility is deeply complex. We learned that auditory accessibility (HoH) requires a fundamentally different UI/UX paradigm than cognitive accessibility—you can't just slap a "make text bigger" button on a screen and call it accessible. What's next for Whale Multi-modal Interaction: Allowing students to use their microphone to ask the AI questions based purely on the real-time lecture context. Slide OCR Integration: Ingesting the professor's presentation slides via camera so the Concept Graph has visual context (e.g., mapping text to diagrams). Server-Side Transcription: Moving from local Web Speech APIs to a centralized transcription server (like Whisper) to handle high-noise lecture hall environments with absolute precision.