LearnHub

LearnHub is a Gemini-powered platform providing accessible education for all. By removing barriers for students of all abilities, we ensure every learner can study effectively without restrictions.

Comment

Inspiration

Growing up, I have seen how education often leaves students with disabilities behind. Even digital platforms fail to cater to the needs of students and help them learn at their own pace. Most platforms are "one size fits all," which really means they fit almost no one with unique needs. For a student with motor impairments, a simple mouse click is a barrier, for a blind student a complex math equation is a wall.

I was inspired to build LearnHub to turn the classroom from a rigid structure into an adaptive environment. I wanted to use AI not just to generate text but to act as a physical and sensory bridge between the content being taught and the student.

What it does

LearnHub is an AI-powered educational ecosystem designed to remove the physical, sensory, and cognitive barriers that prevent students with disabilities from accessing quality education. Unlike traditional platforms, LearnHub is entirely adaptive:

  • Hands-Free Learning: Students with motor impairments can navigate, search, and control lessons using only their voice via a real-time Gemini Live agent.

  • Technical Braille & Math: It converts complex STEM content and LaTeX equations into Grade 2 Braille and Nemeth Code, making science and math accessible to visually impaired students.

  • Granular Lessons (Micro-learning): To prevent cognitive overload, LearnHub automatically breaks down complex topics into bite-sized, "granular" modules. This allows students with ADHD or cognitive disabilities to master one concept at a time, providing frequent "wins" and reducing the mental fatigue associated with long-form lectures.

  • Multi-Modal Content: For students who require diverse learning inputs, the platform offers a "Focus Mode" and uses Imagen 4.0 to instantly turn abstract lesson concepts into clear, educational diagrams.

  • Universal Reach: With multi-language support (English, Spanish, Hindi) and natural Text-to-Speech, it breaks both accessibility and language barriers simultaneously.

How we built it

We built LearnHub using a modern, scalable stack centered around the Google Gemini AI suite:

  • Frontend: Built with React 18 and TypeScript, focusing on ARIA standards and keyboard-less navigation.

  • The Voice Agent: We utilized the Gemini Live API for bidirectional audio streaming. By implementing Tool Calling, we mapped voice intents (e.g., "Take me to the Physics quiz") directly to frontend routes using a custom VoiceAgentProvider.

  • Content Processing: Gemini 2.0 Flash acts as our intelligent parser, extracting structured data from PDFs and raw text while preserving complex mathematical LaTeX strings.

  • Accessibility Engines: We integrated the LibLouis open-source library to handle the heavy lifting of Braille translation and Gemini TTS for expressive, high-fidelity narration.

  • Backend & Storage: A Node.js/Express server manages the data flow, with PostgreSQL (Drizzle ORM) handling user progress and lesson metadata.

Challenges we ran into

Latency and Synchronization

The most significant hurdle was achieving near-instant feedback for a voice-first interface. Orchestrating a bidirectional WebSocket connection for the Gemini Live API while ensuring the React UI updated in real-time required sophisticated state management. We had to ensure the app didn't "fight" the user's voice input, meaning the system had to intelligently handle interruptions and state transitions without losing the context of the conversation.

Mathematical Accuracy & Technical Braille

Braille for mathematics (Nemeth Code) is incredibly sensitive to formatting. A single misplaced character can change the entire meaning of an equation for a visually impaired student. We had to develop rigorous prompt schemas to ensure Gemini 2.0 Flash consistently outputted perfectly formatted LaTeX. For example, to translate complex series like:

$$ sum_{i=1}^{n} i^2 = \frac{n(n+1)(2n+1)}{6} $$

We implemented a validation layer that checks the LaTeX syntax before it ever reaches the LibLouis translation engine. This ensures that the Braille output is 100% accurate, preventing any educational gaps in STEM subjects.

Accomplishments that we're proud of

  • Zero-UI Navigation: Successfully creating a platform that can be fully operated without a mouse or keyboard, providing true independence for students with motor impairments.

  • Bridging the STEM Gap: Automating the conversion of complex technical formulas into Nemeth Code, a feature that is largely absent from mainstream educational platforms, ensuring blind students have equal access to science and math.

  • Creating a Non-Existent Solution: We are proud of building a truly all-in-one accessibility ecosystem that doesn't exist elsewhere; while other platforms might offer simple text-to-speech, none provide the seamless integration of real-time voice control, automated Braille translation, and AI-generated visual aids in a single interface.

  • Seamless Integration: Orchestrating four distinct Gemini capabilities, including Live API for navigation, 2.0 Flash for content parsing, TTS for narration, and Imagen 4.0 for diagram generation, into a cohesive experience that feels intuitive rather than "clunky".

What we learned

We learned that Accessibility is the ultimate stress test for UX. When you design for the most extreme needs (like a user who cannot see the screen or use their hands), the resulting product becomes better and more intuitive for everyone. We also gained deep technical experience in Streaming AI architectures and learned how to leverage Tool Calling to make LLMs "interact" with a React DOM.

What's next for LearnHub

  • Institutional Integration & Personalized Roadmaps: We plan to develop a specialized dashboard for schools and institutions, allowing teachers to use Gemini's reasoning capabilities to generate personalized learning roadmaps tailored to each student's specific accessibility needs and pace.

  • Sign Language Avatars: We aim to integrate real-time sign language generation for deaf and hard-of-hearing students by leveraging Gemini’s video generation capabilities to translate audio and text into visual signing.

  • Offline Mode: To support students in low-bandwidth or remote areas, we are working on a lightweight, edge-compatible version of the Braille translator and core navigation features.

  • Collaborative Learning: We want to build inclusive voice-chat study rooms where a Gemini agent acts as a real-time moderator and accessibility bridge, translating between different modes of communication (e.g., voice to text for some, text to Braille for others) so students with different needs can study together.

Share this project:

Updates