SpectraVox : Inclusive AI Learning which is just not an imagination but a real world solution

Build by Mary Syokau a student at Jomo Kenyatta University of Agriculture and Technology, Kenya

The AI Companion for Students with Down Syndrome

A student-centric, multi-feature platform designed to make learning more accessible, engaging, and joyful.

Inspiration

Students with Down syndrome often thrive in learning environments that are visually engaging, patient, and present information in simple, single-concept steps. We noticed a gap in digital tools that are specifically designed to meet these needs from the ground up.

Our mission began with a simple question: "How can we use AI to create a truly adaptive and joyful learning experience for every child?"

We were inspired by:

*We were deeply moved to come up with the solution when we noticed that out of every 7 children in a group , 2 have down syndrome but only one or none manage to get standard education , thus millions to billions of kids world wide are left unaided and can't support themself and have to mostly rely on assistance.

The unique learning profiles of students with Down syndrome, who benefit from repetition, multi-sensory input (visual, audio), and positive reinforcement.
The dedication of teachers and guardians who spend countless hours adapting standard curriculum.
The potential for AI to automate content simplification, create personalized learning paths, and provide valuable insights into a student's progress and well-being.
The desire to build a tool that empowers students, supports teachers, and reassures guardians.

SpectraVox by AptX is our answer—a platform that champions inclusive education and celebrates every learning achievement.

What It Does

SpectraVoX is a holistic web application designed as a complete learning and support ecosystem for students with Down syndrome, their teachers, and their guardians.

Key Modules

Curriculum Upload & AI Simplification: Teachers can upload existing curriculum documents (PDF, DOCX). Our AI then processes the content, simplifying complex language, generating child-friendly descriptions, creating visual storyboards with AI-generated images (Imagen 2), and producing audio narration (Cloud Text-to-Speech).
Adaptive Learning Player: Students engage with lessons through an interface designed for cognitive accessibility. It features large buttons, one concept per screen, clear audio playback, repeat options, minimal text, and a high-contrast, calming color scheme.
Adaptive Exam Generation: From the simplified curriculum, the platform generates accessible exams with multiple-choice (2-3 options), image-based, and audio-based questions. The difficulty adapts based on student performance.
Guardian Progress Dashboard: Guardians get a clear and positive overview of their child's journey. The dashboard tracks progress, time spent, exam scores, and provides insights into their emotional well-being through features like the "Mood Mirror" and "Positive Mood Streak".
Daily Emotional Check-in: A simple, visual interface for students to log their mood for the day. This feature helps build emotional awareness and provides non-intrusive data for the guardian's well-being dashboard. The student can also use their device's camera to have the AI analyze their emotion.
AI-Powered Chatbot (SpectraVoX Ai): A friendly, patient, and emotionally intelligent AI assistant that understands the needs of users with Down syndrome. It responds with both text and audio, and can help users navigate to different parts of the application.
Role-Based Dashboards: Separate, secure dashboards for Students, Teachers, and Guardians, each tailored with the specific tools and information they need.
Community Hub: Dedicated spaces for students, teachers, and guardians to connect with peers, mentors, and experts for support and collaboration.

How We Built It

Tech Stack

Framework: Next.js (with App Router)
UI Library: React
Language: TypeScript
Styling: Tailwind CSS
UI Components: ShadCN UI
Generative AI: Firebase Genkit + Google Gemini Models
Database: Firestore
Authentication: Firebase Authentication
AI Infrastructure: Raindrop (SmartInference, SmartMemory, SmartBuckets)
Text-to-Speech: ElevenLabs API
Payments & Subscriptions: Stripe
Cloud Infrastructure: Vultr
Deployment: Raindrop + Netlify/Vercel

Core Architecture

+--------------------------------+
|      Client (Browser)          |
|  (Next.js, React, ShadCN UI)   |
+--------------------------------+
             |
             | User Interaction (e.g., clicks a button)
             v
+--------------------------------+
|   Server (Next.js App Router)  |
|      (Server Actions)          |
+--------------------------------+
             |
             | Function Call (e.g., simplifyCurriculum(content))
             v
+--------------------------------+
|       AI Layer (Genkit)        |
|          (AI Flows)            |
+--------------------------------+
             |
             | API Request
             v
+--------------------------------+
|  Google Generative AI Models   |
|         (Gemini, Imagen)       |
+--------------------------------+

AI Flows Built

simplify-curriculum-content: Takes raw curriculum text and breaks it down into simple, easy-to-understand concepts and child-friendly descriptions.
generate-adaptive-assessments: Creates accessible quizzes (multiple-choice, image-based, audio) from the simplified curriculum content.
tts: Converts any string of text into a clear, natural-sounding audio narration data URI using a text-to-speech model.
real-time-adaptive-remediation: Analyzes student performance to suggest remediation if a student is struggling with a concept.
chatbot: Powers "SpectraVoX Ai", providing empathetic, audio-enabled responses and navigation assistance.

Key Integrations

Raindrop (LiquidMetal): Powers our AI infrastructure with SmartInference for real-time content adaptation, SmartMemory for conversation persistence, and SmartBuckets for secure student data storage.
ElevenLabs: Provides natural, high-quality text-to-speech audio generation for all learning content and accessibility features.
Stripe: Handles subscription management, payment processing, and billing for premium tiers (Pro, Pro+).
Vultr: Provides reliable, scalable cloud infrastructure for hosting and managing the application with global edge network for low-latency performance.
Google Generative AI: Gemini models for content simplification and intelligent responses; Imagen 2 for visual storyboard generation.

Challenges We Ran Into

Designing a UI that is extremely simple and accessible without feeling restrictive.
Fine-tuning AI prompts to generate content that is genuinely appropriate and encouraging for students with Down syndrome.
Integrating multiple user roles (Student, Teacher, Guardian) with distinct but interconnected experiences.
Ensuring real-time data synchronization between student actions and the guardian dashboard.

Accomplishments That We're Proud Of

Building a truly multi-user platform that serves the entire support system around a student.
Creating a user experience that prioritizes calmness, clarity, and positive reinforcement.
Successfully integrating multiple GenAI capabilities (text simplification, image generation, TTS, chat) into a single, cohesive application.
Designing an application that goes beyond academics to include emotional well-being as a core feature.

What’s Next for SpectraVoX

We plan to grow the prototype into a full learning companion with:

Deeper personalization of lesson content based on individual progress.
Gamified learning milestones and rewards to boost engagement. *Come up with mobile app flutter based and low latency app *Integrate ussd and sms channel to enhance knowledge
Teacher-facing analytics to identify classroom-wide learning trends.
Offline mode for lessons to ensure accessibility in all environments.
Integration with school information systems (SIS).