Astra: AI Accessibility Layer for Education

Astra uses Vertex AI to make any lesson accessible. It adapts text or speech in real-time via filters for dyslexia, ESL, and ADHD, empowering teachers and personalizing learning.

Comment

Inspiration

Our inspiration comes from the critical need for true educational equity. Today's classrooms are diverse, but the materials are static. Teachers are overworked and cannot manually create five different lesson plans for one class to support students with dyslexia, non-native English speakers, and those with ADHD, all while challenging advanced learners.

We saw the power of Google's generative AI (Vertex AI) not just as a content creator, but as a real-time content adapter. We built Astra to bridge this accessibility gap, ensuring every student can receive material in the format they need to succeed, without adding any burden to the teacher.

What we learned

We learned that the true power of Vertex AI's Gemini models isn't just generating net-new content, but transforming existing data with specific, structured constraints. We dove deep into accessibility standards (like WCAG and dyslexia-friendly formatting) and the challenges of ESL education.

Technically, we learned how to build a cohesive pipeline chaining multiple Google Cloud APIs: converting live speech (Speech-to-Text) and scanned text (Vision OCR) into data, processing that data via Vertex AI, and serving it instantly via a serverless web app.

How we built it

Astra is a full-stack application built entirely on Google Cloud, designed for scalability and speed.

  1. Frontend: We built a clean, responsive, and accessible user interface using React.js.
  2. Ingestion Pipeline: When a teacher uploads content, we use the Google Cloud Vision API (OCR) to extract text from PDFs or images. For live lectures, we use the Google Cloud Speech-to-Text API for real-time transcription.
  3. Core Engine (The "Magic"): The ingested text is sent to our backend API (built in Node.js and deployed on Google Cloud Run). This server calls the Vertex AI (Gemini model) API. This is our core adaptation engine. We engineered specific prompts that act as "filters"—instructing the AI to:
    • Regenerate the text following dyslexia-friendly formatting rules.
    • Simplify the text to a B1 reading level and provide inline Spanish/French translations for ESL students.
    • Summarize the content into key bullet points for focus (ADHD).
    • Generate advanced critical-thinking questions for engaged students.
  4. Output: The adapted content is streamed back to the React frontend, allowing the student to toggle their required filter instantly.

Challenges we faced

The primary challenge was prompt engineering. Instructing an AI to "simplify a text" is easy; instructing it to strictly follow complex typographical formatting rules for dyslexia (like maintaining key concepts while changing line spacing and fonts) required significant refinement.

The second challenge was latency. For a live lecture transcription (Speech-to-Text) and subsequent adaptation (Vertex AI), the process had to feel instantaneous. This required optimizing our API calls and relying on Google Cloud Run's serverless speed to avoid network bottlenecks.

Built With

Share this project:

Updates