AccessEd

We built an AI agent that knows the exact second you stop understanding, and fixes it instantly with personalised, multilingual support to make learning accessible to everyone

Comment

Inspiration

Education access isn’t equal around the world, and the gap in resources contributes to wider social and economic disparities. As international students, we’ve seen firsthand how lucky we are to have opportunities many others don’t. We wanted to create something that helps bridge that gap in any way we can. But accessibility isn’t just about geography or resources—it’s about moments in every student’s learning journey. Everyone knows that feeling: you’re watching a lecture, you get lost for just a few seconds, and suddenly everything that follows stops making sense. AccessEd is designed to catch those moments instantly and make learning truly personal and accessible for everyone, no matter where they come from or what language they speak.

What it does

The problem isn’t lack of content. It’s that no one knows when a student stops understanding. “We built a Chrome extension that detects confusion in real time and responds instantly—by simplifying the concept, generating flashcards, and translating it into the student’s preferred language.

How we built it

1. Confusion Detection & Tracking

We built a real-time system that detects exactly when a student stops understanding. OpenCV captures webcam frames while MediaPipe Face Mesh tracks eye aspect ratio, gaze, and head pose, producing a 0–1 confusion score. This lets the system adapt content instantly, creating a truly personalized learning experience.

2. Chrome Extension & Adaptive Study Widget

A persistent widget overlays every page, showing a live Pomodoro timer and confusion score. Background timers use Chrome Alarms API to keep sessions running even if popups close, ensuring continuous adaptive feedback.

3. AI-Powered Learning & Translation for Accessibility

FastAPI handles session data and connects to the Google Gemini API to generate flashcards, summaries, and translations. The translation feature supports 10+ languages—including Hindi, Urdu, Arabic, and Swahili—making study materials accessible to students in rural and underserved communities worldwide. This ensures that personalization also bridges language barriers, truly opening learning to everyone.

4. Web App & Notebook Dashboard

Our notebook-style dashboard displays session history, AI-generated flashcards, and translated study materials. By combining confusion tracking with AI and multilingual support, students receive a learning experience tailored to their pace, needs, and language—making study smarter, inclusive, and accessible.

Challenges we ran into

Adaptive Study Timer: Initially didn’t adjust correctly to confusion scores; fixed Chrome Alarms API integration to persist and update sessions. Feature Integration: Confusion detection, AI flashcards, and translation weren’t syncing; refactored data flow between Chrome extension and FastAPI backend. CSS & UI Rendering: Stylesheet wasn’t applying due to file path issues; corrected paths to ensure consistent widget and dashboard styling.

Accomplishments that we're proud of

Successfully integrated real-time camera functionality using OpenCV to detect user confusion, enabling our system to respond intelligently and support learners in the moment. Built a feature that extracts on-screen content and automatically generates flashcards, making studying more efficient and accessible.

What we learned

Working with OpenCV and MediaPipe taught us how to capture live video, track facial landmarks, and transform those signals into meaningful data, like detecting confusion in real time. Building a Chrome extension gave us hands-on experience with browser APIs, content scripts, and background services, showing us how to integrate features across web pages. We also learned the challenges and rewards of combining multiple technologies, real-time computer vision, AI APIs, and frontend interfaces, into a cohesive system.

What's next for AccessEd

Integration with popular platforms like Notion, Google Classroom, and Canvas for seamless study workflows. Expand regional language support to reach more students in underserved communities. Improve AI adaptation to individual learning styles for even more personalized learning. Gather user feedback to make AccessEd scalable and classroom-ready. Integrate it with a database to store user history for a more personalized learning experience.

Share this project:

Updates