ClassAssist.AI

Real-time AI that detects student disengagement via webcam and delivers personalized nudges — keeping every student on track while giving teachers live classroom insights.

Comment

Inspiration

We noticed that Gen-Z and Gen-Alpha are the most distracted generation in class. With smartphones, social media, and endless digital stimulation competing for students' attention, teachers are fighting an uphill battle — often without even knowing which students have mentally checked out. We wanted to give educators a superpower: the ability to know, in real time, exactly when a student is losing focus and respond instantly with a personalized nudge before they fall behind.

What it does

ClassAssist.AI monitors student engagement in real time using each student's own webcam. It uses computer vision to analyze head pose, gaze direction, and posture directly on the student's device. When a student is detected as disengaged, the system pulls the last 30 seconds of the teacher's live transcript, combines it with the uploaded lesson plan, and uses Claude AI to generate a personalized, context-aware nudge — delivered silently as a chat message on the student's screen. Teachers get a live dashboard with a heatmap of class engagement, a transcript timeline, and the ability to see every student's nudge history.

How we built it

We built ClassAssist.AI as four tightly integrated components:

  • CV Pipeline (Python): Uses YOLOv8n for person detection and custom algorithms to score head yaw/pitch, gaze, and slouch in real time from webcam frames.

  • Backend (Python + FastAPI): Receives disengagement events from the CV pipeline via Websocket, streams live audio through the Deepgram API for transcription, builds a $\pm30$ second context window, and calls the Anthropic Claude API to generate nudges. A WebSocket hub broadcasts events to both UIs simultaneously.

  • Student UI (Next.js + React): A clean chat interface where students receive AI-generated nudges in real time over WebSockets.

  • Teacher Dashboard (Next.js + React): A live view with a disengagement heatmap, transcript timeline with event pins, drag-and-drop seating chart, and expandable per-student chat logs.

Communicates over WebSocket's for sub-second latency end to end.

Challenges we ran into

The hardest challenge was working with OpenCV, since we didn't have any prior experience. We experimented a lot with adjusting the weights to mark a student engaged or disengaged. Originally, we were going to have ClassAssist.AI use one camera that would be above the teacher's whiteboard so it can see the entire class. The issue with this was that OpenCV struggled to track multiple rows of students. So we pivoted to using the student's own webcam instead, this helped with the tracking issues.

Accomplishments that we're proud of

We are proud that our project was completed with time to spare before the end of the hackathon, especially as we didn't have any prior Computer Vision experience.

What we learned

We also gained experience working with multiple real-time async systems — computer vision, audio streaming, WebSocket hubs, and LLM APIs — and keeping them in sync without race conditions or dropped events. On the AI side, we learned how much prompt context matters: the quality of Claude's nudges improved dramatically once we fed it both the live transcript and the lesson plan together.

What's next for ClassAssist.AI

We want to expand beyond disengagement detection to full emotional state recognition — identifying confusion, frustration, or boredom separately so nudges can be even more targeted. We're also exploring a post-class report for teachers that summarizes engagement trends across the lesson with timestamps, so they can identify which topics consistently lose the class. Longer term, we'd like to track the current tab a student is on to check if they're engaging with the material and integrate directly with LMS platforms like Canvas and Google Classroom

Built With

Share this project:

Updates