ClassPulse

ClassPulse: Real-Time Multimodal Student Feedback Group Members: Sheza Kadri, Wafa Mustafa Durrani, Amania Qureshi

Project Description & Problem Solved

The Problem In online classrooms, teachers or professors can struggle with determining whether students truly understand a concept or are keeping their confusion silent. Students are often hesitant to unmute and speak up to share their confusion. Instructors may not realize this until evaluation results reveal the gap in understanding.

What We Built: ClassPulse ClassPulse turns webcam video into real-time student feedback using AI gesture detection (thumbs up, thumbs down, raised hand). It gives instructors instant alerts when students are confused, allowing them to adjust their lesson in real time. Thumbs up indicates understanding, thumbs down indicates confusion, and raised hand/open palm indicates that the student wants to speak or has a question. There are counters to allow professors and students to know how many students understand, are confused, or need to ask a question. This is to allow better pacing of lectures so that majority students needs can be fulfilled by professor.

How Gemini is Used in the Interaction Loop Gemini acts as the multimodal vision engine behind ClassPulse. It receives periodic webcam snapshots from students and analyzes visual cues such as hand gestures and body posture. We use Gemini to identify simple signals like a thumbs up (understanding), thumbs down (confusion), raised hand (question), or no gesture, along with a confidence level.
Technologies, APIs, and Hardware Used AI: We use the Google Gemini API with the official google-genai Python SDK to analyze webcam images and recognize simple student gestures like thumbs up, thumbs down, and raised hands. Backend: A Python FastAPI server receives the images, sends them to Gemini, and processes the structured results to track class-wide understanding and confusion. Frontend: We built the interface with HTML, CSS, and JavaScript. The browser accesses the webcam, captures occasional frames using the Canvas API, and sends them to the backend using the Fetch API. Hardware: ClassPulse works with a regular laptop, USB, or phone camera