> **ClassPulse** is an AI-powered real-time classroom attention monitoring system that uses **YOLOv8 Pose Estimation** to detect attentive, distracted, sleeping, and phone-using students β delivering instant feedback through a live web dashboard and **ESP32 IoT hardware alerts**.
π Table of Contents
- Overview
- Features
- System Architecture
- Tech Stack
- Hardware Requirements
- Project Structure
- Getting Started
- Configuration
- How It Works
- MQTT Topics
- ESP32 Behaviour
- Troubleshooting
- Author
π― Overview
ClassPulse addresses a real problem in modern education β teachers have no scalable way to know if students are paying attention. This system uses a single webcam, AI pose estimation, and IoT hardware to give teachers instant, data-driven attention feedback without any wearables or student-side hardware.
What makes it different:
- No special student hardware β just a camera
- Works with any laptop webcam (GPU supported, CPU fallback)
- Real-time per-student attention scoring using head pose geometry
- Physical IoT feedback β RGB LED + buzzer on teacher's desk
- Session reports with exportable CSV data
β¨ Features
| Feature | Description |
|---|---|
| π§ YOLOv8 Pose | 17-point skeleton detection per student |
| π Head Pose Estimation | Yaw + pitch angle from facial keypoints |
| ποΈ Eye State Detection | Closed-eye streak detection for sleeping |
| π± Phone Detection | YOLOv8n COCO detects cell phones near students |
| π Live Dashboard | Flask + Chart.js with MJPEG stream + SSE updates |
| π΄ IoT Alerts | ESP32 β RGB LED colours + passive buzzer tones |
| ποΈ Session Logging | SQLite stores every scan with timestamps |
| π Report Generation | Per-session HTML report + CSV export |
| π‘ MQTT Integration | Mosquitto broker bridges Python β ESP32 |
| π― Persistent Tracking | YOLOv8 .track() gives each student a stable ID |
ποΈ System Architecture
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β ClassPulse System β
β β
β [Laptop Camera] β
β β β
β βΌ β
β βββββββββββββββββββββββββββββββ β
β β detector.py (Thread) β β
β β YOLOv8-pose ββββββββββββ 17 keypoints per student β
β β YOLOv8n ββββββββββββ phone detection (class 67) β
β β attention.py ββββββββββββ head pose + eye state β score β
β ββββββββββββββββ¬βββββββββββββββ β
β β β
β ββββββββ΄βββββββ β
β βΌ βΌ β
β database.py mqtt_handler.py β
β (SQLite) (Mosquitto) β
β ββββββββ¬ββββββββ β
β βΌ β
β app.py (Flask) β
β βββββββββββββββββββββββ β
β β /video_feed MJPEG ββββΊ Browser Dashboard β
β β /stream SSE ββββΊ Live Chart.js updates β
β β /report HTML ββββΊ Session report + CSV β
β βββββββββββββββββββββββ β
β β β
β MQTT Broker β
β β β
β ββββββββΌβββββββ β
β β ESP32 β β
β β RGB LED β Green/Blue blink/Red/Purple β
β β Buzzer β Tones based on attention level β
β βββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
π οΈ Tech Stack
| Layer | Library/Tool | Purpose |
|---|---|---|
| Detection | ultralytics YOLOv8 |
Pose + object detection |
| Vision | opencv-python |
Camera capture + annotation |
| Web | flask |
Server, MJPEG, SSE, REST |
| IoT comms | paho-mqtt |
MQTT publisher |
| Data | numpy, pandas |
Geometry + processing |
| Storage | SQLite |
Session + log database |
| Frontend | Chart.js 4.4 |
Real-time timeline charts |
| Embedded | ESP32 + PlatformIO |
IoT feedback node |
| Broker | Mosquitto |
MQTT message broker |
π Hardware Requirements
| Component | Qty | Notes |
|---|---|---|
| ESP32 Dev Board | 1 | Any variant |
| RGB LED (4-pin) | 1 | Common Cathode |
| 220Ξ© Resistors | 3 | One per R/G/B channel |
| Passive Buzzer | 1 | 2-pin, needs PWM |
| Breadboard | 1 | Half-size or larger |
| Jumper Wires | ~15 | Male-to-male |
| USB Cable | 1 | ESP32 power + flash |
| Laptop + Webcam | 1 | Built-in camera works |
Wiring β RGB LED (Common Cathode):
ESP32 GPIO 25 β 220Ξ© β LED Red pin (Pin 1)
ESP32 GPIO 26 β 220Ξ© β LED Green pin (Pin 3)
ESP32 GPIO 27 β 220Ξ© β LED Blue pin (Pin 4)
LED Common GND (Pin 2, longest leg) β GND rail
Wiring β Passive Buzzer:
ESP32 GPIO 18 β Buzzer + leg
GND rail β Buzzer β leg
How to wire a resistor: GPIO pin β resistor leg 1 β resistor leg 2 β LED colour pin. The resistor sits between GPIO and LED on the same breadboard row.
π Project Structure
ClassPulse/
βββ app.py # Flask entry point β run this
βββ detector.py # YOLOv8 background detection thread
βββ attention.py # Head pose + eye state β score
βββ database.py # SQLite session & log manager
βββ mqtt_handler.py # MQTT publisher
βββ config.py # All settings in one place
βββ requirements.txt
β
βββ templates/
β βββ index.html # Live dashboard UI
β βββ report.html # Session report with charts
β
βββ static/
β βββ css/style.css # Dark cyberpunk design system
β βββ js/dashboard.js # SSE listener + Chart.js
β
βββ esp32/
β βββ platformio_project/
β βββ platformio.ini
β βββ src/main.cpp # ESP32 firmware
β
βββ assets/ # Screenshots for this README
β βββ dashboard.png
β βββ detection.png
β βββ report.png
β βββ hardware.jpg
β
βββ sessions/ # Auto-created β SQLite DB
βββ reports/ # Auto-created β CSV exports
βββ .gitignore
βββ README.md
π Getting Started
1. Clone
git clone https://github.com/rafiul254/ClassPulse.git
cd ClassPulse
2. Install Python dependencies
pip install -r requirements.txt
YOLOv8 models download automatically on first run (~20 MB).
3. Start Mosquitto broker
# Windows (run as Administrator)
net start mosquitto
Open C:\Program Files\mosquitto\mosquitto.conf and ensure:
listener 1883
allow_anonymous true
Allow port 1883 through Windows Firewall:
netsh advfirewall firewall add rule name="Mosquitto MQTT" dir=in action=allow protocol=TCP localport=1883
4. Configure ESP32
Open esp32/platformio_project/src/main.cpp, edit the config block:
#define WIFI_SSID "YourWiFiName" // must be 2.4 GHz
#define WIFI_PASSWORD "YourPassword"
#define MQTT_BROKER "192.168.X.X" // your PC's IP (ipconfig)
Build and upload via VS Code PlatformIO: β Build β β Upload
5. Run
python app.py
Open: http://localhost:5000
βοΈ Configuration
All tuning constants live in config.py:
CONF_THRESHOLD = 0.50 # keypoint confidence gate
YAW_DISTRACTED_DEG = 25 # head turn β DISTRACTED
YAW_AWAY_DEG = 50 # head turn β fully sideways
PITCH_SLEEPING_DEG = 30 # chin drop β SLEEPING
EAR_CLOSED_FRAMES = 8 # consecutive closed-eye frames β SLEEPING
ATTENTION_THRESHOLD = 60 # class avg below this β alert fires
ALERT_COOLDOWN = 30 # seconds between repeat alerts
π§ How It Works
Attention Scoring
Camera quality gate β max face conf < 22% β UNCERTAIN (score 10)
Head yaw (left-right):
|yaw| < 25Β° β +45 pts (camera-facing)
|yaw| < 50Β° β +20 pts (partial turn)
|yaw| β₯ 50Β° β +0 pts (sideways)
Head pitch (down):
pitch < -30Β° β cap at 12 (sleeping)
Eye state:
Open β +20 pts
Closed β₯ 8 frames β cap at 12 (SLEEPING)
Confidence scaling:
avg_conf [0.35β1.0] β scale [0.5β1.0]
Phone nearby β hard cap at 25, state = PHONE
State Classification
| Score | State |
|---|---|
| 65β100 | π’ ATTENTIVE |
| 30β64 | π‘ DISTRACTED |
| 0β29 | π΄ SLEEPING |
| any | π± PHONE |
| any | βͺ UNCERTAIN |
π‘ MQTT Topics
| Topic | Payload |
|---|---|
classpulse/stats |
{"class_attention": 72, "total_students": 5, ...} |
classpulse/alert |
{"level": "danger", "message": "Attention dropped..."} |
Test manually:
mosquitto_pub -h localhost -t "classpulse/stats" -m "{\"class_attention\":85}"
mosquitto_pub -h localhost -t "classpulse/alert" -m "{\"level\":\"danger\",\"message\":\"Test\"}"
π‘ ESP32 Behaviour
| State | RGB LED | Buzzer |
|---|---|---|
| Boot | White sweep β R β G β B | C-E-G-C melody |
| WiFi connecting | Yellow blink | β |
| MQTT connected | Cyan double flash | Double beep |
| Attention β₯ 70% | π’ Solid Green | Silent |
| Attention 45β69% | π΅ Blue blink | Silent |
| Attention < 45% | π΄ Solid Red | Warn beep |
| Alert warning | π Purple Γ2 | Double warn |
| Alert danger | π Purple Γ4 | Triple alarm |
π§ Troubleshooting
| Problem | Fix |
|---|---|
| Camera not found | Change CAMERA_INDEX = 1 in config.py |
MQTT rc=-2 |
Open port 1883 in Windows Firewall |
| ESP32 won't connect | WiFi must be 2.4 GHz β ESP32 doesn't support 5 GHz |
| Score always high | Lower MIN_FACE_CONF in attention.py |
| Too many false SLEEPING | Increase EAR_CLOSED_FRAMES to 12+ |
π Medium Article
Full technical deep-dive on Medium: ClassPulse: I Built an AI That Detects If Students Are Sleeping in Class
