Emotions As Inputs

LINK TO VID CAUSE THE AREA DOESN'T ACCEPT DRIVE LINKS: https://drive.google.com/file/d/1eFDkQHZUQnHgP_g_DJLzOfKkiFLVYXZ9/view?usp=sharing

ServiceQuality: Measuring customer service representatives' performance by judging customer emotions.

AutoScroller: Hands-free YouTube Shorts navigation and automated video interactions powered by real-time facial expression tracking.

Project Description (Background & Capabilities)

ServiceQuality AI Dashboard reimagines performance analytics by automating customer satisfaction tracking through passive visual input. In customer-facing roles, manual surveys suffer from low response rates and bias. This module automates the entire feedback loop by leveraging an optimized OpenCV Haar Cascade filter alongside DeepFace neural networks. When a live session is activated, the engine isolates the primary customer's face, continuously tracks facial expressions frame-by-frame, and converts micro-expressions into real-time quality scores based on weighted emotional vectors (e.g., happy = +1.0, angry = -1.0). These scores are fed directly into a multi-threaded SQLite database layer that automatically calculates running performance metrics across multiple sessions. Conceived as a supportive driver-assist tool for managers rather than a rigid monitoring system, it showcases how human emotion can serve as a frictionless automated input stream to elevate service quality.

Challenges & Successes

My biggest challenge with this dashboard was handling computational bottlenecks and data integrity. Running deep learning emotion models frame-by-frame heavily bogs down system resources and causes real-time video lag. I solved this by developing an interval-skipping algorithm (ANALYZE_EVERY_N_FRAMES = 5) to drastically reduce processing overhead without losing data trends. Additionally, managing camera frame streams alongside a responsive database required strict multi-threading architecture. I built a custom CameraWorker thread and isolated database queries behind a thread lock (threading.Lock) to prevent visual freezes or memory corruption. My greatest success was creating a robust multi-face sorting algorithm that automatically isolates and tracks the largest face box by pixel area, ensuring the software intelligently filters out background noise and focuses exclusively on the customer being served.

Built With

• deepface
• python
• sqlite
• tkinter