Inspiration

Road accidents caused by driver fatigue and distraction remain a major global safety challenge. Many advanced driver monitoring systems rely on expensive hardware or continuous cloud connectivity, which makes them impractical for low-cost vehicles and remote regions. During the hackathon, we were inspired to design a lightweight AI-based driver monitoring solution that can work in real time using only a camera and an edge device. Our goal was to combine deep learning and computer vision to create an affordable and intelligent safety assistant that can operate completely offline. By leveraging lightweight neural networks and efficient computer vision techniques, we aimed to build a system that could potentially run on edge devices such as Raspberry Pi, enabling real-time driver monitoring without relying on internet connectivity.

What it does

Vision-Based Driver Drowsiness and Distraction Detection Using Lightweight CNN is an AI-powered driver monitoring system that analyzes a driver's facial behavior in real time. The system detects unsafe driving conditions such as: • Drowsiness • Prolonged eye closure • Yawning • Driver distraction A lightweight Convolutional Neural Network based on MobileNetV2 classifies the driver’s state into three categories: • Active • Distracted • Drowsy To improve reliability, the CNN predictions are combined with facial behavior metrics, specifically: • Eye Aspect Ratio (EAR) for detecting eye closure • Mouth Aspect Ratio (MAR) for detecting yawning When unsafe behavior is detected, the system immediately triggers: • Real-time audio alerts • On-screen visual warning messages The entire system is designed for edge-based execution, enabling fast inference and low latency without depending on cloud services.

How we built it

We built the system using a hybrid AI pipeline that combines deep learning with classical computer vision techniques. First, we trained a lightweight CNN model using MobileNetV2 with transfer learning in TensorFlow/Keras. The pretrained base network was frozen, and a custom classification head consisting of GlobalAveragePooling, Dropout, and Dense layers was added to perform multi-class prediction for Active, Distracted, and Drowsy driver states. For training, we used the Adam optimizer with categorical cross-entropy loss, and implemented Early Stopping to prevent overfitting during training. To make the model suitable for edge-device inference, the trained model was converted into TensorFlow Lite format, and optimization techniques such as model quantization were applied to reduce model size and improve inference speed. For real-time video processing, we used OpenCV to capture frames from the camera and perform preprocessing operations such as resizing and normalization. To analyze facial behavior, MediaPipe FaceMesh was integrated to extract facial landmarks. Using these landmarks, we calculated: • Eye Aspect Ratio (EAR) to detect eye closure • Mouth Aspect Ratio (MAR) to detect yawning These rule-based metrics were combined with CNN predictions to create a hybrid detection system, improving the robustness of the final output. The system is designed with edge-device deployment in mind, where it could run efficiently on hardware such as Raspberry Pi or similar low-power devices. To ensure stable real-time performance, we implemented several optimizations: • Reduced input resolution • Controlled frame rate (~10 FPS) • Periodic CNN inference instead of every frame • Efficient FaceMesh execution to reduce computation For alert generation, we implemented a local audio warning system using Pygame and displayed visual warning overlays using OpenCV. All processing is performed locally on the device, ensuring low latency and independence from cloud infrastructure.

Challenges we ran into

One of the major challenges was achieving real-time performance on limited hardware resources. Deep learning models can be computationally expensive, so designing a lightweight model while maintaining acceptable accuracy required careful optimization. Another challenge was reducing false alerts. CNN predictions alone could sometimes be sensitive to lighting variations or facial differences between users. To improve reliability, we integrated EAR and MAR based rule logic, creating a hybrid detection approach. Handling variations in lighting conditions, camera angles, and facial features also required additional preprocessing and extensive testing. Balancing accuracy, inference speed, and system stability was a key challenge throughout the development process.

Accomplishments that we're proud of

We successfully designed a lightweight AI-based driver monitoring prototype that demonstrates how real-time driver safety monitoring can be implemented efficiently on edge hardware. Key accomplishments include:

• Training and deploying a lightweight CNN model using MobileNetV2 • Optimizing the model for edge inference using TensorFlow Lite • Developing a hybrid detection approach combining CNN predictions with facial metrics • Implementing real-time alerts with minimal latency • Designing a system architecture suitable for edge devices like Raspberry Pi Most importantly, we demonstrated a scalable and cost-effective approach for intelligent driver safety monitoring.

What we learned

Through this project, we gained hands-on experience in building and optimizing AI systems for real-time applications.

We learned about:

• Transfer learning using MobileNetV2 • Freezing pretrained models and adding custom classification layers • TensorFlow Lite conversion and edge optimization • Real-time computer vision using OpenCV • Facial landmark detection using MediaPipe FaceMesh • Performance optimization for edge-based AI systems

We also learned how combining deep learning with rule-based logic can significantly improve reliability in real-world applications.

What's next

Future improvements for this system include:

• Adding head pose estimation for advanced distraction detection • Integrating seatbelt detection using computer vision • Improving robustness under extreme lighting conditions • Expanding the dataset to improve model generalization • Implementing adaptive threshold learning for EAR and MAR • Integrating the system with vehicle hardware interfaces

In the long term, this system could evolve into a smart driver safety assistant that can be integrated into commercial vehicles, public transport systems, and personal vehicles to reduce road accidents and enhance passenger safety.