Dash

Preventative AI driver monitoring with Arduino sensors and instant alerts.

Comment

Inspiration

Distracted driving causes thousands of accidents annually. We wanted to create a real-time monitoring system that could detect driver fatigue and dangerous driving patterns before they lead to accidents, providing immediate alerts to supervisors, parents, or fleet managers.

What it does

Dash is an AI-powered driver monitoring system that uses Arduino sensors and computer vision to detect distracted driving in real-time. The system combines:

  • Arduino Nano 33 BLE sensors for accelerometer, gyroscope, and driving behavior detection
  • OpenCV computer vision for facial landmark detection and drowsiness analysis
  • Dual AI models: one for distraction/drowsiness detection and another for driving behavior analysis
  • Real-time Bluetooth Low Energy (BLE) communication between Arduino and backend
  • Supabase database for live data population and user authentication
  • React dashboard with live driver status, safety scores, and trip analytics
  • Automated email notifications via SMTP for safety violations
  • ElevenLabs Text-to-Speech integration for audio alerts
  • Multi-user monitoring for parents, fleet managers, and ride-share companies

How we built it

Built as a React web app with three AI agents orchestrated through NVIDIA AgentIQ:

Monitoring Agent

  • Process Flow: Arduino sensors + Camera feed → OpenCV face detection → TensorFlow/PyTorch drowsiness detection → Raw distraction scores
  • Key Features:
    • Uses OpenCV haarcascade for facial landmark detection
    • Arduino BLE for real-time sensor data transmission
    • Dual models: computer vision for distraction + ML models for driving behavior analysis

Analytics Agent

  • Process Flow: Raw distraction scores + Historical data → Python data analysis → Statistical models → Safety scores + Alert levels
  • Key Features:
    • Pandas and NumPy for data processing
    • SQLAlchemy for database operations
    • Flask API for real-time data ingestion

Communication Agent

  • Process Flow: Alert severity + User preferences → Email API + Dashboard updates → Real-time notifications
  • Key Features:
    • SMTP integration for automated email alerts
    • Supabase for live dashboard updates
    • ElevenLabs Text-to-Speech for audio notifications

Challenges we ran into

Our built-in Arduino camera module was broken, forcing us to use iPhone camera as a temporary solution - an easy fix for future iterations.

Accomplishments that we're proud of

We successfully implemented live data population with Supabase, which was challenging but crucial for real-time monitoring. The Arduino's BLE Bluetooth capability proved invaluable for seamless wireless communication.

Also the demo video was our first take!

What we learned

Integrating hardware sensors with AI models requires careful calibration and testing. Real-time data streaming between Arduino and web applications has unique challenges that needs robust error handling and connection management.

What's next for Dash

  1. Arduino Camera Integration: Replace iPhone camera with Arduino-compatible camera module (OV7670/OV5640), process camera data directly on Arduino microcontroller, and eliminate Bluetooth video transmission for lower latency.
  2. Voice Analysis Integration: Implement voice analysis for detecting stress or impairment and add real-time audio processing for driver monitoring.

Built With

Share this project:

Updates