Noctua

Inspiration

Drowsy driving kills. Studies show 15-20% of all fatal traffic accidents are linked to driver fatigue, accounting for an estimated 1,200 deaths and 76,000 injuries every year in the US alone. We kept asking ourselves, your phone can unlock with your face, your laptop can detect your hands, so why isn't there an accessible, real-time tool that watches for the one thing that kills drivers every day: falling asleep at the wheel? Noctua is our answer!

What it does

Noctua is a real-time driver drowsiness detection desktop app. Using just a webcam, it monitors the driver's eyes and head position continuously. It calculates Eye Aspect Ratio (EAR) to detect eye closure and uses head pose estimation to catch nodding. When fatigue is detected, it triggers a staged alert system, which gives a warning at 3 seconds, a louder alarm at 5 seconds, and an automatic email alert to an emergency contact at 8 seconds. Every session is personally calibrated to the individual driver's eye shape, making it accurate for everyone.

How We built it

Built entirely in Python using MediaPipe Face Mesh for real-time 468-point facial landmark detection, OpenCV for camera capture and frame processing, and customtkinter for the desktop UI. The detection pipeline computes EAR and PERCLOS metrics per frame, backed by a published NIH research paper on driver fatigue detection. A shared state dictionary bridges the detection thread and UI thread so they run in parallel without blocking each other. Email alerts are sent via Gmail SMTP. The personal calibration system measures each user's open and closed EAR baseline before every session.

Challenges We ran into

Getting MediaPipe, OpenCV, and Tkinter to coexist was a serious battle, as all three frameworks wanted to own the main thread. We had to carefully sequence the app startup: emergency contact setup first, then calibration in a cv2 window, then hand off to the Tkinter UI with detection running in a background thread. We also hit Python 3.13 incompatibility with MediaPipe and had to pin to Python 3.12. Threading the detection loop while keeping the UI responsive at 30fps took several iterations to get right.

Accomplishments that We are proud of

Personal calibration is something most fatigue detection demos skip entirely, but we built it properly. Every driver gets their own EAR threshold based on their actual eye shape, which dramatically reduces false alarms. We also built a fully staged alert system that escalates from audio warning to emergency email, which makes it feel like a real product rather than a demo. Getting the full pipeline — camera → landmarks → EAR → PERCLOS → UI → alert — running end to end in a 24-hour sprint is something we're genuinely proud of.

What We learned

How to architect a multi-threaded Python desktop app where CV, audio, and UI all run simultaneously without blocking each other. The math behind EAR and PERCLOS from actual published research. How personal calibration dramatically improves detection accuracy over fixed thresholds.

Alert Flow

Eyes closed / Head down │ 3 seconds ──► STAGE 1 beep.wav (repeats every 2s) + STOP button │ Eyes open + no STOP ──► sound repeats until STOP pressed │ 5 seconds ──► STAGE 2 warning.wav (repeats) + STOP button │ 8 seconds ──► CRITICAL Email sent + ElevenLabs voice warning

Dismissing alerts:

Click STOP ALARM button on screen
Say "stop" (voice command)

Voice Commands

Command	Action
`"stop"` / `"halt"` / `"cancel"`	Dismiss active alert
`"hey smart car"`	Wake AI assistant
`"hey car, [question]"`	Ask question directly

Database Schema

sql users ( user_id TEXT PRIMARY KEY, first_name TEXT, last_name TEXT, pw_hash TEXT, -- SHA-256 personal_email TEXT, emergency_email TEXT, ear_threshold REAL, -- saved after calibration pitch_baseline REAL, -- saved after calibration -- Driver profile drive_frequency TEXT, -- daily / 2-3x/week / weekends / rarely vision_left TEXT, -- e.g. 20/20 vision_right TEXT, wears_glasses TEXT, -- no / glasses / contacts drive_time_of_day TEXT, -- morning / afternoon / evening / night / mixed avg_drive_duration TEXT, -- under30 / 30to120 / over120 drive_environment TEXT, -- urban / highway / mixed avg_sleep_hours TEXT, -- under5 / 5to6 / 7to8 / over8 caffeine_intake TEXT -- none / light / moderate / heavy )

sessions ( session_id INTEGER PRIMARY KEY, user_id TEXT, date TEXT, start_time TEXT, end_time TEXT, total_minutes REAL, avg_fatigue REAL, -- 0–100 worst_period_min REAL, -- minute into drive best_period_min REAL, alert_1_count INTEGER, alert_2_count INTEGER, critical_count INTEGER, perclos_avg REAL, blink_rate_avg REAL, timeline_json TEXT -- 5s snapshots: t, ear, pitch, blink, perclos, alert, score )

Report Grades

Grade	Condition
A	Avg fatigue < 15
B	Avg fatigue < 30
C	Avg fatigue < 50
D	Avg fatigue < 70
F	Avg fatigue ≥ 70 or any CRITICAL event

Features

Category	Feature
Detection	Eye closure (EAR), head pose (pitch), blink rate, PERCLOS, brow furrow
Alerts	3-stage system — audio beep → warning sound → email + ElevenLabs voice
Voice	"stop" to dismiss alerts, "hey smart car" to chat with Ollama
AI Chat	Ollama (llama3.2) local LLM — works offline
Auth	Sign in / Create account / Guest mode (SQLite)
Calibration	Personalized per user — EAR threshold + head pitch baseline
Driver Profile	Weekly drive frequency, vision, glasses, sleep, caffeine, environment
Reports	Per-session matplotlib report + cumulative history (grade A–F)
Fatigue Signals	Blink rate, PERCLOS, brow furrow — terminal output
Email Alert	Gmail SMTP — sends on CRITICAL with contact info
UI	CustomTkinter dashboard with EAR ring, PERCLOS arc, alert log
Weather	WeatherAPI used for weather conditions on greeting
Traffic	TomTom for traffic conditions announced on greeting

Project Structure

noctua/ │ ├── ui.py # Main app window (CustomTkinter) — run this ├── detection.py # Core detection loop (eye + head pose + alerts) ├── calibration.py # EAR + pitch calibration (3-step) ├── constants.py # Landmark indices, thresholds, EAR/pitch functions │ ├── auth.py # SQLite auth — sign in, create account, driver profile ├── session.py # Per-session data collection (5s snapshots) ├── report.py # Matplotlib reports — single session + history ├── fatigue.py # Blink rate, PERCLOS, brow furrow detector │ ├── voice.py # Speech recognition — "stop" + wake word ├── chat.py # Ollama LLM conversation ├── alert.py # ElevenLabs TTS (blocking + async) ├── sound.py # macOS afplay — beep.wav, warning.wav ├── sms.py # Gmail SMTP emergency email ├── emergency.py # Emergency contact input (OpenCV fallback) ├── weather_greeting.py # Weather + traffic context integration │ ├── config.py # API keys, environment variables, configuration ├── constants.py # Shared constants (thresholds, tuning params) │ ├── beep.wav # Stage 1 alert sound ├── warning.wav # Stage 2 alert sound ├── completion.wav # Voice interaction completion sound │ ├── icons8-alert-100.png ├── icons8-closed-eye-100.png ├── icons8-eye-100.png ├── icons8-up-arrow-100.png │ ├── Noctua Logo with BG.png # Branding asset ├── Noctua symbol no BG.png # Transparent logo ├── Welcome To Noctua.mp4 # Welcome screen video │ ├── users.db # SQLite database (auto-created) ├── .env # Environment variables (API keys, secrets) │ ├── MuseoModerno/ # Custom font assets ├── Inter/ # Custom font assets │ ├── font_test.py # Font rendering test ├── main.py # Alternative entry point │ ├── .gitignore

└── README.md

Tech Stack

Component	Technology
Computer vision	MediaPipe Face Mesh (468 landmarks)
UI	CustomTkinter
Database	SQLite (built-in)
Voice recognition	Google Speech Recognition (SpeechRecognition)
AI assistant	Ollama — llama3.2 (local)
TTS	ElevenLabs Turbo v2.5
Graphs	Matplotlib
Audio playback	macOS `afplay`
Email	Gmail SMTP (smtplib)
Daily Report	Tomtom Weatherapi