NodDetect

A web-based app that tracks movement in the face and eyes to detect fatigue in workers. Its function is to detect if the worker needs rest.

Comment

Inspiration

The inspiration for NodDetect came from recognizing a vulnerability in certain jobs with high-stakes environments, such as drivers and heavy factory work. Cognitive fatigue and microsleeps are huge safety hazards, but the current monitoring systems in place are invasive and use privacy-compromising cloud processing. Our solution guarantees privacy, and real-time accuracy, since the entire pipeline is within the browser.

What it does

NodDetect is a real-time, privacy-first cognitive fatigue detection system that runs entirely inside the web browser. Through the webcam, the system continuously tracks 468 landmarks on the user’s face. It then uses those metrics to determine their blink rate, gaze drift, and head angle. It calculates a “fatigue score” based on those metrics, and alerts the user if it’s getting to dangerous levels. It all happens locally, without biometric data ever being sent to a server.

How we built it

Built entirely on modern, in-browser ML without relying on external APIs. It combines several parts:

  • *Landmark Extraction: * We used MediaPipe FaceMesh (WASM) to continuously extract 468 3D facial landmarks
  • Biometric Signal Processing: * From these coordinates, the system calculates three key metrics in real time: *EAR (Eye Aspect Ratio)””, **Gaze drift, and **Head pose. These three indicators feed a heuristic model to display a live % and set system alarms accordingly.
  • *UI/UX: * The model outputs a continuous fatigue score, transitioning from a green baseline to an orange nudge, and a critical red alert.

Challenges we ran into

We'd a few challenges that complicate a little our project, these are:

  • Getting MediaPipe FaceMesh stable and fast in the browser while keeping frame rate smooth.
  • Tuning EAR thresholds so the system works across different faces and lighting conditions without false positives.
  • Calibrating gaze drift and head pose so the model adapts per user instead of hard‑coding values.

Accomplishments that we're proud of

We’re proud of the entire project, but especially that we achieved full real‑time fatigue detection running 100% in‑browser with no cloud dependency, built a responsive dashboard that shows live signals and clear alert levels, and delivered reliable microsleep and blink detection with a simple, explainable model.

What we learned

We learned that modern in‑browser ML is fast and reliable enough for safety‑critical prototypes when carefully optimized, and that calibration is the key to making biometrics work across different users and environments. We also saw that simple, explainable heuristics can outperform heavier models for real‑time demos, especially when you need stability, speed, and trust during a hackathon.

What's next for NodDetect

Next we plan to package NodDetect as a Progressive Web App so it can run offline on phones and tablets, expand the alert system with haptics/audio for real‑world use, and pilot with a specific high‑risk group (like long‑haul drivers) to validate thresholds in real conditions. We also want to replace the heuristic model with a lightweight on‑device ONNX/TFLite model trained on open fatigue datasets.

Built With

Share this project:

Updates