SENTINEL ZERO — See the Danger Before It Happens
Predictive industrial safety intelligence powered by OpenCV 5, multi-camera spatial tracking, and uncertainty-aware motion forecasting.
Inspiration
In an industrial workcell, a dangerous situation can develop before a collision actually occurs. A worker may approach an operating robot, disappear behind machinery, or remain exposed when equipment fails to stop as expected.
Traditional camera monitoring shows what is happening. We wanted to explore a harder question:
Can computer vision estimate what is likely to happen next, explain the emerging danger, and make that uncertainty visible?
That question became SENTINEL ZERO.
Our goal is to build an intelligent spatial monitoring system that transforms camera footage into a predictive understanding of workers, machines, and industrial hazards.
What It Does
SENTINEL ZERO is a working research prototype for a simulated industrial workcell.
It combines synchronized camera observations, native OpenCV 5 perception, world-coordinate tracking, trajectory prediction, and a telemetry-driven digital twin.
The system can:
- Detect workers, machines, and workcell objects from actual video pixels.
- Process observations from two calibrated camera viewpoints.
- Estimate object positions and velocities in a shared world coordinate system.
- Maintain global tracking identities across camera views and temporary occlusions.
- Forecast short-term motion and estimate future clearance between workers and machinery.
- Represent tracking uncertainty and distinguish observed states from predictions.
- Visualize positions, hazard envelopes, predicted trajectories, and risk estimates in an interactive 2D digital twin.
- Analyze whether machinery continues moving in a simulated stop-failure scenario.
The current implementation is a simulation-based perception and decision-intelligence prototype, not a certified industrial safety controller. It does not actuate real machinery.
How We Built It
1. A reproducible industrial simulation
We built a synthetic industrial workcell with two calibrated camera viewpoints and three principal scenarios:
Worker approach: A worker moves toward a robot operating region, allowing the system to estimate changing clearance and emerging risk.
Camera occlusion: One camera temporarily loses visibility while the other continues observing the worker.
Continued machine motion: Machinery keeps moving after a simulated stop acknowledgment, allowing the perception system to independently observe the motion.
The simulator generates videos and separate ground-truth data. Ground truth is used for evaluation, not as a shortcut for normal visual inference.
2. Native OpenCV 5 perception
We implemented a native C++ perception pipeline using OpenCV 5.0.0, with verified native library linkage.
The pipeline processes video frames, detects objects from image content, tracks their motion, and produces structured telemetry.
It also supports annotated video replay, camera calibration, bounded buffering, and native integration testing.
3. Cross-camera spatial intelligence
Camera-local tracks are combined into world-level observations using timestamp-aware geometric association and uncertainty-aware state fusion.
The spatial engine estimates positions and velocities, predicts short-term trajectories, and evaluates future proximity to machine hazard envelopes.
Our implementation uses deterministic geometry and sampled uncertainty estimates rather than asking a language model to make immediate safety decisions.
4. A telemetry-driven digital twin
We developed an interactive 2D workcell visualization showing tracked objects, their spatial relationships, predicted motion, and risk information.
The twin consumes output from the actual perception and spatial pipelines rather than replaying independent mock coordinates.
5. Verification and reproducibility
We treated verification as part of the project itself.
The OpenCV perception milestone passed its native build, tracking, calibration, and regression checks.
The spatial intelligence implementation also passed 73 Python tests, native integration checks, API checks, and browser visualization tests.
On the original scenarios, global worker identities remained stable, including a 16-frame occlusion interval. Worker position RMSE ranged from approximately 0.0074 to 0.0108 meters.
On six independent held-out scenario families, the risk evaluator detected 61 of 61 evaluable dangerous pair-ticks, but important limitations remained.
These measurements describe controlled synthetic tests, not real
Log in or sign up for Devpost to join the conversation.