SentinelNet: AI Edge Surveillance NVR

Inspiration

Modern CCTV systems are plagued by two major flaws: they are passive recording vaults that only tell you what happened after a break-in has occurred, and consumer smart cameras overwhelm users with constant false alarms caused by pets, shadows, or harmless motion. Furthermore, commercial security subscriptions continuously lock users into recurring cloud fees and pose serious privacy vulnerabilities by streaming private camera feeds to third-party servers.

We built Smart CCTV Guardian to transform any conventional CCTV or webcam into an active, intelligent, edge-native security station that detects actual threats in real time without sending private feeds to the cloud.

What it does

Smart CCTV Guardian is an enterprise-grade, edge-native 4-Channel AI Network Video Recorder (NVR) and Threat Suppression System. It operates across three unified modes: a Windows Desktop App (Electron), a Mobile Touchscreen App (PWA/Capacitor), and a Web Surveillance Station.

Key capabilities:

Real-Time 4-Channel NVR Station: Synchronous live multi-camera monitoring with instantaneous channel swapping, $2\times 2$ grid display, tactical military HUD overlays, and fluctuating on-screen bitrate & timestamp monitors.
YOLOv8 Multi-Object & Intruder Tracking: Tracks people, vehicles, and suspicious handheld objects at sub-50ms latency.
Intelligent False-Alarm Suppression:
- Implements a 2.0-second temporal presence rule ($\ge 2.0\text{s}$) to eliminate fleeting glitches and moving shadows.
- Features an anti-square phone filter verifying aspect ratios: $$\text{Aspect Ratio} = \frac{\max(w, h)}{\min(w, h)} \ge 1.42 \quad \text{and} \quad \text{Confidence} \ge 0.65$$
- Wrist-proximity validation ($< 95\text{px}$) ensures only items actively gripped by an intruder trigger alerts.
YOLOv8-Pose 17-Keypoint Posture Analysis: Identifies abnormal human behavior such as hands raised (robbery/surrender), stealth crouching/sneaking, and sudden falls.
YuNet + SFace Resident Whitelist: Automatically identifies authorized household members using 128-dimensional cosine feature vectors ($\text{Cosine Similarity} \ge 0.50$), suppressing sirens and highlighting them as AUTHORIZED.
Zero-Cloud Video Evidence & Alert Dispatch: Automatically records in-browser .webm evidence clips and dispatches instant SMTP email alerts with attached high-resolution JPEG evidence frames.

How we built it

Backend & AI Engine: Built with FastAPI, OpenCV (DirectShow) with hardware buffer optimization (CAP_PROP_BUFFERSIZE = 1), Ultralytics YOLOv8 (Object Detection & 17-Keypoint Pose Estimation), and YuNet + SFace for facial recognition.
Frontend Dashboard: Built with React (Vite) and styled with Tailwind CSS, featuring dark tactical HUD components, real-time telemetry, and Web Audio API synthesizer alarms.
Desktop Client: Packaged with Electron for zero-latency direct local IPC and window framing.
Mobile Experience: Fully responsive layout with mobile quick pills, swipeable camera cards, bottom navigation, and PWA/Capacitor configuration.
Multi-Camera Protocol Engine: Supports dynamic binding for external IP cameras over RTSP, HTTP/MJPEG, and local webcams with real-time OpenCV handshake validation.

Challenges we ran into

Camera Buffer Latency: Default OpenCV VideoCapture on Windows buffers frames internally, causing significant visual lag. We resolved this by integrating Windows DirectShow (cv2.CAP_DSHOW), enforcing a buffer size of 1, and synchronizing inference workers using Python threading.Condition primitives.
False Positives with Everyday Objects: Generic object detectors frequently classify books, notebooks, or mousepads as cell phones or weapons. We engineered mathematical bounding-box aspect ratio constraints and skeletal wrist-proximity verification to eliminate false alarms.
Face Embedding Performance: Extracting facial embeddings on continuous live feeds can exhaust CPU resources. We built an adaptive pipeline that skips SFace vector extraction unless an enrolled whitelist profile is active and a human face is clearly detected by YuNet.

Accomplishments that we're proud of

Achieving sub-50ms real-time AI inference and zero-buffer streaming on standard consumer hardware without requiring cloud GPUs.
Seamless triple-mode architecture running on Windows Desktop, Mobile, and Web from a single unified codebase.
Completely offline and privacy-first architecture: zero external cloud dependencies, zero subscription costs.

What we learned

How to tune low-level DirectShow hardware capture pipes and manage concurrent MJPEG streaming condition variables.
Combining multi-task computer vision pipelines (tracking + pose + face embeddings) into a unified inference loop without dropping frames.
Designing tactical, responsive UI interfaces that maintain high framerates while rendering continuous video streams and live telemetry.

What's next for Smart CCTV Guardian

Integrating thermal and pan-tilt-zoom (PTZ) camera controls over ONVIF protocols.
Adding spatial audio analytics (e.g., glass break and scream detection).
Building an automated edge backup sync to self-hosted NAS and IPFS storage.