Sentinel

Inspiration

India's land borders run for thousands of kilometres through deserts, marshes, rivers, dense forest and high mountains. Much of this stretch is guarded by small teams of personnel, often at night, in extreme weather and with limited visibility.

Cameras are already being installed along fences and at border outposts, but cameras alone do not stop intrusions. Someone still has to watch every screen, every hour of every night. No human can stare at dozens of feeds without missing something, and most surveillance systems only record. The footage helps investigators after an incident, not the patrol that could have stopped it.

We kept coming back to one question: what if the cameras along the border could watch for themselves? Instead of operators scanning screens, the system would flag only what matters: a person cutting through the fence at 2 AM, a group moving along a nullah, a vehicle stopping near the border road, someone loitering near a vulnerable stretch. And it would tell the nearest patrol within seconds.

Sentinel is our answer. It moves border security from recording what happened to preventing what is about to happen.

What it does

Sentinel is an AI-powered surveillance and early-warning platform for border and perimeter security. It turns existing CCTV, IP and PTZ cameras along the fence, at outposts and on watchtowers into an intelligent detection network.

Works with existing cameras: it connects to fixed, PTZ and IP cameras over RTSP and ONVIF, so no new camera network is needed.
Analyses video at the edge: AI runs on-site at the outpost, so detection keeps working even when connectivity to headquarters is weak or cut off.
Detects and tracks people and vehicles, and follows each one across frames to understand its movement, not just its presence.
Understands threats, not just motion:
- Restricted zone intrusion: virtual zones along the fence line, no-go areas and sensitive stretches trigger alerts the moment someone enters.
- Line crossing with direction: virtual tripwires along the fence tell inward crossings apart from outward ones.
- Loitering: flags people or vehicles that stay near the fence or a sensitive point longer than normal.
- Vehicle and number plate recognition (ANPR): logs vehicles at check posts and on border roads, and flags unknown or watchlisted plates.
- Crowd and group movement: counts people and flags unusual groups.
Instant alerts: patrol officers get a mobile alert with a snapshot, a short clip, the location and a severity level. The command centre sees every event on one live map and alert feed.
Fewer false alarms: tracking, zone rules, time limits and sensitivity settings filter out animals, shadows, rain and headlights. Operators confirm or dismiss alerts, and that feedback improves the models.
Drone ready: the same AI can run on drone camera feeds to cover terrain where fixed cameras cannot reach.

Live demo: ibvap-sentinel.vercel.app

How we built it

Edge AI agent (Python)

Continuous video ingest from RTSP streams with OpenCV, with automatic reconnects for unstable field links.
Object detection with a YOLO-family detector, exported to ONNX for fast inference on rugged edge hardware at outposts.
Multi-object tracking with ByteTrack, so every person and vehicle keeps one identity across frames and is not reported again and again.
A geofencing and event engine for restricted zones, fence-line tripwires, loitering and counting. Each tracked object is checked by its foot point (the bottom centre of its box) against the zones drawn by the operator. A line crossing is counted when an object moves from one side of the line to the other between frames, and the direction of the move tells us whether it crossed inward or outward. Loitering fires when an object stays inside a zone longer than the time limit set for that stretch.
An ANPR module with plate detection, OCR, temporal voting across frames and association with the tracked vehicle, for check posts and border roads.

Command and control backend FastAPI with PostgreSQL for units, sectors, cameras, zones, events and personnel. Edge units at outposts talk to the command centre over MQTT, which works well on low bandwidth. They queue events when the link drops and sync once it is back. Clips are served through secure, time-limited links, and every action is recorded in an audit log.

Mobile app for patrol officers Built in Flutter (Android first) with Riverpod and go_router. It covers instant alerts, live view, alert review, zone setup and per-camera detection settings.

Command centre console A web dashboard that shows all sectors and cameras on one screen, with a live alert feed, severity levels and a confirm or dismiss workflow.

Challenges we ran into

False alarms in the field: animals, wind-blown vegetation, shadows, dust, rain and vehicle headlights all trigger raw detection. We had to combine tracking, zone and line rules, time limits and per-camera sensitivity before alerts became something a patrol would trust.
Night and harsh conditions: most intrusions happen at night. Grainy low-light and IR footage, fog, dust and glare all reduce the accuracy of pre-trained models.
Unreliable connectivity: remote outposts have weak or patchy links. This pushed us to an edge-first design that keeps detecting and queuing events with no network at all.
Unstable video streams: field cameras drop, freeze and change resolution. Keeping continuous processing alive meant building reconnect logic and handling broken streams without crashing the pipeline.
Small and distant targets: a person far along the fence may be only a few pixels tall, so we had to balance input resolution, frame rate and model size on limited hardware.
ANPR on the move: plates at check posts are often blurred, tilted or dirty. One frame is rarely enough, so we vote across several frames to settle on the most likely plate.
Data security: border footage is highly sensitive. Keeping analysis on-site and sending only events and short clips shaped our whole architecture.

Accomplishments that we're proud of

A working end-to-end AI pipeline (detection, tracking, intrusion, fence-line crossing, loitering and counting) running on continuous camera streams.
A working ANPR module with temporal voting and vehicle association.
An edge-first design that keeps working when an outpost loses its link to headquarters.
A working mobile app prototype and a live demo of the platform.
Turning hours of passive footage into a short list of meaningful alerts a patrol can act on in seconds.
Building all of this as a five-member team, with tests in place for the pipeline, tracker and ANPR.

What we learned

In security, trust matters more than raw accuracy. A system that sends fewer, meaningful alerts is far more useful than one that cries wolf every night.
Edge-first is essential for border areas. Detection cannot depend on a network link that may not exist when it is needed most.
Real field footage is very different from benchmark datasets. Night, IR, dust and distance demand our own training data.
Operators need context, not just a red box. Location, direction of movement, severity and a short clip are what turn an alert into action.
Clean contracts between the edge, the backend and the apps (MQTT topics and APIs) let a small team build in parallel.
Security of the system itself (encryption, access control, audit logs) has to be designed in from day one.

What's next for Sentinel

Train our own models on border-specific footage for night, IR and thermal feeds, long range targets, animals versus humans, and local vehicles and plates.
Thermal camera support for detection in complete darkness, fog and dense vegetation.
Drone integration: run the same AI on drone camera feeds for wide-area patrols and terrain that fixed cameras cannot cover, with alerts flowing into the same command centre.
Map-based command centre with sector views, camera coverage, patrol locations and event heat maps.
Live view over WebRTC and instant push alerts for patrol officers.
Field trials at border outposts and sensitive perimeters, then scale-up across sectors.
Extending the same platform to critical infrastructure such as airports, power plants, ports and defence installations.
Long term: an intelligent surveillance grid where every camera along India's borders watches, understands and alerts.