• 

  open cv pipeline

• 

  open cv5 utilization

• 

  beach restriction

• 

  multifeed dashboard

Neonet Vision Agent

What it is

Neonet Vision Agent is an agentic coastal-safety system that turns ordinary beach CCTV into a continuous lifeguard. An autonomous agent watches two live camera feeds, reasons over what it sees using OpenCV 5 vision tools, fuses the two feeds into a single risk picture, and escalates confirmed hazards to a human operator for approval before any action is taken.

The agentic loop (perceive → decide → act → human)

Every tick, on each feed, the agent calls OpenCV 5 tools, and the visual result of one tool decides the next tool call and the next action:

1. Perceive — cv2.createBackgroundSubtractorMOG2 + cv2.findContours segment moving objects in the scene.
2. Geofence — cv2.pointPolygonTest checks whether a blob's foot point is inside the restricted water zone.
3. Inspect — a second look, triggered only when step 1–2 flag a blob: cv2.Canny on a cropped ROI confirms a real swimmer vs. surf glint.
4. Decide + act — above the confidence gate, the agent raises an incident and requests operator approval.
5. Human-in-the-loop — the operator approves (dispatch a lifeguard) or dismisses (false alarm). The decision closes the loop.

Only confirmed detections produce actions; surf glint (small area / wrong aspect ratio) is rejected; a human is always in control before a dispatch.

Cross-feed reasoning (two feeds, fused)

• Feed A — Beach cam: geofenced water zone, swimmer-in-zone detection.
• Feed B — Offshore sea-state cam: surf energy from cv2.absdiff frame-differencing + whitewater cv2.threshold → sea state + rip-risk score.

The offshore camera never sees a swimmer and the beach camera never measures the surf. The agent fuses them into a single risk posture: a swimmer entering the restricted zone is treated as advisory under a calm sea, but escalates to CRITICAL when the sea-state feed reports rough water with high rip risk. This is genuine multi-feed reasoning — one camera's visual state changes how the agent acts on another camera's events.

Where OpenCV 5 is used

All substantive vision runs on OpenCV 5.0.0, surfaced live in the dashboard's OpenCV pipeline view:

cv2.VideoCapture · cv2.cvtColor · cv2.GaussianBlur · cv2.createBackgroundSubtractorMOG2 · cv2.threshold · cv2.morphologyEx · cv2.findContours · cv2.contourArea · cv2.boundingRect · cv2.pointPolygonTest · cv2.Canny · cv2.absdiff · cv2.imencode

Each stage is timed per frame and every call is counted, so you can watch OpenCV 5 doing the work in real time (tens of thousands of calls within the first minute).

AWS component

The service is a single CPU container, designed for AWS:

• Image → Amazon ECR.
• Compute → the container on EC2 (t3.large) or ECS/Fargate, operator console behind an Application Load Balancer.
• Alerts → the alert stream (rip-risk warnings, confirmed incidents, dispatch confirmations) is SNS-ready for operator/lifeguard notification.
• Audit → incidents and the agent's reasoning trace are emitted as JSON (/state) for S3 / CloudWatch.

Operator console

A single web app with six views: Overview, Beach feed, Sea-state feed, OpenCV pipeline, Cross-feed reasoning, and Incidents — live annotated video, the agent's reasoning trace, the OpenCV telemetry, an alert bar, and the human approve/dismiss controls.

Stack

Python · FastAPI · OpenCV 5 · NumPy · vanilla HTML/CSS/JS (no build step). Runs with pip install -r requirements.txt && uvicorn app:app, or docker build. See testing_instructions.txt.

Built With

• fastapi
• gstreamer
• opencv
• python