Mat(a)Mu — Agentic Public Watch

OpenCV 5 + AWS turn citizen road photos into privacy-safe, confidence-gated reports and follow-up actions.

Comment

What is Mat(a)Mu?

Mat(a)Mu is an agentic public-watch platform for safer roads. It turns citizen road observations into structured, privacy-safe, location-aware evidence that communities and road operators can understand and follow up.

Road damage is often noticed by citizens before it enters an official inspection workflow. Reports are fragmented, photos lack consistent context, and operators receive more observations than they can review manually. Mat(a)Mu creates an evidence layer between lived experience and institutional response.

How it works

A citizen captures a road image and live location in the PWA. The system processes the evidence through OpenCV 5, assigns a trust score, applies privacy blur, enriches the observation with AI-assisted interpretation and spatial context, and routes it according to confidence:

  • High-confidence observations can be published automatically.
  • Ambiguous or low-quality observations are routed to an operator IQE review queue.

Published observations appear on a public map with severity, evidence, status, recurrence, and follow-up context. Operators use the dashboard to review, acknowledge, and resolve observations.

The core workflow is:

Capture → Analyze → Structure → Map → Follow

OpenCV 5 and AI implementation

OpenCV 5 is a substantive part of the vision pipeline:

  • OpenCV 5 DNN loads and runs the YOLOv11n ONNX road-condition detector.
  • The detector identifies potholes, longitudinal cracks, alligator cracks, rutting, flooding, and surface degradation.
  • The Inspection Quality Engine combines detection confidence, image sharpness, and brightness into a trust score.
  • YuNet/SFace detects faces and applies Gaussian blur before evidence is published.
  • PostGIS spatial queries identify nearby observations and recurring issues.

The intelligence layer uses:

  • Amazon Nova Lite for concise condition summaries.
  • Amazon Nova Pro for multimodal severity reasoning.
  • TypeSafe AI for structured evaluation of severity, urgency, photo quality, and recurrence.
  • A local fallback mode so the core workflow remains testable when the live model proxy is unavailable.

Agentic Vision

Mat(a)Mu is not only a detector that displays a prediction. The visual result changes what the system does next.

  1. Perception collects detections, confidence, image quality, privacy, structured AI evaluation, and nearby-report context.
  2. Decision gates route the observation to automatic publication, monitored publication, or human review.
  3. Action publishes the observation, queues a high-severity alert, creates a recurring-issue follow-up, or sends the item to the IQE review queue.
  4. A structured JSON trace records the perception signals, decision reason, and resulting action.

This creates a perception → decision → action loop with human oversight for uncertain cases.

Key features

  • Citizen-facing React PWA for capture, GPS, report history, and public map browsing.
  • GPS watermarking to preserve visible location evidence when mobile operating systems strip EXIF metadata.
  • OpenCV 5 + YOLOv11n road-condition detection.
  • YuNet/SFace face privacy blur before publication.
  • Inspection Quality Engine with confidence-gated routing.
  • TypeSafe structured evaluation and AI-generated summaries.
  • Agentic trace viewer for perception → decision → action evidence.
  • Django operator dashboard with map, reports, IQE queue, alerts, and follow-up actions.
  • PostGIS nearby and recurrence queries.
  • AWS deployment using EC2, S3, Amazon Bedrock, and IAM controls.
  • Rate limiting, role-based operator access, audit traces, and human review for edge cases.

Architecture

Citizen PWA → Django / Django Ninja API → OpenCV 5 DNN + YOLOv11n → IQE trust score + YuNet/SFace privacy blur → PostGIS spatial context + TypeSafe/Amazon Nova reasoning → Agentic confidence gates → Public map, operator dashboard, and JSON trace

AWS provides the application and CV runtime on EC2, blurred evidence storage on S3, managed model reasoning through Amazon Bedrock, and IAM access controls.

Evaluation

The current local evidence package reports:

Evidence Result
Integration checks 46/46 pass
Agentic scenarios 4/4 pass
Sidoarjo evaluation photos 6
Average trust score 0.805
Correctly blurred detected faces 2/2
False blurs in evaluation set 0
Mock-mode capture latency approximately 2 seconds

The evaluation includes normal captures, invalid and missing GPS, duplicate and nearby-report logic, privacy processing, confidence gates, and model fallback behavior.

Responsible use and limitations

Mat(a)Mu treats AI output as evidence assistance, not an automatic engineering verdict.

  • Faces are blurred before evidence is published.
  • Low-trust observations require human review.
  • Decision reasons and confidence values are recorded in trace JSON.
  • Operators can accept, reject, acknowledge, and resolve observations.
  • The current evaluation set is small and focused on Sidoarjo.
  • YOLOv11n can miss small, occluded, or edge-of-frame defects.
  • Shadows and paint markings can produce false-positive pothole detections.
  • Low-light images are intentionally routed to review.
  • Local tests use fallback behavior when LiteLLM/Bedrock is unavailable; production claims should be verified on the public deployment.

Links

The final screenshots and demonstration video will be added before submission.

Built With

Share this project:

Updates