CivicGuard

AI-powered civic surveillance system leveraging OpenCV, Google Gemini 2.5 Flash, and AWS Graviton3 for autonomous spatial infrastructure monitoring and agentic telemetry analysis.

Comment

Inspiration

Urban infrastructure degradation—such as severe structural pavement cracking, pothole proliferation, and structural decay—presents significant public safety hazards and costly municipal maintenance backlogs. Traditional inspection methods rely on manual human visual surveys, which are slow, subjective, and labor-intensive. CivicGuard was born out of a desire to automate spatial civic surveillance by combining classical, real-time computer vision with advanced multimodal Generative AI and autonomous agent frameworks on low-cost cloud edge infrastructure.

What it does

CivicGuard provides an end-to-end, agentic surveillance pipeline for automated civic infrastructure telemetry:

  • Spatial Pre-Processing Engine: Utilizes OpenCV for real-time visual preprocessing, region-of-interest cropping, and structural gradient analysis.
  • Multimodal AI Reasoning: Leverages Google Gemini 2.5 Flash to evaluate structural severity, issue risk scoring, and synthesize structured diagnostic telemetry.
  • Agentic Decision Architecture: Integrates Model Context Protocol (MCP) tooling to allow autonomous agents to execute continuous monitoring, dynamic alerting, and automated maintenance ticketing.
  • Live Interactive Dashboard: A Streamlit interface hosted on AWS EC2 allowing city engineers to monitor feed telemetries, upload visual diagnostics, and review real-time AI spatial evaluations.

How I built it

  • Computer Vision Pipeline: Python 3.11 with opencv-python-headless for image matrix transformations and feature isolation.
  • Agentic & Generative Layer: Google Gemini 2.5 Flash SDK (google-genai) paired with custom agent tools for contextual severity assessment.
  • Frontend / Dashboard: Streamlit UI customized for real-time visual telemetries and image analysis.
  • Cloud Infrastructure: Deployed on an AWS EC2 Graviton3 ARM64 instance running Amazon Linux 2023, optimized for high-efficiency continuous inference.

Challenges I overcame

  • Headless GUI Dependencies: Resolving missing X11/XCB and OpenGL driver dependencies (mesa-libGL, libxcb) on headless Amazon Linux 2023 cloud environments.
  • Latency Optimization: Streamlining image buffer transformations between local OpenCV matrices and multimodal LLM API payloads to achieve real-time response times.
  • Remote Git Rebase Workflows: Resolving divergent Git trees and merge states to ensure continuous integration between local development and cloud endpoints.

What I learned

  • How to seamlessly interface classical OpenCV image matrices with Google's multimodal API (google-genai).
  • Efficient configuration and deployment strategies on AWS EC2 Graviton architecture.
  • Designing modular, agent-ready architectures that bridge visual detection with action-oriented civic dispatch pipelines.

What's next for CivicGuard

  • Integration with live IoT transit video streams for vehicle-mounted automated scanning.
  • Automated municipal GIS mapping to auto-pin detected infrastructure hazards directly onto city map overlays.

Built With

  • aws-ec2
  • computer-vision
  • cool
  • google-gemini
  • mcp
  • opencv
  • pydantic
  • python
  • streamlit
Share this project:

Updates

Submission history