Civic Lens

One photo. Ten seconds. Your city, fixed.

Comment

CivicLens

Inspiration

It started with a pothole.

One morning, while walking to class, a teammate twisted their ankle stepping into a road crater that had existed for weeks. They opened the city reporting website, only to face a long form, multiple department dropdowns, mandatory descriptions, and a process that felt slow and impersonal. They closed the tab. They did not report the issue. Most people would not.

That friction is the real infrastructure problem. The United States has an estimated 55 million potholes, contributing to roughly $26.5 billion in vehicle damage annually. Cities spend only about $68 billion of the $185 billion needed each year to maintain road infrastructure. The gap is not just funding. It is data. Cities cannot fix what they do not know about, and complex reporting systems prevent citizens from contributing valuable signals.

We built CivicLens to remove that friction and transform everyday citizens into real time infrastructure sensors.

How We Built It

CivicLens is a full stack AI civic reporting platform designed as a single intelligent pipeline.

Mobile Reporting

A React Native app allows citizens to report issues by taking one photo. No forms. No department selection. No writing required.

AI Processing Pipeline

Why a Fine-Tuned Classifier + OpenAI, Not Just OpenAI?

A FastAPI backend triggers a two model architecture:

We tested sending images directly to OpenAI for classification. It works well for 2–3 obvious classes, but as the number of classes grows, LLMs hallucinate, they invent plausible-sounding but incorrect labels, mix up visually similar classes like cracked pavement and faded lane markings, and return inconsistent output formats that break downstream routing logic.

Our solution: Split the responsibility

EfficientNetV2-S Classifier

A fine tuned computer vision model performs structured classification, returning a label and confidence score within milliseconds. Handles classification only, outputting a single structured label and confidence score in milliseconds at zero marginal cost. It reliably distinguishes cracked pavement from potholes from debris.

OpenAI Vision Generator

Never has to guess the issue type. It receives the confirmed label from EfficientNet and focuses purely on what it's good at, generating a human-readable title, description, likely cause, estimated fix time, and recommended department. When the classifier confidence passes a threshold, OpenAI generates a professional complaint including:

  • Title
  • Description
  • Probable cause
  • Estimated fix time
  • Recommended department

We implemented a production cost gate using a probability threshold:

$$ \text{Call OpenAI only if } P(\text{civic issue} \mid \text{image}) \ge 0.85 $$

Images below (0.60) confidence are rejected as clean road, while images between (0.60) and (0.85) are flagged for manual review.

City Dashboard

An Angular admin dashboard provides:

  • Live KPI cards
  • Severity analytics
  • Interactive Leaflet maps
  • Department workload tracking
  • One click status updates

Dataset and Model Training

We built a balanced dataset by merging multiple Roboflow Universe sources across six classes:

  • Potholes
  • Cracked pavement
  • Road debris
  • Broken road signs
  • Faded lane markings
  • Normal road (negative class)

Each class was balanced to 1,200 images and split 80/10/10 across train, validation, and test sets. Corrupt files and duplicates were removed, and undersized images were filtered out.

Training ran on a T4 GPU using PyTorch with pretrained EfficientNetV2-S weights. Instead of relying entirely on large language models, we separated responsibilities between deterministic vision classification and language generation.

Challenges We Faced

Environment Compatibility

Python 3.13 broke critical dependencies including OpenCV and NumPy. We rebuilt the stack using Python 3.11, pinned packages, and downgraded PyTorch to 2.5.1 after model loading behavior changed.

React Native Architecture Changes

Expo Go forced the Fabric renderer on iOS, causing type errors and breaking older components. We rebuilt several UI screens rather than patch unstable code.

Real World Data Imbalance

Some categories had thousands of images while others had fewer than one hundred. Instead of overfitting, we focused inference on the strongest performing classes and documented dataset limitations transparently.

The “Why Not Just Use OpenAI?” Question

Direct OpenAI classification produced inconsistent labels and unpredictable formatting. EfficientNet provided deterministic structure, while OpenAI added high quality language generation. The pipeline approach improved reliability, latency, and cost control.

What We Learned

The most powerful AI systems are pipelines, not single models.

Separating structure from language generation allowed us to balance performance, scalability, and economic feasibility. Designing with production constraints from the start changed how we approached architecture decisions such as confidence thresholds, deduplication logic, and API gating.

We also learned that civic technology succeeds when it minimizes friction. Reducing reporting to a single photo fundamentally changes participation behavior and creates richer infrastructure data for cities.

What’s Next

We plan to expand CivicLens into a proactive infrastructure intelligence system:

  • Cluster escalation when ≥ 3 reports appear within 100 meters
  • Time based priority escalation after 48 hours of inactivity
  • Push notifications to close the feedback loop with citizens
  • Recurring location detection for chronic infrastructure failures
  • Direct 311 system integrations for automated work order creation

Final Statement

CivicLens represents our belief that artificial intelligence should not just analyze the world, but actively improve how people interact with it. By transforming a complex civic reporting process into a single photo, we aimed to reduce friction, increase participation, and give cities clearer, faster insight into the infrastructure problems that impact everyday life.

This project pushed us to think beyond building a model and instead design a complete system where performance, cost, usability, and real world constraints all mattered equally. We learned that impactful AI comes from thoughtful architecture, transparent limitations, and a deep focus on the people who will actually use the technology.

Our goal is not only to demonstrate what modern AI pipelines can do, but to show how they can rebuild trust between communities and the systems that serve them. CivicLens is a step toward cities that listen better, respond faster, and evolve through the collective input of the citizens who walk their streets every day.

Built With

Share this project:

Updates