GroundWork AI

Inspiration

We were inspired by videos of Zohran Mamdani fixing potholes in New York City. They made us think about how cities discover road damage and decide which neighborhoods receive attention. We wanted to make infrastructure maintenance more proactive and equitable by helping teams identify problems beyond the places that generate the most complaints.

GroundWork AI brings that idea to Seattle. We wanted to identify both existing potholes and cracks that may worsen over time, helping cities address damage earlier.

What it does

GroundWorksAI uses satellite imagery to scan for road discrepancies like cracks and potholes. It stores these discrepancies in a database and evaluates the danger they pose to commuters and pedestrians. Discrepancies tagged as high severity will be prioritized for review helping maintenance teams identify urgent repairs and allocate resources where they are needed most.

How we built it

We built GroundWork AI with React, Vite, and custom CSS, using React Leaflet for the interactive map. Our Python backend uses FastAPI to connect imagery retrieval, model inference, and persistent issue storage.

We integrated Mapillary’s API and coverage vector tiles to discover street-level images within the map’s current bounds. We fine-tuned a pretrained YOLO11 model in Google Colab using the United States subset of RDD2022, converting its XML annotations into YOLO-format bounding boxes. The model recognizes potholes, longitudinal cracks, transverse cracks, and alligator cracks.

We used SQLite to store issues and review decisions, and added logic to consolidate overlapping detections without losing human feedback. Our frontend runs on Vercel, with a Cloudflare Tunnel connecting it to our local backend for the demo.

Challenges we ran into

One major challenge we faced was the limited size of our initial training dataset, which resulted in a model with low accuracy. To address this, we switched to a different model and trained it on a significantly larger dataset. Although this required more time and computational resources, it ultimately improved our model's performance.

Accomplishments that we're proud of

We’re proud of building an end-to-end prototype that connects real street imagery to detection, human review, and work-order tracking.

Our role-based views make the handoffs tangible: a reviewer checks a finding, a coordinator assigns work, and a crew member records progress. Each stage retains the evidence and context behind the original issue.

We also built automatic map-area scanning, persistent review decisions, location adjustments, duplicate handling, and report exports. Most importantly, we made uncertainty visible instead of presenting every model prediction as a confirmed defect.

What we learned

We learned that building a useful computer vision application requires more than training a model. Annotation quality, class balance, camera perspective, geographic accuracy, and testing on unfamiliar imagery all affect whether a prediction is actionable.

We also learned to distinguish three separate concepts: model confidence, human verification, and repair priority. A confident prediction can still be wrong, and a verified defect still requires context before a maintenance decision.

Designing the reviewer, coordinator, and crew views helped us think beyond detection. We learned how important clear handoffs, persistent state, and shared data formats are when building tools for multiple users.

What's next for GroundWork AI

Our next priority is evaluating the model on a larger, independently labeled set of Mapillary images. We want to improve pothole coverage, incorporate representative negative examples, and better handle panoramic imagery.

We also plan to move inference to a persistent hosted backend, introduce authenticated user accounts, and strengthen the role-based workflow with assignment notifications, repair-completion evidence, and an activity history.

We want to improve matching of the same defect across different photographs and develop transparent repair-priority criteria with input from maintenance teams.

Longer term, we hope to expand into sidewalk defects and accessibility barriers, supported by suitable training data and evaluation. Our goal is to help cities discover overlooked damage, document evidence, and coordinate better-informed maintenance decisions.