RoadRangers

A real-time, AI powered service that crowdsources infrastructure damage detection and reporting to the community at large, determine severity, and make recovery efforts more optimized and efficient.

Comment

Inspiration

I grew up in a developing country, Costa Rica to be exact. There is a big sense of community, civic duty, and responsibility for the country among the people, however unfortunately lots of aspects of systems in the government remain inefficient for the 21st Century. One problem that is always brought up by my family and friends still living there is the issue of road infrastructure. The lack of a timely response to ensure repairs are made effectively is also a major concern as delays tend to be long. The Ministerio de Obras Publicas y Transporte in charge of handling the maintenance and construction of all things transportation related has had to deal with limited resources and a lack of a modern system to rapidly assess and track the status of road networks across the country. There is also a lack of transparency when issues do get reported, which only contributes to the lack of quality roadways for the people. Thus, I wanted to think of a way to not only leverage the civic pride present in the people of the country, but also the power of technology to centralize planning and processing of data related to damage to roadways caused by storms, accidents, or deterioration due to age.

What it does

RoadReporters is a system that centralizes the management and planning for conducting repairs to road infrastructure throughout the country. Users from around the country use WhatsApp or regular messaging services to report damages to road infrastructure in their localities to MOPT. They do this by sending a image or text message to a specified number showcasing or detailing potholes, ditches, or other abnormalities present in the road network. This information is sent to a custom AI model that detects if there is damage present, what kind of damage is there and calculate the severity of it. This information is then logged, stored, and displayed in a central dashboard system that the agency will access that highlights the damage present in different sections of the country via an interactive local map/heatmap. It displays updates whenever incoming reports are catalogued and summarizes them based on canton, severity of the damage, and amount of reports present in a given region to help controllers plan where to prioritize allocating resources to ensure work is done where citizens need it the most. In summary, the system converts unstructured citizen reports into structured, decision-ready data, enabling infrastructure planners to allocate limited repair crews based on quantified urgency and regional impact.

How we built it

I built RoadReporters using a Vite + React frontend deployed on Vercel and a Python FastAPI backend. The backend uses a custom ResNet50-based convolutional neural network trained, via transfer learning in PyTorch, to detect potholes from incoming images. For text-only reports, I use the OpenAI Responses API with a structured JSON schema to extract damage type, hazard indicators, and severity hints.

Severity is computed using a hybrid prioritization model combining:

  • Vision model confidence score
  • Text-extracted hazard signals
  • Depth and size descriptors
  • Aggregated regional report frequency

Reports are grouped by canton using reverse geocoding and dynamically update a heatmap visualization in the dashboard. Incoming reports are received via Twilio’s WhatsApp sandbox webhook and processed in real-time before being added to the system.

Challenges we ran into

Getting the messaging section to work with both text and image capability was challenging, as well as fine tuning the algorithm to calculate severity of damage based on what the image showed or what descriptors where in the text. Also ran into some hiccups trying to use the OpenAI API to handle requests as I ran out of usage for the time being midway through testing.

Accomplishments that we're proud of

  • Successfully integrated live WhatsApp image ingestion with an AI vision pipeline.
  • Built a hybrid severity scoring system combining vision + language signals.
  • Fully deployed live demo with frontend on vercel and processing server on local laptop.
  • End-to-end WhatsApp -> AI -> Dashboard pipeline working in real time. ## What we learned I learned a lot about how to work with Vite and implement live, local server features and translate those to be used with a Vercel hosted system.

What's next for RoadRangers

The next steps involve expanding the detection capabilities to cover landslides, flooding, and structural road damage. improving the accuracy for prioritization by implementing time-decay severity scoring to improve prioritization accuracy. Building a public-facing transparency portal for citizens is another key step to ensure accountability. The eventual goal will be piloting deployment with municipal governments and monitoring results.

Built With

Share this project:

Updates