PUPIL

PUPIL (Predictive Urban Pothole Identification Logger) is a cohesive system that enables public works crews to automate the detection, classification, and ticketing of potholes in a community.

Comment

Inspiration

Across Rolla, and many other cities across the US, potholes seem to rule the streets. They appear so easily and yet are left untouched for months while people drive over them. Potholes can cause dangers to drivers and vehicles if not properly treated, so our goal was to make sure that cities are aware of their pothole situations.

What it does

PUPIL gives cities the ability to monitor road conditions at all times and track them online. It includes a camera that sits on the back of public service vehicles, always recording the road behind it, connected to a small computerized device that detects when a pothole is seen. When a pothole is detected by the computer vision algorithm, it is cross-checked against others in our database, verifying that the pothole has not already been detected and has not expanded since the last track. All new or expanded potholes are saved in the database. There is also a public webpage available that shows a map of potholes in several cities across the US based on our database. It also allows city employees to change the status of the pothole and reflect when repairs have been made. It is an integrated, layered system of devices and services that helps cities keep their roads in good condition.

How we built it

We utilized Python for the backend of the web server and camera sensor setup. We leveraged Flask to set up APIs to set up the pothole information database and set up the sensor side to continuously record frames, detect potential potholes using a localized CV model, and use GPS and road detection to verify the exact location of the pothole with respect to the city roadmap.

We heavily used React (JavaScript version) for the front end of the website, which allowed us to include the Google Maps API for mapping potholes across cities and the Auth0 service for authenticating city officials in different jurisdictions. It was very helpful for ensuring that all the right information was displayed for the relevant jurisdiction and specific pothole.

MongoDB Atlas helped us maintain data for all of the detected potholes online and allowed us to quickly update the data for everyone that needed it. It maintains a link an online image that represents the relevant pothole, as well as the size, location, and more relevant pieces.

Auth0 requires online authorization to use editing capabilities. Members are grouped by organization, allowing them to only edit the status of a pothole if it is within their organization's jurisdiction, protecting our pothole data from outside interference.

We used a camera as a sensor to detect potholes and ran an ML model locally via Roboflow using Inference to leverage already existent data and figure out which potential potholes were real matches.

Challenges we ran into

We did not have a camera with GPS capabilities at the time of this project, which seemingly prevented us from completing this project, however we were able to add a GPS connection from another device attached to the local computer managing the sensor system. This allowed us to simultaneously fetch the frame data and the global location when the frame was taken, maintaining pothole locations with a reasonable degree of accuracy.

Accomplishments that we're proud of

We are proud that we were able to join various pieces of hardware and software together to build a cohesive system, and make it flexible enough to be comfortably used by any city whom elects to partner with us on this project to keep roads in good condition. We implemented secure features to ensure that no false information infested our database, and we hope that any city that partners with us will appreciate knowing the condition of their roads and which specific issues need to be prioritized.

What we learned

We learned how to leverage tools that were new to us, including CV models, cloud databases, map embedding, and bridging the communication gap between software and hardware elements. We are proud to say that, while using various different sources, coding languages, and pieces of hardware, we have greatly enhanced our ability to unify them into a cohesive system.

What's next for PUPIL

PUPIL is the future of road condition management. In the future, we hope to partner with cities across the US to find more potholes and ensure that they get fixed within a reasonable time frame. We look to enhance the security of the system and guarantee that, as it scales, attacks from malicious actors are prevented. We also want to ensure that organizations have the ability to manage their own users.

Share this project:

Updates