Info
Github: https://github.com/HarshilGandhi148/PotholePatrol
Table: 11
Inspiration
We're students at Purdue in West Lafayette, and the roads around campus are full of potholes. Everyone complains about them, but almost nobody reports them, so they get worse until someone finally does. We noticed that city vehicles like buses, garbage trucks and maintenance trucks already drive almost every street in Lafayette, every week. If those vehicles could spot potholes on their own, nobody would need to remember to report anything.
What it does
Pothole Patrol turns the cameras on city vehicles into pothole finders. A computer vision pipeline runs on the device to spot potholes, tag them with GPS, and file a report to whoever is responsible for fixing that road. No video ever leaves the vehicle, and the city owns all of the data.
Anyone can also use it directly: take a photo or upload a video, and Pothole Patrol finds the potholes in it.
How we built it
Frontend: An iPhone app built in Xcode on top of the Ultralytics YOLO iOS app. It runs detection live through the camera.
- Swift: the app's programming language
- SwiftUI: the app's screens and navigation
- UIKit: the live camera view and the parts of the Ultralytics app built on it
- AVFoundation: captures the camera feed that the model runs on
- MapKit: shows detected potholes on a map
ML: A YOLO11 object detection model. We converted it to Core ML so it runs entirely on the iPhone, without an internet connection.
Backend:
| Tool | What it does | Why we chose it | At scale |
|---|---|---|---|
| Python | Backend language | Best tools for map data, and fast to build in | Keep it |
| FastAPI | Runs the API routes | Validates input and generates API docs for us | Keep it |
| Uvicorn | Server that runs FastAPI | The standard choice | Run several copies |
| SQLite | Database | One file, no setup, enough for a hackathon | Move to Postgres with PostGIS |
| Shapely | Map geometry | Finds the nearest road and checks city limits | PostGIS could do this |
| pyproj | Converts GPS to meters | "Within 25 m" needs real distances, not degrees | Keep it |
| Pillow | Image processing | Strips hidden photo metadata and resizes images | Keep it |
| pytest | Automated tests | 105 tests run in seconds | Keep it, and add CI |
| ngrok | Public HTTPS link to our laptop | iPhones require HTTPS, and it took minutes to set up | Replace with real hosting |
| Disk storage | Saves photos | Simple | Move to cloud storage |
Challenges we ran into
- Our first model wasn't good enough. On real potholes we photographed, it often labeled them as alligator cracking instead. We had to find, test and compare several models on our own photos before picking one that worked.
- Getting the model onto an iPhone. Converting it to Core ML, setting up code signing with a free Apple account, and getting the app to recognize a custom model each took real debugging.
- Connecting the phone to the backend. iPhones block plain HTTP, so we needed an HTTPS tunnel before the app could send reports to our laptop.
How we used AI and when to correct it
AI was a big help, but we learned where to trust it and where not to:
- Good at debugging. It was fastest at figuring out Xcode signing errors, environment problems, and why the app couldn't find our model.
- Not good at architecture. We had to plan the pipeline ourselves: what runs on the phone, what runs on the server, and how reports get deduplicated and routed. Once we had the plan, AI helped build the pieces.
- Not good at UI. Its interface suggestions felt generic, so we designed the screens ourselves.
What we learned
- Picking a model means testing it on your data. Published benchmark scores didn't tell us how a model would do on Lafayette roads.
- Running AI on the device instead of the cloud is both faster and better for privacy.
- AI tools speed up building, but people still need to own the design and the architecture.
What's next for Pothole Patrol
- LiDAR for depth, so we can measure how deep a pothole is instead of estimating severity
- Pothole validation, so crews or residents can confirm a detection before a repair is scheduled
- Accelerometer data to catch potholes the vehicle actually hits
- A map filter to show the potholes you recorded yourself
- Importing public pothole data, such as existing resident reports, to combine with our detections
- Blurring license plates and faces in any saved image
- Expanding to more towns beyond Lafayette and West Lafayette
