RockyRoad

Inspiration 💡

We wanted to make something that the Mapillary team could potentially use. We knew that Meta's team heavily used machine learning to semantically segment various parts of the image. However, we noticed that there didn't seem to be a detector for potholes!

What it does 💻

Given an image, detect if there's a pothole. This will allow future contributors to label where the potholes are in the image so that the algorithm can be improved in the future!

How we built it 🔨

Javascript React for the frontend. Hugging Face for the backend, fastai, PyTorch, skimage for the data science

Challenges we ran into 🧐

Initially, we were going to try making a classifier for bike racks and ev chargers. There weren't any datasets with these two classes on any of the widely known datasets (cityscapes, KITTI, BDD, COCO, etc). However, after talking with the Mapillary team, it turns out that it already does that! So we had to pivot to potholes pretty quickly. Due to a lack of time, we didn't get to change the methodology on how data was gathered and processed. I'm sure there are much better datasets for potholes than what we ended up using.

Accomplishments that we're proud of 🎉

Deploying a fully functional machine learning model, backend, and frontend within 4 or 5 days is pretty awesome! Training models can be done with a lot less data now with pretrained models.

What we learned 📚

How to use data scraped from the internet to train a model and then transform the data so that it's applicable to Mapillary

What's next for RockyRoad 🔮

One of the things that we ran out of time to implement was the splitting of images into segments. By doing this, it could have more closely resembled the training data. If given even more time, we could have made our own labeled dataset featuring many potholes. This way, our results would be even more accurate as we could have done pixel-wise classification.