Sidewalk Analyzer

Inspiration

We are citizens who utilize the sidewalks in St. Louis and wanted an app to know the quality of our routes. We also wanted to help communities and municipalities track and know where to allocate resources for improvements.

What it does

Using Pegasus to analyze video footage for sidewalk quality, we generate a detailed explanation of where issues are found. It displays these issues on a map with lines corresponding to Overture Maps GERS IDs for sidewalks.

How we built it

We used Python, Flask, Twelvelabs Pegasus, Overture Maps, and Mapillary to get our app working. We use Mapillary to get our images, Overture Maps to get our GERS IDs, Pegasus to analyze the video footage, and Python and Flask to code and deploy our app.

Challenges we ran into

We ran into challenges utilizing Twelvelabs Marengo, as it didn't seem to be the best option for our project. We decided to just use Pegasus instead. We also ran into a challenge in getting data. Mapillary is not really that good for getting images of sidewalk quality. Most of the images of sidewalks are of good quality, so it is hard to know if it can accurately detect errors.

Accomplishments that we're proud of

We are proud of being able to effectively utilize Pegasus to generate a list of problems with the sidewalks. We are also proud of linking it to Overture Maps GERS IDs so that our generated details are meaningful.

What we learned

We learned how to use images to extract useful information, how to analyze video footage using a smart model like Pegasus, and how to utilize AWS.

What's next for Sidewalk Analyzer

We would love to find better data to cover poor sidewalks. It would enable us to be able to tune the prompt and get better results. We would also consider using Marengo if we had a lot of data and we wanted it to find either any sidewalks or poor sidewalks. The future of this app includes a full database where a user could search along their route and see bad sidewalks. We could also set up a list of the worst offenses for municipalities to view where they should spend resources fixing.