We loved the StreetCaster's mission to the city of Boston and its residence. Ever since the inception of the 311 application, a huge number of citizens have submitted reports to make Boston a better place to live. However, not all reports are equally useful to the city. Some may be incorrectly categorized, others may include inconclusive pictures, and some can just serve no purpose at all. We wanted to make a tool to help 311 handle these use cases so that StreetCaster is as successful as possible!
What it does
StreetJudge is a tool meant to detect incorrectly submitted reports. The application uses machine vision to evaluate the user-submitted photo and verifies if it matches the issue. For example, if the user submitted a report that includes a photo of a damaged sidewalk, SmartJudge will be able to validate if the sidewalk is indeed damaged or not through machine vision. The goal is to be able to eliminate, as much as possible, the number of incorrectly submitted reports so that the city can focus on the correct ones. With the goal achieved, 311 employees will save an incredible amount of time. Efficiency will upgrade and the cost of the operation will scale down as a result.
How we built it
We used Python with Flask as our backend. Since the hack is meant to be an extension to the 311's reporting system, we didn't put a huge amount of emphasis into the frontend. Hence, we stuck to the very basics of HTML. For the machine vision, we have used Google's AutoML Vision to validate the photos.
Challenges we ran into
It was our first time using Python-Flask for a web project. Even though it relatively simple in contrast to other languages/frameworks, we had spent the first few hours of the hackathon familiarizing ourselves with the setup process. Setting up Google Cloud was a lot more challenging, although we have managed after many walkthroughs.
Accomplishments that we're proud of
We were extremely proud of training a machine learning algorithm. It was our first time, and seeing the result of hundreds of images was extremely satisfying. Seeing how this can immediately improve the 311 system, we are very proud of this accomplishment!
What we learned
We have learned a great deal regarding Python-Flask. However, the greatest lesson was experiencing first-hand what machine learning can do to make our lives easier. Setting up models to train, selecting the pool of images to feed, and reviewing the results of the process what a very important experience. We can definitely see ourselves leveraging this technology in our future projects.
What's next for StreetJudge
We want StreetJudge to be a full-suite of validation tools. This doesn't just include machine vision, but also leveraging data from the 2014 census to visualize trends. Additionally, we can extend this model to validate more urban issues; such as broken street signs, broken park benches, etc. Seeing how StreetCaster has huge potential, we feel the same about StreetJudge's contribution to the mission.