Visualization showing how buses become late over the course of their route
I was waiting at a bus stop this one time...........
What it does
[Bmore] Transit Tracker collects open data from the MTA's real time GPS trackers equipped on every bus via the ?Swiftly Api and processes it in Logstash. It then references the published route schedule and headways to determine if the the bus is early, on time, or late and by how much via data manipulation in ruby. Then that data is stored in Elasticseach and visualized in Kibana.
How we built it
MTA data protobuf -> logstash ingest/parser-> ruby calculations -> elasticsearch datastore -> Kibana Web visualization. Implemented via AWS EC2
Challenges we ran into
Obtaining api key to access data from swiftly Free tier of AWS EC2 did not have enough memory for logstash (hackathon could provide instances!) MTA Schedule data was so large that the default java heap of logstash was too small The Google provided GTFS protocol buffer definition files were compiled using an incompatible compiler, we had to recompile (with the correct compiler) the buffer definition files required for logstash to read the protocol buffer files.
Accomplishments that we're proud of
Successfully implementing a big data platform that ingests, parses, stores, analyzes, and visualizes previously inaccessible and unusable data for consumption by the public.
What we learned
How to ingest and process big data, and how to mine that data for interesting information. Different parts of the team had different expertise and we each learned a lot about how to bring all of the parts together.
What's next for [Bmore] Transit Tracker
Continue collection and storage of long data for public evaluation of Baltimore's and Maryland's transit systems. Make recommendations to the MTA, our city leaders, and state leaders for improvements to our transit systems. The [Bmore] Transit Tracker ingest to visualization process pulls data from the universally used General Transit Feed Specification (GTFS), our goal would be to pull data into Transit Track from other transit systems across the nation to develop universal metrics and standards for evaluating transit systems worldwide. Eventually becoming an authority on transit system planing, ratings, and improvements.