@robbi5 provided the scraped data of the MVG bikes, so we decided to visualize and analyze them. We first wanted to get an overview where the MVG bikes are used and then make some more in depth analysis.
What it does
We built a framework that makes the provided data easily accessible and provides a nice web front end with different visualizations and views:
- Show availability of bikes over time with a heat map
- Show all movements of the bikes on single days
- Enter a bike ID and show the route the bike has taken in the last months
- Calculate circles of how probable it is to find a bike at a given place and time
How we built it
The back end is implemented with the Python framework Django, using PostgreSQL, PostGIS, Celery and Redis. The available XML data files can be imported into the database, so that is can be used for different types of analysis easily. The back end provides the processed data over a REST API as JSON.
All in all the built software can be extended easily: one has to program some analysis of the data in the back end and add an visualization view to the front end. The front end is built modular so that a new view can be added easily.
Challenges we ran into
The main challenge we ran into was the amount of data: We had over 13 million in the data sets, so the SQL queries took some time. First we added caching where it is possible.
For this weekend we reduced the data to one month so the performance increased.
Accomplishments that we're proud of
We are proud that we can provide some insights in the use of the MVG bikes and build a reusable framework that can be used furthermore.
What we learned
What's next for BikeWatch
We hope to add more data again, but this requires some optimization in the database. We assume that more optimizations are possible in PostGIS so that the queries for coordinates take less time.