Optimizing GT Busing with Reinforcement Learning

GT Bus!!
Heatmap of student distributions
GT Region boundaries

Inspiration

We were frustrated with the exceedingly long bus wait times, constant overcrowding, and poor reliability of the GT bus system. Due to the outspread nature of GT's campus, the bus system is paramount for quick transportation around and beyond campus, which adds to the necessity of a robust design.

Despite its pitfalls, we saw a clear opportunity to seek possibilities where we could improve the bus system's functionality. We focused on what Georgia Tech could do to optimize its transportation infrastructure instead of using a consumer app since 1) the impact of our project on GT's administration and students is larger than other approaches to the problem and 2) we provide a unique insight into a stagnant problem with band-aid solutions.

What it does

This project generates ideal bus allocations for the four most common (and traveled) bus routes to maximize efficiency. Such an approach is flexible on a day-to-day basis (which, with proper scaling, can adapt in real-time), easily implementable, and provides the best use case of the data available within this project's scope.

For example, our idea suggests that GT should have more green and gold buses running one day but shift resources to serve more traveled routes on other days. This helps to maximize bus efficiency while allocating resources effectively.

How we built it

To model bus traffic, we used OSCAR class data to fetch Fall 2022 classes, their locations, and how many students were in each class through a custom API. This provided the baseline of how students moved throughout campus hourly, indicating how buses would serve populations.

We transformed this data into a real-time heatmap of campus. Using that visualization, we divided the campus into five distinct regions where transportation within areas would be largely walking, but between regions would require bus use. This allowed us to simplify the problem to ensure feasibility while still conserving the complexity that this problem posed.

To generate ideal bus allocations, it was decided to utilize a Reinforcement Learning algorithm to train a network. This offered us two advantages - 1) RL algorithms allow the model to generate better bus-route distributions than human-labeled training data. Additionally, if the model was trained on live data, the necessity of a model approaches 0 if it mimics current human logic and doesn’t offer unique insight into the problem and 2) the ability to leverage more powerful training of Reinforcement Learning architectures to develop a strong solution.

Creating a custom Reinforcement Learning algorithm posed many challenges, especially since the reward function is quite difficult to model. As opposed to cart-pole or atari, where reward scores are easily calculated, we had to generate our formula to calculate this (a complex combination of matrix multiplication, weighted averages, and parabolas), which required rigorous testing and endless trial-and-error. Once that was accomplished, the model was constructed, trained, and repeatedly evaluated.

Challenges we ran into

An open-source data set. Despite not finding one readily, we simply modeled the data by writing an API that fetched the OSCAR course data. Cleaning and processing data to remove unnecessary information and smooth model training Developing a custom Reinforcement Learning model to mimic how GT Bus would serve the student population Standing countless hours in the lunch line Debugging code that was written to debug other code

Accomplishments that we're proud of

Develop a scalable solution that will positively impact students at Georgia Tech
Creativity in gathering data from unconventional sources
Utilizing python libraries to clean up and process data
Developing, building, and testing a custom Reinforcement Learning algorithm for transportation optimization
Surviving countless hours in the lunch line
Debugging code that was written to debug other code

What we learned

How to design and implement an end-to-end matching learning system to address real-world problems
Write custom Reinforcement Learning algorithms based on patterns
Time management and problem-solving skills on a strict schedule
How to stand countless hours in the lunch line
How to debug code that was written to debug other code

What's next for Optimizing GT Busing with Reinforcement Learning

Evaluate the viability of implementing the system at Georgia Tech
Build out infrastructure to gather bus usage data & integrate it into model training
Developing custom bus routes that serve locations around campus better
Training, testing, and evaluating the model in a real-world setting