GridIron

Predicting Football - One Play at a Time

Comment

Inspiration

All three of us have been huge fans of NFL football for many years now. There have been many times when I was watching an NFL game and wondered - What if we could predict the type of play the offense was about to run? This question was the main inspiration for this project. All of us are familiar with machine learning, which made this project a very logical idea that combined our love for football with our experience in machine learning.

What it does

Takes video input and predicts the offensive formation and identifies positions and location using computer vision.

How we built it

We used RoboFlow to train vision models to identify labels of players in NFL games as well as put them in bounding boxes based on their location and skill position. We then used Python to create formation predictions using machine learning. Finally, we used AWS, Node, and Typescript for the frontend and deployment.

Challenges we ran into

There were many challenges we ran into - as this was a very ambitious project idea. Here are some of them.

1) Planning out all the components of our project. This was especially challenging considering we had two main machine learning components: A vision-based object detection model that allowed for skill position classification and recognition of score and time elements on the screen and a tabular machine learning component to predict whether the play was a run play or a pass play. These model components introduced a lot of confusion and questions such as: How do we relate the vision model to the tabular model, how do we standardize our features for the two models to prevent feature mismatch errors, etc...

Accomplishments that we're proud of

This was a very ambitious project - and there is a lot to be proud of! Here are a few things:

1) Learning an entire new technology in RoboFlow. None of us had even heard of this technology until the hackathon started - and we were able to learn a good chunk of its core capabilities and create very cool vision models

2) Thinking very big and accomplishing a lot of this initial vision. We knew from the start that this project was as close as you could get to - "Go big or go home". If we could not learn a single component of the project or failed to get a single model to work - then our entire project would fall apart as the parts of our project depend heavily on each other.

What we learned

This project was a very valuable teaching tool. Here are the most important lessons we learned -

1) The importance of a fully fleshed out plan. In the days leading up to the hackathon, we laid out what we thought was a good plan, which consisted of the general components of the project, such as the types of models we needed to use and the general tech stack. However, we quickly discovered how much our plan was lacking as we spent all of Friday just asking questions about our plan and generally being confused. By the end of the night, we had a very specific plan and that made Saturday a lot quicker. For future projects, we should have every detail planned out before writing a single line of code.

2) How important it is to designate roles within a team. When we started experimenting with RoboFlow and some of the general tech related to our project late on Friday, there was a lot of overlap in what each team member was doing (2 people were testing RoboFlow). We got little progress done and we decided to go with a different approach on Saturday, each person focused on a completely separate area of the project. One person would focus fully on frontend, one person would focus solely on the vision workflows and models through RoboFlow, and the final person would focus fully on tabular model development and data cleaning.

What's next for GridIron

While we are proud of what we accomplished - this project did not live up to the vision of the original idea. For example, we initially planned to predict the play at a more granular level (run to the left, deep pass, etc...), but doing so would require many more data points and advanced vision capabilities - which we simply did not have time to explore. We also had plans to integrate a recommendation system (not necessarily machine learning) in our UI which in addition to predicting the type of play that would happen, would also show videos of similar plays from other games. This would require more advanced frontend functionality as well as what looks to be a complex recommendation system. Finally, if we had more time, our plan was to integrate more vision-based workflows within RoboFlow that would allow us to detect team names based on their logos and direction of a run or pass.

Share this project:

Updates