PK Prophet

• 

  Our OpenCV pipeline segmenting areas of interest before a Penalty Kick

Inspiration

Being huge soccer fans we realized that there is a lot of technical analysis that is supposed to go into penalty kicks. It's not just as simple as kicking a ball and hoping it enters the goal, rather its the accurate positioning of the ankle, positioning of the feet, and the run up. With PK Prophet, we aimed to conduct a thorough analysis of all of those elements in order to make an accurate prediction regarding where the ball would end up once it leaves the ground.

What it does

PK Prophet is an AI-powered web application used to predict the location of penalty shots based on the kicker’s body language and run up. All the user has to do is upload a video of a penalty kick, and the pre-processor and LSTM model will extract the striker’s pose, and the coordinates of the goal. Using this data, the machine-learning model will estimate the location of the penalty kick without having to see its actual trajectory.

How we built it

The ML model was built using Python through the TensorFlow library. Prophet first takes the input video data and splices it into individual frames, storing the sum of them as a NumPy array. This array is then fed through a pre-trained ConvNet, which extracts key features from the frames before passing it through the LSTM model. This model receives a frame and makes a prediction, but also has the memory of frames that came before it. This allows it to make more accurate real-time predictions on where the ball will end up.

Challenges we ran into

Throughout Ramblin' Hacks, we encountered numerous challenges ranging from data mining to developing the transition between the loading page and the transformed results. For this hackathon, we analyzed over 12000 frames worth of penalty kicks, measuring the width and height of the goal post while accounting for camera warps to ensure we got an accurate positional analysis of the soccer ball for every penalty kick. This was by far the most time consuming part of the hackathon as we were making our own dataset from raw soccer footage. However, with due time and diligence We ended up getting a comprehensive set of data that made accurate predictions regarding penalty kick expected locations. We were provided with a 2D dataset, and while we did incorporate that into our analysis of the soccer ball, we felt the need for something more thorough, which was why we created our own dataset with 3D values, allowing us for depth perception, which makes our data slightly more enhanced allowing for more accurate predictions.

Accomplishments that we're proud of

The first thing we are proud of is successfully building and utilizing an LSTM model for the first time. Additionally, we successfully took on the challenge of collecting and cleaning our own dataset from scratch, ensuring its accuracy and relevance to our project. Finally, we were able to achieve the successful integration of all three parts of our application—data collection, machine learning, and front-end development—working seamlessly together to create a cohesive and functional product.

What we learned

While we all had previous experience in our areas of interest (frontend, ML, backend, etc...), this project pushed us to improve our skills in each domain. We each had to learn new concepts and ideas and implement them successfully.

The tight time window and the quick turnaround forced us to be creative in our problem solving to keep moving forward and make progress. We had to pivot our plan multiple times and make critical changes based on deadlines.

What's next for PK Prophet

We envision PK Prophet being a way for the entertainment industry to boost fan engagement and interest in sports games. Fans can analyze trends, make their own predictions, and even compete in fantasy-style mini-games based on AI-generated outcomes. This not only deepens their understanding of the sport but also creates new opportunities for broadcasters, streaming platforms, and advertisers to connect with audiences. We also believe that this viewing angle presents exciting opportunities for integration with augmented and virtual reality platforms. By capturing the penalty kick from this perspective, PK Prophet lays the groundwork for immersive fan experiences where users can step into the shoes of the goalkeeper or striker

Built With

• css
• flask
• html
• javascript
• mediapipe
• nextjs
• numpy
• open-cv
• pandas
• python
• tensorflow
• typescript