Soccer Image Multi-Label Captioning

For our project, we want to classify soccer photos into predetermined categories

Comment 3

Initial Post

(This has actually been edited to reflect the new project we pivoted to, also soccer related)

Soccer Image Multi-Label Captioning

Introduction- In celebration of the World Cup and our enjoyment of watching soccer; for our project we want to apply what we have been learning in CSCI 1470 towards soccer image captioning. This can be used for a variety of things, sorting images, FIFA in-game commentary and video assisted refereeing. We will be referring to the articles cited below. We intend to use a CNN architecture and experiment with a Variational Auto Encoder.

Data - https://sites.google.com/view/image-and-video-analysis/home?authuser=0

References - https://ietresearch.onlinelibrary.wiley.com/doi/full/10.1049/ipr2.12543 https://www.analyticsvidhya.com/blog/2019/04/build-first-multi-label-image-classification-model-python/

Metrics - We will be using the test data we separate from training data to measure accuracy.

Ethics - Why is Deep Learning a good approach to this problem?

  • Using deep learning could improve video assistant refereeing. This could have a large impact on the game as it would help prevent missed calls. It is a good approach because non deep learning approaches are unable to deal with the number of features and complexity of a soccer match.

How are you planning to quantify or measure error or success? What implications does your quantification have?

  • We are hoping to have good accuracy. This assumes that the test data is quality.

Final Post

(anyone using a brown university email can view these links)

Our Github - https://github.com/henrypdonahue/soccer-image-captioning

Our Poster - https://docs.google.com/presentation/d/1TzVQ225VZz4zeHpjqYDAsppmrGqruqARci8n4TmOE7w/edit#slide=id.p

Our Final Reflection - https://docs.google.com/document/d/16Src0xh3okh2yLIww_DjE9__z2lJ2lUq1aEWNv3CRqY/edit

Our video - (Still working on this and will have it done by the midnight deadline detail on the assignment, this devpost is due before then for some reason so it may not be included)

Built With

Share this project:

Updates

posted an update —

Our progress since our last mentor meeting has been mainly in the knowledge building category, not in the coding category. After our first mentor meeting, we were told that predicting soccer game outcomes using a feed forward neural network was not within the project guidelines, so we pivoted to this. As such, we have been researching the key differences between image categorization with transformers/RNNs and video categorization with transformers/RNNs.

Log in or sign up for Devpost to join the conversation.

posted an update — 

Our progress since our last mentor meeting has been mainly in the knowledge building category, not in the coding category. After our first mentor meeting, we were told that predicting soccer game outcomes using a feed forward neural network was not within the project guidelines, so we pivoted to this. As such, we have been researching the key differences between image categorization with transformers/RNNs and video categorization with transformers/RNNs.

Log in or sign up for Devpost to join the conversation.