Spike Your Skill

• 

  Graph of the differences between a professional player and regular player

• 

  Pose estimation on Gordon during his serve sequence.

• 

  Pose estimation on a professional player during his serve sequence.

Inspiration

We were inspired with the Covid pandemic, when we were all stuck at home. Both of us are volleyball players, and we both started around the same time as Covid. That left both of us at home, without other people to play with. This inspired us to make this project, to help people improve at home, without the need for a coach.

What it does

SpikeYourSkill is an AI-powered biomechanical analysis tool designed to help volleyball players improve their serve technique.

Spike your skill takes a video upload, and runs various image processing algorithms to determine your technique and biomechanical movements.

• Firstly, using a mediapipe pose estimator, the system extracts joint position and angles for key body parts
  • The data is then sorted and compared to a professional player. It accommodates for the potential difference in speed by using Dynamic Time Warping (DTW), to align the users movement to the professional player.
  • We also have a custom image recognition AI, for the location of the ball as it is hit. This is used to detect the contact time and spot of the ball, ensuring that nothing is left out. This was trained on our own, using training data found on roboflow.
  • Using the data, the system will create graphs representing the difference between the user's joint angles and professionals average.
  • Using the Gemini LLM, the data is read and explained in a simple way. It will highlight areas of improvement, and helps users pinpoint specific aspects of their form that deviate from professional technique, guiding their practice and training efforts.

How we built it

This project was built using python (backend) and react (frontend). The image processing uses various libraries, such as opencv, mediapipe, pandas, etc. It is run in a flask server, for easy access. The ball detection AI used training data from roboflow, and was trained in a Kaggle notebook. The webpage was built with React.

Challenges we ran into

We ran through many challenges, from dealing with API issues to training our own model. Many of our issues came from reading the data, such as aligning the data, training the ball detection model, or even getting the data from the flask api onto the website.

Accomplishments that we're proud of

We are very proud of what we were able to make, both the video processing algorithms, and the working data analysis.

• Robust Pose Estimation Integration: Successfully integrated a real-time 3D pose estimation solution (MediaPipe) to accurately extract biomechanical data from standard video inputs. Intelligent Temporal Alignment: Implemented Dynamic Time Warping (DTW) to effectively align user movements with professional reference data, accounting for varying speeds and durations of movement.
• Clear Visual & Quantitative Feedback: Developed a system that generates intuitive comparison plots (PNGs) and provides precise quantitative metrics to highlight deviations in technique.
• Actionable Insights: Created a tool that transforms raw video into meaningful, actionable insights, empowering users to understand and improve their volleyball serve mechanics against an expert standard.
• Custom AI Model Training: Undertook initial efforts in training and fine-tuning a custom AI model relevant to the project's analysis (e.g., for specific error detection or performance classification).
• End-to-End Workflow: Established a functional pipeline for users to upload video files and receive processed analytical results, demonstrating a viable proof-of-concept for a practical application.

What we learned

We have learned a lot about the intricacies of biomechanical analysis from video, specifically the challenges and nuances involved in accurately capturing 3D human pose from a 2D source and effectively comparing time-series data. This includes understanding the inherent limitations of monocular 3D estimation (like depth ambiguity and occlusion), the critical role of robust data alignment techniques like DTW, and the distinct considerations for presenting analytical data versus creating smooth visual video synchronization for user feedback. We also gained valuable insights into structuring data input and output for a practical application. With the creation of our AI model, we have learned how to gather training data, and how to deploy an AI model.

What's next for Spike Your Skill

A lot of things can be improved with our app itself. The algorithms used to read the video can be made faster and more precise. We would also be interested to add in other actions used in volleyball, such as spiking, passing and setting.

Built With

• cv2
• excel
• fastapi
• figma
• json
• kaggle
• matplotlib
• mediapipe
• next.js
• opencv
• python
• react
• react-native
• roboflow
• tailwind