SportsSync

Inspiration

As big sports fans, we always want more ways to engage with our favorite athletes. Eli and Ford both played baseball and thought of how interesting it would be to be able to compare our swings to our favorite MLB players.

What it does

SportsSync allows sports fans to compare their motions to the motions of the pros and find which star they're in sync with.

How we built it

We used a research library from Facebook called VideoPose3D, which leverages a fully convolutional model for 3D human pose estimation in videos. This approach, rather than using recurrent neural networks (RNNs), employs dilated temporal convolutions over sequences of 2D key points extracted from videos to infer the 3D pose of an individual. This method captures long-term dependencies in motion while maintaining efficiency, as dilated convolutions allow the model to process large temporal contexts without significantly increasing computational complexity.

Our web interface was built using NextJS, React, and Typescript. Our backend was built in Python using fastapi.

Challenges we ran into

Running inference was a challenge due to the lack of GPUs. Fortunately, we were able to use some Vast.ai credits to run inference successfully.

Accomplishments that we're proud of

We started late and yet successfully got a prototype working that matches users' motions to MLB players.

What we learned

We learned how to run inference on Vast.ai, how to implement VideoPose3D, and how to use fastapi.

What's next for SportsSync

Can be expanded to other sports (NBA, Golf, etc.). Add AI feedback to improve user form based on the Generative AI model trained on pro users.

Built With

• nextjs
• python
• react
• typescript