Form Check

"When the heat's on, the truth shows up". Form check is a sports web-app built on MediaPipe which analyzes your sports footage and compares it to a reference video, offering areas of improvement.

Comment

https://github.com/coolkidwrik/nwhacks_2026

Inspiration

We were inspired by our friend Mike who couldn’t make it to this hackathon. He wanted an application which helped analyzed his dribbling footage and compare it to Michael Jordan dribbling, from which it would offer a helpful response.

What it does

  1. First the user uploads two videos, one of himself doing a sport and another of someone he wants to compare it to.
  2. Video is processed through MediaPipe which extracts limb information into structured json
  3. This information is normalized using a discrete time warp algorithm to extract best fit angles (which are kept in coordinate-like format)
  4. Information is then sent off to an OpenAI endpoint with specific instructions, from here it returns its response to the frontend.

How we built it

Our frontend is powered by React/TypeScript and built using Vite, while our backend is powered by Python, using MediaPipe for the limb-extraction and FastAPI for the API.

Challenges we ran into

We had issues with integrating the LLM aspect (we had a non-AI solution in the case the OpenAI response returns badly). We also made a performance decision to store the video virtually and accessing it only once, working with strictly structured JSON data afterwards.

Accomplishments that we're proud of

We are just glad that this application works, we weren’t sure if MediaPipe could handle fast-paced movements well and are very glad that our data extraction and analysis pipeline is viable.

What we learned

We learned that MediaPipe is a very useful tool for rapid prototyping, as well as the difficulties involved in integrating an AI endpoint into our application as well as structuring AI responses to be appropriate.

What's next for Form Check

  1. We want to further quantify this information – we take athlete footage from their “best” games, i.e., MVP, highest scores, and assign this a 10/10 score and their worst footage 1/10. From here we want to create a sliding scale that allows further quantification of what would otherwise be qualitative data.
  2. Perspective normalization would also improve precision of our data. While simply applying a rotation matrix wouldn’t do anything (rotation matrix is isomorphism), there are things we can still do with the position of the limb coordinates i.e., ratioing or similar.
  3. Apply the same data extraction/analysis pipeline to determine live athlete performance, offering numerical information to coaches and analysts so they can offer feedback in real time.

Built With

Share this project:

Updates