RipTrack

Inspiration

Derek is a swimmer and diver, and his team in high school didn't always have a diving coach. Often he would have to look online for tips, only coming up with old, low quality videos that were not of much help. Inspired by this, we wanted to create a service that divers could use when they do not have access to resources such as a coach.

What it does

Our project allows users to upload recorded dives, receive feedback on their performance, and track progress. Additionally, it can give live feedback if a user brings a device to the diving board with them!

How we built it

We built the frontend with Next.js and React, using HTML5 Canvas for real-time body tracking. The Python backend uses a custom computer vision pipeline to visualize joints/motion. We also used Elevenlabs to generate speech from text-based feedback.

Challenges we ran into

Our biggest issue was probably handling noisy data from motion blur and water splashes. We had to write custom algorithms to keep joint angles accurate during fast rotations. Syncing telemetry, video playback, and skeletal UI without lag for live coaching was also hard.

Accomplishments that we're proud of

We were able to combine complex computer vision with a very accessible web interface. We are very proud of our custom scoring algorithm that accurately maps different dive phases. Also getting the live text-to-speech feedback loop makes this an actually helpful tool for a real diver, like Derek (he approves).

What we learned

We learned a lot about computer vision, especially how to apply complex kinematics to 2D video. We also learned how to manage time-series data with TimescaleDB. Additionally, we engineered precise LLM prompts to turn raw telemetry math into actionable, human-sounding coaching.

What's next for RipTrack

We want to implement something further for the workout aspect of our project. Specifically, we want to be able to have users start a workout that was recommended to them, and then the computer vision tracks how well they are performing the motions. It can then give live feedback, such as tips for more muscle activation or other things.

Built With

• fastapi
• gemini
• next.js
• python
• tigerdb
• typescript
• yolo