Inspiration

Most amateur table tennis players never get a coach watching them. I film my matches and save pro clips from YouTube, but the footage just sits on my phone: three minutes of match is maybe forty seconds of actual rallies, and a pro's forehand is over in a third of a second, too fast to learn from. Coaching channels on YouTube fix this with slow motion, freeze frames and a few short cues, but they only cover the clips they chose. I wanted that same style of teaching for any clip you care about, including your own.

What it does

TTSkill takes a video you uploaded or a YouTube link you pasted (up to 3 minutes) and turns it into coaching:

Technique Breakdown. Give it a clip of a player you admire, or describe their training in text. Optionally choose the stroke (forehand topspin, drive, loop vs backspin, backhand topspin, flick, push, chop or serve), or leave it blank to detect it automatically. You get a vertical 9:16 coaching video: a MediaPipe pose skeleton on the player, three two-word cues each with a slow-motion lead-in and freeze frame, a bat-hand trail, a recap and a practice drill. Claude vision writes the cues when an API key is set; offline per-stroke templates are used otherwise. Match Auto-Cut. Upload your own match. TTSkill listens for the sound of ball contact, groups hits into rallies, and gives you either highlights (top rallies ranked by number of shots) or the full match with the dead time removed. Pose is used to filter out the pre-serve "bounce the ball on the floor" routine, which otherwise looks like a rally. Find my pro match. Film your forehand or backhand and optionally enter your height (otherwise it's estimated from the table, which is 76 cm high and 274 cm long). It compares your height and body proportions with pros who use the same hand and return the 3 closest, with a score and notes on where your swing differs (DTW on joint-angle curves). TT Coach. Type one request, such as "here's my match, show my best rallies and fix my forehand". A LangChain deepagents agent plans the work, shows its todo list updating live, hands steps to subagents, runs the tools above and returns the videos plus a short "what to work on". Gear. A rubber finder that ranks rubbers by player ratings within your budget (PingSense data), and a page with the blade and rubbers 20 top pros use. The whole app is mobile-first, because players use it courtside right after filming.

How I built it

Frontend: Next.js 15 (App Router), React 19, TypeScript and Tailwind CSS v4. Mobile-first and dark. Uploads go through XHR so progress shows on the submit button, and job pages poll for live status. Backend: FastAPI with background jobs, which write JSON job records atomically so the page always knows the status, progress and errors. yt-dlp downloads pasted links at 720p or lower and deletes them after the job. Rally detection: a 2 to 6 kHz band-pass, onset spikes over a rolling median, then hits grouped into rallies by gap and minimum hits. ffmpeg cuts each rally with 1.5 s padding, re-encodes it to 720p/30fps and joins the parts with the concat demuxer. Technique pipeline: pick the window (the longest rally by sound, or the most motion), then MediaPipe pose, stroke-type detection, key moments (the swing direction depends on the stroke), the analyzer (Claude vision with structured output, or offline templates) and a StrokeBreakdown schema that the renderer turns into a 1080x1920 video with OpenCV and ffmpeg. Coaching style: I measured the pacing, freeze frames, slow motion and text layout of a coaching channel's recent videos and wrote it into my own style profile (swift_style.json). The renderer, the templates and the Claude prompt all read that profile. No reference footage or branding is used in the app. Pro matching: limb ratios and stance width from MediaPipe world landmarks, a weighted distance on height and proportions for the top 3, and dynamic time warping on joint-angle curves to show where your form differs. Agent: LangChain deepagents with Claude. It plans with write_todos, delegates to the subagents clip_analyst, player_matcher and coach_writer, keeps its notes in the deepagents virtual file system (/plan.md, /pose_summary.json, /matches.json and so on), and calls six tools that wrap my pipelines. Without a key, or if the agent fails, the same tools run on a fixed plan. Built with Claude Code, connected to ChatOverflow throughout the weekend.

Challenges I ran into

Telling the ball from everything else. Shoe squeaks, table bounces and a player bouncing the ball on the floor before serving all make sharp sounds. Squeaks show up as long tonal lines while ball contacts are short spikes, so I filter by shape. The floor-bounce routine needed pose: if nobody is swinging, it isn't a rally. Tracking the right arm. My first breakdown followed the wrong arm of a right-handed player. I added bat-hand detection and a manual left/right option. YouTube's bot check. Downloading pro clips at scale got rate-limited. I chose to wait and retry rather than work around it, so only part of the pro database is built so far. Making it feel like coaching rather than a data dump. It took several passes to reduce the analysis to three two-word cues, each with its own colour, plus a freeze frame and one drill. Time. Building four features, an agent and a polished mobile UI solo in 36 hours meant strict ordering: Auto-Cut first because it is the safest demo, then the breakdown, then the agent on top.

Accomplishments that I'm proud of

A real match (3:35) cut automatically into a highlight reel of its best rallies, with no manual trimming. A pro's forehand turned into a vertical coaching video with a pose overlay and freeze-frame cues. An agent that actually plans: you can watch its todo list tick off as it hands work to subagents. Everything works without an API key (templates and a fixed plan), so the demo never depends on the network. Tests cover every FastAPI endpoint and every Next.js API route.

What I learned

Audio is a surprisingly strong signal for sports video: in table tennis the ball tells you when the action happens. Pose estimation is good enough on a phone video to measure joint angles and body proportions, but choosing the right person and the right arm matters more than model accuracy. Agents work best as orchestrators of solid deterministic tools, with plain-function fallbacks. How to design a phone-first flow for people standing next to a table.

What's next for TTSkill

Build the full pro database: 20 players in the roadmap, then 100, with forehand and backhand clips for each. Side-by-side comparison: your swing next to your matched pro's, synced at the key moments. Progress tracking across sessions ("your elbow angle at contact improved by 12 degrees"). A native iOS/Android app with on-device pose, and hosted uploads past the 4.5 MB Vercel limit. Doubles and multi-camera support, and coach accounts for clubs.

Honest limits (worth saying in the description)

Only Ma Long and some other players that I manually uploaded is fully built in the pro-matching database tonight; the rest of the roster was blocked by YouTube's rate limits. The matching code handles any number of players. Claude coaching and the TT Coach agent need an ANTHROPIC_API_KEY. Without one, offline templates and a fixed plan run instead. Height from the table is an estimate (about ±10 cm); entering your height gives better matches. Large uploads hit Vercel's 4.5 MB body limit until the backend is hosted; pasting a YouTube link avoids it.

Built With

Share this project:

Updates

Submission history