PitchSight

Inspiration

My brother and I both play a lot of sports, and after a game the question is always the same: where should I have been, and what did I do wrong? Pro teams have analysts to answer that. The rest of us get a guess. I wanted to build what we were missing: drop in one clip and get a straight, player-by-player answer.

What it does

• Takes a clip from soccer, basketball, American football, hockey, tennis, volleyball, baseball or cricket and detects the sport.
• Tracks every player, splits the teams and reads each side's shape.
• Rates every player out of 10: what they did right, what they did wrong, nitpicks, where they should have been and the run they should have made.
• Shows it on the video, on a tactics board drawn on that sport's own surface, and as key moments you can jump to.
• Says what it could not see: how often the ball was found and how sure the team split is.

How I built it

Solo, during Hacklanta. YOLOv8 detection with my own tracker, a camera pan-and-zoom model, and a playing-area filter that ignores the crowd, bench and staff. Teams come from kit colour, each sport has its own profile and vocabulary, CLIP recognises the sport, and the app runs on FastAPI and plain JavaScript, with optional Claude vision for the written report.

Challenges I ran into

• A drone's-eye view turns players into a few pixels, so I wrote a motion tracker.
• The "ball" it found was a traffic cone, and a tennis clip counted three players.
• On a sideline football clip, the first version tracked the bench, the mascot and the cheerleaders, so I rebuilt the tracking.
• Player numbers blinked whenever the detector missed a frame, so gaps are now filled and the overlay glides between frames.

Accomplishments that I'm proud of

• It works on messy, real footage, not just clean demo clips, across eight sports.
• It ignores the crowd, and tells you how many people it left out.
• It is honest about its limits.
• The whole flow is real: upload to results in about a minute.
• I built it solo in a weekend, with tests, a README and an open-source repo.

What I learned

Real footage breaks every assumption. When you can't trust a result, measure it and say so. Tests that catch a football word in a tennis report save hours. And AI coding help is fast, but you still have to check it against real clips.

What's next

True pitch coordinates, jersey-number recognition, more footage per sport, and season-long trends

Built solo during Hacklanta with AI coding assistance (Claude). The demo video is real output of the app; its narration is a synthetic voice.

Built With

• anthropic
• canvas
• claude
• clip
• computer-vision
• fastapi
• html5
• javascript
• machine-learning
• numpy
• object-tracking
• opencv
• python
• pytorch
• scikit-learn
• scipy
• sports-analytics
• yolov8