CourtVision

Democratizing elite sports tech with an on-device AI referee, commentator, and scout inside a single phone.

Comment

Inspiration

Youth sports have been gated behind a strict financial barrier. The tools required to gain collegiate or professional visibility, such as advanced statistical analytics, tactical video tracking, and high-quality game film, cost thousands of dollars. Talented athletes across public city parks play elite basketball daily, yet their efforts leave zero data trail, leaving them hidden from scouts. Meanwhile a 40% nationwide referee shortage means casual games are overrun by constant, subjective conflict. We built CourtVision AI to bridge this divide: any smartphone becomes an automated recording, officiating, and scouting suite for underserved communities.

What it does

  • Set a phone on a baseline or sideline tripod, launch the app, and play.
  • Automated officiating. The CV engine maps court lines dynamically and tracks the ball. When the ball crosses a boundary, an instant audio referee whistle resolves the dispute objectively.
  • Playground audio commentary. A contextual engine calls the live score, tracks streaks, and delivers playground, broadcast, or hype style color commentary with offline text-to-speech.
  • Zero-cost pro scouting. Post-game, players get automated highlight reels plus pro-level metrics: vertical jump, shot release velocity, top speed, distance covered, court heatmaps, and shot charts, exportable as public verified scouting links.

How we built it

Computer vision backend (Python, FastAPI, OpenCV, MediaPipe): court boundary detection with homography to real court coordinates, orange-ball detection fused with a Kalman filter that predicts through occlusions, and multi-person pose estimation for athletic metrics. Uploaded game video is processed through a scoring state machine (possession, out-of-bounds with hysteresis, shot and score detection with 2pt/3pt arc logic, streaks) that streams events live over WebSocket. Deterministic-first AI: commentary and narrative scouting reports run on a built-in phrase engine, upgradeable to an LLM with a single env var. The product works end to end with zero keys and zero network. Web app (Vite + React): live in-browser tracking demo, film room, analytics dashboards, and shareable scout cards, backed by the same API. Mobile app (React Native + Expo): native C++/Swift/Kotlin frame-processing scaffold for fully on-device inference at 30 fps / 720p. 91-test pytest suite including end-to-end integration tests on synthetically generated game footage with scripted ground truth.

Challenges we ran into

Single-camera depth ambiguity (solved with perspective homography anchored to court boundaries), ball occlusion when players crowd (solved with constant-velocity Kalman prediction and decaying confidence), and keeping everything offline-capable so players with prepaid data plans lose nothing.

What's next

Jersey-color auto-identification to map tracked players to profiles automatically, multi-sport court profiles, and on-device model quantization for the mobile pipeline.

Built With

Share this project:

Updates