Deep Threes

AI Vision; Combination of 3 AI Models
Landing Page
Film room; Auto-Generated Timestamps, Team Editor, & Knowledge Base Chatbot

Inspiration for our project:

Basketball analytics has transformed how NBA teams scout, strategize, and develop players, but that technology costs millions. High school coaches, college club teams, and recreational leagues are left making decisions based on gut feeling and manual stat-keeping. We're basketball fans who believe every team deserves access to the same kind of data-driven insights, so we set out to build it ourselves.

What our project does:

Deep Threes takes any recorded game footage and automatically runs inference on each frame to extract teams, jersey numbers, shot locations, makes, misses, distances, and more. Powered by Roboflow models and ByteTrack, the entire pipeline is hands-off with no manual tracking or tagging required. Coaches and players can then query their game data through a natural language interface, asking things like "show me all of Kevin's three-point attempts" or "how many mid-range misses in the second half" and instantly get the answers they need.

How we built it:

We started in a Google Colab notebook to prototype and brainstorm the CV pipeline, testing Roboflow models and ByteTrack for player and ball tracking across frames. Our CV pipeline combined 3 AI models that tracked player positions and actions, the colors and teams, and the jersey numbers. Once we had a working approach, we converted the notebook into a model library for sports analytics that our backend can use. On the backend, we used FastAPI to serve the pipeline and MongoDB to store and query game data. The frontend is built with React, Vite, and Tailwind, with a natural language search interface powered by the Gemini API so coaches can ask contextual questions about their game footage and get instant answers.

Challenges we ran into:

Challenges we ran into were consistent video tracking. This was one of our biggest hurdles since players frequently overlap, leave the frame, or get occluded, making it difficult to maintain a reliable identity across frames. Accurately marking shot locations on the court required careful homography calibration, and even small errors in the transformation would skew where a shot appeared on the court diagram. The trickiest math problem was calculating shot distance from the basket correctly for both sides of the court, since the basket coordinates flip depending on which end a team is attacking, and getting that consistently right took more geometric work than expected.

Accomplishments we're proud of:

Getting the full pipeline working end-to-end within a single hackathon was a huge win, given how many moving parts were involved. We're also proud of the natural language query interface, which made the data feel genuinely accessible rather than just a spreadsheet of stats. Nailing accurate shot tracking and distance math for both sides of the court, and getting jersey number detection working automatically meant the whole system stayed true to its goal of requiring zero manual input.

What we learned:

We learned that the gap between "object detection works" and "sports analytics works" is much larger than expected. Sports CV requires as much temporal reasoning and domain knowledge as it does model accuracy, and labeling quality matters enormously when your training set is small. Working as a team under hackathon time pressure also taught us how important it is to prototype fast in a notebook before committing to a full stack.

What's next for Deep Threes:

We want to improve computer vision annotations, as well as improve the pipeline speed. From there, expanding tracking to assists, rebounds, and defensive positioning would make the platform a true all-in-one analytics tool.