AI Basketball Ref

• 

  Clip Analysis Path

• 

  Architecture Diagram

Inspiration

Watching NBA games made the pain point obvious: late-game out-of-bounds and foul reviews can swing the result, but long stoppages often kill the live excitement and momentum of the moment.

## What it does

AI Basketball Ref reviews multi-angle clips and delivers a crew-chief-style verdict with confidence, rule references, and evidence. It supports built-in example clips, game/player metadata context, NBA rules grounding, live WebSocket updates, and interruptible voice delivery.

## How we built it

We built a three-layer system:

• Frontend: Next.js + React + TypeScript for clip selection, synchronized playback, verdict display, and live interaction.
• Backend: FastAPI + Python for session lifecycle, clip handling, metadata/ rules loading, orchestration, and realtime streaming.
• Agents: Google ADK multi-agent workflow powered by Gemini models (session orchestrator, contact/ball/timing/angle/boundary specialists, and a Crew Chief final decision agent).

## Challenges we ran into

The biggest challenge was deployment on Google Cloud. Gemini support inside Cloud Shell was very helpful for debugging and iteration, but we still hit a lot of trial-and-error issues, including a subtle platform constraint where endpoints ending with z are reserved. We also had to harden agent output parsing (strict JSON), fix repeat-session 500s, reduce hallucinations with metadata guardrails, and make interruption behavior reliable during live voice playback.

## Accomplishments that we're proud of

We shipped an end-to-end, working referee-review experience that combines video, metadata, and official rules into a consistent verdict flow. We also integrated a realistic “crew chief” interaction model with turn states, live streaming events, and user interruptions.

## What we learned

Grounding matters more than raw model power. Strong schemas, explicit constraints, and post-generation validation are essential to keep verdicts faithful to game context. We also learned that realtime UX quality depends on orchestration details: state transitions, pacing, cancellation paths, and resilient fallbacks.

## What's next for AI Basketball Ref

Next steps are improving player identification from video, adding richer event timelines per clip, expanding scenario coverage beyond the seeded examples, tightening evaluation against official NBA review outcomes, and polishing production deployment/observability on Google Cloud.

Built With

• adk
• cloud-build
• cloud-run
• fastapi
• fastapi-(backend)-google-adk-multi-agent-framework-(agent-orchestration)-gemini-models-+-gemini-live-api-(analysis-+-realtime-voice)-websockets-(realtime-turn-state
• gemini
• google
• next.js
• next.js-(frontend)-python
• python
• react
• secret-manager
• sqlite
• typescript
• verdict
• websockets