• 

  Dashboard VARLens AI

• 

  Input and Explainable AI

VARLens AI

Inspiration

Sports are built on fairness, precision, and trust, yet even with modern Video Assistant Referee (VAR) systems, controversial decisions continue to spark debates among players, coaches, officials, and fans. While AI and computer vision can assist referees in making more accurate calls, these systems often provide conclusions without explaining the reasoning behind them. We wanted to bridge this gap by creating an AI-powered officiating assistant that not only analyzes game footage but also explains every decision with transparent evidence.

VARLens AI was inspired by the vision of making sports officiating more understandable, accountable, and trustworthy through Explainable Artificial Intelligence (XAI).

What it does

VARLens AI is an Explainable AI-powered sports officiating platform that analyzes match footage and assists referees with critical decisions.

The platform can:

• Detect players and the ball in real time using computer vision.
• Analyze offside situations and key match events.
• Track player movements throughout the game.
• Highlight visual evidence supporting each decision.
• Explain AI reasoning using SHAP-powered feature importance.
• Generate natural-language explanations with GPT-5.5.
• Display confidence scores and decision timelines.
• Provide interactive replay analysis for referees and spectators.

Instead of simply displaying a verdict, VARLens AI explains why the decision was made and presents the supporting evidence.

How we built it

We developed the frontend using Next.js, React, Tailwind CSS, shadcn/ui, and Recharts to create a modern and responsive user interface optimized for match analysis.

The backend was built with FastAPI and Python, handling computer vision pipelines, AI inference, and explainability services.

We implemented YOLOv11 for real-time player and ball detection, ByteTrack for multi-object tracking, and MediaPipe for pose estimation. The prediction pipeline is powered by PyTorch, while SHAP provides explainable AI by highlighting the most influential factors behind each decision.

To make AI outputs understandable for everyone, GPT-5.5 converts technical model results into concise, human-readable explanations. Authentication and data management are handled by Supabase, while the application is deployed using Vercel and Render.

Challenges we ran into

One of our biggest challenges was ensuring that AI decisions remained transparent rather than becoming another black-box system. Balancing real-time performance with explainability required careful optimization, especially when combining computer vision, player tracking, and natural-language explanations.

Another challenge was designing an interface that clearly visualizes evidence, timelines, confidence scores, and AI reasoning without overwhelming referees or users during live match analysis.

Accomplishments that we're proud of

• Built an AI-powered sports officiating assistant with explainable decision-making.
• Integrated real-time player detection and tracking.
• Combined computer vision with Explainable AI.
• Added GPT-powered natural-language explanations.
• Created interactive evidence visualization and replay analysis.
• Developed confidence scoring and transparent decision summaries.

What we learned

Through VARLens AI, we learned that accuracy alone is not enough for AI systems used in high-stakes sports decisions. Transparency, interpretability, and user trust are equally important. We also gained valuable experience integrating computer vision, multi-object tracking, explainable AI, and large language models into a unified real-time application.

What's next for VARLens AI

We plan to expand VARLens AI by supporting additional sports such as basketball, tennis, rugby, and hockey, improving real-time event detection, integrating advanced pose estimation for foul analysis, enabling multi-camera synchronization, and providing post-match analytics for teams, leagues, and broadcasters. Our long-term vision is to make AI-assisted officiating more transparent, consistent, and trusted across every level of competitive sports.

Built With

• bytetrack
• fastapi
• framer-motion
• gpt-5.5
• mediapipe
• next.js
• opencv
• python
• pytorch
• react
• recharts
• render.
• shadcn/ui
• shap
• supabase-authentication
• supabase-postgresql
• tailwind-css
• typescript
• vercel
• yolov11