UI - Light Mode
UI - Dark Mode
Architecture Diagram

GameVision - AI-Powered Gameplay Analysis Platform

Inspiration

The project was inspired by the need to democratize sports video analysis for coaches, analysts, and fans who lack access to expensive professional tools. Our goal was to build an intelligent, scalable platform using Amazon Bedrock that could automatically analyze gameplay footage and enable natural language conversations about the content. We wanted to create a tool that transforms hours of manual video review into instant, AI-powered insights accessible to anyone.

What it does

GameVision is a web-based platform that automatically analyzes sports gameplay videos using AI. Users upload videos through an intuitive interface, and the system uses Amazon Bedrock Data Automation with a custom sports blueprint to extract key moments like goals, penalties, player actions, and crowd reactions. Users can then engage in natural language conversations with an AI agent powered by Amazon Bedrock to ask questions about specific plays, player performance, or game statistics, receiving instant answers with precise timestamps and visual references.

How we built it

The project is a full-stack web application with a React frontend and FastAPI backend, designed for cloud-native deployment on AWS.

Frontend: We built a responsive single-page application using React 18, TypeScript, and Material-UI. The interface features drag-and-drop video upload, real-time video playback with synchronized highlights, interactive analysis displays showing game statistics and key moments, and a conversational chat interface for querying video content.

Backend: We developed a FastAPI application that orchestrates the entire analysis pipeline. It handles presigned S3 uploads for direct video storage, triggers Bedrock Data Automation jobs with custom sports blueprints, manages analysis results in S3, and serves as a proxy for Bedrock Agent conversations with session management.

AI & Analysis: We created a custom Bedrock Data Automation blueprint specifically designed for sports analysis that extracts player actions, game events, spectator reactions, locker room scenes, and team bus footage with precise SMPTE timecodes. The blueprint processes videos into structured JSON with chapters, timestamps, and confidence scores for each detected event.

Agent Architecture: We implemented Amazon Bedrock Agent with three Lambda function action groups that handle video processing, analysis coordination, and intelligent query responses. The agent maintains conversation context and uses session attributes to track which video is being discussed, enabling multi-turn conversations about specific gameplay moments.

Cloud & Deployment: The application is fully serverless and cloud-native, deployed on AWS Elastic Beanstalk for the backend API and AWS Amplify for the frontend. We use Amazon S3 for video and analysis storage, AWS Lambda for serverless compute, and integrated CloudWatch for comprehensive logging and monitoring.

Challenges we ran into

Bedrock Data Automation Integration: Navigating the preview service's API and understanding the correct service names and method calls required extensive experimentation. The lack of comprehensive documentation for the bedrock-data-automation service meant we had to reverse-engineer the correct API patterns.

Custom Blueprint Creation: Designing a sports analysis blueprint that could accurately detect diverse gameplay elements was complex. We had to iterate multiple times on the blueprint prompt to achieve consistent extraction of player names, action types, and precise timestamps.

Agent Event Format Handling: Understanding and correctly parsing the Bedrock Agent's event format, particularly the properties parameter structure as a list rather than a dictionary, required careful debugging. We had to simplify our Lambda functions to return raw JSON and let the LLM intelligently parse the data rather than attempting manual extraction.

Async Video Processing: Implementing a robust polling mechanism for long-running Bedrock Data Automation jobs while maintaining a responsive user experience was challenging. We had to balance between synchronous processing for immediate feedback and asynchronous patterns for production scalability.

Optimizing for Context Window Limits: While Amazon Bedrock Data Automation with Nova models is capable of comprehensive frame-by-frame analysis and full audio transcription, we strategically focused our custom blueprint on extracting key moments like goals, penalties, and highlights. This approach was necessary because processing complete frame-by-frame data with full transcription would exceed the Nova model's context window limits for longer videos. In future iterations, we plan to optimize the model configuration and enhance our custom blueprint to extract more granular information while managing context limits through techniques like chunking, summarization, or selective frame sampling.

Accomplishments that we're proud of

End-to-End AI Pipeline: We successfully built a complete pipeline from video upload through AI analysis to conversational Q&A, all powered by Amazon Bedrock services. The seamless integration of Data Automation and Agent services demonstrates the power of AWS's AI stack.

Custom Sports Blueprint: Creating a production-ready blueprint that accurately extracts complex sports events with precise timestamps and high confidence scores was a major achievement. The blueprint handles multiple scene types including gameplay, locker rooms, and team buses with consistent accuracy.

Intelligent Conversational Interface: We built a natural language interface where users can ask complex questions like "When did Sam Reinhart score?" and receive accurate answers with exact timestamps. The agent maintains context across multiple questions, enabling fluid conversations about the video content.

Production-Ready Deployment: We successfully packaged the entire application for cloud deployment with Elastic Beanstalk configuration, environment variable management, and proper CORS handling. The deployment package is ready for immediate production use with comprehensive documentation.

What we learned

This project was an intensive exploration of building production-grade AI applications on AWS. Key learnings include:

Bedrock Data Automation: We gained deep expertise in creating custom blueprints, understanding the video analysis pipeline, and working with structured output formats. We learned how to optimize prompts for consistent extraction of sports-specific events and handle the nuances of SMPTE timecode formatting.

Bedrock Agent Architecture: We learned how to design effective agent instructions, implement Lambda function action groups with proper event handling, and manage session state for contextual conversations. Understanding the agent's event format and response structure was crucial for building reliable integrations.

Serverless Orchestration: We mastered coordinating multiple AWS services including S3, Lambda, Bedrock, and API Gateway to create a cohesive application. We learned best practices for async processing, error handling, and maintaining data consistency across distributed services.

Production Deployment: We gained practical experience with Elastic Beanstalk deployment, including Procfile configuration, environment variable management through .ebextensions, and proper WSGI server setup with Gunicorn for FastAPI applications.

What's next for GameVision

Real-Time Analysis: Implement streaming video analysis to provide live insights during ongoing games, enabling coaches to make real-time tactical decisions based on AI-detected patterns and player performance metrics.

Advanced Analytics: Integrate machine learning models for predictive analytics, player performance trends, and tactical pattern recognition. Add features like heat maps, player tracking, and formation analysis to provide deeper strategic insights.

Team Collaboration: Build collaborative features allowing coaches and analysts to share annotated videos, create highlight reels, and collaborate on game strategy. Add role-based access control and team workspace management for professional organizations.

Mobile Application: Develop native iOS and Android applications for on-the-go video upload and analysis, enabling coaches to analyze game footage immediately after matches. Include offline support for reviewing previously analyzed videos without internet connectivity.