About SportsSnap

Inspiration

As avid sports fans and tech enthusiasts, we've often found ourselves watching game highlights or replays on YouTube, wishing we could quickly access more information about the players, teams, and specific moments we were seeing. We realized there was a gap between video content and real-time sports data that we could bridge, and thus, SportsSnap was born.

What We Learned

Developing SportsSnap has been an incredible learning journey. We've deepened our understanding of:

Browser extension development and its intricacies
Video processing techniques for screenshot capture and analysis
Integration with sports APIs (particularly ESPN's database)
Computer vision and image recognition for identifying players and game situations
Overlay technologies for seamlessly displaying information on video content
Leveraging AI and machine learning models for natural language processing and multimodal search

How We Built It

SportsSnap was built using a combination of cutting-edge technologies:

Browser Extension: Developed using JavaScript, HTML, and CSS to create the user interface and handle interactions with the video player.
Backend Server: Built with Flask, a lightweight Python web framework, to process requests from the extension, communicate with external APIs, and perform image analysis.
AI Integration:
- Utilized Groq, a high-performance AI inference engine, for rapid processing of complex queries.
- Integrated ChatGPT for computer vision.
Multimodal Search: Implemented Twelve Labs' technology for advanced video understanding and search capabilities, allowing us to extract and analyze visual and audio information from sports content.
API Integration: We integrated with ESPN's API to fetch game data, player stats, and highlight information.
Data Overlay: Developed a custom overlay system using HTML5 Canvas and WebGL to display information on top of the video without affecting playback.

Challenges We Faced

Video Identification: Accurately matching YouTube videos to specific games in ESPN's database proved challenging due to variations in video titles and content. Twelve Labs' multimodal search capabilities helped significantly in improving accuracy.
Real-time Processing: Ensuring that screenshot analysis and data retrieval happened quickly enough to provide a seamless user experience was a significant hurdle. The use of Groq's high-performance AI inference engine was crucial in overcoming this challenge.
Computer Vision: Implementing ChatGPT to recognize player jerseys based on the data we provided
Multimodal Data Integration: Combining visual, audio, and textual data from videos with external sports statistics and real-time information required complex data processing and synchronization.
User Interface Design: Creating an intuitive and non-intrusive UI that enhances rather than detracts from the viewing experience required multiple iterations and user feedback sessions.

Despite these challenges, we're proud of what we've accomplished with SportsSnap. By leveraging advanced technologies like Flask, Groq, ChatGPT, and Twelve Labs' multimodal search, we've created a powerful tool that enhances the sports viewing experience for fans around the world. Our project demonstrates the potential of combining AI, machine learning, and traditional web technologies to create innovative solutions in the sports entertainment space.