SportSense

• 

  User flow

• 

  Technologies and architecture we implemented.

• 

  Demo of what video annotations look like(includes voice agent coaching).

Inspiration

We wanted to make high-quality sports coaching accessible to everyone, not just athletes who can afford private trainers. Using AI and agentic voice assistance, we saw an opportunity to deliver affordable, personalized coaching through a simple mobile app.

What it does

Players can record themselves practicing — for example, shooting a basketball — and receive live audio feedback on form and technique, along with visualized progress tracking to guide long-term improvement.

How we built it

Mobile Frontend: React Native with Expo for quick camera integration and smooth recording on any device.

Backend: FastAPI to handle video uploads and communication with the AI services.

AI & Orchestration: LangChain agents orchestrated by Agentuity, connecting tools like Gemini for reasoning, OpenCV and MediaPipe for motion detection, and ElevenLabs for real-time audio feedback.

Pipeline: End-to-end flow from recording to analysis to instant tips to stored session history.

Challenges we ran into

It was a challenge to develop the agent orchestration framework since there were so many different technologies involved in robust video processing. We also had struggles integrating the mobile app to the backend agentic frameworks. We initially wanted to have complete real-time feedback while recording, but the Gemini LLM doesn't have capabilities for simultaneous video processing and reasoning.

Accomplishments that we're proud of

-We successfully built and deployed a LangChain-powered agent orchestrated by Agentuity. -We delivered a working MVP that gives immediate feedback on recorded sports movements. -We integrated multiple AI tools seamlessly — including Gemini, OpenCV, and ElevenLabs — into a single experience.

What we learned

-How to build an agentic system with multiple tools orchestrated by another service. -The importance of moving fast and iterating on features. It's also important for an MVP to focus on the features absolutely critical to the functionality of the app, and then later if time permits, adding extra features. For our app, user experience is critical(we don't want the user to wait 5 minutes to get feedback on their shot), and so figuring out clever ways to process the data while keeping the UX nice was crucial.

What's next for SportSense

As real-time video processing technology improves for LLMs, SportsSense will be able to provide genuine real-time actionable feedback rather than a slight delay. We also want to flush out the functionality for storing previous sessions and offering a visual way(graphs, charts, etc.) for the user to see their progress and what they improved on over sessions. Right now, we only target basketball, but we'd love to expand to other sports as shown in our app like baseball, volleyball, fitness, etc. SportsSense targets a huge market for a very relevant painpoint, and we're just getting started.

Built With

• agentuity
• elevenlabs
• expo.io
• fastapi
• gemini
• langchain
• mediapipe
• opencv
• python
• react-native