Swing Sense

• 

  swing_analysis

• 

  swing_metrics

• 

  upload_video

Inspiration

Most golfers are stuck in "swing limbo"—they know something is wrong, but they can't afford a $20,000 Trackman or a $150/hour coach. We wanted to democratise coaching. Inspired by the precision of professional biomechanics, we set out to build a "Pro in your Pocket" that uses nothing but a smartphone camera and the reasoning power of Gemini 3 to provide world-class feedback.

What it does

Swing Sense is an Gemini-powered web app that transforms phone camera swing videos into a comprehensive 3D bio-mechanical analysis.

Body Dynamics: Extracts 33+ pose landmarks using Google's MediaPipe Pose model to create a 3D skeleton.

Agentic Analysis: Uses a multi-agent system (via Google ADK) to analyse specific swing phases: Address, Top of Swing, and Impact.

Visual Feedback: Provides interactive 3D trajectories, rotation dials, and velocity graphs.

Expert Synthesis: Gemini 3 acts as the "Head Pro," synthesising raw data into a conversational, actionable coaching plan.

How we built it

The web app was built with SvelteKit and Bun. We utilised Google’s ADK to orchestrate parallel Gemini 3 sub-agents. One agent focuses on biomechanics (angles/rotations), while another focuses on kinematics (velocity/timing).

Google's MediaPipe Pose handles the client-side landmark extraction, which is then processed through our custom math engine to calculate "X-Factor" and wrist velocity. Three.js renders the 3D trajectory, while TailwindCSS styles the web app.

SQLite and Better Auth manage user sessions and historical swing data.

Challenges we ran into

Mapping 2D pixels to a 3D coordinate system for something as fast as a golf swing is notoriously difficult. Detecting key poses, such as top of swing and impact proved extremely difficult, but we felt that giving Gemini as much information as possible was essential for accurate coaching advice.

Accomplishments that we're proud of

We successfully built a fully agentic workflow where Gemini isn't just a chatbot, but a reasoning engine. Seeing a sub-agent correctly identify a "1° deviation" at address and having the "Head Pro" agent translate that into a specific drill was a major breakthrough. We’re also proud of the math implementation behind the key metrics calculations, such as shoulder and hip rotations.

What we learned

We learned that Gemini 3 is an extremely powerful model which is capable of being a top level golf coach, with no fine-tuning and very little additional help needed. By providing Gemini with both the visual frames of the swing and the raw numeric telemetry, the quality of advice improved drastically compared to text-only prompts. We also loved building the agentic app using Google's ADK framework.

What's next for Swing Sense

We want to take Swing Sense from the range to the course.

Voice assist: An audio feedback after each swing, allowing users to focus on the game. Mobile app: While a web app is great for demo purposes and is easy to distribute, the power of the golf coach would be unlocked on mobile.

Built With

• betterauth
• gemini
• google-adk
• mediapipe
• sveltekit