BetaBreak - Your AI Climbing Coach

BetaBreak Home Screen
BetaBreak Video Upload
Video analysis screen
Analysis result screen
Before and after comparison
Architectural Flow
Social Media Campaign for BetaBreak

Inspiration

I started climbing at 20.

Most national team climbers start at six or seven. I was already 14 years behind, and I could feel it every time I stepped into the gym. I'd watch kids half my age flash problems I'd been projecting for weeks, and I knew I needed help—real, technical coaching to fast-track my progression.

But here's the thing: professional climbing coaches charge $200/hour. As someone desperately trying to go pro despite starting late, that simply wasn't sustainable. I needed feedback after every session, not once a week when I could afford it.

So I started thinking: what if I could get expert-level feedback instantly, for every attempt, at a fraction of the cost?

The idea hit me while reviewing my climbing videos one night. I was watching myself fail the same V6 for the tenth time, and I could see something was wrong with my hip positioning—but I didn't know exactly what to fix. That's when it clicked: what if I could feed my climbing videos into a computer vision model, track all my body angles and positions, and then use Gemini 3 to analyze my form and tell me exactly what I was doing wrong?

Not generic advice. Not "try harder." But specific, actionable feedback like:

"Move 3: Your hips are 45cm from the wall during the reach. Rotate your left hip in before reaching—this will shift your center of mass closer and make the hold easier to grab."

The kind of feedback a $200/hour coach would give you.

What it does

BetaBreak is your AI climbing coach that analyzes your form in real-time and gives you expert-level feedback instantly.

Here's how it works:

Film your attempt - Use your phone to record yourself climbing at the gym
Upload instantly - Process the video while you're still at the wall (< 60 seconds)
Get specific feedback - Receive move-by-move analysis with actionable adjustments
Try again - Apply the feedback immediately and iterate until you send

The app leverages computer vision to track your body mechanics and Gemini 3 to translate raw data into coaching that actually makes sense.

Gemini 3 Integration Overview

BetaBreak leverages Gemini 3's multimodal capabilities as the core intelligence layer that transforms raw climbing biomechanics into expert-level coaching feedback.

How Gemini 3 Powers BetaBreak

1. Biomechanical Analysis Engine

MediaPipe extracts 33 body keypoints per frame from climbing videos, calculating metrics like hip-to-wall distance, body angles, and center of mass positioning. This structured data is fed to Gemini 3, which interprets these measurements in climbing-specific context. Unlike traditional CV models that only detect poses, Gemini 3 understands why a 45cm hip distance matters for a static reach versus a dynamic move.

2. Natural Language Coaching

Gemini 3 translates technical metrics into actionable feedback: "Move 3: Your hips are 45cm from the wall—rotate your left hip in to shift your center of mass closer." This contextual understanding makes the difference between raw data and real coaching.

3. Adaptive Feedback Loop

Gemini 3 tracks progression over time, adjusting coaching intensity and focus areas as technique improves, creating a truly personalized AI coach experience.

Why Gemini 3 is Essential: The app's value isn't in pose detection—it's in intelligent interpretation. Gemini 3's ability to reason about biomechanics, understand climbing technique, and communicate in natural language transforms BetaBreak from a tracking tool into a genuine coaching platform.

Technical Stack

Frontend: React Native (learned from scratch)
Backend: Supabase for database and authentication
Computer Vision: MediaPipe for real-time pose estimation (33 keypoints tracked)
AI Analysis: Gemini 3 API for form analysis and personalized coaching
Development: Cursor AI for rapid prototyping and debugging

Architecture

The processing pipeline works like this:

User films attempt → MediaPipe pose estimation → Calculate metrics → Gemini 3 analysis → Actionable feedback

Challenges we ran into

Challenge 1: Gemini API Blocked in Hong Kong

My family is from Hong Kong, where Gemini API is blocked. I'm building a climbing app that needs Gemini to work. The irony wasn't lost on me.

Solution: I set up routing services to reroute API calls through supported regions. I used proxy servers and VPN routing to make it work. Not ideal, but it taught me resilience—if you believe in an idea enough, you find workarounds.

Challenge 2: No Prior Art in "AI Climbing Form Analysis"

This has never been done before in climbing. There's no "AI beta analysis" category. No competitors to learn from. No user research to reference.

Solution: I became my own first user. I used BetaBreak in my training for months, documenting everything—successes, failures, iterations. Then I shared my journey on social media.

The response was overwhelming: nearly 600k views across platforms. Climbers worldwide reached out saying they needed this tool.

Challenge 3: Product-Market Fit

Getting views is one thing. Building something people would pay for? That required real validation.

Solution: I conducted over 20 user interviews with climbers who followed my journey:

"What's missing in the current version?"
"Would you use this at your gym?"
"What would make you choose this over posting videos in your gym's Discord?"

The feedback shaped everything:

✅ Users wanted move-by-move breakdown (not general analysis)
✅ They wanted Gemini to generate training plans based on weaknesses
✅ They wanted progression tracking over time

I implemented all of it.

Challenge 4: Learning Full-Stack Development

I had zero frontend experience before this. React Native, Supabase, API integration, mobile deployment—I learned it all through:

Building with Cursor AI as my pair programmer
Watching countless YouTube tutorials (then using Gemini to summarize them!)
Iterating rapidly and breaking things constantly

The learning curve was steep, but necessary. You can't outsource understanding your own product.

Accomplishments that we're proud of

1. Real Users, Real Impact

I've conducted 20+ user interviews and have active beta testers across 5 countries. The feedback loop is incredible:

"I sent my first V7 after using BetaBreak's hip rotation feedback. This is game-changing." — Beta tester, Singapore

2. Solving My Own Problem

The biggest accomplishment? BetaBreak actually works for me. My sending rate improved measurably after using my own app's feedback. When your product solves your own desperate need, you know you're onto something.

What we learned

Technical Skills

Frontend development: React Native, component architecture, state management
API integration: RESTful APIs, authentication, rate limiting
Computer vision: Pose estimation, keypoint tracking, confidence thresholds
Prompt engineering: Crafting prompts that turn raw data into actionable coaching
Mobile deployment: App Store submission, provisioning profiles, TestFlight

Product Thinking

User research matters: Those 20 interviews fundamentally changed my feature priorities
Iteration > perfection: Ship fast, get feedback, improve. Repeat.
Solve your own problem first: Authentic need shows in every design decision

Resilience

Work around constraints: API blocked? Find another way.
Learn in public: Sharing my journey created accountability and brought users who shaped the product
Fail forward: Every bug, every rejected feature taught me something

What's next for BetaBreak - Your AI Climbing Coach

Short-term (Next 3 Months)

1. Fine-tuned Models for Specific Moves

Train specialized models on real user data for:

Flags and drop knees
Dynamic moves (dynos, deadpoints)
Mantles and top-outs

Expected improvement in accuracy: $\Delta_{accuracy} \approx 20-30\%$ for move-specific analysis.

2. Gym Partnerships

Validate business model through partnerships with climbing gyms:

Offer BetaBreak as a premium member benefit
Install dedicated filming stations
Gather data on real usage patterns

3. Training Plan Automation

Leverage Gemini's memory retention to create weekly training plans:

Week 1: Focus on hip mobility (detected weakness: hip distance averaging 42cm, should be <30cm)
Week 2: Flag technique drills
Week 3: Integration - apply improved technique to project routes

Medium-term (6-12 Months)

4. Social/Community Features

Compare your form against successful attempts by climbers with similar body types
Share progress with training partners
Leaderboards for improvement rate $\frac{\Delta_{form_score}}{\Delta_{time}}$

5. Multi-angle Analysis

Support filming from multiple angles simultaneously:

$$\text{Form Score} = f(\theta_1, \theta_2, ..., \theta_n)$$

where $\theta_i$ represents analysis from angle $i$.

6. Wearable Integration

Integrate with climbing-specific wearables to track:

Grip strength over time
Heart rate during attempts
Recovery metrics

Long-term Vision

Make expert climbing coaching accessible to everyone, regardless of location or budget.

Currently, high-quality coaching is concentrated in major climbing hubs (Boulder, CO; Innsbruck, Austria; etc.) and costs prohibitive amounts. BetaBreak can: