Markus - AI Fitness Coach

Get personalized, real-time feedback on your workout form. Upload videos, analyze your posture, and chat with an AI coach to unlock your full potential. Powered by Google Gemini.

Comment

🧠 Markus – AI Fitness Coach

Inspiration

During our workouts, we often lack real-time guidance and objective feedback on our form — something only available with expensive personal trainers or motion-capture devices. Markus was born from the idea of democratizing smart coaching: combining AI posture analysis, Gemini intelligence, and serverless scalability on Google Cloud Run. Our goal: help anyone train safely and efficiently, anywhere in the world, with just their smartphone camera.

What it does

Markus – AI Fitness Coach analyzes workout videos to give users instant feedback on their form.

  • 📸 Upload your workout video (squat, push-up, lunge).
  • 🧍 Pose estimation extracts keypoints (hips, knees, shoulders, spine).
  • 🧩 Gemini feedback engine generates personalized, natural-language advice (“Keep your back straight”, “Avoid knee collapse”).
  • 🤝 AI Agents collaborate to design personalized workout plans and progressive challenges.
  • 💬 Chat with Markus, your AI coach, to get explanations, tips, or a new program instantly.

Everything runs serverlessly on Cloud Run, scaling automatically with zero maintenance.

How we built it

Markus is composed of several Cloud Run services working together:

  • ⚙️ Backend (Node.js + Express)

    • REST API to handle uploads, job dispatch, and AI feedback.
    • Deployed as a Cloud Run Service.

  • 📦 Video processing (Worker Pool)

    • Uses Pub/Sub to trigger pose estimation tasks.
    • Runs inference on open-source models like RTMPose / MoveNet (optionally on L4 GPU).

  • ☁️ Storage & Data Layer

    • Cloud Storage for videos and frames.
    • Firestore for user profiles, metrics, and progress tracking.

  • 🧠 AI Layer

    • Gemini API via AI Studio for chat and natural-language coaching.
    • Agent Development Kit (ADK) for collaboration between agents:
    • Form Coach Agent → analyzes movement & flags errors.
    • Program Designer Agent → adapts the workout plan based on results.

  • 🧰 Frontend (optional)

    • Simple Vite/React interface for uploads and chat.
    • Deployed as a second Cloud Run Service.

[ \text{Data Flow: Video → Storage → Pub/Sub → Pose Inference → Gemini → Feedback → User} ]

Challenges we ran into

  • 🎞️ Video handling: Managing uploads, size limits, and asynchronous processing efficiently on Cloud Run.
  • 🧩 Pose inference latency: Balancing performance vs cost when using GPUs for short tasks.
  • 🧠 Prompt engineering with Gemini: Getting concise, consistent, and human-sounding fitness advice.
  • 🔄 Multi-agent coordination: Ensuring both agents (Form & Program) share context through Firestore and stay in sync.
  • ⚖️ Scaling: Optimizing autoscaling and quotas (min instances = 0) to stay within free credits.

Accomplishments that we're proud of

  • 🚀 Fully serverless architecture: deployable in minutes on Cloud Run.
  • 💪 Real-time AI feedback using Gemini + open-source pose models.
  • 🤖 Collaboration between multiple AI Agents through the ADK.
  • 🧩 Clean modular Node.js backend that anyone can fork and extend.
  • 🎥 A working MVP that processes videos and provides actual fitness advice in seconds.

What we learned

  • How to combine AI Studio, Cloud Run, Pub/Sub, and Firestore into a scalable end-to-end AI system.
  • The power of Google’s AI stack — from Gemini to L4 GPUs — for building real-world AI experiences.
  • Best practices for deploying and debugging serverless AI pipelines.
  • The importance of prompt iteration to make AI feedback feel motivating, not robotic.
  • How multi-agent systems can cooperate to deliver context-aware personalization.

What's next for Markus – AI Fitness Coach

  • 🧩 Add 3D pose reconstruction and rep-count tracking.
  • 🗣️ Integrate voice feedback powered by Gemini or Veo for real-time corrections.
  • 🧠 Expand to multi-exercise workouts and advanced metrics (symmetry, stability, fatigue).
  • 📊 Add a weekly progress dashboard with BigQuery analytics.
  • 🌍 Launch a public beta on Cloud Run + Firebase for early users and fitness creators.

Markus is more than a project — it’s the beginning of a new era in AI-powered fitness coaching 🏋️‍♂️.

🧱 Tech Stack

Layer Technology Description
Frontend React + Vite (AI Studio-generated) Upload videos, display feedback & chat with Markus
Backend API Node.js + Express Handles uploads, Pub/Sub messages, and Gemini requests
AI Models Gemini 2.0 (AI Studio), MoveNet/RTMPose Gemini for natural language feedback; pose model for keypoint extraction
Multi-Agent System Google Agent Development Kit (ADK) 2 agents — Form Coach & Program Designer — collaborate on user improvement
Serverless Compute Google Cloud Run Hosts backend, frontend, and worker services
Async Processing Pub/Sub + Worker Pool Manages asynchronous video processing and inference
Storage Cloud Storage Stores user-uploaded videos and frame snapshots
Database Firestore Saves user profiles, posture metrics, and agent chat context
Optional GPU NVIDIA L4 (europe-west1/west4) Accelerates real-time pose inference
Monitoring Cloud Logging + Metrics Observability for events, latency, and autoscaling

🧩 System Architecture

graph TD

subgraph User Side
A[📱 User Uploads Video] --> B[🌐 Frontend (React/Vite)]
B -->|HTTP POST| C[/API Endpoint/]
end

subgraph Cloud Run Services
C --> D[⚙️ Node.js Backend API]
D -->|Store| E[(☁️ Cloud Storage)]
D -->|Trigger| F[[Pub/Sub Topic]]
end

subgraph Worker Pool (Cloud Run Jobs)
F --> G[🧠 Pose Estimation Worker]
G -->|Keypoints + Angles| H[(Firestore)]
end

subgraph AI Layer
H --> I1[🤖 Form Coach Agent]
H --> I2[🏋️ Program Designer Agent]
I1 -->|Gemini Prompt| J[(Gemini API)]
I2 -->|Personalized Plan| H
end

subgraph Output
J --> K[💬 AI Feedback & Recommendations]
K -->|Display| B
end

⚡ Cloud Run Deployment Overview

Component Type Region Autoscaling Notes
Backend API Service europe-west1 Min 0, Max 5 Handles main REST endpoints
Worker Pool Worker europe-west1 Min 0 Processes video inference jobs
Frontend Service europe-west1 Static hosting with API integration
AI Agents Service europe-west1 ADK runtime, connects to Gemini
GPU Pose Service (optional) Job europe-west4 1 GPU L4 Real-time keypoint extraction

☁️ Deployment Summary

  1. AI Studio → “Deploy to Cloud Run” for initial prototype generation.
  2. gcloud run deploy markus-api --source . --region europe-west1
  3. gcloud run jobs create pose-worker --image gcr.io/...
  4. Connect Pub/Sub → Worker → Firestore → Gemini.
  5. Add Frontend Service (React/Vite) → connected to API endpoint.
  6. Done. 🚀 Markus is live and scales automatically.

Built With

Share this project:

Updates