Porch Fitness

Voice agent fitness coach Samantha guides you through gentle chair-based exercises entirely by voice. The agent can remember your history, track pain patterns, and provides personalized insights.

Comment

Inspiration

What inspired us to create PorchFitness was watching family members struggle to find accessible exercise options. Many seniors and people with limited mobility want to stay active but face barriers like expensive gym memberships, intimidating fitness classes, or complicated technology. We saw an opportunity to combine ElevenLabs' natural conversational AI with Google's Gemini to create something truly special - a voice-first fitness companion that feels like having a personal coach on your porch.

The name "PorchFitness" embodies our vision: exercise should be as comfortable and accessible as sitting on your front porch. No special equipment needed, no pressure, just gentle movements guided by Samantha's supportive voice.

What it does

PorchFitness is a simplified webpage with 15 safe and effective exercises that makes gentle exercise accessible through AI-powered voice coaching and intelligent progress tracking:

Voice Coaching with Samantha (ElevenLabs):

  • Natural conversation for logging workouts: "I did the hamstring stretch, pain level 3, it felt easy"
  • Samantha automatically extracts exercise name, pain level, and difficulty
  • Real-time guidance during exercises
  • Personalized greetings based on your recent workout history

AI-Powered Insights (Google Gemini 2.0 Flash):

  • Generates motivational weekly summaries analyzing your progress
  • Creates personalized workout greetings
  • Identifies pain trends and provides encouragement
  • Adapts messaging based on your consistency

15 Gentle Exercises:

  • 12 seated exercises (perfect for limited mobility)
  • 3 standing exercises (for those who want more challenge)
  • Evidence-based movements targeting flexibility, strength, and mobility
  • Clear instructions with rep counts and hold times

Smart Progress Tracking:

  • Visual charts showing workout frequency
  • Pain level tracking over time for safety monitoring
  • Difficulty progression tracking (easy/medium/hard)
  • Weekly statistics and trends

Accessible Design:

  • Clean white-first aesthetic with subtle color guidance
  • Large fonts (18px base) and high-contrast text
  • 56px+ touch targets for easy clicking
  • Full keyboard navigation and screen reader support
  • Mobile-responsive on all devices

How we built it

Frontend:

  • HTML5 with Tailwind CSS for responsive, accessible design
  • Progressive Web App (PWA) with service worker for offline capability
  • Clean white-first design system with CSS variables for consistency
  • Large fonts and touch targets optimized for seniors

AI Integration:

  • ElevenLabs Conversational AI: Embedded widget with 3 custom client tools
    • getRecentHistory: Returns recent workouts for personalized greetings
    • logWorkout: Logs exercises with 6 parameters (exerciseName, painLevel, notes, repsCompleted, durationSeconds, difficultyLevel)
    • getWeeklySummary: Returns weekly stats for Gemini to analyze
  • Google Gemini API (Gemini 2.0flash): Generates personalized insights and weekly summaries

Backend (Firebase/Google Cloud):

  • Firebase Hosting for static site deployment
  • Cloud Functions (Node.js) for serverless API endpoints
  • Firestore for real-time database (users/{userId}/sessions collection)
  • Firebase Authentication (Google Sign-In)

Development Process:

  1. Designed 15 evidence-based exercises with clear instructions
  2. Built accessible UI with senior-friendly design principles
  3. Integrated ElevenLabs widget and configured client tools for voice logging
  4. Created Firebase Cloud Functions to handle workout logging and AI queries
  5. Connected Gemini API for intelligent weekly summaries
  6. Implemented progress tracking with visual charts
  7. Optimized for mobile and tested accessibility features
  8. Deployed to Firebase Hosting

Challenges we ran into

1. ElevenLabs Widget Not Loading: Widget wouldn't appear - script URL had changed from elevenlabs.io/convai-widget/index.js to unpkg.com/@elevenlabs/convai-widget-embed. Critical fix that unblocked voice features.

2. Agent Credits Exhausted: Testing the agent from ElevenLabs agent uses significant amount of token/credits. So instead of using them all up, decided to save some for the judges and testers.

3. Firestore Timestamp Issues: Progress page showed no data. Fixed by converting JavaScript Dates to firebase.firestore.Timestamp.fromDate() for proper query comparisons.

4. UI/UX Iterations: Started with colorful emoji-heavy design that overwhelmed seniors. Pivoted to clean white-first aesthetic with selective color only for guidance. Multiple font size adjustments to find the right balance (settled on 18px base).

5. Authentication Flow: Duplicate sign-in buttons across pages confused users. Centralized auth to homepage only, simplifying the entire user experience.

6. Mobile Widget Blocking Buttons: ElevenLabs widget covered action buttons on mobile. Resized widget in dashboard and adjusted positioning to fix UI conflicts.

What's next for PorchFitness

**Goals:

  • Add more exercises (balance, breathing, coordination)
  • Goal-setting feature: "I want to do 3 workouts this week"
  • Shareable progress reports for healthcare providers

Growth:

  • Multi-language support for diverse communities
  • Video demonstrations alongside images
  • iOS/Android native apps
  • Partner with senior centers and physical therapists

We want PorchFitness to become the trusted platform for accessible exercise, helping people stay active and independent through the power of conversational AI.

Accomplishments that we're proud of

  • Successfully integrating two powerful AI systems (ElevenLabs + Gemini) to work seamlessly together
  • Creating a truly accessible app that seniors can actually use (18px+ fonts, 56px+ touch targets)
  • Building natural voice logging - users can just say "I did the stretch, pain was 3" and it works
  • Achieving a clean, luxury wellness aesthetic without sacrificing functionality
  • Deploying a full-stack serverless app with Firebase + Google Cloud Platform

What we learned

Technical Insights:

  • ElevenLabs client tools are incredibly powerful for voice-first apps - they extract structured data from natural speech
  • Always use CDN package URLs (unpkg.com) for third-party widgets rather than direct domain links
  • Firebase Timestamp handling requires careful date conversion - never compare Date objects directly
  • Authentication state management should be centralized to avoid UX confusion

AI Integration:

  • Combining ElevenLabs (conversational) with Gemini (analytical) creates a powerful duo - each AI handles what it does best
  • Gemini's model is perfect for generating personalized, empathetic fitness insights
  • Structured prompts with clear context (workout history, pain trends) produce much better AI responses

Design for Accessibility:

  • Seniors prefer clean, uncluttered designs with generous white space over colorful, busy interfaces
  • Font size matters more than we thought - 18px base is the minimum, not a luxury
  • High contrast and large touch targets (56px+) make a huge difference in usability
  • Testing on actual mobile devices reveals issues that desktop responsive mode doesn't catch

Development Process:

  • Start with accessibility requirements first, not as an afterthought
  • Progressive enhancement works - build core features that work without JavaScript, then enhance
  • Good documentation (like our EXERCISE_KNOWLEDGE.txt) helps both development and judges understand the project

What's next for PorchFitness

Immediate (Post-Hackathon):

  • Add more exercises (targeting balance, breathing, coordination)
  • Implement goal-setting feature ("I want to do 3 workouts this week")
  • Create shareable progress reports for healthcare providers -Create and release the Google-Play Store app/aab version when we figure out the tokens/credits issues.

Short Term:

  • Multi-language support (Spanish, Mandarin for diverse senior populations)
  • Voice customization (different coach personalities)
  • Integration with wearable devices (heart rate monitoring during exercises)
  • Group challenges for community engagement

Long Term:

  • Build iOS/Android native apps for better offline support
  • Add video demonstrations alongside images
  • Create personalized exercise plans based on pain patterns and progress
  • Telehealth integration for physical therapy follow-ups

Built With

Share this project:

Updates