Somavo Team

Somavo maps Apple Watch heart-rate intensity to exact spoken words, turning physiological speech anxiety into visual intelligence for high-stakes interview and presentation rehearsal.

Comment 1

Inspiration

Public speaking, job interviews, executive board presentations, and high-stakes negotiations trigger acute physiological stress responses. Most speakers are unaware of the exact moments their composure breaks, leading to filler words, rapid speech cadence, or memory blanking.

Existing tools offer either text-only transcript analysis (like Otter.ai) or isolated biometrics (like Apple Fitness). None connect what you said with how your body reacted.

Somavo bridges this gap. It maps Apple Watch heart-rate intensity directly onto live audio transcripts, turning physiological stress into an actionable, interactive pressure map.

What It Does

Somavo is a native iOS + watchOS app backed by a high-performance Go serverless API that transforms verbal practice into a closed-loop physiological rehearsal system:

  • Biometric-Speech Alignment: Captures real-time heart rate (BPM) via Apple Watch HealthKit and syncs it down to millisecond accuracy with live iPhone speech recognition.
  • Replayable Pressure Heatmap: Generates an interactive visual timeline where users can tap high-BPM spikes to hear the exact phrase spoken when stress peaked.
  • Privacy-Conscious Architecture: Audio capture, transcript generation, and physiological alignment remain 100% local on-device by default.
  • Cloud-Assisted AI Coaching: Opt-in integration with Anthropic Claude 3.5 for targeted interview/speech advice and Cartesia for voice synthesis—processed securely backend-side so API keys never ship on mobile devices.
  • Contextual Interview Prep: Generates customized interview briefs based on user-provided job descriptions and CVs, avoiding generic static question banks.

Operations Research & AI Integration (Orion Hackathon Special Focus)

Somavo applies Operations Research (OR) concepts to human performance and communication:

  • Optimization of Rehearsal Loops: Uses biophysiological data to solve a resource allocation problem—identifying the exact 10% of a speech responsible for 80% of physiological stress, maximizing rehearsal efficiency.
  • Time-Series Alignment & Queue Dynamics: Models speech cadence and cardiovascular variance as continuous time-series signals to detect stress spikes before vocal disfluency occurs.

$$\text{Rehearsal Efficiency Ratio} = \frac{\Delta \text{BPM Variance Mitigated}}{\text{Minutes Spent Practicing Targeted Spikes}}$$

How We Built It

  • iOS & watchOS Frontends: Built using Swift and SwiftUI. Utilizes HealthKit for biometric telemetry, Speech framework for live transcription, and WatchConnectivity for sub-second cross-device data streaming.
  • Go Backend API: Built in Go, deployed as serverless functions (AWS SAM / Cloud Run) with a single-table AWS DynamoDB database for fast metadata queries.
  • AI Engine: Integrated backend-side with Anthropic Claude 3.5 for structured coaching insights and Cartesia for realistic voice feedback.
  • Infrastructure & Testing: Docker Compose for local emulation (LocalStack DynamoDB & S3), XcodeGen for reproducible project generation, and Cloudflare Tunnels for live physical device debugging.

Challenges We Ran Into

  • Sub-Second Biometric Synchronization: Apple Watch HealthKit samples heart rate at variable intervals, whereas speech audio streams continuously. We engineered a temporal interpolation algorithm in Swift to align discrete BPM data points seamlessly across transcript timestamps.
  • Privacy-First Security Boundary: Balancing rich AI insights with strict user privacy. We established a strict boundary: raw audio and raw HealthKit samples stay on-device, sending only consent-gated, anonymized text segments for server-side AI evaluation.
  • Cross-Device Reliability: Maintaining stable WatchConnectivity sessions during active speech recording without dropping packets or draining watch battery.

Accomplishments That We're Proud Of

  • Native Apple Watch & iPhone integration that feels effortless during practice sessions.
  • Fully functional no-auth live sample experience so reviewers can test the pressure heatmap instantly.
  • Production-grade cloud & local stack ready for App Store deployment and enterprise team usage.

What We Learned

Biometrics add a missing dimension to NLP. Combining physiological signals with speech analysis reveals hidden anxiety triggers that traditional transcript analysis completely misses.

What's Next for Somavo

  • Enterprise Team Dashboards: Group analytics for sales teams, media training agencies, and executive coaches.
  • Predictive Stress Alerts: Real-time haptic cues on Apple Watch during live presentations to signal when to slow down or take a breath.
  • Multi-Sensor Support: Expanding support to Garmin, Oura Ring, and Whoop hardware.

Built With

  • aws-dynamodb
  • aws-sam
  • cartesia-ai
  • claude-3.5
  • docker
  • go
  • healthkit
  • ios
  • operations-research
  • speech-recognition
  • swift
  • swiftui
  • watchos
Share this project:

Updates

Submission history