The Hydra Heads

From patient standing to personalized recovery session in 90 seconds no wearables, no setup, no guesswork.

Comment

Inspiration

Recovery practitioners — physical therapists, sports trainers, MedSpa clinicians — make critical protocol decisions every session with almost no objective data. They ask "how are you feeling?" and guess the rest. We wanted to change that. The Hydrawav3 device is powerful, but the intake process before every session is slow, manual, and subjective. We saw an opportunity to make the device smarter by giving it eyes, ears, and a brain.

What it does

Hydra Heads is an AI recovery intelligence layer for the Hydrawav3 wellness device. In under 90 seconds it:

  1. Assesses posture using MediaPipe's 33-landmark pose detection — measuring shoulder elevation, knee flexion, hip asymmetry, and ROM flags in real time
  2. Measures contactless vitals — heart rate, breathing rate, and HRV from the patient's face and shoulders using skin-color rPPG (CHROM + POS algorithms), with no wearables or physical contact
  3. Analyzes posture photos with Claude Vision — detecting compensation patterns and recommending Sun/Moon pad placement from a single image
  4. Generates a personalized protocol using Claude AI, combining camera data, vitals, and patient intake into a specific pad placement, goal, intensity, and duration recommendation
  5. Speaks the protocol aloud via ElevenLabs voice synthesis — fully hands-free for the practitioner
  6. Fires the session to the device via a single MQTT API call, starting the Hydrawav3 session autonomously
  7. Logs outcomes and generates a downloadable PDF report with all vitals, protocol, and session history

How we built it

The entire application is a single-file React + Vite web app — no backend required. Key technical components:

  • Contactless vitals: CHROM rPPG (de Haan & Jeanne 2013) + POS rPPG (Wang 2017) on skin-filtered ROIs. We use YCbCr color space to isolate skin pixels only, ITA angle for skin-tone calibration, Welch PSD with parabolic interpolation for HR frequency estimation, and motion gating to exclude high-motion frames from the signal.
  • Pose assessment: MediaPipe Pose (33 landmarks via CDN) computing joint angles, asymmetry scores, and mobility flags against the UI-PRMD clinical reference dataset.
  • Claude AI: Protocol generation from structured JSON bundles containing pose angles, vitals, and patient intake. Claude Vision analyzes posture photos for pad placement recommendations.
  • ElevenLabs: Text-to-speech for protocol delivery and vitals readout — the app is fully voice-navigable.
  • MQTT device control: JWT-authenticated REST API → MQTT publish to HydraWav3Pro/config topic with full session configuration including PWM values, vibration ranges, and thermal pad assignment.
  • PDF reports: jsPDF-native text generation (no html2canvas) producing clean multi-page reports with protocol, vitals, and session history.

Challenges we ran into

The hardest problem was contactless HR accuracy. Green-channel rPPG sounds simple but fails badly in practice — camera auto-exposure adjustments create 0.5–1 Hz artifacts that completely mask the actual pulse frequency. We went through four major iterations:

  1. Single green channel → too noisy, auto-exposure interference
  2. CHROM algorithm → better, but skin tone dependent
  3. Added YCbCr skin-pixel filter → dramatically improved SNR by removing hair/shadows/background
  4. Added motion gating + linear detrending + parabolic FFT interpolation → finally got consistent ±3–6 bpm accuracy

We also hit browser CORS issues calling the Anthropic API directly and had to add the anthropic-dangerous-direct-browser-access header. The PDF generation was completely blank initially because html2canvas cannot render backdrop-filter and rgba transparency — we replaced it entirely with jsPDF text APIs.

Accomplishments that we're proud of

  • Contactless vitals that actually work — watching the live BPM update in real time as someone goes from resting (68 bpm) to post-exercise (140+ bpm) and the AI automatically shift the protocol from activation to recovery is genuinely impressive
  • A fully closed loop — camera → AI → voice → device → PDF report, with zero manual steps between assessment and session start
  • All skin tones supported — ITA-based alpha calibration means the rPPG algorithm self-adjusts for the patient in front of it
  • One file — the entire application is a single App.jsx with no backend, deployable instantly

What we learned

Building contactless vitals taught us that signal quality matters more than algorithm complexity. A mediocre algorithm on clean skin-only pixels beats a sophisticated algorithm on noisy full-frame data every time. We also learned that browser-based AI integration is surprisingly capable — Claude Vision analyzing a posture photo and returning structured pad placement recommendations in 3 seconds felt like magic the first time it worked.

What's next for The Hydra Heads

  • Mobile app (Expo React Native) with the same pipeline running on phone camera
  • Longitudinal tracking — comparing vitals and ROM across sessions to measure recovery trends over time
  • Practitioner dashboard — multi-patient history, protocol analytics, and outcome scoring
  • HRV-guided protocol adaptation — dynamically adjusting session intensity mid-session based on real-time HRV changes
  • Integration with EHR systems for clinical-grade documentation

Built With

Share this project:

Updates