Volante

A fitness pod device for your badminton racket!

Comment

Inspiration

While playing badminton, we kept running into the same problem: there's no good way to measure your own badminton game accurately. Smartwatches count steps and heart rate, but they can't tell a smash from a drop shot or how hard you actually swing. Racquet sensors for badminton are rare, expensive, and lock your data inside closed apps.

So we set ourselves a challenge to build a small, open sensor pod that clips onto any racquet and turns every swing into data that you can use to improve your badminton skills! We wanted to create a polished app and then wondered: what if your playing style could become an AI player?

This led us to focus on:

  • Measuring every swing at 500 samples a second (count, estimated head speed, rallies, tempo) and showing it in a way that's simple and never overclaims
  • Combining racquet data with health data from Apple Watch, Fitbit, Apple Health, and Google Health, so a session shows effort and recovery as well as swings into multiple services
  • Turning your style into a "twin": an AI badminton player trained to play at your tempo and power, competing on a public ladder
  • Labeling your swings accurately and precisely: speeds are labelled as estimates, the experimental stroke model is labelled experimental, and insights always show the numbers behind them

What it does

  • The Volante pod (an ESP32-S3 with a 6-axis motion sensor) clips onto your racquet. It streams live over Bluetooth, or records on its own and uploads afterwards.
  • The iPhone app detects every swing, estimates head speed in km/h, finds rallies and tempo, and gives each swing an experimental power/mid/touch stroke group from an on-device model we trained on our own recorded data.
  • After every session it opens a Summary:
    • 2–4 insights from rules, each citing its evidence, e.g. "Your hard swings faded 42% in the last third", "Median 21 shots per rally".
    • An intensity timeline.
    • A heart-rate card with time in 5 zones, an effort score and 1-minute heart-rate recovery.
  • Health integration: heart rate from Apple Watch, Fitbit (through the Google Health API) and Apple Health, or from an imported CSV/JSON file. You can add heart rate after a session if your tracker hadn't synced.
  • The Arena: a public website and in-app tab where AI badminton players trained with our free kit play ranked matches 24/7 in a physics simulator. Every match can be watched in 3D. Sign-in is with GitHub, and there are Elo ladders and weekly tournaments.
  • Volante Twins: the app turns your sessions into a style profile (power, consistency, tempo). You train an AI twin from it with reinforcement learning, and the server only admits it to the Twins ladder once it actually plays like you.

Core experience

  • A customizable Summary dashboard: weekly goal ring and streaks, last session with health, heart-rate trends, effort, time in zones, recovery, personal bests. You can add, remove, reorder and resize widgets, and there's a home-screen widget.
  • Live play mode with a Live Activity on the Lock Screen and Dynamic Island showing swings and speed as you play.
  • The Arena in your pocket: ladders, model pages, match replays in 3D, tournament brackets, and uploading your own AI from your phone.
  • Your style → your twin: export my_style.json, train, upload, and watch an AI play with your rhythm.

How we built it

  • Hardware & firmware: Seeed XIAO ESP32-S3, MPU-9250 IMU, Rust (Embassy) firmware with a custom BLE protocol, 500 Hz recording to flash and live streaming
  • iPhone & Apple Watch: Swift, SwiftUI (Liquid Glass), Swift Charts, CoreBluetooth, CoreMotion, HealthKit, WidgetKit, ActivityKit (Live Activities), WatchConnectivity, AuthenticationServices, XcodeGen
  • Shared analysis library: OsoKit (pure Swift), byte-for-byte compatible with our Rust and Python reference implementations through shared test vectors
  • 3D replay & simulator viewer: Rust, wgpu, WebAssembly
  • Website: TypeScript, Vite, a strict Content Security Policy (no innerHTML), Phosphor icons, a Liquid Glass design system shared with the app
  • Arena backend: Python, FastAPI, SQLite, systemd-sandboxed workers that run every untrusted uploaded model with no network access, GitHub OAuth (with a PKCE-style flow for the iPhone app)
  • AI / ML:
    • Arena players: PyTorch PPO self-play, exported to safetensors.
    • Stroke classifier: scikit-learn gradient boosting plus a 1D CNN, exported to ONNX and to pure-Swift tree models.
  • Infrastructure: Raspberry Pi 5, Tailscale, GitHub Actions CI plus continuous deployment to the Pi
  • Health APIs: Google Health API (Fitbit), Apple HealthKit
  • Dev tools: Claude Code, GitHub, Xcode

Challenges

  • Physical-sensor reality: a gyro clips at ±2000°/s on hard smashes, packets drop over Bluetooth, and mounts rotate. We built quality checks, labelled clipped speeds as lower bounds, and made the analysis rate-aware across the pod (500 Hz) and the watch (200 Hz).
  • Keeping four implementations identical: swing detection and the style profile exist in Rust, Python, Swift and TypeScript. We pinned them together with shared test vectors that must match byte for byte.
  • Networking in the real world: campus Wi-Fi isolates devices, iCloud Private Relay silently intercepted the app's requests on cellular, and one phone was signed into the wrong Tailscale account. We fixed it by serving tailnet-only HTTPS and having the app explain failures in plain language.
  • ** HandlingML with little data**: our stroke model was trained on one drill session. Its per-stroke accuracy is low, so we ship the reliable 4-group model and clearly mark the rest as experimental.

Accomplishments

  • A working racquet pod and iPhone app that recorded real sessions alongside Apple Watch and Fitbit heart rate integration
  • AI players that actually play: our House Base model wins 200/200 games against the scripted bot.
  • A real AI twin: trained with reinforcement learning on our own swings, and it passes the Arena's fidelity check (it hits at 1.73 s between shots vs. our 1.67 s).
  • A live public platform on a Raspberry Pi with GitHub sign-in, sandboxed uploads, ladders, tournaments and 3D replays, deployed automatically on every merge.
  • A polished, unified iPhone app: health data from every source becomes the same zones, effort and recovery numbers, on a customizable dashboard.

What we learned

  • Ground every number in its data: insights that cite their evidence, and labels that say "estimated" or "experimental", build more trust than impressive-looking claims.
  • Shared test vectors keep a multi-language system honest. They caught real rounding and edge-case bugs between Rust, Python and Swift.
  • Security and usability have to be designed together: sandboxing untrusted models, PKCE-style app sign-in, and keeping private servers off the public internet without making setup painful.
  • Small UX decisions matter: one unit (km/h) everywhere, collapsible long lists, failures explained in plain words, and a dashboard users arrange themselves.

What's next for Volante

  • More data, better strokes: record more players and sessions so the stroke model can name individual strokes and forehand/backhand reliably
  • Apple Watch as a second sensor: wrist motion plus heart rate alongside the pod for a fuller picture of every rally
  • Twins for everyone: in-app training (no computer needed), with weekly Twins tournaments where your AI plays your friends'
  • Coaching over time: trends across weeks, recovery-aware suggestions, and shareable session reports on the web
  • A smaller, sealed pod with a proper battery gauge, plus pairing pods to your Arena account

Built With

Share this project:

Updates

Submission history