IronVision

IronVision turns your phone into a real-time lifting coach — tracking barbell path and joint angles on-device to catch bad form before it becomes an injury. Free to start, zero lag

Comment

IronVision

An on-device AI biomechanics coach for Android — no internet, no cloud, no compromise on privacy.

Submitted for RevenueCat Shipaton 2026 — Next Gen Award (Student Track)

Inspiration

Every gym has someone lifting with bad form and no one there to catch it. Personal coaching is expensive and most fitness apps either need a subscription-gated cloud backend or don't actually watch your form at all. IronVision was built to put real biomechanical analysis in everyone's pocket — using nothing but the phone's camera and on-device inference.

What it does

IronVision turns an Android phone into a real-time motion capture and coaching system:

  • Live skeletal tracking at 30+ fps using Google ML Kit's BlazePose model, extracting 33 3D body landmarks per frame
  • Joint angle calculation (e.g. squat depth, elbow flare) using vector dot-product math on the tracked landmarks
  • Automatic rep counting via a five-state rep state machine (Idle → Eccentric → Inflection → Concentric → Lockout) that detects full reps without manual input
  • Bar path / medial drift tracking — flags when the barbell (proxied via wrist midpoint) drifts more than ~4.5cm off the vertical path over midfoot, a key form fault in squats and deadlifts
  • Winter Arc, a built-in 8-week structured Push/Pull/Legs program with adherence tracking
  • A local training dashboard aggregating weekly volume from on-device data
  • Offline reminders via WorkManager, with zero reliance on Firebase or any backend

How we built it

  • ML pipeline: Google ML Kit pose-detection-accurate, using the bundled model (not the unbundled/downloadable one) so the app works fully offline immediately after install, with CPU/GPU hardware acceleration for real-time inference
  • Camera integration: CameraX LifecycleCameraController + MlKitAnalyzer with COORDINATE_SYSTEM_VIEW_REFERENCED, which handles the notoriously tricky sensor-rotation-to-screen-coordinate transform automatically
  • Biomechanics engine: a custom BiomechanicsEngine.kt that converts raw (x, y) landmark coordinates into joint angles via the vector dot-product / arccosine formula, and drives the rep state machine
  • UI: 100% Jetpack Compose (Material 3), with a custom Compose Canvas overlay (PoseOverlayCanvas.kt) drawing the skeleton, bounding boxes, and drift plumb lines at 60Hz
  • Persistence: Room (SQLite) with KSP for compile-time codegen — all sets, sessions, and rep-by-rep telemetry stored locally
  • Monetization: RevenueCat SDK, gating a Pro tier (historical form comparisons, bar-path exports) behind a subscription while live tracking and rep counting stay free
  • Security: R8 full-mode obfuscation on the biomechanics engine, and native libraries packaged uncompressed for Android 16KB page-size compliance

Challenges we ran into

  • Getting camera sensor rotation to map cleanly onto the UI overlay across different device orientations
  • Keeping joint-angle readouts stable frame-to-frame despite natural pose-detection jitter
  • Designing a rep state machine general enough to work across different lift types (squat, deadlift, press) without hardcoding per-exercise logic

Accomplishments we're proud of

  • A fully offline, privacy-first pipeline — no video or pose data ever leaves the device
  • A real biomechanics engine (vector math + state machine), not just a pose-detection demo
  • Clean RevenueCat integration separating free live-tracking from a premium analytics tier

What we learned

Building real-time computer vision on-device means the hard problems aren't the ML model itself — they're the surrounding plumbing: coordinate transforms, frame-rate-stable state tracking, and turning noisy landmark data into something a lifter can actually trust mid-set.

What's next for IronVision

  • Smoothing (EMA) on joint angle calculations to reduce jitter further
  • Per-exercise calibration for more accurate real-world drift measurements
  • Expanding the exercise library and Winter Arc program content
  • Publishing to Google Play and the Samsung Galaxy Store

Built with

Kotlin · Jetpack Compose · CameraX · Google ML Kit (BlazePose) · Room · KSP · WorkManager · RevenueCat

Built With

Share this project:

Updates

Submission history