BounceBoard

BounceBoard turns an iPhone and Apple Watch into a practical gymnastics trampoline training lab—built by a coach, with Codex, for athletes and coaches.

Comment

Inspiration

For about 15 years, trampoline gymnastics was a major part of my life. I competed at national level in Germany, and today I still work in the sport as a coach. This lets me see practice from both sides: performing a skill and trying to explain what happened during the few seconds it took to complete it.

Alongside sport, software development has always been my other passion. I have worked professionally as an SAP software developer since 2016, began experimenting with machine learning and computer vision in 2017, and closely follow developments in LLMs, coding agents, and agent harnesses.

For years, I wondered whether computer vision could identify athletes, detect bounces, recognize body positions, and document skills or routines during practice. Video helps coaches, but trampoline is such a niche sport with few specialized models or large public datasets. General pose models often struggle when an athlete is small in the frame, rotating quickly, upside down, or blurred by motion. Building something useful alone therefore seemed to require data and resources I did not have.

Recent advances in coding agents inspired me to revisit the idea from a different angle: combine iPhone video with acceleration, rotation, and orientation data from an Apple Watch. Together, these devices can describe much more of a practice than either source alone while keeping the setup simple enough for everyday training.

What it does

BounceBoard is a local-first trampoline training workflow across Apple Watch, iPhone, and Mac:

  1. The Apple Watch records workout, accelerometer, and device-motion data for each training segment.
  2. The iPhone controls the practice, records synchronized video, aligns the device clocks, safely receives the Watch recording, and stores the session.
  3. The macOS Review Hub brings video, sensor signals, local Apple Vision pose analysis, athlete identity, and trampoline skill annotations into one timeline.

A coach can inspect a segment, compare movement with the sensor signals, select the correct athlete, and confirm or correct suggested skill intervals. Automated results remain suggestions: if athlete identity is ambiguous, BounceBoard asks the coach instead of guessing, and a detected skill is not treated as reviewed truth until a person confirms it.

The goal is not to replace a coach. It is to give athletes and coaches better shared evidence, make training easier to document, for example with an automated training dairy for athletes and coaches, with trustworthy practice history using a phone and Watch they may already own. Athlete video, motion, and pose data remain on the user's devices unless a session is explicitly exported.

How we built it

Although I built BounceBoard as a solo developer, Codex was my engineering collaborator throughout the entire project. I supplied the trampoline expertise, coaching workflow, requirements, physical-device testing, and final product decisions. Codex helped with architecture, Swift and SwiftUI implementation, debugging, research experiments, refactoring, UI iteration, and verification across three connected applications.

The project uses HealthKit and Core Motion for Watch capture, WatchConnectivity for communication and transfer, AVFoundation for iPhone video, Apple Vision for local 2D pose extraction, and GRDB/SQLite for Review Hub data.

One of the best ideas from my Codex work began with an observation: the Watch often records strong rotation and orientation changes exactly when a conventional pose detector stops recognizing the athlete. Codex suggested treating this partly as an input-orientation problem—retry the same video frame under alternative rotations, then transform detected joints back into the original coordinates.

Testing also exposed an important limitation: a Watch measures the wrist, not the torso. We therefore refined the idea into a conservative two-pass recovery pipeline. BounceBoard keeps reliable standard Vision results and retries only missing or weak frames using a tracked athlete region and alternative quarter-turn orientations. Recovered poses must pass confidence, coverage, scale, and temporal-continuity checks. In a focused experiment, this approach produced pose observations in 29 of 33 sampled frames where the normal pass had failed.

BounceBoard existed as a working capture-and-review prototype before Build Week. During the event, I used Codex with GPT-5.6 to turn it into a resilient end-to-end workflow. The Build Week work added receipt-driven Watch transfer, recoverable recording and video finalization, clock alignment, conservative athlete identification, exact skill intervals, human-confirmed label states, preserved-source recovery, privacy and release checks, and a redesigned cross-device experience. The seven event commits touched 190 files, and the test suite grew from 69 to 177 test methods.

GPT-5.6 and Codex were development-time collaborators. BounceBoard does not require an OpenAI model or API at runtime.

Challenges we ran into

The first major challenge was reliability across three devices. Connections can disappear, apps can be suspended, and transfers can be delayed. I learned that a queued file is not necessarily a safe file, so the Watch retains its recording until the iPhone confirms a durable import. Original recordings are treated as irreplaceable, while indexes, caches, analyses, and exports can be rebuilt.

Synchronization was equally difficult. Watch motion, phone video, segment commands, and annotations originate from different clocks. Even a small error matters when a complete skill may last only a fraction of a second, so BounceBoard records alignment evidence and distinguishes exact from approximate timing.

Computer vision introduced another trust problem. A video may contain a coach, another athlete, or people walking behind the trampoline. The most confidently detected body is not always the athlete wearing the Watch. BounceBoard therefore preserves uncertainty and lets the coach resolve ambiguous cases.

Finally, the lack of a large sport-specific dataset meant I could not begin with a polished automatic skill classifier. Instead, I focused on multimodal evidence, transparent detector suggestions, and a human-review workflow that is useful now while creating better labeled data for the future.

Accomplishments that we're proud of

I am proud that BounceBoard connects my experience as an athlete, coach, and developer in one working product. It provides a complete path from a quick phone-and-Watch setup to synchronized, reviewable training evidence across three Apple platforms.

I am especially proud that reliability and uncertainty are visible product features. Recordings survive interrupted transfers, source data survives database rebuilds, athlete ambiguity is surfaced instead of hidden, and automated labels remain proposals until a coach confirms them. A synthetic no-Watch demo also lets judges explore the stored-session workflow without exposing personal athlete footage.

What we learned

The most important technical lesson was that sensor fusion is not just about collecting more data; it can change the shape of a problem. Watch motion cannot replace video, and wrist orientation is not torso orientation, but it can reveal rotational structure, help locate important moments, and explain why visual analysis becomes uncertain.

I also learned that reliability matters more than an impressive chart if a coach cannot trust the recording to survive. That realization shaped the storage and transfer architecture across all three devices. ( I have made multiple times the personal unfortunate experience during the development process, of going to the gym, recording my whole practice with the intend of gathering more sample data, to then sadly discover at home, that some sort of silent failure, connectivity issue or bug stopped the whole recording )

Finally, Codex was most effective when I supplied clear domain constraints rather than only implementation tasks. Requirements such as “do not lose a practice,” “do not guess the athlete,” and “a detected skill is not reviewed truth” influenced the architecture more deeply than any individual framework choice.

What's next for BounceBoard

Next, I want to test BounceBoard with more athletes and coaches in real practices, improve onboarding and capture feedback, and collect consented, human-reviewed examples across different athletes, camera positions, and skill levels.

Over time, that dataset could support a multimodal temporal model combining pose sequences with Watch motion. Any future classifier should preserve the principle established by the current application: automation assists the coach, but a person owns the final training record.

Built With

  • apple-vision
  • apple-watch
  • avfoundation
  • codex
  • computer-vision
  • core-motion
  • gpt-5.6
  • grdb
  • healthkit
  • ios
  • macos
  • sqlite
  • swift
  • swift-charts
  • swiftui
  • watchconnectivity
  • watchos
  • xcode
  • xcodegen
  • xctest
Share this project:

Updates