RepRoom

companion page
architecture diagram
main interface
feedback
summary

Inspiration

A television is a natural screen for a home workout: it is large, visible from across the room, and leaves your hands free. But most workout videos are one-way experiences. They demonstrate an exercise without responding to what you actually do.

I wanted to explore a more interactive alternative using equipment people may already own: a television and a phone or webcam.

That became RepRoom—a Fire TV application that brings camera-based repetition counting and exercise feedback to the living room.

The guiding principle was simple: when the system cannot measure something, it should say so rather than display a confident-looking score.

What it does

RepRoom connects a companion camera to a television-based workout interface.

The intended flow is straightforward:

Choose an exercise on the TV using the remote.
Join the TV's session from the companion camera page.
Position the camera according to the exercise guidance.
Perform the movement while the service processes incoming frames.
Review counted repetitions, timing measurements, and the available exercise checks on the TV.

The exercise catalogue includes bodyweight squats, push-ups, overhead presses, bicep curls, and split squats or lunges.

The counting engine distinguishes completed repetitions from movements rejected by its range-of-motion or timing rules. It also contains logic for abandoning interrupted movements when tracking is lost.

Exercise checks depend on camera placement. For example, the application marks squat-depth feedback as unavailable from a front-facing camera instead of treating an unsupported measurement as a successful check.

These are experimental, camera-derived exercise heuristics—not medical assessments, validated coaching advice, or guarantees of safe technique.

How I built it

A React Native application for Fire TV

The TV application uses React Native and TypeScript with Amazon's React Native Kepler package for Vega OS.

I built it with the Vega Developer Tools, generated application packages, and installed and launched the application on the Vega Virtual Device. No physical Fire TV was used for this prototype.

The interface includes an exercise picker, API connection setup, a session identifier for camera pairing, workout readouts, and a set-summary screen. Remote-focus styling makes the controls visibly selectable without relying on touch input.

A companion-device camera

A browser-based capture page uses the device camera and sends JPEG frames to a Python FastAPI service.

This separates the camera from the television: the camera can be positioned for the exercise while the TV remains the display. The prototype does not depend on a camera built into a television or Fire TV device.

Pose estimation and deterministic counting

OpenCV decodes the images, and MediaPipe Pose Landmarker produces body landmarks.

A separate Python package converts those landmarks into joint-angle measurements and feeds them into a repetition state machine. Smoothing reduces short-term fluctuations, and separate movement thresholds help prevent small oscillations from being counted repeatedly.

The core geometry and counting package uses Python's standard library. MediaPipe is an optional detection dependency, allowing the counting logic to be tested independently of the vision model.

Amazon Bedrock integration

I implemented a coaching adapter using Amazon Bedrock's Converse API. Its intended role is narrow: turn measured session facts into a short coaching sentence.

Bedrock does not decide whether a repetition occurred. Counting and exercise checks remain deterministic.

The adapter excludes checks marked unmeasurable from its prompt and provides a labelled rule-based fallback when generation is unavailable. The interface distinguishes Bedrock-generated text from built-in rules.

A successful Bedrock generation has not yet been verified with configured account credentials. The fallback and error handling have been tested, including an authentication failure returned by the live AWS endpoint. The current demonstrated coaching path is the rule-based fallback.

Challenges I ran into

Building from a directory containing spaces

The React Native Vega build encountered a manifest-generation failure when the project was located beneath a directory containing a space.

Moving the project to a space-free location allowed the build to complete. This was a useful reminder that a successful earlier build stage does not prove the final application bundle was generated correctly.

Making remote focus visible

A mobile interface can rely on touch. A TV interface cannot.

I implemented explicit focus highlighting through focus and blur events so the selected control remains visible when navigating with a remote.

Connecting the emulator to the local service

The application running inside the virtual device needed access to the API running on the Mac.

I verified reverse port forwarding and confirmed that the application's own health and exercise-catalogue requests reached the service. This provided evidence of application execution and network connectivity, rather than merely successful package installation.

Separating tests from real-world accuracy

Synthetic landmark sequences are useful for testing counting transitions, timing rules, and exercise-check calculations. They do not establish whether a camera will recover a person's movement accurately.

Keeping those claims separate became one of the most important engineering decisions in the project.

Accomplishments that I'm proud of

Building and launching a React Native application on the Vega Virtual Device.
Connecting the running TV application to a local FastAPI service.
Processing a real image through OpenCV, MediaPipe, and the session API.
Separating the reusable counting engine from its camera and UI dependencies.
Building automated tests for geometry, counting, API behaviour, coaching fallbacks, and UI components.
Publishing the source under the Apache-2.0 license.
Documenting development friction and verification gaps rather than hiding them behind a polished interface.

What I learned

A passing test suite and a validated user experience are different things.

I learned how to build and package a Vega application, connect an emulator to a local service, design visible remote-focus states, and separate camera processing from the display.

I also learned why confidence and missing-data states belong in the product itself. A percentage can look authoritative even when some of its underlying checks could not run.

Finally, I learned to give generative AI a bounded role: explaining measurements is different from producing the measurements that determine the result.

Current validation and limitations

The application has been built and launched on the Vega Virtual Device, and its connection to the local API has been verified. Automated tests cover the counting logic, service endpoints, and UI behaviour. Pose inference has also been exercised on a real image.

However:

A complete live workout through the phone-camera-to-TV flow has not yet been validated end to end.
Repetition accuracy has not been benchmarked against a labelled corpus of real exercise videos.
Exercise feedback has not been validated by a qualified trainer.
A successful Amazon Bedrock generation remains unverified.
The application has not been tested on a physical Fire TV.

RepRoom is a working development prototype, not a production-ready fitness assessment system.

What's next for RepRoom

The next priority is validation rather than adding more exercises.

I want to test the complete camera-pairing and workout flow, compare repetition counts with human-labelled recordings, and evaluate performance across camera positions, lighting conditions, clothing, and movement variations.

After that, I plan to verify live Bedrock generation, improve pairing and connection recovery, test on physical Fire TV hardware, and refine the exercise checks with informed feedback.

The goal is a useful living-room workout companion that makes both its capabilities and its limitations clear.