Smart gym

We make any gym smart with automatic workout tracking—no manual logging and no equipment replacement, the gym do the job for you.

Comment

Inspiration

Smart Gym began with a simple frustration: strength training is still mostly manual and poorly traceable. Members count reps, remember weights, and log workouts after the fact. Gyms, meanwhile, have little objective data about how equipment, zones, and schedules are actually used.

Wearables help in some contexts, but they require active participation and are often unreliable for strength exercises. We wanted to explore how an installed camera system and connected equipment could turn an ordinary gym into a smart, data-aware environment without requiring members to interact with each machine.

What it does

Smart Gym automatically builds a member’s workout session from gym-floor signals:

  • AI-powered cameras observe an exercise and count repetitions.
  • Connected equipment supplies load context, such as paired dumbbells.
  • A backend creates a live workout timeline linked to the member's wristband identity.
  • Members can review and correct detected reps or load.
  • The gym can eventually use aggregated, privacy-conscious operational data to understand equipment and space usage.

The demo replays local gym videos through the same inference-to-backend flow used by the system.

How we built it

The submission packages the project as a reproducible Docker Compose application, while keeping clear production service boundaries:

  • React + TypeScript member web application.
  • FastAPI + PostgreSQL backend for authentication, workout state, and corrections.
  • YOLO pose inference service that processes video frames and emits set events.
  • NFC dumbbell-pairing fixture to demonstrate connected-equipment load context.
  • Gym equipment profiles for known barbell and plate configurations.

The edge inference service is stateless and uses a wristband identifier rather than personal data. The backend is the source of truth for the workout lifecycle.

Challenges we faced

The central challenge was balancing a compelling demo with technical honesty.

The demo uses real pose inference from video frames to produce rep counts. However, reliable vision-based plate recognition requires labeled images of a specific gym's plates, lighting, camera angles, and layouts. We have not claimed that a plate-recognition model is already trained. Instead, barbell load in the demo comes from a transparent gym profile, while dumbbell load comes from a simulated NFC pairing.

We also faced the practical challenge of making an ML-heavy stack easy to evaluate. The inference image is published through GitHub Container Registry so reviewers can pull the prepared environment instead of rebuilding large vision dependencies.

Privacy and scaling

The intended deployment is local edge computing inside each gym: cameras and inference run on-site, reducing latency and avoiding a dependency on continuous internet connectivity. The product direction is to process video locally and retain structured workout events rather than personal video footage.

The architecture is modular: a gym can add processing capacity by zone, and the same setup can be replicated across multiple locations. During installation, each gym would be calibrated for its layout, equipment, and plate types.

What we learned

We learned that trustworthy gym intelligence is not one model. It is the combination of uncertain observations, equipment context, deterministic state management, and clear provenance.

We also learned that gym-specific data collection is a product requirement, not an afterthought. Pose inference provides a strong foundation today; gym-calibrated object detection, form analysis using 3D landmarks, and history-aware coaching are the next layers.

What's next for Smart Gym

The next milestone is a real-gym pilot: install cameras, calibrate each monitored zone, collect consented gym-specific data, and validate the system under real lighting, layouts, equipment, and member traffic.

From that foundation, Smart Gym can add:

  • Gym-specific vision models for plates, bars, dumbbells, and machines.
  • More advanced training metrics and form analysis using richer pose landmarks.
  • Equipment, zone, and occupancy analytics to help operators optimize layouts, schedules, and maintenance.
  • Performance reports and personalized services for members, grounded in confirmed workout data rather than generic estimates.
  • A privacy-first local data layer that keeps processing in the gym while letting members export their own workout history.

We also envision an on-premise operations agent running on the gym's local computer, such as a Mac mini. It would help staff register members and equipment, monitor cameras and services, explain system alerts, guide routine troubleshooting, and escalate failures that need human attention.

Over time, the same structured data could support assistants for both members and gym operators: querying training history, understanding facility usage, and generating recommendations based on the real environment.

How GPT-5.6 and Codex helped

GPT-5.6 and Codex accelerated the work across architecture, implementation, and verification. They helped consolidate a multi-repository prototype into a reviewer-friendly Docker Compose project, trace contracts across inference, backend, and frontend, add migrations and tests, diagnose live-update issues, and document both capabilities and limitations clearly.

That collaboration let us focus on validating the end-to-end workout flow: video frames, pose inference, set creation, live UI updates, and member corrections.

Built With

Share this project:

Updates