Inspiration

Rehabilitation exercises often involve repeating the same movements, with little immediate feedback or encouragement. We wanted to explore a simple question: could those repetitions become part of an adventure?

That idea became ReHA, a browser-based rehabilitation exercise prototype that connects physical movement with a small fox’s journey. Our goal was to make exercise more engaging while keeping the setup simple: a computer, a webcam, and a browser.

What it does

ReHA turns squats into progress in an interactive adventure. Using a webcam, it tracks body landmarks, counts completed repetitions, and provides real-time feedback on issues such as insufficient squat depth or excessive forward leaning.

Each completed repetition advances the fox’s reward animation, giving users a visible reason to keep going. After the session, users receive a summary of their repetitions and experimental movement metrics, including range of motion, smoothness, and consistency. Saved session history supports comparisons over time.

Camera frames are processed locally in the browser rather than uploaded to a server. The current prototype focuses on squats, and its scores are demonstration metrics rather than clinical assessments.

How we built it

We built the interface with React and JavaScript, using Vite for development and production builds. MediaPipe Pose Landmarker detects body landmarks from the webcam feed, which our movement analysis code uses to estimate joint angles and identify exercise phases.

A state machine follows the transition from standing to squatting and back to standing, allowing us to count completed movements instead of individual frames. We added angle smoothing, tracking checks, and feedback cooldowns to reduce unstable measurements and repeated prompts.

We used the Canvas API to render the fox’s frame-by-frame animation and Recharts to visualize training results. Our modules communicate through a shared event bus, connecting pose tracking, repetition counting, feedback, analytics, and animation.

Challenges we ran into

One major challenge was turning noisy camera observations into dependable movement events. Small landmark fluctuations, changes in camera angle, and temporary tracking loss could affect joint-angle estimates or produce incorrect counts. We worked on this through movement thresholds, state transitions, and explicit handling of interrupted tracking.

Another challenge was making feedback useful without making it distracting. A warning repeated on every frame would quickly become frustrating, so we added smoothing, duration checks, and cooldowns.

Integrating the team’s modules also required clear agreements about event names and data formats. We had to ensure that a completed repetition consistently updated the counter, recorded the right measurements, and triggered the animation.

Accomplishments that we're proud of

We connected webcam tracking, movement analysis, visual feedback, storytelling, and session reporting into one working browser experience.

We are especially proud of the link between exercise and animation: a repetition produces an immediate, visible reward. We also kept pose processing on the user’s device and packaged the model with the application.

What we learned

We learned that detecting a pose is only the beginning. Building an interactive movement application requires deciding when measurements are trustworthy, what counts as a complete repetition, and how to handle missing information.

We also learned that more feedback is not always better. Timing and clarity matter as much as detection. Clear module interfaces and shared test data helped us develop different parts of the project together.

What's next for ReHA

We want to expand beyond squats, improve camera-position guidance, and test the experience with a wider range of users and movement patterns.

We would also like to develop more adventures and work with rehabilitation professionals to evaluate exercise-specific thresholds and feedback. Our next goal is to understand whether the experience helps users stay engaged while producing useful, interpretable movement information.