Inspiration

Our group was inspired by the challenge given by Accenture to try solve a problem that everyone accepts. We identified a common issue in physiotherapy: patients often struggle to complete their prescribed rehabilitation programmes due to a lack of motivation and accountability. We wanted to explore how gamification could transform physiotherapy from something that is widely known as repetitive and tedious into more of an engaging experience. Our goal is to ensure patients are completing their exercises to a precise level of accuracy and also encouraggreater consistency. We were also inspired by this year's Durhack theme, "Composition", by combining physiotherapy, motion-tracking technology and interactive gaming to create a better experience.

What it does

The platform uses webcam-based motion tracking to measure joint angles and range of motion in real time, allowing the patients to control the game through their physical movements. As patients perform their exercises, they navigate obstacles, collect coins and compete against their own personal best. Beyond gameplay, Theraplay provides guided exercise demonstrations, target ranges of motion and structured workout sessions. The platform also tracks number of repetitions, accuracy of movement and completed sessions which provides patients and their physiotherapists with detailed feedback on their performance.

How we built it

We built Theraplay using HTML, CSS and JavaScript for the frontend, Node.js and Express for the backend and MongoDB Atlas for data storage. The core of the application uses Google's MediaPipe Pose Landmarker to track body landmarks through a patient's webcam in real time. We used JavaScript algorithms to calculate the angles between these landmarks, allowing the platform to measure joint movement and range of motion. An exercise recognition was also incorporated to identify completed repetitions and translate physical movements into controls for the game. We used MongoDB to store patient information, prescribed exercises and session data (reps, joint angles, movement duration and range of motion). Out Express.js backend processes data to generate summaries of performance which are displayed through interactive charts on the physiotherapist dashboard.

Challenges we ran into

The main challenge we faced was integrating everyone's code through GitHub since we encountered merging issues during development. Additionally, working collaboratively under time pressure meant we had to ensure resolving merge conflicts and ensure that implementing new features didn't break existing functionality. We also had to ensure as greatest accuracy as we could with motion tracking by trying to stop small fluctuations in detected joint positions affecting the platform's ability in distinguishing intentional movements .

Accomplishments that we're proud of

We're proud of our creativity when first coming up with a common issue that people face and finding a fun way to try and solve it. We feel we we're imaginative and flexible when developing it, combining every member of the team's ideas into a cohesive project that everyone contributed to - managing merge conflicts from multiple branches well, taking everyone's best ideas and work and combining it into one product.

What we learned

We learned how to effectively all collaborate on the same code without overwriting other people's work Using separate branches and reviewing all merges to make sure that no one's work was disregarded or lost.

What's next for Theraplay

We would develop more exercises and the recognition of lower body, currently it's more designed towards sitting upper body exercises as that's easiest to capture in the camera because you can be closer to the screen. Potentially having an external camera that you can link to your laptop would have a wider lens and higher detail, making the movement recognition more accurate.

Built With

Share this project:

Updates

Submission history