Motivation
Traditional rehabilitation often lacks engagement, and many patients struggle to stay motivated—this project emerged from a genuine desire to change that reality. We saw how difficult it can be for patients to remain committed to a long and demanding recovery process, and we felt compelled to create a solution that could make therapy not only effective, but meaningful and uplifting. Our goal was to design an experience that helps patients reconnect with their progress, regain confidence, and feel supported every step of the way.
To achieve this, we developed two complementary modalities. The first is a therapy mode, where the patient connects via BLE to an existing exoskeleton to perform guided rehabilitation exercises. This modality includes a tutorial based on object detection to help the patient correctly position the exoskeleton on their arm. The second modality is a game mode, where the patient completes recipe-based tasks inside a virtual kitchen, transforming therapy into an engaging and motivating activity.
Building this system required integrating several key technologies. We implemented gesture detection for the game mode to ensure that therapeutic movements were performed correctly, creating a direct link between physical action and in-game progress. We also developed a BLE communication system capable of maintaining a stable, real-time connection between the Meta Quest headset and the physical exoskeleton. Additionally, we incorporated object detection for the setup tutorial, allowing the system to recognize the exoskeleton in the patient’s environment and guide them step by step.
We integrated a real-time digital twin of the exoskeleton, designed to mirror the exact movements of the device throughout the session. This synchronized model provides immediate visual feedback, reinforces correct motor execution, and strengthens the connection between physical rehabilitation and virtual interaction. Together, these elements allowed us to build an intuitive experience that helps patients stay focused on their recovery.
The development process brought several challenges. Accessing the Meta Quest cameras and integrating the Sentis model into Unity required creating and adapting our own solutions. Building the digital twin demanded a deep understanding of Unity’s joint systems to accurately reflect the exoskeleton’s kinematics. Designing a reliable BLE library also required extensive testing, while gesture detection for the game mode forced us to develop different strategies for each movement.
These challenges reminded us why this project mattered. Each obstacle pushed us to learn, refine, and design with purpose. Looking ahead, the system can continue evolving by improving gesture detection, optimizing the object-detection model and dataset, enabling cloud-based patient records, and adapting the exoskeleton for patient use. Ultimately, our motivation comes from the belief that technology can humanize rehabilitation—helping patients feel motivated, supported, and empowered on their path to recovery.
More than a technical achievement, this project became a reminder of what innovation can represent for someone striving to regain independence. Every line of code, every model trained, and every obstacle overcome brought us closer to creating a tool that could genuinely impact a person’s life. Our hope is that this work continues to grow, inspire new ideas, and open doors for patients who deserve therapies that are not only effective, but dignifying and filled with possibility.
Built With
- android-studio
- blender
- c#
- exoskeleton
- image-proccesing
- psoc-creator
- python
- unity
- yolov11
Log in or sign up for Devpost to join the conversation.