LimbVR

VR-based physiotherapy module aiming to improve injury recovery process

Comment

Inspiration

After an injury, people are often traumatized into avoiding the same movements that led to such outcome. How to make sure they recover healthily and efficiently? LimbVR proposes a new, VR-based method of physical therapy, by combining Oculus Rift, Myo bands, and state-of-the-art software to recreate that person’s movements in VR and compare it to the ideal motion. By helping the person see themselves doing the movements on VR and juxtaposing that against the ideal motion, we can provide quantitative measurements on how far people are in their recovery, keep track of their progress and provide a tool for them to learn the physical motion.

What it does

Buckle up and get ready for action. Once you put the headset on, you are immersed into your healthy, ideal body. Wary of moving in some way that might get you injured again? LimbVR shows you how to perfect your moves and build trust in yourself once again. Using the Myo band, we recreate the movements of your arm and compare it to the ideal motion in VR and provide quantitative measurements how close to the ideal motion you currently are. This can vastly speed up physical therapy and injury recovery.

How we built it

Oculus Rift. Myo. Unity. The three structural pillars of our product are integrated seamlessly, capturing the best that each element has to offer. We designed a virtual reality atmosphere where the user can feel as comfortable as if he/she were in a clinic with a therapist. Unity was our main development platform, where we outlined the structure of the environment, designed the main menu and integrated the animations of the ideal motion. Myo’s data is streamed via JavaScript bindings, parsed and processed by three.js, vectorizes human arm movement in real time and logs that data into our database. This data will then be compared to the movement depicted in Oculus, as seen by the user, to provide useful feedback. In a further stage, Myo would also be used to program Oculus’ animations.

Tools: in Unity, we used mainly C# scripts to control the animations and design the user interface. With Myo, we used Myo's JavaScript API and three.js for calculation and visualization of vectors. For the animations on Oculus, we used Python in Blender to dissect and plot every move and have a Python Flask back-end.

Streamcast of Unity + Oculus: https://youtu.be/I8EC6BgXfhE

Challenges we ran into

  • Myo vectorization--turning the raw Myo data into realistic vectorized arm movement.
  • Integrating Unity with the Oculus SDK and lacking documentation
  • Find/develop adequate animations to display through the Oculus Rift
  • Calibrating both Myo sensors to provide proper orientation and joint-movement

Accomplishments that we're proud of

  • Getting the hardware to cooperate with the software!
  • Using two Myo bands simulateneously, combining forearm and bicep movement into a cohesive swing.
  • Taking raw data from accelerometers and gyroscopes and turning it into a realistic human arm
  • Integrating ideal motions into Oculus Rift and Unity

What we learned

  • How to integrate a variety of software and hardware into a cohesive package
  • The vast potential impact that VR and generated movement can have towards personal treatment

What's next for LimbVR

  • Superimposing the vectorized Myo vectors into Oculus Rift animations which can be visually compared against the ideal motion
  • Building out a time-series metric to calculate the difference between the ideal and current motions
  • Personalized dashboard featuring user movement analytics
Share this project:

Updates