UpER G

An Upper Extremity Rehabilitation system frees nurses from repetitive rehabilitation work and gives patients a low-price recreational rehabilitation environment.

Comment 2

Inspiration

Physical therapy is a promising avenue for helping relieve pain and restore bodily functions for people who are injured or suffer from debilitating conditions, such as stroke. According to the U.S. Census Bureau, 91 percent of nurses are female. However, most of them had a large workload while taking care of rehabilitation patients and worked under harsh condition. As most of the rehabilitation jobs are repetitive, my team is trying to free these female labors from repetitive jobs that could be done by machines. To help solve this problem, my team developed an Upper Extremity Rehabilitation system that uses a game to guide patients, who needs wrist rehabilitation, to perform rehabilitation by themselves in a recreational way and report rehabilitation message back to doctors. By meeting the increasing demand of patients’ rehabilitation, we provide a possible solution for the next generation of nurses a more comfortable and enriched working environment. Moreover, our device provides patients with low income an economically competitive method to do rehabilitation, which supervised by professional medical staff remotely, at home without physically going to the hospital. This will bring the local community together as more and more patients can get access to scare medical resources. Also, with the COVID2019 outbreak, our Upper Extremity Rehabilitation system significantly decrease physical contact between patients and medical staff, which largely eliminates infection rate.

What it does

It uses Leap Motion to detect the wrist gesture of patients. Patients can move their wrist gesture to play the pygame, which will guide the patients to play the space shooter game to make patients move their wrist in an entertaining way. The patient can use different gesture to pause or end the game proactively. After each game, the medical stuff will get a text message that contains the latest patients' rehabilitation training data and his/her past data to help doctors better diagnose patients' situation.

How we built it

Our Upper Extremity Rehabilitation system is divided into four parts: Pygame, Leap Motion, Twilio, and Azure SQL database. First, we use pygame to create a spaceship shooter game that will guide the patients to move their hands according to where meteorites are coming from. We program the meteorites that will come from relatively same direction to ensure patients can follow along with the game. Then we learn the Leap Motion documentation and use it to sense patients’ hand movement so that patients can play the spaceship shooter game through rehabilitation process. After patients' complete each treatment, we use Twilio to send a SMS message to the doctor that contains the latest patients' rehabilitation training data and his/her past data. We use Azure to create the database that tracks patients’ past rehabilitation situation. Finally, the Twilio will send a SMS message report with these data back to medical staff.

Challenges we ran into

During our hacking, the one of the most tricky problems we met is the communication between our Pygame and Leap Motion platform. Our teammates develop the two parts of our system separately; however, we use different python version- python 2.7.3 for Leap Motion and python 3.7.2 for PyGame. As a result, during communication, we have to either downcast the game python files to Python 2 and fix all the related conflicts or find a way to recompile the LeapPython.so that is only supported till 2.7.3 on the official site. However, no existing online source has done the communication before. After hundreds of experiments, we finally decide to risk it -- changing its backbone 1000-line C++ code and recompile it through swig. Luckily, we succeeded, and our team all raised a furor for the resolution. We also encountered a "502 error" while Twilio is trying to receiving from the using. We searched many websites tried to solve it, which turns out to be that we should run the server and the python program at the same time. We also meet a brainstorming problem at the beginning of our hackathon. We came up with three disparate ideas: Auto Upper Extremity Rehabilitation, Female Sport Community, and Female Safety Protector, but we have to weigh the practicableness and novelty of each idea in a limited time. We were really anxious at the beginning, but as long as our team reach an agreement, we all started to enjoy the hackathon.

Accomplishments that we're proud of

Each of our team member learns to program a completely new platform within 23 hours: leap motion, Twilio, Azure. We can utilize a programming language that we never encountered before to complete the system. Our Upper Extremity Rehabilitation system incorporates different platforms: Pygame, Leap Motion, Twilio, and Azure SQL database. We researched professional medical documents to ensure that our pygame provides professional and proper medical training guidance for patients. Our Azure SQL database provides a whole data history of the patients past medical situation, which gives doctors a visualized sense of the patients' situation, and promises better rehabilitation surveillance and treatment.

What we learned

Throughout the developing process, our team not only learned to use python to develop a game, but also learned the API of cutting age hardware Leap Motion, cloud communications platform Twilio, and Azure SQL database provided by Athenahack sponsors. Teamworking is also an indispensable part throughout the whole Hackathon. We realize the importance of helping each other when one of our teammates has some programming problems. AthenaHack also brings a unique experience of females helping females, which makes our team realize the power of female engineers and would like to continue the influence in the society. As a beginner of hacker and programmer, I learned to use and understand basic programming tools like GitHub and learn API of different platform by myself as well as apply them into a product, which is a very exciting process.

What's next for Upper Extremity Rehabilitation

Medical staff can also incorporate our pygame to not only help wrist rehabilitation, but also arm, shoulder, leg, finger rehabilitation with different size and accurate types of leap motion. We can also use the Twilio to incorporate an AI chatbot that sends SMS messages that may answer some basic medical questions for patients. As the COVID2019 is wide spreading, it may help relief the scarce medical human resource in hospitals. The remote control without the need of physical contact ensures self-quarantine at home while getting treatment at the same time. Our system can incorporate other games to provide different rehabilitation treatments or just simply converts in a more personal involved game.

Share this project:

Updates