Each year in the US, around 2 million people go to their doctors because of rotator cuff issues. 600,000 have surgery for rotator cuff tears or tendonitis. Besides the volume of injuries, the rotator cuff and shoulder girdle are one of the key body areas in determining posture and shoulder health.

One of the most common remedies, and best prevention methods, is performing corrective exercises. Although rotator cuff exercises can be very effective and simple to perform when done right, there are multiple areas where patients can perform the exercise incorrectly, which can actually worsen their issues. This is where Asclepius comes in.

What it does

Our Android app Asclepius is geared towards physical therapy patients performing corrective exercises. It serves to monitor and guide their form on key exercises, providing real time feedback via 2 features: an accelerometer that ensures the patient maintains a safe and stable speed of motion and a camera-based tracker that visually examines the user’s form and guides them toward a correct range of motion.

The app does not require any expensive features or programs, operating off the hardware present in every smartphone and any wearable object provided by the user.

The camera tracker implements openCV, a computer vision library, to monitor the patient’s form. To activate it, the user has the camera lock on to a wearable object, such as a neon band. The camera then uses the object’s movement during the exercise to determine the patient’s range of motion and form. Depending on the exercise being performed by the user. Asclepius can tell the user to adjust their movement via a green outline for correct movement and a red outline incorrect movement. Asclepius tracks for a blue band by default, but a calibrator is provided that allows the user to adjust the camera to the object they have available by changing the target color.

The second method for real time feedback, the accelerometer, works by tracking the acceleration of the user's movement, from which we know whether the user is maintaining a constant velocity. We want users to maintain a constant velocity to increase stability and reduce the chance of injury, and Asclepius can tell them to adjust when needed via a red screen for "adjustment needed" and a green screen for "good movement."

Our final feature is a tutorial section that provides instructions and video demonstrations of each exercise and the features of the app.

While Asclepius is meant to aid the recovery of injured PT patients, it can also be used in casual exercise for injury prevention, which is another angle with massive potential. The ultimate aim of the app is to help users perform various exercises safely and with proper form without the need for hiring personal trainers or other expensive options. We hope that in making a free, effective tool in Asclepius, people can live happier and more productive lives.

How we built it

Our main tools for making this application was Android Studio, Google's integrated development environment, which uses Java to program the application and XML to design the application. We also added the openCV module to utilize computer vision in our app and Firebase as our backend.

Challenges we ran into

One of the most difficult aspects was effectively implementing computer vision. We had to figure out how to implement openCV, a computer vision library with unique methods we had little experience with. Additionally, we wanted to add the capability for real time feedback to the user, which also brought the issue of not just tracking results, but communicating them in a way the user could understand. And of course, the whole structure would have to be simple enough to use and understand that new users would be able to intuitively take full advantage of the app’s features. Eventually, we designed a relatively simple user interface, with the user tapping on the screen to signal to the camera tracker to activate and lock on to the provided object. Once the screen is tapped, Asclepius creates a "box" at the point where the object is located. When the user completes a rep, the user's band should return to about the same spot where it was when the screen is tapped, so Asclepius then knows that the user has completed a rep. Firebase also provided a technical challenge in transferring data on the phone to the cloud. We required a certain organization of all the data, so the phone could then translate the data back into something useful for visualization.

Accomplishments that we're proud of

We're proud to have been able to apply our skills in openCV to a more effective and meaningful manner in our own everyday lives, while still keeping our user interface simple enough to easily understand and operate. We also feel accomplished successfully incorporating Firebase into an app for the first time, despite the difficulties it presented. Asclepius, originally made a little less than a year ago as an entry for the Congressional App Challenge, already gave us a sense of accomplishment, as it won in our respective congressional district and earned us the opportunity to both meet our congressman and travel to Washington DC to present our app. Since then we have still continued to develop and improve the app to allow to be of greater help to more users, and truly feel the greatest sense of accomplishment helping people in our community stay healthy and avoid further injury.

What we learned

Creating Asclepius gave us the opportunity to learn more about how to use openCV in an everyday environment, and design an app with a simple UI centered around it, since we had initially only ever used it in the FIRST Robotics Competition to detect light reflecting from reflective tape. We also learned about how to use Firebase as a backend and the process of transferring and organizing data. Beyond the technical aspects of the app, we learned more about treating injury and the importance of performing corrective exercises correctly, further stressing the importance of an app like Asclepius.

What's next for Asclepius Physical Therapy Assistant

One important improvement we want to add is to use machine learning in conjunction with computer vision to track a user's arm movement because as of right now, Asclepius requires users to wear a band or other wearable object that it can track. Tracking arm movement instead of a band would be more convenient for users.

In addition, we think that we could provide a much better way of counting the user's reps because it currently relies on a timed period after the user has completed a rep. Once the time period ends, Asclepius can then begin counting reps, and this may not work out in real life scenarios where the user is performing reps at a faster pace than Asclepius can count. Incorporating machine learning to recognize arm movement would surely help in counting reps.

In regards to Firebase functionality, we would like to provide a home page where the user can track progress over time. By tracking progress over time, Firebase could give data for Asclepius to determine which exercise the user should complete in the short term to avoid injuries.

Although our app currently focuses on exercises designed to strengthen the rotator cuff and shoulder girdle, areas of the body where injuries are common in various sports and workout regimes, we could expand to cover other types of activities and exercises, widening the audience we can help.

Share this project: