COVID-19 has hit all of us very hard. There's no denying it. Malls have been shut, shops shuttered, gyms locked up - and what's even worse? We have to self quarantine and stay in isolation, to try and curb the spread. This gives birth to 2 major complications, the mental health of individuals, and their physical health. 4 in 10 adults in the U.S. have reported symptoms of anxiety ¹. And sitting at home, with no access to expensive gym equipment, 85% of study participants reported that they did not engage in any moderate or strenuous exercise at all ².
Thus, we narrowed our topic down to one of the most versatile of exercises -
Yoga helps reduce stress, anxiety and depression, and helps energize and keep the person physically fit ³. We wanted to do our part to try and help the people of this world during such trying times.
What it does
The app, via the video camera feed, analyses the person's yoga pose. The user selects a pose, reads the rules, and proceeds to start the timer. During this 15 second countdown, the person's pose is analysed at each frame, the necessary angles for that pose are computed, and checked against the angles stored in the database. Based on how close the angles are (° values), a score is assigned to each angle. The average is then computed and used as a reference. This is the score that is reflected in the leaderboard as well. The app can be used by beginners to try and understand how to perform certain poses, while more seasoned individuals can use it for perfecting their pose.
How we built it
We used react-posenet to identify the keypoints from the pose. These points are then passed throught the functions we wrote, in order to generate angles about 8 different, crucial joints.
A score is then determined by checking how much the user's angle about a joint deviates from that of the professional (test images from professionals were used to make these). If the deviation is more than a certain threshold, then a score of
0 is automatically awarded. Scores are saved in Firestore, and can be retrived in the form of a leaderboard and a graph of the user's history.
The frontend has been coded in
ReactJS, and the backend uses
Challenges we ran into
Calculating angles from each pose, and deciding WHICH angles to calculate was very difficult, since not every pose returned the same set of keypoints. The mountain yoga pose returned 17 points, whereas the cow pose only returned 6 points. Thus, we had to take into account the variance in frontal and side-profile poses. For the longest time, we had a cyclic dependency in our Firebase system, which took a lot of effort on Manak's part to rectify. Deciding on a scoring system was also a paramount task.
Accomplishments that we're proud of
We're proud of the color schema we used, since it's said that shades of purple are best associated with yoga ⁴, and the fact that we've incorporated blissful music as well into the app. We are also extremely proud of the UI we managed to create in less than 24hrs, and also the functions to compute the angles.
What we learned
We improved our frontend and backend skills (I re-learnt React and node.js). We learnt how to use
Firebase (and Firestore),
Firebase Hosting, website deployment and about SSL certifications. Colors have a big impact on not only the aesthetics of the app, but also on the psychology of the user. Thus, we've used soothing colors that help promote a sense of calmness and spirituality. Positivity inducing music has also been incorporated via the Spotify API. Finally, we learnt the importance of writing clean code.
What's next for Yogasaan
We hope to clean up the UI, and build dedicated mobile apps for the same. We hope to improve the performance of the pose detector, perhaps by GPU acceleration on the devices, since now even mobiles have GPUs with decent computing power. We don't intend to monetize the app, since it's solving problems that directly affect millions of people every day.
¹ https://www.kff.org/coronavirus-covid-19/issue-brief/the-implications-of-covid-19-for-mental-health-and-substance-use/ ² https://www.physio-pedia.com/Physical_Activity_and_COVID-19 ³ https://www.healthline.com/health/depression/yoga-therapy ⁴ https://studiogrowth.com/what-color-to-paint-a-yoga-studio/