GymLens

• 

  A team member demonstrating technique that needs to be adjusted.

• 

  A team member demonstrating proper technique.

Inspiration

The pandemic has affected the health of billions worldwide, and not just through COVID-19. Studies have shown a worrying decrease in physical activity due to quarantining and the closure of clubs, sports, and gyms. Not only does this discourage an active lifestyle, but it can also lead to serious injuries from working out alone at home. Without a gym partner or professional trainer to help spot and correct errors in movements, one can continue to perform inefficient and often damaging exercises without even being aware themselves.

What it does

Our solution to this problem is GymLens, a virtual gym trainer that allows anyone to workout at home with personal rep tracking and correct posture guidance. During the development of our Minimum Viable Product, we implemented a pose tracker using TensorFlow to track the movement of the person’s key body points. The posture during exercises such as pushups can then be evaluated by processing the data points through a linear regression algorithm. Based on the severity of the posture, a hint is provided on the screen to re-correct the posture.

How we built it

We used a Tensorflow MoveNet model to detect the positions of body parts. These positions were used as inputs for our machine learning algorithm, which we trained to identify specific stances. Using this, we were able to identify repetitions between each pose.

Challenges we ran into

From the beginning, our team had to navigate the code editor of Sublime Text and Floobits, which proved to be more difficult than we imagined since members could not log in and sign in to Github. Our front end members who were coding with HTML and CSS ran into problems with margins and paddings with divs and buttons. Aligning and making sure the elements were proportional caused a lot of frustration, but with moral support and many external sources, we were able to get a sleek website with which we could host our project.

Incompatibilities between machine learning tools and the library used for pose detection were a major hurdle. We were able to solve this issue by using our own custom-coded machine learning library with a simple feed forward neural network.

Lastly, our struggles with Floobits ended up being one of our biggest setbacks. It turned out, our entire team soon realized, that when two people were on the same file, the lines of code would severely glitch out, causing uncontrollable chaos when typing. Due to the separate nature of front end and back end programmers, it was inevitable that members would step on each others’ toes on the same file and accidentally undo, delete, or add too many characters into one line of code. We ended up having to code cautiously in the fear of deleting valuable code, but we had many laughs over the numerous errors that transpired due to this glitch. Furthermore, Floobits’ ability to overwrite code turned out to be an asset and liability. Although we were able to work on the same files in real-time, destruction from one member of the team turned out to be collateral. On the last evening of the hackathon, one of our team members accidentally overwrote the remote files that the rest of the team had worked hours on instead of the local ones. In a frantic effort to get our code back, our group tried to press ctrl-z to get to the point where the deletion occurred, but it was too late. Unfortunately, there was nothing we could do to get about 3 hours of work back. Luckily, with our excellent team morale, we separated into groups to repair what had been lost. However, our problems with this code editor did not end here. Nearing the end of the hackathon, our front end and back end duos came together triumphantly as we presented our accomplishments to each other. This final step turned out to be an unsuspecting hurdle once more. As the back end merged their final product into the website, many errors with Github pushes and the integration with Floobits became apparent. Progress had not been saved from branch to branch, and the front end code ended up being set back another 2 hours. Having dealt with this problem before, our team put our heads down, pushed away the frustration of restarting, and began to mend the lost progress once more.

Accomplishments that we're proud of

One significant milestone within our project was the successful alignment of the canvas-drawn posture overlay with the body of a user. Its occurrence brought the team to a video call, where we offered congratulations while hiding our faces behind our freshly made overlay. Its successful tracking later became the main highlight of multiple demonstrations of exercises and jokes surrounding bad posture.

The front end development team enjoyed the challenge of coding in unfamiliar territory. Their encounters with unfamiliar functions as well as their first attempts at Javascript to create a stopwatch with working buttons all resulted in greater feelings of pride as they were incorporated into the site. They are proud to have designed a self-reassured, visually-appealing website by applying the knowledge they’ve gained over the last 36 hours.

Personally, we're proud to have stayed awake for 21 hours.

What we learned

This hackathon became a giant learning experience for all of our members, despite the range of coding abilities and experience. Our more experienced back end members tackled Tensorflow’s MoveNet pose detection library for the first time. Meanwhile, the members responsible for the user interface, design, and website navigated Floobits and Sublime text as beginners. Our understanding of the different css functions greatly increased, most notably with layering and element positioning with inspect element. Additionally, members ventured into the world of Javascript for the first time, and realized its handiness with HTML and CSS.

Overall, our team surprised ourselves with our mental fortitude when numerous obstacles were thrown our way, and our ability to learn different functions, languages, software, and platforms so quickly. We worked cohesively and efficiently, all the while discovering the true extent of our capabilities under time pressure and during late nights calls.

What's next for GymLens

In the future, we hope to expand the functionality of our project by making greater use of its movement correction and movement recognition capabilities. We plan to offer a greater library of exercises sourced from online trainers’ videos used as training data. This extra variety will allow users to take total command of their workout regimes, all while ensuring the safety and efficacy of all exercises.

Built With

• css
• html
• javascript
• movenet
• node.js
• ola.js
• tensorflow.js