Physio Assist

Mediapipe overlay on our website

Inspiration

We have all gone through physical therapy, and noticed that it was hard to make sure our form was correct when we were practicing the physical therapy exercises at home, after being with at the PT's office.

What it does

Our web app tracks user movement through mediapipe, and calculates the angles made by all of their joints. Depending on the exercise, our code will analyze the movement, and give feedback on improving form, as well as insight into the positive gain from the movement.

How we built it

We explored pose detection models and determined that mediapipe was the best for our idea. We then spent time figuring out how to use mediapipe within a next.js environment, and used typescript to handle the majority of the product function. After the data is collected, it is passed into a python file which does calculations on the data, and then sends it to an OpenAI API to construct the feedback paragraph and graphs.

Challenges we ran into

Some challenges we ran into were differences in operating systems that made working on the project together difficult, as windows would often not be able to run the code correctly if it was written on a mac

Accomplishments that we're proud of

We built an entire app in Next.js even though we had very minimal previous exposure with javascript/typescript frameworks.

What we learned

We learned a significant amount about developing in Next.js, as well as connecting with the OpenAI API. We also learned about many of the existing problems in the physical therapy space, and how deep tech can work to solve these issues.