YogaVision

a screenshot during testing

Inspiration

We wanted to create a project that involved computer vision and cameras. After some brainstorming, we've settled with the idea of combining computer vision with yoga and exercises, where we can track if the user is performing the yoga/exercise with accuracy through a camera. Our goal is to help users perform routines with greater accuracy and confidence by providing real-time feedback on their posture and alignment.

What it does

YogaVision is a web-based application that uses computer vision to guide users to perform yoga poses and exercises with greater accuracy. Users begin by selecting a specific yoga or exercise routine and setting the desired duration. Once selected, the app prompts the user to enable their camera, stand in front of it, and to put their arms extended horizontally for calibration. Once the session starts, the webcam will track the user's movements in real time. The user's pose will be compared with prerecorded references using computer vision. how does this sound

How we built it

We created an Ubuntu web server through digital ocean. We used NextJs for our front end, and python for our image processing and back end. To track poses we used MediaPipe a framework for machine learning models made by Google.

Challenges we ran into

Some of our biggest challenges came from cameras and integration between nextjs and python. We had wanted to use a raspberry Pi to run the camera, which would process the image and sent the data to the web server. However, there was no overlapping python version that we could find that the picamera2 library and mediapipe could both be on. We then tried to use the computer webcam. However, most browsers will not allow camera access on sites using HTTP. We tried for a while to get HTTPS to work but abandoned it for the sake of time. Instead we found a workaround in chrome to allow camera access on specific HTTP pages. Additionally, we had a while of trying to configure network ports, since some services kept trying to use the same port

Accomplishments that we're proud of

We are proud to have built a mostly* working web app. We are happy to have been use technologies we have been interested in learning about in our project.

What we learned

Through this project, we got to learn about computer vision, nextjs, multiprocessing in python, using web-sockets to transfer data between our python back-end and the nextjs front end. None of have really done much in SQL so that was also a great learning opportunity to use a mariaDB instance for our database.