Kinai

logo
home screen
how to perform exercise
how to perform dropdown
pose estimation + live tracking
summary + insights/suggestions

Inspiration

Ever since COVID, alot of physical therapy and training has shifted online due to convenience and outreach. Physical therapists can help out tens to hundreds of clients a day by just responding to a video message on their phone.

That's the problem though. As an athlete myself, I went through this virtual physical therapy and virtual training many times. My coach would tell us what to do, we would have to record a video of us performing that exercise, send it to him, and it would often take hours or days for him to get back because of the amount of other people doing the exact same thing.

This is why I developed Kinai

What it does

Kinai uses CV and LLMs to analyze a user's movement and give them feedback on their technique. The way it does this is by comparing specific joint angles for specific movements to that of an experts/coach. It then rates how well the user did the movement relative to the coach, highlights specific angle measurements if needed, and provides suggestions on how the user can improve their technique.

So no more having to wait hours and days just to hear that you need to bend your knees more when you squat.

How I built it

Backend in python, frontend with react, javascript, HTML/CSS pose estimation with media pipe and openCV, AI insights with openai API

Challenges we ran into

Lots

Motivation: this being my first solo hackathon, it was alot tougher than I thought it would be
Prompt Engineering: Figuring out how to prompt the AI to take in all this data and make insights on it was something that took many hours, and it still isn't perfect even now
Finding the right angles for each movement: Because this is 2d pose estimation, its hard to get depth perception and joint calculations for certain things like pushups.
Frontend development: Not a fan of frontend development so this took alot of work