Axxess Kids Joint

Inspiration

Almost 11.6 million kids in the United States are diagnosed with a developmental disorder, such as Autism, Down Syndrome, or Cerebral Palsy not to mention physical disorders such as scoliosis or osteoporosis (centers for disease control and prevention). While disorders such as Autism and Scoliosis are wildly different, kids who are diagnosed with them have one thing in common, the need to attend physical therapy. Physical therapy includes performing exercises, stretches and motions that contribute to increasing muscle growth, mobility and strength and while these exercises may seem easy for an adult, it’s often difficult to encourage kids to remain consistent with their exercises every day. Furthermore, it is often disorienting for children to receive hands-on and comprehensive physical therapy treatment in an unknown environment such as a clinic. We created Kids Joint in order to provide a gamified version of physical therapy that encourages and excites kids about following their therapy routine, while also allowing them to remain comfortable at home, creating an overall positive environment around doing physical therapy.

What it does

Kids Joint is an interactive site that guides kids through key movements utilized in physical therapy in an engaging and entertaining form. Kids Joint utilizes computer vision techniques to scale pose landmark body detection to guide patients through exercises as well as to monitor their progress. Using the ML model and classification algorithms, the computer utilizes data visualization to analyze the joint pose landmark patterns to depict the yoga poses. Exercises are presented in a game-like fashion, where executing movements correctly earns points. The system recognizes the specific movement and provides feedback to the patient while guiding them through the exercise set.

How we built it

We implemented OpenCV and Google MediaPipe libraries using built-in body pose models for pose landmark estimation. Through data visualization, the program tracks the various body joints to identify yoga poses ensuring the users are achieving accurate pose alignment. We achieved real-time detection through OpenCV that captures the user through webcam live stream. In addition, we incorporated a gamification of the model using player scores based on their accuracy and modules ranging from various difficulties. The program is built mainly using Python using Jupyter Notebook that emphasizes unit testing and integration testing blending in pose detection algorithms and player score counting logic.

Challenges we ran into

It was our first time utilizing AI/ML using the ML models to track the pose estimation so identifying the exact joints and parameters to track the specific yoga pose was tricky. Furthermore, we had never before designed or coded a webpage, so it was challenging to learn and execute the software and processes of creating the front-end.

Accomplishments that we're proud of

We are extremely proud of our ability to create a successful AI that is able to detect motions and provide feedback. Furthermore, we are proud to have attempted things that we have never done before and emerge with a successful project.

What we learned

We learned how to implement trained models for computer vision tasks and pose estimation. It was also our first time coding using Jupyter Notebook so it was interesting to see the data visualization and analysis through the application.

What's next for Axxess Kids Joint

In the future, we would like to add more features to further connect patients utilizing Kids Joint with their physicians, such as a record option that will allow a physician to see the patient complete a movement and provide feedback. Moreover, we would like to further gamify Kids Joint by adding a point system as well as comprehensive modules with progressive difficulty levels. Lastly, we aim to add more complex poses and movements that are more interactive and engaging and that would widen the scope of exercise sets available.