Background subtraction and limb tracking
Body part tracking

Wallaby

What it Does

Wallaby uses machine vision to automate the analysis of physical therapy patient's exercises to make it easier and more efficient to get insight on how they are improving.

How we built it

We used openCV with Python to create a program that can analyze and track multiple body parts and then evaluate its own trackers as it goes to prevent mistakes. Using the trackers it detects how well the person is doing specific exercises and uploads them to a database. The data is sent to a server we hosted on AWS, and all the data is stored in JSON format. The website was coded with bootstrap and HTML, along with some javascript to display graphs and add animations, and to retrieve data from the database.

Accomplishments we're proud of

Making a product that has the potential to make people more mobile faster and to lower costs, making physical therapy more accessible to lower income patients
Got angle detection working for body parts
Got tracking working with openCV -Making a website that has 5 separate pages that are responsive using bootstrap -Processing data with JSON in order to display information for the patient to see -Used an AWS server to get data from the machine vision part to be displayed using canvasJS
Automated detection and tracking of body parts
Uses adaptive algorithm to verify correctness of software processing and detection

Challenges we ran into

One of the major challenges we ran into was training the computer to accurately and consistently locate certain parts of the body. One possible cause for this was to the inaccurate data sets we had and the laptops built-in camera we were using. A lot of times, we had to learn a brand new topic in order to continue with the project. For example, both machine vision and bootstrap were brand new to us, so we had to spend extra time learning that. Besides that, making a good looking line graph/bar graph out of a given database was difficult for us since we had to do it in html. Overall, what was hard was that we "hack" and learn at the same time.