Different interfaces offered
The idea was born through the realization that there could be a way to identify and improve our body poses in different environments by looking from an external point of view. It is always easier to see what others do right or wrong, but it's often quite harder with our own body movements. Therefore it would be cool to have a tool that would allow us correct what we are doing in real time! Specifically we thought about archery, since it requires being able to replicate the exact same pose every time to achieve good scores.
What it does
Our program allows the user to choose a position to replicate, for example from yoga, archery or the gym. It gives a model from an expert and records the user in real time recognizing his/her body position as soon as s/he stops moving. This is compared to the inclinations of the right way to perform the chosen position and indicates by voice what movements should be done to improve until it becomes similar enough to the model. As soon as the user achieves that point he can choose an other pose freely.
How we built it
We used OpenCV for the pose recognition and motion detection. Also a cloud service to greatly improve the performance and scalability of our code. The specifics are better kept for the presentation ;)
Challenges we ran into
Besides some technical issues with one of the computers, the main deal was to coordinate all parts and versions when building up the full platform. Installing CUDA was not possible but we managed to solve it by using a Cloud Service, so that we can guarantee that even devices that lack processing power are able to execute the program without performance problems. Also the field of computer vision generally requires a great deal of fine tuning, which wasn't quite so easy to do when we hadn't used most of the software before and the total time to code was only 24h.
Accomplishments that we're proud of
This was an almost completely new field for us, both related to the problem at hand and to the tools used to try and solve it. We had to play to our individual strengths, assigning every member with the tasks he had the highest proficiency with. The team worked hard and together for it so we are happy to say that we could overcome the challenges through the day and create a working prototype of what we had envisioned.
What we learned
It's our first time doing full body recognition, which was different from the (very little) experience we had with face recognition. We have designed our first GUI in python and managed to implement a livestreaming multithreaded system.
What's next for PoseTrainer
As mentioned above, this kind of work requires time to tweak the finer details to make things go the smoothest way possible. That would be one of our primary goals going forward, as well as adding more variety to the models supported and implementing GPU acceleration of our DNNs to reduce the current bottleneck in processing time.