PoseCore

My idea is to enable people with limited possibility/desire to go out for activities to play games where you actively have to move your body with equipment most people already have at home (computer)

Comment 1

Inspiration

A lot of people around the world are now in quarantine or unwilling to spend to much time outside and have a limited possibility to exercise or do activities with friends that are active in any way. This will have a negative impact of peoples fitnesse and health.

I really liked the Kinect/Wii capabilites where you used you body as the controller for gaming, this is a way to enable this again but without any new hardware requirements.

What it does

This solution uses a normal computer & web camera (or iPhone in step 2) as a "Kinect-like" device that interprets your body movements and sends these to another application (game, meeting app...) where the position of your body parts can be used to visualize your movements or use them in a game to control your avatar.

The PoseCore server application will send the body part locations in a standardized format that any game/app developer can use to do games/apps that uses body tracking for

How I built it

I built a server application that contains a communication platform through which one or more body tracker clients can communicate with one or more client applications using the body joint information (to control an avatar or other entities in a game/application).

I then created a library in the Unity game engine platform that connects to this platform & uses this body information to control the wanted game elements.

Challenges I ran into

Simplifying the connection between clients & the server, which was solved by a UDP broadcasting mechanism allowing clients to automatically find the PoseCore server.

The complexities of controlling an avatar with body joint positions where you don't have a clear floor level.

Accomplishments that I'm proud of

What I learned

What's next for PoseCore

  • Newer iPhone models has built-in support for body tracking, so creating a new client using this API and then send the body info to the Posecore server as another input channel.
  • Create server executables for Mac
  • Enable multiple persons using the same server in a multiplayer scenario
  • Publish the server application & API specifications for others to make games/apps using the same body-tracking framework
  • Make some fun games and applications for people to use:-)
  • Polish the GUI on the server module.
  • Enable the PoseCore server to handle certificates so that we can have separate web clients connecting to the same server.

Built With

Share this project:

Updates