Cardboard Spaces is a virtual reality telepresence experience.
What it does
Users can broadcast themselves, as avatars to any Google Cardboard. It is all web-based which means it needs no expensive hardware or lengthy setup.
How I built it
It was built by retrofitting an existing WebRTC messaging app to support the Posenet implementation by Tensorflow.js. The server sends the positions of the user, and the client picks it up, applies some filters, and displays the avatar.
Challenges I ran into
My laptop has no GPU, so running neural networks in the browser is rather slow. It was difficult to fill in the missing information in a meaningful way.
Accomplishments that I'm proud of
What I learned
I learned a bit better how WebRTC works.
What's next for Cardboard Spaces
Multiple users in one call!
Tensorflow.js: https://github.com/tensorflow/tfjs-models/tree/master/posenet WebRTC sample: https://webrtchacks.com/datachannel-multiplayer-game/