Inspiration
We have always had the dream about a zero-friction screen sharing experience in social multiplayer VR environments. For example, being able to watch your buddy play a game on Xbox Cloud Gaming in a virtual living room you can join in VR.
Until now, that required setting up a stream from an external PC or other heavy workarounds that add a ton of overhead and setup.
What it does
We added true zero-friction screen sharing to our immersive web browser, Fluid. In any private multiplayer room, a user can click a single button and everyone in the room will see and hear a shared "Smart TV" screen in the room. No additional computer or device is needed.
How we built it
We built an Android View rendering system that has a shared rendering context which allows us to display an android view in multiple users' devices at the same time.
Challenges we ran into
Capturing generic Android views with low enough CPU overhead to maintain frame rate and quality on Quest was very challenging. Most of the early methods we tried cause poor performance and constant FPS drops.
Accomplishments that we're proud of
Making a super simple 1 click system to get users sharing their screens in a quest without any additional hardware required.
This version of this capture system is extremely efficient allowing 1080 30fps 32bit image capture with stereo audio while keeping the quest CPU and GPU at level 3 or lower while simultaneously rendering our 3D environment and multiplayer avatars.
What we learned
We gained a deep understanding of how native views render as well as advanced texture passing techniques between difference process that didn't incur any additional copies.
What's next for Fluid Immersive Screen Sharing
We are looking to make screen sharing features even more powerful in Fluid. We want to give people the ability to share multiple screens at the same time and have multiple people sharing at the same time.
Log in or sign up for Devpost to join the conversation.