LaptopVR

Stereoscopic view on iPhone
Waiting for device to connect
After device connection successful

Inspiration

For the last few years, I’ve had back problems, most likely caused by poor posture when using the computer.

A few weeks ago, I had a back spasm, and for a few days, I was unable to move most of my body without sharp pain in my back. Using the computer was painful, but I still needed to get work done, especially during midterm season. Both sitting in a chair and slouching in bed cause me to crane my neck, which is not good for my posture.

As I was lying there in pain, I wished I had a way to just have the computer screen right above my face, while keeping the keyboard on my lap, so I could keep my spine fully straight when using the computer. I knew I could do this with a VR headset like the Oculus Rift, but those are expensive — around $400.

But what if I could use my phone as the VR headset? Since I already own it, the only cost is the VR headset holder. Google Cardboard costs around $5, and a more comfortable one that I purchased off Amazon was only $20. Simply slide the phone in, and you’re ready to display content.

I tried searching for an app that would allow me to mirror my screen in a stereoscopic view on my phone, but I couldn’t find one in existence. So I made one!

What it does

First, launch the app on your computer. It waits for the companion app to be launched on a phone connected by USB. Once it’s connected, you can start streaming your desktop to your phone, and there is a stereoscopic view displayed on the companion app. Although it looks small, modern phones are pretty high-resolution, so it’s really easy to read the text on the screen. Now simply slide it into the holder and put it on your head. It's just like an extra monitor, but you get a full view and you can look in any direction without hurting your neck!

How we built it

There were a lot of technical challenges implementing this. In order to have a low latency stream to the phone, I had to do a wired connection over USB, but Apple doesn’t support this natively. I used an external framework called PeerTalk, but they only allow raw TCP packets to be sent over the USB from the computer to the phone, so I had to serialize each frame, de-serialize it on the phone, and then display it stereoscopically, all in real-time.

Challenges we ran into

I had a lot of trouble with Objective-C memory safety, especially during deserialization when the data was received by the phone. I was obtaining the data on the phone but was unable to turn it into an actual image for around 24 hours due to numerous bugs. Shoutout to Bhushan, a mentor who helped me debug some annoying memory issues!

Accomplishments that we're proud of

I'm super proud that it actually works! In fact, I made one of the final commits using the headset instead of my actual computer screen. It's practical and it works well!

What we learned

Learned a ton about graphics serialization and Objective-C memory safety.

What's next for LaptopVR

I want to add the optional ability to track movements using the gyroscope, so you can have a slightly more zoomed-in version of the screen, and look up/down/left/right to see the extremes of the screen. This would make for a more immersive experience!