Virtual Screens

Inspiration

At home you could have multiple monitors which helps people like us be more productive. However, when taking a laptop and going elsewhere you normally don't take your monitors with you or maybe you don't have multiple monitors. But, we would somehow like to have the similar level of productivity we would have at home. So we came up with Virtual Screens.

What it does

Virtual Screens uses virtual reality technology to emulate multiple monitors. You look into the VR goggles and you will see multiple screens. You can then move windows into the different screens and turn your head to view and focus on them. You can also use them while lying on your bed, while your away from your computer, the possibilities are numerous.

How we built it

We had Google Cardboard at our disposal so we created an Android application using the Cardboard SDK and Java. We also created a desktop application in C++ to control the windows. The two applications communicate with each other using UDP on a local area network. Physically, you have one monitor so what the desktop application does is it brings into focus (to the front) the window that's on the current virtual screen the user is viewing (focused on). Then it takes screenshots and sends the data to the android application where it is rendered. The other screens not in focus are not updated but are continued to be rendered from the last frame.

Challenges we ran into

We encountered a lot of bugs due to UDP networking which was fairly new to us in terms of actual programming. The Cardboard SDK which was also quite new to us used OpenGL and so we had some rendering bugs. We also had some wifi issues since our phone and computer couldn't connect to the same network due to different frequencies. In the end we used our phone's hotspot capabilities to get onto the same network.

Accomplishments that we're proud of

Some of us had never done Android programming, UDP programming, VR programming, amd OpenGL programming before and so we are very proud of the fact that we managed to learn all these in just a few days and that we built applications utilizing them as well.

What we learned

We learned a lot about UDP communication programming, Android programming, VR programming, and OpenGL programming.

What's next for Virtual Screens

We still have a lot of ideas that we could implement for Virtual Screens such as augmented reality, more gesture actions, etc. and we think this project could possibly be marketable. But even just for our own personal use, Virtual Screens can definitely be polished up, improved, and optimized.

Built With

• android
• c++
• google-cardboard
• opengl
• udp-networking