We have done a lot of VR projects in the past, and most of what we do is interfacing with the real world through VR creating a mixed reality. We wanted to do this again and bring real world objects into the VR environment.
What it does
Our VR environment is built with unity and using the vive and it's camera we pull objects from the Real world, and bring them to the VR where they can be manipulated.
How we built it
We made a unity environment that was catered for the vive. Once we figured out how the vive would interface with the environment we needed to take a picture with the front facing camera with the vive. Once we had the picture of the real world we used the cloudsite api to use computer vision to tell us what the main object of the picture was. Once we had that data we could dynamically create an object that the api gave and placed that into the scene.
Challenges we ran into
Unity web services are not very friendly for post requests. When we went to go send the picture to the cloudsite api it couldn't get the encoding correct. So we had to get creative and execute command prompt commands to cURL the website to send the data. Again, nothing is easy and unity doesn't really like to do that as well so we had to write the command we wanted to run to a batch file and execute that when we need to.
Accomplishments that we're proud of
This is the first time we have used any api in a hackathon and relied on other services to do computation for us. It took some work to get it usable for each time we take a picture with the vive but we got it all working.
What we learned
VR is always challenging but rewarding once you get it working. The things that we can do in 36 hours in VR are pretty crazy in terms of how humans react to what we have made. If we had more time to work on these things then we could really make something that could change the world.
What's next for VVoRld
We came into the hackathon and wanted to take a picture of a chair on the floor and be able to sit on that in vr. However there are limitations with only one camera. We would really like to nail the positioning of the objects and be able to recreate rooms in only a couple image gathers.