The fact that almost everyone around us was sitting with an OcculusVR or a CradBoard and hacking on stuff. We wanted to lower the barrier of entry for Virtual Reality story telling and application creation. StoryVR lowers the learning curve for making your own VR content and utilizes WebVR to do that. Anybody without any programming knowledge can go to StoryVR and just write a imaginary scene in natural language, and StoryVR will instantly create a Virtual Reality scene in WebVR right in your browser which can be experienced using Google Cardboard,Samsung Gear VR, Oculus Rift or HTC Vive. Practically any device which can show content from a web browser. Our good old mobile phones are enough to experience and immerse into our own creation in Virtual Reality.
What's more! StoryVR supports speech recognition and Live Input. So you can essentially go into a virtual reality scene and live edit it by narrating your story to StoryVR and it will keep changing the scene live while you are in it! Inception all over again
What it does
StoryVR is a web application which creates dynamically generated Virtual Reality Scenes, worlds and content for it's users. It utilizes WebVR which makes the Virtual Reality scenes it creates cross platform and ready for any kind of device which can run a web browser decently. You can drag and drop your images to be included in the scenes too. In addition it supports Augmented Reality mode(only if you have a camera).
W/O Log In
The user goes to the website and writes his imagination using natural language. The natural language processor of watson picks it up and understands what you are trying to do. The web app connects your intent to visualization and starts creating your virtual world for you in a live preview pane. Once you are done writing, your virtual world is finished too. It being live makes it easier to do incremental and rapid changes to your world too! The user has option to write his story or narrate it too! StoryVR listens and understands you!
With Log In
The user creates his scene. But along with previous options, also has option too save the stories or what we call worlds. He can later visit them. Edit them or remix them to something new! They can also share the world. A handy share button creates a premalink to that Virtual Reality scene which can be opened in any standalone device and can be shared anywhere. Also can be embedded in any web page using iframe
How I built it
The backbone of the web application is aframe from Mozilla and threejs. Which makes rendering of webVR applications possible. Amazon EC2 takes care of the hosting of resources and asset uploads since according to specs Google Chrome and other won't support non https and CORS disabled resources in webvr. EC2 is the only host I found where I could easily enable it, Though our micro isntace makes it slow sometimes. IBM watson does an excellent job at making StoryVR understand what the user actually wants and gives us insight which we convert to actionable content for VR creation. The web app itself was created in nodejs and uses monodb as db (as if I could pull of anything in any other decent language...). The editior is a form of codemirror (which i got from github). My most favorite part is the login/registration part. Which essentially only creates a token for registration, bypassing the need to collect and store personal information.
Challenges I ran into
Everything. From finding out concept tagging in alchemy was deprecated and trying to use the one ported to watson. Trying our hands on npm first and dealing with the dependency hell as well realizing each version change can break your app. Trying to get EC2 servers to use CORS header, trying to install mogodb in Ubuntu 16 and realizing their key signing for Ubuntu 16 is broken.
Accomplishments that I'm proud of
We finally could get the app running. It runs in web and mobile (mostly). It works pretty awesome considering users are writing in a normal way and not following a format or giving commands. Proud that actually we can create simple VR scenes just by a narration (even though sometimes they need tweaking)
What I learned
Never believe on documentation unless you actually have tried and tested something. The cncept tagging api got deprecated on 19th August,2016 -_-
What's next for StoryVR
Exciting times! Try to support proper animated models! Right now it supports Collada models. But once it can support others and we ahve augmented reality....you can create live Pokemon Go type models :D