We had the idea of creating an immersive experience for reading books. Readership is continually decreasing among our demographic each year. The problem we posed to ourselves is, how can we bring magic back to reading?
What it does
vReader is a virtual reality audiobook player. Using Unreal Engine 4, the app renders a 3D environment reflecting the story being read. Parts of the scenery will animate throughout the narration at key events in the plot.
How we built it
This demo scene was built from the ground up using Unreal Engine 4.14.2, tailored to the Oculus Rift virtual reality headset and the Oculus Touch Controllers. The excerpt we chose for our demonstration was a scene from Harry Potter and the Philosopher's stone, wherein the reader gets to experience Charms Class and the famous levitation scene. Sculpting and modeling the environment came from a combination of default Unreal Engine assets and Autodesk Maya. The animations for cued and scripted events were key-framed manually in the engine, as were the particle effects.
Challenges we ran into
Virtual reality integration in Unreal Engine 4.14 is brand new, and the Oculus touch controllers are even newer. We had little to no documentation to help us program our player controller and level blueprint. We had to improvise scripting methods, and hit several deads ends at times when what we wanted to have happen on screen was impossible with current technology.
Accomplishments that we're proud of
We created something we feel could be a viable consumer product, and that we would be happy to buy and use. With a partnership from a publisher and the streamlined level design, we have the chance to perfect the shared reading experience. Improvements in mobile VR would make this prospect even more possible; children will be able to learn in a new way that's fun and engaging. Adults will be able to fall in love with reading again, and experience the stories they love as if for the first time. We had a vision and, with the tools and time we had to work with, we think we proved our point.
What we learned
We learned event scripting, animating static meshes, audio cues, motion controller input, and dynamic lighting in Unreal Engine. We also learned about important design standards for making a virtual reality experience. vReader was a culmination of all the skills we've gained in our careers as computer scientists, combined with a new venture into unexplored territory.
What's next for vReader
If possible, we'd like to take vReader to a publisher or audiobooks distributor like Audible/Amazon and present this as a product that can feasibly be brought to market. Having the licenses to a great bevy of audiobooks with professionally done narrations already available (as well as having the business capital needed for a commitment like this), it's reasonable that this is something Amazon could distribute through a mobile Virtual Reality platform like the Google Daydream headset or the Samsung Gear VR. The experiences are self-contained scenes and settings -- no characters or anything besides the scene is depicted. This helps both with the suspension of disbelief, and also saves development time and funding that it would otherwise take to bring those to life for every scene in a book.