Back to school!
Our app presents some ideas on the potential of having "holographic" human representations in VR (think Oliver Kreylos videos :-) ). We built a VR classroom experience with a 3D-recorded teacher, so that you can experience a very special lesson, as if you were there.
- Virtual reality course by a "real teacher" captured with a Kinect 2 and Q3D software
Controls and mechanics : No controls for this one, just enjoy the show.
- humans : we bring humans to VR. We have designed a much lighter and cheaper solution than the mocap studio solutions to embed real humans (or holographic representations of humans) in a VR environment. With just a Kinect 2, we're able to display a human representation that feels really natural though not photo-realistic.
- crossing the uncanny valley : We're able to deliver emotions and "humanity" quite efficiently in our opinion (the screenshots don't really reflect the final experience, hopefully). We find the resulting communication less "uncannyvalleyish" than motion-capture, and more intimate.
- real-time : it isn't shown in the demo because we didn't know how or when the jury will be reviewing the apps, but we're able to deliver this experience in real-time!
Milestone 4 Update : a real teacher this time!
We did a recording with a real teacher (and what a teacher!) His name is Declan Mc Cavana and he is actually one of my favorite teachers ever, so it was a real honor to have him on board! The experience changed a little bit from what you can see on the video. There's much more stuff happening! The app would need some polishing we couldn't provide due to time, but you'll get the idea.
The social interactions that happen between a student and a teacher are correlated to the quality of learning, as shown by several papers coming from the Stanford Virtual Human Interaction Lab. A lot of e-learning content (such as classroom videos or Massive Open Online Courses) currently lack these interactions for obvious reasons. We wanted to see if a 3D recording of a teacher could lead to better e-learning, and what could change between a real and a virtual classroom experience.
We use for this experience a specific software we've developed, Q3D. Q3D is meant to capture humans in 3D very easily with a cheap off-the-shelf 3D sensor - such as the Microsoft Kinect-, transmit this "holographic self" in real-time and display it in VR. Think "3D Skype" : natural interactions between holographic people (à la "Help me Obi-Wan Kenobi").
Our first objective for this demo was to have the VR lesson happen in real-time (which is something we're able to do technically on the Gear VR) but since we didn't know when the app will be tested and our beloved teacher is not always standing in front of our Kinect, we decided to go with the pre-recorded stuff. Our VR chat with John Carmack will be for the next time (John, if you read us...)