"Long ago, the four nations lived together in harmony. Then, everything changed when the Fire Nation attacked. Only the Avatar, master of all four elements, could stop them, but when the world needed him most, he vanished. A hundred years passed and my brother and I discovered the new Avatar, an airbender named Aang, and although his airbending skills are great, he has a lot to learn before he's ready to save anyone..."
But, lucky for him we have developed an AR simulation that help him master all four elements with only his trusty iphone and Oculus Gear VR.
What it does
Enter the Avatar state and play as Aang, the Avatar. Experience the power of the Avatar, manipulating water, earth, fire, and air in AR. Fight against enemies that will track your location. You can also take snapchats of your Avatar adventures with the click of a button and share it with all your friends! Compatible with all stereoscopic headsets. Clicking the z button will screencap the current VR overlay and prompt you to publish it to your snapchat story!
How we built it
The Oculus Gear VR is not meant for AR nor an iphone. But, we leveraged the fact that when the cover is off, you can house an iphone on the case with the camera exposed! ARKit is also not meant to play well with stereoscopic headsets (think google cardboard), with no native support. We implemented the stereoscopic view by mirroring two ARSCNViews next to each other and copying the Frames.
The User can interact with the world via a gamecube controller. The gamecube controller is mapped to specific commands in the game, connecting the app and the hardware commands using websockets. Specific moves are controlled by the direction of the user's camera.
Enemies are spawned when real-world images/markers are recognized by ARKit. Enemies will track the user location and hone in on the user, disappearing and re-spawning after a dedicated amount of time.
We used Snapchat's API to implement creative kit, which allows us to share our adventures mid-game through the Snapchat app.
Finally, we created all the sprites ourselves.
Challenges we ran into
Initially, we used a Flask server implementation, but had trouble connecting it to the iOS implementation of Socket.io, so we switched over to using a Node.js server to facilitate socket connections between controller inputs and the app.
Because the physics engine in ARKit is very bare bones, we had to implement a lot of vector mathematics to calculate object transformations and positions.
Accomplishments that we're proud of
The name & making our first AR App
What we learned
A lot! About Sockets, ARKit, and game design.
What's next for Avatar: the Last AR Bender
Multiplayer implementation with ARKit 2 (currently in Beta)