However, as we explored the project, we ran into an issue; how can we have user interaction without the complex requirements of controllers and menus? Having used headsets like Oculus in the past, we felt that VR controls have been too clunky or disorienting. That's where Alexa came in. Having seen all the capabilities of Alexa at previous hackathons, we knew that Alexa offered the capability to give simple user interface without sacrificing the visual aspect of VR.
What it does
Air Alexa allows the user to act as an air traffic controller in a virtual reality airport. The user can decide to either let planes land or circle. This is done through vocal commands by Alexa. By giving an immersive visual and auditory experience, we hope to help train the next generation of air traffic controllers.
How we built it
We used A-Frame, a framework dedicated to making 3D graphics in the browser, as our graphics engine. With an extremely talented community lead by Mozilla, A-Frame offers various packages for animation. We then employed React and aframe-react to bundle the individual entities into React components. Afterwards we used a Rails 5.1 application with SQLite hosted on a Digital Ocean server to run the application. On the Alexa side, we coded a lightweight NodeJS app to interface with the Rails application (not easy!) and design our own custom responses.
Challenges we ran into
Due to A-Frame's relative development, there aren't a whole lot of models out there. As a result, we designed all of our models from scratch. While React made it significantly easier to reuse components, it was still a challenge making designs with our toolset.
We also ran into some trouble linking Alexa to Rails. As it turns out, Alexa is very particular in its data inputs and outputs. We had to use AWS Lambda as an interface, while also debugging POST requests by hand. Not fun!
Finally, Rails 5.1 was only prereleased a few days ago, so we had to deal with the standard early adopter bugs. Not anything crazy, but still frustrating at times.
Accomplishments that we're proud of
We managed to design our models from scratch! We weathered through design iterations and many setbacks such as accidentally deleting models to build our idea.
Two of our team members had no experience with VR or React coming in, so we're particularly proud of their quick learning.
What we learned
Virtual reality is more than an emerging technology; it is easier and easier to implement on a wide variety of devices.
Beware of new releases! Sometimes it's good to have tech mature before you use it.
Finally, be willing to stick with a project, even if it means learning new tech and having to constantly debug and trawl through forums.
What's next for Air Alexa
- Porting Air Alexa to React Native to provide full native features
- Implement more robust physics such as heat and air resistance
- Implementing a web socket connection between Alexa and Rails app for better communication
- Processing more commands and teaching Alexa more skills
- More plane designs !!!!