Unlike many major historical disasters, the coronavirus pandemic did not strike us with one catastrophic event. Instead, the deadly problem has snaked its way around the world, killing millions and causing a global health crisis. With mass-scale border restrictions and home-quarantine, the tourism industry began to nosedive. As the pandemic remains a public health concern, many still wonder about the prospects of travelling. Hence, ‘vrTravellers’ seeks to solve this problem!
What it does
You don’t need to pay for expensive flight tickets or hotels to see the world anymore! Within the comfort of your home, ‘vrTravellers’ will take you to natural attractions, historical museums, and infamous landmarks! Moreover, it provides insightful information, via a virtual assistant, whom you can converse with about the food, people, languages, time, weather, traditional dances and music of each city you choose to travel.
How we built it
Flutter Flow was used to create the UI of our application. Flutter was then used to create the mobile application compatible with both iOS and Android. With the base application created, the team then shifted their focus to Unity VR. The team used simple models, and uploaded them to echoAR, deciding on positions and data points. Next, the team wrote voice recognition scripts in C#, that take in user input in the form of voice command, to navigate to different parts of the scene in Unity (different parts of the virtual tour), and giving the user the option to seamlessly interact with the virtual tour guide. Next, the team added in key bindings for connecting bluetooth controllers with the mobile application. In our demonstration, we have made use of a PS4 controller, but a different one can be used as well. Lastly, we added in the google cardboard VR SDK, and rendered the player and camera to give a fluid VR experience. As the scenes switch, the models are added from the echoAR platform into Unity, which is connected to an iPhone XR via Wifi for testing. Once tested, we used the flutter-unity-api for integrating the unity project and our flutter app to make the final product - vrTravellers.
Challenges we ran into
- Integration of Flutter and Unity
- Adding individual models to echoAR platform
- Unity model render time issues
- Deprecation of google vr sdk - the need to hardcode certain aspects of the unity project
Firstly, there are not many production value mobile applications that achieve the purpose of tour guides with the VR experience. The current market doesn’t have seamless VR-flutter applications with cloud integration. With the help of echoAR, we seek to solve the problem of render time which is usually faced by VR apps as they are really heavy, and discourages most developers from using virtual reality. We believe VR has the ability to give the best, seamless experience of 'virtual' surroundings, and with the aid of echoAR, vrTravellers is able to surpass the boulders of using VR technology - which sets us apart from our AR competitors as well.
We hope people will travel around the world, sustainably and comfortably, and most importantly safely through our app.
Accomplishments that we're proud of
- Learning to create a Flutter App in a day.
- Unity and Flutter Integration
- Making seamless Unity Projects - with minimum render time thanks to echoAR
- Mobile-bluetooth Controller integration via scripting
- Clean, simple, easy to use VR project, with patch fixing google-cardboard-sdk bugs
- Code a voice recognition AI on Unity
What we learned
- Integrating Flutter and Unity
- Using echoAR to make seamless and lightweight Unity Apps
- Voice recognition using C# scripting
- Adding VoiceOver audio files on Unity
What's next for vrTravllers
Collaborate with travel agencies to get their inputs and increase the virtual databases of various locations across the world.