What inspired us
Our whole team is made of individuals where everyone had to learn a second language. We realized that everyone had their own different learning styles that allowed them to learn the language. There are some visual learners, spacial learners, audio learners, and kinetic learners. Classic curriculums have always catered to the paper based memorizing style of learning. We saw a potential for the Magic Leap to allow us to create new language learning experiences that cater to specific learning styles. By using the Magic Leap spatial computing capabilities, with the power of Google's Cloud API we thought we could create a learning experience catered to visual-spatial learners.
What it does
The Tarnia application uses a task-based learning approached backed by research to set the user on a quest to find items in their space. When they find the items, Tarnia will teach them the German word for the object in the room that they are pointing at. The Magic Leap will keep the translated label of the word in the 3d space and the user can revisit if necessary.
How we built it
We used the Magic Leap SDK for Unity as a Starting Point to capture a 2D Image with the Magic Leap. Then we worked on a Way to use the Google Vision API, Text to Speech and Translate with Unity. After that was done, we were able to create a MVP that captures a 2D Image, upload it to Google, analyze it and then tells the User the what he is seeing in his chosen Language.
Challenges we ran into
We had some Problems with Github as well as with the Magic Leap SDK as it was a completely new toolset for us.
Accomplishments that we are proud of
The Integration and process flow to identify and image from the magic leap in live view is our greatest accomplishments. We believe this will be the backbone for creating a free flowing task base learning experience.
What's next for Tarnia
Next we want to continue to develop more modes for learning, more games, and more stories. We think that the magic leap will enable easier language learning experience for different learning styles. For the foreseeable future, we will be designing task based games (proven by research), visual spatial (enables spatial learners), and story based(bringing in human narratives to language learning) experiences.