Hololearn - IVO
Hololearn on Hololens
Education has enormous transformative power, and the potential to change the world. However, the potential is unrealized and we strongly believe this can be changed with AR.
Studying alone is boring, especially when you only read or simply listen. It's also ineffective, as content is only retained at a 10% rate with this method. This leads to frustrated students and teachers. What a waste of time!
Studying is fun and engaging when you can explain the material to someone else. It's also the most effective way of retaining information, at a rate of 90%. Win-Win!
IVO - The name of the avatar “IVO” comes from the Japanese word with the same pronunciation “Aibou” which means pal, partner, or buddy.
What it does
Users insert customized content of what they are learning at school, such as; definitions, concepts, and key terms. The content is from the users traditional educational materials, such as; textbooks, study guides, or flashcards. The content is inputted through speech recognition or via the web.
The user is greeted by an avatar upon setting up their Hololens AR headset. The avatar has a positive, asset-based minded personality, who encourages the growth mindset and lifelong learning of the user. The avatar is always curious about the topic that the user is learning, and really wants to learn more.
Stage 1 Learn::
In stage 1, the user has a time period to review flashcards of the content. The time period is dependent on the amount of content that has been inputted. Next to the flashcards, there are retrieval image cues that are related to keywords within the content.
Stage 2 Explain:
In stage 2, the avatar is ready and excited to learn from you.The avatar asks you to explain the topic of each flashcard, with various conversational prompts. When the avatar prompts the user, the retrieval image cue is shown. The conversation is structured within a time period based on the amount of content. When the user is answering the prompt, there is a visual in the form of a brain, that is progressively colored in while the user is answering the prompt correctly.
The avatar reacts with body gestures to the user responses based on whether the response is correct or incorrect.
Once all of the prompts have been answered, the avatar either congratulates the user or directs the user to review the flashcards again in stage 1.
How we built it
This application was built in Unity using the UWP platform. We used Microsoft's Mixed Reality Toolkit in order to build the groundwork for HoloLeARn.
The animation was built using a character and animations from mixamo. The voice of the avatar was recorded by one of the team members.
Challenges we ran into
We started the project with Magic Leap, and ran into issues with the speech recognition capabilities. After switching platforms to the Hololens, we had to tackle the issue of keyword and phrase recognition. We also ran into issues with the Unity build process and application lag.
Accomplishments that we're proud of
We are proud of identifying a huge gap in the education space and leveraging AR to develop to open retrieval practice, the most effective method of enhancing memory, to everyone. The avatar that we have built, embodies characteristics that encourage positive-minded and lifelong learners, through the connection built during the game.
Creating a virtual avatar that will interact and react with the user in Mixed Reality is a huge accomplishment as well. Getting the avatar to respond and actually listen to and recognize our voices was a big step for us.
What we learned
Majoring learning came from starting with the Magic Leap. We learned what the platform is capable of and isn't capable of. We also learned more about the Hololens platform and how spatial perception comes into play.
What's next for HoloLeARn - IVO
Adding additional avatars to have a wide variety of representation for users to identify with.