-
-
Training in Moonvale: Voice and Ink
🌙 Inspiration
Imagine learning a language not by repeating lines, but living it through a Moonvale adventure.
Created for the Skill Building category, this project reimagines how learners practice real speaking, vocabulary, and pronunciation skills through immersive interaction. We wanted language learning to feel lived, not memorized. Training in Moonvale, Voice and Ink places learners inside a magical world where a mysterious mask companion is trapped in an enchanted mirror and Moonvale is threatened by dark forces. Most learners want to improve vocabulary, speaking ability, and pronunciation, so we focused on those needs throughout the journey. As a global team from Brazil, Canada, India, Iran, and Zimbabwe, we set out to combine XR and AI to create an immersive, meaningful learning experience. Step into the adventure and put your language skills to the test!
🌙 What It Does
Built for skill development, users begin in mixed reality, practicing speaking and writing to free their mask companion. This unlocks entry into the VR world of Moonvale, where language continues to drive the adventure. AI speech assessment offers real-time feedback while XR environments make learning feel lived rather than repetitive.
🌙 How We Built It
We built the experience in Unity with OpenXR for MR-to-VR transitions, designed wand-based writing mechanics, and integrated Microsoft Azure AI for voice evaluation. We created narrative flow, magical UI prompts, and intuitive interactions that blend learning with story.
🌙 Tools We Used
Meta SDK
Voice SDK
Passthrough API
Hand-Tracking API
OpenXR
Microsoft Azure Language Services (speech recognition, pronunciation assessment)
Unity Engine (core development environment)
🌙 Challenges We Ran Into
Object recognition and tracking were not reliable enough with current technology, so we removed them and redesigned the experience. Integrating external AI into XR for real-time speech assessment and creating a smooth MR-to-VR transition tied to performance were significant challenges and valuable learning moments.
🌙 Accomplishments We Are Proud Of
We built a cohesive world where XR and AI work together, created intuitive speaking and writing interactions, and developed a reusable framework for future Kore Meta language experiences.
🌙 What We Learned
We learned how powerful XR becomes when paired with AI, how simplifying interactions increases immersion, and how story-driven environments naturally motivate learners to practice.
🌙 What Is Next
We will expand Moonvale into a larger Speak Quest series, deepen AI-driven interactions, evolve the mask companion, and continue building XR and AI language worlds that turn learning into a lived adventure. We also plan to implement object passthrough camera API and object tracking to enhance immersion, and integrate more advanced AI language assessment to strengthen feedback and learner progress.
Built With
- microsoft
- unity
