VerbaVisio

Logo

Inspiration

Languages and communication have always been the key to interacting with other people. So we thought about how to make this easier. We believed there must be a way to make it more fun and even more engaging than the current solutions.

What it does

You will be joining a mixed-reality experience where familiar objects around you are recognized by the app. Their names will appear automatically, along with options to translate them into various languages.

If you feel confident with the vocabulary, you can switch to a game mode where you must touch the objects that match the words shown by the app.

How we built it

Basic setup: Unity + Meta XR SDK

We built our project on top of the OneShots Toolkit provided by SenseAI.

To enable translation, we integrated DeepL, which handles all vocabulary conversions.

For spoken output, we used ElevenLabs, allowing users to hear each word pronounced clearly in the chosen language.

Challenges we ran into

We had to combine a wide range of different functionalities within a limited timeframe.
Finding solutions to properly integrate all these systems. Scanning, translation, speech synthesis, interaction, and game mechanics was one of our biggest challenges.

Accomplishments that we're proud of

We successfully created two modes: a learning mode and a game mode.
We implemented multiple selectable languages, with the system being essentially unlimited in how many languages can be added in the future.

What we learned

We learned how important it is to make good use of everyone’s distinct skill sets and communicate effectively as a team.
Time management was also key and we learned that sleep is important, because late at night, productivity drops fast (but the pizza was delicious)!

What's next for VerbaVisio

Gamification, gamification, gamification. We want to expand the game mode further to keep users engaged in learning.
We also plan to add features that track how well a user knows certain words, helping them build long-term knowledge and making the experience more personalized and educational.