Inspiration
Our team is driven by the desire to make language learning an enjoyable and accessible experience for children. We draw inspiration from the idea that technology can play a pivotal role in creating interactive and immersive educational platforms. Not only do we see the potential of connecting children from different parts of the world through our learning platform, but we also wish to instill a sense of curiosity and learning amongst children.
What it does
Penpals makes language learning fun and interactive for pen pals. In this time, we made a mixed reality word learning game that prompts users to find the 3D model representation of the word that matches the flashcard. If correct, the user gets the opportunity to throw the object at an alien to win the game. If incorrect, the object disappears and the correct definition of the word is given.
Given more time, we would make a web app to allow users to send words for their penpals to learn. Once a word is submitted, our app would turn it into a flashcard and create a 3D model representation of the word in real-time using AI. Users can then learn words from their pen pals through the game and be able to better connect to their penpals.
How we built it
PenPals is a Unity application built for the Meta Quest 3. We used the new Presence Platform toolset, which allowed us to add utility like handtracking and passthrough mixed reality with relative ease.
To generate our 3D assets, we used Luma AI's Genie tool to quickly create models for the words. For the prototype, we pre-generate the models based on a set word list, but future versions will generate 3D models in real-time.
The UI elements like the flashcard are a combination of custom 3D-modeled Blender assets made by us and elements from the Presence Platform and Unity's UI.
Challenges we ran into
We ran into many issues with building to the headset from Macs, and this required a day of troubleshooting. We also had a difficult time figuring out how to use MRUK to randomly spawn objects into the room. Some other issues we ran into involved game logic with figuring out how to identify which object the user was picking up.
Accomplishments that we're proud of
We are proud of being able to maintain our vision throughout the whole process. Because of this, we didn't have to pivot later to simplify the MVP on or remove features that weren't possible. We also faced build issues for almost a full day but persisted and were eventually able to build to the headset from all devices (both Windows and Mac), allowing us to test faster given that in-editor testing wasn't possible on the Macs.
What we learned
We learned a lot about troubleshooting and prototyping effectively.
What's next for Penpals
Although the web app was a part of our vision for the product, this was out of our scope given the time and nature of this hackathon. In the future, it would be great to make the web app and integrate it with the MR game. We would also like to add more sharing features to make it more like a sharing platform between penpals. Pen pals would be able share stats and pictures of them learning through the game.
Log in or sign up for Devpost to join the conversation.