Point to Point Game

Boss Fight
Vocab Check
Intro screen
Level

Inspiration

Our group has a relatively diverse language background. We speak Japanese, Spanish, Turkish, Polish, and more, but we have found it is difficult to point people to resources to practice speaking. Research shows that speaking and active usage of language is the best learning method, yet it remains difficult to find fluent speakers who are willing to help you learn. We were inspired by this to create a game-based language learning platform, specifically for heritage speakers.

What it does

Our game takes the user through a series of cities in their target country's language. For example a user learning Polish will be tasked with visiting Kraków, Gdańsk, Warsaw, etc. In each city, they will interact with a "local" who is trying to teach them a few words in the language. The local is an eleven labs agent who is prompted to only advance the user once they are happy with their learning usage. Between cities the user plays small review games that reinforce their learning. Finally, there is a boss fight where the user uses everything they have learned to beat a boss.

How we built it

We leaned into AI to build this. We have never built a game on the phaser platform, so we were leaning into Ai for code generation. We tried to refine our prompting for speed of development. Likewise, we also leveraged eleven labs for the voice integration. We thought it would integrate well into our project. We used Gemini to source the target words based on user interest and relevance to the region. For example a mountainous region will teach the user words like "snow" and "mountain".

Challenges we ran into

The original implementation used a few different API tools from eleven labs and Gemini. We would capture the user's input, transcribe it, send it (alongside context) to Gemini, get a response back, call eleven labs for audio, and finally play it to the user. This worked well, but had terrible latency. We tried different audio formats, compression schemes, and other optimizations to no avail. Eventually we found out about eleven labs agents. We used the socket-based agent to cut down on latency from 5+ seconds to under a second, significantly improving the user experience.

Accomplishments that we're proud of

We are proud that we managed to create a real working game in a few hours. We have never built a game before and are happy that we managed to create something interactive and relevant to our original goals.

What we learned

We learned that game dev is really tough. From physics to language to level design, there are many things that gamers often take for granted. We also learned a ton about eleven labs and how voice agents work. We see ourselves using that platform in the future for other non-game related use-cases.

What's next for Point to Point Game

We plan to refine the gameplay experience by making each level more interactive and complete. Right now the experience is limited, but we want to add intra-level quests, tasks, and a more complete player system that introduces health, sleep, etc. that the player must manage.