Inspiration

Anyone who has learned a language knows the gap: you can ace the flashcards and still freeze when a real cashier asks a real question. Apps teach you words, but not how to speak under pressure. We wanted practice that feels like actually being abroad, minus the queue of people behind you.

What it does

HelloWorld is a VR game on Roblox where you practise a language by speaking it. You pick your own language, the language you want to practise and a scenario where you would commonly communicate with people.

  • A living scene. Every character can be talked to: the cashier, the cook, the cleaner and a regular customer. The staff also chat among themselves.
  • Full immersion. People inside scenarios never speak or understand your own language, just like abroad.
  • A safety net. Lumi, a floating ball outside the building, explains things in your language and teaches you the phrase you need.
  • Real consequences. For example, cooks make what you actually ordered. You pick the food up off the tray and eat it by bringing it to your mouth.
  • Three scenarios. A fast food restaurant, an airplane cabin and a dinner out.
  • 17 languages to practise.

How we built it

The game is built in Roblox Studio with Luau. The client handles the intro screen, the building pop-in, character movement, speech-to-text, text-to-speech and effects. The server spawns the player and calls the Claude API for every line of dialogue.

Claude replies in a fixed JSON format: what the character says, whether the order is complete and which items were ordered. That structure is what lets a free-form conversation drive the game, so the cook knows what to make without any scripted dialogue tree.

Challenges we ran into

  • Voice coverage. Roblox can understand 17 languages but only has voices for 11 of them, so six languages fall back to subtitles.
  • VR in general. VR is very niche technology that does not have numerous resources that other technologies have.
  • Turning talk into gameplay. Getting from "uh, a burger and a cola please" to a tray with the right food on it took the structured replies described above.

Accomplishments that I'm proud of

  • A whole conversation loop with no typing: your voice in, a spoken reply out.
  • No scripts and no multiple choice. Every conversation is different.
  • A world that reacts to what you say, down to the food you end up holding.

What we learned

  • How to make an AI model drive game logic reliably by asking for structured replies instead of free text.
  • The limits of Roblox's speech tools, and how to design around them.
  • This is the first time I made something with VR, it was very interesting.

What's next for HelloWorld

  • More scenarios: a hotel check-in, a taxi ride, a doctor's visit.
  • Feedback after each conversation on what you said well and what to try next time.
  • Difficulty levels, from patient characters who speak slowly to a busy lunch rush.

Built With

  • claude
  • luau
  • roblox
Share this project:

Updates

Submission history