Incantation

Two of the devs battling it out in the arena!

Inspiration

Incantation is inspired by Mage Arena! We wanted to create a game where players can voice and create their own spells.

What it does

Incantation is a Roblox game in which you can cast spells using your voice. Battle against one or more players, and whoever is the last standing wins! Select your spells from a premade list, or create your own using Gemini API to turn a description of a spell into a real one in game!

How we built it

Incantation uses the built in Roblox Studio’s microphone to take in audio, which is then sent to Elevenlabs API for speech to text translation via a local HTTP connection. If a set trigger word is used for a spell, it sends a message to a public-facing HTTPS relay server (which is hosted on one of the players' devices and uses a Cloudflare Quick Tunnel) to execute the spell in the game, which then relays the message to the remote Roblox server when it is requested. Each player's ElevenLabs process is identified using a randomized code given in-game that the player inputs into their companion process — which is how we know which player sent which spell and what spell commands to look for. For the personalized spells, a user is able to input a short description, which is analyzed by Gemini API and then turned into code imported into the game for a user-unique spell!

Challenges we ran into

About halfway into the hackathon, we tried to test our game by playing it together through Roblox Studio Team Test. However, we soon realized our approach — a local HTTP connection running on each machine between the ElevenLabs API and our Roblox game — was completely incompatible with Team Test, since the server running our game was hosted at a remote Roblox data center. This caused us some distress because we worried that the crux of our game, voice-operated spells, might not work with multiple players. However, we resolved this issue by creating an HTTPS relay. Each player runs an ElevenLabs companion process on their computer that converts their speech to text through ElevenLabs. Then, if a spell command is detected, it sends a message to a public-facing HTTPS relay server (which is hosted on one of the players' devices and uses a Cloudflare Quick Tunnel) to execute the spell in the game, which then relays the message to the remote Roblox server when it is requested. Each player's ElevenLabs process is identified using a randomized code given in-game that the player inputs into their companion process - which is how we know which player sent which spell and what spell commands to look for.

We also encountered some issues with Gemini API token limits, but we worked around it by having our code utilize less performant backup models if we run out of tokens for the higher tiers.

Accomplishments that we're proud of

We were amazed by the AI-generated spells Gemini created during our testing. When designing the game, we originally believed that providing a textbox where players could generate anything would be too out of scope for Gemini to make impressive-looking, entertaining spells. Yet, through having a pre-defined library of effects defined in module scripts that Gemini simply refers to, it became much easier for it to quickly generate spells that still looked as stunning as our pre-made spells. We could ask it to generate a wide variety of different spells, like an explosion of ducks or a water droplet that bounced throughout the arena, and it accomplished it without needing to reference a spell we had already made.

What we learned

Before this hackathon, the majority of us didn't know how to use AI or MCP servers to code video games. This project taught us how to use Claude and Codex efficiently to add our ideas directly to Roblox without any manual intervention. Learning how to integrate the ElevenLabs Speech-to-Text API to solve the problem of voice-to-text latency was also intriguing; creating a program that sends and receives HTTP calls from Roblox using HTTPService to communicate between ElevenLabs and our game, in addition to deploying an HTTPS relay to allow remote Roblox servers to communicate with the programs on our machines, was a novel idea to us that was curiously nuanced. Learning the Gemini API to integrate user-to-AI spell generation was also incredibly rewarding for the range and depth of spells it could create.

What's next for Incantation

If given more time, we would love to turn Incantation into a game that everyone can enjoy. Currently, Incantation requires running an ElevenLabs companion program on each player's computer to quickly and accurately convert speech into text that can be used to trigger spells in-game. It also requires one computer to host an HTTPS relay so the program can communicate with Roblox servers. We would like to experiment with a solution that does not require any extra setup — just join the Roblox game, and all dependencies would be resolved automatically. We could do this using Roblox's Speech-to-Text API service, although we found it much slower than ElevenLabs. We would also need to find a cost-efficient solution for using Gemini or another AI for custom spell creation, as right now, if this game were published, our tokens would run out fast. Hence, we hope to make Incantation much more accessible in the future.