AI Dungeoneering

Inspiration

Some of the earliest marks that both LLM's and Gaming had on culture in their early years focused on exploring a fantasy dungeon, fighting monsters and avoiding perils along the way. On the LLM side, there are projects such as AIDungeon that set the scene for what AI interactive words could look like. On the gaming side you have games such as Zork and Rogue.

These two mediums have opposing issues. Games are often too static and limited to provide a fully free experience, from invisible walls to NPC's with limited voicelines. LLM's, on the other hand, have the curse of being too dynamic. One moment you're adventuring in a meadow, the next moment you're in a volcano.

Our goal was to get the best of both worlds. We want to use the LLM to free the game, and use the game to regulate the LLM.

What it does

It is a simple dungeon crawler, but many actions the player takes are deferred to an LLM to compute the result. This includes spawned weapons and enemies, as well as outcomes of the player and enemies abilities. However, the LLM is also prompted to come up with damage values for the player and enemy, and those values are used in a more traditional way.

How we built it

We use a Llamacpp server running locally as a backend, with an HTML/JS game client as the frontend.

In order to get our LLM to output items, enemies, etc in the particular formats we need, we made heavy use of the grammar field in the completion request body that Llamacpp offers (this does not appear to be a standard feature of OpenAI-like API's, however). This allows us to specify that it must output a 2-digit damage number, for example. We similarly modulated the temperature values throughout generation to get results that are both more fun and consistent.

Challenges we ran into

Our grand budget of 0$ for this project did not leave us with much room to use paid API's. This is why we opted to use Llamacpp, as it allows us to run an LLM locally. This led to another challenge, which is the fact that only one of us had the laptop specs to run it. Our stopgap solution was to have "dummy" outputs that would be returned in case the LLM wasn't present, which allowed the other two in the group to test their code.

Accomplishments that we're proud of

We each had a way of bringing each other down to earth. Each one of us had ideas and goals that were valuable, but we each also had ideas that were beyond the scope of the weekend we had available. Yet, each one of us had the courage to tell each other we don't have infinite time, which can often be the hardest part of hackathons.

What we learned

HTML and CSS were mostly new to all of us. Each person had vague ideas of how they worked, but together we managed to fit the pieces together and make our understanding more concrete.

What's next for AI Dungeoneering

This has an opportunity to become a great hobby project, and if it does we'd like to implement many of the features we came up with during development, such as:

• Generating images for the weapons and enemies
• Scaling the damage numbers as the game progresses
• Increasing the wackyness of items and enemies as the game progresses
• More concrete numbers than just health (such as defense)
• Better CSS

etc...

It's also a useful jumping off point for implementing AI's in other games. We can use it as a reference for modding in AI generated enemies to an existing game, for example.

Built With

• html
• javascript
• llamacpp
• mixtral