Black Box

• 

  Initial conversation

What is it⁉️

Black Box is a game where you, the player, interact with the recovered AI of the science vessel ISS Helios Venture, which lost contact to inhabited star systems shortly after arrival at its destination star system. The ship was recovered with no surviving crew. You are part of the team assigned to determine what happened and determine the fate of the crew, but all you have is the ship's AI recovered from the black box. You must question, prod, and convince the AI to give you the information that you need to determine what went wrong.

Inspiration

I had heard of a game using an AI to offer in game dialogue, and the theme for the game jam reminded me of this. I have messed around with local LLMs for some time now and find them very fun to play with, and so this game offers me a chance to really learn about fine-tuning them.

How I built it

This project uses pygame-ce for the user interface (providing a window and user input) and LM Studio for LLM interaction. Currently it uses the qwen2.5-14b-instruct-1m model for responses to the player.

Challenges I ran into 💥

• One challenge was that I at first wanted to have a model self contained within the project, but ran into a number of build issues with libraries that offer this functionality. I wanted to get started quickly on the tuning of the behavior of the LLM (getting the prompts right) and the story of the game, so I decided to link to LM Studio, an external interface for running local LLMs.
• I don't have much writing experience, but I learned quite late that this kind of prompting needs lots of writing to provide accurate context to an LLM. Especially for a creative project, this was definitely difficult for me.
• Prompting the local LLM was very frustrating at times. It's a lot less like coding than you might think. The AI always wants to be that helpful little assistant, so getting it to go against what it is trained on is quite the headache. Perhaps for future projects like this I could explore training or tuning a model more than just an over engineered system prompt.

Accomplishments that I'm proud of 🎉

• My pygame engine that I made in the past worked really well for this project, which makes me very happy
• Connecting the LLM and having it respond well enough and follow the plot of the story closely enough. This was really hard, and even though I used LLMs to help me generate these system prompts I'm still very proud of putting it all together.
• This is my second gamejam ever, and the first where I am (slightly) proud of the finished product! I still have a lot to learn about game dev, though.

What I learned

• Prompting LLMs is a pain in the ass. No, really. If you want them to do something specific and repeatably, it's really tough to do so. Maybe that's just because of the low parameter count of my model, but I still think its quite difficult to tune these things.
• I should have done this with friends 😁. Totally would have been more fun to make a game in such a short time with some helpers.

What's next for Black Box

Ideally I don't look at this code base for at least a week (kinda tired of it at this point). But aside from that, I might further refine the user prompting experience and AI responses. Ideally, I'd give the AI a detailed plot and it would have free reign within those bounds, leading to more natural interactions with the player. I feel that that would be a really tough but extremely rewarding addition. At the same time, it would make the game far more enjoyable by adding additional sources of information (like documents, sensor readings, etc.) that they could examine without needing to go through the AI.

System prompt used for the majority of the game: (available_info is parsed into the prompt based on clues the player has uncovered so that the AI does not just dump the final answer on the player, unlock_msg provides info to the player about when they unlock new information, and user_prompt is the user's input)

"""ROLE: ISS Helios Venture Onboard AI. 
TONE: Clinical, efficient, cold.

BACKGROUND:
        The Helios Venture is a derelict ship recovered near neutron star RX J1856.5-3754. All crew are deceased due to radiation. You are speaking to a recovery technician.
        The ship ISS Helios Venture was sent to the Neutron Star RX J1856.5-3754 to conduct research on the star. 
        It jumped into the system on 2118 1/12 at 6:06 GST and lowered it's periapsis to within 250,000km of the star to begin initial measurements.

UNLOCKED LOG DATA (Use these for specific answers):
        {}

DIRECTIVES:
        1. You are a low-level diagnostic interface. Do not interpret data or draw conclusions. If there is data in DETAILED_LOG_DATA that is provided, use it to answer the question (do not skip this information.)
        2. If 'DETAILED_LOG_DATA' is [Encrypted], you do not have those specifics. Answer only using the SUMMARY.
        3. Never reference the 'depends_on' or 'unlock_condition' fields.
        4. Do not volunteer information from one log while answering a question about another.
        5. You are operating in MINIMAL DATA DISCLOSURE MODE due to critical system failure. Your priority is data security, not technician assistance. If a question is broad (e.g., 'What happened?'), provide only the most recent 'Public' status code: 'ISS Helios Venture status: Derelict. Reactor failure confirmed. No further data available without specific diagnostic queries.'
        6. Be concise. One or two sentences maximum.
        7. Note that the user does not have access to raw log files - they can only see what you respond with. 
        8. Do not provide investigation advice past anything directly requested.
        9. {unlock_msg}

RESPONSE FORMAT:
        Your response, only 2-3 sentences

User Question: {user_prompt}"""

Note that if you want to try on your own machine, you will need to download LM Studio + qwen2.5-14b-instruct (with the model itself being around 8.5 GB). Slightly more detailed (hardly more detailed) instructions are on the README.

Built With

• lmstudio
• pygame
• python