Minecraft AI Agent Mod

Tired of memorizing Minecraft command IDs? The AI Agent Mod uses Gemini to turn plain English into instant and lag-free game commands.

Comment

Inspiration

Minecraft commands are powerful, but the syntax is a real obstacle for new players. Trying to remember exact item IDs and coordinate formatting just to do a basic task is frustrating. We wanted a way for players to simply say what they want, and have the game handle the technical details.

What it does

This is a Fabric mod for Minecraft 26.2 that adds an /agent command. Instead of typing out a complex string, you can type something like “/agent make it night” or “/agent give me 64 golden apples”. The mod sends your prompt to the Gemini API, gets a raw command back, and executes it in-game using your existing position and permissions.

How we built it

The mod is written in Java (JDK 25) using the Fabric API. We used the current Commands and sendSuccess methods for Minecraft 26.2. For the AI, we implemented the Gemini 3.6 Flash model using Java's native HttpClient and Gson for parsing JSON. To prevent the game from freezing, all network calls are handled on background threads using CompletableFuture, and results are passed back to the server thread using getServer().execute(). The API key is read from an environment variable (GEMINI_API_KEY) rather than being hardcoded in the file, so it isn't exposed in the compiled jar.

Challenges we ran into

The biggest issue was keeping up with the recent API changes in both Minecraft 26.2 and Gemini. A lot of the initial code errors came from outdated class names and dependency paths. We also had to swap the AI model from an older version to gemini-3.6-flash because the previous one was deprecated. We spent a fair amount of time dealing with the threading to make sure the game didn't pause while waiting for the AI.

Accomplishments that we're proud of

Getting a large language model to run smoothly inside a client-side game without causing lag is the main achievement here. We also solved the issue of the AI adding conversational text to its responses by using a strict system prompt, which forces it to return only the command itself. Using environment variables for the API key was a deliberate choice to keep the code safe to share.

What we learned

This project was a solid lesson in asynchronous programming, especially in an environment where blocking the main thread is not an option. It also made it clear how much detail you need to include in a system prompt. Without strict instructions, a model will default to behaving like a chatbot rather than a tool.

What's next for Minecraft AI Agent Mod

The next step would be adding context awareness so the AI knows the player's biome or time of day. We are also looking at voice input to make it even easier to use, and possibly expanding it to handle larger tasks like building structures.

Built With

Share this project:

Updates

Submission history