Minecraft MCP Narrator

Inspiration

We wanted to discover building an MCP and make it fun, so what better way to do it than by creating a personal anecdote. We wanted something that had feedback to YOU, not just an MCP that made boring API calls.

What it does

This is a joint minecraft mod and MCP server. The minecraft mod part turns your game into an API and sends your actions to the MCP server,, which narrates your game out loud (in a funny way).

How we built it

We used the Fabric Mod Loader to build the mod, using Java, mixins, and event calls to detect player actions and create an http request to the MCP server to process.

The MCP server was made in python, and utilized many useful libraries, such as google generative AI for generating the narration, elevenlabs does the TTS, anthropic's MCP library, and a myinstants API to fetch sound effects to play. It also has the capability to take a screenshot of your screen using your operating system for context.

Challenges we ran into

We wanted the mod and MCP to run on any operating system, so we struggled to make it compatible on both Windows and MacOS, and if we wanted to potentially self host, on Linux.

Accomplishments that we're proud of

We are proud to have dove deep into creating an MCP server with complex logic to make the narration as good as possible, as well as creating an extremely functional minecraft mod that anyone can download and use in Minecraft!

What we learned

We learned a lot about API calls in python and event hooks in java. We learned to use many libraries in python, like playsound, REST API (requests), flask, and gemini api/eleven labs.

What's next for Minecraft MCP Narrator

We want to expand the mod to detect more actions, perhaps add functionality for the MCP to return actions to the game, such as punishing/rewarding the player by running commands in game. We also think this is a great project for content creation.