Tesseract

Inspiration

I wanted Minecraft building to feel like using a modern creative tool, where you can describe what you want and it appears instantly. i was also inspired by tools like Cursor that turn natural language into real changes in a developer’s workspace.

What it does

Tesseract lets you select a region in Minecraft, describe a build in plain English, and have an AI generate a safe JSON build plan that gets placed in your world. It also includes a web UI that generates a paste link you can use in-game with a command.

How we built it

I built a Fabric mod for Minecraft 1.18.2 that handles region selection, commands, validation, and block placement. I used Gumloop to orchestrate the AI workflow and a local Python plan registry plus a small web server to generate, store, and serve build plans via URLs.

Challenges we ran into

Getting the AI to reliably output strict JSON was hard, especially for counting and formatting. I also had to adapt to asynchronous Gumloop runs with polling, deal with Java version requirements for Minecraft, and debug local networking and SSL issues during development.

Accomplishments that we're proud of

The end-to-end demo flow is smooth and reliable: prompt to plan to blocks placed in-world. The web UI makes it easy to generate a build and share it into Minecraft with a paste link, which avoids the in-game chat length limit.

What we learned

Reliability beats complexity for a strong demo, so I focused on validation, constrained palettes, and robust error handling. I also learned that AI outputs need guardrails and post-processing to be dependable in real software.

What's next for Tesseract

Add richer context input like screenshots and better style transfer from existing builds. Improve build quality with smarter structure primitives, symmetry rules, and support for more block palettes. Make the paste workflow usable beyond local demos by hosting the plan registry remotely and adding multiplayer-friendly flows.