OpenCraft

Agent farming cows with the sword it was able to locate
2 agents collaborating and working simultaneously
Internal Dashboard for logging and viewing agent orchestration
Tech Design

Why?

Everyone Knows Minecraft Can Be Tedious

Grinding ≠ Fun
Even in a game we play for fun, the grind inevitably feels tedious.

We wanted to see what it would look like if a team of agents could handle that coordination for you. Three agents, each with a role, talking to each other in a shared group chat and figuring out the division of labor on their own. You send a message saying "go get me copper"...lol. and the scout finds a vein, the miner heads over, and the crafter turns the haul into nothing... bc its copper.You stay in the loop without having to micromanage every step.

The part that excited us most was the iterative piece. The agents remember how you like things done. If you tell them once to always rebuild a pickaxe after mining, they carry that forward. The system gets better the more you use it, and that felt like a genuinely useful thing to build.

How to use it??

Minecraft is fun, everyone loves it.
Wouldn't ai take away all the fun? No.

Think of creative ways to employ ai to create new automations in your world: Think of the endless farms that could be created, no more limitations with pesky villagers....
Beyond this, what about the workflows that could be created in your bases. Think about hermit craft, if you were in their shoes would you grind hundreds of hours just to get the resources needed for your builds?? :(

If you enjoy it, great.
If not, welcome to OpenCraft.

What It Does

You interact with three specialized Minecraft agents through a shared group chat interface. Each agent has a defined role (mining, crafting, scouting) and they coordinate in real time through a Supabase-backed shared world state, with Redis handling distributed job locking so multiple agents never claim the same task.

When you send a command, the orchestrator parses intent, creates a job, and the appropriate agent picks it up. Agents query a PostgreSQL database for live world data like ore locations and chest inventories, and they pull user preferences from a RAG pipeline powered by pgvector to know how you want things done. After every action, they write observations back to persistent memory, update the world state, and report results to the group chat.

The system is iterative. Agent behavior is governed by an embedded markdown directive file that you can update at any time. Change a preference, re-embed it, and the agents pick up the new behavior on their next task. The more you use it, the more context the agents accumulate, and the better they get at executing the way you want.

How we built it

We connected iMessage to a backend using Photon, which lets us receive and send messages from a group chat. Those messages get processed by our agent layer, where different AI roles interpret the request, plan what to do, and generate actions. We then send those actions into Minecraft using Voyager, which executes them in-game. At the end of the run, we trigger a reward flow through a separate service, which also handles our API integration and sends an event back to the game to show the delivery animation.

Challenges We Ran Into

Getting the Dedalus VMs to cooperate was difficult as they went down repeatedly throughout the hackathon, so we'd have to wait for things to come back online before we could test anything.

The Photon SDK gave us a lot of trouble when we tried to wire up the group chat functionality. The documentation was sparse in places and getting multiple agents to communicate reliably through the service took more debugging than we expected.

Multi-agent orchestration turned out to be harder than any of us anticipated. Making sure agents didn't duplicate work, coordinating task handoffs between them, and keeping the shared world state consistent when all were acting at the same time required a lot of careful thinking about locking, state management, and race conditions. Getting one agent to do something is straightforward. Getting multiple to work together without stepping on each other is a fundamentally different problem.

Accomplishments that we're proud of

We got a full end-to-end system working where iMessage actually controls what happens inside Minecraft. You can send a message in a group chat, the agents understand it, coordinate, and execute it in-game. We also built a multi-agent setup where different roles handle different parts of the task, instead of just one bot doing everything. On top of that, we created a full end-of-game experience where the world reacts to what you did, including a pig delivering a reward package, which ties into a real API integration.

Thanks :)