ROBO BRIDGE MCP

Browser robot arm demo. Build structures with an AI Agent through WebMCP tools. Place bricks, plan structures and more!

Comment

Inspiration

I loved building with lego for years, and wanted to see what it would look like if a robot could build with bricks in an interactive simulation. Thought this was a bit ambitious, it is buggy in areas, but hopefulyl it shows what future interactive games could look like with WebMCP.

What it does

ROBO BRIDGE MCP is a browser-based robotics simulator built around a robot arm and a shared construction environment.

Through WebMCP, the AI can inspect the workspace, plan structures, control the robot, place bricks, request more materials and react to changes made by you in the scene.

There are over 31 native WebMCP tools available for controlling the robot, planning, placing the bricks, and modifying the environment.

When testing it out, ask your agent to explore the tools, and iterate with it to build different structures. I was suprised when i asked it for a checkerboard pattern and rubiks cube that it nailed it!

How I built it

The simulator runs entirely in the browser using Three.js in a static website with no server-side processing.

A shared build area, inventory and revision state are used by both the Human and the AI agent. You decide what to build, and the agent plans on how to build it and which tools to use. The WebMCP tools are used to trigger code that drives the robot movement to place bricks.

I tried to get the agents to do the full robot joint commands and it took too long, so it essentially controls the TCP and gripper of the robot, and inverse kinematics handles the rest to position the robot joints within its workspace.

Challenges

The hardest part was getting the robot arm to build structures AND adapt if the user would place a brick in the middle of its plan.

I also had to make the interaction fast enough for a natural live demonstration while keeping placement validation deterministic. So a real robot arm wouldnt move that quickly, but you can play around with the speed settings in the menu. (Note some larger builds may take a while, even with 1 second brick pick-and-place cycle time, the planning and number of bricks will greatly increase the time the build takes)

What I learned

WebMCP is awesome! This live demo game opens up a lot of possibilities. Hopefully i can get it to build larger buildings, streets and towns later on. You could also potentially control a real robot arm through the browser. Or even its own dedicated web game, where people can connect their agents to it to build cool stuff.

Built With

Share this project:

Updates

Submission history