Inspiration

Neptune North asked for systems that keep working towards a shared goal when nobody is in charge and the network can't be trusted. We wanted to make that idea fun to watch and useful beyond a demo.

Termites build huge mounds with no architect: each one reacts to what it can see and what its neighbours leave behind. So we built Termite Builders, a swarm of robots that build a 256-tile mosaic together, with no central server telling anyone what to do.

The twist is that the same brain runs a trading desk. Press Reveal and every word on screen changes: robots become trading engines, tiles become orders, the mosaic becomes a book of work. Nothing else changes. The coordination problem is the same, and so is the code. The same pattern fits drone light shows, warehouse robots, or a team of AI agents splitting up a big task.

What it does

  • 6 robots, 256 tiles, one shared goal. Each robot owns a zone of the mosaic and decides for itself what to place next.
  • No hub. Robots gossip with whichever neighbours they can reach and merge what they hear into their own copy of the board.
  • Break it on purpose. Judges scan a QR code and become the storm: cut radio links, steal a robot, jam the network, black everything out, or knock tiles off.
  • It still finishes, with 0 duplicate tiles. A cut-off robot keeps building its own zone from what it last knew. When it reconnects, its work merges back in.
  • Augmented reality. Point a phone or laptop camera at a table and the build appears on it in 3D, with ants walking the tiles, grass around the edges, and your hand drawn in front of the scene.

How we built it

The brain is pure Python with no dependencies, so the same file runs on the server and in the browser through Pyodide. Each robot keeps an append-only log of events (placed tile, claimed cell, heard from robot X). Merging two logs is a set union:

$$ \text{state}_i \leftarrow \text{state}_i \cup \text{state}_j $$

So it doesn't matter in what order messages arrive, or whether they arrive twice. Every robot ends up with the same board.

Avoiding duplicates without a leader. Every cell has exactly one owner, so two robots never place the same tile. If a robot goes silent, its neighbours don't jump straight in. They wait a timeout $T$ and then take over only the tail of its unfinished work:

$$ \text{take over cell } c \iff \text{owner}(c)\ \text{silent for} > T \ \wedge\ c \notin \text{seen}_i $$

Each robot runs in one of three modes, chosen from how many neighbours it can still hear: connected, cautious or alone.

The server (FastAPI + WebSockets) only runs the simulation and streams snapshots. It never makes a decision for a robot. All the words come from a labels file, which is how the termite and trading versions share one codebase.

The AR view is raw WebGL2, with no framework. A homography maps the board onto the table:

$$ \begin{bmatrix} u \ v \ w \end{bmatrix} = H \begin{bmatrix} x \ y \ 1 \end{bmatrix} $$

From $H$ we recover the camera pose, so tiles have depth and ants stand on top of them. The ants walk with a tripod gait driven by distance travelled, so their feet don't slide. Hands are found by comparing each frame with a photo of the empty table.

Testing: 30 automated tests, including a robot that stays cut off for the whole run and still completes its zone.

Challenges

  • The tradeoff between being consistent and staying available. A partitioned robot can either keep working or wait for agreement, not both. Tiles are safe to place blind because ownership prevents clashes. Real trades are not, so the trading version marks which actions are safe to do while disconnected and which must wait.
  • Deciding when a robot is "gone". If neighbours give up on it too early, you get duplicates. If they wait too long, the build stalls. Tail takeover after $T$ was the balance we found.
  • Making AR feel real in a browser. Early versions were flat and frantic. We rebuilt the view in 3D with a proper gait and depth, and added hand detection so the scene sits behind your hand.
  • Venue networking. Phones couldn't reach the laptop at first, because the QR code pointed at localhost and the venue Wi-Fi isolates devices from each other. We now detect the laptop's real network address and run the demo on a phone hotspot.

What we learned

  • Agreement doesn't need a leader if the data structures are designed so that merging always gives the same answer.
  • Stigmergy, coordinating by reading the shared work rather than by messaging, is surprisingly robust: the board itself is a message.
  • Some actions can't be made safe offline. Knowing which ones matters more than any algorithm.
  • A demo people can break themselves is far more convincing than slides that say "resilient".

What's next

  • Run real robots on separate phones over peer-to-peer connections.
  • Repair knocked-off tiles on their own.
  • Test it with a real trading simulator.

Built With

Share this project:

Updates

Submission history