Inspiration

Crowd control at events still relies on static barriers and human staff. We asked: what if the barrier could think, move, and adapt?

Inspired by how border collies herd sheep — no central command, just local perception and coordination — we set out to build a swarm of robots that guides crowds the same way.


What It Does

Vee is a fleet of wheeled robots that autonomously coordinate to guide and redirect foot traffic.

Each unit:

  • moves independently
  • communicates with its neighbors
  • uses onboard LLM inference to make real-time decisions
  • operates with no central controller

The V-shaped inflatable structure on top is soft, safe, and highly visible. The swarm can:

  • form lines
  • encircle areas
  • dynamically redirect flow in response to crowd density

How We Built It

Each robot runs on a Raspberry Pi 5 with a local language model handling perception-action decisions.

The system is divided into two layers:

Low-level control layer

  • written in C
  • handles motor driving
  • handles sensor polling
  • executes real-time movement
  • provides deterministic latency and direct hardware access

High-level decision layer

  • written in Python
  • handles LLM inference
  • manages inter-robot messaging over a peer-to-peer mesh
  • translates high-level decisions into C control calls

We first built a Godot simulation to validate swarm behavior before deploying to hardware.

The physical robot uses:

  • a Mecanum wheel chassis for omnidirectional movement
  • a 3D-printed V-bracket
  • two inflatable columns
  • embedded LED strips

Challenges We Ran Into

1. On-device LLM latency

LLM inference latency on-device is real. The robot has to keep moving while waiting for a decision.

We solved this by having the robot continue following its last valid instruction until a new one arrives. This mirrors how real biological agents behave.

2. Honest simulation

Getting realistic simulation was also hard. It was tempting to cheat with global state, but we enforced a strict rule:

each simulated robot could only perceive what its real-world counterpart could physically sense


Accomplishments That We're Proud Of

  • A clean separation between the C control layer and the LLM decision layer
  • Continuous robot responsiveness, because motion control never waits on inference
  • A fully decentralized swarm with no single point of failure
  • The ability for the rest of the swarm to regroup even if one robot is removed
  • A practical inflatable structure that is:
    • soft enough to be safe in crowds
    • tall enough to be visible
    • collapsible to the size of a water bottle

What We Learned

Swarm coordination does not require individually smart robots. It requires robots that share the right information at the right time.

We also learned that LLMs are especially useful not for executing known tasks, but for handling situations outside their training distribution, such as:

  • unexpected crowd behavior
  • robot failures during operation
  • novel spatial configurations

What's Next for Vee

Next steps for Vee include:

  • larger swarms
  • real event deployments
  • tighter hardware-simulation co-development so behavior validated in Godot transfers directly to the physical fleet

We are also exploring the same platform for:

  • agricultural herding
  • warehouse dynamic zoning
  • any domain where the boundary itself needs to move

Built With

Share this project:

Updates