Commotion

home page

Inspiration

We wanted to see what happens when you change one thing in a crowded place, without scripting the answer. Commotion gives each simulated person their own goals and lets the outcomes come from their decisions.

What it does

We start by describing a place in one sentence, like "a busy amusement park with two rides, food stalls, and a gift shop." Commotion searches the web for it, fills gaps with labeled assumptions, and builds a low-poly 3D scene with about 40 people and six to ten places.

Type in any event to simulate it: a store promotion, a closure, or even an alien! Each event runs an agent for each person in parallel with Jev to simulate 30 seconds and is recorded to view. The timeline is interactive, and you can click any person or store and look into what their action's are and how they are influenced by the prompt.

How we built it

Frontend: React, Next.js, React Three Fiber, Drei.

Backend: A Cloudflare Worker with SQLite Durable Objects for session state, WebSockets, and recorded timelines.

Baseten (GPT-OSS-120B) handles language. It is an open-weight mixture-of-experts model with 117B parameters, of which 5.1B are active per token. That keeps latency and cost low enough to call on every setup and every event. It does two jobs:

Setup: It reasons over live search results and produces the environment definition: places, products, service points, and the starting population. Every field gets a provenance tag (researched, inferred, or assumed).

Events: Events are created by user input, including but not limited to discounts, stock changes, closures, or threats.

Jev (typesafe/jev on Cloudflare Workers AI) handles user behavior in response to events. Jev cannot generate text. Instead, it reads a state and answers typed questions (i.e. choice, score, or yes/no probability) in one parallel pass, returning probabilities and a confidence value with each answer.

For each decision, we send the person's stats: goals, remaining budget, hunger, fatigue, current activity, recent experiences, and known events. For every second, Jev evaluates the actions the engine says are valid right now (move, browse, join queue, purchase, receive service, eat, rest, flee, leave, re-enter, wait). We then store the returned probabilities and visualize them by showing how users move as time progresses.

Engine: Deterministic code owns movement, queues, checkout, stock, service capacity, and metrics. Prices and stock are integers, and a purchase commits exactly once even when two people reach for the last unit.

Challenges we ran into

Late responses. Every decision carries a setup ID, run ID, decision ID, and the state revision it was based on. We drop it after a reset, regeneration, or newer decision. Before applying one, we recheck target, budget, stock, and capacity.

Bounded inference. Each segment allows at most four concurrent Jev requests, 240 attempts, and 180 seconds. Simulated time waits for pending decisions. If a limit is hit, the segment finishes on existing activities and says so. A failed call keeps the person's current behavior and is logged as a failure, never as a Jev decision. Playback and scrubbing make no model calls.

Global events with local effects. Every person learns every event immediately, including people who left. Only validated targets change mechanically, so a gate change updates matching passengers and no one else.

Generated worlds. The model proposes a semantic layout, and deterministic code validates and repairs it. It works with generic shapes when custom assets are missing.

Accomplishments that we're proud of

The same event produces different reactions because each person decides from their own goals, budget, and history. The full loop works end to end: research, generate, event, individual decisions, recording, replay, comparison. It also runs on non-retail places, such as a ride modeled as a timed service, so it isn't just stores with new labels.

What we learned

Each layer does one job. The language model turns open-ended text into structure. Jev picks from a constrained set of options. Code guarantees what must be true. Calibrated probabilities gave us something concrete to inspect when behavior looked odd. Jev only knows the state it's handed, so what goes into that state decides what a person can know.

What's next for Commotion

Larger populations, memory and relationships between people, more environment types, better generated visuals, and tools to run and compare many scenarios at once.

NOTE: Hack the North's provided Baseten workspace crashed in the middle of our development. Our research agent and world generation relies on Baseten, so, world generation usually does not work. Enter "Yorkdale" to try out our pre-generated world.