PeopleOs

Inspiration

At a 1,000+ person hackathon, venue problems live in Slack threads. A broken outlet gets no reply, a simple question about supplies takes hours to answer, and even pinning down where a problem is can take several back-and-forth messages. Meanwhile the people who could fix it are somewhere else in the building, with no idea who needs them.

We wanted something anyone can use with zero setup: call a number, say what's wrong, and the right person walks over by the right route.

What it does

PeopleOS is an AI phone line plus a live operations map for an event venue.

• Call and report. A Cartesia voice agent (Claude as the brain) takes the report, matches the location to one of 406 real places in the venue (304 hacker tables, sponsor booths, rooms, entrances), and confirms it with exactly one question: "Table 142 on the main floor, right?"
• Answers questions. Callers can also ask event questions (meals, showers, wifi rules) answered from a Moss index, and get walking directions that avoid blocked paths.
• Dispatches the right person. The incident appears on a live 3D model of the Palace of Fine Arts. PeopleOS picks a staff member by role, availability, current workload, and walking distance through the venue graph, then draws the route and buzzes their phone. Staff accept, decline (auto re-dispatch to the next best person), or complete from a mobile page.
• Reroutes around blockages. Report a blocked hallway and the path closes instantly. Every open task is re-planned, and if someone else is now closer, the task moves to them. Operators can undo with one click.
• Spots duplicates. A repeat report about the same problem increments a report count instead of creating a second task.
• Hands emergencies to humans. Injury, fire, or safety reports are never auto-dispatched. The agent escalates to a human operator and tells the caller to call 911, and the server enforces the same rule independently of the prompt.

How we built it

• Cartesia Managed Agents: the agent, five webhook tools, and the phone number, all provisioned by script. Tools call Convex HTTP actions with bearer auth.
• Convex: zones, edges, locations, staff, incidents, and assignments; the report pipeline; and live queries that update the dashboard and staff phones. Applying a blockage and re-planning every assignment happen in a single transaction.
• Bitdeer AI Model Studio: bge-m3 embeddings plus DeepSeek-V4.1-Flash for duplicate verdicts and one-line staff briefs, with every call logged with its latency.
• Moss: a search index of event facts and venue areas, queried from a small Node server with a 2-second limit and a keyword fallback.
• React Three Fiber: the 3D venue (from the organizers' map data), with incident markers, staff dots, and routes that follow the real walkable graph, including the mezzanine.
• One source of truth: the agent's location list, prompt, venue guide, and search documents are all generated from the same venue data that seeds the database, so the agent can never name a place the backend can't resolve.
• Next.js, TypeScript, Tailwind, and 100+ Vitest tests, including the exact demo scenario.

Challenges we ran into

• Latency inside a phone call. Every AI step had to be bounded: duplicate detection is capped at 4 seconds with a rule-based fallback, event search at 2 seconds with a keyword fallback, and the staff brief runs after the call.
• Moss in a serverless backend. Moss's embedding model could not load inside Convex actions, so queries run on a small search server over HTTP, and the system degrades to keyword search if it is unreachable.
• A reasoning model that ran out of tokens. The brief model spent its token budget on reasoning, so a small limit returned empty briefs. We found it by logging every Bitdeer call and raised the budget.
• Turning a map into a graph. The organizers' map has tables and walls, not walkways. We drew 18 zones and 27 walkable connections with door and stair waypoints so routes don't cut through walls.
• Rerouting without thrashing. When a blockage changes the graph, the current assignee competes with their own task removed from their workload, so an unchanged graph never reassigns a task.

Accomplishments that we're proud of

• The blockage scene: one phone call closes a hallway, and the task visibly moves to a different staff member on a new route, live on the 3D map and on both phones.
• Emergencies are handled by design: two independent layers, never auto-dispatched, never dropped even when the location is unclear.
• The voice line never fails silently: every AI dependency has a time limit and a non-AI fallback.
• Built solo during the hackathon: backend, voice agent, 3D venue, staff and ops apps, and four sponsor integrations.

What we learned

Voice is the right input for people who will never install an app, but an agent is only reliable when its world is closed. A fixed location list generated from data beat free-form extraction. And keeping dispatch as explainable rules, with AI only at the edges (listening, deduplicating, summarizing), made the system easier to trust and to test.

What's next for PeopleOS

• Onboarding other venues from a floor plan, the event's policy documents, and a staff list. The dispatch, routing, and duplicate-detection engine contains no venue names; zones and walkways are drawn by hand today, and generating them automatically is the next step.
• Timeout re-dispatch for unanswered offers, and paging the ops lead on escalations.
• A status tool for callers ("is my report being handled?") and SMS/web reporting on top of the existing channel-agnostic pipeline.
• Finer staff location (check-in or indoor positioning) instead of zone-level presence.

Try it

Call +1 681 272-0158 and say something like "the outlet at Table 142 isn't working", or ask "where can I take a shower?" Then open the live map: https://peopleos-mu.vercel.app/demo

Built With

• anthropic
• bge-m3
• bitdeer
• cartesia
• claude
• convex
• css
• deepseek
• dijkstra
• drei
• moss
• next.js
• node.js
• react
• tailwind
• three.js
• typescript
• vitest
• voiceai
• webhooks