Ground-Control

Inspiration

Picture Mo, the safety lead at a 15,000-person festival with 300 volunteers and one radio channel. On a 38°C afternoon someone collapses at the water station. The call comes over the radio for a few seconds, through static, and then it's gone. Who's closest? Who's trained? Did anyone else already call it in?

Festivals see a steady stream of medical presentations, around 12 per 1,000 attendees, which at this size is roughly

$$15{,}000 \times \frac{12}{1{,}000} \approx 180 \text{ a day.}$$

Reviews of disasters like Astroworld (2021) and the Manchester Arena Inquiry keep pointing to the same thing: information doesn't reach the right people fast enough. We wanted to give the person in charge a clear picture and a fast, safe way to act, without taking the decision away from them.

What it does

Ground Control is a phone app for festival crews, with a different view for volunteers, location leads and the safety lead.

Report by voice. A volunteer taps the mic and says what they see. AI turns it into a structured incident: type, location, urgency and what's needed. The volunteer confirms it before it's sent.
The right skills, nearest first. Suggestions are matched to the incident: first aid for a medical emergency, a working-with-children check (with crowd-management support) for a lost child, security and de-escalation for a fight, RSA at the bars.
A human approves everything. The AI never acts on its own. Every suggested response goes to the safety lead to approve, edit or reject. If Mo is tied up, the zone's location lead can step in after 2 minutes, or 30 seconds for a critical incident, and Mo is told. Every decision is logged with a name.
A spoken brief that fits the walk. The responder's phone reads a brief aloud as they move. Under 100 m it's one sentence, up to 300 m it adds the location and what to expect, and further out they get the full picture.
A live map of the whole site. Zones, crew, incidents and responders' routes, updating as people move, so Mo can see who's where at a glance.
Location only while checked in. A volunteer's position is shared only while they're checked in for a shift. Check out and they disappear from the map.
Off-site alerts. If someone leaves the festival grounds while still checked in, after a minute both they and the safety lead are notified, so nobody is counted as on shift when they've gone.
Duplicate detection, decided by people. Two reports of the same fight are flagged as possibly related. Mo compares them side by side and decides; they're never merged automatically.
Staffing that keeps every zone covered. Mo sets how many people with each skill every zone needs, adjusts it through the day, and gets alerted when a zone falls short. AI suggests who to move, using the nearest people who can leave without creating a new gap. One tap applies surge presets like "Concert at the Lawn Stage", or Mo can just type what he needs.
Simulation mode. Judges can try it without a festival. It has a seeded roster of 300 volunteers moving on a live map, guided stories (heat collapse, a fight reported twice, a busy safety lead) and a speed-up clock.

How we built it

App: Expo (React Native) with Expo Router and TypeScript, so one codebase runs on iOS, Android and the web. On iPhone it uses Apple's native tab bar, with a custom floating dock elsewhere. Zustand holds state, and Reanimated and Gesture Handler drive the motion.
AI: the Claude API through a small Hono server, so the API key never reaches the phone. Each model does what it's best at:
- Haiku 4.5 structures incidents and writes the spoken briefs (fast and cheap).
- Sonnet 5.5 writes suggested responses, checks for duplicate and related incidents, and proposes placements.
- Opus 5.5 (low effort) turns Mo's plain-English staffing requests into concrete changes.
Code decides who, AI decides how. Code picks the 3–5 eligible candidates by skill, distance and availability. The AI only chooses among them, and the server rejects anyone not on that list. Every response is validated with Zod plus semantic checks, and every call has a timeout and a non-AI fallback: a manual form, rule-based flags or a template brief.
Voice: the device's built-in speech recognition, and the browser's on the web.
Deployment: the web build and AI routes run on Vercel. On a desktop the app sits inside an iPhone-sized frame so judges see exactly what a phone shows. The public AI endpoints only answer our own site and are rate-limited.

Challenges we ran into

Keeping a human in control without slowing things down. "AI only suggests" can mean nobody acts when the one person who can approve is busy. Time-boxed escalation to the location lead (2 minutes, or 30 seconds for critical) solved that while keeping a named human on every decision.
Trusting AI output in a safety tool. Structured outputs, schema validation, candidate allow-lists and a fallback for every call mean a bad or slow AI response degrades the app gracefully instead of breaking it.
Speech recognition. We chose the device's built-in recognition over Whisper. It's a little less accurate, but it's free, works with poor festival signal, and keeps working if our server goes down. A human reviews every transcript before it becomes an incident, so the trade-off is safe. A production deployment could revisit this, but built-in recognition is normally good enough.
Getting it into judges' hands. Expo Go stopped opening other people's projects in May 2026, so we shipped the app as a web build that runs in an iPhone frame on desktop and full-screen on phones.

Accomplishments that we're proud of

The whole loop works end to end, live: voice report, AI write-up, duplicate check, suggested responder, human approval, a spoken brief, and the responder's dot walking to the incident.
No path where AI acts alone, and no path where a failed AI call leaves someone stuck.
27 unit tests over the parts that matter for safety: matching, escalation windows, duplicate pre-filtering, brief distances, coverage gaps and placement validation.

What we learned

In safety tools, the most important design decision is where the human sits. AI is most useful doing the legwork (structuring, matching, drafting) and least useful making the call.
Letting code constrain the AI (who's eligible, what's valid) makes it far more reliable than asking it to get everything right.
Picking the right model per job matters: small and fast where speed counts, bigger where judgement does.

What's next for Ground Control

Pattern-based staffing suggestions: spotting trends such as a rising number of altercations and suggesting more security or de-escalation-trained volunteers before it escalates.
Real multi-device sync, so every crew member's phone shares one live picture (the demo runs on one device with a role switcher).
Proper sign-in and integration with Fieldday's rostering.
Native builds through TestFlight and Google Play, and offline-first reporting for patchy festival signal.
Optional Whisper transcription at scale, where accuracy is worth the cost.