-
-
FireBreak Home Page
-
FireBreak in action
-
GrokBot Interaction
-
Voiskey Implementation
Inspiration
During a wildfire, a single fake post can do real damage. Picture this one: a photo of flames next to a freeway, captioned "The fire jumped I-15 at Rancho Bernardo. Poway, get out now." People jam evacuation routes, flood 911, and the next real order gets ignored. County emergency comms teams are small, and the same few people are posting real orders, answering the press and watching social media. By the time they confirm a rumor is false and write a correction, the rumor has already won.
We wanted to give that team their minutes back. Firebreak is a response desk that catches the rumor, checks the evidence, drafts a correction, tests it on a simulated town, and lets the officer approve it by voice. The officer stays in control: nothing goes out without their explicit approval.
What it does
Firebreak walks a comms officer through one incident, start to finish, with a single guided button:
- Detect. A Grok Scout groups related posts into one incident and pulls out the core claim, in English and Spanish.
- Verify (Gemini). Three Gemini agents debate the actual image:
- The Prosecutor looks for evidence that the photo isn't what the post claims, and draws a box on each finding. In our scenario it reads the road sign: "101 NORTH, Scottsdale Rd. That's Arizona, not I-15."
- The Defender makes the strongest honest case that the post could be real.
- The Judge rules (False, Misleading, Unverified or True), gives a calibrated confidence, and lists the missing evidence that would change its mind, with who to ask. One click drafts that request.
- Draft (Grok). A Grok Comms Bot writes three corrections: a fact check, an action-first version, and a neighbor's voice, each in English and Spanish.
- Simulate (Grok Bots). Six native Grok Bots play 20 personas across six communities (Poway residents, Spanish-speaking Escondido parents, local influencers, retirees on Facebook, reporters, skeptics) on a 240-account social graph. Each round, every persona decides to share, question, verify, reply, retract or wait, based only on what it has actually seen.
- Edit by voice (Voiskey). The officer holds Right Alt and talks: "Use action first. Send it to Poway, the parent groups and the influencers. Do Spanish too." Voiskey types the cleaned note, and Gemini turns it into edited copy, a chosen framing and target communities.
- Approve and send. The officer approves the exact text on screen. The correction enters the town as real messages and the map turns from orange (rumor) to blue (correction).
- Respond. The Broadcaster answers residents' questions using only the approved copy and verified facts.
You can also hand it your own image: the full Gemini debate runs live on it.
How we built it
Backend. Python with FastAPI. A Server-Sent Events stream pushes every episode update to the browser. Pydantic validates every agent output, so the UI never renders malformed model output.
Gemini (vision, structured output, voice).
- The Prosecutor uses Gemini's image understanding with spatial output: each finding comes back as
box_2d = [ymin, xmin, ymax, xmax]on a 0 to 1000 grid. Against the real sign at roughly[336, 583, 491, 811], Gemini returned[333, 581, 492, 812]. - The Prosecutor and Defender run in parallel; the Judge reads both and returns strict JSON that drives the interface directly.
- Gemini also transcribes the backup mic recording, parses the officer's commands from the voice note, and drafts evidence requests.
- The Gemini result becomes a versioned evidence packet that the Grok desk imports. The community Bots learn about it only through messages they actually receive.
Grok (native Grokbots). Seven official Grokbots routines, triggered by webhooks: six community Bots and one Comms Bot (Scout, Drafter, Broadcaster). Instead of stuffing a big task into the webhook, which gets truncated, each trigger carries a short-lived, scoped token. The Bot fetches its task over HTTPS and posts its result back to a callback-only server exposed through a Cloudflare tunnel. A SQLite job store checks every result against its episode, round, community and a hash of what that Bot saw. A round only advances when all six communities have answered. Each persona's belief stays in $[0, 1]$ and can move at most
$$|b_{t+1} - b_t| \le 0.35$$
per round, so no single message flips anyone instantly.
Voiskey. Voiskey is a system-wide speak-to-type desktop app with no public API, so we integrated it through the interface itself. Its dictation hotkey types a cleaned note into the Editor, and its translation hotkey writes straight into the Spanish post. Voice is the officer's main control, not an add-on.
Safety by design.
- Approval is bound to a SHA-256 hash of the exact copy. If the text changes, it has to be approved again.
- Voice can propose a correction but never approve one.
- Everything is a fictional exercise: no real posting, no real accounts.
- What-if branches fork the town at the decision point, so outcomes can be compared. Finished runs can be saved and replayed.
Frontend. Plain HTML, CSS and JavaScript, no build step. The town is drawn on a canvas: node colors blend between rounds, and every real message delivery travels its edge. The evidence reveal plays once: a scan line while Gemini works, then each box appears with its finding, and the verdict stamps in. Reduced motion is respected throughout.
Testing. 87 automated tests cover the Bot task and callback flow, round barriers, exact-copy approval, branching, replay, and the Gemini and Voiskey wiring.
Challenges we ran into
- Voiskey has no API. We checked its site and found no SDK or developer docs. We redesigned the Editor around speak-to-type instead, which turned out to be a more natural experience for an officer under pressure.
- Gemini free-tier limits. Our key allowed 5 requests per minute and 20 per day per model, with no Google Search grounding. The newest Flash models were also overloaded that day, with calls taking over 40 seconds. We added a chain of backup models that kicks in when a model is overloaded or out of quota, plus an on-disk cache so the demo never waits on the API. A search-based check for earlier copies of the photo is built but needs a paid key, so the Prosecutor says so honestly instead of inventing a match.
- Honest AI cost us a finding. Our demo image is an illustration, and Gemini correctly kept pointing that out. We kept that honesty for uploaded images and only tell Gemini about the stand-in for the scripted scenario.
- Two teammates, two architectures. One of us built the Gemini and Voiskey side; the other built the native Grokbots society. Merging them meant agreeing on a JSON evidence contract and an intervention contract, then wiring both into one desk.
- Native Bot plumbing. Webhook bodies get truncated, tunnel hostnames change on every restart, and every routine has to allowlist the exact callback host. The task-pull design solved truncation; the rest is careful setup.
- Cross-platform bugs. Spanish text read on Windows came out garbled until we forced UTF-8 everywhere.
Accomplishments that we're proud of
- Gemini finds the real evidence in the image and draws a box on it, not just a text verdict.
- The town's behavior comes from Bot decisions and actual message deliveries, not a hard-coded probability curve.
- Voice is a real control, and the officer always has the final say.
- The whole flow fits on one screen and one button, which matters when every minute counts.
What we learned
- Adversarial agents (Prosecutor vs Defender, then a Judge) give more balanced rulings than a single model asked "is this true?"
- Structured output is what turns an LLM into a reliable UI component.
- Free-tier limits and outages are part of the design problem. Caching and fallbacks belong in version one.
- In a crisis tool, what the officer must approve matters as much as what the AI can do.
What's next for Firebreak
- Connect to real feeds instead of the scripted scenario, and pilot with a county comms team during fire season.
- Turn on Google Search grounding to find earlier copies of recycled photos.
- Measure the Judge's accuracy on a labeled fact-check set. The script is ready; it needs a paid key to run.
- More languages beyond English and Spanish, matched to each community.
Log in or sign up for Devpost to join the conversation.