Project Story
The problem: The first minutes are critical to save lives.
On October 1, 2016, a fire broke out at Morton Hall on Dartmouth College just after midnight. Firefighters from Hanover and nine other neighboring towns arrived to find a growing blaze. The building's design challenged them, because visibility on the fourth floor was almost zero. The fire burned for six hours. 67 students were displaced. Thankfully, no one died, but it was close, and it could have been worse.
The most important question is: why? Because when fire trucks arrived, they had no idea what to expect. They didn't even know how many people were there. Neither where the exits were. They had to learn the basics of the building while fighting fire, while people were trapped.
Every minute spent gathering information in a critical situation is a minute not spent on rescuing lives. In fires, those minutes are the difference between life and death.
We consulted Dartmouth Office of Safety and Security (DOSS) and the Hanover Fire Department (HFD) to understand the real problem. DOSS tells Hanover Fire the location and nature of the call, and then they assess the situation on scene. But that assessment happens without any prior information about the building?!
FlameScout solves this. Before the truck arrives, the crew knows exactly what to expect.
What we built:
FlameScout turns live fire-dispatch calls into a building-specific advisory brief on the crew's tablet before arrival.
Here's how:
- Fuse the calls. Multiple callers report different floors, different trapped people, and conflicting information. We merge them into one incident record, flag contradictions for the dispatcher, and never guess which one is right, because there should always be a human mind to decide.
- Location. CoStar property search
- Pull three layers for the same address:
- Matterport: interior floors, rooms, stairs, exits (the building's layout)
- Vexcel: aerial exposures, roof, neighboring structures
- CoStar: use, occupancy, tenants, year built, hazard clues...
- Synthesize a brief. Size-up facts, hazard flags, up to five ranked advisory considerations. Every factual claim carries a source and confidence label so firefighters can trust it and verify it.
- Show what's missing. The tablet brief arrives before the truck does. Size-up confirms and refines instead of starting from nothing.
How we built it
- Backend: LangChain deepagents orchestrator running parallel subagents for CoStar, Matterport and Vexcel. Gemini extracts facts with structured output and quote verification. A fast path writes one brief in a single model call; an agent path plans and critiques as a fallback. Fusion is pure code: there is no model, so it's auditable. Tiger Data stores timelines; MongoDB stores the incident workspace.
- Frontend: React + Vite, one WebSocket per incident. The MDT shows the confirmed address, incident type, scan age, and a 3D/2D house viewer from Matterport. Crews can mark facts correct, wrong, or outdated; their feedback trains the system for the next brief and future incidents at that building.
- Data layers: Three providers behind adapters with fixtures so the demo works offline. CoStar (3,339-property kit), Matterport (live plans), Vexcel (aerial reads by vision).
- Deployment: Docker Compose on Vultr behind Caddy with HTTPS.
What we learned
- Follow the handoff, don't replace it. DOSS describes to Hanover Fire: location and nature of the call. Our brief opens with those two items and adds detail underneath. We didn't guess at the workflow; we asked.
- Performance matters when lives are at stake. A 12-second timeout on model calls, one retry, and a 75-second run deadline kept briefs coming even when a layer was slow or a model rate-limited. We measured; we didn't assume.
- Deterministic first, model second. Routes, distances, contradictions, and place resolution are computed in code, not by a model. That way crews can audit what they're reading.
- Crew feedback is the learning signal. A building changes. A brief goes stale. Letting crews mark a fact wrong and applying it to the next incident at that building is how FlameScout learns and gets better.
Challenges
- Three data layers, three APIs, three different speeds. Matterport takes 5 seconds; Vexcel slower; CoStar is local. Running them in parallel with timeouts meant the brief was always ready, even if one layer lagged. A facts-only fallback when an agent run timed out kept the product honest.
- Bridging two roles. The dispatcher console and the MDT are different people. One event bus, two screens, one incident. Copilot questions for the dispatcher; evidence inspector for the crew.
Next steps
- Live CAD and MDT integration so briefs go to the actual tablet in the engine.
- Hydrant and water-supply data from public records.
- Structured crew-review loop so feedback trains the system in real time.

