GeoArbiter

COMPETING CLAIMS. ONE VERDICT.

Comment

Inspiration

After a disaster, emergency teams are not dealing with a lack of information. They are dealing with too much scattered information.

There may be drone footage, news video, field reports, FEMA-style documents, maps, and geospatial datasets, but someone still has to manually answer the most important questions:

What was damaged? Where did it happen? How severe is it? Do the sources agree? What damage was missed?

GeoArbiter was inspired by this problem of fragmented disaster intelligence. We wanted to build a system that does not just summarize one source, but compares multiple sources and helps analysts find a more defensible version of the truth.

What it does

GeoArbiter is a multimodal geospatial intelligence tool for disaster damage assessment.

The system ingests disaster video and damage reports, then extracts structured findings from each source. It compares video-derived observations against report claims and geospatial reference data to determine whether sources agree, disagree, or reveal missing information.

For each damaged location, GeoArbiter creates a fused intelligence object with:

damage severity facility type location and coordinates video timestamp evidence report excerpt evidence Overture/geospatial match confidence score fusion status recommended action

The frontend displays these findings on an interactive map. Each point opens a context tab showing the full evidence chain behind that location.

GeoArbiter can classify findings as:

confirmed damage unreported damage found in video unsupported report claim conflicting severity uncertain / needs manual review

The final output is a map-based analyst dashboard and a FEMA-style damage assessment report.

How we built it

We built GeoArbiter as a full multimodal pipeline.

The backend accepts a disaster video and damage report through an upload endpoint. The video is analyzed with TwelveLabs models through AWS Bedrock. Pegasus is used to generate structured video descriptions, while Marengo supports video understanding and retrieval. The report is parsed into structured claims, including entities, locations, severity levels, and supporting excerpts.

The system then performs fusion between the extracted video findings and report findings. It compares entities using semantic similarity, severity alignment, source agreement, and geospatial context. Overture Maps data is used as a reference layer to ground findings to real-world buildings, places, and roads.

The frontend is built with React, Vite, Tailwind CSS, Leaflet, and React-Leaflet. It visualizes fused findings on a map, color-codes points by severity, displays Overture reference layers, and opens a context tab for each location. The context tab shows the complete evidence chain: video, report, geospatial match, confidence score, and recommendation.

Challenges we ran into

The hardest part was deciding how to connect sources that describe the same real-world event in different ways.

Videos may mention a place verbally but not show a street sign. Reports may describe a location vaguely. Geospatial datasets may have official names that differ from common names. Damage severity may also be described differently across sources.

We had to think carefully about:

entity resolution alias matching geospatial matching missing data source disagreement confidence scoring avoiding hallucinated locations or damage claims

Another challenge was scope. Full disaster analysis is a huge problem, so we narrowed the MVP to a clear workflow: video + report + geospatial reference data → fused findings → interactive map and damage report.

Accomplishments that we're proud of

We are proud that GeoArbiter goes beyond a basic disaster map.

Instead of only showing where damage happened, it shows why the system believes that damage happened and which sources support the conclusion.

We built a working interface where each map point has a complete evidence chain. That makes the system more useful for real analysts because they can inspect the reasoning behind each finding instead of trusting a black-box output.

We are also proud of the product framing. GeoArbiter is not just a damage detector. It is a source-validation system for disaster intelligence.

What we learned

We learned that multimodal intelligence is less about adding more data and more about making different data sources speak the same language.

A video clip, a PDF report, and a map feature can all describe the same location in different ways. The real value comes from aligning those sources, measuring agreement, and making uncertainty visible.

We also learned how important provenance is. In serious geospatial and emergency-response workflows, every conclusion needs to trace back to evidence. A useful system should not only say “this building is damaged.” It should say:

“This building appears severely damaged because this video segment shows roof loss, this report mentions the same location, and this geospatial object confirms the facility.”

What's next for GeoArbiter

The next step is improving the fusion engine and expanding the system beyond the demo workflow.

Future improvements include:

live processing of larger disaster video collections stronger Overture-based entity resolution better geospatial polygon generation support for additional sources such as satellite imagery, social media posts, sensor feeds, and field inspection databases real-time query support for analysts more advanced confidence scoring and uncertainty modeling export formats for FEMA, local emergency management, and geospatial intelligence teams

Long term, GeoArbiter could become a general multimodal evidence engine for emergency response, infrastructure monitoring, and geospatial intelligence.

Built With

Share this project:

Updates