Disaster-Response-Data-Hub-Google-Cloud-Fivetran-Elastic-

Inspiration

In disasters, minutes matter. During Hurricane Katrina, it took days to get a full picture of county-level needs. Maria knocked out communications in Puerto Rico for weeks. Sandy left New York scrambling to coordinate shelters. We asked ourselves: what if real-time data pipelines and AI could give emergency managers a trusted dashboard within minutes, not days? That was the seed for the Disaster Response Data Hub.

What it does

Our platform ingests live disaster feeds (FEMA alerts, NOAA weather, local emergency reports) into Google Cloud through Fivetran. The data is cleaned, stored in BigQuery, and indexed in Elastic. Emergency managers or insurers can open a Looker Studio dashboard—or ask our AI agent powered by Vertex AI Gemini—questions like: “Which Florida counties are at highest hurricane risk this week?” “Where are the shelters most needed?” “How many severe events are affecting New Jersey right now?” Instead of static reports, decision-makers get real-time intelligence, hybrid search, and AI-driven answers.

How we built it

Data pipeline: Fivetran connector stub pulls disaster data → Cloud Storage → BigQuery. Analytics layer: BigQuery dataset + views structure risk data. Search & reasoning: Elastic hybrid search + Vertex AI Gemini Q&A agent. Visualization: Pre-built Looker Studio dashboard (KPIs, geo heatmaps, trendlines). Demo polish: Seeded with Katrina, Maria, and Sandy so the dashboard lights up immediately.

Challenges we ran into

Harmonizing schemas from FEMA, NOAA, and local feeds. Designing IAM roles that were both secure and hackathon-fast. Making Elastic hybrid search and Vertex AI actually “talk” to each other in a way that felt seamless. Keeping latency low—our pipeline refreshes every 3 minutes, which was trickier than it sounds.

Accomplishments that we're proud of

Built an end-to-end pipeline from ingestion to AI answers in <48 hours. Preloaded largest FEMA responses in U.S. history for instant credibility. Created a hybrid agent that can answer county-level questions with context, not just keyword search. Designed a demo where the “wow moment” comes in under 90 seconds.

What we learned

Cloud-native tools like Fivetran + BigQuery reduce data engineering time by 80%. Elastic’s hybrid search is powerful when paired with an LLM—it grounds AI in real, trustworthy data. The difference between a “hack” and a “solution” is storytelling: showing how this helps a real emergency manager or NGO worker on the ground.

What's next for Disaster-Response-Data-Hub-Google-Cloud-Fivetran-Elastic-

Real API integrations with FEMA, NOAA, and USGS feeds. Predictive ML models in BigQuery ML to forecast severity before it hits. Mobile-first dashboard so first responders can use this in the field. NGO partnerships: American Red Cross, World Central Kitchen, and insurers for proactive risk management. Scalability: Multi-region deployment so response teams in Florida, California, or Puerto Rico can each have their own localized hub.

Built With

• elasticsearch
• elasticsearch-rest-api
• fivetran
• fivetran-connector-sdk
• google-bigquery
• google-cloud
• google-cloud-bigquery-api
• python
• sql