KHABAR — AI Crisis Intelligence & Response Orchestrator

Project Overview

KHABAR is a decentralized, AI-driven emergency response orchestration platform designed for Pakistan. It connects citizens, emergency coordinators, public alerts, maps, resource management, and AI decision-making into one complete civic response workflow.

Citizens can report emergencies through a Flutter mobile app using text, photo, voice, and GPS. The backend receives the report, creates a structured crisis signal, and sends it through a 4-stage AI agent pipeline. The AI verifies the incident, analyzes its severity, plans the response, executes emergency tools, updates resources, sends alerts, and displays the outcome on a React-based admin dashboard.

The goal of KHABAR is not only to collect emergency reports, but to transform raw citizen signals into verified, prioritized, and actionable emergency response decisions.

Problem

In Pakistan, emergency response is often delayed because crisis information is scattered across multiple sources such as phone calls, social media posts, local reports, news updates, and informal communication channels.

During floods, fires, road accidents, traffic blockages, heatwaves, medical emergencies, or infrastructure failures, citizens may report the problem quickly, but authorities often struggle to:

Verify whether the report is real.
Identify the exact crisis type and location.
Decide which incident is most urgent.
Estimate the severity and possible impact.
Allocate rescue teams, ambulances, fire trucks, or supplies.
Inform citizens through timely public alerts.
Coordinate multiple agencies from a single dashboard.
Track what actions were taken and what changed after execution.

Because of this, emergency response can become slow, fragmented, and difficult to monitor.

KHABAR solves this problem by using AI to connect citizen reporting, incident verification, emergency planning, resource dispatch, alert generation, and coordinator visualization in one system.

Why This Problem Matters

Emergency response is a direct civic and public service issue. In cities like Islamabad and Rawalpindi, incidents such as urban flooding, road accidents, fires, heatwaves, and road blockages can quickly affect thousands of people.

A small delay can lead to:

More people being trapped or affected.
Delayed ambulances and rescue teams.
Poor coordination between departments.
Public panic due to lack of verified information.
Duplicate or fake reports wasting resources.
No clear audit trail of what decision was taken and why.

KHABAR matters because it gives citizens an easy way to report emergencies and gives coordinators an intelligent dashboard to verify, prioritize, and respond faster.

Solution

KHABAR provides an AI-powered emergency response workflow with two main user sides:

Citizen side — Flutter mobile app for reporting and receiving alerts.
Coordinator side — React admin dashboard for monitoring, decision-making, and action execution.

The system works like this:

Citizen Report / Automated Signal
        ↓
FastAPI Backend
        ↓
RawCrisisSignal Created
        ↓
4-Agent AI Pipeline
        ↓
Detection → Analysis → Planning → Execution
        ↓
Database + Alerts + Resources + Dashboard
        ↓
Citizens and Coordinators receive live updates

KHABAR turns unstructured emergency reports into structured civic intelligence.

How KHABAR Works

1. Citizen Reporting

Citizens can submit emergency reports using the Flutter mobile app.

Supported input types include:

Text report
Photo report
Voice report
GPS location

The system supports local language reporting, including:

English
Urdu
Roman Urdu
Punjabi

Example citizen report:

Nullah Lai overflow ho rahi hai, Murree Road Rawalpindi band ho gayi!

This makes the system accessible for ordinary citizens because they do not need to write formal English. They can report in the language and style they naturally use.

2. Multi-Source Input

KHABAR accepts crisis signals from multiple sources.

Text Input

Text reports are submitted through:

POST /report/text

The text is passed directly to the Detection Agent with GPS metadata.

Image Input

Photo reports are submitted through:

POST /report/image

The image is analyzed by Vision AI. The system can detect visual evidence such as flooding, fire, road blockage, damaged vehicles, or infrastructure damage.

Voice Input

Voice reports are submitted through:

POST /report/voice

The voice note is transcribed using speech AI. The transcription is then converted into a crisis signal and passed into the same AI pipeline.

Automated Feeds

KHABAR can also monitor automated external sources such as:

Open-Meteo weather data
TomTom traffic data
Google News RSS
External emergency signals

These automated signals can trigger proactive crisis detection when dangerous conditions are detected, such as heavy rainfall, heatwave conditions, or abnormal traffic slowdown.

Signal Lifecycle

When a report is submitted, KHABAR creates a structured signal called RawCrisisSignal.

A signal includes:

Signal ID
Source type
Raw content
Timestamp
Latitude and longitude
Vision result, if image was submitted
Speech result, if voice was submitted

The incident is first saved with this status:

PROCESSING

Then the background AI pipeline starts.

Final status can become:

PIPELINE_COMPLETE

or:

REJECTED

depending on whether the report is verified and processed successfully.

AI Multi-Agent Pipeline

KHABAR uses a 4-stage sequential AI pipeline:

Detection Agent → Analysis Agent → Planning Agent → Execution Agent

Each agent performs a specific role and passes structured output to the next agent.

Stage 1: Detection Agent

The Detection Agent reads the raw citizen report or automated signal.

It performs:

Crisis classification
Location extraction
Priority assignment
Confidence scoring
Spam or invalid report detection
Verification against external data when available

It identifies crisis types such as:

Urban flooding
Fire
Road accident
Building collapse
Heatwave
Medical emergency
Road blockage
Infrastructure failure

It assigns priority from:

P1 = Critical
P2 = High
P3 = Medium
P4 = Low
P5 = Informational

If the signal is fake, irrelevant, or unverified, it can be rejected.

Stage 2: Analysis Agent

The Analysis Agent studies the real-world impact of the verified incident.

It estimates:

Severity score
Affected population
Stranded vehicles
Nearby hospitals
Hospital ETA
Critical infrastructure at risk
Bilingual public warning summary

This stage helps coordinators understand how serious the incident is and what kind of emergency response may be required.

Stage 3: Planning Agent

The Planning Agent creates an ordered emergency response plan.

It recommends actions such as:

Dispatch rescue teams
Allocate supplies
Broadcast public alerts
Reroute traffic
Create emergency tickets
Update incident status
Query emergency SOP knowledge base

The plan includes:

Response strategy
Required units
Target agencies
Priority of each action
Recommended sequence

Example target agencies:

Rescue 1122
WASA
Traffic Police
NDMA
Edhi Foundation
Hospitals
Fire services

Stage 4: Execution Agent

The Execution Agent executes the recommended plan using emergency response tools.

It can:

Dispatch rescue teams
Allocate supplies
Send bilingual alerts
Close roads
Set detour routes
Create emergency tickets
Query SOP knowledge base
Update incident status

Every execution produces a before/after state so coordinators can see exactly what changed.

Example:

Before:
rescue_team: 0
ambulance: 0
alerts_sent: 0
closed_roads: []

After:
rescue_team: 3
ambulance: 2
alerts_sent: 2
closed_roads: ["Murree Road"]
detour_routes: ["Via Expressway"]

This makes the system transparent and auditable.

Emergency Action Tools

KHABAR includes seven emergency tools used by the Execution Agent.

1. Dispatch Rescue Team

Deploys rescue teams or emergency units from the resource inventory.

2. Allocate Supplies

Reserves supplies such as medical kits, food packs, or equipment.

3. Broadcast Alert

Sends bilingual Urdu and English public alerts using Firebase Cloud Messaging.

4. Update Traffic Route

Marks roads as closed and sets alternate detour routes.

5. Create Emergency Ticket

Creates an inter-agency emergency support ticket.

6. Query Knowledge Base

Looks up relevant emergency SOPs and response protocols.

7. Update Incident Status

Updates the incident status, such as from PROCESSING to PIPELINE_COMPLETE.

Admin / Coordinator Dashboard

KHABAR includes a dedicated React-based Admin / Coordinator Dashboard for emergency dispatchers, civic authorities, and response teams.

While the Flutter app is for citizens, the admin dashboard is for coordinators who need to monitor, verify, manage, and respond to incidents.

The dashboard acts as a real-time civic emergency command center.

Purpose of the Admin Dashboard

The dashboard solves a major coordination problem. Emergency coordinators need one place where they can see:

What incident was reported
Where it happened
How serious it is
Which priority level it has
What the AI agents decided
Which resources are available
Which resources have been dispatched
What alerts were sent
What roads are closed
What actions still require manual control

KHABAR provides this through a single dashboard interface.

Dashboard Features

1. Live Map

The dashboard includes a live map that shows:

Incident markers
Resource markers
Rescue team locations
Priority-based incident colors
Closed roads
Detour routes
Nearby emergency resources

This helps coordinators quickly understand where the crisis is happening and which resources are nearby.

2. Resource Manager

The Resource Manager shows live emergency resource inventory.

It tracks:

Ambulances
Rescue teams
Fire trucks
Dewatering pumps
Emergency supplies
Resource status
Available quantity
Assigned incident ID

When a resource is dispatched, its status changes from available to deployed or en route. This prevents duplicate assignment and improves coordination.

3. Agent Panel

The Agent Panel displays the output of each AI agent.

Coordinators can inspect:

Detection Agent result
Analysis Agent result
Planning Agent result
Execution Agent result
AI reasoning trace
Allocated resources
Tool execution logs
Before/after system state

This makes KHABAR explainable. The AI is not a black box because the coordinator can see why a priority was assigned and why an action was recommended.

4. AI Command Chatbot

The dashboard includes an AI Command Assistant for emergency coordinators.

Instead of manually filling forms, a dispatcher can type natural language commands such as:

Dispatch Rescue 1122 to SIG-123 immediately

or:

Send flood alert to sector G-10

or:

Mark SIG-123 as resolved

The chatbot reads current incidents and resources, understands the command, and executes backend actions through:

POST /admin/chat

Supported actions include:

Dispatch resources
Send public alerts
Reroute traffic
Create emergency ticket
Update incident status
Add resource
Clear/reset incidents

This gives KHABAR a human-in-the-loop control system where AI assists the coordinator, but the coordinator remains in command.

5. Stats Grid

The Stats Grid shows quick KPI cards such as:

Total incidents
Active resources
Alerts sent
Pipeline success rate

This gives emergency managers a quick overview of the current situation.

6. Case Tracker

The Case Tracker shows the distribution of incidents by priority.

It tracks:

P1 critical incidents
P2 high-priority incidents
P3 medium incidents
P4 low-priority incidents
P5 informational incidents

This helps authorities focus on the most urgent cases first.

7. Alerts Panel

The Alerts Panel shows public warnings sent through the system.

It displays:

Alert content
Alert time
Incident reference
Urdu message
English message
Firebase notification history

This helps coordinators confirm that citizens were informed.

8. Situation Summary

The dashboard includes an AI-generated situation summary.

It summarizes the overall emergency landscape so coordinators can understand the situation quickly without reading every incident manually.

Manual Control and Human Oversight

KHABAR does not remove human decision-making. It supports human coordinators with AI assistance.

Coordinators can manually trigger actions using:

POST /action/execute

They can also use natural language through:

POST /admin/chat

This human-in-the-loop design is important because civic systems require safety, accountability, and human judgment.

The AI recommends and assists, but the coordinator can review, trigger, modify, or override actions.

Public Alert System

KHABAR sends real-time bilingual push notifications using Firebase Cloud Messaging.

Alerts are sent in:

Urdu
English

Alert types include:

Flood alert
Urban flooding alert
Fire alert
Road accident alert
Building collapse alert
Heatwave warning
Medical emergency alert
Road blockage alert

All Flutter app users can subscribe to the public alert topic:

khabar_public_alerts

If Firebase credentials are missing, the system falls back to simulated delivery logs so the pipeline does not crash.

Maps and Location Intelligence

KHABAR uses map and geocoding services to improve location-based emergency response.

It supports:

Text location to coordinates
GPS coordinates
Incident markers
Resource markers
Hospital lookup
ETA calculation
Closed road visualization
Detour route visualization

The geocoding fallback chain is:

Google Maps API → Local Pakistan City Dictionary → OpenStreetMap Nominatim → Islamabad Default Location

This makes the system more reliable if one location service fails.

Database and Resource Tracking

KHABAR uses Supabase PostgreSQL to store:

Incidents
Resources
Agent traces
Before/after states
Allocated units
Alert records

The system also includes an in-memory fallback. If Supabase is unavailable, the backend can still continue using thread-safe local memory instead of crashing.

Resources include:

Ambulances
Rescue teams
Fire trucks
Dewatering pumps
Medical supplies
Emergency units

When a resource is dispatched, KHABAR updates its status and assigns it to a specific incident.

Offline and Resilience Features

Emergency systems must work even when internet access is unstable. KHABAR includes a local offline AI fallback.

The AI fallback chain is:

AIML API → Local GGUF Model → Hardcoded JSON Fallback

Local models include:

Qwen GGUF
Gemma GGUF

The local model can power offline citizen chat through:

POST /local-chat

If the local model is not available, the system uses keyword-based fallback responses.

This ensures the system can still provide emergency guidance even when cloud AI is unavailable.

API Endpoints

KHABAR includes API endpoints for reporting, monitoring, actions, resources, chat, and logs.

Important endpoints include:

GET /health
POST /report/text
POST /report/image
POST /report/voice
GET /incidents
GET /incident/{id}
GET /resources
POST /resources/add
POST /action/execute
GET /logs/{id}
GET /geocode
POST /chat
POST /local-chat
POST /admin/chat
GET /live-news

These endpoints connect the mobile app, backend, AI pipeline, dashboard, database, maps, alerts, and admin control system.

AI Component

KHABAR uses AI in several meaningful ways.

1. Multilingual Text Understanding

AI understands citizen reports written in English, Urdu, Roman Urdu, and Punjabi.

2. Vision AI

Photo reports are analyzed to detect crisis evidence such as flooding, fire, damaged vehicles, road blockage, or disaster scenes.

3. Speech AI

Voice reports are transcribed and converted into structured emergency signals.

4. Multi-Agent Reasoning

The 4-agent pipeline performs:

Detection → Analysis → Planning → Execution

This is the core intelligence of the system.

5. RAG-style SOP Lookup

The Planning Agent can query emergency response knowledge base content to recommend better actions.

6. AI Command Assistant

The admin dashboard includes a chatbot that allows coordinators to issue natural language commands.

7. Offline AI

Local GGUF models provide AI fallback when cloud AI is unavailable.

Why AI Is Appropriate

AI is appropriate because emergency reports are often:

Unstructured
Noisy
Multilingual
Time-sensitive
Incomplete
Location-dependent
Difficult to verify manually at scale

AI helps KHABAR by:

Understanding natural language reports.
Extracting crisis type and location.
Verifying report relevance.
Assigning priority.
Estimating severity.
Generating response plans.
Creating bilingual alerts.
Assisting coordinators through chat commands.
Reducing response delay.

Without AI, this system would only be a reporting form. With AI, it becomes an emergency response orchestration platform.

Who Will Benefit

Citizens

Citizens can report emergencies easily and receive real-time safety alerts.

Emergency Coordinators

Coordinators get a live dashboard to monitor incidents, resources, AI decisions, and response actions.

Rescue Agencies

Rescue teams receive structured incident details and priority-based dispatch information.

Government and Civic Organizations

Public-sector teams can improve transparency, accountability, and service delivery.

Vulnerable Communities

People in flood-prone, high-risk, or low-connectivity areas can receive faster warnings and support.

Civic and Social Impact

KHABAR can create strong civic impact by:

Reducing emergency reporting delays.
Improving incident verification.
Supporting faster resource dispatch.
Helping citizens receive timely alerts.
Improving coordination between agencies.
Increasing transparency in emergency response.
Creating traceable AI decision logs.
Supporting bilingual communication.
Helping authorities prioritize critical cases.
Making emergency response more data-driven.

In the long term, KHABAR can help cities become safer, more responsive, and more resilient.

Innovation

KHABAR is innovative because it does not stop at reporting.

It moves from:

Report → Verify → Analyze → Plan → Execute → Alert → Visualize

Most civic reporting apps only collect complaints or incidents. KHABAR goes further by using AI agents to convert reports into actions.

Its key innovation is the combination of:

Citizen reporting
Multi-source input
Multi-agent AI reasoning
Emergency tool execution
Resource allocation
Public alerts
Admin command dashboard
Agent trace transparency
Offline AI fallback

This makes KHABAR a complete civic emergency intelligence system.

Technical Implementation

KHABAR is implemented using a modular architecture.

Layer	Technology
Citizen App	Flutter / Dart
Admin Dashboard	React + Vite
Backend	Python FastAPI
AI Pipeline	Multi-agent LLM orchestration
Primary AI	AIML API with Gemini 2.5 Flash
Offline AI	Qwen / Gemma GGUF with llama-cpp-python
Database	Supabase PostgreSQL
Database Fallback	In-memory Python store
Notifications	Firebase Cloud Messaging
Maps	Google Maps + OpenStreetMap
Weather	Open-Meteo
Traffic	TomTom
News	Google News RSS
API Documentation	FastAPI Swagger

Prototype Status

The current prototype includes:

Flutter citizen reporting app.
Text, image, and voice reporting.
GPS-based incident reporting.
FastAPI backend.
4-agent AI pipeline.
React admin dashboard.
Live map visualization.
Resource manager.
AI command chatbot.
Firebase alert integration.
Supabase incident and resource storage.
Local offline AI fallback.
Manual action execution.
Agent trace logs.
Before/after state comparison.
External integrations for maps, weather, traffic, and news.

This demonstrates a complete end-to-end civic AI workflow.

Feasibility

KHABAR is feasible because it uses available technologies and modular services.

The prototype already demonstrates the core flow:

Citizen report → Backend API → AI pipeline → Execution tools → Database update → Alerts → Dashboard visualization

Each component can be improved independently:

Mobile app can support more languages.
Backend can scale on cloud.
Database can connect to real agencies.
AI pipeline can use stronger models.
Dashboard can add more operational controls.
Alerts can expand to SMS and WhatsApp.

Scalability

KHABAR can scale from a hackathon prototype to a city-level or national civic platform.

Future scaling options include:

Add more cities across Pakistan.
Integrate with Rescue 1122, NDMA, WASA, hospitals, and traffic police.
Add SMS alerts for citizens without smartphones.
Add WhatsApp reporting.
Add more regional languages.
Add predictive disaster risk analysis.
Connect with IoT sensors, CCTV, or drone feeds.
Add role-based access for agencies.
Create open data dashboards for public transparency.
Train models on local emergency datasets.
Use historical incident data for heatmaps and risk prediction.

Because KHABAR is modular, it can support more agencies, more cities, and more emergency types over time.

User Experience

KHABAR is designed for two user groups.

Citizen Experience

Citizens can:

Submit reports easily.
Use text, image, or voice.
Attach GPS location.
Track incident status.
Receive alerts.
Use online or offline AI chat.

Coordinator Experience

Coordinators can:

View live incidents.
Inspect AI decisions.
Manage resources.
Dispatch teams.
Send alerts.
Track priority cases.
Use AI command chatbot.
Review trace logs.

This dual-interface design makes KHABAR useful for both public participation and official response coordination.

Demo Scenario

A possible demo scenario:

A citizen reports urban flooding near Nullah Lai using Roman Urdu text and GPS.
KHABAR creates a crisis signal and saves the incident as PROCESSING.
Detection Agent verifies the report and classifies it as URBAN_FLOODING.
Analysis Agent estimates high severity, affected population, and nearest hospital ETA.
Planning Agent recommends dispatching Rescue 1122, sending a public alert, and closing a risky road.
Execution Agent dispatches rescue teams, sends bilingual FCM alerts, creates a road closure, and updates the incident status.
Citizens receive alerts.
The admin dashboard shows the incident on a live map with resources, priority, trace logs, and before/after state.
The coordinator uses the AI chatbot to manually send an additional alert or mark the incident resolved.

Future Roadmap

Future improvements include:

Real-time agency integration.
SMS and WhatsApp alert support.
More accurate traffic and routing intelligence.
Predictive crisis risk maps.
Citizen trust scoring.
Duplicate incident clustering.
CCTV or drone image confirmation.
More local languages.
Role-based admin accounts.
Public transparency portal.
Historical analytics dashboard.
Mobile app deployment for Android users.

Conclusion

KHABAR is an AI-powered civic innovation project designed to improve emergency response in Pakistan.

It allows citizens to report crises through accessible channels, uses AI to verify and prioritize incidents, creates response plans, executes emergency actions, sends bilingual alerts, and gives coordinators a live dashboard for transparent decision-making.

By combining citizen participation, artificial intelligence, emergency response tools, public alerts, maps, resource management, and human-in-the-loop coordination, KHABAR can help communities become safer, more informed, and more resilient.