NEXORA

Build Beyond Limits!!!

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for NEXORA

🚨 CrisisMesh AI — Project Story

About the Project

💡 Inspiration

During emergencies such as fires, floods, accidents, and mass-casualty events, the biggest challenge is often not the absence of information, but the inability to process and act on it quickly enough.

Emergency reports can arrive as panicked phone calls, text messages, images, or mixed-language descriptions such as:

"Building mein aag lagi hai, log phanse hue hain."

A human operator may need to understand the language, determine the severity, identify people at risk, locate the incident, find suitable emergency units, and decide which resources should be dispatched. Every manual step can introduce valuable delays.

This inspired us to build CrisisMesh AI — From Chaos to Coordinated Response: an AI-powered emergency intelligence and dispatch platform designed to transform unstructured emergency reports into prioritized, actionable information for command-center operators.

Our goal was not simply to build another AI chatbot. We wanted to build an operational system where AI understands the emergency, algorithms determine its priority, geospatial logic finds suitable nearby resources, and real-time communication keeps the command center synchronized.

🎯 What We Built

CrisisMesh AI creates an end-to-end pipeline:

Emergency Report → AI Triage → Structured Intelligence → Priority Scoring → Resource Matching → Dispatch → Real-Time Operations

A citizen can submit an emergency through text, voice, or an image. Gemini AI analyzes the available information and extracts important details such as:

  • Emergency category
  • Severity
  • People potentially at risk
  • Hazards
  • Required emergency resources
  • Language
  • Sentiment/urgency
  • Recommended action
  • Confidence

Instead of allowing an AI model to independently decide everything, we designed a deterministic priority engine on top of the AI output.

This creates a more predictable and auditable decision-support workflow.

🧠 How We Built It

1. Multimodal Emergency Ingestion

The first layer accepts different types of emergency reports:

  • 💬 Text
  • 🎙️ Voice/audio
  • 📷 Images

The system is designed to understand English, Hindi, Urdu, and Roman Urdu, allowing people to communicate naturally instead of requiring them to translate their emergency into a specific language.

For example:

"Building mein aag lagi hai, log phanse hue hain."

can be interpreted as an urgent building-fire incident rather than simply being treated as raw text.

2. Gemini AI Triage

We integrated Google's Gemini models through the @google/genai SDK.

Instead of requesting an unrestricted natural-language response, CrisisMesh asks Gemini to produce a strictly structured JSON response.

The AI extracts information such as:

Category: FIRE
Severity: CRITICAL
People at Risk: 20+
Hazards: Fire, Smoke, Trapped Occupants
Required Resources: Fire Truck, Ambulance
Sentiment: Urgent

This structured approach makes the AI output easier for the rest of the application to process.

We also implemented a model fallback chain, allowing the system to attempt alternative Gemini models when the preferred model is unavailable.

⚖️ 3. Deterministic Priority Engine

One of the most important design decisions we made was not to rely entirely on AI for emergency prioritization.

AI can understand context extremely well, but operational ranking should be consistent and explainable.

Therefore, CrisisMesh calculates a deterministic priority score using:

$$ P = S + V + H + U $$

where:

  • (S) = Severity contribution
  • (V) = People-at-risk contribution
  • (H) = Hazard-density contribution
  • (U) = Urgency/sentiment contribution

The maximum contributions are:

Factor Maximum Contribution
Severity 40
People at Risk 30
Hazard Density 18
Urgency 12
Total 100

The final score is bounded to the operational range used by the platform.

This means that two similar incidents are ranked consistently rather than depending on small variations in an AI-generated response.

🗺️ 4. Geospatial Resource Matching

After understanding an emergency, CrisisMesh needs to answer another critical question:

"Which available emergency resources should respond?"

We built a geospatial matching system using Leaflet, OpenStreetMap, and the Haversine distance formula.

For an incident and responder unit, we calculate the spherical distance between their coordinates:

$$ d = 2R\arcsin\left( \sqrt{ \sin^2\left(\frac{\Delta lat}{2}\right) + \cos(lat_1)\cos(lat_2) \sin^2\left(\frac{\Delta lon}{2}\right) } \right) $$

The system then estimates an arrival time based on distance and a configurable response speed.

This allows CrisisMesh to recommend resources based on factors such as:

  • Distance
  • Availability
  • Resource type
  • Required equipment
  • Incident requirements

For example, a critical building fire can automatically surface nearby fire trucks and ambulances instead of making an operator manually search through the fleet.

🚒 5. One-Click Dispatch

Once recommended resources are displayed, operators can dispatch one or multiple units directly from the incident interface.

Resources can transition through operational states such as:

AVAILABLE
     ↓
ASSIGNED
     ↓
EN_ROUTE
     ↓
ARRIVED
     ↓
COMPLETED

The corresponding incident and resource information is updated as the response progresses.

This creates a complete operational lifecycle rather than stopping at AI classification.

📡 6. Real-Time SSE Event Architecture

Another major challenge was ensuring that multiple command-center consoles remain synchronized.

Instead of repeatedly polling the server, CrisisMesh uses Server-Sent Events (SSE) to push operational updates to connected clients.

Events include:

INCIDENT_CREATED
INCIDENT_UPDATED
DISPATCH_UPDATED
AUDIT_LOG

For example:

Operator A dispatches a fire truck → Server broadcasts event → Other connected command consoles receive the update instantly.

This allows the dashboard, incident queue, fleet status, and operational log to remain synchronized without requiring page refreshes.

🔊 7. Tactical Audio Alerts

We also implemented browser-based Web Audio API alerts.

Critical incidents can generate distinctive tactical sounds, while dispatch events can trigger additional notification tones.

This is important because a command-center operator may not always be looking at the exact incident panel when a critical event occurs.

The goal is to make the interface feel like an operational environment rather than a conventional dashboard.

🤖 8. AI Command Assistant — SITREP

We also wanted operators to interact with the system using natural language.

CrisisMesh includes an AI command assistant capable of answering operational questions such as:

"Show all critical incidents."

"Which incident has the highest priority?"

"Which emergency resources are available?"

"Show me the active fire incidents."

"Which incident needs attention first?"

Instead of forcing operators to manually search multiple screens, the assistant acts as a command-center copilot for querying live operational information and generating situation reports.

🏗️ Technology & Architecture

We built the platform using a modern web stack:

Frontend

  • React 19
  • TypeScript
  • Tailwind CSS
  • Motion
  • Recharts
  • Lucide Icons

Backend

  • Node.js
  • Express.js
  • TypeScript

AI

  • Google Gemini
  • @google/genai
  • Structured JSON/schema-based AI output
  • Multimodal analysis

Geospatial

  • Leaflet.js
  • OpenStreetMap
  • Haversine distance calculation

Real-Time

  • Server-Sent Events (SSE)

Data & Cloud

  • Firebase/Firestore-ready architecture
  • In-memory demo state for fast hackathon operation
  • Firebase authentication/hosting configuration

Browser Intelligence

  • Web Audio API for tactical alerts

🧩 Challenges We Faced

Building CrisisMesh AI presented several interesting challenges.

Challenge 1: AI Reliability

We initially considered allowing AI to directly determine the emergency priority. However, we realized that relying entirely on an LLM for operational ranking could produce inconsistent results.

Our solution:

We separated AI understanding from deterministic prioritization.

Gemini extracts the relevant emergency information, while our scoring engine calculates the final priority using predefined mathematical weights.

Challenge 2: Multilingual Emergency Reports

Emergency communication is rarely perfectly structured. People may switch between languages or use Roman Urdu/Hindi while describing a stressful situation.

For example:

"Building mein aag lagi hai, log phanse hue hain."

Our solution:

We designed the AI ingestion layer to understand multilingual and mixed-language descriptions directly rather than forcing users through a translation step first.

Challenge 3: Selecting the Right Emergency Resource

Finding the nearest vehicle isn't enough.

A nearby police cruiser is not necessarily the correct resource for a building fire.

Our solution:

We combine:

Incident requirements + resource type + availability + geographic distance

to produce ranked resource recommendations.

Challenge 4: Real-Time Synchronization

Emergency operations require multiple screens to remain synchronized.

A dispatch made on one operator console should immediately appear on another.

Our solution:

We implemented an SSE event bus so important operational events are pushed to connected clients in real time.

Challenge 5: Designing for Human Operators

We didn't want CrisisMesh to become an "AI makes every decision" system.

Emergency response requires human judgment.

Our solution:

CrisisMesh is designed as a human-in-the-loop decision-support platform.

AI provides intelligence and recommendations, deterministic algorithms provide consistent prioritization, and the operator remains responsible for the final dispatch decision.

📚 What We Learned

Building CrisisMesh AI taught us that developing an AI application is not simply about connecting an LLM to a frontend.

We learned how to combine different engineering disciplines into one operational workflow:

  • Multimodal AI
  • Structured AI outputs
  • Deterministic algorithms
  • Geospatial mathematics
  • Real-time event systems
  • API architecture
  • Interactive data visualization
  • Emergency resource optimization
  • Human-in-the-loop AI design

One of our biggest lessons was:

AI should not replace deterministic systems where consistency and accountability matter.

Gemini is excellent at understanding messy, unstructured information. Mathematical algorithms are better suited for predictable ranking and distance calculations. Real-time event systems are better suited for synchronizing operational state.

CrisisMesh brings these capabilities together instead of trying to make one technology solve the entire problem.

🚀 What Makes CrisisMesh AI Different

Many emergency applications focus on reporting an incident.

CrisisMesh focuses on the complete journey:

REPORT
  ↓
UNDERSTAND
  ↓
PRIORITIZE
  ↓
LOCATE
  ↓
MATCH
  ↓
DISPATCH
  ↓
MONITOR
  ↓
AUDIT

The core idea is simple:

Don't just tell emergency teams what happened. Help them understand what matters most and what resources can respond.

🔮 What's Next

CrisisMesh AI is designed as a foundation that can evolve beyond the current prototype.

Our future roadmap includes:

🛰️ Offline Disaster Mesh

Integrate LoRaWAN/P2P networking for areas where cellular infrastructure is unavailable.

🚁 Autonomous Drone Response

Use drones for rapid aerial assessment and live situational awareness before responders arrive.

📊 Predictive Resource Staging

Use historical incidents, weather, population density, and geographic risk factors to predict where emergency resources should be positioned before an incident occurs.

🌐 Larger Multi-Agency Networks

Connect fire, police, medical, rescue, disaster-management, and shelter resources into a unified emergency intelligence network.

❤️ Final Vision

CrisisMesh AI was built around one question:

What if the time between an emergency being reported and the right response being mobilized could be reduced from minutes to seconds?

Our answer is CrisisMesh AI.

By combining Gemini multimodal intelligence, deterministic priority scoring, geospatial resource matching, real-time SSE communication, and human-controlled dispatch, we aim to create a smarter emergency command layer that helps responders spend less time processing information and more time responding to people who need help.

🚨 CrisisMesh AI

From Chaos to Coordinated Response.

Built With

Share this project:

Updates

Submission history