CrisisLens

An AI powered command center that transforms chaotic emergency audios, videos, images into clear, real time intelligence so first responders can act faster when lives are on the line.

Comment

Inspiration

CrisisLens was born from witnessing the chaos during emergencies. Growing up, I often saw news about wildfires, building fires, floods, and major accidents, and how first responders struggled to manage the flood of information coming from emergency calls, videos, images, and social media. I realized that in those critical first minutes especially during fast-spreading fires human lives were at stake simply because no one could process everything fast enough. That became my mission. I wanted to build a system that doesn’t just respond, but truly understands a crisis in real time giving emergency teams a clear, unified picture so they can act decisively and save lives. CrisisLens is more than a project it’s a commitment to turning overwhelming chaos into actionable clarity, and ensuring that when fires break out, every critical second counts.

What it does

CrisisLens is a multi-role, AI-powered emergency response platform that transforms chaotic incident reports into actionable intelligence, enabling faster and more informed decisions during critical emergencies. It brings together citizens, dispatchers, and responders through a unified, intelligent workflow. Key features include:

  1. Citizen Reporting (Text, Audio, Image, Video):
    Citizens can quickly report emergencies using text, audio recordings, images, or videos. Optional location data can be attached automatically. Submissions are designed to be stress-free, and users receive calming confirmation feedback once their report is successfully sent.
    AI Role: Each report is immediately analyzed using multimodal AI to extract structured information, including incident type, hazards, location, and affected population.

  2. AI-Powered Incident Analysis:
    CrisisLens applies advanced AI to every submission to:

    • Identify the type and severity of the incident
    • Detect hazards (fire, flood, accident, etc.)
    • Estimate the number of people affected
    • Flag contradictions across multiple reports (e.g., conflicting eyewitness inputs)
    • Calculate a priority score for rapid triage
      Impact: Converts fragmented, chaotic information into a unified, actionable overview for dispatchers.

  3. AI Recommendations for Dispatchers:
    Based on incident type, hazards, severity, and people affected, the system generates recommended resource allocation strategies. This guidance ensures that first responders are dispatched efficiently and appropriately.
    Impact: Reduces human decision-making time in emergencies and helps save lives.

  4. Dispatcher Dashboard & Command Center:
    Dispatchers can view a real-time operational dashboard with:

    • Metrics and trends for all active incidents
    • AI reasoning behind each incident’s priority and contradictions
    • Resource management and assignment tools
    • Activity feeds for all ongoing responses
      Impact: Provides complete situational awareness for informed, time-sensitive decision-making.

  5. Incident Merging & Contradiction Handling:
    When multiple reports describe the same incident, the system intelligently merges them and highlights conflicting information. Dispatchers can then resolve inconsistencies with confidence.
    Impact: Prevents duplication, ensures accuracy, and reduces cognitive overload during crises.

  6. Responder Dashboard & Field Reporting:
    Responders see only incidents assigned to them, with all relevant AI insights, and can submit field updates, including status changes, images, or other evidence.
    Impact: Maintains up-to-date ground truth and allows dynamic adaptation of the response strategy.

  7. Real-Time Incident Monitoring:
    Dispatchers and responders track incidents continuously with live status updates, elapsed time, deployed resources, and activity feeds.
    AI Role: AI continuously evaluates incoming updates to adjust priority scores and flag emerging hazards.
    Impact: Ensures that evolving emergencies are managed proactively, not reactively.

  8. Unified Multi-Role Coordination:
    CrisisLens connects citizens, dispatchers, and responders in a single ecosystem. From submission to resolution, every update flows seamlessly across roles, powered by AI analysis and prioritization.
    Impact: Reduces delays, avoids miscommunication, and accelerates emergency response.

Category: CrisisLens falls under Smart Use of Data & AI, leveraging advanced AI technologies for multimodal understanding, real-time reasoning, and actionable recommendations in life-critical scenarios.

How we built it

CrisisLens combines advanced AI, cloud technologies, and modern web frameworks to deliver real-time emergency intelligence:

  • Google Cloud Gemini 3 API: Used for multimodal understanding of text, audio, images, and video. Extracts incident type, hazards, location, people affected, detects contradictions, and generates AI-powered dispatch recommendations.
  • Frontend: Built with React.js, Redux Toolkit, Tailwind CSS, and Bootstrap for a responsive, role-based interface for citizens, dispatchers, and responders.
  • Backend: Implemented with Node.js and Express.js, handling APIs for input ingestion, incident management, AI integration, and real-time updates.
    Database: MongoDB provides scalable storage for incidents, reports, and user data.
  • File Storage: AWS S3 is used for secure, reliable storage of uploaded images, videos, and audio files from citizens and responders.

By integrating Gemini 3 AI with real-time infrastructure, CrisisLens transforms chaotic emergency reports into actionable intelligence, enabling faster, life-saving decisions.

Challenges we ran into

  • Multimodal Input Analysis: Processing text, audio, images, and video from citizen reports in real time was challenging. Ensuring accurate extraction of incident type, hazards, and affected population required fine-tuning AI pipelines.

  • Contradiction Detection: Handling conflicting information from multiple reports demanded robust AI reasoning to flag inconsistencies without delaying dispatch decisions.

  • Priority Scoring & Recommendations: Designing an AI system that could reliably prioritize incidents and suggest resource allocation involved balancing severity, hazard type, and affected people.

  • User Experience for Stressful Situations: Creating a calm and intuitive interface for citizens under emergency stress, while providing dispatchers and responders with actionable intelligence, required careful UX design and iterative testing.

  • Performance Optimization: Ensuring the backend could handle multiple simultaneous incident submissions and AI analysis without lag was a significant technical challenge.

Accomplishments that we're proud of

  • Real-Time Multimodal AI Analysis: Successfully implemented AI pipelines that process text, audio, images, and video in real time, extracting actionable insights from chaotic citizen reports.

  • AI-Powered Incident Prioritization & Recommendations: Developed an intelligent system that scores incident severity, flags contradictions, and generates dispatch guidance, enabling faster, data-driven decisions.

  • Seamless Multi-Role Coordination: Built a unified platform connecting citizens, dispatchers, and responders, ensuring smooth communication and up-to-date situational awareness during emergencies.

  • Efficient File Management with AWS S3: Integrated secure and scalable storage for images, audio, and video uploads, ensuring reliability and easy access for field reporting and incident analysis.

  • Intuitive User Experience Under Stress: Designed interfaces that are calm, minimal, and easy to use for citizens during high-stress situations, while providing dispatchers and responders with clear, actionable intelligence.

  • Scalable Frontend and Backend Architecture: Built using React.js, Redux Toolkit, Tailwind CSS, Node.js, Express.js, and MongoDB, ensuring responsiveness, reliability, and efficient handling of multiple simultaneous incidents.

  • Positive Early Testing Feedback: Initial testing with simulated emergency scenarios showed that dispatchers and responders could make faster, more informed decisions, demonstrating the system’s real-world impact.

What we learned

  • Deepening AI Expertise: Working with the Gemini 3 model for multimodal understanding (text, audio, images, video) expanded our knowledge of AI reasoning, contradiction detection, and priority scoring in real-time scenarios.

  • Real-Time Incident Management: Learned the challenges of building pipelines that handle multiple simultaneous reports, process data quickly, and deliver actionable insights to dispatchers and responders.

  • Frontend & Backend Integration: Strengthened skills in React.js, Redux Toolkit, Tailwind CSS, Node.js, Express.js, and MongoDB, connecting responsive dashboards with robust backend logic.

  • File Handling & Cloud Storage: Gained experience implementing AWS S3 for reliable, secure storage of audio, images, and videos, ensuring accessibility for real-time incident analysis.

  • User-Centered Design Under Stress: Learned how to design interfaces that are intuitive and calming for citizens during emergencies, while providing clear, actionable information to dispatchers and responders.

  • System Scalability & Performance: Understood the importance of optimizing backend APIs and AI pipelines to handle high traffic without latency.

  • Real-World Problem Solving: Recognized how AI and technology can have life-saving impact, and the critical need for clarity, reliability, and accuracy in high-pressure emergency systems.

What's next for CrisisLens

  • Expanded AI Capabilities: Improve incident type detection, hazard recognition, and priority scoring by fine-tuning AI models with more diverse emergency scenarios.

  • Mobile App Development: Launch iOS and Android versions to allow citizens to report emergencies and responders to update incidents on-the-go.

  • Enhanced Multimodal Support: Incorporate additional input types, such as live social media feeds, to capture incidents faster.

  • Automated Dispatcher Assistance: Add predictive features that suggest optimal resource allocation and estimated arrival times for responders.

  • Integration with Emergency Services: Collaborate with local authorities and first responder networks for real-world deployments and validation.

  • Continuous UX Improvements: Iteratively refine dashboards and reporting interfaces based on user feedback from dispatchers, responders, and citizens.

  • Community & Awareness Initiatives: Build public awareness about the platform, encouraging citizen participation to improve emergency response effectiveness.

Share this project:

Updates