Caligo Dynamics

“Firefighters often enter fires blindly. unable to see victims or hazards. This AI powered swarm drone uses ROS2, PX4, Kiro, and Bedrock to detect fires, locate civilians, and improve response.”

Comment

Inspiration

We are living in an era where humanity is curing diseases, exploring space, and building artificial intelligence that can write code in seconds. Yet, despite all our technological leaps, when a fire breaks out, we are still losing lives in the most primal, terrifying way possible. My faculty dean, Atakan Yalçın a brilliant mentor who shaped my vision and his family lost their lives in a tragic hotel fire in Bolu. It was a place I had stayed at over a dozen times. The distance from the fire station meant that by the time help arrived, it was too late. I couldn't stop thinking about how fast a fire consumes, and how utterly helpless we are in the face of it. I saw this same helplessness during the recent wildfires. People were running toward raging infernos with 0.5-liter water bottles. Bravery shouldn't require blinding oneself in smoke. The core problem in firefighting isn't a lack of courage it's a lack of time and a lack of vision. To truly understand the problem, I stood in a Seattle Fire Department and spoke with Julien, a hero with 22 years of experience. "Even with thermal imaging, I can see fire and bodies, but I can't see trip hazards or holes in the floor. We search by maintaining contact with the wall because of no visibility. Smoke inhalation kills, and with poor visibility, we often can't tell the way out." First responders are forced to make life-or-death decisions in the dark. Caligo Dynamics exists to change that. Caligo Dynamics is an AI-powered autonomous drone swarm system for emergency response in zero-visibility environments. Caligo is Latin for mist, darkness, fog exactly the conditions where our system excels. Before a single firefighter steps through a burning doorway, autonomous drones have already been inside mapping the floor plan, locating trapped civilians, identifying fire temperature and handed the incident commander a complete tactical picture in under 45 seconds.

What it does

Every year, firefighters die in zero-visibility conditions. According to the National Fire Protection Association: 60-80 firefighters die annually in the United States alone Zero visibility is a contributing factor in 40% of firefighter fatalities Response time is critical - every minute counts when lives are at stake Traditional drones fail in smoke, fog, and darkness The Impact Caligo Dynamics addresses this life-or-death challenge by providing: Situational Awareness - Drones equipped with thermal and LiDAR sensors can "see" through smoke and darkness, providing real-time intelligence to ground teams Rapid Deployment - Strategic pod placement across cities enables <45 second deployment to any incident location AI-Powered Decision Making - Amazon Bedrock analyzes sensor data in real-time, identifying trapped civilians, optimal evacuation routes, and fire suppression priorities Coordinated Response - Multi-drone swarms work together, covering more ground and providing redundancy if one unit fails Real Wildfire Integration - NASA FIRMS satellite data provides real-time wildfire detection across the USA, enabling proactive deployment Social Impact This isn't just about technology - it's about saving lives: Firefighters get critical intelligence before entering dangerous buildings Trapped civilians are located faster using thermal imaging Incident commanders make better decisions with real-time AI analysis Communities benefit from faster, more effective emergency response Wildfire response teams get early detection and real-time fire tracking The system is designed to be deployed in urban fire departments, wildfire response teams, and disaster relief organizations worldwide. The reality is staggering: First responders are forced to make life-or-death decisions in the dark.

How we built it

The system uses a four-layer architecture that separates concerns while enabling real-time communication: LAYER 4: Command Dashboard (Next.js 15 + React + Leaflet) │ - Real-time tactical map with drone positions
│ - NASA FIRMS wildfire data integration
│ - Click-to-deploy interface
│ - AI feed with Bedrock recommendations
│ - Live metrics and fleet management
│ → Polls Flask bridge every 2 seconds

↕ HTTP/WebSocket

│ LAYER 3: Cloud Integration (AWS Lambda + S3 + CloudWatch) │ │ - AWS Lambda: Decision logging and Bedrock API handler │ │ - Amazon S3: Stores all autonomy decisions as JSON logs │ │ - CloudWatch: Real-time metrics (SeverityLevel, Confidence)│ │ - CloudTrail: Security and compliance logging │ │ → Serverless, scales automatically

↕ HTTP/REST API

│ LAYER 2: AI Brain (Flask + Amazon Bedrock + ROS2 Bridge) │ │ - caligo_server.py: Flask API server │ │ - Amazon Bedrock integration (Claude 3 Haiku) │ │ - ros2_bridge.py: Reads ROS2 topics → POSTs to Flask │ │ - Endpoints: /api/status, /api/bedrock, /api/firms |

↕ ROS2 Topics

│ LAYER 1: AWS Kiro Autonomy (ROS 2 + Bedrock Integration) │ │ - bedrock_autonomy.py: Kiro + Bedrock ROS 2 node │ │ - Real-time sensor data → AI decisions │ │ - Decision types: PROCEED, HOVER, RETREAT, EVACUATE │ │ - Severity levels: LOW, MEDIUM, HIGH, CRITICAL │ │ - Confidence scores: 0.0-1.0 with reasoning |

Robotics & Simulation: ROS 2 Humble - Robot Operating System for sensor fusion and control Gazebo Fortress - Physics-based 3D simulation environment PX4 Autopilot - Industry-standard drone flight controller Custom ROS2 packages - caligo_sensors, caligo_autonomy, caligo_sim

AI & Cloud: Amazon Bedrock - Claude 3 Haiku for tactical decision-making AWS Lambda - Serverless Bedrock API handler and decision logging Amazon S3 - Incident log storage and decision archives Amazon CloudWatch - System metrics and monitoring NASA FIRMS API - Real-time wildfire satellite data AWS Kiro Integration - ROS 2 + Bedrock autonomy decision layer

Frontend & Dashboard: Next.js 15 - React framework with App Router TypeScript - Type-safe development Leaflet - Interactive mapping library Tailwind CSS - Utility-first styling shadcn/ui - Component library

Backend: Flask - Python web framework for bridge server Python 3.10+ - Core language for AI integration WebSocket - Real-time bidirectional communication

Challenges we ran into

The drone that wouldn't move. The first simulation used the X3 UAV model downloaded from Gazebo Fuel. It looked right a proper quadcopter mesh, correct dimensions, loaded cleanly. But when the autonomous controller published velocity commands to /caligo/cmd_vel, nothing happened. The drone sat on the ground. The commands were being published, the bridge was forwarding them, the topic was live and the drone didn't move. The problem was that the X3 model is a 3D mesh with no flight controller plugin attached. It has no physics. We had to scrap it entirely and build a custom quadcopter model from scratch with a gz-sim-multicopter-control-system plugin, four rotors with proper inertia tensors, and PID gains tuned by hand. Two days lost before the drone moved a single meter.

The state overwrite bug. The dashboard polls /api/status every two seconds to keep drone telemetry live. That's fine for battery and altitude. It's fatal for mode. When Bedrock parsed a natural language command "recall UNIT-01 and send UNIT-02 to Fire Zone Alpha" the mode update happened in React state. Two seconds later the poll fired, Flask returned the last known modes from _sim_state, and the AI command was silently erased. The drone appeared to respond, then snapped back. The fix was a commandedModesRef a useRef<Map<number, string>> that lives outside the render cycle. AI-commanded modes get locked in the map for 30 seconds. The poll handler checks the map before writing. Without that lock, every AI command in the system was effectively a no-op.

NASA FIRMS returning 23,000 fires. The FIRMS API call with area=world&days=1 returns every active fire detection on the planet roughly 23,000 data points. Rendering all of them on a Leaflet map caused the browser to freeze for 8–10 seconds on load. The fix was a combination of geographic filtering (USA bounding box), confidence filtering (high and nominal only, dropping low-confidence detections), and a 1-hour cache in firms_client.py so the API isn't hit on every dashboard load. The filtered result is around 200 fires renderable in under 200ms.

Accomplishments that we're proud of

Phase 1: Foundation (Weeks 1-2) Set up Ubuntu 22.04 development environment Installed ROS 2 Humble and Gazebo Fortress Created basic drone model with PX4 integration Established project structure and build system.

Phase 2: Sensor Integration (Weeks 3-4) Implemented LiDAR obstacle detection Added thermal imaging for fire detection Created visibility sensor for smoke/fog simulation Built sensor fusion node to combine all data streams

Phase 3: Autonomy & AI (Week 5) Integrated Amazon Bedrock for decision-making Developed obstacle avoidance algorithms Created autonomous navigation system Implemented multi-drone coordination

Phase 4: Dashboard Development (Weeks 7-8) Built Next.js + Bedrock command center interface Integrated Leaflet for interactive mapping Created real-time data visualization Added click-to-deploy functionality

Phase 5: Real Data Integration (Week 9) Integrated drone simulation with dashboard Connected ROS2 telemetry to backend Linked AI decisions to interface Unified deployment commands across system Synchronized real-time drone telemetry streams Integrated wildfire data with dashboard

Phase 6: Polish & Testing (Week 9) Improved dashboard responsiveness and stability Optimized real-time telemetry data updates Fixed system integration and UI bugs Tested autonomous drone deployment scenarios Validated AI decision outputs accuracy Conducted end-to-end system testing

What we learned

  1. Multi-Layer Architecture is Essential Building a robotics system taught me that separation of concerns isn't just good practice - it's survival. By splitting the system into simulation, AI brain, and dashboard layers, I could: Develop and test each layer independently Replace components without breaking the whole system Scale different parts based on load (e.g., multiple dashboard instances) Debug issues in isolation The ros2_bridge.py was a game-changer - it decoupled ROS2 from the web stack entirely.
  2. Real-Time Systems Require Careful State Management Managing state across three layers with different update frequencies was challenging: ROS2 publishes at 10-50 Hz Flask polls ROS2 at 2 Hz Dashboard polls Flask at 0.5 Hz I learned to use refs in React to avoid stale closures, implement proper cleanup in useEffect hooks, and handle race conditions when multiple async operations update the same state.
  3. AI Integration Needs Fallbacks Amazon Bedrock is powerful, but network latency and API limits are real. Key learnings: Always implement intelligent fallbacks for critical operations Cache common AI responses locally Use streaming when possible to reduce perceived latency Provide immediate feedback even if sometimes AI response takes seconds The fallback AI I built actually handles 40% of scenarios without Bedrock, saving costs and improving response time.
  4. Real Data is Messy Integrating NASA FIRMS taught me to train our real world model and real-world data which requires extensive filtering: 23,000+ fires globally, but only ~200 relevant to USA operations Confidence levels vary - need to filter low-confidence detections Coordinate precision varies by satellite Data updates every 10 minutes, not real-time I learned to always validate, filter, and sanitize external data before rendering.
  5. User Experience Matters in Emergency Systems This isn't a consumer app, it's a life-safety system. Every design decision considered: Clarity over beauty - High contrast colors, large fonts, clear labels Speed over features - One click deployment, no confirmation dialogs Feedback is critical - Every action shows immediate visual response Error states must be obvious - Red warnings, yellow cautions, clear messages

What's next for Caligo Dynamics

Caligo Dynamics represents more than just a technical achievement, it's a vision of how AI and robotics can save lives in the most dangerous situations. By combining Amazon Bedrock's intelligence with autonomous drone technology and real-time wildfire data, we've created a system that gives first responders superhuman capabilities in zero visibility conditions. The journey from concept to working prototype taught me invaluable lessons about system architecture, AI integration, real-time systems, and the importance of user experience in critical applications. Every line of code was written with one goal in mind: helping firefighters and emergency responders do their jobs more safely and effectively.

On hardware: The simulation is the validation layer, not a shortcut. Every PID gain, every formation offset, every obstacle avoidance threshold is tuned before it goes on a physical aircraft. The Webots physics engine uses the actual DJI Mavic 2 Pro PROTO model with real flight dynamics. Parts are being sourced one by one. The next phase: NVIDIA Jetson for onboard inference, real thermal cameras and LiDAR, controlled field testing. When the hardware arrives, the software is ready.

On cost: One prevented firefighter fatality saves over $1,000,000. A full pod deployment costs ~$90,000 that's an 11× ROI on the first incident. Drone units run $3,000–$8,000. Deployment pods $10,000–$20,000. AWS infrastructure ~$200/month serverless. For comparison, aerial firefighting helicopters cost $3,000–$10,000 per hour to operate.

This is just the beginning. With continued development and real-world testing, Caligo Dynamics has the potential to become a standard tool in emergency response operations worldwide, saving countless lives and protecting the brave men and women who run toward danger when others run away.

Caligo Dynamics, founded by Furkan Serhatlioglu and Siva Abishikth Mylavarapu, bridges human bravery with intelligent technology to transform emergency response.

Built with ❤️ for firefighters and first responders everywhere.

Built With

Share this project:

Updates