FireForce

Inspiration

On January 7th, California faced an unprecedented wildfire crisis, with multiple large-scale fires erupting simultaneously across different regions of the state. The Eaton fires destroyed more than 9400 buildings and killed 17 civilians. The Palisades fires destroyed more than 6800 buildings and killed 12 civilians. Fueled by a combination of strong winds, prolonged drought, and dry vegetation, the fires spread rapidly, overwhelming firefighting resources and prompting mass evacuations. Thanks to the current firefighters who risk their own lives to save the lives of others, casualties and damages could have been worse. This is a real problem which requires a real solution which is why I started FireForce.

What it does

FireForce allows us to take in a pre-configured flight plan, and perform reconnaissance on buildings. In my current simulator, I have a drone that initially spawns near a hydrant, and then tries to locate a fire that is shown from one of the building windows. Once it locates it, it will center itself with respect to the window. I also have a web application which is used to visualize the drone data which includes GPS coordinates and camera feed.

How I built it

In order to build FireForce, I had to utilize a variety of tools including methods of simulating an environment. Specifically, I utilized ROS2, PX4 SITLs, and Gazebo Classic. ROS2 is the leading robotics framework in terms of robotics research, and provides a really intuitive approach to creating robotics applications. Note that I use "ROS2 nodes" which publishes and subscribes to "ros topics" which one could think of as mediums of communication. For example, if I want to listen to specific sensor data coming from a drone, I can subscribe to a ros topic associated with that specific sensor data. Note that I used ROS2 to write all of my drone control code. ROS2 also has great integrations with Gazebo which is the simulator I utilized. In terms of Gazebo, I had to create my own world file and sdf/dae models. The world file contains configuration code of how exactly you want your simulation to look like, and you can place different models such as buildings. Lastly, I used PX4 SITLs which gave a much easier approach to simulating a drone. SITLs allows you to simulate PX4 drone hardware without actually having PX4 drone hardware. PX4 SITLs specifically is an open source software library which uses "uXRCE-DDS" as a medium for messages streaming between uORB topics and ros topics. uORB topics are communication channels used in PX4 Autopilot. To completely simplify everything I mentioned above, Gazebo is my main simulator, ROS2 is how I write my control code, and PX4 SITLs is an open source library which I use to help simulate drones and write the ROS2 control code.

I also created a react application, using Javascript, and a Supabase Cloud Storage which allows me to effectively send various data from my ros2 subscribers to view on the web. Essentially, I utilized libcurl, a popular HTTP library, to stash my captured drone frame and a file containing sensor measurements on Supabase. I then retrieve the data on my web app and display it on different visualizations including maps and various charts.

Challenges I ran into

The biggest challenge I ran into was actually developing the buildings for my Gazebo Sim and training models. In order to detect fire, I trained my own Yolov5 model and converted it to ONNX. I cannot actually run inferences on a yolo model using a .pt extension in OpenCV DNN (C++) which made it harder for me when developing my PX4 ROS2 control code. In terms of the building that I developed for my simulation, I had to create my own STL file, and then download it and modify in blender to allow for integrating a fire PNG.

What I learned

I previously had ROS2 and PX4 SITL experience, but not to this complexity. This not only helped develop my previous skills in world building, ros2 control code, and PX4 SITLs, but I learned how to build and integrate my own custom models using onShape and blender, and learned how to combine external libraries like libcurl to allow for data transmission to other sources.

What's next for FireForce

I actually have a lot of ideas for the future of FireForce:

Utilize Drone Swarm techniques allowing for multiple drones to exchange messages with each other to locate and take out fires.
Create a more comprehensive command control UI for providing manual control for firefighters just in case the drone goes off-mission.
Create the actual PX4 drone using Pixhawk, cube orange, and an Nvidia Jetson Orin nano for faster compute and real life testing
Experiment with different materials such as carbon fiber and see how close we can get a drone near a a fire
Upgrade to alternative simulators such as AIRSIM and Isacc Sim for more functionalities
Look into removing the pre-configuring aspect of drones and see if it can truly-autonomously navigate its environment to start taking out fires