DroneStuff: FireDrone

Inspiration

The inspiration for our approach was to put ourselves in the shoes of first-responders on the ground and monitoring the scene and to imagine what kind of resources and approaches would be most helpful in supporting those people.

What it does

direct_run.py Runs a direct run that allows a person to control the drone and analyze the current field of view using keyboard input. The current field of view of the drone is displayed in a matplotlib figure. Note: matplotlib figure freezes/pauses between key inputs. Simply press another valid button to unfreeze

reverse_run.py Runs a reverse_run. Scans the entire scene, stitches the images together, identifies the fires, and exports a clean image of the scene, an image of the scene marked with fire, and a json fire containing coordinates of the instances of fire to Azure Blob Storage.

control_panel.py Imagined as the 'control panel code.' Allows user to select between continuously scanning a selected scene and exporting data to Azure Blob Storage (monitoring the scene), running a normal direct run, or starting a direct run centered on a previous detected fire using data stored in Azure Blob Storage.

Note: run the code using a terminal

How we built it

This project was built using Azure Cognitive Services and Azure Storage API's for interfacing with Microsoft Azure. The image processes used during direct run primarily utilize matplotlib. The image stitching done in reverse run utilize the PIL libraries.

Challenges we ran into

One of our goals was to simulate the control panel and the drone as two separate entities by not running the drone code on the same device as the control panel, but we were unable to do this.

Some of the arguments for Azure's API's had different names than what could be found on Azure's portal. This caused significant delays.

Accomplishments that we're proud of

Learned and successfully implemented Azure in Python code The project works in testing Finishing a hackathon after 1-2 failures

What we learned

Azure Cognitive Services

Azure Storage Services

Image manipulation with PIL and Matplotlib

How to finish a hackathon

What's next for DroneStuff: FireDrone

We were considering using Azure Cognitive Services to identify smoke in the field of view of the drone and then using reinforcement learning to teach the drone to follow the smoke to the fire. However, this was out of our league in terms of knowledge and experience and will likely be saved for another time.

Built With

azure
keyboard
matplotlib
numpy
pil
python
readchar

Submitted to

FireDrone.AI
- Winner It Works! Honorable Mention

Created by

I worked on the coding portion of this project, being the member that had the most experience with python, though I still had fairly little.

This project is a series of firsts for me. This is not my first hackathon (I've participated in 2, in fact), but this is the first time my team was able to submit something that sort of worked. This is also my first large-scale and cohesive project in the python language (aside from small tinkerings and weekend projects here and there). Through this experience, I've really learned how versatile python can be, with the enormous variety of libraries where there's one for every need I had. This was also my first experience using the Microsoft Azure platform. Using it, I've realized that Azure has so much functionality, much of which this project does not utilize (because I was learning the platform as I went). However, its versatility makes me want to look into the platform further in the future.

This was a really great learning experience and a great way to spend my summer.

As a final note, I truly hope that the projects created as part of this hackathon will find their way into real-world situations to assist firefighters with their lifesaving duties.

Vincent Yang
Leonardo Espinoza

Updates

Vincent Yang started this project — Aug 04, 2019 10:28 PM EDT

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.