The inspiration for our approach was to put ourselves in the shoes of first-responders on the ground and monitoring the scene and to imagine what kind of resources and approaches would be most helpful in supporting those people.
What it does
direct_run.py Runs a direct run that allows a person to control the drone and analyze the current field of view using keyboard input. The current field of view of the drone is displayed in a matplotlib figure. Note: matplotlib figure freezes/pauses between key inputs. Simply press another valid button to unfreeze
reverse_run.py Runs a reverse_run. Scans the entire scene, stitches the images together, identifies the fires, and exports a clean image of the scene, an image of the scene marked with fire, and a json fire containing coordinates of the instances of fire to Azure Blob Storage.
control_panel.py Imagined as the 'control panel code.' Allows user to select between continuously scanning a selected scene and exporting data to Azure Blob Storage (monitoring the scene), running a normal direct run, or starting a direct run centered on a previous detected fire using data stored in Azure Blob Storage.
Note: run the code using a terminal
How we built it
This project was built using Azure Cognitive Services and Azure Storage API's for interfacing with Microsoft Azure. The image processes used during direct run primarily utilize matplotlib. The image stitching done in reverse run utilize the PIL libraries.
Challenges we ran into
One of our goals was to simulate the control panel and the drone as two separate entities by not running the drone code on the same device as the control panel, but we were unable to do this.
Some of the arguments for Azure's API's had different names than what could be found on Azure's portal. This caused significant delays.
Accomplishments that we're proud of
Learned and successfully implemented Azure in Python code The project works in testing Finishing a hackathon after 1-2 failures
What we learned
Azure Cognitive Services
Azure Storage Services
Image manipulation with PIL and Matplotlib
How to finish a hackathon
What's next for DroneStuff: FireDrone
We were considering using Azure Cognitive Services to identify smoke in the field of view of the drone and then using reinforcement learning to teach the drone to follow the smoke to the fire. However, this was out of our league in terms of knowledge and experience and will likely be saved for another time.