Charizard is inspired by my capstone design project (https://www.ece.rutgers.edu/sites/default/files/capstone/capstone2018/posters/S18-54-poster.pdf), in which we developed a drone that is able to deploy multiple Micro-UAVs and perform 3D mapping of an environment. I envisioned Charizard as a perfect application to this project. My vision is to build a complete fire detecting drone system for the fire department. So, I envisioned a hexacopter deploying multiple smaller UAVs in an area of interest which send images to the hexacopter and which is able to detect fire efficiently using Azure ML.
What it does
Charizard is designed to be easy to use and hassle free with minimal training required for the user (fire fighter). As shown in the video, the user can control the drone through "Manual Control" using simple buttons on any device with a web browser. The user can also use the auto run feature to let the drone move autonomously and detect fire. The drone moves in a spiral pattern from the edges of the environment towards the center. This path was chosen because it allows the drone to continuously move to unexplored parts of the environment, which reduces the time taken by the drone to search the environment. The main principle is to reduce extra work on the fire department in times of urgency and let drone controls be intuitive.
How I built it
The website is hosted as a web app on Azure App Service. This app communicates with the deployed ML model on Azure Container Instance (ACI). The ML model is firenet, a convolutional neural network trained for non-temporal real-time fire detection (https://github.com/tobybreckon/fire-detection-cnn). The neural network is built using tensorflow. The frontend webapp was built using Flask.
Challenges I ran into
I had to learn a lot about Azure and cloud in general. I also learned how to build models in Tensorflow.
Accomplishments that I'm proud of
I really enjoyed building an app that is so useful. Azure was really easy and intuitive to develop on after the initial struggle.
What I learned
I learned to develop ML + Cloud apps.
What's next for Charizard
Since the simulation only allowed a single run at a time, I was not able to simulate multiple drones at once. The next step is to explore the environment by multiple drones simultaneously. This dramatically reduces the time to search an environment and provides a scalable solution as the smaller UAVs can be made cheaper, which allows them to explore dangerous situations faster.