FIREFLY SYSTEMS

GIF
FireFly Systems
Initial Plan of the Idea
An example of our image output.
An example of our image output.
An example of our image output.
An example of our image output.
An example of our image output. Incorrect classification as 90 is about the threshold, this may be down to lack of training.
Scott's computer suffering, during the training of the AI.

Inspiration

These days with climate change, wildfires are becoming ever more prevalent and even more powerful. They can spread over 10 MPH and can easily cause dangerous situations for both people and wildlife by causing building and environmental damage. That is why we decided to help with preventing and reducing the effect for rangers and anyone else, in a cheap and cost-effective way.

What it does

FIREFLY SYSTEMS is a cost-effective and secure remote monitoring multipart system to enable prevention and controlling wildfires. It has a secured central system which receives a video feed, from remote devices in key areas, which can detect fire or smoke from the feed and log and monitor any potential danger to minimise the possible effect if it was left untouched. The central system has a portal which gives a live feed from the devices along with their location and detection of a fire. The central system can also be connected to automated suppression systems to enable a potential fire to be delt with remotely without a person needed to be there in person. This system gives an addition step which can be cheaply added to using an internet connected device with a camera.

How we built it

Initial Plan and how it developed

Going into the project we chose; we realised that it was a complex task that required complex tools for the project to be successful. We decided to utilise TensorFlow for the machine learning for its more abstracted approach to machine learning and being a part of DataBricks. We also used MongoDB for its ease of storing and accessing dumps of data in a standard web format. Auth0 was also utilised as MongoDB isn't the best suited for sessions, user management or authentication and used it as a replacement of a typical database session management. StreamLit was also utilised for it's ease and simplicity in running and representing python code in a web environment.

Challenges we ran into

During our hack, there were always going to be two systems that would meet in the middle. The prior being allowing a user to sign up, create and view cameras / camera feed in order to get a camera IO. We accomplished the very basics of photo IO early on, then adapted it to be a more multi machine, several service architecture. This however, although difficult for a smaller team was the least of our concerns. To finish the project we required the other half... A functional AI Classifier. Although Databricks did make this incomprehensibly easier to implement, through the use of TensorFlow, we did hit road blocks at both the modelling and the training. Having to briefly teach ourselves more complex AI networks, than the simple K-NN we have only just been introduced to, and many rewrites, documentation and dataset filtering suggestions, we managed to implement a model that learnt to classify images as either a fire, or as to whether there is a fire.

Accomplishments that we're proud of

Finishing! During the long hours of many failed AI builds, Serialisation issues and deprecated code causing issues with saving the model, we finally finished.

What we learned

AI is much harder than "Import AI Brain", it takes thought, research and patience... Oh and a computer that doesn't melt when training. Training an AI is no small task and with the 8GB and 16GB of our computers we just about capped out our memory Scott's computer trying to learn far too many images at once

During the training we learnt that the models we were trying to emulate were far too complex for us at this time, so we were required to cut back on the accuracy of our AI. With our initial plan of an ai returning bounding boxes for both smoke and fires with classification to an accurate degree, our cutbacks resulted in us finding a new dataset trying to implement it, then going back to our old dataset and converting the bounding box data to separate fires from non fires.

This was a learning curve.

What's next for FIREFLY SYSTEMS

If we were to take FireFly Systems, even further we would most likely try to attach these models onto some form of drone. This would allow a drone to communicate via HTTPS, and report back to a main server so that rangers could see and respond to wild fires. As the location of the drone / camera is sent back and displayed on the server. This would allow rangers, to see more of their local territories and forests, keeping them safer from wild fires, when compared with static devices like old phones running the camera feed

In addition to this we would train the model with the whole dataset over a series of days, rather than the shorter sample we used to detect a fire. This would improve the accuracy of the model. As the model runs locally, it would be interesting to see if the drone could also run it allowing it to travel towards fires. It would also be very interesting to see other views of fires like heat maps further improving our AI's accuracy.